CRC setup fails on Hyper-v virtual switch

General information
Hi, I seem to be running into a common error (based on issue search) with the Hyper-V virtual switch. during CRC setup. I have relogged & rebooted. I appear to be a user in Hyper-V admin group. Hyper-V service also appears to be running.
PS C:\Users\bgoslin\OpenShift\CRC\crc-windows-1.2.0-amd64> net start "Hyper-V Virtual Machine Management"
The requested service has already been started.

OS: Windows
Hypervisor: Hyper-V
Did you run crc setup before starting it (Yes/No)? yes

PS C:\Users\bgoslin\OpenShift\CRC\crc-windows-1.2.0-amd64> .\crc setup  --log-level debug
INFO Checking if running as normal user
INFO Caching oc binary
DEBU oc binary already cached
INFO Unpacking bundle from the CRC binary
INFO Check Windows 10 release
INFO Hyper-V installed
INFO Is user a member of the Hyper-V Administrators group
INFO Does the Hyper-V virtual switch exist
ERRO Virtual Switch not found
DEBU Virtual Switch not found
ERRO Unable to perform Hyper-V administrative commands. Please make sure to re-login or reboot your system
FATA Unable to perform Hyper-V administrative commands. Please make sure to re-login or reboot your system
CRC version
PS C:\Users\bgoslin\OpenShift\CRC\crc-windows-1.2.0-amd64> .\crc version
crc version: 1.2.0+c2e3c0f
OpenShift version: 4.2.8 (embedded in binary)
CRC status
# Put the output of `crc status`

CRC config
# Put the output of `crc config view`

Host Operating System
Host Name:                 DUBLPJCVGQV2
OS Name:                   Microsoft Windows 10 Enterprise
OS Version:                10.0.17763 N/A Build 17763
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Member Workstation
OS Build Type:             Multiprocessor Free
Registered Owner:
Registered Organization:
Product ID:                00329-00000-00003-AA627
Original Install Date:     25/06/2019, 09:16:52
System Boot Time:          09/12/2019, 21:20:39
System Manufacturer:       Dell Inc.
System Model:              Precision 5530
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
[01]: Intel64 Family 6 Model 158 Stepping 10 GenuineIntel ~2592 Mhz
BIOS Version:              Dell Inc. 1.8.1, 01/02/2019
Windows Directory:         C:\WINDOWS
System Directory:          C:\WINDOWS\system32
Boot Device:               \Device\HarddiskVolume2
System Locale:             en-gb;English (United Kingdom)
Input Locale:              en-gb;English (United Kingdom)
Time Zone:                 (UTC+00:00) Dublin, Edinburgh, Lisbon, London
Total Physical Memory:     16,095 MB
Available Physical Memory: 9,930 MB
Virtual Memory: Max Size:  18,527 MB
Virtual Memory: Available: 10,577 MB
Virtual Memory: In Use:    7,950 MB
Page File Location(s):     C:\pagefile.sys
Domain:
Logon Server:
Hotfix(s):                 10 Hotfix(s) Installed.
[01]: KB4514366
[02]: KB4470788
[03]: KB4487038
[04]: KB4504369
[05]: KB4509095
[06]: KB4512577
[07]: KB4512937
[08]: KB4516115
[09]: KB4523204
[10]: KB4523205
Network Card(s):           3 NIC(s) Installed.
[01]: Intel(R) Wireless-AC 9260
Connection Name: WiFi
DHCP Enabled:    Yes
DHCP Server:     <IP_ADDRESS>
IP address(es)
[01]: <IP_ADDRESS>
[02]: fe80::2966:3d7f:6725:9e58
[03]: 2a02:8084:8081:c080:65da:c7b9:f62c:8d3f
[04]: 2a02:8084:8081:c080:2966:3d7f:6725:9e58
[02]: Bluetooth Device (Personal Area Network)
Connection Name: Bluetooth Network Connection
Status:          Media disconnected
[03]: Cisco AnyConnect Secure Mobility Client Virtual Miniport Adapter for Windows x64
Connection Name: Ethernet 2
Status:          Hardware not present
Hyper-V Requirements:      A hypervisor has been detected. Features required for Hyper-V will not be displayed.
Steps to reproduce

run .\crc setup

Expected
CRC setup completes successfully
Actual
CRC setup fails
Logs
You can start crc with crc start --log-level debug to collect logs.
Please consider posting this on http://gist.github.com/ and post the link in the issue.

@BrianGos From Windows version 1709 there is a Default Switch present which is created when you install Hyper-V, but you don't have that. at least from the systeminfo you provided this seems to be the case.
Can you check in the Hyper-V management GUI under the Virtual switch manager if the switch is there.

@anjannath Thank you. Creating the external switch manually seems to have worked well.
PS C:\users\bgoslin\OneDrive - daa> cd C:\Users\bgoslin\OpenShift\CRC\crc-windows-1.2.0-amd64
PS C:\Users\bgoslin\OpenShift\CRC\crc-windows-1.2.0-amd64> .\crc setup
INFO Checking if running as normal user
INFO Caching oc binary
INFO Unpacking bundle from the CRC binary
INFO Check Windows 10 release
INFO Hyper-V installed
INFO Is user a member of the Hyper-V Administrators group
INFO Does the Hyper-V virtual switch exist
INFO Found Virtual Switch to use: crc
Setup is complete, you can now run 'crc start' to start the OpenShift cluster

@BrianGos Since it is worked for you,  can we close this issue?

@praveenkumar Please do. Thanks for the help!

This needs some coordination when it comes to documenthing this. Ideally we want to avoid this situation, but perhaps a code change to the preflight might be needed to make people attentive of the fact a virtual switch needs to be created.
for the creation process; would this be automated or manual? we can automate the process, as we can provide a selection of interface and then create an external switch for this. although, we do not have any verification checks in place.
If manually created, we need to be clear about the need of a DHCP server on the network segment. Again, we do not check for this.

I'm running into the same issue - I have verified my account is part of the local hyper-v administrators group, and I still get the below.
PS C:\crc> .\crc.exe setup --log-level debug
INFO Checking if oc binary is cached
DEBU oc binary already cached
INFO Checking if CRC bundle is cached in '$HOME/.crc'
INFO Checking if running as normal user
INFO Checking Windows 10 release
INFO Checking if Hyper-V is installed and operational
INFO Checking if user is a member of the Hyper-V Administrators group
DEBU User is not a member of the Hyper-V administrators group
INFO Adding user to the Hyper-V Administrators group
INFO Will run as admin: add user to hyperv admins group
INFO Checking if Hyper-V service is enabled
INFO Checking if the Hyper-V virtual switch exist
DEBU Virtual Switch not found
ERRO Unable to perform Hyper-V administrative commands. Please make sure to re-login or reboot your system
FATA Unable to perform Hyper-V administrative commands. Please make sure to re-login or reboot your system
Windows 10 enterprise - and I have the built in Default Switch and a second switch named crc that's external.

Sorry running 1.6 should have mentioned.

I've landed here as I've searched for
INFO Checking if the Hyper-V virtual switch exist

since crc setup was hung there.
I am not sure I will ever be able to reproduce, but to un-stuck it, I simply opened the Hyper-V Manager and as I was fiddling with trying to add a new Virtual Switch, the crc setup completed with:
INFO Found Virtual Switch to use: Default Switch
Setup is complete, you can now run 'crc start' to start the OpenShift cluster

I don't know if simply opening the GUI somehow triggered something in the background which enabled the cec setup to complete, or it was me fiddling with the add/remove Virtual Switch panel, but hey... I thought worthy for reporting it :)
Win 10 Pro Version 2004 OS Build 19041.685 Experience Pack 120.2212.551.0

I've landed here as I've searched for
INFO Checking if the Hyper-V virtual switch exist

since crc setup was hung there.
I am not sure I will ever be able to reproduce, but to un-stuck it, I simply opened the Hyper-V Manager and as I was fiddling with trying to add a new Virtual Switch, the crc setup completed with:
INFO Found Virtual Switch to use: Default Switch
Setup is complete, you can now run 'crc start' to start the OpenShift cluster

I don't know if simply opening the GUI somehow triggered something in the background which enabled the cec setup to complete, or it was me fiddling with the add/remove Virtual Switch panel, but hey... I thought worthy for reporting it :)
Win 10 Pro Version 2004 OS Build 19041.685 Experience Pack 120.2212.551.0

you can not add your own 'Default Switch'. it either exists or you can create a 'crc' named override switch, but this means you have to provide your own DHCP and DNS servers on the same segment.
likely what happened is that your hadn't finalized the Hyper-V install with a reboot.

you can not add your own 'Default Switch'. it either exists or you can create a 'crc' named override switch, but this means you have to provide your own DHCP and DNS servers on the same segment.
likely what happened is that your hadn't finalized the Hyper-V install with a reboot.


[SPARK-12016][MLlib][PySpark] Wrap Word2VecModel when loading it in pyspark

JIRA: https://issues.apache.org/jira/browse/SPARK-12016
We should not directly use Word2VecModel in pyspark. We need to wrap it in a Word2VecModelWrapper when loading it in pyspark.

Test build #47056 has started for PR 10100 at commit 3ba2a10.

Test build #47061 has started for PR 10100 at commit 6ac03ac.

Test build #47056 has finished for PR 10100 at commit 3ba2a10.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/47056/
Test PASSed.

Merged build finished. Test PASSed.

Test build #47061 has finished for PR 10100 at commit 6ac03ac.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/47061/
Test PASSed.

Merged build finished. Test PASSed.

This is only a suggestion, but looking at the other Java loadable models which are also wrapped in python it seems the standard approach is to instead add a load[X] method to the PythonMLLibAPI instead of doing the wrapper in Python. Might be worth doing that way just for consistencies sake - what are your thoughts?

It makes sense. Thanks. I will update this later.

Test build #47122 has started for PR 10100 at commit 56c250e.

Test build #47122 has finished for PR 10100 at commit 56c250e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/47122/
Test PASSed.

Merged build finished. Test PASSed.

ping @davies @mengxr

ping @davies Can you take a look of this? It should be straightforward one.

LGTM, merging this into master, thanks!

I just backported this to 1.6 and 1.5


Current measured by CT sensor

Hi all.
I'm new user of ESPAltherma and manage to hook up everything fairly easy. Most of the things are functioning well and now starting to look more deeply on the pump behavior.
I have Daikin Altherma EGSAX10DA9W ground heat pump here in Finland fitted.
I have also CT sensors fitted and they work as I can see the current values from heat pump display.
I'm trying to read CT sensor values from register 0x63 without any success.
It only shows Conv 161 not avail.
Could there be any other register where these CT sensor readings could be found?
Thanks in advance

Similar to #313


keyring.Remove() returns error: Keychain Error. (-25244)

Using OSX Catalina (10.15.7) and calling keyring.Remove(key) returns an error Keychain Error. (-25244) instead of removing the provided key from the OSX keychain.  For now I'm just calling keyring.Set() and over writing the old values with zero.
you can see how I'm using the code here if you have issues reproducing: https://github.com/synfinatic/onelogin-aws-role/blob/main/cmd/keyring.go#L153

I have a similar issue, I'm on MacOS 12.3.1 Monterey, and using poetry config http-basic.my-org <username>, not sure if I already have an existing key, but I get the error:

Can't store password on keychain: (-25244, 'Unknown Error')


"Error when parsing watch.html, maybe YouTube made a change."

Describe the bug
Since yesterday I only get this error.
Error: Error when parsing watch.html, maybe YouTube made a change.
Please report this issue with the "1726751716488-watch.html" file on https://github.com/distubejs/ytdl-core/issues.
    at getWatchHTMLPage (/home/ben/downloader/node_modules/@distube/ytdl-core/lib/info.js:187:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async retryFunc (/home/ben/downloader/node_modules/@distube/ytdl-core/lib/info.js:121:16)
    at async exports.getBasicInfo (/home/ben/downloader/node_modules/@distube/ytdl-core/lib/info.js:44:16)

Debug File
1726751716488-watch.html
Environment

@distube/ytdl-core version: v4.14.4
Node.js version: v20.17.0
Operating system: Linux

this project and all projects based on similar approaches are dead. better move away from youtube scraping if this is all you rely on this approach .

this project and all projects based on similar approaches are dead. better move away from youtube scraping if this is all you rely on this approach .

At the moment I don't know any alternatives (in js) to @distube. Do you?)

this project and all projects based on similar approaches are dead. better move away from youtube scraping if this is all you rely on this approach .

At the moment I don't know any alternatives (in js) to distubejs/ytdl-core. Do you?)

you can set up your own instance of https://github.com/imputnet/cobalt (see info in their repo somewhere) or use their api or figure out what they do to avoid the ban

this project and all projects based on similar approaches are dead. better move away from youtube scraping if this is all you rely on this approach .

At the moment I don't know any alternatives (in js) to distubejs/ytdl-core. Do you?)

If you want to use the same ytdl-core (usage is different), I recommend https://github.com/ybd-project-ver1/ytdl-core. cobalt is a good option, but not suitable in this case.

this project and all projects based on similar approaches are dead. better move away from youtube scraping if this is all you rely on this approach .

At the moment I don't know any alternatives (in js) to distubejs/ytdl-core. Do you?)

you can set up your own instance of https://github.com/imputnet/cobalt (see info in their repo somewhere) or use their api or figure out what they do to avoid the ban

same issue there...
https://github.com/imputnet/cobalt/issues/647


fix: Added fix for word-wrap and semver npm packages

What does this PR do?
🛑 Please open an issue first to discuss any significant work and flesh out details/direction - we would hate for your time to be wasted.
Consult the CONTRIBUTING guide for submitting pull-requests.

Fixes for Dependabot security alerts
added fix for word-wrap and semver npm packages

Motivation

More

[ ] Yes, I have tested the PR using my local account setup (Provide any test evidence report under Additional Notes)
[ ] Mandatory for new blueprints. Yes, I have added a example to support my blueprint PR
[ ] Mandatory for new blueprints. Yes, I have updated the website/docs or website/blog section for this feature
[x] Yes, I ran pre-commit run -a with this PR. Link for installing pre-commit locally

For Moderators

[ ] E2E Test successfully complete before merge?

Additional Notes

Don't merge this. Website pipeline is failing so I need to fix that first

Closing this issue as this is introducing more security warnings


Fix search for rustup rust-src component

Fixes japaric/xargo#85

Oops, I pushed the wrong thing. One moment!

Thanks for tackling this @jdub!
@homunkulus r+

:pushpin: Commit 853b343 has been approved by japaric

:hourglass: Testing commit 853b343 with merge 853b343...

:broken_heart: Test failed - status-travis

Hmm, looks like an unrelated test suite failure on nightly? I'll take a look.

So, the problem is that the test runners, the binaries you get from compiling the stuff in the tests directory, have moved from e.g. target/debug to target/debug/deps and that broke the current_exe().pop().join(exe_name) trick that's used to test CLI applications.
cc @alexcrichton is that change a regression or was it intentional?
In any case, this is not hard to fix.

That should do the trick.
@homunkulus r+

:pushpin: Commit 634c00a has been approved by japaric

:hourglass: Testing commit 634c00a with merge 634c00a...

:broken_heart: Test failed - status-travis

:pushpin: Commit 6b6d189 has been approved by japaric

@homunkulus r+

cc @alexcrichton is that change a regression or was it intentional?

Yeah that was intentional, the change looks good though!

:hourglass: Testing commit 6b6d189 with merge 6b6d189...

:sunny: Test successful - status-appveyor, status-travis
Approved by: japaric
Pushing 6b6d189955c535b260bbf5ccc624caad0cee033f to master...


Optimize etl::Buffer

Higher-level alternative to PR #207. Kudos to @Giulio2002

Ok, so there have been an increase in performance compared of normal silkworm. This branch takes 57:56 minutes to complete vs 1:03 hours of normal silkworm. However, my branch is still taking 51:02 minutes. so its choosing between 7% improvement(this branch) with no boilerplate code or 18% improvement with boilerplate code. it all comes down to which trade-of we prefer (unless it can be optimized even further). We could also make a custom container to wrap the boilerplated code, this could also allow us to use it somewhere else without everything looking ugly. I also feel like stage 5 could be optimized to a certain extent by doing something like that. but that's just me wanting to experiment :).

@Giulio2002 I've compared performance between the two versions on a Google Cloud machine and it is the same (the difference is less than 1%). This this be reasonably accurate as no other processes were running on the machine.
This PR: optimize_etlb816bca.txt
PR #207: etl_buffer_experiment0ff087f.txt


Running organize imports quickly ends up corrupting code

Issue Type: Feature Request
Repro

In a python file:

import time
import os

Run organize import quickly twice in a row (I did this by mistake because the first command didn't seem to do anything for a half second)

Bug
The document is often changed to:
import os
import os

Extension version: 2020.4.68060-dev
VS Code version: Code - Insiders 1.44.0-insider (efd7548df65de8e775015b67863190e6bd4934fc, 2020-03-13T06:01:51.669Z)
OS version: Darwin x64 19.3.0

@mjbvz, thanks for letting us know about this.  We'll get back to you as soon as possible.

@mjbvz, I was able to reproduce the problem.  We'll look into the solution.

Fixed in the next release


investigate todo-mvc binary size increase

-rw-r--r--   1 root root  14937 Dec 12 14:41 todo-mvc-spasm-example-opt-wast-as-opt
-rw-r--r--   1 root root   5797 Nov 19 15:15 todo-mvc-spasm-example-opt-wast-as-opt.gz

vs.
-rw-r--r--   1 root root  15534 Mar  7 10:51 todo-mvc-spasm-example-opt-wast-as-opt
-rw-r--r--   1 root root   6090 Mar  7 10:51 todo-mvc-spasm-example-opt-wast-as-opt.gz

Got it down to:
-rw-r--r--   1 root root  14015 Mar  7 21:28 todo-mvc-spasm-example-opt-wast-as-opt
-rw-r--r--   1 root root   5801 Mar  7 21:28 todo-mvc-spasm-example-opt-wast-as-opt.gz

Mainly by adding -boundscheck=off to compiler flags. Not recommended. On the other hand, there are genuine cases where llvm should elide them.

Closing, since using brotli hardly makes a difference.

Also, there is a change in llvm that allows us to reduce binary size even further. It works by doing more aggressive dead code/data elimination.
Currently it is not activated because it removes too much. LLVM has fixed the issue by propagating @assumeUsed to lld, but it might take some time before this is released.
See https://github.com/ldc-developers/ldc/issues/3023


Release

dask / distributed / scikit-learn all have Python 3.10 support at this point. We should add support for dask-ml too

Hi @jrbourbeau , I am wondering do you know when will dask-ml new version be released, the one that supports python 3.10?

Thanks for the ping @hungbui-io. In addition to Python 3.10 support, it'd also be good to get a new release out for supporting recent versions of sklearn.
@TomAugspurger is this something you have bandwidth for?

Sure, just fixing some metadata in https://github.com/dask/dask-ml/pull/966 quick (hopefully).

Created https://github.com/dask/dask-ml/releases/tag/v2023.3.24. https://github.com/dask/dask-ml/actions/runs/4515711405/jobs/7953319610 is releasing to PyPI.
We'll want to bump the python version in the conda-forge package before that's merged.

Thanks @TomAugspurger!

@jrbourbeau @TomAugspurger  Thanks for the update. However, I am still facing an issue with 2023.3.24 release when import dask_ml due to the conflict with dask.distributed. Can you pls have a look on this?
Here is the error:
`Python 3.10.10 (tags/v3.10.10:aad5f6a, Feb  7 2023, 17:20:36) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import dask_ml
Traceback (most recent call last):
File "", line 1, in
File "C:\Users...\site-packages\dask_ml_init_.py", line 4, in
from dask_ml.model_selection import normalize
File "C:\Users...\site-packages\dask_ml\model_selection_init.py", line 6, in
from ._hyperband import HyperbandSearchCV
File "C:\Users...\site-packages\dask_ml\model_selection_hyperband.py", line 12, in
from ._incremental import BaseIncrementalSearchCV
File "C:\Users...\site-packages\dask_ml\model_selection_incremental.py", line 19, in
from dask.distributed import Future, default_client, futures_of, wait
ImportError: cannot import name 'Future' from 'dask.distributed' (C:\Users...\site-packages\dask\distributed.py)`

You might try installing things in a new environment to see if the problem is reproducible.


[Settings::Fees] Buggy input field

Settings -> Default Fee related isssues.

[ ] Input field should be visible while typing

[ ] When typing numbers

Previously mentioned issues are fixed.
But the auto-correction seems little annoying.

When I type 500
It reset to 2
Now I decide to go with 20 and type a 0
It removed the 0 and resets to 2
Typing 0 again, sets the field to 20

Suggestion
Let the user type whatever number they want, even 500000. But just show validation error and disable the Done button.
https://github.com/SatoshiPortal/bullbitcoin-mobile/assets/701985/8441e56d-f8f7-4b85-a4be-0c6e8394cbc2

@i5hi  @saiy2k i am not seeing this behaviour anymore
pls recheck with latest release and close if fixed

all good now!

Hi.
Just retesting this. Above issues are fixed now.
But 1 small thing still. Setting anything above 50, saves the value as null

@saiy2k  can't replicate the null error
do u think it could be caused by something different ?

Fixed on main.


How to correctly add a flutter project to a version control system such as git?

I'm working on a Flutter school project and we need a version control system. What are the correct steps to follow to create a GitHub repository for a flutter projects? Including files to include in the repo and files to ignore.

flutter create generates project with proper .gitignore files which is similar to what flutter has in it's own repo: https://github.com/flutter/flutter/blob/master/.gitignore


[explorer] task: update symbol-sdk to 2.0.2

What was the issue?

Old SDK has a problem handling meta info when the value is undefined.

What's the fix?

updated latest SDK 2.0.2

This latest SDK fix issue #1106 #1105

@AnthonyLaw issues:

https://github.com/symbol/explorer/issues/1106
https://github.com/symbol/explorer/issues/1105
are resolved.

I just have a question regarding the partial transaction presentation (fee field):

I think that for partial transactions we should show a max fee because currently for partial it's always zero (I expect it's calculated as for confirmed tx). If you agree I will raise a separate issue for that.

@AnthonyLaw issues:

Partial transactions list for account not presented #1106
Details for partial transaction not presented #1105

are resolved.
I just have a question regarding the partial transaction presentation (fee field):  I think that for partial/unconfirmed transactions we should show a max fee because currently for partial it's always zero (I expect it's calculated as for confirmed tx). My proposition:

Paid Fee (for confirmed transactions)
Max Fee (for unconfirmed/partial transactions)

If you agree I will raise a separate issue for that.

yes, I think that sounds good to me. let do it

@AnthonyLaw reported it here -> https://github.com/symbol/explorer/issues/1130


HDXDSYS-914 Keep crisis tags on dataset update

coverage: 97.517% (-0.06%) from 97.578%
when pulling 7b2a762c26d5d4c3b773ef63a21419cf778825f1 on keep_crisis_tags
into a0a43456c746f54086027cf2ce1811234875e8fc on main.


Idempotence shouldn't have prerequisite steps

A molecule v2 tracking issue.
Idempotence task shouldn't have prerequisite tasks prior to running.  If the instances have not been converged, it should report an error.

Implemented by #698.


chore(deps): upgrade dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-k8s-26-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 2374

Questions ?
Please refer to the Backport tool documentation


Repo is MIT but MapObject.h is All rights reserved.

So what's the actual license status?

That's a Typo. It should be MIT licensed.


Simplify API to upload MARXML

Originally on 2013-01-31
Currently, the most developer friendly way to upload MARCXML is to save it to disk, and then call task_low_level_submission.
E.g. in Insert_Record.py (websubmit function:
[...]
from invenio.bibtask import task_low_level_submission, bibtask_allocate_sequenceid

[...]
    sequence_id = bibtask_allocate_sequenceid(curdir)
    if os.path.exists(os.path.join(curdir, "recmysql")):
        recfile = "recmysql"
    else:
        raise InvenioWebSubmitFunctionError("Could not find record file")
    initial_file = os.path.join(curdir, recfile)
    tmp_fd, final_file = tempfile.mkstemp(dir=CFG_TMPDIR,
                                          prefix="%s_%s" % \
                                          (rn.replace('/', '_'),
                                           time.strftime("%Y-%m-%d_%H:%M:%S")))
    os.close(tmp_fd)
    shutil.copy(initial_file, final_file)
    bibupload_id = task_low_level_submission('bibupload', 'websubmit.Insert_Record', '-r', '-i', final_file, '-P', '3', '-I', str(sequence_id))
    open(os.path.join(curdir, 'bibupload_id'), 'w').write(str(bibupload_id))
[...]

In principle it should be possible to simply have an API such as:
from invenio.bibupload import schedule_bibupload
[...]
schedule_bibupload(user, mode, marcxml, ...)
[...]

that will take care of saving somewhere to disk the marcxml and schedule a bibupload task for you and return the task ID.
Additionally we can think of a synchronous version that will block and wait for termination (e.g. by polling the schTASK table.

The issue seems to be stalled for 3 years. Check https://github.com/inveniosoftware/invenio-records-rest for new REST API.
cc @inveniosoftware/triagers

On legacy one might also want to profit from an INSPIRE utility: https://github.com/inspirehep/inspire/blob/master/miscutil/lib/bibtaskutils.py


Land Cover CCI display too slow

Expected behavior
Land Cover CCI products should be displayed as fast as other CCI products.
Actual behavior
Displaying Land Cover CCI from NetCDF files is unacceptably slow, single tiles occur only after tens of seconds.
Specifications
Cate 0.9.0.dev6

Will be partly addressed by #378


Publish new version on Crates.io

This is one of the remaining crates that the Ion shell requires to be able to be published on Crates.io.

2.2.0


Fix import fix on react or react dev

Fixes #41920

@RyanCavanaugh Any update on this?

@xiaoxiangmoe No.

I'm sorry to be that guy, but this PR has been approved almost two months ago. Is there any plan to include it in the next TypeScript release?
This bug makes it very hard when working with the new jsx factory trasform, since auto-import code actions basically just stop working and you end up with a lot of spurious (and unused) imports from 'react'.

Thank you @andrewbranch for the merge! 🙏

Sorry @gabro, I wanted to give others a chance to review, and it just slipped through the cracks. This will be released in 4.3 proper, or in the nightly starting tomorrow.

No worries, thanks and keep up with the great work!


What about Visual Studio Express?

I tried downloading and executing the new Installer, but when it ran, it told me there are no instances of Visual Studio on my machine. I am running VS Express 2012 on a Windows 8 machine.
Is there an alternative installation approach I can use, if the Installer doesn't recognize Express versions?

Hi @sjcullen ,
Yes, AJAX Control Toolkit can not be installed on Visual Studio Express editions as stated in documentation because it does not support installation of Visual Studio extensions.
You have 2 options to overcome this:

Install the AjaxControlToolkit NuGet package to a project. You can install the AjaxControlToolkit.StaticResources and AjaxControlToolkit.HtmlEditor.Sanitizer packages if needed.
Upgrade to the Community edition of the Visual Studio. It is free and allows installation of Visual Studio extensions.


Verify SSWG with Swift.org ownership?

Is your feature request related to a problem? Please describe.
No.
Describe the solution you'd like
Can we verify SSWG with Swift.org, to mark this plugin as "official" in VS Marketplace?
Describe alternatives you've considered
Keep it as is now.
Additional context
https://code.visualstudio.com/api/working-with-extensions/publishing-extension#verify-a-publisher

Duplicate of #86

cc @tomerd


refactor code

use maps instead of for loops that feed back to new arrays
use setuptools to install packages instead of including lib folder in the repo

Closing this as we are now going to refactor into node code.


Referencing model twice in another model

I need one of my model to reference another model twice. Example:
model Post {
  id              String      @id @default(cuid())
  draftVersion    PostVersion
  publishedPost   PostVersion?
}

model PostVersion {
  id            String         @id @default(cuid())
  post          Post
  title         String
}

Prisma doesn't let me do this. It has a problem with Ambiguous relation detected... I have tried various approaches from the docs like @relation directive but I was not able to make it work properly.
Is there a way to make this work or is it a bug?

Fixed by adding second reference to PostVersion model too.


Provide an API for deleting views

Description
This feature is particularly useful for time-based frames which can have a number of views growing over time. An example is a frame with timeQuantum = "D" where the user is interested in a rolling window of time like "that last 30 days". In this case, it would be useful to provide the ability for views older than 30 days to be deleted while the server is still running.
Success criteria (What criteria will consider this ticket closeable?)
An endpoint exists that will delete a view:
curl -X DELETE localhost:10101/index/i/frame/f/view/standard_20170601

All directories and files associated to the view are removed.
I don't think it makes sense to delete anything other than time-based views at this point, so we could be very strict and return an error if the view is not time-based.

To simplify the rolling window case, perhaps a frame option could be set to delete views automatically?

That would be amazing :)

While planning this task I see that the Frame SetBit and ClearBit functions implicitly create Views with CreateViewIfNotExists.  This could be an issue while streaming in time data, and Pilose were processing time views that were recently truncated.  Resulting in new empty Views being created.  I think we should handle that and auto truncating Views with a rolling window in a separate ticket.

@travisturner in the case where we have inverse views setup should the Delete View API explicitly name the standard or inverse view to delete them?  Or should Pilosa automatically delete the inverse view?

@raskle I think it should be explicit.
Also, I'm not sure I follow your example regarding the CreateViewIfNotExists issue you mention above. Do you mean that if you receive a timestamp represented by a view that does not exists, instead of creating it we should verify that the timestamp falls within a defined window. If it does, then we proceed with creating the view. If it does not, then we discard the data as out of bounds?


a little problem

Status       : Error: fetch failed. retrying in 120 seconds
(To overcome this error, for me personally, I first had to go to the web and verify wallet manually.)

mines      :
Status       : Error: 423 - Locked
(stuck)

423 It's because the officials think you're operating too often, or think you're operating as a bot and need to lock up for over an hour


[BUG] Cannot start plugin. 404 client error: Not found

Describe the bug
Since updating Hassio, my Ring devices are showing as not available.  I get the following error in Hassio:
20-03-31 11:10:31 ERROR (SyncWorker_10) [supervisor.docker] Can't create container from addon_fdb328a7_ring_mqtt: 404 Client Error: Not Found ("No such image: fdb328a7/aarch64-addon-ring_mqtt:3.1.0")

Not sure what the issue is.  I uninstalled both the plugin and the repository.  I also re-flashed Hassio and recovered my configs from a snapshot.  I have uninstalled the plugin a couple times.
I am on the 64bit RPi 3B+ version, with HassOS 3.12 and Supervisor 214.  Any idea what is wrong?
I included the logs from the system page.  I was not able to get the full logs because when I used the process described, it said "~bash: docker: command not found.  There is no toggle to disable protections in the addons that I see any longer.
Debug Logs
20-03-31 23:12:46 INFO (MainThread) [supervisor.ingress] Update Ingress as panel for core_ssh
20-03-31 23:12:49 INFO (SyncWorker_19) [supervisor.docker.interface] Stop addon_core_ssh application
20-03-31 23:13:14 INFO (SyncWorker_19) [supervisor.docker.interface] Clean addon_core_ssh application
20-03-31 23:13:14 WARNING (SyncWorker_15) [supervisor.docker.interface] Can't grep logs from homeassistant/aarch64-addon-ssh: 404 Client Error: Not Found ("No such container: e2b546463171bf4c2dbe5c1b4b53ef770bc759f779dd0218013cb15f8f2ab3fa")
20-03-31 23:13:35 INFO (SyncWorker_3) [supervisor.docker.addon] Start Docker add-on homeassistant/aarch64-addon-ssh with version 8.5.3
20-03-31 23:15:28 ERROR (SyncWorker_9) [supervisor.docker] Can't create container from addon_fdb328a7_ring_mqtt: 404 Client Error: Not Found ("No such image: fdb328a7/aarch64-addon-ring_mqtt:3.1.0")
20-03-31 23:17:23 ERROR (SyncWorker_15) [supervisor.docker] Can't create container from addon_fdb328a7_ring_mqtt: 404 Client Error: Not Found ("No such image: fdb328a7/aarch64-addon-ring_mqtt:3.1.0")

This is almost certainly a HassIO issue and not an issue with this addon.  The error you are receiving is Hassio trying to build/create the addon but for some reason not finding it locally even though it thinks it is there.
If you perform a Google search on this issue you will find a number of similar reports on various addons with 64bit Hassio with the same message.  I've been unable to find a common cause, one seems to be space, another DNS, but almost everyone reports success after a clean re-install of Hassio.  The other common issue seems to be Docker db corruption.  I'd suggesting searching for the error generically (not specific to this addon) and reading all the threads.
You might try installing the portainer addon, unhide the home assistant containers and make 100% sure that the container is completely removed after removing the addon and repository, or do the same from the command line.
The 64bit version of Hassio is a mess IMO, there's a reason it's not recommended.  I'd strongly suggest using the 32bit version.  I don't have a 64bit version to test with so I'm not going to be much help I'd guess.

I figured it might be.  Seems that anytime they update Hassio, something breaks.  I tried a fresh reinstall and that did not help, although I restored by config using a snapshot (just the config, not the OS).  I was using the 32-bit verison of Hassio prior to the reinstall and it was under that version that I first noticed the alarm panels were unavailable.  I have been Googling like crazy and not really finding anything that helps.  I will try the portainer suggestion as I had not thought about that.  I am not very savy with the CLI in hassio (I am barely competent in the Linux shell :D  ).

So, looking at the docker containers, I find this container stopped...

I assumed it was the Ring Hassio addon container and opening it, I found it is.  I started the container and tried to start the addon in Hassio, but it still will not start.
The container is not named correctly.  The container is also linked to an untagged image.  The image referenced in the container is also incomplete as it shows an alphanumeric string rather than the name of the image (ring-mqtt).
It appears that the install script did not complete and it left the docker container present but misinstalled.

That's kind of what I expected (an incomplete container image) but then the question is why is it failing.  The Dockerfile for this thing is pretty simple overall.  I've had some issues with the Alpine repo's not being reachable, but it feels like your image didn't even get that far.  I'd just make sure to completely remove this image and all of it's components and try again.
I just did an install of the addon on my two test platforms (an RPi3B+ 32-bit and a VMware VM) and the addon installed on both with no issues and I've had no other reports of this issue even though the addon gets ~5-10 pulls/day so I'm at a loss what it might be from the addon perspective, unless I'm missing something architecture wise, but I looked at some of the other addons and how they deal with mulitple architectures and, because I'm not compiling/building anything, I don't need any arch specific tweaks as far as I can tell.  I wonder if some particular package isn't available for aarch64 bit on Alpine?  I do have another spare RPi3B+, I guess I could try installing 64bit Hassio on it but it will probably be a few days before I can get around to it.

No worries.  I am going to try a couple more things and see if I can get it working under the 64 bit.  If not, then I will switch back to the 32bit and see if it works again.  Thanks for the help!

I messed with it some more tonight, but I did not get it working.  The issues I see is that the 64 bit version is not running the dockerfile and the ring-mqtt is not cloned.  I ran each command in the dockerfile separately and got the ring-mqtt cloned.  I ran part of run.sh.  I am not really sure how to install a docker file via CLI  though.  I am just going to reinstall the 32bit version of Hassio for now.  I am always available to help test things if needed.  Thanks again.


Please, add https://eugene-andrienko.com to 512kb club

This PR is:

[x] Adding a new domain
[ ] Updating existing domain size
[ ] Changing domain name
[ ] Removing existing domain from list
[ ] Website code changes (512kb.club site)
[ ] Other not listed

[x] I used the uncompressed size of the site
[x] I have included a link to the Cloudflare report
[x] This site is not an ultra minimal site
[x] The following information is filled identical to the data file

I confirm that I have read the FAQ section, particularly the two red items around minimal pages and inappropriate content and I attest that this site is neither of these things.

[x] Check to confirm

- domain: eugene-andrienko.com
  url: https://eugene-andrienko.com/
  size: 222.45
  last_checked: 2024-09-19

Cloudflare Report: https://radar.cloudflare.com/scan/4f7dff1e-0672-4284-8f2d-97a2f9f49b4f/summary

Thanks!


Application callback on PDU transmission complete

A callback that notifies the application that a PDU (L2CAP or ATT) has been successfully transmitted to the peer. It would essentially follow the flow of HCI Number of Completed Packets with additional counting to make sure a single callback per PDU is issued.

@pdunaj we would like to have a usecase for this functionality. Can you let us know why exactly you need this? Thanks!

Hi @carlescufi ,
This is needed for a proper generation of a key events sequence sent to the host via HOG service.
In case the sequence of keys is pressed fast we need to make sure that:

each key event is transmitted separately (to maintain the order in which sequence was recorded)
no key event is missed (obvious reasons).

For example. Let's say that user pressed A quickly. We should send two notifications: first notification with "A" and then empty notification (indicating key released). If due to earlier events notification cannot be added to the queue we will lose the key press or key release (or both).

Perhaps Im missing something but if bt_gatt_notify cannot fail due to not be able to be added to the queue, it will block until the a buffer becomes available, or perhaps let me ask if there has been reproduced already or it a theoretical problem? Also what guarantee do we have that once a key failed to delivered it won't fail again? If the keys are pressed in a quick succession and one of them fails to be sent, would you resend it? That means one will have to reference all buffers queued and so they can be resent before the others which I don't think it a good application API, so I wonder if for this use case the application actually has to queue by itself and not flush everything to bt_gatt_notify and risk it to block.
Also because bt_gatt_notify blocks it must not be used in interrupt context which I assume is where the inputs would be generated, that means it will need to be deferred to a thread which might already be using a fifo/queue, that way there should already be a queue in between so when that thread is using bt_gatt_notify if that blocks nothing is lost since the inputs can still be queued and the thread will be resumed once a buffer becomes available and then bt_gatt_notify can be called again, so I don't see how we can fail to send a packet because of queueing, except if interrupt cannot queue anything anymore but that has nothing to do with Bluetooth.

Hi @Vudentz , I don't think bt_gatt_notify blocks. I don't see any wait in the code and practical test (I tried to send 50 notifies in a loop) shows only 15 notifications are queues. Rest is simply ignored with no notification to the application.
As for the application side, I don't really mind to queue the notifications here. I only would like to get notified when there is no space to send key event sequence. This way we would be able to react (e.g. drop remaining events, clear state, etc.) to maintain the sanity of the state.
Note that, to save energy, we don't want to transmit the state periodically. If we sent key release but this notification is dropped host may keep key pressed (until we press and release the key again on the device).

Hi @pdunaj Im pretty sure it blocks if you send more packets than there are buffers:
https://github.com/zephyrproject-rtos/zephyr/blob/master/subsys/bluetooth/host/gatt.c#L630
https://github.com/zephyrproject-rtos/zephyr/blob/master/subsys/bluetooth/host/att.c#L1944
https://github.com/zephyrproject-rtos/zephyr/blob/master/subsys/bluetooth/host/conn.c#L1935
If it would not block it would fail immediately so it is really surprising to me that you could lose packets since we never discard them or anything. In the other hand if bt_gatt_notify blocks as it should it would stop sending and then resume as soon as the driver free a buffer, and you can actually have multiple threads blocking in the same manner.

Hi all,
I would like to get back to this issue. I have run tests today and the code blocks. I must have done some mistake when I tried it earlier.

@pdunaj does that mean we can close it?

Now about the use case. To create a responsive HID device:

we need to be able to create a round robin of notifications,
we also cannot allow more then one notification to be in the transmission buffer.

With current infrastructure we could create a thread that sends the notifications and pass data to it to be sent. This way we would be able to perform round robin as we would control the flow. There are however two problems here.

First, we will block on a second notifications being sent. Imagine data creation module creates and sends packet1 (accepted), then immediately packet2 (this will block). After packet1 is sent, packet2 is accepted and creator blocks on packet3. And so on. Note that in this case data producer is always 1 interval late (pipeline is always 2 elements - one being sent and one that is blocked).
Second, we need a thread to block, and this requires RAM for stack and complicates the design.

With callback we race with the next interval to create data but data is always fresh. Also as we are notified no thread is be needed and RAM is spared.

@pdunaj thanks for the explanation, this is clear enough. Adding @joerchan.

@carlescufi I would prefer to have something like HVN_TX_COMPLETE from SDK. This was used in SR3 to trigger data processing. Having this callback would give us just that. If there is any cheaper way of telling that we are CTS maybe we can use it but blocking on notify seems not good option.

@carlescufi Thank you. Sorry it took so long - I lost focus on BLE part for some time but we need to catch up with it reasonably fast.

@pdunaj yep, makes sense to me to have the (optional) callback. Don't you think we could merge #9056 into this issue? That loos like a particular case of the present one.


Extend DataLoader with transform arguments to keep that logic in the DataModule

🚀 Feature

Extend DataLoader to accept transform.
Possibly split into 2 arguments:

sample_transform
batch_transform

The new process of data retrieval would go as follow:

Retrieve sample tuple with (input, target)
sample_transform(sample)
Concatenate to batch
batch_transform(batch)

Motivation
The new DataModule page of the documentation incorrectly includes a transform=transforms argument for the DataLoader:
class MNISTDataModule(pl.LightningDataModule):
    def train_dataloader(self):
        transforms = transform_lib.Compose([
            transform_lib.ToTensor(),
            transform_lib.Normalize(mean=(0.5,), std=(0.5,)),
        ])
        return DataLoader(self.train_dataset, transform=transforms, batch_size=64)

This extra argument would allow to truly separate the Dataset and DataModule in terms of functionality.
If we do transforms on the samples after retrieving it from the Dataset, we could also logically include not only sample transforms, but also possible batch transforms.
Pitch
Dataset and DataModule are not truly separated in functionality yet.
While the documentation suggests that transforms should be specified in DataModule, in the end the transform is still passed on to Dataset.
This requires Dataset to be transform-aware. If a Dataset is not configured with something similar to:
if self.transform:
    input = self.transform(input)

, it will not be possible to transform the data.
If it is necessary to write transform code in the Dataset class, why not specify it directly there?
Extra 1: If Dataset does not have to be tuned to expect PyTorch transforms, this Dataset could be used for both PyTorch and scikit-learn without encoding specific logic for this in the Dataset class.
Extra 2: If we move transform to DataLoader, we can also both specify per-sample transforms and per-batch transforms.
p.s. Would it be faster to transform a batch of numpy arrays into Tensor, or is there no different with transforming the arrays 1-by-1 into a Tensor first before batching them?
Alternatives
Keep it as-is, passing the transform to a Dataset and require the user to make sure the Dataset can handle potential transforms.
Additional context
Liked by: @Borda
Others mentioned by him: @nateraw @awaelchli @williamFalcon

I like your point to decouple also this part of the pipeline as most DL training contains some augmentations...
cc: @PyTorchLightning/core-contributors

This split in functionality can already be found in the PyTorch based FastAI(2) library in their DataBlock. This class accepts a item_tfms and a batch_tfms argument.
FastAI also has support for something they call presizing:

...
The challenge is that, if performed after resizing down to the augmented size, various common data augmentation transforms might introduce spurious empty zones, degrade data, or both. For instance, rotating an image by 45 degrees fills corner regions of the new bounds with emptyness, which will not teach the model anything. Many rotation and zooming operations will require interpolating to create pixels. These interpolated pixels are derived from the original image data but are still of lower quality.
To work around these challenges, presizing adopts two strategies ...:

Resize images to relatively "large" dimensions—that is, dimensions significantly larger than the target training dimensions.
Compose all of the common augmentation operations (including a resize to the final target size) into one, and perform the combined operation on the GPU only once at the end of processing, rather than performing the operations individually and interpolating multiple times.

The first step, the resize, creates images large enough that they have spare margin to allow further augmentation transforms on their inner regions without creating empty zones. This transformation works by resizing to a square, using a large crop size. On the training set, the crop area is chosen randomly, and the size of the crop is selected to cover the entire width or height of the image, whichever is smaller.
In the second step, the GPU is used for all data augmentation, and all of the potentially destructive operations are done together, with a single interpolation at the end.

This picture shows the two steps:

Crop full width or height: This is in item_tfms, so it's applied to each individual image before it is copied to the GPU. It's used to ensure all images are the same size. On the training set, the crop area is chosen randomly. On the validation set, the center square of the image is always chosen.
Random crop and augment: This is in batch_tfms, so it's applied to a batch all at once on the GPU, which means it's fast. On the validation set, only the resize to the final size needed for the model is done here. On the training set, the random crop and any other augmentations are done first.

By having separated the transforms from Dataset, it might become possible for PyTorch Lightning to speed up transforms by smartly, under the hood, combining all transforms into 1, as done by FastAI.

So, we don’t enforce transforms to be passed in but they CAN be passed in via init. Why wouldn’t this work?
init(self, train_transform, etc...)
the datamodule is just an interface. the ones we codes have default transforms. and in fact, you can set them however you want or pass them in.
dm = DataModule()
dm.train_transforms = ...
or
dm = DataModule(train_transform)

@williamFalcon because there is no defined way, where these transforms will be called. In most cases they are transforms for the dataset. But here the request is to have them as part of the data loader

got it.
ok, unlikely that we will add a custom dataloader to lightning since the api is finalized and needs to stay thin.
but we can consider for another project

The issue is not that it cannot be passed to the DataModule. The issue is a conceptual one. The Dataset class should be disconnected from possible transforms, but now the DataModule still has to pass it on to the Dataset which also has to know how to use that transform, before the transform can be used.
How to transform data should not be handled in the Dataset class.
Although I can see this is at a deeper level that should be tackled from PyTorch, and not Lightning. The DataModule would be a perfect fit for this separation however.

I very much agree with this Issue philosophy. Conceptually speaking, it is very clean to have the Dataset logic focus on loading the data and the DataModule to focus on augment & normalize it. When we started porting our code to Pytorch Lightning, about 2 weeks ago, we considered this approach in or designs:

With this approach, the data folks can load or import the data from wherever they want and we, from the AI team, can define how that data will be augmented, transformed & normalized. So basically is just a matter of interfacing between 2 teams and between 2 logically separated tasks.
We are sticking to our designs because it is almost a must for us but it would be really nice to have this the standard library

in bolts we separated the transforms from the data:
https://github.com/PyTorchLightning/pytorch-lightning-bolts/blob/master/pl_bolts/models/self_supervised/simclr/simclr_module.py#L248

in bolts we separated the transforms from the data:
https://github.com/PyTorchLightning/pytorch-lightning-bolts/blob/master/pl_bolts/models/self_supervised/simclr/simclr_module.py#L248

Now @ https://github.com/Lightning-Universe/lightning-bolts/blob/master/src/pl_bolts/models/self_supervised/simclr/simclr_module.py


Make sure fits file reading is correct in general

There's some code about reading .fits vs. .fz that doesn't seem to be right in general

generalized to look for 'SCI' extension, and when that's not available, use the first HDU with a data array.  See f10f763


Date parsing issue

Data from ALA currently consists of mixture of dmy and mdy dates.

This effects the "pulling more recent record" function

the issue seems to be coming from here: https://github.com/traitecoevo/plant-map-app/blob/11484c06a41ea8219eb8c973cca159b803fbdd9e/ala_processing.R#L36. might be related to time zones.

there are two outstanding issues with respect to dates:

we're not handling timezones very well, which is making things off by one day in some cases
the date sorting thing in datatable doesn't work with this date format.

NSW_plants_tzdate  <- NSW_plants_renamed |>
  mutate(collectionDate = ymd_hms(eventDate) |> with_tz())

seems to work well, but we're losing some data at this step

some parsing failures, that lead to data dropped later on.

maybe that's OK, not sure

some of them are like this to start with "2016-04-06 UTC" and others are like this ""2021-12-22 09:48:00 UTC"

Hmm is this for plant data or tassie butterflies

Butterflies.  but I think I fixed it on the generalization branch

I think some dates dates do get parsed oddly by ALA so sometimes its good to grab verbatimeventDate just in case

So in the tibble, you can't see the timezone info. I had to use $ to see

some parsing failures, that lead to data dropped later on.

maybe that's OK, not sure

I think the 6k records that get dropped may not have UTC in the eventdate?
Need to go back and check

i think this is working as far as i can tell

Reopenning this so I can try fix it again!

Addressed in fe57d18


Skip AntiPhishing page on user click/interaction

I know that per #151 (and the docs) we have a way to skip the anti-phishing page and avoid that unnecessary redirect by passing in the header X-Tunnel-Skip-AntiPhishing-Page: True, however if the initial interaction to the devtunnel link comes from a user click/interaction from within a browser page (i.e. clicking on an anchor tag), there is no way to set that header (that I am aware of). I am requesting that we allow for a query param to be passed in the URL to essentially perform the same behavior (i.e. https://{TUNNEL_ID}.devtunnels.ms/?SkipAntiPhishing=true).

Thanks for taking the time to write this up. However, the current behavior is by design and for security/privacy reasons.
Introducing a query param would reduce the effectiveness of the interstitial ("anti-phishing") page.

Unfortunate, because in my code i then have to write two workflows for how to get the correct referral header, and this is only an issue when using dev tunnels on that first access attempt. The only thing that would alleviate this is if an additional header is passed through the tunnel to show the original referral header IF the anti-phising page was just shown to the user.

Can you expand on your latest comment in relation to the referral headers and share more on the scenario?
There may be another way to achieve your desired goal that doesn't involve adding a query param to allow skipping of the interstitial page.

Normal/desired flow:

User clicks link under site X
Browser navigates to site Y behind dev tunnel
Site Y inspects referer header and sees site X listed ✅

Problem flow (on first access with dev tunnel):

User clicks link under site X
Browser navigates to site Y behind dev tunnel
Since anti-phishing cookie has not been set yet, the anti-phishing confirmation page hijacks the request chain and prompts the user to confirm before continuing to site Y behind dev tunnel
After user confirms, Browser re-navigates to site Y, however referer header now shows the dev tunnel URL, not site X ❌

The click action that initiates these series of requests is performed on an anchor tag, therefore it is not possible to specifiy the header to skip the anti-phishing page alongside the GET navigation request.
Subsequently, I dont think it is possible for the Browser to send along the correct referer header once the phishing confirmation page hijacks the request flow. So besides opting for another way of specifying intent to skip the anti-phishing page (i.e. query param), the hijacking dev tunnel logic would need to somehow pass the original referer header to the down stream request.
Note, I know that people tend to not reccomend using the referer header for any sort of security measure, however I can assure you that is not the intended use case here.
Let me know if you still need some clarification.

Understood. I have observed what you've described.
When the anti-phishing cookie is not set, the confirmation page is displayed and on confirmation, the Referer is set to the tunnel URL.
However, on subsequent requests when the cookie has been already set, the Referer is set to site X.
In the flow where the user goes from site X to site Y, it's important for us to ensure the anti-phishing page is displayed so it is clear to the user that they are accessing a dev tunnel. This is why we can't introduce a query param that allows the page to be skipped with no user interaction.


Test connection with op-program

This commit contains:

1 unit test for bidirectional channels
1 integration test (a shell script and a part of our optimism VM)
fixes to the issues found during the testing phase

I don't know if it is expected, but we get:
t=2023-12-07T19:34:31+0100 lvl=eror msg="failed to close pre-image server" err="signal: interrupt"

when stopping the zkVM with CTRL-C.
Not urgent IMHO, and definitely not in the scope of this PR, but would be nice to open a follow-up.

I don't know if it is expected, but we get:
t=2023-12-07T19:34:31+0100 lvl=eror msg="failed to close pre-image server" err="signal: interrupt"

when stopping the zkVM with CTRL-C.
Not urgent IMHO, and definitely not in the scope of this PR, but would be nice to open a follow-up.

Opened https://github.com/o1-labs/proof-systems/issues/1487


Update NPM to 2.8

NPM still has 2.7.1 from 5 months ago.

+1

It is now properly published

Thanks a lot for notifying me


AsyncAPI as part of OpenAPI ?

I'm not sure if this has been discussed already. However, I did not find related discussion so just asking here now. AsyncAPI is reusing the basic concepts from OpenAPI, see also https://www.asyncapi.com/docs/getting-started/coming-from-openapi and I was thinking about why AsyncAPI was never integrated into OpenAPI spec. I personally think this would be a perfect match. Was this considered already?
Of course also in regard of tooling such a combination could become beneficial for many people, e.g. https://github.com/swagger-api/swagger-ui/issues/6632

See #2431 - one of the reasons for not merging the specs is that doubling the size of the spec would make it harder for new users to get to grips with it.

thanks for the super fast feedback @MikeRalphson ! I agree, it might hinder development velocity a bit. However, as the core elements are very close and OpenAPI has been extended with SLA, webhooks, json schema I was thinking that this could be valuable. Also considering use cases were you have full blown APIs based on OpenAPI including webhooks and some async endpoints in a single spec.

You succinctly explain the counter point!

@fmvilas has thoughts on this area - when he returns from parental leave, maybe we can schedule a discussion.

@fmvilas Could you maybe share your thoughts on this?

😊 You actually raised a common topic and I presented a few times about it: https://www.youtube.com/watch?v=z3nA_2Wu91A&list=PLbi1gRlP7pihtIxQOd51YMqtUn_htgsrj&index=3. Hope this sheds some light on why we're not merging the two specs.
That said, it's in our roadmap to let users embed OpenAPI, GraphQL, and other API specifications (or parts of it) inside AsyncAPI. So, eventually, you should be able to use AsyncAPI and OpenAPI together using the same tools.

Thanks for the insights and the link to your presentation on the topic!

Looking at https://github.com/APIs-guru/asyncapi-directory there isn't much interest in AsyncAPI

@rafalkrupinski I think that's an artifact of who tends ot look at APIs.guru rather than lack of interest in AsyncAPI.

@handrews Might be the case. But there was traffic in that repo, just died three years ago...

I'm closing this issue as I don't think there's anything directly actionable here that merits being on our backlog. OpenAPI as an organisation stays in touch with the other API specification standards and of course the topic is always open for discussion.

Is there any way to unite OpenAPI and AsyncAPI so that it is possible to create api documentation for WebSockets with other http requests?


feat: Allow to transform result dataframe before aggregation in plotting

This allows to plot a derived metric, which is not contained in the results dataframe, but which depends on the content there. The first application of this is to plot the hypervolume index in the multi-objective context.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

In fact, I added the hypervolume indicator code. @jgolebiowski , would have to confirm that this gives the same results as your special code

Rather than transforming the dataframe, as an alternative approach, have you considered appending the new information into the results.csv.zip file, using the feature you added earlier this year? https://syne-tune.readthedocs.io/en/latest/faq.html#how-can-i-write-extra-results-for-an-experiment

@wesk The problem is that you often would like to plot something you have not recorded explicitly during experimentation, but which can be derived from what you recorded. I think this is legitimate use.

OK, I think I have now addressed the concerns


chore: add simple bb installer script

As of #6320, we're now expecting users to maintain an installation of bb and update its version as required.
To help with this, this PR adds an ultra dumb bbup script fashioned from noirup which only allows installing released versions of bb. This means that users can run bbup -v 0.38.0 to install 0.38.0 of bb.
Happy to make any changes as requested.

I think it's worth progressing with this in the short term pending the client/server model switch and have bb.js install the binary. We can delete this after that work is complete.

Benchmark results
Metrics with a significant change:

batch_insert_into_append_only_tree_32_depth_ms (512): 317 (+18%)
batch_insert_into_append_only_tree_32_depth_ms (1024): 611 (+21%)
batch_insert_into_append_only_tree_32_depth_ms (2048): 1,193 (+21%)
batch_insert_into_append_only_tree_32_depth_ms (4096): 2,364 (+21%)
batch_insert_into_indexed_tree_20_depth_ms (512): 455 (+21%)
batch_insert_into_indexed_tree_20_depth_ms (1024): 897 (+21%)
batch_insert_into_indexed_tree_20_depth_ms (2048): 1,777 (+21%)
batch_insert_into_indexed_tree_20_depth_ms (4096): 3,534 (+20%)
circuit_witness_generation_time_in_ms (merge-rollup): 110 (-30%)
node_history_sync_time_in_ms (5): 18,426 (+20%)
node_history_sync_time_in_ms (10): 33,845 (+19%)
l2_block_processing_time_in_ms (8): 1,606 (+18%)
l2_block_processing_time_in_ms (32): 6,050 (+19%)
l2_block_processing_time_in_ms (64): 12,019 (+19%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
8 txs
32 txs
64 txs

l1_rollup_calldata_size_in_bytes
740
740
740

l1_rollup_calldata_gas
6,692
6,680
6,656

l1_rollup_execution_gas
586,877
586,865
586,841

l2_block_processing_time_in_ms
:warning: 1,606 (+18%)
:warning: 6,050 (+19%)
:warning: 12,019 (+19%)

l2_block_building_time_in_ms
24,911 (+5%)
97,855 (+5%)
196,014 (+6%)

l2_block_rollup_simulation_time_in_ms
24,739 (+5%)
97,244 (+5%)
194,817 (+5%)

l2_block_public_tx_process_time_in_ms
9,986 (+5%)
39,394 (+6%)
79,133 (+7%)

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric
5 blocks
10 blocks

node_history_sync_time_in_ms
:warning: 18,426 (+20%)
:warning: 33,845 (+19%)

node_database_size_in_bytes
22,917,200
39,587,920

pxe_database_size_in_bytes
29,868
59,425

Circuits stats
Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit
circuit_simulation_time_in_ms
circuit_input_size_in_bytes
circuit_output_size_in_bytes
circuit_witness_generation_time_in_ms
circuit_proving_time_in_ms
circuit_proof_size_in_bytes

private-kernel-init
130 (+4%)
18,212
41,446
N/A
N/A
N/A

private-kernel-inner
361 (+2%)
66,727
41,446
N/A
N/A
N/A

private-kernel-reset
617 (+4%)
210,456
41,446
N/A
N/A
N/A

private-kernel-ordering
355 (+1%)
61,368
55,522
N/A
N/A
N/A

base-parity
8.18 (+5%)
128
64.0
309 (+12%)
12,730 (-5%)
2,208

root-parity
51.3 (+6%)
27,064
64.0
228 (+1%)
154,072 (-15%)
2,720

base-rollup
712 (+8%)
111,094
925
8,537 (+12%)
138,886 (-14%)
3,104

root-rollup
72.7 (+13%)
11,454
789
196 (+3%)
15,411 (-8%)
3,424

public-kernel-app-logic
135 (+4%)
68,697
59,431
313 (-7%)
489 (-2%)
73,024

public-kernel-tail
470 (+5%)
174,138
7,691
480 (-1%)
396 (-7%)
10,112

public-kernel-setup
110 (+4%)
68,697
59,431
300 (+2%)
457 (-8%)
73,024

public-kernel-teardown
114 (+5%)
68,697
59,431
279 (-3%)
403 (-4%)
73,024

merge-rollup
6.79 (+6%)
2,696
925
:warning: 110 (-30%)
3,554 (-4%)
3,104

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
16 leaves
64 leaves
128 leaves
512 leaves
1024 leaves
2048 leaves
4096 leaves
32 leaves

batch_insert_into_append_only_tree_16_depth_ms
13.1 (+19%)
21.5 (+20%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.7
31.8
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.769 (+19%)
0.663 (+21%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
62.0 (+20%)
97.4 (+19%)
:warning: 317 (+18%)
:warning: 611 (+21%)
:warning: 1,193 (+21%)
:warning: 2,364 (+21%)
N/A

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
95.9
159
543
1,055
2,079
4,127
N/A

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
0.634 (+20%)
0.601 (+19%)
0.577 (+19%)
0.571 (+21%)
0.567 (+21%)
0.565 (+21%)
N/A

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
73.6 (+20%)
144 (+21%)
:warning: 455 (+21%)
:warning: 897 (+21%)
:warning: 1,777 (+21%)
:warning: 3,534 (+20%)
N/A

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
106
208
692
1,363
2,707
5,395
N/A

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
0.645 (+21%)
0.648 (+21%)
0.620 (+21%)
0.618 (+21%)
0.618 (+21%)
0.616 (+21%)
N/A

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
80.8 (+20%)

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
108

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0.710 (+21%)

Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.

Metric
0 registered classes
1 registered classes

tx_size_in_bytes
58,427
1,049,033

Transaction size based on fee payment method
| Metric |  |
| - |  |
Transaction processing duration by data writes.

Metric
0 new note hashes
1 new note hashes
2 new note hashes

tx_pxe_processing_time_ms
2,653 (+5%)
2,797 (+6%)
7,112 (+7%)

Metric
0 public data writes
1 public data writes
2 public data writes
4 public data writes
8 public data writes

tx_sequencer_processing_time_ms
715 (+5%)
988 (+5%)
1,062 (+7%)
1,980 (+8%)
1,258 (+8%)

Actually, that code referencing NOIRUP_  env vars should be removed entirely as it's not relevant. Thanks for pointing that out.

For posterity's additional context, bbup is currently recommended in the Noir docs' installation section: https://noir-lang.org/docs/dev/getting_started/barretenberg/
If anyone reading is has an updated recommended way to install bb, please update the Noir docs and give the Noir team a heads up. Thanks!


Font of Jupyter Notebook output is not monospaced anymore

Applies To

[X] Notebooks (.ipynb files)
[ ] Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?
I open VScode and it automatically updates to 1.65.0 just now. When I open Jupyter notebook I find the font of the output is not monospaced anymore. I wonder if this issue comes with the new version because I worked with Jupyter notebook just several days ago. I try to find any configuration items in the settings but there is nothing.

Thanks!
VS Code Version
Version: 1.65.0 (user setup) Commit: b5205cc8eb4fbaa726835538cd82372cc0222d43 Date: 2022-03-02T11:12:08.962Z Electron: 13.5.2 Chromium: 91.0.4472.164 Node.js: 14.16.0 V8: 9.1.269.39-electron.0 OS: Windows_NT x64 10.0.19043
Jupyter Extension Version
v2022.2.1020642448
Jupyter logs
No response
Coding Language and Runtime Version
Python 3.8.10
Language Extension Version (if applicable)
Python Extension; Pylance Extension
Anaconda Version (if applicable)
Not using Anaconda
Running Jupyter locally or remotely?
Local

Same issue here. Jupyter output on VS code no longer in monospace font

Also same issue here.

Same issue, and it also applies to the Python Interactive Window


For multi-page scores, only prints the current page

For example, on https://musescore.com/user/19710/scores/58553 , it only loads the first page to print. I have made sure to load every page, but that does not seem to work. I am using Brave, which is derived from Chromium.

Sorry, I just fixed up the search.  The scale is a bit weird but if you print and set the scale to 25% it should be okay.  I'll have to dig into this later to see if there's a way to get the page to resize.  Bit weird.


Dependency "tar" - high severity vulnerability

npm is reporting that the "tar" (gulp-sass > node-sass > node-gyp > tar) dependency version is vulnerable to Arbitrary File Overwrite. https://nodesecurity.io/advisories/803
This issue only affects tar <4.4.2, would it be possible to update Gulp-Sass's dependency to a more recent version to prevent this?
Many thanks

this is currently being worked on by the node-gyp team: https://github.com/nodejs/node-gyp/pull/1713

@stof Looks like node-gyp merged the fix.

@RobertAKARobin they currently merged it only in the master branch, which is the dev version of the upcoming 4.0.
There is no release containing the fix yet. The work on backporting it to their 3.8 branch to create a patch 3.8.1 release is in progress in https://github.com/nodejs/node-gyp/pull/1718

Tracking in https://github.com/sass/node-sass/issues/2625.
Locking this issue in the mean time.

node-tar released a fix. Run npm update node-tar to resolve the npm audit warninds.


SQL Server monitoring support

Describe the feature request
Hi!
It would be awesome to be able to monitor my SQL Server to see if my database is reachable and 'works'.
This could be using the ODBC protocol for example.
A bit more extensive but Datadog uses this method to collect all their data, in this case I would only need to know if the database 'is alive'.
https://docs.datadoghq.com/database_monitoring/setup_sql_server/
Why do you personally want this feature to be implemented?
It would be nice to check if the database is alive and is working.
How long have you been using this project?
No response
Additional information
No response

Not a big fan of proprietary systems. If you can get the ODBC protocol working for mysql/postgres/etc and it happens to also support SQL server, I don't mind the feature, but I don't want to add a feature that only works for a proprietary system.

Understandable, I would be more a fan of a protocol that supports many vendors not just one.
The reason for requesting is a lot of my clients use a variant of Microsoft SQL Server and want to monitor it using a 'uptime' or connectivity tool.
I almost always recommend Gatus!
This is a missing feature for some of them that makes it hard to check connectivity to the database.


GitHub actions pipeline to build, test and deploy collector

          > > Also, the test framework might need to be executed in a CI/CD pipeline.

Yes that would be ideal! The question is can we spin up bare-metal VMs per run. I know the GCP console allows spinning up nodes with a timeout. I think it might be worth a search for a GitHub action that supports spinning up VMs for testing.
Seems like we need to get a short list of cloud providers to run the tests on, and an infra to spin those up ad-hoc (and the framework would need to support whatever provider we choose?)

Depending on the target cloud, we should look into a couple of Github actions. See action for terraform that can both provision and reclaim the test nodes or more cloud-specific actions (https://github.com/google-github-actions/setup-gcloud). We need an eventing/webhook action to listen and trigger the VM/Baremetal reclamation workflow. See - https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows
Either way, the CI/CD pipeline deserves its issue, too. I will create one.
Originally posted by @amosomokpo in https://github.com/perfpod/memory-collector/issues/2#issuecomment-2460284150

Q: Is terraform our best option or is there a solution with github actions? (And of terraform is best can we use opentofu instead)
A: If we are targeting multiple clouds, yes terraform is the best option. Not all cloud providers will have a GitHub action but all of them will more likely have a terraform provider. Opentofu sounds great, that’s the best option since it’s completely open source. I’ll look into their support for embedding in GitHub actions.

@yonch Is multiple clouds a requirements? Seems like only support for intel an AMD is a requirement at this point?

Yes. 👍
I think we can have a lighter-weight check to verify support in different cloud providers (as in issue #3), and creates a more extensive test suite in a single provider.

Here is an idea, following GitHub's "Autoscaling with self-hosted runners":

Run a Kubernetes cluster on one of the cloud providers
Use GitHub Actions to trigger tests
Tests run self hosted on the Kubernetes cluster. The actions-runner-controller seems to be the official controller for this.
Each test that requires a specific node type will trigger a runner that only runs on that node type

I believe we can add a nodeSelector in the AutoscalingRunnerSet from the values.yaml when deploying the controller (under template.spec). So this might be a controller deployment per node type.

This does not describe how to run the test workload on the test node, only how to provision and autoscale nodes.
Brainstorming ideas:

use dind to run a Kubernetes control plane on the node, and run workload containers through that control plane
add more containers to the runner template (but then we cannot use the helm charts for workloads)
modify workload helm charts to always run automatically on nodes of a certain type with DaemonSets (but that reduces the flexibility to modify workloads)

visited the GitHub booth at Kubecon, they validated the idea of running the Actions controller with runners that select a specific node pool, and autoscaling (e.g., with Karpenter). They also suggested running a Kubernetes cluster on the node under docker-in-docker with e.g., k3s.
Octopus Deploy folks described a way to use Terraform in their actions to spin up VMs explicitly, then add a step to create an artifact with the results. This seems like a more involved effort though than the Actions controller. The engineer mentioned testing is more of a CI than CD, and noted they had experience with TeamCity.
Shivansh from Loophole Labs noted that even an Kubernetes node with no Pods has a lot of "stuff" already running, and on GKE it is quite a lot (he showed an example where an empty node already had close to 1GB of memory occupied by these infrastructure services). These are noisy, so for benchmarks we should consider running on VMs rather than Kubernetes nodes. Loophole Labs offers a CI service that creates bare metal VMs with only the GitHub Actions runner, so we might use that for benchmarks. He put us on the waiting list.

@yonch do you mean running memory collector or telegraf in VMs? That might not be possible. https://github.com/intel/intel-cmt-cat/issues/207
There's also some earlier info here - https://github.com/intel/intel-cmt-cat/issues/162#issuecomment-663046431

https://github.com/aws/karpenter-provider-aws
https://github.com/Azure/karpenter-provider-azure
https://github.com/cloudpilot-ai/karpenter-provider-gcp (Still in development)


the typo in Docs

file on https://github.com/flutter/flutter/blob/master/packages/flutter/lib/src/widgets/async.dart has a typo in line 265 (at present):
  /// This will not be null iff [error] is not null. Consequently, [stackTrace]
typo in word "iff".
And yes, it is worthy of issue :D

Are you sure the devs didb't abbreviated "if and only if" as iff?

May be. Didn't think about it.

It clearly reads Only If (abbr : iff). This is Not an Issue. Kindly close the Same.

Closing, as this isn't an issue with Flutter itself. If you disagree, please write in the comments and I will reopen it.
Thank you


Allow Importing of External MIDI files

It'd be nice to allow importing of external MIDI files, since I have some that wouldn't be possible to include with the site itself due to copyright constraints, but I would still like to use them.

Hi,
thanks for your comment. I do not plan to develop the game further as I mainly made it for my own needs :). I also do not plan to extend hosting for musicope.com. This github page will however remain as it is. And to be honest, you are the first one interested in the game in many years :).
I can however recommend you to

download the game from this github page ( https://github.com/musicope/game/archive/master.zip )
unpack it
go to game-master/Musicope
execute mongoose-3.4.exe
in e.g. chrome go to http://localhost:8080/

To change the songs you can

contents of the songs/songs.json to fit your needs
alternatively you can point the game to an alternative json file e.g.: localhost:8080/list/?l_songsUrl=../songs/private.json

Hope it helps

Thanks for the reply! I'll have to give that a shot. :) Awesome job with the game as well. Not sure how difficult it would be, but you might be able to deploy it to Heroku or Openshift for free, so the hosting costs would be mitigated. I might actually try something similar to this using the upcoming MIDI APIs for the web. Anyway, awesome job on the site!

Thanks. I will investigate the option to host the game somewhere. Is there any browser that supports the MIDI API? I also would like to get rid of the Jazz plugin.

Glad to help! I know the W3 spec of the MIDI API is still a Working Draft, but some browsers, e.g. Chrome, may support it experimentally in some forms. I'm not sure if Firefox nightlies support it yet, but I'll have to check into that.
I know I found Musicope today, and thought it was pretty cool, so I'd hate to see it disappear. Regardless, thanks for all your hard work!

I have changed into nw.js and implemented web midi api. You can find the first win-32 release in the release section.


cmake build fails on flutter run

Hi,
I'm using agora_uikit as a package in my flutter app that installs this package as a dependency
However when I run the project, iris_method_channel fails to compile with errors,
(compiling this for an android emulator with sdk version 33)
I'm using the latest stable versions of flutter at the time of posting,

Flutter 3.10.2 • channel stable
Dart 3.0.2 • DevTools 2.23.1

here are the agora and iris versions that are installed,

iris_method_channel 1.1.0-rc.5
agora_uikit 1.3.3

I have limited experience in c/c++ and I would really appreciate any help to identify and fix the problem or any workarounds
Thank you so much,
Below are the logs
Execution failed for task ':iris_method_channel:buildCMakeDebug[arm64-v8a]'.
> com.android.ide.common.process.ProcessException: ninja: Entering directory `/home/heaust/.pub-cache/hosted/pub.dev/iris_method_channel-1.1.0-rc.5/android/.cxx/Debug/722d6s2f/arm64-v8a'
  [1/4] Building C object CMakeFiles/iris_method_channel.dir/dart-sdk/include/dart_api_dl.c.o
  [2/4] Building CXX object CMakeFiles/iris_method_channel.dir/iris_life_cycle_observer.cc.o
  [3/4] Building CXX object CMakeFiles/iris_method_channel.dir/iris_event.cc.o
  FAILED: CMakeFiles/iris_method_channel.dir/iris_event.cc.o
  /opt/android-sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ --target=aarch64-none-linux-android21 --gcc-toolchain=/opt/android-sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64 --sysroot=/opt/android-sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DDART_SHARED_LIB -Diris_method_channel_EXPORTS  -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security   -fno-limit-debug-info  -fPIC -MD -MT CMakeFiles/iris_method_channel.dir/iris_event.cc.o -MF CMakeFiles/iris_method_channel.dir/iris_event.cc.o.d -o CMakeFiles/iris_method_channel.dir/iris_event.cc.o -c /home/heaust/.pub-cache/hosted/pub.dev/iris_method_channel-1.1.0-rc.5/src/iris_event.cc
  In file included from /home/heaust/.pub-cache/hosted/pub.dev/iris_method_channel-1.1.0-rc.5/src/iris_event.cc:6:
  /snap/flutter/current/usr/include/c++/9/memory:121:25: error: cast from pointer to smaller type 'uintptr_t' (aka 'unsigned int') loses information
    const auto __intptr = reinterpret_cast<uintptr_t>(__ptr);
                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /home/heaust/.pub-cache/hosted/pub.dev/iris_method_channel-1.1.0-rc.5/src/iris_event.cc:90:17: error: use of undeclared identifier 'memcpy'; did you mean 'wmemcpy'?
                  memcpy(abuffer, obuffer, abufferLength);
                  ^~~~~~
                  wmemcpy
  /snap/flutter/current/usr/include/wchar.h:262:17: note: 'wmemcpy' declared here
  extern wchar_t *wmemcpy (wchar_t *__restrict __s1,
                  ^
  /home/heaust/.pub-cache/hosted/pub.dev/iris_method_channel-1.1.0-rc.5/src/iris_event.cc:90:24: error: cannot initialize a parameter of type 'wchar_t *' with an lvalue of type 'uint8_t *' (aka 'unsigned char *')
                  memcpy(abuffer, obuffer, abufferLength);
                         ^~~~~~~
  /snap/flutter/current/usr/include/wchar.h:262:46: note: passing argument to parameter '__s1' here
  extern wchar_t *wmemcpy (wchar_t *__restrict __s1,
                                               ^
  3 errors generated.
  ninja: build stopped: subcommand failed.

  C++ build system [build] failed while executing:
      /opt/android-sdk/cmake/3.18.1/bin/ninja \
        -C \
        /home/heaust/.pub-cache/hosted/pub.dev/iris_method_channel-1.1.0-rc.5/android/.cxx/Debug/722d6s2f/arm64-v8a \
        iris_method_channel
    from /home/heaust/.pub-cache/hosted/pub.dev/iris_method_channel-1.1.0-rc.5/android

duplicated https://github.com/AgoraIO-Extensions/Agora-Flutter-SDK/issues/1086


update django to 4.2

if you do not want cacheops in debug mode, you should set CACHEOPS_ENABLED = False in your settings now.
django-select2-forms is not compatible with django 4+. the way i understand it, it has not been developed for a while, since django 2 supposedly supports select2 directly. i don't really see where we even use select2, so i simply removed it. removing it is not possible, unless we squash migrations.

just noticed that django 5 was released in the meantime. will soon work on upgrading.

LGTM, a few minor cosmetic things @lakinwecker can we get this merged?

i'll fix the minor stuff on tuesday/wednesday.

I trust you all more than I trust myself on this stuff at the moment. I won't have time to review it in depth, anytime soon. Do you need me to mash the merge button? Or does one of you have the perms to do that?
I can deploy whenever you are ready

I ran into an error testing this locally so don't merge yet.
Django 4.1 upgraded the ManifestStaticFilesStorage method for collecting static files and that's playing havok with our mapless bootstrap files (I suspect that's the reason that the tests are using the more tolerant StaticFilesStorage).
ref: https://github.com/glbert-does/heltour/blob/70a1edcbec3923029412ad179967cc6e5e49097a/heltour/tournament/static/lib/css/bootstrap.min.css#L6

i wonder why i did not have that local error.
do i understand this correctly that you're saying a line like the one you pointed to needs to be added to all css files that are missing them?
so e.g. heltour/tournament/static/lib/css/BootstrapXL.css, which does not have it currently
needs /*# sourceMappingURL=BootstrapXL.css.map */?
i will fix this tomorrow/day after tomorrow as well. also, i will try to see why i didn't have the error locally.
PS: lakin, you gave me the merge perm. but the only way for me to use it responsibly is to not use it.

do i understand this correctly that you're saying a line like the one you pointed to needs to be added to all css files that are missing them?

no. I'm suggesting that some of the current files are missing maps for their minified content but are still referencing them. The old version of ManifestStaticFilesStorage  was permissive of this, but the new version isn't.
I'm not sure what settings you are running locally but if you change the staticfiles finder to ManifestStaticFilesStorage and try python manage.py collectstatic that should produce the error I was seeing. off the top of my head it was complaining about no being able to find /lib/css/bootstrap.css.min.map or similar

@nicwest fixed all the minor issues you commented about above.
i also tried to read up on the ManifestStaticFilesStorage. and i did create the map files. i understand more about static file storage now (easy, as opposed to zero before), but i won't pretend to fully comprehend why empty files would be needed.

you can grab the files from here:  https://stackpath.bootstrapcdn.com/bootstrap//css/
https://github.com/glbert-does/heltour/blob/2d83b28015ea49240f94d481f56860c75077b6a8/heltour/tournament/static/lib/css/bootstrap-theme.min.css#L2C15-L2C20 the version is at the top of the file.

right. i added those map files, but this is some dark magic. locally, the empty files did not lead to an error for me already, so i am not convinced i fixed it.
i turned debug on and off again, and switched around some folders and STATIC*-settings, and in the end i did manage to break everything. doesn't seem related to those map files though, so i am still in the dark.

you may need to to run poetry lock --no-update


Restructure code, and rename InfExtended to InfExtendedReal

closes #5
I tried moving everything in the way we discussed previously. Let me know if I moved something in a place you weren't wanting.
I also decided to do the renaming of InfExtended to InfExtendedReal. Since this is a breaking change, and this package isn't at 1.0.0 yet, I just renamed it and bumped the version to 0.2.0. Not sure if we want to set a deprecation cycle for this.

Codecov Report

Merging #12 into master will decrease coverage by 3.27%.
The diff coverage is 46.29%.

@@            Coverage Diff             @@
##           master      #12      +/-   ##
==========================================
- Coverage   53.27%   50.00%   -3.28%
==========================================
  Files           8       13       +5
  Lines         122      118       -4
==========================================
- Hits           65       59       -6
- Misses         57       59       +2

Impacted Files
Coverage Δ

src/Infinity.jl
100.00% <ø> (ø)

src/infextendedreal/io.jl
0.00% <0.00%> (ø)

src/infinite/io.jl
0.00% <ø> (ø)

src/infinite/rand.jl
100.00% <ø> (ø)

src/infextendedreal/conversion.jl
20.00% <20.00%> (ø)

src/infinite/arithmetic.jl
25.00% <25.00%> (ø)

src/infextendedreal/base.jl
36.36% <36.36%> (ø)

src/infextendedreal/comparison.jl
46.15% <46.15%> (ø)

src/infextendedreal/arithmetic.jl
53.33% <53.33%> (ø)

src/infinite/conversion.jl
57.14% <57.14%> (ø)

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fa73ee8...29b1d11. Read the comment docs.

Thanks very much. Yes this and InfExtendedTime will become v0.2.0. As this is still "alpha" a deprecation cycle is not necessary.


[BugFix] Fix error message when exceeding big query mem limit

Big query mem limit directly uses query-level memory tracker. Therefore, it will report Mem usage has exceed the limit of single query, when the query memory usage exceeds the big query mem limit.
Mem usage has exceed the limit of single query, You can change the limit by set session variable query_mem_limit.

Modify the error message to
Mem usage has exceed the big query limit of the resource group [Group=group1, Query46ea64e2-213a-11ee-8f21-daaa37d1a479].
You can change the limit by modifying [big_query_mem_limit] of this group

What type of PR is this:

[ ] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Checklist:

[x] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.1
[x] 3.0
[x] 2.5
[x] 2.4
[x] 2.3

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

@Mergifyio backport branch-3.1

@Mergifyio backport branch-3.0

@Mergifyio backport branch-2.5

@Mergifyio backport branch-2.4


feat: add liferay-portal/no-explicit-extend rule

I wanted to put this in the repo because it is an example of a liferay-portal-specific lint that:

Only applies to certain files.
Includes an "autofix" functionality.

Test plan:
Run this in liferay-portal without:
https://github.com/brianchandotcom/liferay-portal/pull/75813
See:
modules/apps/data-engine/data-engine-taglib/.eslintrc.js
  16:12  error  `liferay/portal` applies automatically and can be omitted  liferay-portal/no-explicit-extend

modules/apps/dynamic-data-mapping/dynamic-data-mapping-form-builder/.eslintrc.js
  16:12  error  `liferay/portal` applies automatically and can be omitted  liferay-portal/no-explicit-extend

modules/apps/dynamic-data-mapping/dynamic-data-mapping-form-web/.eslintrc.js
  16:12  error  `liferay/portal` applies automatically and can be omitted  liferay-portal/no-explicit-extend

modules/apps/layout/layout-content-page-editor-web/.eslintrc.js
  16:12  error  `liferay/portal` applies automatically and can be omitted  liferay-portal/no-explicit-extend

modules/apps/segments/segments-web/.eslintrc.js
  16:12  error  `liferay/portal` applies automatically and can be omitted  liferay-portal/no-explicit-extend

✖ 5 problems (5 errors, 0 warnings)
  5 errors and 0 warnings potentially fixable with the `--fix` option.

ESLint checked 562 files, found 5 errors, found 0 warnings, 5 issues potentially fixable with lint:fix

Which are the same files reported by git grep 'extends:.+liferay':
apps/app-builder/app-builder-web/.eslintrc.js:16:       extends: ['liferay/react']
apps/data-engine/data-engine-taglib/.eslintrc.js:16:    extends: ['liferay/portal', 'liferay/metal']
apps/dynamic-data-mapping/dynamic-data-mapping-form-builder/.eslintrc.js:16:    extends: ['liferay/portal', 'liferay/metal']
apps/dynamic-data-mapping/dynamic-data-mapping-form-web/.eslintrc.js:16:        extends: ['liferay/portal', 'liferay/metal']
apps/layout/layout-content-page-editor-web/.eslintrc.js:16:     extends: ['liferay/portal', 'liferay/react'],
apps/segments/segments-web/.eslintrc.js:16:     extends: ['liferay/portal', 'liferay/react']

After running autofix, see this diff:
https://gist.github.com/wincent/6874c549f011937104266800ab724475

LGTM, @andresfulla, just FYI since we talked about this this morning.


This adds a test harness so we can do future tests more easily

I am going to auto merge this.  If its not needed it can be deleted.
I think this will help us build better tests ongoing.


Stopping Audio Not Working

The audio event has a default value of nothing, and should stop audio playback like that. But currently it does not, it will do nothing. It can play audio files just fine, just does not stop any currently playing if it's left blank

Can you specify if this is for the 1.x or 2.x branch? Sorry these are confusing times.

This is 1.4.x. I'll be taking a peak later on it.

@zaknafean is this still relevant?

Not sure when it was fixed, but it is definitely working properly as of now.


Update dependencies inclusding Kubernetes v1.21

Codecov Report

Merging #202 (4aafb81) into master (c8e8f0b) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #202   +/-   ##
=======================================
  Coverage   75.99%   75.99%
=======================================
  Files          28       28
  Lines        1462     1462
=======================================
  Hits         1111     1111
  Misses        263      263
  Partials       88       88

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c8e8f0b...4aafb81. Read the comment docs.


Proposal: Add NamedGet and NamedSelect Methods to sqlx

Proposal:
I propose the addition of two new methods, NamedGet and NamedSelect, to the sqlx library. These methods would work similarly to their counterparts but would allow for the use of named parameters, enhancing consistency and usability for developers who prefer or require named parameters in their queries.
Looking forward to hearing your thoughts and feedback on this proposal.

Sounds like an idea to me, we could do. But then we would need documentation for this. So we need a follow-up issue to set up gh-pages with the documentation and update it.


Roll Engine from 3c9308faad0b to 57bb508a6605 (1 revision)

https://github.com/flutter/engine/compare/3c9308faad0b...57bb508a6605
2020-09-15<EMAIL_ADDRESS>Roll Skia from 22aa7d791515 to d911c91d8895 (5 revisions) (flutter/engine#21181)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/master/autoroll/README.md

Commit queue failed; closing this roll.


Short solution needed: "How to read file with Lua" (nginx-lua)

Please help us write most modern and shortest code solution for this issue:
How to read file with Lua (technology: nginx-lua)
Fast way
Just write the code solution in the comments.
Prefered way

Create pull request with a new code file inside inbox folder.
Don't forget to use comments to make solution explained.
Link to this issue in comments of pull request.

How to read file with Lua


Certs support in decode transaction

adp-1245
Support in tx for certificates:

delegation certs belonging to a given wallet -> then also reward account's derivation path is added

delegation certs not belonging to a given wallet

pool registration/deregistration certs

genesis cert

MIR certs

[x] I have extended support in basic types and in shelley for all certificates

[x] I have extended decodeTx to account for certificates

[x] I have extended ApiDecodedTransaction for "certificates" and added all needed bits in swagger

[x] I have supported extended ApiDecodedTransaction in Api.Server.Types

[x] I have implemented this support in Api.Server's decodeTransaction

[x] I have added illustrative integration tests

Comments
When constructTx will have delegation support we will need to check ours delegation certs in txs. For now txs with external delegation certs (in the context of a given wallet) are checked in integration testing. Also the detection of txs with pool registration/deregistration, and mir certs is tested

Issue Number

@piotr-iohk @Anviking I refactored json outputs - now everywhere "certificate_type" is present. Also introduced Certificate wich is a sum type, and we maintain certificates order now. Some refactoring of Shelley.Compatibility was needed (but I think it makes things clearer and simpler in the end)

bors r+