Execution is stuck until termination when running on CUDA in WSL2

Reproducing the behavior
The issue
Hi,
I am having issues running code with bend run-cu on CUDA inside the WSL2. There are not errors, but code is not executing either. Execution is frozen, similary to while (true) {}.
Code executes without any issues when using bend run or bend run-c
Compiling with bend gen-cu  and nvcc has the same result.
I've tried both 12.4 and 12.5 and result is the same.
What I attempted
I tried different code examples from the repo, but result is always the same. Since bend allowed to compile code to cuda with gen-cu, I tried to find what is broken inside generated file (assuming bend run-cu will use the same code). This issue happened inside gnet_normalize function, where code could never exit the for loop. This break is never callen (rlen always has the same value)
CUDA verification
Just to rule CUDA out, I have successfully installed CUDA and can confirm it is recognised by compiling and running this code.
cuda-test.cu:
#include <cuda.h>
#include <stdio.h>

int main(int argc, char** argv) {
  int driver_version = 0, runtime_version = 0;

  cudaDriverGetVersion(&driver_version);
  cudaRuntimeGetVersion(&runtime_version);

  printf(
    "Driver Version: %d\nRuntime Version: %d\n",
    driver_version,
    runtime_version
  );

  return 0;
}

output
~> nvcc cuda-test.cu -o cuda-test && ./cuda-test
Driver Version: 10010
Runtime Version: 12040

nvidia-smi
Calling from wsl
>  nvidia-smi.exe
Sun Jun  2 16:32:08 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 552.44                 Driver Version: 552.44         CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                     TCC/WDDM  | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3070      WDDM  |   00000000:08:00.0  On |                  N/A |
| 49%   56C    P3             41W /  220W |    6496MiB /   8192MiB |     36%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      3832    C+G   ...oogle\Chrome\Application\chrome.exe      N/A      |
|    0   N/A  N/A      8292    C+G   C:\Windows\explorer.exe                     N/A      |
|    0   N/A  N/A     21004    C+G   ...\cef\cef.win7x64\steamwebhelper.exe      N/A      |
|    0   N/A  N/A     21416    C+G   ...wekyb3d8bbwe\XboxGameBarWidgets.exe      N/A      |
|    0   N/A  N/A     27344    C+G   ...ience\NVIDIA GeForce Experience.exe      N/A      |
....
+-----------------------------------------------------------------------------------------+

System Settings
Example:

HVM: 2.0.18
Bend: 0.2.27
OS:  Ubuntu 20.04.6 LTS
WSL:  <IP_ADDRESS>
CPU: AMD Ryzen 9 5900X
GPU: RTX 3070
Cuda Version: 12.5, V12.5.40

Additional context
No response

Is this for any program you try to run?
Also, what compiler version is nvcc using? By default it should be g++ and you can check its version with g++ --version

Yes, all exmples from examples folder have the same beheviour.
For example compiled fib and it fail to break the same loop, because rlen is aways has the same value (which is different from run to run, but never zero)
nvcc and g++
~ > nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Wed_Apr_17_19:19:55_PDT_2024
Cuda compilation tools, release 12.5, V12.5.40
Build cuda_12.5.r12.5/compiler.34177558_0

~ > g++ --version
g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I would like to also add that I have this same issue on my laptop running ubuntu linux. I have tried the sorter.bend script from the README on three machines now and I have gotten it to run on my other two (one has a laptop 3080 gpu and the other running dual RTX A4000s) however it won't run on my laptop with a Quadro P600 (cuda version 12.2). It does the same thing @wtfil is describing where it simply hangs.
Looking at the processor usage, it seems as though it might be having an issue allocating GPU memory (?). I can see it running 100% on a single cpu core but never load either the RAM or VRAM and there is no processing being run on the GPU either. Here are some specs from the machine in question if this might be a bug to be fixed in the future. (Already love this programming language BTW, really hoping I can start switching over to it at work when more support is added in).
Info:
HVM: 2.0.19
Bend: 0.2.27
OS: Ubuntu 22.04.4 LTS
Kernel: 6.5.0-35
CPU: Intel i5-9300H
GPU: Nvidia Quadro P600 Mobile
g++: 11.4.0
nvcc: 12.2
nvidia driver: 535.171.04

I'm getting the same issue with:
HVM: 2.0.19
Bend: 0.2.33
Ubuntu 22.04.4 LTS x86_64
Kernel: 6.5.0-35
CPU: AMD Ryzen 7 5800H
GPU: NVIDIA GeForce RTX 3070 Mobile
g++: 11.4.0
nvcc: 12.5
nvidia driver: 550.67

Same issue here. I tried debugging this issue in the official discord server here.
As @keaneflynn observed, it doesn't allocate vram correctly, and hangs for a long time until it crashes with the following message:
Failed to launch kernels. Error code: an illegal memory access was encountered.
Errors:
Failed to parse result from HVM.

I waited 45 minutes while another discord server member only waited 30 minutes with his example in a virtual machine. I don't think the time is as important since the program crashed shortly after I launched another application (steam in my case).
A quick summary of the debugging we did in discord: Downgrading from cuda 12.5 to 12.4 doesn't help. Examples unrelated to bend compiled by nvcc work just fine, so it's not a (simple) issue with cuda. Using run and run-c work for the examples provided by bend while using run-cu directly or gen-cu then compiling with nvcc hangs. A different member of the server mentioned it could be an issue with the smaller L1 cache size of my GXT 1660, but that doesn't seem to be the case due to multiple 3070's listed here also not working.

My Specs

HVM: 2.0.18
Bend: 0.2.27
Ubuntu 22.04.4 LTS x86_64
Kernel: 6.5.0-35
CPU: AMD Ryzen 7 5800X 8-Core Processor
GPU: NVIDIA GeForce GTX 1660 (ti I think?)
g++: 11.4.0
nvcc: V12.4.131
nvidia driver: 555.42.02

I'm getting the same issue with: HVM: 2.0.19 Bend: 0.2.33 Ubuntu 22.04.4 LTS x86_64 Kernel: 6.5.0-35 CPU: AMD Ryzen 7 5800H GPU: NVIDIA GeForce RTX 3070 Mobile g++: 11.4.0 nvcc: 12.5 nvidia driver: 550.67

This is fascinating as I have a laptop with nearly identical specs that does manage to use the run-cu properly. I have the 5800H on ubuntu 22.04 except it has a 3080 mobile. I am pretty sure the cache on these two chips are identical per @nmay231 inquiry. Hoping to see some bug fixes here soon!

Same issue on GTX 1050 Ti, Windows 10 22H2, Intel i5-6400. Not at home right now, but I'll get more info later!

Hey, I am also facing the same issue. All the dedicated GPU memory gets full. And the process is stuck.
bend 0.2.33 hvm 2.0.19 nvcc V12.5.40 wsl-ubuntu 22.04.3 LTS gpu NVIDIA GeForce RTX 2060

@Imran-S-heikh I'm not certain if we have the same issue. My Video RAM for the bend/hvm process never went above 100 MiB.
Perhaps we should all make sure we are experiencing the same thing. Here's a very simple program, that shouldn't need much memory. It hangs with bend run-cu. I checked my GPU memory usage with nvtop.
def main:
  return (1 + 1)

Also, I forgot to mention I did try running run-cu --verbose and this is the output of the program above.

bend run-cu --verbose simple.bend

% bend run-cu -v simple.bend
(Map/empty) = Map/Leaf

(Map/get map key) = match map = map { Map/Leaf: (*, map); Map/Node: switch _ = (== 0 key) { 0: switch _ = (% key 2) { 0: let (got, rest) = (Map/get map.left (/ key 2)); (got, (Map/Node map.value rest map.right)); _ _-1: let (got, rest) = (Map/get map.right (/ key 2)); (got, (Map/Node map.value map.left rest)); }; _ _-1: (map.value, map); }; }

(Map/set map key value) = match map = map { Map/Node: switch _ = (== 0 key) { 0: switch _ = (% key 2) { 0: (Map/Node map.value (Map/set map.left (/ key 2) value) map.right); _ _-1: (Map/Node map.value map.left (Map/set map.right (/ key 2) value)); }; _ _-1: (Map/Node value map.left map.right); }; Map/Leaf: switch _ = (== 0 key) { 0: switch _ = (% key 2) { 0: (Map/Node * (Map/set Map/Leaf (/ key 2) value) Map/Leaf); _ _-1: (Map/Node * Map/Leaf (Map/set Map/Leaf (/ key 2) value)); }; _ _-1: (Map/Node value Map/Leaf Map/Leaf); }; }

(IO/MAGIC) = (13683217, 16719857)

(IO/wrap x) = (IO/Done IO/MAGIC x)

(IO/bind a b) = match a = a { IO/Done: (b a.expr); IO/Call: (IO/Call IO/MAGIC a.func a.argm λx (IO/bind (a.cont x) b)); }

(call func argm) = (IO/Call IO/MAGIC func argm λx (IO/Done IO/MAGIC x))

(print text) = (IO/Call IO/MAGIC "PUT_TEXT" text λx (IO/Done IO/MAGIC x))

(get_time) = (IO/Call IO/MAGIC "GET_TIME" * λx (IO/Done IO/MAGIC x))

(sleep hi_lo) = (IO/Call IO/MAGIC "PUT_TIME" hi_lo λx (IO/Done IO/MAGIC x))

(main) = (+ 1 1)

@nmay231 I get the exact same output. I also have the same results- it uses all my CPU but no GPU

Can someone with the issue try running cuda version 11.x? (sudo apt install nvidia-cuda-toolkit will get version 11)
With my 980Ti, on cuda 11, I would instantly get the Failed to launch kernels error, whereas on cuda 12.5, the program just hangs.
This issue is most likely same as #364, where GPU memory architecture is the cause because bend was only tested on an rtx 4090.

@TimotejFasiang For me, sudo apt install nvidia-cuda-toolkit installed version 12.0.140~12.0.1-4build4, and I couldn't find any full 11.x version numbers to specify (e.g: E: Unable to locate package nvidia-cuda-toolkit@11.8.0-1). I don't use Linux very often so I might be doing something silly, though

I have a little update on the issue, hope this will help to understand it better.
Initially I faced this issue when used<EMAIL_ADDRESS>from WSL.
Today I tried my second image -<EMAIL_ADDRESS>on the same machine and it worked! This image was almost fresh so I run installations steps from the README and bend run-cu worked right away
Here are the version of relevant tools for both images

name
<EMAIL_ADDRESS><EMAIL_ADDRESS>

os
Ubuntu 20.04.6 LTS
Ubuntu 22.04.3 LTS

gcc
gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

nvcc
release 12.5, V12.5.40
release 12.5, V12.5.40

hvm
2.0.19
2.0.19

bend
0.2.33
0.2.33

cargo
1.80.0-nightly
1.78.0

The only major different is gcc between two
I also noticed that nvidia-smi (not the nvidia-smi.exe) is failing on<EMAIL_ADDRESS>with following error:
NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

nvidia-smi  on<EMAIL_ADDRESS>works fine
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 555.85                 Driver Version: 555.85         CUDA Version: 12.5     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3070      WDDM  |   00000000:08:00.0  On |                  N/A |
|  0%   46C    P8             22W /  220W |    2652MiB /   8192MiB |     36%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

<EMAIL_ADDRESS>~/www/bend-examples > time bend run bitonic_sort.bend
Result: 16646144

real    0m35.908s
user    0m33.658s
sys     0m2.250s
~/www/bend-examples > time bend run-c bitonic_sort.bend
Result: 16646144

real    0m10.611s
user    1m3.683s
sys     1m44.188s
~/www/bend-examples > time bend run-cu bitonic_sort.bend
Result: 16646144

real    0m2.410s
user    0m1.996s
sys     0m0.080s

I am noticing similar. When running on WSL2, I noticed that the default parallel_hello_world would not finish (before I got bored of waiting and figured something was wrong). I have a standard RTX 3070, btw. I rewrote things to play around, and found that it ran plenty fast when running gen(13), just not gen(16). My guess is some issue with using too much GPU memory and getting stuck attempting to get more, as some have mentioned here.
Moreover, when running gen(16), my GPU continued to be fully running after CTRL+C the command line and attempting to terminate the process. Is this related to having no IO?

@evbxll I'm not sure if that issue is related, because the issue I and others are describing happens even for this
def main:
  return (1 + 1)

which shouldn't consume much (if any) GPU memory. Also, the issue we're describing results in all our CPU being used, but none of our GPU, and CTRL+C does stop it from using all the CPU.

@evbxll Sounds like a different issue. The issue I and others are describing happens even for a very simple program like the one below
def main:
  return (1 + 1)

Also, the issue we're describing results in all our CPU being used, but none of our GPU, and CTRL+C does stop it from using all the CPU for me

Eh, I feel like the original issue these comments are under is similar to me. Execution stuck when running CUDA WSL2

Yeah, one also get this issue, so one followed a few steps:

#364 solution (since one had an 1080 Ti) but modify on the current latest version hvm v2.0.19 (instead of the v2.0.13 @hubble14567 originally mentioned).
Follow @wtfil suggestion to install ubuntu 22.04 (and 24.04 LTS just in case, but both are installed on separate disk because you can't install both on the same folder same disk, otherwise, it'll share a single virtual disk and that's a huge issue); in the end, both works.
Update driver from 536.xx (forgot the exact version) to 555.99 (the current latest version). If one tries to run nvidia-smi now should get segmentation error something like that. Restart your computer, then the error should be gone. Now, run bend run-cu simple.bend -s and it'll have no problem.

The fix is quite stupid, because it seems to take so long to move data from cpu to gpu that running the simple.bend suggested above took 5-6 seconds.
<EMAIL_ADDRESS>$ bend run-cu simple.bend -s
Result: 2
- ITRS: 2
- LEAK: 0
- TIME: 5.83s
- MIPS: 0.00

is there a fix for this?
running into same issue


Add Swagger UI

Added Swagger UI to run on admin path.
Implementation is a bit tricky, but all the steps are documented in the Java doc for HelloWorldApplication.registerSwaggerAtAdminPath method.
Insights into the implementation are at the following links:
https://www.reonomy.com/blog/post/augmenting-dropwizard-with-swagger
https://spin.atomicobject.com/2014/10/11/serving-static-assets-with-dropwizard/
Adding swagger libraries entails a lot of exclusions, otherwise there are compilation errors of the 'ClassNotFound' type.
Open Question:  Current Swagger configuration will make ALL endpoints (Resource endpoints, admin endpoints, internal(?) endpoints) available in Swagger. Execution of the admin endpoints does not work due to path issues. This is what our Swagger page looks like:

Question: Do we want to try to limit Swagger to just our resource endpoints? I think it is probably possible via some config, but it will take some figuring out. The drawback is that we may need to be extending the config to include new resources in Swagger as we add them. What do you think, Greg?
My inclination is to leave Swagger as it is for now and worry about enhancing its config later if we would so wish.

Yeah, copying Swagger UI dist into our code base isn't pretty. But as far as I can tell, this is what most solutions (leakycauldron, and https://www.reonomy.com/blog/post/augmenting-dropwizard-with-swagger) do.
I agree, the approach you suggest sounds better. I will add a TODO to basecamp for it.
For now, we go with the current implementation.


CI: RTD build is useless

Hi there,
while working on #349, I found the RTD build on CI to be broken ^1. However, it is not related to Sphinx 5, because it already happens beforehand. I've verified it with #350 and the outcome is likewise broken ^2.
For the RTD build, the crate-docs project is used. However, it is apparently not suitable for being inspected on CI. It would be so sweet if it could be used as a canary for verifying updates to the theme without needing them to be propagated downstream, no?
With kind regards,
Andreas.
/cc @msbt

@amotl I've compared the raw logs from the Run build before upgrading to Sphinx 5 docs #338 build with some of earlier runs (e.g. SEO improvements (content wider than screen) and other smaller fixes build #283) and it seems that there are a few things missing.
I can't find the line with .env/bin/pip3 install -r requirements.txt which subsequently would install nodeenv and all the webpack stuff. It also says Using legacy 'setup.py install' for livereload, since package 'wheel' is not installed. which it doesn't on the older build. I reckon that's why the bundle/main.js with its assets is missing.

Now I am seeing it. I think there are two anomalies.
Application bundle missing
This is what you reported, @msbt, right?
GET https://crate-docs-theme--349.org.readthedocs.build/en/349/_static/bundle/main.js?ver=0.24.5
Status: 404 Not Found

Wrong project association
I  believe the documentation of crate-docs-theme thinks it is both associated with the project crate-docs-theme, as well as fake, which looks like it is not published at the moment.
GET https://crate.io/docs/fake/_/static/javascript/readthedocs-doc-embed.js?ver=0.24.5
Status: 404 Not Found

Outlook

Investigate why the application bundle file bundle/main.js is not present.
Investigate the misguidedness of the project name/url, wrt. crate-docs-theme vs. fake.

@amotl yes, exactly, I was investigating why main.js (and related assets) from the webpack bundler aren't present and those were the attempts to figure out why. Thanks for checking those out!

Investigate why the application bundle file bundle/main.js is not present.

Oh it just occurred to me that this would actually be expected. While the JavaScript/CSS bundle is part of the package published to PyPI, it is obviously missing on a bare repository checkout and would need to be bundled beforehand.
So, this is actually a natural implication of #255. webpack probably can't run on RTD, so we will have to come up with a different solution.

GH-379 fixed this issue, so rendering previews and finals on RTD works well now.
-- https://crate-docs-theme.readthedocs.io/


compiling busybox file system stucked in 'mknod' step

Version
10.0.18363.418
WSL Version

[ ] WSL 2
[X] WSL 1

Kernel Version
No response
Distro Version
No response
Other Software
No response
Repro Steps
cd busybox "_install/dev" folder , when execute "mknod console c 5 1" but get result "operation not permitted",
get the same result when i switch to root user .
Expected Behavior
execute mknod console c 5 1 and succeed
Actual Behavior
execute "mknod console c 5 1" but get result "operation not permitted"
Diagnostic Logs
No response

I also test this step on wsl2 and get result of "mknod: console: Operation not supported"

Maybe it's time to upgrade to WSL2.

Maybe it's time to upgrade to WSL2.

Have tested on WSL2 . but also failed with different error message "mknod: : Operation not supported"

i have the exactly same problem，anyone got an idea why its not working？

Same question, may be WSL didn't implement some feature of linux device


Allow users interactively choose some options.

When there is more than one device and/or snapshot available the user can select one of them to download. Additionally, when passing the option --domains, the user can optionally select one of the domains.
I really appreciate your work on this project. As I'm not a crypto expert, this was the way I found to contribute.

Sorry the delay and thank you! I'll work through the merge conflicts at some point this week.


syz-manager: ignore repro.Run for suppressed crash

Before sending a pull request, please review Contribution Guidelines:
https://github.com/google/syzkaller/blob/master/docs/contributing.md

Hi Joey!
Are you observing suppressed bugs being reproduced?
For the non-dashboard mode, it should be prohibited here
https://github.com/google/syzkaller/blob/867e44df36d93e8127938eca6f6a5c339a2ba0b8/syz-manager/manager.go#L688-L691
If you connect syz-manager(s) to some web dashboard, it would make more sense to disable suppressed bug reproductionin your config:
https://github.com/google/syzkaller/blob/867e44df36d93e8127938eca6f6a5c339a2ba0b8/dashboard/app/config.go#L110-L111


Should not be installed directly on VCenter7

Hello,
Should warn on the readme that it should not be installed directly on Vcenter 7, because internal process vmware-rhttpproxy won't work anymore since your installer overwrite the pyvmomi library.
Will break your Vcenter.

There was a modicum of 'assumed knowledge' here that most people know better than to install pretty much anything on a managed appliance.


OAL Logos Improperly Scaling

Describe the bug
Various partner airlines are not scaling as expected
To Reproduce
Steps to reproduce the behavior:
https://components.alaskaair.com/catelog/demo/airline-helper
See all logos that are improperly displayed. Expecting these SVGs to scale out to the edges of the container.
Expected behavior
SVG fills container

Screenshots
Impacted logos:
BA, AY, HU, HA, QF, LH, JL, DE, DL, FI, AM
Should we consider moving the "Pacific" under "Cathay" for CX, as well as "airlines" for JL?

Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Issue accidentally associated with unrelated PR.

Here is what I have come up with in regards to this issue. Here is a demo page of all the logos http://ota-logo.surge.sh/ota
In your code you are brute-forcing the width and the height of the SVG and that's not going to work. Not any of these SVGs have the same viewBox dimensions. Because of that, there is no way that you can do what you are trying to do.
Notice in the demo I am only restricting the width and some of the boxes have heights. That is because the height is baked into the viewBox of the SVG and that will be a LOT OF WORK to try and edit those by hand on code.
I'd suggest a couple of things to try here.

I'd put a div around the SVG and control the size of that, not the SVG itself. That way you don't need to use all those important statements.
If you can, use the auro-icon component and you can address the size of that container versus a div.

In the end, these SVGs and this design pattern are pretty incompatible. I feel like there is a better design solution somewhere for this.

This issue has been taken up by the design team to normalize the OTA logos.

I am assuming that the solution to this logo issue is the new tail icons? If that is so, am I to assume that the logos can be removed from the lib?

I am assuming that the solution to this logo issue is the new tail icons? If that is so, am I to assume that the logos can be removed from the lib?

That is the solution, but we will be in production with the fullsize logos on the matrix page (not the details modal like we discussed) for several months. I will let you know once we retire that old experience so we can remove it here.

That is the solution, but we will be in production with the fullsize logos on the matrix page (not the details modal like we discussed) for several months. I will let you know once we retire that old experience so we can remove it here.

If that is the case, I am going to close this issue as there is no point in keeping this open for months. When the time comes to remove the legacy logos, we can create a new issue at that time.

If that is the case, I am going to close this issue as there is no point in keeping this open for months. When the time comes to remove the legacy logos, we can create a new issue at that time.


RS Website Content - Sender onboarding

Problem statement
As a sender of the RS public website's resources section, I would like to see guidance on onboarding for a sender.
Currently, there is only one for a receiver - https://reportstream.cdc.gov/resources/elr-checklist
What you need to know

Collected O&O feedback on site improvements
Acceptance criteria

[ ] Updated wireframe on product design/layout (if needed)
[ ] Work with O&O to author this content
[ ] Review proposed updates with O&O and internal
[ ] Once approved, supply implementation details and/or prep an implementation ticket if being handled by the design team

No longer needed. Being encompassed by new website redesign currently in progress.


Accept https callbacks with valid (not self-signed) certificate

It's not unusual for a project to interact with other systems, and good practice requires data to be sent over https. Ddev uses its own certificate in when responding to https callbacks from external systems, and this certificate is often not acceptable to services sending data, which makes it difficult to use ddev to develop when external systems are involved.
Notes

ddev uses the same certificate for all sites. In fact the router software jwilder/nginx-proxy has the ability to use the correct certificate. I think the nginx template file would need alteration to more closely match the jwilder version. https://github.com/jwilder/nginx-proxy/blob/master/README.md#ssl-support
jwilder/nginx-proxy has an associated container which works with LetsEncrypt. This requires port 80, so it won't work with projects that use other ports. https://github.com/jwilder/nginx-proxy/blob/master/README.md#ssl-support-using-letsencrypt

Workarounds

As a hacked workaround, you can set up a non-ddev Nginx reverse proxy outside of ddev with the correct certificate. This terminates the https request, and passes the request into ddev. It's clutzy as a solution because the whole project is no longer in ddev. If you want to use LetsEncrypt, you'll need to set this up on port 80. You will need to switch your ddev containers off port 80. This workaround requires port 80 to be reachable from the internet, and of course having a domain correctly set up.
An alternative workaround is to tunnel from a computer on the internet, through to your dev machine. See rfay's article here - https://randyfay.com/content/tunneling-http-debug-or-develop-external-webservice-call . This will not work if the external service does not accept the ddev self-signed certificate.

Hmm, #1540 is going to give us trusted certs (trusted by the host, trusted inside the container) but is not a "valid" cert. You can actually get your certificate revoked for using it on a non-internet IP, as I understand, right?
I think ngrok might satisfy the need here (as in https://github.com/drud/ddev/issues/375), maybe we should push forward on that one? Would love to hear from you @JeffVeit

IMO the combination of trusted mkcert certs and ddev share solve this problem. Happy to continue the conversation here @JeffVeit if you have any concerns or don't think that has done the job.


npm ERR! Darwin 14.5.0

I was trying to install angular-cli using the following command
npm install -g angular-cli@latest and the get the error described below,
npm ERR! Darwin 14.5.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "-g" "angular-cli@latest"
npm ERR! node v4.6.0
npm ERR! npm  v2.15.9
npm ERR! code EPEERINVALID
npm ERR! peerinvalid The package<EMAIL_ADDRESS>does not satisfy its siblings' peerDependencies requirements!
npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>npm ERR! Please include the following file with any support request:
npm ERR!     /Users/Chaklader/npm-debug.log
What should I do to resolve the issue ?

I have this exact same issue! Arghh

I thought this may only the Mac Os issue. Today I try to install Angular 2 in Windows 10 and still having the same issue. Is that because this is still in the Beta version?

I used angular cli on my laptop earlier today. I don’t get it! Both of my computers are OS X .

uninstalling and reinstalling angular-cli should fix the issue
https://github.com/angular/angular-cli/issues/1190 https://github.com/angular/angular-cli/issues/1190

You need npm3 to install the CLI: https://github.com/angular/angular-cli#prerequisites.
You can update npm by doing npm install -g npm

Thanks Filipesilva. Its good suggestion to check our NPM version. Its help me.


Hugo 0.118.0 fails on Netlify

10:17:12 AM: $ hugo --gc --minify
10:17:12 AM: hugo: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by hugo)
hugo: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by hugo)
hugo: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by hugo)
hugo: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by hugo)

We updated the build image to use Go 1.21, which also seemed to bump GLIBC, which Netlify's Ubuntu 20.04 build image does not support ... Bummer.

I confirm that a similar problem occurs in Vercel. (It was fine up until 0.117.0) It can also happen with other web services.

I tried Hugo 0.118.1 with Vercel and still have the same problem:
2023-08-31T10:06:09.251Z  Running build in Cleveland, USA (East) – cle1
2023-08-31T10:06:09.380Z  Retrieving list of deployment files...
2023-08-31T10:06:09.586Z
2023-08-31T10:06:10.117Z  Downloading 441 deployment files...
2023-08-31T10:06:12.097Z  Skipping build cache since Node.js version changed from "16.x" to "18.x"
2023-08-31T10:06:12.174Z  Running "vercel build"
2023-08-31T10:06:12.672Z  Vercel CLI 32.1.0
2023-08-31T10:06:13.271Z  Installing Hugo version 0.118.1
2023-08-31T10:06:14.415Z  Installing dependencies...
2023-08-31T10:06:14.707Z  yarn install v1.22.17
2023-08-31T10:06:14.718Z  warning package.json: No license field
2023-08-31T10:06:14.724Z  info No lockfile found.
2023-08-31T10:06:14.728Z  warning No license field
2023-08-31T10:06:14.729Z  [1/4] Resolving packages...
2023-08-31T10:06:14.729Z  [2/4] Fetching packages...
2023-08-31T10:06:14.730Z  [3/4] Linking dependencies...
2023-08-31T10:06:14.735Z  [4/4] Building fresh packages...
2023-08-31T10:06:14.737Z  success Saved lockfile.
2023-08-31T10:06:14.740Z  Done in 0.04s.
2023-08-31T10:06:14.760Z  hugo: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by hugo)
2023-08-31T10:06:14.760Z  hugo: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by hugo)
2023-08-31T10:06:14.770Z  Error: Command "hugo --gc --minify" exited with 1
2023-08-31T10:06:15.373Z

FWIW, 0.118.1 works fine in GitHub Actions with ubuntu-latest, which as of now is Ubuntu 22.04.

@fu-sen that is odd. The Hugo docs site built fine on Netlify, which I assumed meant that this was ... OK.

Vercel + Hugo 0.118.2 was successfully built.

@fu-sen can I ask; do you know if there's a newer build image available in Vercel compared to the one that failed for you above?

@bep We have a choice about the Node version, but there doesn't seem to be a choice about the build image.

Vercel is still using Amazon Linux 2. Long term support (LTS) for Amazon Linux 2 was to have ended in June 2023. Unfortunately, Amazon decided to extend LTS until June 2025. Had they not extended the date, organizations such as Vercel would have been forced/encouraged to update to Amazon Linux 2022 or Amazon Linux 2023. Hugo v0.118.0 runs fine on both.

I have a roundabout fix for this which I will apply in a minute, but a note to my future self. Running this from an AMD64 PC will get a running Amazon Linux 2 image:
docker run --rm -v "$PWD":/aws -w /aws -i -t amazonlinux:2 /bin/bash


luci-base: Update Japanese translation

Updated Japanese translations.
Signed-off-by: INAGAKI Hiroshi<EMAIL_ADDRESS>

merged, thank you!


BUG: Inconsistent results from dgesv on different platforms

Describe your issue.
We use the RBFInterpolator, and in a simple pytest, which succeeds on other developers' machines, it failed on mine. Upon further investigation, it appears that dgesv returns a different result on my machine than on theirs. We have the same lapack installed. I am going to assume the result on my machine is wrong, because I get outvoted. so I wrote the test, assuming it succeeds. I am not entirely sure how to continue debugging this. The error message is from the test, not from the reproducing code. The reproducing code just gives an assert error: on my machine, info == 10, and this leads to the RBFInterpolator raising the LinAlgError I pasted in the error message.
Reproducing Code Example
import numpy as np
from numpy.testing import assert_array_almost_equal

from scipy.linalg.lapack import dgesv
from scipy.interpolate._rbfinterp_pythran import _build_system

y = np.array([
        [-1.],
        [20.],
        [20.],
        [21.],
        [20.]]
)

d = np.array([[0.1],
       [0.3],
       [0.5],
       [0.7],
       [0.7]])

kernel = 'thin_plate_spline'

epsilon = 1.0

powers = np.array([[0],
       [1],
       [2],
       [3],
       [4]])

smoothing = np.array([2., 2., 2., 2., 2.])

lhs, rhs, shift, scale = _build_system(
        y, d, smoothing, kernel, epsilon, powers
        )

lhs_out = np.array([[ 2.00000000e+00,  1.34263440e+03,  1.34263440e+03,
         1.49606455e+03,  1.34263440e+03,  1.00000000e+00,
        -1.00000000e+00,  1.00000000e+00, -1.00000000e+00,
         1.00000000e+00],
       [ 1.34263440e+03,  2.00000000e+00,  0.00000000e+00,
         0.00000000e+00,  0.00000000e+00,  1.00000000e+00,
         9.09090909e-01,  8.26446281e-01,  7.51314801e-01,
         6.83013455e-01],
       [ 1.34263440e+03,  0.00000000e+00,  2.00000000e+00,
         0.00000000e+00,  0.00000000e+00,  1.00000000e+00,
         9.09090909e-01,  8.26446281e-01,  7.51314801e-01,
         6.83013455e-01],
       [ 1.49606455e+03,  0.00000000e+00,  0.00000000e+00,
         2.00000000e+00,  0.00000000e+00,  1.00000000e+00,
         1.00000000e+00,  1.00000000e+00,  1.00000000e+00,
         1.00000000e+00],
       [ 1.34263440e+03,  0.00000000e+00,  0.00000000e+00,
         0.00000000e+00,  2.00000000e+00,  1.00000000e+00,
         9.09090909e-01,  8.26446281e-01,  7.51314801e-01,
         6.83013455e-01],
       [ 1.00000000e+00,  1.00000000e+00,  1.00000000e+00,
         1.00000000e+00,  1.00000000e+00,  0.00000000e+00,
         0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
         0.00000000e+00],
       [-1.00000000e+00,  9.09090909e-01,  9.09090909e-01,
         1.00000000e+00,  9.09090909e-01,  0.00000000e+00,
         0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
         0.00000000e+00],
       [ 1.00000000e+00,  8.26446281e-01,  8.26446281e-01,
         1.00000000e+00,  8.26446281e-01,  0.00000000e+00,
         0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
         0.00000000e+00],
       [-1.00000000e+00,  7.51314801e-01,  7.51314801e-01,
         1.00000000e+00,  7.51314801e-01,  0.00000000e+00,
         0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
         0.00000000e+00],
       [ 1.00000000e+00,  6.83013455e-01,  6.83013455e-01,
         1.00000000e+00,  6.83013455e-01,  0.00000000e+00,
         0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
         0.00000000e+00]])

rhs_out = np.array([[0.1],
       [0.3],
       [0.5],
       [0.7],
       [0.7],
       [0. ],
       [0. ],
       [0. ],
       [0.

        Parameters
        ----------
        y : (P, N) float ndarray
            Data point coordinates.
        d : (P, S) float ndarray
            Data values at `y`.
        smoothing : (P,) float ndarray
            Smoothing parameter for each data point.
        kernel : str
            Name of the RBF.
        epsilon : float
            Shape parameter.
        powers : (R, N) int ndarray
            The exponents for each monomial in the polynomial.

        Returns
        -------
        coeffs : (P + R, S) float ndarray
            Coefficients for each RBF and monomial.
        shift : (N,) float ndarray
            Domain shift used to create the polynomial matrix.
        scale : (N,) float ndarray
            Domain scaling used to create the polynomial matrix.

        """
        lhs, rhs, shift, scale = _build_system(
            y, d, smoothing, kernel, epsilon, powers
            )
        _, _, coeffs, info = dgesv(lhs, rhs, overwrite_a=True, overwrite_b=True)
        if info < 0:
            raise ValueError(f"The {-info}-th argument had an illegal value.")
        elif info > 0:
            msg = "Singular matrix."
            nmonos = powers.shape[0]
            if nmonos > 0:
                pmat = _polynomial_matrix((y - shift)/scale, powers)
                rank = np.linalg.matrix_rank(pmat)
                if rank < nmonos:
                    msg = (
                        "Singular matrix. The matrix of monomials evaluated at "
                        "the data point coordinates does not have full column "
                        f"rank ({rank}/{nmonos})."
                        )

>           raise LinAlgError(msg)
E           numpy.linalg.LinAlgError: Singular matrix. The matrix of monomials evaluated at the data point coordinates does not have full column rank (3/5).

SciPy/NumPy/Python version and system information
>>> import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info); scipy.show_config()
1.12.0 1.26.4 sys.version_info(major=3, minor=10, micro=14, releaselevel='final', serial=0)
Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /usr/local/include
    lib directory: /usr/local/lib
    name: openblas
    openblas configuration: USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
      NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= HASWELL MAX_THREADS=2
    pc file directory: /usr/local/lib/pkgconfig
    version: 0.3.21.dev
  lapack:
    detection method: pkgconfig
    found: true
    include directory: /usr/local/include
    lib directory: /usr/local/lib
    name: openblas
    openblas configuration: USE_64BITINT= DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
      NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= HASWELL MAX_THREADS=2
    pc file directory: /usr/local/lib/pkgconfig
    version: 0.3.21.dev
  pybind11:
    detection method: config-tool
    include directory: unknown
    name: pybind11
    version: 2.11.1
Compilers:
  c:
    commands: cc
    linker: ld.bfd
    name: gcc
    version: 10.2.1
  c++:
    commands: c++
    linker: ld.bfd
    name: gcc
    version: 10.2.1
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 3.0.8
  fortran:
    commands: gfortran
    linker: ld.bfd
    name: gcc
    version: 10.2.1
  pythran:
    include directory: ../../tmp/pip-build-env-oj5r1eje/overlay/lib/python3.10/site-packages/pythran
    version: 0.15.0
Machine Information:
  build:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
  cross-compiled: false
  host:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
Python Information:
  path: /opt/python/cp310-cp310/bin/python
  version: '3.10'

Note that your lhs matrix is rather poorly conditioned:
In [10]: import numpy as np
    ...: from numpy.testing import assert_array_almost_equal
    ...:
    ...: from scipy.linalg.lapack import dgesv
    ...: from scipy.interpolate._rbfinterp_pythran import _build_system
    ...:
    ...: y = np.array([
    ...:         [-1.],
    ...:         [20.],
    ...:         [20.],
    ...:         [21.],
    ...:         [20.]]
    ...: )
    ...:
    ...: d = np.array([[0.1],
    ...:        [0.3],
    ...:        [0.5],
    ...:        [0.7],
    ...:        [0.7]])
    ...:
    ...: kernel = 'thin_plate_spline'
    ...:
    ...: epsilon = 1.0
    ...:
    ...: powers = np.array([[0],
    ...:        [1],
    ...:        [2],
    ...:        [3],
    ...:        [4]])
    ...:
    ...: smoothing = np.array([2., 2., 2., 2., 2.])
    ...:
    ...:
    ...: lhs, rhs, shift, scale = _build_system(
    ...:         y, d, smoothing, kernel, epsilon, powers
    ...:         )

In [11]: np.linalg.cond(lhs)
Out[11]: np.float64(3.692339037612239e+19)

So it's probably a relative of https://github.com/scipy/scipy/issues/14150.

Hm, ok. That makes sense. If it's unstable, then some floating point operations going one way or another in the umpteenth decimal could allow the coefficients to be computed, and going another way it errors out, and the underlying hardware determines what happens in any given instance. I guess following the discussion in the related issue, we should (1) change the unit tests, and (2) add some tests around the interpolation to create a warning if we are in an area of the problem space where this type of instability arises, and do some experimenting to see how common this occurrence is. Thanks!


[WFLY-14960] Upgrade commons-io from 2.5 to 2.10.0

Thanks for submitting your Pull Request!
Please delete this text, and add a link to the Jira issue solved by this PR.
Remember to use the Jira issue ID in the PR title, and any commits.
Issue: https://issues.redhat.com/browse/WFLY-14960

@bstansberry @jimma @jamezp May I ask from you to evaluate this upgrade? thanks :)

@gaol  This is good for webservice subsystem.

@gaol I submitted https://github.com/wildfly/wildfly-core/pull/4650 to see if the core testsuite failure that sparked our hesitancy about moving on from 2.5 is still there. My impression though is that it is.
If it is we're going to have to decide what to do. The biggest issue before was that failure made us concerned about other failures, but if our actual production use of commons-io is looks ok we can deal with the core test thing in some other way.


feature: Add true streaming APIs to reduce client-side memory usage (for SynapseML)

Background
Currently, Azure SynapseML wraps the LightGBM library to make it much easier for customers to use it in a distributed Spark environment. The initial approach taken by SynapseML uses the LGBM_DatasetCreateFromMat API to create a Dataset for either each Spark partition or each executor (user option). However, this method requires the SynapseML Scala code to load the entire uncompressed data into memory before sending it to LightGBM, and merge partition data together manually. This amounts to using an order of magnitude or more memory (raw double arrays and multiple copies) over what LightGBM Datasets use internally (binned data). This requires larger Spark clusters than are really needed, and often causes OOM issues.
In order to improve the memory performance of SynapseML-wrapped LightGBM, we decided to convert the Dataset creation into more of a “streaming” scenario (as opposed to the above “bulk” input matrices), where we do NOT create large arrays on client side. After initial investigation, there were existing LightGBM APIs that seemed to fit this purpose: LGBM_DatasetCreateFromSampledColumn and LGBM_DatasetPushRows[byCSR].  This seemed to allow creation of a “reference” Dataset with defined feature groups and bins, and to push small micro-batches of data into the set. However, these APIs suffered from several significant issues as currently implemented:

Not thread safe for parallelism, since FinishDataset is called when the literal last index is pushed.
Still requires knowing Dataset size up front (not convenient for Spark Row Iterator or general streaming)
Only pushes feature data, and Metadata must still be handled client side
Does not handle partial Datasets or merging Datasets together

Changes
This PR adds APIs to LightGBM to fix these issues and create a true “streaming” flow where microbatches of full rows (including metadata) can be pushed directly into LightGBM Dataset format. The general idea is that we still create an initial Dataset with DatasetCreateFromSampledColumn, but then create other “real” Datasets from that reference. Note that this also fixes a problem where DatasetCreateFromMat does not use the full data corpus to sample the data, and only the data from each separate partition.
The new general flow within SynapseML can be described as 2 approaches: simple streaming and coalesced streaming. With simple streaming, the count of rows must be known up front and uses the following APIs:
Driver:

DatasetCreateFromSampledColumn (final number of rows) with samples from full dataframe
DatasetSerializeReferenceToBinary to create a reference ByteArray to pass to workers

Workers:

Count partition size
DatasetCreateFromSerializedReference(size) using passed buffer
DatasetSetWaitForManualFinish(true), to take manual control over Dataset Finish
DatasetPushRows[byCSR]WithMetadata over the Spark Row Iterator
DatasetMarkFinished

This simple streaming scenario works in some limited cases, but requires counting up front and is not particularly flexible.  To address a true “unbounded” streaming scenario, we also added APIs for a “coalesced flow”.  The basic approach is to create some intermediate Datasets of some fixed “chunk size” (which are never “Finished”), and then merge them into 1 final Dataset to actually use for training iterations. Datasets keep track if they are only “partially” loaded by keeping a num_pushed_rows. The Driver flow is the same, but the worker flow is different:
Workers

DatasetCreateFromSerializedReference(semi-arbitrary chunk size)
DatasetSetWaitForManualFinish(true), to take manual control over FinishLoad
DatasetPushRows[byCSR]WithMetadata over Spark Row Iterator
If dataset fills, repeat from 1 with new Dataset and add to list
Count sum of num_pushed_rows from all Datasets, creating a final num_rows
Final Dataset = DatasetCreateFromSerializedReference(final num_rows)
DatasetCoalesce(target final Dataset, source list of Dataset chunks) – note this calls FinishLoad internally on the target after coalescing

This “coalesced streaming” flow is very flexible, and allows us to optimize the final training Datasets by Spark partition, executor, or whatever we want.  There is no client-side memory overhead, except for a small fixed amout to hold a “microbatch” of pushed data (which can be pushed down to 1 row if desired). And there is no extra special processing for Metadata, so we can send full Spark dataset “Rows” and LightGBM handles it all.
We added the above APIs in this review, which largely do not affect existing APIs.  I only refactored some existing code where it made more sense to share sections between old and new methods (e.g. reading header from either file or serialized buffer).
I also added a new ByteBuffer class, to make it easier to pass unbounded buffer back to client layer. This avoids the odd flow of 1) call and get size, 2) allocate some native array, 3) call final API to load buffer. The ByteBuffer can do this in one pass. This also required a little refactoring of the Writer since I wanted to share code between file and memory buffer operations.
Note that there shouldn’t be any impact of this PR on training iteration code.  The changes are mostly limited to initial Dataset creation and loading.
Testing
I added C++ tests for all of the above so we could test the functionality intensively. There are now C++ test files for serialization, streaming, coalescing, and the new byte buffer.  Both dense and sparse scenarios are covered, 4bit vs 8+bit, microbatch sizes of 1 and 2+, and all types of Metadata.
Also, the jar created from this PR was used in SynapseML, and passed an extensive list of tests covering old “bulk” mode vs new “streaming” mode, sparse vs dense, binary classification vs multiclass, regression and ranking, weights, initial data in binary and multiclass, validation data, and more.  Basically our entire SynapseML LightGBM test suite now passes in both streaming and the older bulk mode (we kept them both to be able to compare and test performance).

@svotaw thanks for the contribution! I'm not ready to review it yet, but since I noticed that you've been pushing multiple commits to try to work around linter issues...wanted to let you know that you can find all the commands run and tools used by the lint CI job for this project here:
https://github.com/microsoft/LightGBM/blob/f96c4ceb804634d0ed3e28ebbe2a1505063e43e8/.ci/test.sh#L70-L91
You might find you have a faster development cycle by running those things directly in your local development environment.

@jameslamb Thanks for the tip!  Sadly I have all linting passing now, but next time I will know. :)  Looks like everything is green, except 1 R build for some reason, and I have a path issue where tests find the file in Windows but not in Linux/Mac. Will fix that when I get a chance.  The Windows cxproj file was updated by me to Win10 (must have been done locally automatically), but I can revert those if needed once things are all settled.

Related issue created here: https://github.com/microsoft/LightGBM/issues/5298

We are going to do some restructuring of the PR to split out things into separate PRs, so hold off on review for now.

split out things into separate PRs

That would be VERY appreciated, thank you!

New PR is here: https://github.com/microsoft/LightGBM/pull/5299.  It's less than half the size and only adds streaming changes, not serialization or coalescing.  Leaving this PR here for a bit until new one has been looked at by our team, and I've done more testing with the limited build.  We will extract some more changes from it later.


License

Hi - could you add a license to this repo? I'd like to build off what you've created, but I'm not sure we can use it until we know the terms. Thanks. If all else is equal, the MIT License is preferable.

Thanks for your remind
I add MIT license for this repo


Customer-reported issue with NNBD migration tool: the tool adds * to types.

Steps to reproduce:

Run dart migrate --skip-import-check for DevTools at commit 2933ba1e07cb7575c05062b3249c19e2ebcceae8.
Unselect everything except https://github.com/flutter/devtools/blob/master/packages/devtools_app/lib/src/shared/table.dart
Find added *: ColumnData<T*>

Dart SDK version: 2.16.0-134.5.beta

CC @stereotype441

As of 1c7fe7196dd7a542f7f91cd6ea2ac81e1a13146c, the null safety migration tool has been removed from active development and retired. No further work on the tool is planned.
If you still need help, or you believe this issue has been closed in error, please feel free to reopen.


Fix/get wrong version

when customer use --version flag, the version format include the Build
metadata, the metadata will be ignored.
The change is when customer input exact version ,we just need compare whether the version is same as chart, when the version is empty, we just continue use the old method to get a latest version.

#4838

@mattfarina Thanks for your review. I have updated the code change. Could you help to take a look at it again?

@mattfarina should we consider this for the v2.12.0 release or do we think it needs more time to bake before we merge? Just going down the PR queue looking for potential features to merge for 2.12. Thanks!

this didnt fix the issue. at least when running helm dep update
i've opened an issue here:
https://github.com/helm/helm/issues/7563


Datalib related improvements

We shouldn’t remove lib/datalib.js because index.html is referring to it.
We can refer to datalib.js via node_modules/datalib/datalib.js but we also need to modify publish.sh to publish node_modules/datalib/ to gh-pages.

Other than that.  This PR looks good.

@kanitw True but I removed that it actually needs it. Fixed in d7d43e5

Okay.  Make sense.  Thanks.


Downloaded file is larger than expected

[2023-06-17 12:41:47]  DEBUG     root
Locals:
{
'self': <httpcore.AsyncConnectionPool object at 0x7f60354c67d0>,
'requests_still_in_flight': 1,
'connection': <AsyncHTTPConnection ['https://esgf-data.ucar.edu:443', HTTP/1.1, CLOSED, Request Count: 1]>
}
[2023-06-17 12:41:47]  ERROR     root
Traceback (most recent call last):
File "/gpfscmip/gpfsdata/esgf/miniconda/envs/esgpull/lib/python3.11/site-packages/esgpull/processor.py", line 88, in stream
raise DownloadSizeError(ctx.completed, ctx.file.size)
esgpull.exceptions.DownloadSizeError: Downloaded file is larger than expected: 5605850 > 5605846
During handling of the above exception, another exception occurred:

I found 4 files with a size of 5605846 bytes:

file
data_node

CMIP6.DCPP.EC-Earth-Consortium.EC-Earth3-CC.dcppB-forecast.s2020-r2i1p1f1.Amon.pr.gr.v20220503.pr_Amon_EC-Earth3-CC_dcppB-forecast_s2020-r2i1p1f1_gr_202611-202710.nc
esgf-data1.llnl.gov

CMIP6.DCPP.EC-Earth-Consortium.EC-Earth3-CC.dcppB-forecast.s2020-r2i1p1f1.Amon.pr.gr.v20220503.pr_Amon_EC-Earth3-CC_dcppB-forecast_s2020-r2i1p1f1_gr_202611-202710.nc
esgf.bsc.es

CMIP6.DCPP.EC-Earth-Consortium.EC-Earth3.dcppA-hindcast.s1960-r2i1p1f1.Amon.pr.gr.v20200213.pr_Amon_EC-Earth3_dcppA-hindcast_s1960-r2i1p1f1_gr_196211-196310.nc
esgf.bsc.es

CMIP6.DCPP.EC-Earth-Consortium.EC-Earth3.dcppA-hindcast.s1960-r2i1p1f1.Amon.pr.gr.v20200213.pr_Amon_EC-Earth3_dcppA-hindcast_s1960-r2i1p1f1_gr_196211-196310.nc
esgf.nci.org.au

I cannot download the files from esgf.bsc.es, the data_node seems to be down at the moment.
For the other two, downloading went fine.
Since I can't reproduce your issue, can you provide a bit more context ? Was the file you tried to download one of the above ?

File(file_id='CMIP6.DCPP.EC-Earth-Consortium.EC-Earth3.dcppA-hindcast.s1992-r3i1p1f1.Amon.psl.gr.v20200213.psl_Amon_EC-Earth3_dcppA-hindcast_s1992
-r3i1p1f1_gr_199611-199710.nc', dataset_id='CMIP6.DCPP.EC-Earth-Consortium.EC-Earth3.dcppA-hindcast.s1992-r3i1p1f1.Amon.psl.gr.v20200213',
master_id='CMIP6.DCPP.EC-Earth-Consortium.EC-Earth3.dcppA-hindcast.s1992-r3i1p1f1.Amon.psl.gr.psl_Amon_EC-Earth3_dcppA-hindcast_s1992-r3i1p1f1_gr_
199611-199710.nc',
url='https://esgf.bsc.es/thredds/fileServer/esg_dataroot/a1ua-DCPP-r3/CMIP6/DCPP/EC-Earth-Consortium/EC-Earth3/dcppA-hindcast/s1992-r3i1p1f1/Amon/
psl/gr/v20200213/psl_Amon_EC-Earth3_dcppA-hindcast_s1992-r3i1p1f1_gr_199611-199710.nc', version='v20200213',
filename='psl_Amon_EC-Earth3_dcppA-hindcast_s1992-r3i1p1f1_gr_199611-199710.nc',
local_path='CMIP6/DCPP/EC-Earth-Consortium/EC-Earth3/dcppA-hindcast/s1992-r3i1p1f1/Amon/psl/gr/v20200213', data_node='esgf.bsc.es',
checksum='9d5bc9c4d16cf738601a2fcd5120f5d6470bcca98f7e45f25fe87de122051836', checksum_type='SHA256', size=4398846, status=<FileStatus.Starting:
'starting'>)

This should now be fixed with 8e0de5679d8a62526c899e20d9671a2b5a6f5962, I'm currently making the 0.5.5 tag and conda package.


opt: add StorageImageWriteWithoutFormat to trimm pass

This will be required to fix https://github.com/microsoft/DirectXShaderCompiler/issues/6981

pushing an empy commit to unblock CI-check-format


_posts/0000-01-02-Darkangel1470.md

Created a branch, created a file , made commit and and opened a pull request

__

Done


nci-openmpi: from Spack v0.21 an explict setup_run_environment() call is NOT required.

Spack v0.20 required an explicit call to setup_run_environment() from setup_dependent_build_environment(). e.g.
    # The following is reproduced from the builtin openmpi spack package
    def setup_dependent_build_environment(self, env, dependent_spec):
        self.setup_run_environment(env)

Spack v0.21 has a get_env_modifications() function that calls setup_run_environment().
See https://github.com/spack/spack/issues/42700 .

Once we transition our deployment from v0.20 to v0.21, we should commit the change.


Bug report : Restore JAR locates backup folder wrongly

Description
When I set an absolute path to the backup folder in the .properties file on my computer, backup process runs well, but when I try to run the script file (or run the 2 lines of command manually), it says
Could not find backup directory!
/path/to/the/backup/folder
Have you made any backups before?

From the error message, it can be recognized that the  Restore JAR locates backup folder wrongly. It only adds the backup path value set in the .properties file to the end of the .minecraft folder, which creates a wrong path.
To Reproduce
Steps to reproduce the behavior:

Set an absolute path in the .properties file: config.advancedbackups.path=/Users/username/Files/Games/Minecraft/1.20.1/Backups
Launch Minecraft and join a world.
Use /backup start to create a backup.
Save your world, and close Minecraft.
Navigate to your backup folder set in the .properties file.
Run the script or run the jar manually. You should notice the error message described above.

Expected behavior
Normal backup restore process is expected.
Versioning

OS: macOS Sonoma 14.3
Minecraft Version(s): 1.20.1
Modloader(s): Fabric 0.15.6
Mod Version: 3.4 (Latest version when this issue is submit)
Java Version: Zulu17.46+19-CA (build 17.0.9+8-LTS)

Additional questions

Can a backup folder path have characters other than ASCII character (I.E. /Users/username/文件/My Minecraft Backup Folder)? If yes, should it be quoted in " or '? (I.E. set config.advancedbackups.path="/Users/username/文件/My Minecraft Backup Folder" in the .properties file)
Can a backup folder path be set a Windows path format (I.E. D:\Games\Minecraft\1.20.1\Backups) on Windows? If not, should I set /d/Games/Minecraft/1.20.1/Backups in the example above?

Thanks so much!

AdvancedBackups-fabric-1.20-3.4-clihotfix.zip
Got a hotfix here that should sort it - that'll propagate to all versions in v3.5. Do let me know how that goes - note that you'll need to boot the server once to update the restoration script with the new jar, alternatively you can rename this one to have the same name as the old one or just run the jar manually.
As for your questions -

All UTF-16 characters that are supported by the filesystem should work. No quotes should be needed.
I would however suggest sticking to ASCII characters, as non-ASCII paths can have inconsistent behaviours on different filesystems and are therefore hard to test and debug in the event of any issues.

You can use the Windows path format, however double backslashes must be used - so D:\\Games\\Minecraft\\1.20.1\\Backups.
D:/Games/Minecraft/1.20.1/Backups also works.

Thank you for solving my problems that quickly! But there is somehow another problem… When I try to export backup as zip, it completely crashed. Same system settings as above. Terminal screenshot below.

I also want to let you know that macOS auto-generates a .DS_Store file used by system, and (if I choose "restore single file" or "restore entire world") the restore process regards it as a "backup". So continue with that … it runs good but I'm not sure if something would go wrong. Maybe you'd like to add an ignore list.

Thanks for the info on macos, I'm unable to test there so it's nice to know about the file it adds.
As for your issue with exporting the backup, I might know the cause already - but just in case, could you please show the file structure for the backups folder, and the same for your saves folder (or server root if this is a server)?
If its what I think it is, I can get you another hotfix out tomorrow, as its almost 3am right now. I'll also aim to get 3.5 as a whole released this weekend.
(I'm thinking the issue is a edgecase to do with how I've used a few different variables, but I'd need to reproduce the error to properly test a fix.)

Have a good rest. I'm really looking forward to the new 3.5 update. My world save is fine. I want to check what my world used to be like simply. And I'm glad to help this project, too.

Here's two screenshots. The first one shows my backup folder. Every column shows files inside the selected folder on its left one. The second one is world folder. The column info (from left to right) shows Name, Modified Time, Size and Type. You should be able to understand each… (Additionally, I set to dark mode for your eyes!👀)
If you need more information, talk to me anytime!

Confirmed repro, and hopefully a confirmed fix - and here's one for you to use.
AdvancedBackups-fabric-1.20-3.4-clihotfix-2.zip
Also added a couple of utility changes - mainly, better error reporting and a filter for that .DS_Store file you mentioned, could you check that too please?
I need to rewrite the whole CLI at some point. Perhaps that'll be the update after 3.5.

Somehow a problem again…
Please select your world. Default for servers is "world".
1. mods test
2. mods test but infinite world
3. Huanqiu 空岛生存
3
Select a backup to restore.
Exception in thread "main" java.lang.UnsupportedOperationException: remove
	at java.base/java.util.Iterator.remove(Iterator.java:102)
	at java.base/java.util.Collection.removeIf(Collection.java:577)
	at co.uk.mommyheather.advancedbackups.cli.AdvancedBackupsCLI.getBackupDate(AdvancedBackupsCLI.java:269)
	at co.uk.mommyheather.advancedbackups.cli.AdvancedBackupsCLI.main(AdvancedBackupsCLI.java:161)

However, when I chose world 1, it works. I checked the backup folder in world 1 and 3, and there is a .DS_Store file under Huanqiu 空岛生存/differential. I'll try to build one removing some code later.

AdvancedBackups-fabric-1.20-3.4-clihotfix-3.zip
Wonderful. I missed that a method I used returned an internal java class with the exact same name as one that supported the remove method I was using.
Definitely partially my fault. Regardless, this is hopefully the last hotfix. You should no longer see that .DS_Store file, and logging should also be better.

Thanks for consistently helping with this by the way!

Thanks! Sorry for my delay… I was busy editing a video yesterday and forgot about this. I'm now away from home and I'll try the new hot fix as soon as I get home. Hopefully that would be the last hot fix. 💪

Good to hear it's all working for you.
As for exporting into the saves folder, that's definitely an option but I'd still prefer to have the user unzip, so there'd be very little benefit. regardless, mind making it a separate feature request so this issue can stay on topic?

Oh, of course I don't mind! I originally mean that the current one chooses to export the whole zip file to the minecraft folder, which makes me really confused. After the restore process, I checked the whole backup folder, the saves folder, the mods folder and even the user root (~) folder! Finally I found it in the minecraft folder. So I think it's a better way to change the export path or remind that in the prompt. However, opt on your own!

One solution there is to give a full path to the file, rather than just the filename.
and come to think of it, i do already know if the user is on a server or not, so i could export to the saves directory if so... hmm.

Maybe output a path is enough and, if user changes the world folder name set in server.properties, it's still able to find the export file. Is there trouble with identifying the world name in server.properties?
Another option is to create a in-game GUI that is able to open the folder directly, but then we know it's client.

reopening because the CF workflow is erroring. you can still get jars from here


Volume.MountPoint should be populated after a mount operation

Upon success, libstorage/client.Integration().Mount() returns a mountPoint string, a pointer to struct libstorage/api/types.Volume, and an error.  It would be nice if field Volume.MountPoint()  was pre-populated with the mount point value as well.

@vladimirvivien Are you referencing that the MountPoint() method is not returning the expected field? I am not clear and what you are describing here.

@clintonskitson Yes, correct. Volume.MountPoint() is nil.

@clintonskitson and I just talked about this, we're going to update the MountPoint on the client side during a Mount op to avoid having to do a second round-trip to reinspect the volume.


Use clang static code analyzers

Read about clang-analyzer and clang-tidy and try to call them in Travis CI #9.

Installation on Ubuntu 18.04 https://askubuntu.com/questions/1033808/how-can-clang-static-analyzer-scan-build-be-installed-on-ubuntu-18-04
Using with CMake https://stackoverflow.com/questions/19050461/cmake-add-target-for-invoking-clang-analyzer
Example of Travis CI configuration https://github.com/robotsthatdream/cafer/blob/fa614347e44e045fcf1a87252dbe73fe3321e391/.travis.yml
Available checkers https://clang-analyzer.llvm.org/available_checks.html
clang-analyzer vs clang-tidy http://lists.llvm.org/pipermail/cfe-dev/2015-September/044966.html
?clang-analyzer vs clang warnings?

clang-tidy
Setup issues? https://github.com/travis-ci/apt-package-safelist/issues/1905
Add it to CMake http://mariobadr.com/using-clang-tidy-with-cmake-36.html
Usage instruction https://www.kdab.com/clang-tidy-part-1-modernize-source-code-using-c11c14/


package manager not found using create-lunaria

┌  create-lunaria
│
◇  Which @lunariajs package would you like to set up?
│  @lunariajs/core
│
◇  Where should we set up Lunaria?
│  ./lunaria
│
◇  Does your project use TypeScript?
│  Yes
│
◇  Do you wish to install @lunariajs/core and its dependencies?
│  Yes
└  Could not find your package manager. Setup wizard cancelled.

npm ERR! code 1
npm ERR! path /Users/alexanderniebuhr/Developer/tmp
npm ERR! command failed
npm ERR! command sh -c create-lunaria

npm ERR!  Thanks for reporting!


Configure AM issued remote consent JWT expiration time

As part of Remote Consent, AM produces a consent_request JWT which it sends to the Remote Consent server.
The initializer is currently configured to give the request a time limit of 0 seconds, which means the "exp" claim equal to "iat" claim. This means that the JWT has effectively expired as soon as it is issued.
https://github.com/SecureApiGateway/securebanking-openbanking-uk-fidc-initializer/blob/master/pkg/securebanking/oauth2.go#L81-L84
The request time limit needs to be configured.
The value needs to be a sensible amount of time for the PSU to complete the remote consent flow.

I think we will configure this as 30 minutes to begin with.
The value will be configurable via initializer config.


Question about how the hook affect the gradient pass to Generator

Hi Dingfan, Thank you for your work!
I'm a postgraduate student in Beihang university.Recently I've read your paper and tried to find out how the Mechanism affect the gradient in the backward process.
In your code ( source/main.py line294) , you defined the dynamic_hook_function = dp_conv_hook, which means you changed the dummy_hook to dp_conv_hook to let the DP Mechanism( clip-gradient and add noise) work.
However, I noticed that in line301-302 p.requires_grad = False, you actually set the netD parameters fixed and it seems the dp_conv_hook will not modify the gradients in the backward process, so I wander how could the hook take effect？Or what should I do to let the dp_conv_hook work?
Thank you !

Hi, thanks for your question. Briefly speaking, setting requires_grad=False will also affect the gradient passing through non-leaf tensors (See the pytorch document for more details).
Also, we have uploaded a colab notebook in the repository which 'verifies' that the gradient of the generator is indeed changed (when setting requires_grad=False for the discriminator parameters) as we expected. You can play with it and check the results for now. We may include the Unittest later.

Thanks a lot! Your notebook really helps!
I really appreciate your patience and detailed step-by-step demonstration, which means a lot to me!
I sincerely wish you all the best in your research in Germany!


arch/arm/s32k1xx: Disable interrupt during FTFC operation

Summary
AN12003: only one FlexNVM operation can be executed at a time.
See: https://www.nxp.com/docs/en/application-note/AN12003.pdf
Disable ISR during FTFC operations so an ISR doesn't cause a hardfault from a simultaneous read.
Impact
arch/arm/src/s32k1xx/s32k1xx_progmem.c
Testing
Tested on a custom target based on the S32K148.
Set up printf to output to LPUART0.
Called printf and write to flash in a loop, no hardfault appears.

Hi @sammytranGeo nice to see more a contributor from Geotab here! ;-)
Sammy, please verify the function wait_ftfc_ready(), if it kept blocked for long time the interrupts will become disabled for a long time inside the critical section, then bad things could happen.

@acassis Glad to see you here!
wait_ftfc_ready() is a while loop that exits when the CCIF bit of the FSTAT register in the FTFC (Flash Memory Module) is 1. If a command is complete the CCIF is set to 1. If a command is unable to complete (ex invalid parameters, protected memory, execution error), the command is terminated and the CCIF is set to 1. Errors are reported by other bits in the FSTAT register.
I've looked through erratas, I could not find any documentation related to the CCIF bit getting stuck.
Is there anything else you would like me to verify?

@acassis Glad to see you here!
wait_ftfc_ready() is a while loop that exits when the CCIF bit of the FSTAT register in the FTFC (Flash Memory Module) is 1. If a command is complete the CCIF is set to 1. If a command is unable to complete (ex invalid parameters, protected memory, execution error), the command is terminated and the CCIF is set to 1. Errors are reported by other bits in the FSTAT register.
I've looked through erratas, I could not find any documentation related to the CCIF bit getting stuck.
Is there anything else you would like me to verify?

No, everything fine!


Make it easier to send stx between accounts

Currently, either when testing or in real life usage, it's very cumbersome to send stx between your accounts. Currently you have to do this:

click menu
click switch account
select the account you want
click copy address button
click menu
click switch account
select previous account
click send
continue as normal

Ideally it'd be like this:

click send
have the ability to select any of your other addresses as recipient
optional: generate new address and select that one

This was added a while back with the recipient modal


"numpy" is not accessed pylance

import numpy
import scipy.io.wavfile
from scipy.fftpack import dct
sample_rate, signal = scipy.io.wavfile.read('harvard.wav')  # File assumed to be in the same directory
signal = signal[0:int(3.5 * sample_rate)]  # Keep the first 3.5 seconds
I executed above code in python virtual environment. then numpy is not work. its show the error of
( WavFileWarning: Chunk (non-data) not understood, skipping it.
sample_rate, signal = scipy.io.wavfile.read('harvard.wav')  # File assumed to be in the same directory )
And, In the code section, the numpy modulus are in transparent word. I point that it shows numpy is not accessed in pylance

Thanks for the issue but this is by design. That code is not directly using numpy. That's what the error message means. It would run the same with or without the import numpy statement.


Fix the get_db example function

Creation of the SessionLocal should be outside the try block.

Hey @bcb , thanks for the suggestion.
Why do you think that it should be outside of the try block?

Great, thanks for your contribution @bcb ! :rocket: :cake:


handle leak in x509_crt.c

under windows there is handles leak in function mbedtls_x509_crt_parse_path - hFind going out of scope leaks the handle on line 1146
simple patch:

*** D:\work\src\crypto\mbedtls-2.3.0\library\x509_crt.c Mon Jun 27 21:50:36 2016
--- D:\work\src\mbedtls-2.3.0\library\x509_crt.c        Thu Sep 01 16:03:21 2016
*************** int mbedtls_x509_crt_parse_path( mbedtls
*** 1144,1150 ****
--- 1144,1153 ----
                                       p, (int) len - 1,
                                       NULL, NULL );
          if( w_ret == 0 )
+         {
+             FindClose( hFind );
              return( MBEDTLS_ERR_X509_FILE_IO_ERROR );
+         }

          w_ret = mbedtls_x509_crt_parse_file( chain, filename );
          if( w_ret < 0 )

ARM Internal Ref: IOTSSL-945

Fixed in merged PR's #738, #751, and #752. Issue can now be closed.


Desktop timezone issue

A few reports or the desktop app not choosing the correct time or date in app but appearing correct in browser.

Switzerland we add 1 hour. But on Windows 10, this is considered as UTC;1 not UTC;2. On Toggl.com it;s considered as UTC;2.
The result is that in the desktop app, I create an entry at 5PM (UTC), it;s marked 6PM when it;s actually 7PM but on Toggl.com it;s correctly set as 7PM.

/case/450995

Screenshot included case/451352
"Here I found another bug. Pretty strange one. I cannot sent the date of my entry to the 5th of April. It stays on 4th of April. I had to delete and re-create the entry...
To reproduce that I simply create an entry click the Start/Stop button and then edit it and set the date to 05/04/2017. The entry is placed on 04/04/2017.
My date setting is DD.MM.YYYY but in the desktop app it's not taken in consideration."

I have the same issue on Windows 10.
I had the correct time two days ago and since yesterday it's off by one hour.
there was also daylight recently..


Mempool Size chart shows 0 MvB

Current setting

Mempool.space chart as an example

Fixed in b6a3538. The mempool size chart will now switch to kvB units when the maximum Y value of the chart is less than 1 MvB.


Resolve Ruby compiler warnings during tests

This is a follow-up to https://github.com/codeclimate/codeclimate-services/pull/107

LGTM


Frame header format

Needs specification,  esp event/ref uuids

What is a header? Is it mandatory? Must there be exactly 1 header?

A reducer's output is always a frame that MUST start with a header.
A reducer MAY produce several headers thus delimiting chunks of output.
A general frame may contain a mix of raw ops and reducer outputs, thus it can contain any number of headers (may have none if empty or all-raw).

Consider frame containing reducer output and then some raw ops. What does it look like in the text format? Are there any means to signal an end of the frame or start another chunk in the text format?

*lww #obj @time1+origin !
    :one =1 ,
    :two 'два' ,

@time2+origin :three ^3.0 ;

.

(lww reducer output plus one yet-unreduced op)

another chunk starts with another header;
a raw op does not belong to any output, hence any chunk;
a frame is explicitly finalised with a period . (required for separating frames in a byte stream, otherwise optional)

Also, the document states header is an op, but this header isn't, it has no location.

Also comma and semicolon seem not documented. What do they mean?

Oops. Comma terminates reduced ops, semicolon terminates raw ops.
Comma is optional.

another chunk starts with another header;

Let's reword this: a chunk consists of a header ! or a query header ? op followed  by reduced , ops belonging to the chunk. Raw ops form their own one-op chunk.

Also, the document states header is an op, but this header isn't, it has no location.

Omitted UUID is set to the default value, which is the same UUID of the previous op. For the first op, the default is 0.
e.g. see https://github.com/gritzko/ron-test/blob/master/05-rga.in.ron#L38

I'll remove that zero default UUID. Indeed, it adds confusion but makes no huge savings.
e.g. *lww #object @version ! or *lww #object @version :0 !


Error: failed to prune cache with v0.111.0 and later on Windows

hugo --gc

Error: failed to prune cache...

Two reports, both Windows users.
Reference: https://discourse.gohugo.io/t/hugo-gc/43261

I can reproduce with the quickstart steps
hugo new site quickstart
cd quickstart
git init
git submodule add https://github.com/theNewDynamic/gohugo-theme-ananke themes/ananke
echo "theme = 'ananke'" >> config.toml
Then copying the exampleSite to content and:
hugo
hugo --gc
Windows 10
Error: failed to prune cache "getjson": remove C:\Users\photo\AppData\Local\Temp\hugo_cache\quickstart\filecache\getjson: The process cannot access the file because it is being used by another process.
Seems to be an issue trying to delete empty directories.

@alexandear I suspect this is the ReadDir update.  Do you have a Windows machine to test this on?

I can reproduce with the quickstart steps

Tried the same on Win 11. Can't reproduce. Tried running commands from CMD, WSL, PowerShell, Windows PowerShell, and Git Bash.

OK, looking at this now, I'm pretty sure this is a Go 1.20 issue. I have seen similar things in unrelated tests on Windows and I have seen this issue mentioned in Go's issue tracker recently. I suspect that this issue is more prominent on certain Windows versions, and the workaround I have seen (and also remember to implement somewhere) is to just wait and try again. Also, in this particular situation we could just print a warning -- the deletion would most likely succeed on the next run.

OK; I can reproduce this on Windows 8 (!). I'll apply a workaround + create an upstream issue.

The process cannot access the file because it is being used by another process.
Usually, Windows can't delete a file when the file is not closed from the Go code.

I have a Windows 10 machine and will try to reproduce it later today.

Multiple runs don't solve it. Tried the next day, reboot and it persists.
Only seems to affect empty folders, it isn't that any files are locked (by anything other than the running hugo --gc command I suppose)
As per forum, only way to fix is to put a file in each of the folders that complains.
Andrew@R5 MINGW64 ~/quickstart (master)
$ hugo --gc
Start building sites …
hugo v0.111.1-39a4a3cf676533859217805c36181c7a3dfa321c+extended windows/amd64 BuildDate=2023-03-02T10:08:26Z VendorInfo=gohugoio
15 runs og hugo --gc, the error repeats, though which folder it complains can change, it never deletes any of them.
Error: failed to prune cache "images"
Error: failed to prune cache "getjson"
Error: failed to prune cache "getresource"
Error: failed to prune cache "getjson"
Error: failed to prune cache "getresource"
Error: failed to prune cache "images"
Error: failed to prune cache "getresource"
Error: failed to prune cache "images"
Error: failed to prune cache "getcsv"
Error: failed to prune cache "images"
Error: failed to prune cache "getjson"
Error: failed to prune cache "getresource"
Error: failed to prune cache "getjson"
Error: failed to prune cache "getjson"
Error: failed to prune cache "images"

@bep I fixed the problem in #10788
Also, I copied the integration test from your PR #10786 to prove it. If you want you can use my solution instead of a workaround.

@alexandear see my comment in the PR:


Openmetrics Parser

Did I miss something? I still can not find exactly the open metrics standard defined properly somewhere. Feels strange to start the implementation before we have that.

This is blocking on me, unfortunately.
Is this a general comment or a hard blocker to get things in as EXPERIMENTAL?

I don't see this as a hard blocker, this will actually help us flesh things out.

It is not a hard blocker, I was just wondering on which "draft'  this PR is based :)


Create FFI for enumerating all header values for memo

Context: https://github.com/subconsciousnetwork/subconscious/pull/389#discussion_r1069964171.
We currently do not expose any way of enumerating all headers for a given memo. This means there is no way to know which headers are present via the FFI.
Possibilities:

A function that returns an array of memo header keys as strings
A function that returns an array of key-value string pairs

#2 may be slightly more convenient, but either one works for me.

Fixed by https://github.com/subconsciousnetwork/noosphere/pull/202


An unexpected error occurred: "http://registry.npmjs.org/compression/-/compression-1.7.2.tgz: read ECONNRESET"

Hello Jan,
I got the error mentioned in the title when running yarn.
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
error An unexpected error occurred: "http://registry.npmjs.org/compression/-/compression-1.7.2.tgz: read ECONNRESET".
info If you think this is a bug, please open a bug report with the information provided in "C:\gitprojects\react-oidc-client-js\src\yarn-error.log".
I ran yarn into other projects without issues. Also, I could download that compression.1.7.2.tgz without issues if I put that url in the browser.
Please, let me know if you know how to fix this issue.
I'm running the latest version of each: node v10.16.0 and yarn v1.16.0
yarn-error.log

Sorry for delay, I will check it. :)

I have updated whole project - all dependencies, please if you can - try to update to the latest master version. ;)
Thanks!

I am closing this, feel free to reopen.
Thank you


(mio v0.7) Fix compiling net::udp::UdpSocket doctest with Rust 1.71

With Rust 1.71 this doctest fails to compile because of a new dropping_copy_types warning, which is elevated to a hard error because of "deny(warnings)".
---- src/net/udp.rs - net::udp::UdpSocket (line 30) stdout ----
error: calls to `std::mem::drop` with a value that implements `Copy` does nothing
  --> src/net/udp.rs:82:1
   |
54 | drop(buffer); // Silence unused assignment warning.
   | ^^^^^------^
   |      |
   |      argument has type `[u8; 9]`
   |
   = note: use `let _ = ...` to ignore the expression or result
note: the lint level is defined here
  --> src/net/udp.rs:29:9
   |
1  | #![deny(warnings)]
   |         ^^^^^^^^
   = note: `#[deny(dropping_copy_types)]` implied by `#[deny(warnings)]`
error: aborting due to previous error
Couldn't compile the test.
failures:
    src/net/udp.rs - net::udp::UdpSocket (line 30)

This PR changes the doctest from calling "drop(buffer)" to using the "assign to underscore to drop" trick.

Well ... it looks like my PR is equivalent to https://github.com/tokio-rs/mio/commit/958200c0ddcc57efbf6130d86d5ad54b8fc16ec1 on the master branch.  Both are fine with me.

@decathorpe you're right, I didn't realise it was already in master. Thanks for the pr.


netclient: init at 0.20.6

Description of changes
Added new package netclient https://github.com/gravitl/netclient
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[x] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 244197 run on x86_64-linux 1

2 packages blacklisted:

nixos-install-tools
tests.nixos-functions.nixos-test

1 package built:

netclient

From ofborg x86_64-darwin
trying https://github.com/gravitl/netclient/archive/v0.20.6.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  311k    0  311k    0     0   2401      0 --:--:--  0:02:13 --:--:--     0
curl: (92) HTTP/2 stream 1 was not closed cleanly: CANCEL (err 8)
error: cannot download source from any mirror
error: builder for '/nix/store/gsqsmbipncg3qm4s62ppiv4fnrlgv3s6-source.drv' failed with exit code 1;
       last 8 log lines:
       >
       > trying https://github.com/gravitl/netclient/archive/v0.20.6.tar.gz
       >   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       > 100  311k    0  311k    0     0   2401      0 --:--:--  0:02:13 --:--:--     0
       > curl: (92) HTTP/2 stream 1 was not closed cleanly: CANCEL (err 8)
       > error: cannot download source from any mirror
       For full logs, run 'nix log /nix/store/gsqsmbipncg3qm4s62ppiv4fnrlgv3s6-source.drv'.
error: 1 dependencies of derivation '/nix/store/fxj7161d9x4yz6iw1s0bkn612j2gb9vv-netclient-0.20.6-go-modules.drv' failed to build

From ofborg aarch64-darwin
unpacking source archive /private/tmp/nix-build-source.drv-0/v0.20.6.tar.gz
gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
do not know how to unpack source archive /private/tmp/nix-build-source.drv-0/v0.20.6.tar.gz
error: builder for '/nix/store/is1qv86jnr5nr21nvxv3sz8mgm9mpd72-source.drv' failed with exit code 1;
       last 10 log lines:
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       > 100 3392k    0 3392k    0     0   9442      0 --:--:--  0:06:07 --:--:--     0
       > unpacking source archive /private/tmp/nix-build-source.drv-0/v0.20.6.tar.gz
       >
       > gzip: stdin: unexpected end of file
       > tar: Unexpected EOF in archive
       > tar: Unexpected EOF in archive
       > tar: Error is not recoverable: exiting now
       > do not know how to unpack source archive /private/tmp/nix-build-source.drv-0/v0.20.6.tar.gz
       For full logs, run 'nix log /nix/store/is1qv86jnr5nr21nvxv3sz8mgm9mpd72-source.drv'.
error: 1 dependencies of derivation '/nix/store/jz7krwaxjmzd79ifiyi966rns7b85in3-netclient-0.20.6-go-modules.drv' failed to build

From ofborg x86_64-darwin
trying https://github.com/gravitl/netclient/archive/v0.20.6.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  311k    0  311k    0     0   2401      0 --:--:--  0:02:13 --:--:--     0
curl: (92) HTTP/2 stream 1 was not closed cleanly: CANCEL (err 8)
error: cannot download source from any mirror
error: builder for '/nix/store/gsqsmbipncg3qm4s62ppiv4fnrlgv3s6-source.drv' failed with exit code 1;
       last 8 log lines:
       >
       > trying https://github.com/gravitl/netclient/archive/v0.20.6.tar.gz
       >   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       > 100  311k    0  311k    0     0   2401      0 --:--:--  0:02:13 --:--:--     0
       > curl: (92) HTTP/2 stream 1 was not closed cleanly: CANCEL (err 8)
       > error: cannot download source from any mirror
       For full logs, run 'nix log /nix/store/gsqsmbipncg3qm4s62ppiv4fnrlgv3s6-source.drv'.
error: 1 dependencies of derivation '/nix/store/fxj7161d9x4yz6iw1s0bkn612j2gb9vv-netclient-0.20.6-go-modules.drv' failed to build

From ofborg aarch64-darwin
unpacking source archive /private/tmp/nix-build-source.drv-0/v0.20.6.tar.gz
gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
do not know how to unpack source archive /private/tmp/nix-build-source.drv-0/v0.20.6.tar.gz
error: builder for '/nix/store/is1qv86jnr5nr21nvxv3sz8mgm9mpd72-source.drv' failed with exit code 1;
       last 10 log lines:
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       > 100 3392k    0 3392k    0     0   9442      0 --:--:--  0:06:07 --:--:--     0
       > unpacking source archive /private/tmp/nix-build-source.drv-0/v0.20.6.tar.gz
       >
       > gzip: stdin: unexpected end of file
       > tar: Unexpected EOF in archive
       > tar: Unexpected EOF in archive
       > tar: Error is not recoverable: exiting now
       > do not know how to unpack source archive /private/tmp/nix-build-source.drv-0/v0.20.6.tar.gz
       For full logs, run 'nix log /nix/store/is1qv86jnr5nr21nvxv3sz8mgm9mpd72-source.drv'.
error: 1 dependencies of derivation '/nix/store/jz7krwaxjmzd79ifiyi966rns7b85in3-netclient-0.20.6-go-modules.drv' failed to build

Can you help me here ? If I understand it correctly, the source tar failed to download but only for darwin arch ? Why that could be ?

@Lord-Valen Can you help me here ? If I understand it correctly, the source tar failed to download but only for darwin arch ? Why that could be ?

I'm not sure what caused that myself. Maybe it was a fluke?
@ofborg build netclient

Ah, it was for passthru.tests. The actual problem:
# golang.design/x/clipboard
vendor/golang.design/x/clipboard/clipboard_darwin.go:16:9: fatal error: 'Cocoa/Cocoa.h' file not found
#import <Cocoa/Cocoa.h>
        ^~~~~~~~~~~~~~~
1 error generated.
error: builder for '/nix/store/8qpa4gphzlvhbi1s6zcavml034zw4r1a-netclient-0.20.6.drv' failed with exit code 1;
       last 10 log lines:
       > Building subPackage ./config
       > Building subPackage ./daemon
       > Building subPackage ./firewall
       > Building subPackage ./functions
       > Building subPackage ./gui
       > # golang.design/x/clipboard
       > vendor/golang.design/x/clipboard/clipboard_darwin.go:16:9: fatal error: 'Cocoa/Cocoa.h' file not found
       > #import <Cocoa/Cocoa.h>
       >         ^~~~~~~~~~~~~~~
       > 1 error generated.
       For full logs, run 'nix log /nix/store/8qpa4gphzlvhbi1s6zcavml034zw4r1a-netclient-0.20.6.drv'.

https://github.com/ofborg build netclient

@ofborg build netclient

@ofborg build netclient

@ofborg build netclient

@ofborg build netclient

Ah, it was for passthru.tests. The actual problem:
# golang.design/x/clipboard
vendor/golang.design/x/clipboard/clipboard_darwin.go:16:9: fatal error: 'Cocoa/Cocoa.h' file not found
#import <Cocoa/Cocoa.h>
        ^~~~~~~~~~~~~~~
1 error generated.
error: builder for '/nix/store/8qpa4gphzlvhbi1s6zcavml034zw4r1a-netclient-0.20.6.drv' failed with exit code 1;
       last 10 log lines:
       > Building subPackage ./config
       > Building subPackage ./daemon
       > Building subPackage ./firewall
       > Building subPackage ./functions
       > Building subPackage ./gui
       > # golang.design/x/clipboard
       > vendor/golang.design/x/clipboard/clipboard_darwin.go:16:9: fatal error: 'Cocoa/Cocoa.h' file not found
       > #import <Cocoa/Cocoa.h>
       >         ^~~~~~~~~~~~~~~
       > 1 error generated.
       For full logs, run 'nix log /nix/store/8qpa4gphzlvhbi1s6zcavml034zw4r1a-netclient-0.20.6.drv'.

@Lord-Valen  Can you please take a look if the buildInputs looks good. It makes sense to me to have it like this, but it might not be the correct way

LGTM

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/1135

@wegank Do you think you can merge this?

@wexder you have merge conflicts

@wegank thank you for resolving merge conflicts

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/netmaker-options/36906/1


Allow plain strings for nixpkgs maintainer fields

Implements first point as mentioned in #391 to prevent issues with plain maintainer names (#389) from happening
cc. @garbas @ncfavier

Is this a symptom of something missing in the Flakes interface?
Or is the flakes indexing assuming too much from the structure of packages?

This has nothing to do with flakes a priori.

Yes it’s not a flakes problem
It was assuming nixpkgs metadata to be structured correctly which isn’t enforced in nixpkgs (or can easily slip a review as in this case.

Yes this has to do with Flakes. If we're indexing flake outputs, it means there is a de-facto interface we can index.
If there is no interface defined, we shouldn't index the data.

Coorect me if I'm wrong, but I think flakes dont have the meta structure in the first place. At least I'm deriving everything i did get from nixpkgs from the flake

I'm really not using Flakes, so I really don't know.
I assume you're probably right, that there is no meta interface defcribed in Flakes. Thus the fact that we are indexing them is probably problematic, as it makes it a de-facto interface not defined in Flakes. I suspect this should be circled back to the Flakes authors. (@edolstra and who else?)

well were not exactly indexing meta as is, but some information (license, maintainers, systems) that nixpkgs chose to wrap on a derivation basis.
with flakes this creates a bit of redundancy and little additional benefit to the user as that information might already be encoded by the flake author and system attribute