Permission denied for asciidoctor-diagram

I'm using the asciidoctor-diagram extension but I believe this is more related to the jekyll-asciidoctor project. Feel free to relocate the issue wherever it might be relevant.
The extension nicely generates the diagram, when configured with the root (i.e. :imagesdir: /). However, whenever I try to use a different folder, it fails:
ERROR: Failed to generate image: Permission denied @ dir_s_mkdir - /svg

I've tried with different folders, to no avail. I also tried use the verbose command-line parameter without getting more information.
Thanks for your help.

I understood the issue, it's about the leading /. The configuration is about the filesystem at generation time, not the web server at runtime.
If I configure :imagesdir: to assets/resources, it works.
If I configure it to :imagesdir: to /assets/resources, it doesn't.
Hence, the overall configuration should be:
:imagesoutdir: my/path
:imagesdir: /my/path

Feel free to close the bug, I think it might still be useful to others.

For awareness: I followed these instructions, and I was finally able to get the diagram generated and placed in the correct location.  However, if I edit the file which generates the file and save it while running jekyll serve, then Jekyll seems to go into an infinite loop of regenerating the .svg file:
Configuration file: /home/wessven/Sandbox/jekyll/myblog/_config.yml
            Source: /home/wessven/Sandbox/jekyll/myblog
       Destination: /home/wessven/Sandbox/jekyll/myblog/_site
 Incremental build: disabled. Enable with --incremental
      Generating...
       Jekyll Feed: Generating feed for posts
                    done in 0.514 seconds.
 Auto-regeneration: enabled for '/home/wessven/Sandbox/jekyll/myblog'
    Server address: http://<IP_ADDRESS>:4000/
  Server running... press ctrl-c to stop.
      Regenerating: 1 file(s) changed at 2019-04-26 22:50:02
                    _posts/2019-04-25-an-asciidoctor-test.adoc
       Jekyll Feed: Generating feed for posts
                    ...done in 0.407418476 seconds.

      Regenerating: 1 file(s) changed at 2019-04-26 22:50:03
                    assets/img/dot-example.svg
       Jekyll Feed: Generating feed for posts
                    ...done in 0.41863548 seconds.

      Regenerating: 1 file(s) changed at 2019-04-26 22:50:03
                    assets/img/dot-example.svg
       Jekyll Feed: Generating feed for posts
                    ...done in 0.418895928 seconds.

      Regenerating: 1 file(s) changed at 2019-04-26 22:50:04
                    assets/img/dot-example.svg
       Jekyll Feed: Generating feed for posts
                    ...done in 0.385898415 seconds.

      Regenerating: 1 file(s) changed at 2019-04-26 22:50:04
                    assets/img/dot-example.svg
       Jekyll Feed: Generating feed for posts
                    ...done in 0.407666232 seconds.

      Regenerating: 1 file(s) changed at 2019-04-26 22:50:05
                    assets/img/dot-example.svg
       Jekyll Feed: Generating feed for posts
                    ...done in 0.413507921 seconds.

      Regenerating: 1 file(s) changed at 2019-04-26 22:50:05
                    assets/img/dot-example.svg
^C       Jekyll Feed: Generating feed for posts

I am guessing that this is a bug in Jekyll, but I am posting it here for posterity.  My configuration is:
keep_files:
  - assets

asciidoctor:
  safe: unsafe
  attributes:
    imagesdir: /assets/img
    imagesoutdir: assets/img

I'm using Ruby 2.6.0, Jekyll 3.8.5, asciidoctor 2.0.8 and asciidoctor-diagram 1.5.16.

This happens, to my knowledge, because the generation of the image triggers the watcher, which then causes the image to be generated again (because it is cleaned).
In the 3.0.0 version of the plugin, I'll be looking into a cleaner integration with Asciidoctor Diagram out of the box (whether in code or documentation) to avoid these sorts of issues.
As for this issue, it is closed, so any further discussion should happen in a fresh issue. We don't develop on close issues.


fm.ini

Is is possible for angelload to write back the fm.ini files with the contents of angeloader's DB?

Oh, you mean like FMSel's Tasks->Create new fm.ini option? Yeah, I guess I could do that.

Are you making an FM and wanting to use AngelLoader to create your fm.ini file? The slight problem here is that AngelLoader doesn't store the "Descr" (long form description) value, so you would have to write it or copy-paste it yourself into a field in the Export window. Fine if you export only once I suppose.
Here's the window

Hi Fen! No the Descr field isn't necessary. But imagine you have all the FM's and all have their fm.ini, and you want to save the changes you make in AngelLoader back to those ini files.

Ah. Well, you wouldn't be able to rebuild your full AngelLoader data from just fm.ini files - you would lose rating, comments, finished status, game type, size, last played, date added, per-FM disabled mods...
Also, we would have to support writing these files back into zip and 7z archives in case the FM isn't installed. I'm not sure I'm comfortable with that conceptually.

For now, no support would be needed to write/back to 7z files. just unzipped missions


ID DownloadScreen:
User feedback: something
(fee_waiver_request_download)

Question ID
DownloadScreen

Details
Can you overwrite the value of accessible_instructions_link and resource_page_link?

Since we don't have instructions for this as such, I think it makes sense to set accessible_instructions link to go to the article since it does have some instructions. Both the URL and display text should be  https://michiganlegalhelp.org/node/2066/
And for resource_page_link, both url and text should be: https://michiganlegalhelp.org/node/10945
Additionally please set cover_sheet_select_county to "" (empty)
Variable being sought | fee_waiver_request_download
Package version | 0.2
Filename | docassemble.FeeWaiverRequest:data/questions/fee_waiver_request.yml

(There were some additional cover sheet edits that I think you didn't see, which prompts some of this. And seeing the fee waiver version in action just prompted me to make a couple more tweaks. But I think with the changes above it should all work.)

Hey Brett, I thought I had this all worked out and just needed to merge, but now I'm seeing some weird unintended effects, so I'll come back to this on Monday and sort it out.

OK, this is good to go now. I believe if you set the variables as requested above and run the latest version of the framework (0.4.19, currently installed on dev), this should work. Thanks!


build_package.csharp_package_unity_linux continuously broken on master

It looks like the first failed build happened yesterday at 6:17PM Pacific.
+ cp Grpc.Core.Api/bin/Release/net45/Grpc.Core.Api.dll unitypackage/unitypackage_skeleton/Plugins/Grpc.Core.Api/lib/net45/Grpc.Core.Api.dll
+ cp Grpc.Core.Api/bin/Release/net45/Grpc.Core.Api.pdb unitypackage/unitypackage_skeleton/Plugins/Grpc.Core.Api/lib/net45/Grpc.Core.Api.pdb
+ cp Grpc.Core.Api/bin/Release/net45/Grpc.Core.Api.xml unitypackage/unitypackage_skeleton/Plugins/Grpc.Core.Api/lib/net45/Grpc.Core.Api.xml
+ cp Grpc.Core/bin/Release/net45/Grpc.Core.dll unitypackage/unitypackage_skeleton/Plugins/Grpc.Core/lib/net45/Grpc.Core.dll
+ cp Grpc.Core/bin/Release/net45/Grpc.Core.pdb unitypackage/unitypackage_skeleton/Plugins/Grpc.Core/lib/net45/Grpc.Core.pdb
+ cp Grpc.Core/bin/Release/net45/Grpc.Core.xml unitypackage/unitypackage_skeleton/Plugins/Grpc.Core/lib/net45/Grpc.Core.xml
+ cp nativelibs/csharp_ext_linux_x64/libgrpc_csharp_ext.so unitypackage/unitypackage_skeleton/Plugins/Grpc.Core/runtimes/linux/x64/libgrpc_csharp_ext.so
+ cp nativelibs/csharp_ext_macos_x64/libgrpc_csharp_ext.dylib unitypackage/unitypackage_skeleton/Plugins/Grpc.Core/runtimes/osx/x64/grpc_csharp_ext.bundle
+ cp nativelibs/csharp_ext_windows_x86/grpc_csharp_ext.dll unitypackage/unitypackage_skeleton/Plugins/Grpc.Core/runtimes/win/x86/grpc_csharp_ext.dll
cp: cannot stat 'nativelibs/csharp_ext_windows_x86/grpc_csharp_ext.dll': No such file or directory
+ FAILED=true
+ '[' artifacts '!=' '' ']'
+ mkdir -p /tmpfs/src/github/grpc/artifacts
+ docker cp build_and_run_docker_4bd4df45-84cc-40f0-85d4-be15bb652a1f:/var/local/git/grpc/artifacts /tmpfs/src/github/grpc
+ docker rm -f build_and_run_docker_4bd4df45-84cc-40f0-85d4-be15bb652a1f
build_and_run_docker_4bd4df45-84cc-40f0-85d4-be15bb652a1f
+ '[' true '!=' '' ']'
+ exit 1

2021-06-15 10:23:31,357 FAILED: build_package.csharp_package_unity_linux [ret=1, pid=8436, time=220.7sec]
2021-06-15 10:23:31,357 WAITING: ETA 147.2 sec; 0 queued, 2 jobs running, 3 complete, 2 failed (load 2.00)

https://source.cloud.google.com/results/invocations/f2ea0b84-cee3-45eb-977a-b91c3bf76849/log

@jtattermusch Looks like you're already on top of things: https://github.com/grpc/grpc/pull/26465. I will monitor and close this issue if that PR resolved the problem.

It looks like this failure is currently breaking the following jobs:

master/linux/grpc_build_packages
master/linux/grpc_distribtests
master/macos/grpc_distribtests
master/windows/grpc_build_artifacts
master/windows/grpc_distribtests

The following commits are the diff between the last good build and the first bad build:

https://github.com/grpc/grpc/commit/22cbc3a388999b92c2a3b083a65a3dcc4167c903
https://github.com/grpc/grpc/commit/f90716a8210d26ac3fb65693b6bd9965106be2ac
https://github.com/grpc/grpc/commit/1e2f19b286fb52e6c58dd0d36a08a0e3170ef777

These look completely unrelated and I see no compilation errors, so this was likely caused by a non-hermetic change in the C# toolchain.

I think the real root cause is the windows/grpc_build_artifacts job suddenly getting broken on master.
https://fusion2.corp.google.com/invocations/80f58a79-e645-460e-b0af-6ef8528c5f65/targets/github%2Fgrpc;config=default/tests
I think the real problem is that nasm installation in the prepare step has failed, which later in the build results in failure to build grpc_csharp_ext. Since grpc_csharp_ext.dll cannot be build, all the subsequent steps (build packages & distribtests fail).
This is not the first time we're seeing the problem, unfortunately the nasm installation is not reliable (from time to time it gets broken for a few days and then it starts working again).
T:\src>choco install nasm -y --limit-output

Installing the following packages:

nasm

By installing you accept licenses for the packages.

Progress: Downloading nasm 2.14.02... 15%
Progress: Downloading nasm 2.14.02... 49%
Progress: Downloading nasm 2.14.02... 82%
Progress: Downloading nasm 2.14.02... 100%

nasm v2.14.02 [Approved]

nasm package files install completed. Performing other installation steps.

Attempt to get headers for https://www.nasm.us/pub/nasm/releasebuilds/2.14.02/win64/nasm-2.14.02-installer-x64.exe failed.

  The remote file either doesn't exist, is unauthorized, or is forbidden for url 'https://www.nasm.us/pub/nasm/releasebuilds/2.14.02/win64/nasm-2.14.02-installer-x64.exe'. Exception calling "GetResponse" with "0" argument(s): "Unable to connect to the remote server"

Downloading nasm 64 bit

  from 'https://www.nasm.us/pub/nasm/releasebuilds/2.14.02/win64/nasm-2.14.02-installer-x64.exe'

ERROR: The remote file either doesn't exist, is unauthorized, or is forbidden for url 'https://www.nasm.us/pub/nasm/releasebuilds/2.14.02/win64/nasm-2.14.02-installer-x64.exe'. Exception calling "GetResponse" with "0" argument(s): "Unable to connect to the remote server"

This package is likely not broken for licensed users - see https://chocolatey.org/docs/features-private-cdn.

The install of nasm was NOT successful.

Error while running 'C:\ProgramData\chocolatey\lib\nasm\tools\chocolateyInstall.ps1'.

Looks like starting from build 17102 the problem has fixed itself (nasm installation through chocolatey started to work again?): https://fusion.corp.google.com/projectanalysis/summary/KOKORO/prod:grpc%2Fcore%2Fmaster%2Fwindows%2Fgrpc_build_artifacts


Update PR workflow to trigger on PRs

Signed-off-by: grantseltzer<EMAIL_ADDRESS>

@grantseltzer Hi, are you still working on this? I’d like to give this a shot if you're not.

Hi @mozillazg ! I do have this on my long to-do list, but please feel free to give it a shot! You're welcome to open a new PR and i'll review it accordingly. Thanks!

@grantseltzer I opened a new PR: https://github.com/aquasecurity/libbpfgo/pull/201


Unable to mock a HTTPForm service

https://github.com/SalesforceCommerceCloud/dw-api-mock/blob/cfc46f8fbd644772e7491e462e47d528cb20a1d9/dw/svc/LocalServiceRegistry.js#L17
as service id is lowercased, 'httpForm' will never match

It's literally a single character change, appreciate if you could create a PR real quick next time. Thanks!

I thought the repository was dead, since no pull request were handled from 2022.
I probably wrote this issue anyway to prevent others and avoid them to loose time debugging.
However I am happy to see this repository alive. Thank you for maintaining it !


Module does not work with react-native 0.28

Module does not work with react-native 0.28  the view is not orderned

PR: https://github.com/nick/react-native-carousel/pull/59

Merged


Resync with apache-spark-on-k8s upstream

We merged things into palantir/spark in a slightly different order than upstream -- revert to get to a clean slate, then cherry pick the PRs in order of upstream
Notable inclusion: the PR that allows small-files to skip RSS

Looks ok once the build passes.


reviewdog warning: -fail-on-error is deprecated. Use -fail-level=any, or -fail-level=error for github-[pr-]check reporter instead

level=WARN msg="reviewdog: -fail-on-error is deprecated. Use -fail-level=any, or -fail-level=error for github-[pr-]check reporter instead. See also https://github.com/reviewdog/reviewdog/blob/master/CHANGELOG.md"

https://github.com/reviewdog/reviewdog/blob/master/CHANGELOG.md

https://github.com/suzuki-shunsuke/github-action-actionlint/pull/784


Default to "synth_xilinx -nodsp" for all families but "xc7"

Vivado upgrades primitives automatically, confirmed working even going through EDIF, so imo there's no real harm in leaving DSP inference turned on for UltraScale (I'll look at doing it properly some time but it's a lot better than nothing)

You can merge it if you want, but I intend to do a better patch this week (with proper support for techmapping to DSP48A1/DSP48E2, but no pmgen yet)


Add Nong Song: Dj Nate, Robva Topper - Skibidynamix

Song Name
Skibidynamix
Artist Name
Dj Nate, Robva Topper
Source
Direct file download
Youtube Link
https://www.youtube.com/watch?v=k13uYQ6P48Y
Direct File Link
https://gamebanana.com/sounds/73394
Song ID
14
Start Offset [ms]
No response
Extra
No response

reject
already in the mod, put it as a local song, as I told you in dms


Now we have pagination it's wrong for the groups

user report : When logged in as "" and clicking "public(21)" I see a few
pagination oddities.
The top right has a really high count: (see attachment)
Showing 1 - 100 of 219 [1][2][3]
I see 8 CVES on page 1, 12 on page 2, and the final one on page 3. The
inconsistency was weird and made me afraid I might be missing results.
An option for longer results, or a bigger default if it was really
meant to be 100 per the paginator widget,  would be nice (to
control-f)

Okay, to fix this properly we need to not filter but alter the query; but that's more work than we need to do right now, let's just move the page limit to 2000 and it gives us a few years to fix it properly


Define metadata fixture pk values in a consistent way

Codecov Report

Merging #328 into develop will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #328   +/-   ##
========================================
  Coverage    92.83%   92.83%
========================================
  Files           71       71
  Lines         2567     2567
  Branches       231      231
========================================
  Hits          2383     2383
  Misses         151      151
  Partials        33       33

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0888787...45b654c. Read the comment docs.


Distributed.jl-based reduce

Thread-based reduce is unsafe/scary to use especially when calling functions from external packages.  Using Distributed.jl would be much easier and safer in many cases.

I was wondering if there is any way to have parallelization of mapfoldl. Apparently it's on the roadmap :).
Have you considered https://github.com/JuliaParallel/Dagger.jl or https://github.com/invenia/Dispatcher.jl?
As Transducers can be seen as a DAG. It might be an easy path toward parallelization.

FYI, reduce implemented in Transducers.jl already supports thread-based parallelism. This issue is about adding process-based parallelism (which is much safer to use).
I've never used Dagger.jl or Dispatcher.jl but both seem to be about "model parallelism" rather than data parallelism which is what reduce is about. I think using Distributed.jl directly might be simpler.


”Andrid”, ”Dindești”, ”Irina” are listed in the drop-down menu

Description:
When user writes ”Andrid”, ”Dindești”, ”Irina” the locations appear in the drop-down menu next to the comune and the county which is a part according to the law.
Precondition:
The website is up an running.

Step 1
Write in the search bar "Andrid"

Expected results
The location appears in the drop-down menu as "Sat Andrid, SATU MARE (Andrid)".

Step 2
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 3
Write in the search bar "Dindești"

Expected results
The location appears in the drop-down menu as "Sat Dindești, SATU MARE (Andrid)".

Step 4
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 5
Write in the search bar "Irina"

Expected results
The location appears in the drop-down menu as "Sat Irina, SATU MARE (Andrid)".


Container storage leak when the podman process is SIGKILL

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug
Description
When the podman process is SIGKILL midway, the container storage leaks and it is not obvious how to clean it up.
Steps to reproduce the issue:
Run this script:
#!/bin/python3

import subprocess, time

argv = ["podman", "run", "--rm", "--name", "test-server",
        "-e", "PYTHONUNBUFFERED=1",
        "fedora", "python3", "-m", "http.server"]

while True:
    try:
        print("Press ctrl-C if http server start...")
        proc = subprocess.Popen(argv)
        try:
            proc.communicate()
        except KeyboardInterrupt:
            proc.terminate()
            proc.kill()
            raise
    except KeyboardInterrupt:
        subprocess.Popen(["podman", "kill", "test-server"]).wait()
    time.sleep(1)

Sample output:
$ python3 repro.py
Press ctrl-C if http server start...
Serving HTTP on <IP_ADDRESS> port 8000 (http://<IP_ADDRESS>:8000/) ...
^C
Keyboard interrupt received, exiting.
Error: no container with name or ID test-server found: no such container
Press ctrl-C if http server start...
Serving HTTP on <IP_ADDRESS> port 8000 (http://<IP_ADDRESS>:8000/) ...
^C
Keyboard interrupt received, exiting.
Error: no container with name or ID test-server found: no such container
Press ctrl-C if http server start...
Serving HTTP on <IP_ADDRESS> port 8000 (http://<IP_ADDRESS>:8000/) ...
^C
Keyboard interrupt received, exiting.
Error: no container with name or ID test-server found: no such container
Press ctrl-C if http server start...
Error: error creating container storage: the container name "test-server" is already in use by "478dbcf9e5f5d262ad6b21ad3fe94907260c086b718ce29d54d05a311caf2abf". You have to remove that container to be able to reuse that name.: that name is already in use
Press ctrl-C if http server start...
Error: error creating container storage: the container name "test-server" is already in use by "478dbcf9e5f5d262ad6b21ad3fe94907260c086b718ce29d54d05a311caf2abf". You have to remove that container to be able to reuse that name.: that name is already in use

Describe the results you received:
Once this happens, to be able to re-use the test-server name, the container storage needs to be removed manually:
$ podman container inspect test-server
Error: error looking up container "test-server": no container with name or ID test-server found: no such container
$ podman container inspect 478dbcf9e5f5d262ad6b21ad3fe94907260c086b718ce29d54d05a311caf2abf
Error: error looking up container "478dbcf9e5f5d262ad6b21ad3fe94907260c086b718ce29d54d05a311caf2abf": no container with name or ID 478dbcf9e5f5d262ad6b21ad3fe94907260c086b718ce29d54d05a311caf2abf found: no such container
$ podman rm test-server
Error: no container with name or ID test-server found: no such container
$ podman rm -f test-server
Error: no container with name or ID test-server found: no such container
$ podman rm -f --storage test-server
test-server

Describe the results you expected:
podman inspect should tell that the container storage still exists instead of "no such container".
Additional information you deem important (e.g. issue happens only occasionally):
The reproducer is probably not correct as it SIGKILL podman while it is cleaning up the container, but perhaps podman could remove the storage first?
Output of podman version:
Version:            1.4.5-dev
RemoteAPI Version:  1
Go Version:         go1.12.2
OS/Arch:            linux/amd64

Output of podman info --debug:
debug:
  compiler: gc
  git commit: ""
  go version: go1.12.2
  podman version: 1.4.5-dev
host:
  BuildahVersion: 1.9.0
  Conmon:
    package: podman-1.5.1-3.fc30.x86_64
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.0, commit: d728afa06cd2df86a27f32a4692c7099a56acc97-dirty'
  Distribution:
    distribution: fedora
    version: "30"
  MemFree:<PHONE_NUMBER>4
  MemTotal:<PHONE_NUMBER>0
  OCIRuntime:
    package: containerd.io-1.2.6-3.3.fc30.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc8
      commit: 425e105d5a03fabd737a126ad93d62a9eeede87f
      spec: 1.0.1-dev
  SwapFree: 0
  SwapTotal: 0
  arch: amd64
  cpus: 8
  hostname: ocp.rdocloud
  kernel: 5.2.9-200.fc30.x86_64
  os: linux
  rootless: true
  uptime: 1h 31m 26.12s (Approximately 0.04 days)
registries:
  blocked: null
  insecure: null
  search:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  ConfigFile: /home/fedora/.config/containers/storage.conf
  ContainerStore:
    number: 9
  GraphDriverName: vfs
  GraphOptions: null
  GraphRoot: /home/fedora/.local/share/containers/storage
  GraphStatus: {}
  ImageStore:
    number: 6
  RunRoot: /tmp/1000
  VolumePath: /home/fedora/.local/share/containers/storage/volumes

Package info (e.g. output of rpm -q podman or apt list podman):
podman-1.5.1-3.fc30.x86_64

Additional environment details (AWS, VirtualBox, physical, etc.):
The host is running in OpenStack

Hi @TristanCacqueray, thanks for opening the issue. There's something odd with the version since podman-version claims 1.4.5-devbut the package is podman-1.5.1-3.fc30.x86_64. I have the very same package installed on my machine but podman returns the correct version.
Can you if you used the correct binary?

Oops, it was a devel version in ~/.local/bin... Using the system podman resulted in the same behavior. Here are the new debug info:
Version:            1.5.1
RemoteAPI Version:  1
Go Version:         go1.12.7
OS/Arch:            linux/amd64

debug:
  compiler: gc
  git commit: ""
  go version: go1.12.7
  podman version: 1.5.1
host:
  BuildahVersion: 1.10.1
  Conmon:
    package: podman-1.5.1-3.fc30.x86_64
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.0, commit: d728afa06cd2df86a27f32a4692c7099a56acc97-dirty'
  Distribution:
    distribution: fedora
    version: "30"
  MemFree:<PHONE_NUMBER>4
  MemTotal:<PHONE_NUMBER>0
  OCIRuntime:
    package: containerd.io-1.2.6-3.3.fc30.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc8
      commit: 425e105d5a03fabd737a126ad93d62a9eeede87f
      spec: 1.0.1-dev
  SwapFree: 0
  SwapTotal: 0
  arch: amd64
  cpus: 8
  eventlogger: file
  hostname: ocp.rdocloud
  kernel: 5.2.9-200.fc30.x86_64
  os: linux
  rootless: true
  uptime: 1h 45m 53.63s (Approximately 0.04 days)
registries:
  blocked: null
  insecure: null
  search:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  ConfigFile: /home/fedora/.config/containers/storage.conf
  ContainerStore:
    number: 10
  GraphDriverName: vfs
  GraphOptions: null
  GraphRoot: /home/fedora/.local/share/containers/storage
  GraphStatus: {}
  ImageStore:
    number: 6
  RunRoot: /tmp/1000
  VolumePath: /home/fedora/.local/share/containers/storage/volumes

If we finally have a consistent reproducer for this, that's a very good thing - removing storage happens before containers are evicted from the database, so figuring out why it's failing but the container is still not in the Libpod DB might let us fix this for good.
On inspect showing the container - I would not expect inspect to be able to locate containers not in our database. There is some talk of a podman ps --storage will display containers in storage (in a somewhat limited form, as we can't get status, etc).

@mheon, I can share some of my cycles. Want me to have a look?

Here is a new sample log using system podman:
$ python3 repro.py
Press ctrl-C if http server start...
Serving HTTP on <IP_ADDRESS> port 8000 (http://<IP_ADDRESS>:8000/) ...
^C
Keyboard interrupt received, exiting.
Error: no container with name or ID test-server found: no such container
Press ctrl-C if http server start...
Error: error creating container storage: the container name "test-server" is already in use by "1c5629586bd866f80c05cc90d38b6e395c0114aac01c000446c52dfe3708d841". You have to remove that container to be able to reuse that name.: that name is already in use
Press ctrl-C if http server start...
Error: error creating container storage: the container name "test-server" is already in use by "1c5629586bd866f80c05cc90d38b6e395c0114aac01c000446c52dfe3708d841". You have to remove that container to be able to reuse that name.: that name is already in use
Press ctrl-C if http server start...
^CError: no container with name or ID test-server found: no such container
^CTraceback (most recent call last):
  File "repro.py", line 22, in <module>
    time.sleep(1)
KeyboardInterrupt
$ podman container inspect 1c5629586bd866f80c05cc90d38b6e395c0114aac01c000446c52dfe3708d841
Error: error looking up container "1c5629586bd866f80c05cc90d38b6e395c0114aac01c000446c52dfe3708d841": no container with name or ID 1c5629586bd866f80c05cc90d38b6e395c0114aac01c000446c52dfe3708d841 found: no such container
$ podman container inspect test-server
Error: error looking up container "test-server": no container with name or ID test-server found: no such container
$ podman rm test-server
Error: no container with name or ID test-server found: no such container
$ podman rm 1c5629586bd866f80c05cc90d38b6e395c0114aac01c000446c52dfe3708d841
Error: no container with name or ID 1c5629586bd866f80c05cc90d38b6e395c0114aac01c000446c52dfe3708d841 found: no such container
$ podman rm 1c5629586bd866f80c05cc90d38b6e395c0114aac01c000446c52dfe3708d841^C
$ podman run -it --rm --name test-server fedora
Error: error creating container storage: the container name "test-server" is already in use by "1c5629586bd866f80c05cc90d38b6e395c0114aac01c000446c52dfe3708d841". You have to remove that container to be able to reuse that name.: that name is already in use
$ podman rm -f 1c5629586bd866f80c05cc90d38b6e395c0114aac01c000446c52dfe3708d841
Error: no container with name or ID 1c5629586bd866f80c05cc90d38b6e395c0114aac01c000446c52dfe3708d841 found: no such container
$ podman rm -f test-server
Error: no container with name or ID test-server found: no such container
$ podman rm -f --storage test-server
test-server
$ podman run -it --rm --name test-server fedora echo it is working
it is working

@mheon perhaps the podman run failure in that situation could hint at using "podman rm --storage" or something. It currently says "You have to remove that container to be able to reuse that name.: that name is already in use".

podman should act with /var/lib/containers/storage/overlay-containers/containers.json based on ACID. SIGTERM or SIGKILL shouldn't corrupt containers.json. Having containers.lock is not enough to implement ACID.

I haven't seen any corrupted containers.json yet. Here, we're seeing a failure to remove for reasons we've yet to determine - but the storage metadata is intact. The container can be removed, but later.
@holser If you have a corrupted containers.json that's a separate issue - and significantly more serious, because that sounds like it would break storage a lot worse than this does.

I've identified something that may be part of the problem - Podman may not be unmounting the SHM mount before the removal. I'm adding some patches that might address it to #3931

I'm going to test this now that #3931 is merged and see if we can call it fixed

Does not appear to be fixed. Looking deeper...

SHM is still mounted, but I'm not seeing any error messages out of Podman.

Alright, I think the reproducer here has introduced a wierd race.
podman run --rm invokes container removal twice: once in the Conmon cleanup process, once in the main podman process. This duplication is to ensure that, if the user detaches from the container, it will still be removed in the background; but if the container remains alive, it is guaranteed to be removed before podman run --rm exits. With our locking model, this is quite safe - one of the processes will lose, but the resulting "no such container" error is safely handled, and the container is definitely gone.
In this case, podman run --rm appears to be consistently winning the race as to who gets to remove the container; which makes sense, because it doesn't need to launch a fresh process. It begins container removal, successfully deletes it from the database, and then is immediately whacked by a SIGKILL from the reproducer before it can completely clean up the mount.
The cleanup process then arrives and attempts to remove the container, but it's already gone from the database; that's one of the first steps we take, to ensure that we don't leave unusable partially-deleted containers around. As such, podman cleanup can't find the container, determines it has been removed by the other Podman process, and as such doesn't complain much.
I don't know how well we can handle this. If podman takes a SIGKILL in the middle of removing a container, things are going to be left undone, so we're left with the decision of whether to leave partially-removed, unusable containers around in podman ps, or potentially leave storage containers around that block names from re-use.
Perhaps the simplest takeaway here is to not whack podman processes with SIGKILL unless they are actually stuck and timing out (well, really, Podman processes that are expected to remove containers - podman run --rm, podman rm).
I'll think about this more, but I don't know if there's an easy solution on our end.

(For reference - adding a time.sleep(1) before the proc.kill() completely fixes the bug for me.
Podman will handle other signals via sig-proxy, but we can't catch SIGKILL, and can't sanely handle the situation.

Please note we cannot in general protect podman from sudden "SIGKILL", AFAIK, a fenced controller would end up with the same situation, like if using SIGKILL but doing a force power off instead.
So there needs to be some transactions replaying or something

@mheon SIGKILL is a valid signal which can be used by user. There is no way to intercept it, that's true. Though, there are many ways how to handle it. A lot of software guarantees constancy data whatever happens (databases, file systems, clustering software). Thus I mentioned ACID. For instance there can be a transaction log (journal) of podman operations. If podman was killed by SIGKILL the transaction log won't be clean. It means that extra steps should be taken to assure that consistency is present and nothing will blow up.

Not sure if my issue is related to SIGKILL issue here, but I'm encountering this case of container that seems to didn't make to the execution of command inside the container. The podman ps -a didn't even detect the container.
Oct 09 19:50:34 prod.moe.ph podman[6966]: time="2019-10-09T19:50:34+08:00" level=debug msg="setting container name nginx"
Oct 09 19:50:34 prod.moe.ph podman[6966]: time="2019-10-09T19:50:34+08:00" level=debug msg="created OCI spec and options for new container"
Oct 09 19:50:34 prod.moe.ph podman[6966]: time="2019-10-09T19:50:34+08:00" level=debug msg="Allocated lock 21 for container 6e3076659ef6c90a9c35d7f2e7d43cccb0a25fd7d2ab5edc9859c8deb64c2da0"
Oct 09 19:50:34 prod.moe.ph podman[6966]: time="2019-10-09T19:50:34+08:00" level=debug msg="parsed reference into \"[overlay@/var/lib/containers/storage+/var/run/containers/storage:overlay.mountopt=nodev,metacopy=on]@0dac5b41d811ca6e1bfe68d31bec5fb1f5c37485b741674619a1a2a3dec5cc0e\""
Oct 09 19:50:34 prod.moe.ph podman[6966]: time="2019-10-09T19:50:34+08:00" level=debug msg="exporting opaque data as blob \"sha256:0dac5b41d811ca6e1bfe68d31bec5fb1f5c37485b741674619a1a2a3dec5cc0e\""
Oct 09 19:50:34 prod.moe.ph podman[6966]: time="2019-10-09T19:50:34+08:00" level=debug msg="created container \"6e3076659ef6c90a9c35d7f2e7d43cccb0a25fd7d2ab5edc9859c8deb64c2da0\""
Oct 09 19:50:34 prod.moe.ph podman[6966]: time="2019-10-09T19:50:34+08:00" level=debug msg="container \"6e3076659ef6c90a9c35d7f2e7d43cccb0a25fd7d2ab5edc9859c8deb64c2da0\" has work directory \"/var/lib/containers/storage/overlay-containers/6e3076659ef6c90a9c35d7f2e7d43cccb0a25fd7d2ab5edc9859c8deb64c2da0/userdata\""
Oct 09 19:50:34 prod.moe.ph podman[6966]: time="2019-10-09T19:50:34+08:00" level=debug msg="container \"6e3076659ef6c90a9c35d7f2e7d43cccb0a25fd7d2ab5edc9859c8deb64c2da0\" has run directory \"/var/run/containers/storage/overlay-containers/6e3076659ef6c90a9c35d7f2e7d43cccb0a25fd7d2ab5edc9859c8deb64c2da0/userdata\""
Oct 09 20:17:36 prod.moe.ph systemd[1]: Stopping nginx container service...
Oct 09 20:17:36 prod.moe.ph podman[9147]: Error: no container with name or ID nginx found: no such container
Oct 09 20:17:36 prod.moe.ph systemd[1]: cnginx.service: Succeeded.
Oct 09 20:17:36 prod.moe.ph systemd[1]: Stopped nginx container service.

I'm assuming that if the podman didn't started the container successfully, it won't be able to cleanup the shm mounts afterwards. On my case, the container was still in the containers.json, so I have to remove it using podman rm --storage.
Here's my podman info details.
[root@prod storage]# podman info --debug
debug:
  compiler: gc
  git commit: ""
  go version: go1.12.9
  podman version: 1.6.1
host:
  BuildahVersion: 1.11.2
  CgroupVersion: v1
  Conmon:
    package: conmon-2.0.1-1.fc30.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.1, commit: 4346fbe0b2634b05857973bdf663598081240374'
  Distribution:
    distribution: fedora
    version: "30"
  MemFree: 332312576
  MemTotal:<PHONE_NUMBER>
  OCIRuntime:
    package: runc-1.0.0-93.dev.gitb9b6cc6.fc30.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc8+dev
      commit: e3b4c1108f7d1bf0d09ab612ea09927d9b59b4e3
      spec: 1.0.1-dev
  SwapFree:<PHONE_NUMBER>
  SwapTotal:<PHONE_NUMBER>
  arch: amd64
  cpus: 2
  eventlogger: journald
  hostname: prod.moe.ph
  kernel: 5.2.14-200.fc30.x86_64
  os: linux
  rootless: false
  uptime: 19h 17m 42.67s (Approximately 0.79 days)
registries:
  blocked: null
  insecure: null
  search:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  ConfigFile: /etc/containers/storage.conf
  ContainerStore:
    number: 9
  GraphDriverName: overlay
  GraphOptions:
    overlay.mountopt: nodev,metacopy=on
  GraphRoot: /var/lib/containers/storage
  GraphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  ImageStore:
    number: 44
  RunRoot: /var/run/containers/storage
  VolumePath: /var/lib/containers/storage/volumes

Oops. I thought the bot closed this.
Adding do-not-close label.

Thanks @mheon !

Took us half a year, but it's finally done :smile:


cmd/compile: &T{} generates different assembly than new(T)

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

$ go version
go version go1.7rc3 darwin/amd64

(Also tested with go1.6.2)

What operating system and processor architecture are you using (go env)?
darwin/amd64

What did you do?
Create a simple struct User, and benchmark the difference between &User{} and new(User):
https://github.com/prashantv/go-bench/blob/master/new_bench_test.go

What did you expect to see?
Both to have the same performance and disassembly.

What did you see instead?
new is consistently faster. Looking at the disassembly, new(User) does less work than &User{}.

new does:
MOVQ BX, 0(SP)
CALL runtime.newobject(SB)
MOVQ 0x8(SP), CX
MOVQ 0x40(SP), AX

and then increments the counter and jumps to the if check.
&User{} is similar but then also stores a lot of 0s before jumping to the if check:
MOVQ BX, 0(SP)
CALL runtime.newobject(SB)
MOVQ 0x8(SP), CX
XORL BP, BP
MOVQ BP, 0(CX)
MOVQ BP, 0x8(CX)
MOVQ BP, 0x10(CX)
MOVQ BP, 0x18(CX)
MOVQ BP, 0x20(CX)
MOVQ 0x40(SP), AX

It seems like &T{} is zeroing out the value even though runtime.newobject already returns a zero'ed out object.

I think this is a duplicate of a very similar issue I raised about 4-6
months ago.
On Tue, 26 Jul 2016, 07:34 Prashant Varanasi<EMAIL_ADDRESS>wrote:

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

$ go version
go version go1.7rc3 darwin/amd64
(Also tested with go1.6.2)

What operating system and processor architecture are you using (go env
)?
darwin/amd64
2.
What did you do?
Create a simple struct User, and benchmark the difference between
&User{} and new(User):
https://github.com/prashantv/go-bench/blob/master/new_bench_test.go
3.
What did you expect to see?
Both to have the same performance and disassembly.
4.
What did you see instead?
new is consistently faster. Looking at the disassembly, new(User) does
less work than &User{}.
new does:
MOVQ BX, 0(SP)
CALL runtime.newobject(SB)
MOVQ 0x8(SP), CX
MOVQ 0x40(SP), AX
and then increments the counter and jumps to the if check.
&User{} is similar but then also stores a lot of 0s before jumping to the
if check:
MOVQ BX, 0(SP)
CALL runtime.newobject(SB)
MOVQ 0x8(SP), CX
XORL BP, BP
MOVQ BP, 0(CX)
MOVQ BP, 0x8(CX)
MOVQ BP, 0x10(CX)
MOVQ BP, 0x18(CX)
MOVQ BP, 0x20(CX)
MOVQ 0x40(SP), AX
It seems like &T{} is zeroing out the value even though runtime.newobject
already returns a zero'ed out object.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/16494, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAcAyKc3f7BGHGxDG0MP-oLjncdGQ54ks5qZSvUgaJpZM4JUkkP
.

dupe of #15199 ?

That's the one, closing as dup.


Features with all 0 after subsetting

After filtering to remove my controls, several features now contain all zeros. I believe the default function should then remove all columns that sum to 0. If there is any reason to keep these features, maybe make a flag?

I assume you are meaning something like this?
ct <- data.frame(
  s1 = c(1,1,1),
  s2 = c(100, 0, 100),
  s3 = c(50, 0, 50),
  row.names = letters[1:3]
)

FeatureTable$
  new(t(ct))$
  keep_samples(function(x) sum(x) > 10)$
  data

     a b   c
s2 100 0 100
s3  50 0  50

If so, then I normally add one more function call like keep_features(function(x) sum(x) > 0) to remove any features with zero counts.  Or there are some "wordy" helpers too: keep_features(that_are_present).
FeatureTable$
  new(t(ct))$
  keep_samples(function(x) sum(x) > 10)$
  keep_features(that_are_present)$
  data

     a   c
s2 100 100
s3  50  50

As to the reasoning behind it, I generally try to avoid magic or implicit behavior.  In my opinion, having the keep_samples function also have the potential to drop features is surprising/unexpected.  Obviously, that is a pretty subjective criteria, and I'm not saying that I'm 100% right about it, so I would be willing to listen to an argument in favor of implicitly dropping features with zero counts after filtering by samples.

I can understand and appreciate that mindset. After I realized what was happening, I implemented a second step to remove all offending features. Maybe a more conservative approach is to throw a warning that one or more features now have a sum() = 0?

Yeah a warning message would be a good idea I think.


Update and rename ae.txt to ue.txt

Academy of Economics changed name 7 years ago into University of Economics in Poznan

Can you remove the second line from ue.txt and actually remove ae.txt from the PR?

@sebastianborbis Flagging as inactive.


Fix 404 redirect URL

Docs Build status updates of commit 591c45b:
:clock10: Full build: the average full build time is 11 min(s) 46 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit 591c45b:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


nvidia-x11 - apply only patches for the right kernel

Motivation for this change
The nvidiaLegacy390 driver failed building on stable kernel (4.19) due to patches for 5.1 kernel
being applied. Should fix #65937.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Notify maintainers
cc @baracoder

Would only fix the issue for 4.19 while it would still fail on 5.x since the patch is still being applied. If the patch is still needed on 5.2 editing the patch so that it applies cleanly would be the solution but if it isn't needed just drop it.

This should be closed since #69851 fixed it the right way.

OK. Closing. Thanks for all the effort.


Create Documentation for User Registration Page

For the added registration page make a guide with using code snippets.

Claiming this!


Delete buildozer.spec

This file is for creating an Android .apk file and not needed here. The README.md refers readers to the Kivy web site if they want to follow along that path.

Thanks!


Chips Input issue (initialValue & patchValue)

Assigning the initialValue in the main FormBuilder Widget using a map doesn't assign any values to the FormBuilderChipsInput Widget. As a workaround, I just assigned the initialValue to the Chips Input widget instead.
Also currentState.patchValue doesn't work if an initialValue is specified (initial value just stays there if specifying a new list for the input field or doesn't clear it's field by assigning an empty list for the input field).
Just an update on this.
currentState.reset() doesn't also revert to the initialvalue provided.
Last update on my issue.
Values can actually be changed but the UI is not being updated.

Hi @harlanx , I have run into a similar issue.  In my case too reset is clearing the value in currentState but leaving the chips in place on the screen.  Have you made any progress on resolving this?  Thanks. -Dan

Hello @SubutaDan . I was able to achieve the field clearing that reflects on UI but have not yet tested adding values or brining back it to initial values.
I have created a
ChipsInputState _chipsState = ChipsInputState();
Then passed it on the
chipBuilder: (context, state, value) {
_chipsState = state;
....}
then on my clear field method I just need to do
formBuilderKey.currentState.fields['selectedContacts'].value.forEach((val){
_chipsState.deleteChip(val);
});

Hello @SubutaDan . I was able to achieve the field clearing that reflects on UI but have not yet tested adding values or brining back it to initial values.
First I imported
import 'package:flutter_chips_input/flutter_chips_input.dart' (this is from the dependency of formbuilder package);
then created
ChipsInputState _chipsState = ChipsInputState();
Then assign a state from the FormBuilerChipsInput
chipBuilder: (context, state, value) {
_chipsState = state;
....}
then on my clear field method I just need to do
formBuilderKey.currentState.fields['selectedContacts'].value.forEach((val){
_chipsState.deleteChip(val);
});

Thanks very much, @harlanx . I have started experimenting with your technique and so far it is working for me.  -Dan

This still an issue?

Maybe related with #1052

Due to lack of response and an old error, I will close this issue.
If the bug still exists, feel free to open a new issue


Testport fails to run on an origin from an overlay

So I have the following scenario:

/usr/ports is where my ports tree is located.
/opt/overlay is where my overlay is located. It has only one port (/opt/overlay/devel/myport).

Here's what I do:
# poudriere ports -c -p default -m null -M /usr/ports
# poudriere ports -c -p overlay -m null -M /opt/overlay

Now, poudriere bulk -j myjail -O overlay devel/myport works just fine, whereas poudriere testport -j myjail -O overlay -o devel/myport does not, failing with:
[00:00:03] Error: Nonexistent origin devel/myport

I took a look into testport.sh and I found the following lines: https://github.com/freebsd/poudriere/blob/master/src/share/poudriere/testport.sh#L208-L210
I am not sure how hard it would be to support running testport on origins from an overlay.

Yeah the whole file needs to learn about overlays apparently.
For now just use bulk -t instead.

related: #352

Fixed by #790


Error from sync_datasets

If I backup large amount of data - arround 3 TB, one dataset with 10 subdatasets - with your script, zfs-autobackup will fail after sending/receiving three out of ten datasets. It works with standalone zfs-autobackup command.
https://www.truenas.com/community/threads/udev-trigger-zfs-autobackup-to-removable-disk-offsite-backup.100570/post-803248
2024-01-14 01:51:59 - UDEV-Trigger - INFO - Starting ZFS-Autobackup for pool backuppool with parameters:
zfs-autobackup-v -d --strip-path=1 --allow-empty --clear-mountpoint --keep-source=10,1d1w,1w1m,1m1y --keep-target=10,1d1w,1w1m,1m1y backuppool backuppool
2024-01-14 01:52:43 - UDEV-Trigger - ERROR - An unexpected error occurred. Backup mail have failed. Please investigate.
Error:
[Errno 5] Input/output error
Traceback (most recent call last):
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsAutobackup.py", line 362, in sync_datasets
source_dataset.sync_snapshots(target_dataset, show_progress=self.args.progress,
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsDataset.py", line 1181, in sync_snapshots
source_snapshot.transfer_snapshot(target_snapshot, features=features,
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsDataset.py", line 785, in transfer_snapshot
target_snapshot.recv_pipe(pipe, features=features, filter_properties=filter_properties,
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsDataset.py", line 699, in recv_pipe
self.zfs_node.run(cmd, inp=pipe, valid_exitcodes=valid_exitcodes)
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ExecuteNode.py", line 176, in run
if not cmd_pipe.execute():
^^^^^^^^^^^^^^^^^^
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/CmdPipe.py", line 118, in execute
self.__process_outputs(selectors)
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/CmdPipe.py", line 158, in __process_outputs
item.stderr_handler(line)
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ExecuteNode.py", line 131, in stderr_handler
self._parse_stderr(line, hide_errors)
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsNode.py", line 171, in _parse_stderr
self.parse_zfs_progress(line, hide_errors, "STDERR > ")
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsNode.py", line 155, in parse_zfs_progress
self.logger.progress(
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/CliBase.py", line 104, in progress
self.log.progress(txt)
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/LogConsole.py", line 66, in progress
print(">>> {}\r".format(txt), end='', file=sys.stderr)
OSError: [Errno 5] Input/output error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsAutobackup.py", line 501, in run
fail_count = self.sync_datasets(
^^^^^^^^^^^^^^^^^^^
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsAutobackup.py", line 376, in sync_datasets
source_dataset.error("FAILED: " + str(e))
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsDataset.py", line 59, in error
self.zfs_node.error("{}: {}".format(self.name, txt))
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsNode.py", line 177, in error
self.logger.error("{} {}".format(self.description, txt))
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/CliBase.py", line 98, in error
self.log.error(txt)
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/LogConsole.py", line 33, in error
print("! " + txt, file=sys.stderr)
OSError: [Errno 5] Input/output error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsAutobackup.py", line 501, in run
fail_count = self.sync_datasets(
^^^^^^^^^^^^^^^^^^^
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsAutobackup.py", line 376, in sync_datasets
source_dataset.error("FAILED: " + str(e))
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsDataset.py", line 59, in error
self.zfs_node.error("{}: {}".format(self.name, txt))
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsNode.py", line 177, in error
self.logger.error("{} {}".format(self.description, txt))
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/CliBase.py", line 98, in error
self.log.error(txt)
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/LogConsole.py", line 33, in error
print("! " + txt, file=sys.stderr)
OSError: [Errno 5] Input/output error
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/mnt/ssdpool/scriptroot/scripts/autobackup/backup.py", line 39, in backup
captured_output = run_zfs_autobackup(pool_config.autobackup_parameters, logger)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ssdpool/scriptroot/scripts/autobackup/backup.py", line 89, in run_zfs_autobackup
cli()
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsAutobackup.py", line 546, in cli
failed_datasets=ZfsAutobackup(sys.argv[1:], False).run()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/ZfsAutobackup.py", line 532, in run
self.error("Exception: " + str(e))
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/CliBase.py", line 98, in error
self.log.error(txt)
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/LogConsole.py", line 33, in error
print("! " + txt, file=sys.stderr)
OSError: [Errno 5] Input/output error

Does the backup always fail during the same third dataset? Or is it a timeout thing?
Does zfs-autobackup log anything more with the --debug-output options? I see you put -d -v in the config but I don't see any output in the logs. You could compare that output from this script and when you run it zfs-autobackup directly.

Does the backup always fail during the same third dataset? Or is it a timeout thing?

It fails after the first subdataset with a larger amount of data, about 600 GB. The error is shown before completion of send/recv of the subdataset, so about a couple of minutes after the backup script have started. I do not think it's something like a timeout. Probably it depends on the number of datasets to backup or the amount of data?

Does zfs-autobackup log anything more with the --debug-output options? I see you put -d -v in the config but I don't see any output in the logs. You could compare that output from this script and when you run it zfs-autobackup directly.

There are no more log entries. When I run zfs-autobackup directly without your trigger script the error does not occur.

Today I made two incremental backup of my main dataset with 10 subdatasets, just about 3 GB. No errors. Probably the error on initial run - about 3 TB - has indeed something to do with a timeout?

@wolmeh I have pushed a change that might fix this. Do you have a possibility to test this again?
ChatGPT pointed out that the error comes from zfs-autobackup writing to std:err
File "/mnt/ssdpool/scriptroot/scripts/autobackup/venv/lib/python3.11/site-packages/zfs_autobackup/LogConsole.py", line 33, in error
print("! " + txt, file=sys.stderr)
OSError: [Errno 5] Input/output error

and recommended to capture the output instead of letting it accumulate. I had indeed only captured std:out and not std:err, which I now changed.
I'll close this ticket as I'm otherwise unable to reproduce it.


Add option for checking class constructor docstrings in __init__

Closes #195
Not sure if the test setup will be exactly as you want - let me know what you think, I'm happy to change. In particular, while it does check that things fail and succeed as expected, it's not currently checking the exact error/output, which is a shame, but I couldn't think of a good way of doing it. Equally the actual code change is probably small enough that the tests don't have to be super exhaustive - really 6 lines of changes 'logic', which is really one if/else.
I made --check-class and --check-class-constructor mutually incompatible, as it wasn't obvious how you could made it work if both were specified without ending up with ugly code (and even what the behaviour 'should' be if a user passes maybe isn't obvious).

gah, I messed up checking for merge conflicts - forgot to sync my fork with the upstream.
It's getting late my time, so I'll look at this tomorrow. Feel free to look at the code if you want, I wouldn't think the merge conflicts will be catastrophic to the whole approach.

This looks great
I really like this, I wouldn't have through to do it
➜ docsig -cC
usage: docsig [-h] [-v] [-c | -C] [-D] [-o] [-p] [-P] [-i] [-a] [-k] [-n] [-S] [-s STR] [-d LIST] [-t LIST] path [path ...]
docsig: error: argument -C/--check-class-constructor: not allowed with argument -c/--check-class

I resolved the conflicts locally and ran everything and this is all holding up, everything passed, and I ran the cli and it looks good
So we can let the pipeline pass before I merge this to master you can either
git rebase master, resolve the conflicts and then git push --force, that would probably be cleanest?
Or if you don't like rebase you can merge master into this branch, resolve the conflicts and push the updated commit to this PR
With the package's conflicts there only seem to be a few changed args and maybe one inheritance change
The tests were a little messier, I'd probably just checkout the tests to HEAD and then add your new tests to the bottom of the files. Look forward to seeing your update
Cheers mate!
Stephen

Rebase done!
One change I made in addition to the rebase - I noticed in the docstring of test_check_class_constructor I had accidentally left in two lines that didn't relate to this test (I wrote the test by copy-pasting another one and modifying):
    All tests that fail such as with ``--check-class`` should be prefixed
    with ``FClass``, and these particular tests should all fail.

So I've deleted these now.

Awesome, I'm ready to merge
The CI will pass if you just run pyaud whitelist -f
It's a bit redundant, as only the line numbers have changed on it, I originally wrote it to exclude the full path and only from the repository root as the diff would be huge from machine to machine, dir to dir. Really it should just be there to update new funcs/classes, I'll look at that some time
but yeah, in the meantime just run that and amend the commit and it should be good to go
Thanks!
Stephen

I hadn't thought too much about the rendering of the errors - it looks the same as for with --check-class, so with a separate 'class' for the description, and the __init__ for the actual signature. I think that's fine, but let me know if you think it needs changing (random example from my project):
class OntologyMatcher:
    """
    :param span_key: ✖
    :param None: ✖
    :param None: ✖
    :param None: ✖
    :param None: ✖
    """

    def __init__(✖nlp, ✖name, ✖span_key, ✖match_id_sep, ✖parser_name_to_entity_type)?:

E101: parameters out of order
E103: parameters missing

imo that's fine
I don't really want to support the rendering of errors anymore, I use the summary option
It seemed to make sense when I first started this, but it doesn't overall make that much of a difference, imo

just fixed the whitelist - sorry about that - didn't deal with it properly in the rebase, but thought I was in the clear with the pre-commit hooks all passing. Oops!


Can't get to loading page for voter frontend

I started the demo using ./docker-prebuilt-up.sh and got to the voting stage. But when I use the voter frontend, I can't seem to login and get to the loading page. I'm using the default credentials found in identity-provider-backend/src/database/identities like alex:alex.
When I try to login, it flashes "Currently no vote ongoing." from the NotOpen page. This made me think that the switch for votingState was going to the default case:
https://github.com/alexscheitlin/master-project-evoting/blob/48c18ac80e22d6673c29153d2bf746cad030aac5/voter-frontend/src/AppManager.tsx#L80-L81
But when I checked state page for the access provider at http://<IP_ADDRESS>:4002/state, I see {"state":"VOTING","address":"0x5F3DBa5e45909D1bf126aA0aF0601B1a369dbFD7"}. This made me realize it should be going into the VOTING case and is probably going into the !state.authenticate branch where it's returning me to the login page:
https://github.com/alexscheitlin/master-project-evoting/blob/48c18ac80e22d6673c29153d2bf746cad030aac5/voter-frontend/src/AppManager.tsx#L68-L70
So then, this made me think the credentials were invalid or not registered. But, I tried running:
curl -X POST \
  http://<IP_ADDRESS>:4003/registerVoters \
  -H 'Content-Type: application/json' \
  -d '{
    "voters": [
        "9980280d-32d1-41e9-8959-7c483e43256b",
        "afd948fe-7b48-421b-accb-389619f8456c",
        "5342b7e8-3f8e-4520-ac4e-e0b54f1d1ead"
    ]
}'

Which returned:
{"success":true,"msg":"The provided voters are already registered!"}

So, this makes me think the default credentials such as alex:alex should work, but it doesn't. What is going on here?

So upon further inspection, I found that the root cause of this issue originates from the handleLogin function that is called when you try to login:
https://github.com/alexscheitlin/master-project-evoting/blob/48c18ac80e22d6673c29153d2bf746cad030aac5/voter-frontend/src/pages/LoginPage.tsx#L12
Here, there is this line:
https://github.com/alexscheitlin/master-project-evoting/blob/48c18ac80e22d6673c29153d2bf746cad030aac5/voter-frontend/src/pages/LoginPage.tsx#L16
In this getToken function, a POST request is made:
https://github.com/alexscheitlin/master-project-evoting/blob/48c18ac80e22d6673c29153d2bf746cad030aac5/voter-frontend/src/services/eIdentityProviderService.ts#L18
However, upon checking the web console in the browser, I can see that no POST request is being sent. I'm not sure why it's not and where the code goes from there, but this seems like it would be the source of the problem.


covtracer open sourced and integrated

[x] covtracer opensourced
[ ] included in image and working in template

covtracer is open sourced: https://github.com/Genentech/covtracer

@dinakar29 - if we try to install covtracer, the API limit for our IP is already exhausted. I put the code to install it below (commented out right now). Do you know if it's possible to inject a token, or would users need to setup there own token as a secret everytime this is used?
https://github.com/insightsengineering/r-pkg-installation-validation/blob/add/generator/report-generator.R#L13

@dinakar29 - if we try to install covtracer, the API limit for our IP is already exhausted. I put the code to install it below (commented out right now). Do you know if it's possible to inject a token, or would users need to setup there own token as a secret everytime this is used?
https://github.com/insightsengineering/r-pkg-installation-validation/blob/add/generator/report-generator.R#L13

It really depends on who/what is installing it.

If it's via a Github Action, the GITHUB_PAT is readily available to the action and is dynamically generated
It it's an unauthenticated user, then anonymous requests are rate limited to the IP address from where the request is initiated
If it's an authenticated user, then requests are rate limited to the user's account


[android][test] XFAIL memory-layout-silgen in Android ARMv7

Change the original UNSUPPORTED to XFAIL, to get signal if the test
starts working.
PR #35707 should allow this test to work properly.
/cc @buttaface

@swift-ci please smoke test

Trying to complete #34491 for Android ARMv7.

Thanks, I thought linux-android would cover them both, guess not.


Straitjacket Functionality.

You can now equip the straitjacket to a humanoid or yourself similarly to equipping a set of handcuffs. The straitjacket is a slower but more effective way of restraining someone.
https://github.com/space-wizards/space-station-14/assets/113240905/a8dfbd5d-85d5-4a2c-8aed-9bad9c8f2382

[ ] Being restrained with strait jacket should reduce movement speed.

[ ] possibility of removing a strait jacket far more quickly using a bladed weapon.

[x] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog

:cl:

add: Strait Jackets can now be used to restrain individuals.

WOOOO LETS GO i needed this functionality

though, looking at it more closely... I believe that it should take even longer for it to apply considering its not a big big change compared to handcuffs and it takes so long to get out. It also shouldnt hurt since unlike handcuffs its soft fabric; not hard metal. If you really want it to still do damage, it at least wouldn't cause you to bleed or deal bloodloss damage as it'd be closer to brute.

Another idea to throw out there, is making the strait jacket unescapable but take an extremely long time to put on; so its only reserved for prisoners that want to kill themselves or permas that are disobedient and actively try to escape.

Another idea to throw out there, is making the strait jacket unescapable but take an extremely long time to put on; so its only reserved for prisoners that want to kill themselves or permas that are disobedient and actively try to escape.

i think making it ocmpletely unescapable would likely result in a lot of griefing such as  clown putting it on an afk player.

Another idea to throw out there, is making the strait jacket unescapable but take an extremely long time to put on; so its only reserved for prisoners that want to kill themselves or permas that are disobedient and actively try to escape.

i think making it ocmpletely unescapable would likely result in a lot of griefing such as clown putting it on an afk player.

yeah i suppose thats true, but it still needs a bit of a change


Use latest tfw bits + vanilla bionic on GCE

What is the problem that this PR is trying to fix?
The GCE way of running travis-worker is not taking advantage of the latest tfw goodness as was developed for use on Packet.
What approach did you choose and why?
Use latest tfw and switch to a vanilla Ubuntu Bionic image.
How can you test this?

[x] it works in staging :tada:

What feedback would you like, if any?
WDYT? This would also mean that several things could be removed in the packer-templates, and maybe also travis-cookbooks repo(s).

Generally huge 👍!
I I'll link travis-ci/worker-gcp-standalone since it is sort of the barebones version (though it's missing stuff like ssh keys from github, papertrail logging).


Add testing for sync_fd

Modify the external semaphore extension test to use SYNC_FD, if available on the device.
Also, made tests compatible with blocking semaphore spec updates (https://github.com/KhronosGroup/OpenCL-Docs/pull/882)
This PR incorporates the functionality from #1608 and supersedes it.

I've been asked by @kpet to have a look on behalf of ARM. I'll try and get this reviewed by weeks end.

With SYNC_FD enabled on Linux platforms, the tests ran through and worked properly for me.
Is it the plan eventually to loop over all supported export handle types? Currently the tests only run with the first export handle_type in the vector.

With SYNC_FD enabled on Linux platforms, the tests ran through and worked properly for me.
Is it the plan eventually to loop over all supported export handle types? Currently the tests only run with the first export handle_type in the vector.

I agree, the current structure of the test is insufficient to test all handle types. I have a local change that generalized the test to iterate over all handle types.  It was originally as a part of https://github.com/KhronosGroup/OpenCL-CTS/pull/1676, however the PR was too large, and I removed all unnecessary refactoring and improvements.  This PR is the minimal change to test SYNC_FD.  Once this PR and 1676 merge, I can merge the changes to iterate over all handle types.  I was finding it difficult to push it as an independent PR due to massive merge conflicts that would occur.

We will push a follow-on PR that cleans up the test structure once #1676 and this one merges. That will add support for iterating over all handle types.


Update dependencies

Hi @MarkPieszak, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by .NET Foundation and real humans are currently evaluating your PR.
TTYL, DNFBOT;


Derive ./data (of default dataDir) from PRJ Spec?

https://discourse.nixos.org/t/prj-spec-direnv-lib/30294
Looks like PRJ_DATA_HOME could be used.

Looks like a good idea to me, we could be one of the early adopters of the specification.


Length of weight input in estimation is not constrained by length of observation set

I am currently estimating space debris using optical observations (RA / Decl). Once I tried implementing a weight vector, I noticed something odd.
Say I want to estimate an orbit on 30 observations. This means I will technically have 60 observations in my observation set (30 RA, 30 Decl. observations). I accidentally used a weight vector in my estimation input that was 30 entries long, instead of 60.
This showed odd behaviour (because 60 are required in this case), LS iterations were behaving almost randomly everytime I clicked the button. But I never received an error message that my weight vector was not the right size.
It could be useful to implement a check if the weight vector is the same size as the observation set, otherwise returning an error message.

Corrected in latest dev version


[JENKINS-29600] added support for the CloudBees Docker Build and Publish plugin

Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests

Thx for the plugin! One question though... as far as i can see there is no option to use a specific Dockerfile ("docker build -f Dockerfile.xyz ...") like the cloudbee-plugin offers (https://wiki.jenkins-ci.org/display/JENKINS/CloudBees+Docker+Build+and+Publish+plugin - BuildContext vs. Dockerfile Path)?


[BUG] - v2 textarea doesn't allow spaces inside a Table row

NextUI Version
2.1.13
Describe the bug
When trying to type in the Textarea box, spaces are not allowed.
Your Example Website or App
https://stackblitz.com/edit/stackblitz-starters-xdwdac?file=app%2Fpage.tsx
Steps to Reproduce the Bug or Issue
<Table>
          <Textarea
            label="Justification"
            placeholder="Explain what change meets this criteria and why it's needed"
            className={`${question.answered_yes ? 'h-32' : 'h-0'} transition-all duration-300 overflow-hidden w-full mt-5`}
            defaultValue={comment}
            onBlur={
              (event) => { handleCommentUpdate(index, event.target.value) }
            }
          />
</Table>

Expected behavior
As a user I expect to be able to type full sentences including spaces in the Textarea box.
Screenshots or Videos
No response
Operating System Version
macOS
Browser
Chrome

Anyone have any idea of what is going on here? It seems that text boxes allow spaces until placed inside a map or a dynamic table (which I'm guessing under the hood uses something similar). @ceastman-r7 included a simple, reproducible example in the linked Stackblitz.

It's similar to this bug here https://github.com/nextui-org/nextui/issues/1608. In my case, spacebar randomly stops working in a <textarea/> placed inside a ` (https://github.com/nextui-org/nextui/issues/1608#issuecomment-1811629800)

Same with

Experienicng the same issue.

@BjornsBot are you saying the issue is happening when creating the text areas inside of a map too? But works fine when outside of a map? Or is it happening when a text area is inside of a NextUI <Table> element, but putting the same text area inside of a pure HTML table works? Assuming the second one?

I am experiencing the same bug. The textarea appears to recognize a spacebar press if:

The spacebar press is the first input after selecting the textarea
The spacebar press is the first input after pausing for ~1 second
The previous input was a spacebar press
Otherwise it does not seem to recognize pressing the spacebar.

This issue is caused by useTypeSelect in @react-aria/selection:
/**
 * Handles typeahead interactions with collections.
 */
export function useTypeSelect(options: AriaTypeSelectOptions): TypeSelectAria {
  let {keyboardDelegate, selectionManager, onTypeSelect} = options;
  let state = useRef({
    search: '',
    timeout: null
  }).current;

  let onKeyDown = (e: KeyboardEvent) => {
    let character = getStringForKey(e.key);
    if (!character || e.ctrlKey || e.metaKey || !e.currentTarget.contains(e.target as HTMLElement)) {
      return;
    }

    // Do not propagate the Spacebar event if it's meant to be part of the search.
    // When we time out, the search term becomes empty, hence the check on length.
    // Trimming is to account for the case of pressing the Spacebar more than once,
    // which should cycle through the selection/deselection of the focused item.
    if (character === ' ' && state.search.trim().length > 0) {
      e.preventDefault();
      if (!('continuePropagation' in e)) {
        e.stopPropagation();
      }
    }

    state.search += character;

    // Use the delegate to find a key to focus.
    // Prioritize items after the currently focused item, falling back to searching the whole list.
    let key = keyboardDelegate.getKeyForSearch(state.search, selectionManager.focusedKey);

    // If no key found, search from the top.
    if (key == null) {
      key = keyboardDelegate.getKeyForSearch(state.search);
    }

    if (key != null) {
      selectionManager.setFocusedKey(key);
      if (onTypeSelect) {
        onTypeSelect(key);
      }
    }

    clearTimeout(state.timeout);
    state.timeout = setTimeout(() => {
      state.search = '';
    }, TYPEAHEAD_DEBOUNCE_WAIT_MS);
  };

  return {
    typeSelectProps: {
      // Using a capturing listener to catch the keydown event before
      // other hooks in order to handle the Spacebar event.
      onKeyDownCapture: keyboardDelegate.getKeyForSearch ? onKeyDown : null
    }
  };
}

and the value of TYPEAHEAD_DEBOUNCE_WAIT_MS is exactly 1000, as @des5771 stated
Here's a workaround. You can create a dummy keyboardDelegate to replace the default one. But it'll prevent user from navigating the table elements by keyboard and may also cause other trouble.

Same with <Input />

Same with  inside

Any updates on this issue?

This error is very serious, is there a solution?

side of

Yes, putting the elements inside a regular table works fine.

@BjornsBot are you saying the issue is happening when creating the text areas inside of a map too? But works fine when outside of a map? Or is it happening when a text area is inside of a NextUI <Table> element, but putting the same text area inside of a pure HTML table works? Assuming the second one?

Yes, putting the elements inside a regular table works fine.

yo! this bugs killing me. any update?


Debug queries

Debug queries

Fails

:no_entry_sign:
The version was not updated. Please update the version.

:no_entry_sign:
The changelog has not been updated. Please update the changelog.

Warnings

:warning:
The README has not been updated. Please update the README.

Messages

:book:
We were able to automatically fix some formatting issues in this PR for you!

:book:
Could not find any browser results.

Some things that were possibly fixed:

Code that could be fixed via the --fix flag
Formatting that could be fixed by prettier

Take a look at this commit to see what happened in detail: 4b41689f319ca9c199f5be562833ae6868ccf2e7
And look at this wiki page to see the reasoning behind the ESLint rules: https://github.com/smartprocure/eslint-config-smartprocure/wiki/Rules-and-Why-We-Chose-Them

  Generated by :no_entry_sign: dangerJS against e14e3e2bb81c09bab3fa3788d5e01e6d58ab8a3a


Empty files with trailing slash are sometimes treated as directories and sometimes treated as regular files

import boto3
from s3fs import S3FileSystem
from pprint  import pprint
TEST_AWS_S3_PORT = 5555
TEST_AWS_ENDPOINT_URL = f'http://<IP_ADDRESS>:{TEST_AWS_S3_PORT}/'

boto_client = boto3.client('s3', endpoint_url=TEST_AWS_ENDPOINT_URL)
fs = S3FileSystem(client_kwargs={'endpoint_url': TEST_AWS_ENDPOINT_URL})

boto_client.create_bucket(Bucket='test-bucket')
boto_client.put_object(
    Bucket='test-bucket', Key='empty-dir/', Body='',
)

pprint(fs.ls('test-bucket', detail=True))
pprint(fs.info('test-bucket/empty-dir/'))
print(fs.isdir('test-bucket/empty-dir/'))
print(fs.ls('test-bucket/empty-dir/'))

The code above first creates an empty file using that ends with a trailing slash. Then it tries to run s3fs's ls on the parent directory, which identifies that file as a directory;
[{'Key': 'test-bucket/empty-dir',
  'Size': 0,
  'StorageClass': 'DIRECTORY',
  'name': 'test-bucket/empty-dir',
  'size': 0,
  'type': 'directory'}]

Also the second and the third calls (info() and isdir()) claims it is a directory;
{'Key': 'test-bucket/empty-dir',
 'Size': 0,
 'StorageClass': 'DIRECTORY',
 'name': 'test-bucket/empty-dir',
 'size': 0,
 'type': 'directory'}
True

though when we try to do ls/walk etc it behaves like a file. The following is the result of .ls('bucket/empty-dir/');
['test-bucket/empty-dir/']

instead I would have expected it to return an empty list

@martindurant today I was bitten by a similar issue in s3fs.core.S3FileSystem.isfile. I had an s3 bucket like the (currently existing) bucket modin-datasets and it had an empty file testing/ in it, i.e. an object at s3://modin-datasets/testing/. There were also objects like modin-datasets/testing/test_data.parquet.
When I list the contents of 'modin-datasets/testing/', I see my object at 'modin-datasets/testing/':
from fsspec.core import url_to_fs

fs, path = url_to_fs("s3://modin-datasets/testing/")
# this prints a list including 'modin-datasets/testing/',  'modin-datasets/testing/test_data.parquet', ...
fs.ls('modin-datasets/testing/')

but my filesystem doesn't recognize modin-datasets/testing/ as a file!
assert not fs.isfile('modin-datasets/testing/')

Is this a bug in s3fs? Is it a separate issue? How does it relate to #562?

There are a few ideas in conflict with this kind of thing, where a file and directory have exactly the same name, including the trailing "/". This situation could not, of course, happen on a posix FS.
The ls method is designed to provide a list of outputs, and so the same name can appear twice, with different details. However, info only fetches one of these, and isfile/dir uses info.

there is an assumption that a pat must be one or the other, not both; wrong in this case
ls is often much slower; in this case, the directory cache should be doing it's thing to make this not a problem.
s3fs's info is quite complex and contains logic to fall back to HEAD when the user doesn't have list permission and may return versions - it finds files first and only checks for a directory if that fails.
the convention that a 0-length file with trailing "/" is used by the AWS console and accepted by the CLI tool, but not part of the official API. Should we accept this convention everywhere?


Update ammonite to 1.9.3

Updates com.lihaoyi:ammonite from 1.9.2 to 1.9.3.
Release Notes/Changelog
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.lihaoyi", artifactId = "ammonite" } ]

labels: test-library-update, semver-patch

Closed by #193