Unable to install modules

Bug description
I get the following error in the console when I try to install a module:
FoundryVTT | 2023-01-11 12:30:14 | [warn] The requested manifest at "https://bitbucket.org/rpgframework-cloud/shadowrun6-eden/downloads/system-staging.json" did not provide system manifest data.
Steps to reproduce
This happens if I select install from the following dialogue:

Expected behavior
I expect that the module will install successfully.
Container metadata
com.foundryvtt.version = "10.291"
org.opencontainers.image.authors =<EMAIL_ADDRESS>org.opencontainers.image.created = "2022-12-02T23:36:29.879Z"
org.opencontainers.image.description = "An easy-to-deploy Dockerized Foundry Virtual Tabletop server."
org.opencontainers.image.licenses = "MIT"
org.opencontainers.image.revision = "bf4e42e9570aab7e8de39d7f760cb5f16e19862e"
org.opencontainers.image.source = "https://github.com/felddy/foundryvtt-docker"
org.opencontainers.image.title = "foundryvtt-docker"
org.opencontainers.image.url = "https://github.com/felddy/foundryvtt-docker"
org.opencontainers.image.vendor = "Geekpad"
org.opencontainers.image.version = "10.291.0"

Relevant log output
Entrypoint | 2023-01-11 18:57:06 | [info] Foundry Virtual Tabletop 10.291 is installed.
Entrypoint | 2023-01-11 18:57:06 | [info] Not modifying existing installation license key.
Entrypoint | 2023-01-11 18:57:06 | [info] Setting data directory permissions.
Entrypoint | 2023-01-11 18:57:06 | [debug] Completed setting directory permissions.
Entrypoint | 2023-01-11 18:57:06 | [info] Starting launcher with uid:gid as foundry:foundry.
Launcher | 2023-01-11 18:57:06 | [debug] Ensuring /data/Config directory exists.
Launcher | 2023-01-11 18:57:06 | [info] Generating options.json file.
Launcher | 2023-01-11 18:57:07 | [info] Setting 'Admin Access Key'.
Launcher | 2023-01-11 18:57:07 | [info] Starting Foundry Virtual Tabletop.
FoundryVTT | 2023-01-11 11:57:07 | [info] Running on Node.js - Version 16.18.1
FoundryVTT | 2023-01-11 11:57:07 | [info] Foundry Virtual Tabletop - Version 10 Build 291
FoundryVTT | 2023-01-11 11:57:07 | [info] User Data Directory - "/data"
FoundryVTT | 2023-01-11 11:57:07 | [info] Application Options:
{
  "awsConfig": null,
  "compressStatic": true,
  "fullscreen": false,
  "hostname": null,
  "language": "en.core",
  "localHostname": null,
  "passwordSalt": null,
  "port": 30000,
  "protocol": null,
  "proxyPort": null,
  "proxySSL": false,
  "routePrefix": null,
  "sslCert": null,
  "sslKey": null,
  "updateChannel": "stable",
  "upnp": false,
  "upnpLeaseDuration": null,
  "world": null,
  "adminPassword": "••••••••••••••••",
  "serviceConfig": null
}
FoundryVTT | 2023-01-11 11:57:07 | [info] Software license verification succeeded
FoundryVTT | 2023-01-11 11:57:07 | [info] Server started and listening on port 30000
FoundryVTT | 2023-01-11 11:57:12 | [warn] Could not reach IP discovery service
FoundryVTT | 2023-01-11 11:57:36 | [info] Created client session 2381f78ce0f7e9cfec306dba
FoundryVTT | 2023-01-11 11:57:39 | [info] Created client session 90bf245526d5a6a41e48bd3e
FoundryVTT | 2023-01-11 11:57:44 | [info] Administrator authentication successful for session 90bf245526d5a6a41e48bd3e
FoundryVTT | 2023-01-11 11:58:14 | [warn] The requested manifest at "https://bitbucket.org/rpgframework-cloud/shadowrun6-eden/downloads/system-staging.json" did not provide system manifest data.
Entrypoint | 2023-01-11 19:18:06 | [debug] Timezone set to: UTC
Entrypoint | 2023-01-11 19:18:06 | [info] Starting felddy/foundryvtt container v10.291.0
Entrypoint | 2023-01-11 19:18:06 | [debug] CONTAINER_VERBOSE set.  Debug logging enabled.
Entrypoint | 2023-01-11 19:18:06 | [info] Foundry Virtual Tabletop 10.291 is installed.
Entrypoint | 2023-01-11 19:18:06 | [info] Not modifying existing installation license key.
Entrypoint | 2023-01-11 19:18:06 | [info] Setting data directory permissions.
Entrypoint | 2023-01-11 19:18:06 | [debug] Completed setting directory permissions.
Entrypoint | 2023-01-11 19:18:06 | [info] Starting launcher with uid:gid as foundry:foundry.
Launcher | 2023-01-11 19:18:06 | [debug] Ensuring /data/Config directory exists.
Launcher | 2023-01-11 19:18:06 | [info] Generating options.json file.
Launcher | 2023-01-11 19:18:06 | [info] Setting 'Admin Access Key'.
Launcher | 2023-01-11 19:18:06 | [info] Starting Foundry Virtual Tabletop.
FoundryVTT | 2023-01-11 12:18:07 | [info] Running on Node.js - Version 16.18.1
FoundryVTT | 2023-01-11 12:18:07 | [info] Foundry Virtual Tabletop - Version 10 Build 291
FoundryVTT | 2023-01-11 12:18:07 | [info] User Data Directory - "/data"
FoundryVTT | 2023-01-11 12:18:07 | [info] Application Options:
{
  "awsConfig": null,
  "compressStatic": true,
  "fullscreen": false,
  "hostname": null,
  "language": "en.core",
  "localHostname": null,
  "passwordSalt": null,
  "port": 30000,
  "protocol": null,
  "proxyPort": null,
  "proxySSL": false,
  "routePrefix": null,
  "sslCert": null,
  "sslKey": null,
  "updateChannel": "stable",
  "upnp": false,
  "upnpLeaseDuration": null,
  "world": null,
  "adminPassword": "••••••••••••••••",
  "serviceConfig": null
}
FoundryVTT | 2023-01-11 12:18:07 | [info] Software license verification succeeded
FoundryVTT | 2023-01-11 12:18:07 | [info] Server started and listening on port 30000
FoundryVTT | 2023-01-11 12:18:12 | [warn] Could not reach IP discovery service
FoundryVTT | 2023-01-11 12:18:36 | [info] Created client session a83a0978d8bf52cad551fe0c
FoundryVTT | 2023-01-11 12:26:34 | [info] Created client session 9fcb16e221a7666cb7168f8c
FoundryVTT | 2023-01-11 12:26:34 | [info] Created client session ace04160ba3a369dd643471f
FoundryVTT | 2023-01-11 12:26:36 | [info] Administrator authentication successful for session ace04160ba3a369dd643471f
FoundryVTT | 2023-01-11 12:27:03 | [warn] The requested manifest at "https://bitbucket.org/rpgframework-cloud/shadowrun6-eden/downloads/system-staging.json" did not provide system manifest data.
FoundryVTT | 2023-01-11 12:29:54 | [warn] The requested manifest at "https://bitbucket.org/rpgframework-cloud/shadowrun6-eden/src/master/system-template.json" did not provide system manifest data.
FoundryVTT | 2023-01-11 12:30:14 | [warn] The requested manifest at "https://bitbucket.org/rpgframework-cloud/shadowrun6-eden/downloads/system-staging.json" did not provide system manifest data.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

I have got a module installing so I do not think this is an issue for the developer responsible for this container.
This thread on reddit made me think it might be something to do with the container.
https://www.reddit.com/r/FoundryVTT/comments/y4oof6/unable_to_installupdate_most_modules_since_v10/
My apologies.

No problem.  I'm glad you figured it out.

I'm not sure if it's possible, but I think we should re-open this.  I am seeing this same issue when trying to install MANY systems, modules, and add-ons.  One specifically that can be tested with is the Blade Runner RPG: https://github.com/fvtt-fria-ligan/blade-runner-foundry-vtt
The reason that I suggest reopening this issue is that when I was initially configuring my docker instance, I did so through the Synology Docker GUI tool.  Once the container was up and running, I added that specific system and it worked just fine.  I ran through my tests and then killed the container and reconfigured using docker-compose.  I got it up and online and all seemed to be working, but that's when I noticed that I've barely been able to add any systems at all.  But that's not to say none will work.
So far I've had no issues installing Cyberpunk Red, FATE, and Starfinder, but Blade Runner, DnD5e, and Call of Cthullu have all failed with similar to the following console output seen:
setup.js:292 Error: Unable to load valid system manifest data from "https://github.com/fvtt-fria-ligan/blade-runner-foundry-vtt/releases/download/10.0.2/system.json"
The requested manifest at "https://github.com/fvtt-fria-ligan/blade-runner-foundry-vtt/releases/download/10.0.2/system.json" did not provide system manifest data.
at Module.installPackage (file:///home/foundry/resources/app/dist/packages/views.mjs:1:1634)
at runMicrotasks ()
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async SetupView.handlePost (file:///home/foundry/resources/app/dist/server/views/setup.mjs:1:1980)

I am reopening this now as I installed the windows version and I'm not getting those arrors at all.

Alright, so I've been able to narrow this issue down further and I find it quite strange.  I'm running 4 separate instances of this docker container on my Synology DSM with differing results.
Scenarios:

CLI using Docker-Compose with Foundry Admin Credentials
CLI using Docker-Compose with Temporary URL
Docker GUI with Foundry Admin Credentials
Docker GUI with Temporary URL

Results:

Manifest errors are observed on both containers that are created via Docker-Compose
No issues observed adding content in containers created through Docker GUI

My initial testing had been done by creating the container in the GUI as a simple test and I'd followed the community guide found here [jump to "Setting up with Docker (no command line)" section]:
https://foundryvtt.wiki/en/setup/hosting/Synology
I had noticed that the GUI configuration automatically creates additional environmental variables not seen in the Docker-Compose writeup, so I modified the yaml files to be identical to the conditions of the GUI containers (port configurations, mount points, environment variables, etc) and I'm still unable to add the majority of content to containers created via Docker-Compose.
Additional Note:
The method of using the temporary URL for creating containers does not seem to be practical.  Every time I recreate the container (say for update purposes) the temporary URL is no longer valid.  It truly seems the best practice is to use the actual credentials for creating containers.

I'm using this docker via the unraid app store

I'll take a look at this and see if I can reproduce it.
Looking through the discord I think I see many reports of the same issue.  I suspect that it is a rate-limiting issue with GitHub.
See:

https://discord.com/channels/170995199584108546/689889940590690323/1043201978865553438
https://discord.com/channels/170995199584108546/697845559435853865/1039185153169301555
https://discord.com/channels/170995199584108546/722566083499786352/1035879944292667443
https://discord.com/channels/170995199584108546/486930822465716249/1027727874473590906

I saw a lot of those comments as well, but I don’t think this is GitHub rate limiting for two reasons:

Some GitHub downloads work consistently while others fail. Working installations can be added from GitHub repeatedly and work every time, while others fail consistently every time.
I appear to only experience these issues when deploying via docker-compose. If I create the container from the Docker GUI on my Synology, the same installs will work. I can literally run both containers simultaneously and it will work on the GUI deployed container and fail on the DC deployed container.

Hey @felddy, just wanted to update once more and let you know I've confirmed the above.  Last night I performed the following test:

Access docker container that was created via docker-compose
Attempt to install several modules that have failed consistently
Each attempt is met with same failure as seen previously
Executed "sudo docker-compose down" to stop and remove container
Created new container in Synology Docker GUI using the exact same configurations from the docker-compose.yaml file
Note: I also pointed the new container to the same existing directory
New container came up with all of the previous configurations and installations in place
Attempted to then install same modules that had just failed and each installed without issue.

At this point it seems abundantly clear that there is some sort of variance under the hood in how these deployment methods are working.

I haven't been able to reproduce these errors.  I've installed several systems, modules, and worlds successfully.
I attempted to get rate-limited by hammering on the Update All buttons without error.
I've included the configuration information I used below.
Could you try adding NODE_DEBUG=http,net to your environment variables as described here, and in the docker-compose.yml file below:

https://github.com/felddy/foundryvtt-docker#nodejs-variables

This will configure node to emit a ton of information about the network requests.
With NODE_DEBUG defined, I expect that you'll see better information about the failure.  Please post back anything interesting that you find.
Installations
ls -1 data-565/Data/*

data-565/Data/modules:
README.txt
dark-mode-5e/
dice-so-nice/
nice-more-dice/
polyglot/
tidy5e-sheet/
trs-foundryvtt-dice-set/

data-565/Data/systems:
README.txt
blade-runner/
dnd5e/
pf2e/
shadowrun6-eden/

data-565/Data/worlds:
README.txt
kobold-cauldron/

docker-compose.yml
---
# version: "3.8"

secrets:
  credentials:
    file: credentials-mine.json

services:
  foundry:
    image: felddy/foundryvtt:10.291.0
    hostname: felddy_foundryvtt
    init: true
    restart: "no"
    volumes:
      - type: bind
        source: ./data-565
        target: /data
    environment:
      - CONTAINER_VERBOSE=true
      - FOUNDRY_GID=20
      - FOUNDRY_UID=501
      - NODE_DEBUG=http,net
    secrets:
      - source: credentials
        target: config.json
    ports:
      - target: 30000
        published: 30000
        protocol: tcp

So the Docker compose version isn't even making the createConnection call?  Or was that cut off in the log?
I've got another idea that we could try.  I've got a version on the container published with Node v18 in an attempt to resolve an IPv6 problem in issue #531 .  Could you try testing with felddy/foundryvtt:node-18 ?
See:

#531

I probably just cut it off. The logs from both were the same until that point specifically.  I’ll try the newer node version and see if it makes a difference.As a side note, in delving into this all I learned that Synology maintains the Docker application as well as the Docker Compose module. Both are version restricted based on the DSM version.  Are you testing with Docker on a Synology or something else? And what versions of Docker and DCompose (DSM as well if you’re using Synology)?-NickOn Jan 23, 2023, at 1:29 PM, Mark Feldhousen @.***> wrote:
So the Docker compose version isn't even making the createConnection call?  Or was that cut off in the log?
I've got another idea that we could try.  I've got a version on the container published with Node v18 in an attempt to resolve an IPv6 problem in issue #531 .  Could you try testing with felddy/foundryvtt:node-18 ?
See:
#531
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

Hi - I am encountering the same issue as well.
Running the container on a Synology via docker-compose (served through Traefik) and seeing The requested manifest at ... did not provide module manifest data.
I tried the felddy/foundryvtt:node-18 container and I'm still seeing the same issue - although subjectively it seems to maybe happen a bit less, hard to tell since there is a large amount of variance from attempt to attempt.
Log output from a run of felddy/foundryvtt:node-18 with CONTAINER_VERBOSE=true and NODE_DEBUG=http,net. I started the container and clicked Update All for the modules.
_FoundryVTT_logs.txt
Versions:
Synology DSM 7.1.1-42962 Update 1
Docker version 20.10.3
docker-compose version 1.28.5
These are all the latest available stable versions. As mentioned by others, the Synology Docker releases usually trail the official ones a bit and include some Synology specific adjustments. They seem to release about 1-2 updates a year maybe.
FWIW I don't have any similar issues with any other docker containers running on the same NAS (about 25-30 in total) although I don't think any of the others are Node based.

Hey @felddy, sorry for the delay in getting back.  I was finally able to give the node-18 release version a shot and it seems to be behaving the same way.  First I purged out all of the remnant files, then I retooled the DC file to not have explicit version configurations and set the image to the node-18 release.  Foundry came up like it always does and was showing the correct version.
I tested downloading the Blade Runner RPG (as it is always a failure case) and sure enough it failed again due to the manifest problem.
@Jnosh Just as a heads up, I've been able to run the same exact container image via the docker GUI in DSM without any issues.  It's a super easy workaround because you can literally copy all of the same environment variables and point it to the existing folder structure and run it without losing anything you've created.  You can literally just shutdown the GUI container and then start it back up via docker compose.

@salsabeard good to know, thanks!

Hello, I had the same error, in my case it was a docker on a raspberry 4. After much looking at all the log-debug docker to look at the raspberry syslog I saw that returned a DNS error. The error was because it had as first DNS server localhost, I have no idea why it was there but it was there, the rest of the servers were correct. When I modified it and set the DNS of my IPS I have no more errors when I click on update all the modules.
I had been having this error for a long time and every now and then I consulted to see if you had found the problem. Maybe this is not your case, but maybe the Synology does something similar.
I hope it helps you, greetings!

@felddy - I'm having the same issue. Not only with your docker, but others as well.
I've turned on all the logging and captured a success and failure against felddy/foundryvtt:release.
Failure to install Pathfinder 2e:
https://pastebin.com/dZ25pd11
Successfully installed DnD 4e:
https://pastebin.com/qDgj4vy9
I wonder if it has something to do with the number of redirections from the original manifest URL.
-Bearded Gnome

Retested with felddy/foundryvtt:node-18.
Same error for PF2e:
https://pastebin.com/SQHj6izV
Also tried cURL from the container, it worked:
https://pastebin.com/2v5tnTxq
The base URL gets a redirect:
https://github.com/foundryvtt/pf2e/releases/latest/download/system.json
gets redirected to
https://github.com/foundryvtt/pf2e/releases/download/4.7.4/system.json
Back in Foundry, attempting to manual manifest install pointing directly to 4.7.4 also fails:
https://pastebin.com/2kn41sUF
The 4.7.4 URL gets a redirect to a long, temporary githubusercontent URL. Which, if you manually enter it fast enough into the Manifest URL form, will be successful:
https://pastebin.com/24tf5zPU
Speculation: The redirect from github.com to githubusercontent.com is failing for some reason.
-Bearded Gnome

Finally I found something regarding the series of bugs that plague my entire docker setup recently.
I have observed the same problems as everyone else commented here, but in addition my foundry docker also consistently fails to contact the foundry licensing signature servers, preventing me from starting the app at all recently.
I also noticed, anything accessing github hosted content can sometimes time out in compose created containers. For example, the following url seems to always time out in all of my docker containers, no matter the origin:
https://raw.githubusercontent.com/CubeCoders/AMPTemplates/main/xonoticserver.cfg (it is a configuration file for a game server panel and just used as an example to show that this problem exists also outside of foundry).

So I have been digging around a bit and it seems like recently a lot of containers have started to show timeout errors when trying to reach resources on the outside they should 100% be able to reach and I have yet to find any solution or even official issue raised for this.
It could of course be coincidence, but it would explain a lot of problems I have had recently and fits this issue.
Some examples:
https://www.reddit.com/r/docker/comments/11541kv/need_a_docker_expert_for_a_weird_problem/
https://www.reddit.com/r/docker/comments/1137147/curl_request_error_52_from_docker_container_but/
https://www.reddit.com/r/docker/comments/10tvpjv/last_time_for_minecraft_forge_server/

Edited Foundry to print out the error from fetch:
foundryvtttest    | AbortError: The operation was aborted.
foundryvtttest    |     at abort (file:///opt/foundryvtt/resources/app/node_modules/node-fetch/src/index.js:70:18)
foundryvtttest    |     at AbortSignal.abortAndFinalize (file:///opt/foundryvtt/resources/app/node_modules/node-fetch/src/index.js:89:4)
foundryvtttest    |     at AbortSignal.dispatchEvent (/opt/foundryvtt/resources/app/node_modules/event-target-shim/dist/event-target-shim.js:818:35)
foundryvtttest    |     at abortSignal (/opt/foundryvtt/resources/app/node_modules/abort-controller/dist/abort-controller.js:52:12)
foundryvtttest    |     at AbortController.abort (/opt/foundryvtt/resources/app/node_modules/abort-controller/dist/abort-controller.js:91:9)
foundryvtttest    |     at Timeout._onTimeout (file:///opt/foundryvtt/resources/app/common/utils/http.mjs:23:18)
foundryvtttest    |     at listOnTimeout (node:internal/timers:559:17)
foundryvtttest    |     at processTimers (node:internal/timers:502:7) {
foundryvtttest    |   type: 'aborted'
foundryvtttest    | }

Still need to investigate the root cause.

Confirmed it's not Foundry/Node. Curl is taking 9 seconds in the container, 0.4 directly on the host.
Time to reach out to Docker support.

@BeardedGnome What did you test specifically? I can't reproduce any timeouts with curl in the foundry container with your example URLs:
time curl -L -s -o /dev/null https://github.com/foundryvtt/pf2e/releases/latest/download
/system.json
real    0m 0.65s
user    0m 0.05s
sys     0m 0.01s

I fixed it in my setup.
It was the default docker MTU that was at fault.
By default the docker daemon has an MTU of 1500. However many network cards can have an MTU of less, and even if you have an MTU of 1500 on your main outside facing network interface on the host machine, some things like a VPN connecting, lower the effective MTU to something like 1420 (which was the case for me).
That means that packets that come from any docker container using the bridge network with its higher MTU have to be fragmented or result in fragmented return packages. These are often dropped by firewalls, resulting in the timeouts we see here. Services running directly of the host or containers using host networking are aware of the lowered MTU and thus work without problems.
So why would this only happen in compose for some? My best guess is that your default docker daeomon MTU is actually adjusted accordingly, but docker compose creates new default bridge networks for every service it starts and it doesn't use the daeomon's MTU parameter but defaults to a hardcoded 1500. This has to be adjusted for each compose network individually (or you create one overarching adjusted compose bridge and connect new containers to it instead of using a default network).
Below is a tutorial that explains the way how to adjust the MTUs for docker. You can check if it worked with "netstat -i" (please note that the docker0 interface shows MTU 1500 always, and only changes in netstat to your configured MTU when a container is connected to it).
https://www.civo.com/learn/fixing-networking-for-docker

Try pulling the Blade Runner system. That one has failed on me every time.Release 10.0.2 - Saint Nicholas 2037 · fvtt-fria-ligan/blade-runner-foundry-vttgithub.com-NickOn Feb 21, 2023, at 7:42 AM, Janosch Hildebrand @.***> wrote:
@BeardedGnome What did you test specifically? I can't reproduce any timeouts with curl in the foundry container with your example URLs:
time curl -L -s -o /dev/null https://github.com/foundryvtt/pf2e/releases/latest/download
/system.json
real    0m 0.65s
user    0m 0.05s
sys     0m 0.01s
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

@Jnosh - I used the instructions found here:
https://stackoverflow.com/questions/18215389/how-do-i-measure-request-and-response-times-at-once-using-curl
Host
mdahrea@fatsia:~/foundryTest$ curl -L -w @curl-format.txt -o /dev/null -s https://github.com/foundryvtt/pf2e/releases/latest/download/system.json
     time_namelookup:  0.002033s
        time_connect:  0.026777s
     time_appconnect:  0.238662s
    time_pretransfer:  0.238941s
       time_redirect:  0.425002s
  time_starttransfer:  0.642463s
                     ----------
          time_total:  0.656259s

Container to PF2e
mdahrea@fatsia:~/foundryTest$ sudo docker exec foundryvtttest sh -c "curl -L -w @/host/curl-format.txt -o /dev/null -s https://github.com/foundryvtt/pf2e/releases/latest/download/system.json"
[sudo] password for mdahrea:
     time_namelookup:  8.054977s
        time_connect:  8.080959s
     time_appconnect:  8.150718s
    time_pretransfer:  8.151287s
       time_redirect:  4.394603s
  time_starttransfer:  8.728799s
                     ----------
          time_total:  8.732226s

@salsabeard  - I see the same behavior with Blade Runner:
Container to Blade Runner
mdahrea@fatsia:~/foundryTest$ sudo docker exec foundryvtttest sh -c "curl -L -w @/host/curl-format.txt -o /dev/null -s https://github.com/fvtt-fria-ligan/blade-runner-foundry-vtt/releases/download/10.0.2/system.json"
     time_namelookup:  8.006251s
        time_connect:  8.028398s
     time_appconnect:  8.080048s
    time_pretransfer:  8.080311s
       time_redirect:  4.191276s
  time_starttransfer:  8.393353s
                     ----------
          time_total:  8.393802s

@Leolele99  - I don't think it's a MTU issue for me.
mdahrea@fatsia:~/foundryTest$ netstat -i
Kernel Interface table
Iface      MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
br-47612  1500     6006      0      0 0         25288      0      0      0 BMRU
br-52168  1500   145047      0      0 0        176684      0      0      0 BMRU
br-58522  1500     2002      0      0 0          2310      0      0      0 BMU
br-68033  1500   223337      0      0 0        284573      0      0      0 BMRU
br-a19c1  1500    50610      0      0 0         54403      0      0      0 BMRU
br-ea940  1500      640      0      0 0           830      0      0      0 BMU
docker0   1500     5589      0      0 0         20954      0      0      0 BMU
enp2s0    1500  1223022      0      4 0       1633509      0      0      0 BMRU
lo       65536      632      0      0 0           632      0      0      0 LRU
veth8347  1500   223337      0      0 0        284616      0      0      0 BMRU
veth3828  1500   145047      0      0 0        176726      0      0      0 BMRU
veth3e28  1500      126      0      0 0           130      0      0      0 BMRU
vethcf42  1500    50610      0      0 0         54446      0      0      0 BMRU

From the timing reports out of curl it looks like a DNS lookup issue. When I left off I hadn't gotten dig to run in the container.

I'm 99% sure my specific issue is that I'm running my local DNS server (pi-hole) in a Docker container. By default, docker-compose creates a unique network for each container and they're not configured to connect to each other. It takes until the default docker DNS server (<IP_ADDRESS>) times out after 4 seconds before falling back to another DNS server and getting the answer.
The two look ups (github.com and githubusercontent.com) and two 4 second timeouts coincide well with a 9 second total time.
I've tested this theory by bringing up the Foundry container on the pi-hole network and specifying the pi-hole IP address as the DNS. Seems to solve the problem:
mdahrea@fatsia:~/foundryTest$ sudo docker exec foundryvtttest sh -c "curl -L -w @/host/curl-format.txt -o /dev/null -s https://github.com/foundryvtt/pf2e/releases/latest/download/system.json"
     time_namelookup:  0.061142s
        time_connect:  0.083173s
     time_appconnect:  0.134501s
    time_pretransfer:  0.135124s
       time_redirect:  0.417493s
  time_starttransfer:  0.838467s
                     ----------
          time_total:  0.908495s

Now I need to determine the correct way to do this. Probably not by hard coding the IP address.

Quite an interesting find. I too am running a pi-hole container.-NickOn Feb 22, 2023, at 1:42 AM, Michael Dahrea @.> wrote:
I'm 99% sure my specific issue is that I'm running my local DNS server (pi-hole) in a Docker container. By default, docker-compose creates a unique network for each container and they're not configured to connect to each other. It takes until the default docker DNS server (<IP_ADDRESS>) times out after 4 seconds before falling back to another DNS server and getting the answer.
The two look ups (github.com and githubusercontent.com) and two 4 second timeouts coincide well with a 9 second total time.
I've tested this theory by bringing up the Foundry container on the pi-hole network and specifying the pi-hole IP address as the DNS. Seems to solve the problem:
@.:~/foundryTest$ sudo docker exec foundryvtttest sh -c "curl -L -w @/host/curl-format.txt -o /dev/null -s https://github.com/foundryvtt/pf2e/releases/latest/download/system.json"
time_namelookup:  0.061142s
time_connect:  0.083173s
time_appconnect:  0.134501s
time_pretransfer:  0.135124s
time_redirect:  0.417493s
time_starttransfer:  0.838467s
----------
time_total:  0.908495s
Now I need to determine the correct way to do this. Probably not by hard coding the IP address.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

So I did some additional investigation on this and found that Docker Compose specifically handles DNS differently than other Docker deployment methods.  Docker run and the GUI will pull DNS entries from the /etc/resolve.conf file of the host.Dock compose on the other hand does not inherently update the container’s resolve.conf with the host’s file without explicit configuration. So it appears that it relies on the default configuration of <IP_ADDRESS> and attempting to resolve DNS via the configured gateway.The solution to this that I keep finding is to specify DNS servers in the docker compose field.-NickOn Feb 22, 2023, at 1:54 AM, Nick Saylor @.> wrote:Quite an interesting find. I too am running a pi-hole container.-NickOn Feb 22, 2023, at 1:42 AM, Michael Dahrea @.> wrote:
I'm 99% sure my specific issue is that I'm running my local DNS server (pi-hole) in a Docker container. By default, docker-compose creates a unique network for each container and they're not configured to connect to each other. It takes until the default docker DNS server (<IP_ADDRESS>) times out after 4 seconds before falling back to another DNS server and getting the answer.
The two look ups (github.com and githubusercontent.com) and two 4 second timeouts coincide well with a 9 second total time.
I've tested this theory by bringing up the Foundry container on the pi-hole network and specifying the pi-hole IP address as the DNS. Seems to solve the problem:
@.***:~/foundryTest$ sudo docker exec foundryvtttest sh -c "curl -L -w @/host/curl-format.txt -o /dev/null -s https://github.com/foundryvtt/pf2e/releases/latest/download/system.json"
time_namelookup:  0.061142s
time_connect:  0.083173s
time_appconnect:  0.134501s
time_pretransfer:  0.135124s
time_redirect:  0.417493s
time_starttransfer:  0.838467s
----------
time_total:  0.908495s
Now I need to determine the correct way to do this. Probably not by hard coding the IP address.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

Alright, so I can confirm that this issue is specific to using pi-hole DNS container. Even when manually configured, it times out and fails. But if I changed to use external servers only, it works without any issue.-NickOn Feb 22, 2023, at 8:40 AM, Nick Saylor @.> wrote:So I did some additional investigation on this and found that Docker Compose specifically handles DNS differently than other Docker deployment methods.  Docker run and the GUI will pull DNS entries from the /etc/resolve.conf file of the host.Dock compose on the other hand does not inherently update the container’s resolve.conf with the host’s file without explicit configuration. So it appears that it relies on the default configuration of <IP_ADDRESS> and attempting to resolve DNS via the configured gateway.The solution to this that I keep finding is to specify DNS servers in the docker compose field.-NickOn Feb 22, 2023, at 1:54 AM, Nick Saylor @.> wrote:Quite an interesting find. I too am running a pi-hole container.-NickOn Feb 22, 2023, at 1:42 AM, Michael Dahrea @.> wrote:
I'm 99% sure my specific issue is that I'm running my local DNS server (pi-hole) in a Docker container. By default, docker-compose creates a unique network for each container and they're not configured to connect to each other. It takes until the default docker DNS server (<IP_ADDRESS>) times out after 4 seconds before falling back to another DNS server and getting the answer.
The two look ups (github.com and githubusercontent.com) and two 4 second timeouts coincide well with a 9 second total time.
I've tested this theory by bringing up the Foundry container on the pi-hole network and specifying the pi-hole IP address as the DNS. Seems to solve the problem:
@.:~/foundryTest$ sudo docker exec foundryvtttest sh -c "curl -L -w @/host/curl-format.txt -o /dev/null -s https://github.com/foundryvtt/pf2e/releases/latest/download/system.json"
time_namelookup:  0.061142s
time_connect:  0.083173s
time_appconnect:  0.134501s
time_pretransfer:  0.135124s
time_redirect:  0.417493s
time_starttransfer:  0.838467s
----------
time_total:  0.908495s
Now I need to determine the correct way to do this. Probably not by hard coding the IP address.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

It would seem that this error message can occur anytime the update checks fail or time out so there isn't a common cause, especially not one related to this docker image.
I checked the various DNS suggestions and none seemed to apply in my case...
I finally tracked the issue in my case to Foundry issuing update checks in batches of 10 with this error message being returned if they do not finish within a set timeout.
It would seem that for my NAS hardware these parallel requests often simply do not finish quickly enough and are aborted once the timeout is reached. I verified by modifying the foundry frontend to serialize the update checks which leads to them completing successfully for me.
I have contacted Foundry support and hopefully they can increase the timeout, provide an option to serialize the update checks for people on weaker hardware or provide some other solution.

@salsabeard

Alright, so I can confirm that this issue is specific to using pi-hole DNS container. Even when manually configured, it times out and fails. But if I changed to use external servers only, it works without any issue. -Nick

Glad we agree on the symptom.
I haven't figured out how to add the correct dns entry to the yml file yet. Host IP doesn't work, docker0 IP doesn't work,  host.docker.internal doesn't work.
Using the pihole IP only works when on the same network. Even then, only until the IP changes.
Using the gateway IP works, until a reboot and it gets a new subnet. I guess I could specify the subnet for every container.
I could try putting all of my services into one yml file or predefining a network for them all to join.
I'm a little shocked I haven't been able to find a simple guide - 'how to' use container to container DNS. We can't be the first people to run into this issue.

@BeardedGnome
You beat me to saying what I was going to say, I just hadn’t had a chance to validate new findings.
According to Docker’s official documentation regarding compose, a compose instance will build a default network (if an external network is not configured).  This network is used for all services in the compose instance to communicate with one another, and they do so using their host names.
In my current configuration, my pihole and foundry containers are part of separate compose instances, and so foundry cannot communicate with the DNS server that is running in the other instance.
For my next test, I will be creating an explicitly configured “network” and binding all of my compose instances to it.  If my understanding  is correct, this should allow all of the compose services communicate with one another, regardless of their container instance.  Good write up of this process here:
https://stackoverflow.com/questions/38088279/communication-between-multiple-docker-compose-projects
Now all that being said, I’m coming to realize that this behavior is likely impacting ALL of my containerized externally reaching services that aren’t in the same container as my pihole instance, but the failover to the external DNS servers is never really an issue with them.
This is where I think the Foundry source code may need to be tweaked.  As it seems to me, there isn’t enough fault tolerance to allow a DNS resolution timeout to occur with a primary server before essentially abandoning the download attempt and throwing an error.
Once I have a chance, I’ll get the new network configuration applied and report back.

I GOT IT!!!  FINALLY!!!!!!!
TL;DR - The compose yaml needs to include " network_mode: 'bridge' " to allow the container to use the host network and not automatically create a separated network.  Without this, you cannot use local DNS if it resides on the same server.
@felddy can you check into this and update the example you have listed?
My last theory regarding the problems with inter-container communication was not exactly far off base, but it wasn't as on the money as I'd hoped.

I created the external network and bound my containers to it, but the same issue persisted.
I shutdown my pi-hole container and brought up the native Synology DNS server, but again, the same issue persisted
I also identified that if I list only the local DNS server in the DNS server list, the foundry container fails to even auth with the foundry servers

I then started doing inspections on the containers and found a variance in behavior and inner working configuration from the Foundry compose container and every single other container on my server.

I did exec shell to each container and found that only the container with issues was resolving DNS via <IP_ADDRESS>
Every other container was using the local DNS server IP address

As for configuration, every container except for the one in that wasn't working has either host or bridge listed in the Networks field, but the container with problems was showing "dctest1-creds_default" indicating that it had created its own separate network for this container.
It then hit me like bricks and I realized that there is a difference between this bridged container and every other one I have on my server.  I added the line " network_mode: "bridge" " to the docker compose yaml, and now the compose container is using the local DNS and is having absolutely no issues downloading any content.

This solution seems to be working for me:
https://forums.docker.com/t/dns-issues-with-local-resolver-and-containers-on-the-same-host/102319
Advantage:

No changes to other containers
No changes to your networks

Disadvantage:

Have to hard code the host IP address for your DNS container

In my case the host IP address is static, so I can get away with this configuration.
Although, to be honest, I'm not really sure why that works.

Had the same issue with modules not installing due to timeouts ultimately related to a complicated docker bridge (and local pihole) network configuration.
An alternative simple work around that I ended up using is to add a --dns <IP_ADDRESS> parameter (or any other public DNS server) to have the container use public DNS server explicitly. Curl timing test went from around ~8s to ~.4s
Thanks for the lead and discussion! 🎉


Spacing between list items is too big

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:
Using the following code, I get an unordered list with a lot of white space between the lines. Using the debugger it seems that every bullet point is placed in it's own paragraph.
+ Test
+ Test
+ Test

or a full example:
var document = Document.Create(container =>
    {
        container.Page(page =>
        {
            page.Size(PageSizes.A4);
            page.Margin(1, Unit.Centimetre);
            page.PageColor(Colors.White);
            page.DefaultTextStyle(x => x.FontSize(10));

            page.Content().Row(row =>
            {

                row.RelativeItem(1)
                    .Markdown("Test line above list.\r\n* Test\r\n* Test\r\n* Test");
            });
        });
    });

document.GeneratePdfAndShow();

Expected behavior
I would like to see the bullet points in one paragraph and each bullet point on a new line, resulting each line without the white space. Wether this is default behavior or configurable is less of a concern to me. It just seems more logical for me to have items of the same list without whitespace. Just like in HTML <ul> with <li> behavior. Like:

Test
Test
Test

Screenshots

Note: Only the Test bullets are part of the Markdown code. The rest of layout/texts are not.
I can reproduce this behavior in various places.

Environment (please complete the following information):

OS and version:  macOS 14 Sonoma
OS Platform x64, arm64
QuestPDF 2023.12.1
QuestPDF.Markdown 1.3.0

Additional context
No comments at this moment.

Thanks for submitting this issue. The nested paragraphs seem to be a limitation of how markdig renders the markdown document, but it should be no issue to make the spacings within lists configurable. This think should address your issue.


Implemented a noscroll option

In reply to #206 i've created a different approach to handle the onscroll handler option.
This way you can use Turbolinks.visit(url, {scroll: 'noscroll'}); for noscroll.
For now the back button makes the content to scroll to top but maybe someone else can point me in the right direction.
Works great for page-filters or calendar navigation.

+1 on this merge.

Thanks for doing this! turbolinks-classic has equivalent function via enableTransitionCache, so the absence was a regression.

Is this available via data-attribute ?


helm - Incorrect zap log level value override

The value override for zap log level is defined as zap.logLevel in the documentation. However, the actual value is zapLogLevel
See:
https://github.com/metacontroller/metacontroller/blob/master/deploy/helm/metacontroller/templates/statefulset.yaml#L39
and
https://github.com/metacontroller/metacontroller/blob/master/docs/src/guide/helm-install.md?plain=1#L50

:tada: This issue has been resolved in version 2.6.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Maybe add a refresh property

Add a dummy property to shape_base that marks the shape as dirty.
Can be used to force refreshing the shape.
See also discussion in https://github.com/mphe/SmartShape2D/issues/5.

Like a custom property editor with a "Refresh" button?

Just a regular bool property with a setter that does nothing but marking the shape dirty. When you click the checkbox in the inspector, it won't change, but the setter will be called and the shape refreshes eventually.


Fix incorrect statusPageUrl and healthCheckUrl when management.server.port not set

Hello,
this PR is an attempt to fix incorrect urls being reported to Eureka since 2.0.0.M5.
It changes the following condition:
(managementPort == null && isRandom(serverPort))

which always evaluated to false due to serverPort having fallback to a default, as in:
int serverPort = Integer.valueOf(propertyResolver.getProperty("server.port", propertyResolver.getProperty("port", "8080")));

and hence always being != RANDOM_PORT (unless one sets the server.port or port to 0 in props but I don't really see a use case for that and how it could affect the Eureka registration part 🤔).
It also assumes that healthCheckUrl and statusPageUrl should not be set explicitly when management.server.port is not set (based on the note at the end of this paragraph of Spring Boot docs).
I hope it is useful. Any comments will be appreciated.
BR,
Barbara

@brbsk Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@brbsk Thank you for signing the Contributor License Agreement!

Codecov Report

Merging #2557 into master will decrease coverage by 0.03%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #2557      +/-   ##
==========================================
- Coverage   65.87%   65.83%   -0.04%
==========================================
  Files         207      207
  Lines        7654     7655       +1
  Branches      950      948       -2
==========================================
- Hits         5042     5040       -2
- Misses       2248     2250       +2
- Partials      364      365       +1

Impacted Files
Coverage Δ

...ka/metadata/DefaultManagementMetadataProvider.java
73.68% <100%> (-10.53%)
:arrow_down:

.../netflix/eureka/EurekaClientAutoConfiguration.java
92.64% <100%> (+1.34%)
:arrow_up:

...cloud/netflix/feign/support/SpringMvcContract.java
87.5% <0%> (+0.17%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 06398da...6570ef0. Read the comment docs.

@brbsk is this related to #2524?

I think it might be, however I originally noticed the issue somewhere else.
I am using just the eureka.instance.nonSecurePort and eureka.instance.ipAddress options, no management.server.port option set. I noticed that since M5 the nonSecurePort option doesn't get applied to managementPort as it gets set too early (specifically management.port key in metadata map in InstanceInfo; the same occurs for statusPageUrl and healthCheckUrl).
The issue reveals itself in an Eureka aware Turbine - it loses capability of connecting to an instance registered in such a way. For an M4 instance having same options set (eureka.instance.nonSecurePort and eureka.instance.ipAddress), Turbine has no issue.

This seems OK to me @spencergibb what do you think?


Added enumeration descriptions for the VM Connector Reference

Fixed DOCS-4294 by adding descriptions to the enumerations for the Transactional Action parameter.

Took notes of review comments for other parts of the doc so I can implement them when the Reference Guide version is updated.


Test launches additional browser before test starts

I am having all the default configs as is for jest.config, jest-puppeteer.config,babel.config,eslintrc.js. I am running simple test to launch the browser in headless mode but I see browser opens as soon as I start test by "npm test"  ? why and from where does the browser gets launched ?
Test:
const puppeteer = require('puppeteer');
import 'expect-puppeteer'
let browser;
let page;
describe('Google', () => {
beforeAll(async () => {
browser = await puppeteer.launch();
page = await browser.newPage();
await page.goto('https://google.com')
})
it('should display "google" text on page', async () => {
await expect(page).toMatch('google')
await page.close();
await browser.close();
})
})

see the example app in https://github.com/smooth-code/jest-puppeteer/tree/master/examples/create-react-app.
You don't need to launch puppeteer yourself in the tests the jest-puppeteer-environment will do that.

OK, so jest-puppeteer-environment launches the default browser. And that is the recommended option? I have modified my test to not launch browser and in headless mode. But it will still launch the non-headless browser (from jest-puppeteer-environment).
Can we make jest-puppeteer-environment browser headless ? and How ?

headless option

Thanks UziTech,
headless: process.env.HEADLESS !== 'false', makes it headless.
Can we configure to make it work from commandline ?

Hi UziTech,
figured out to run in commandline both headless and non-headless
"test": "cross-env HEADLESS=true jest --verbose --no-cache --runInBand",
"test:nonheadless": "cross-env HEADLESS=false jest --verbose --no-cache --runInBand"

You can also configure it in jest-puppeteer.config.js
// jest-puppeteer.config.js
module.exports = {
  launch: {
    headless: true
  }
}


My honor just went from 479 to 79!

http://www.codewars.com/users/joebandenburg
I am rank 2 kyu, half way towards 1 kyu. I just completed a 4 kyu problem and upon completion my honor dropped by 400 points. Did I do something very dishonorable?

In the "Allies" table on the home page my correct honor (493) is displayed along with my correct position in that table.

Maybe it was some g'glitch? Actually system shows that you have 493 honor points. Take look at http://www.codewars.com/users/joebandenburg/site-events.

@adam-tokarski Good spot. It seems to have been an issue with my session. My honor is now displayed correctly after I signed out and back in again.
I'll leave this issue open as there is clearly a bug lurking somewhere.


Add visual to show the connections between feature points when using SIFT

Add visual to show the connections between feature points when using SIFT. This allows to visually inspect the succes of aligning images when using SIFT

Scikit-image has some example code we can re-use for this:
https://scikit-image.org/docs/stable/auto_examples/features_detection/plot_sift.html

Ok, great! I was already inclined to use opencv for this

I'm on it, in feature_points branch!

Yeah, it's a nice way to reduce clutter


GPTQ Formats that work (and don't)

Now that we can load GPTQ files that haven't been quantized by TGI's quantization script, I thought I'd do a set of tests to see which formats work and which don't. I'm using https://huggingface.co/TheBloke/OpenOrca-Preview1-13B-GPTQ as an example set.

The 'most compatible' format ([main] branch) doesn't work. This throws the following error: RuntimeError: weight model.layers.0.self_attn.q_proj.g_idx does not exist
Fortunately, the other formats provided by TheBloke do seem to work. In particular: gptq-4bit-128g-actorder_True definitely loads correctly. To use this, you need to set the following environment variables: GPTQ_BITS = 4, GPTQ_GROUPSIZE = 128  (matching the groupsize of the quantized model). Additionally, you need to pass in REVISION = gptq-4bit-128g-actorder_True to pull the correct version of this model (rather than the default version which still doesn't work).

So overall this is great news - we can now load GPTQ files that other people have converted rather than relying on the inbuilt quantizer in TGI!

That's great to hear.  I started adding those extra quant formats recently with software like TGI and ExLlama in mind.
To the developers of the GPTQ plugin I'd like to ask: is there any chance you could add support for the quantize_config.json file?  It's produced automatically by AutoGPTQ when making a quantisation, and I provide it with every one of my GPTQ files, even the ones made with GPTQ-for-LLaMa.  It contains all the GPTQ parameters, and it could easily be used as a source for the GPTQ params, saving the user the need to set them manually via env vars.
Here's an example quantize_config.json:
{
  "bits": 4,
  "group_size": 128,
  "damp_percent": 0.01,
  "desc_act": true,
  "sym": true,
  "true_sequential": true,
  "model_name_or_path": null,
  "model_file_base_name": null
}

Pretty self explanatory. You'd just need to read bits and group_size from this file, found in the model folder, and it could work automatically.
I'd be happy to PR a change myself, if someone confirms it would be merged.

Similar issue for me, trying to get Vicuna 7B GPTQ models to run with TGI
RuntimeError: weight model.layers.0.self_attn.q_proj.weight does not exist
So far haven't gotten it to work. Looking forward to the README updates with more info about using these models with TGI @TheBloke !

@kaleko I updated the Vicuna 7B v1.3 GPTQs yesterday.  Download one of the models from the other branches, as listed under Provided Files.  They will work if you manually set the GPTQ_BITS and GPTQ_GROUPSIZE as ssmi153 mentioned in the first post.
You can download from alternate branches using the REVISION parameter in TGI.
https://huggingface.co/TheBloke/vicuna-7B-v1.3-GPTQ

is there any chance you could add support for the quantize_config.json file?

This is actually much cleaner than the ENV variables I added. I'm more than happy to switch to it.
Is it correct to assume that for inference we can discard all other config other than bits and groupsize ?
Just want to avoid loading/running a model and output garbage if we can raise an error early.
I'm actually very fine aligning and outputting such a config instead of putting the values in the weights. Wdyt @OlivierDehaene  ?
We also have some work to use a better GPTQ kernel: https://github.com/huggingface/text-generation-inference/pull/553 if that's interesting.
The reason for missing g_idx is the use of autogpt, correct ?

The AutoGPTQ quants from TheBloke seem to work actually. It's the ones that TheBloke is converting using GPTQ-for-Llama that are causing the missing g_idx error. Weirdly though, the GPTQ model file that I've created using GPTQ-for-Llama loads correctly, so there must be a difference somewhere in the way TheBloke is processing those files, or in the settings he's chosen for them.
@TheBloke , out of interest, how do your GPTQ-for_Llama conversion settings compare to this?:
%run -i 'GPTQ-for-LLaMa/llama.py' {INPUT_MODEL_FULL_NAME} wikitext2 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors {OUTPUT_MODEL_FULL_NAME_SAFETENSORS}
My conversion of a 33B Llama model with these settings works with the current TGI implementation. Maybe we can identify any different settings on your side which might allow us to isolate where the problem is.

My settings are effectively identical.  The reason that my GPTQ-for-LLaMA quants aren't working is that they're using the "old" GPTQ format. There's never been any official GPTQ version naming, but some implementations call it "v1", versus the current "v2" format which has g_idx.
AutoGPTQ produces the new format and can load either format, but TGI uses an implementation that can only load the newer v2 format.  It's a shame that TGI chose an implementation that can't load the old format, as it meant that at that time it couldn't support most of the GPTQs on Huggingface.
Going forward I will always be making multiple GPTQs available, nearly all of which will be made with AutoGPTQ and will work.  For now I am still also providing an old GPTQ-for-LLaMa produced version which doesn't work.  I haven't decided yet if if I will continue making that old format version as well - I  need to take a survey of the current state of all the various UIs out there and confirm they can all load the v2 format.
I wanted to switch to making all quants with AutoGPTQ a while back, but as soon as I did I got complaints from users using KobaldAI that the newer format models didn't work with CPU offload.  So to avoid hassle (and because I lacked the time to test it on KoboldAI myself), I just went back to GPTQ-for-LLaMA for making Llama GPTQs.
That issue may even be fixed now, so I should re-evaluate that and hope to do so soon.
Over the last few days I've uploaded multiple options of AutoGPTQ-produced quants to 62 GPTQ repos. And all my non-Llama models (Falcon, MPT, Bloom, Starcoder) were already produced with AutoGPTQ, so should already be TGI compatible. Most of the repos I haven't done are ones I don't plan to do, because they're old and superseded. Eg I didn't do Vicuna v1.1, just v1.3.
My next step is to update my SuperHOT GPTQ repos also - that should happen this weekend.

is there any chance you could add support for the quantize_config.json file?

This is actually much cleaner than the ENV variables I added. I'm more than happy to switch to it. Is it correct to assume that for inference we can discard all other config other than bits and groupsize ? Just want to avoid loading/running a model and output garbage if we can raise an error early.

Fantastic, thanks!
The other parameter that sometimes matters is desc_act, which in GPTQ-for-LLaMa is called Act Order.   With AutoGPTQ it needs to know whether that's true or false else inference will produce gibberish.
But as you don't have an ENV var for it, I assume that GPTQ-for-LLaMa is able to auto detect that somehow, so it presumably isn't required.  It isn't required in ExLlama so that's further confirmation that it is possible to auto detect it.

I'm actually very fine aligning and outputting such a config instead of putting the values in the weights. Wdyt @OlivierDehaene ? (If we can reduce the current split around GPTQ it's all for the best)
We also have some work to use a better GPTQ kernel: #553 if that's interesting.
The reason for missing g_idx is the use of autogpt, correct ?

Sorry I missed your earlier reply when I replied to ssmi's ping, else I'd have replied to you directly, but this is answered above: AutoGPTQ outputs g_idx and the files work fine, it's the old GPTQ-for-LLaMa CUDA version I have been using that doesn't output g_idx.
I would really love to stop using that old GPTQ-for-LLaMa code and will do as soon as I've confirmed there's no need to do so any more.
But either way, I'll always have AutoGPTQ-produced GPTQs in future which it's confirmed TGI can load OK.

I would really love to stop using that old GPTQ-for-LLaMa code and will do as soon as I've confirmed there's no need to do so any more.

You mean using https://github.com/PanQiWei/AutoGPTQ instead of https://github.com/qwopqwop200/GPTQ-for-LLaMa correct ?
I used specifically https://github.com/qwopqwop200/GPTQ-for-LLaMa because I found the code easier to reason about.
Notably we don't need the modeling code at all in either lib.
I was able to refactor the code to load nothing on the CPU by default, and pull only layer per layer (on CUDA, but could be CPU) for quantization.
Makes everything a bit easier to work with. We currently don't provide a way to select the data sent during quantization but I'm not sure how much that really matter (Didn't seem to matter that much for llama derived models)
All that to say that I'm hesitant to pull from either codebase because they are quite large and we only use a very tiny fraction of the code. Trying to find common grounds would be nice.

@TheBloke, TGI seems to have issues with H100s I'm not sure why yet.
Any chance you could test on another device? I was able to launch the model on 1xA10 for example.
You can also use ghcr.io/huggingface/text-generation-inference:sha-44acf72  with the env var LOG_LEVEL=info,text_generation_launcher=debug for more logs.

@Narsil  Sorry, I didn't mean you should stop using GPTQ-for-LLaMa.  I was just talking about my own quantising processes. I meant I would like to stop using the old CUDA fork of GPTQ-for-LLaMa for making new quants and uploading them to HF.
I have used an old CUDA fork because for a long time for my 'main' branch GPTQs because there were UIs out there that couldn't support the g_idx GPTQ format in all scenarios.  But that might not be the case any more and I plan to check that.
If you don't even use the modelling code then it sounds like you have a very lightweight implementation so that's great.
@OlivierDehaene  Ah I see, OK thank you.  I wanted to use the H100 specifically as I'm benchmarking TGI vs vLLM and seeing what the max performance I can get in a multi-request scenario on a single GPU.
I tried the logging container but unfortunately I get no extra logs around the crash:
2023-07-17T20:15:20.153880Z  WARN text_generation_launcher: We're not using custom kernels.
2023-07-17T20:15:26.260462Z  INFO text_generation_launcher: Server started at unix:///tmp/text-generation-server-0
2023-07-17T20:15:26.267192Z  INFO shard-manager: text_generation_launcher: Shard 0 ready in 9.516731042s rank=0
2023-07-17T20:15:26.363369Z  INFO text_generation_launcher: Starting Webserver
2023-07-17T20:15:27.692885Z  INFO text_generation_router: router/src/main.rs:346: Serving revision 738964d963bd0aa13be29cb3ded3590a803b3b7e of model TheBloke/vicuna-13b-v1.3.0-GPTQ
2023-07-17T20:15:27.701918Z  INFO text_generation_router: router/src/main.rs:212: Warming up model
2023-07-17T20:15:28.912342Z ERROR warmup{max_input_length=1024 max_prefill_tokens=4096 max_total_tokens=16000}:warmup: text_generation_client: router/client/src/lib.rs:33: Server error: transport error
Error: Warmup(Generation("transport error"))
2023-07-17T20:15:28.969656Z ERROR text_generation_launcher: Webserver Crashed
2023-07-17T20:15:28.969692Z  INFO text_generation_launcher: Shutting down shards
2023-07-17T20:15:28.994843Z  INFO shard-manager: text_generation_launcher: Shard 0 terminated rank=0

Later I'll see if a local install, or a custom Docker container with later pytorch version, makes any difference.

Ah I just realised it's already reported here: https://github.com/huggingface/text-generation-inference/issues/613 . I'll post there and stop de-railing this thread as it's obviously H100 specific.

@TheBloke - I just did some benchmarking of TGI on Runpod instances using a range of GPU combinations. (https://docs.google.com/spreadsheets/d/1Ph_GeybAtNVoTs7w4mkCfd7p1lGywsNhJf9z-8fTcUE/edit?usp=sharing - if you're interested). This benchmarking was designed to reflect how I would use TGI and may not be as robust as some of the more formal benchmarks. One thing to note when benchmarking using the Docker image is that I get (and also you look like you've got) a warning saying WARN shard-manager: text_generation_launcher: We're not using custom kernels. . I think this is because by default the combination of Runpod + the docker image doesn't have NVIDIA NVCC installed (it requires the developer version of the NVIDIA container toolkit rather than the standard one), so it can't build the custom kernels or vLLM for PagedAttention. I'm struggling to work out whether this warning is a red herring or not, as TGI is still impressively fast. If it is indeed correct, then there should be a further uplift in performance above and beyond what I've seen so far. I haven't yet created an issue about the warning because it felt like I was raising a million Runpod related issues, but if other people are also seeing this (and on other platforms using the Docker image) then it might be worth exploring it further.

@ssmi153, this warning is a bit dismissive. If you don't see import errors and your architecture is one of the optimized architecture (as displayed in the README), you are using flash and paged attention.
This warning only applies to BLOOM and non-flash neox.

@ssmi153 Can you confirm the new llama 2 gptq versions working? I am getting error with Bloke's autogptq branch:
RuntimeError: weight model.layers.0.self_attn.q_proj.weight does not exist
WizardCoder is also giving an error when trying to load gptq version of it.:
raise RuntimeError(f"weight {tensor_name} does not exist")
RuntimeError: weight gptq_bits does not exist
I am running the docker command with the variables you mentioned at start and there are models that do work so wondering why this one is throwing error as it works fine in fp16.

The fix seems to work for llama 70b but is pretty slow.
Was wondering if you got the time to check wizardcoder. Still having the issues of loading the gptq version:
raise RuntimeError(f"weight {tensor_name} does not exist")
RuntimeError: weight gptq_bits does not exist

@bloodsucker99 You need to pass the environment variable GPTQ_BITS (though I think gptq_bits and gptq_groupsize could be directly inferred from the shapes of qweights, qzeros, g_idx?)

Narsil was also looking into adding automatic support for quantize_config.json which would provide the bits and group_size without the user needing to specify any env vars.
But yeah, I think ExLlama can automatically detect quantise config, so maybe neither env vars nor quantize_config.json are needed and it can just be auto detected?  That would be the ideal scenario.

@TheBloke I am trying to run your quantized model: TheBloke/Llama-2-70B-chat-GPTQ on V100 but TGI is complaining about compute capability < 7.5 detected. Is it that flash attn is not supported for V100 GPU? Does that mean all GPTQ models can only run on Ampere or later cards as well?

Indeed flash isn't supported on V100, and sharding requires flash for llama.

Indeed flash isn't supported on V100, and sharding requires flash for llama.

I tried to set env var for Sharding to False. But it doesn’t work too. Means I definitely have to load full model on v100 only? I rmb I tried bitsandbytes and it worked but was relatively slower since there is offloading to ram.

older models?

what do you mean ? All TheBlock models have this quantization configuration, no ?

Sorry i meant older GPU cards such as V100.

So the problem is in flash santacoder modeling, its trying to use to bits values from the weights and not using environment values. Manually forced the use if env variables fixed the issue :)

@TheBloke When I tried to run with TheBloke/Llama-2-7b-Chat-GPTQ, I got the following error:
warmup{max_input_length=4096 max_prefill_tokens=4096}:warmup: text_generation_client: router/client/src/lib.rs:33: Server error: Not enough memory to handle 4096 prefill tokens. You need to decrease --max-batch-prefill-tokens
Error: Warmup(Generation("Not enough memory to handle 4096 prefill tokens. You need to decrease --max-batch-prefill-tokens"))
Actually, I am able to serve the official meta-llama/Llama-2-7b-chat-hf with --quantize bitsandtyes on single T4 GPU. When I change the model to TheBloke/Llama-2-7b-Chat-GPTQ with --quantize gptq. I got NOT enough memory error. Even when I changed --max-batch-prefill-tokens=2048, this error still happens. Since the bitsandtypes-quantized version can be served on single T4 GPU, gptq-quantized version should be of no problem, do you happen to know why?

I have the same error when trying to load TheBloke/Llama-2-7b-Chat-GPTQ

I'm hitting the same issue as @taori when trying llama 2 70b chat. The error message:
"level":"ERROR","message":"Server error: Not enough memory to handle 4096 prefill tokens. You need to decrease `--max-batch-prefill-tokens`","target":"text_generation_client","filename":"router/client/src/lib.rs","line_number":33,"span":{"name":"warmup"},"spans":[{"max_input_length":1024,"max_prefill_tokens":4096,"name":"warmup"},{"name":"warmup"}]}

This is my YAML manifest:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: llm
spec:
  replicas: 1
  selector:
    matchLabels:
      app: llm
  template:
    metadata:
      labels:
        app: llm
    spec:
      containers:
      - name: llm
        image: ghcr.io/huggingface/text-generation-inference:1.0.3
        resources:
          limits:
            nvidia.com/gpu: 2
        env:
        - name: MODEL_ID
          value: TheBloke/Llama-2-70B-chat-GPTQ
        - name: NUM_SHARD
          value: "2"
        - name: QUANTIZE
          value: gptq
        - name: GPTQ_BITS
          value: "4"
      nodeSelector:
        cloud.google.com/gke-accelerator: nvidia-l4


Enable translucent TextureView configuration

Resolves https://github.com/tangrams/tangram-es/issues/2235
The changes to the demo app will be reverted before merging. This change enables a map using a TextureView to use alpha blending with views behind it in the layout - like in this image:

To achieve this, an application needs to:

Call getMapAsync with a TextureViewHolderFactory that has translucency enabled: https://github.com/tangrams/tangram-es/compare/textureview-translucent?expand=1#diff-80d33429d8f446a1f050db50f09ee177db5eb37139545d51a32ec6bf4fa0e799R111
Render parts of the map scene with the desired opacity, such as the background: https://github.com/tangrams/tangram-es/compare/textureview-translucent?expand=1#diff-6eeced4c21ee33c82ee5a8fdb6750e5bb4497b265688c744c191c68b41173dd9R17

Great use of TextureView handler. Will try to look at the code sometime next week, but if you want to merge before that, please go ahead.
Thanks Matt


Add LLaVA support, modify generate function

This is continuation of https://github.com/TransformerLensOrg/TransformerLens/pull/818 topic. Now make PR to dev branch.
Also I've decided that it would be good idea for generate function work with any type of input type and return type. I decided that it would be more readable and simply if generate function will generate sequence using embeddings as input to forward function. It is because string and tokens can be casted to embeddings but embeddings cannot be casted backward to tokens or string. And I made possible to give any input (string, list of strings, tokens, embeddings) ang get any output of function. Maybe there is something that I didn't make correct. For example, I'm not sure if my implementation is okay with positional embeddings.
I've met problem, LLaVA processor = AutoProcessor.from_pretrained(model_id, revision="a272c74") do not have apply_chat_template method at transformer version of project. So it is needed to update transformer library for this method work.
I've made test for any type of input and return types with gpt-2 model and it passed. But outputs of gpt-2 looks very poor though maybe for gpt-2 it is normal

I've also rerun all LLaVA.ipynb file and even tested other types of output of generate function and all worked pretty well.

[x] New feature (non-breaking change which adds functionality)

[x] I have commented my code, particularly in hard-to-understand areas

[x] I have made corresponding changes to the documentation

[x] My changes generate no new warnings

[x] I have added tests that prove my fix is effective or that my feature works

@zazamrykh I saw the initial struggles with the branch selection (dev-3) and appreciate the effort you're making to get this PR in shape for dev. Will be incredibly useful to have transformerlens work also with llava (vision) models. Just wanted to show some early appreciation for the effort in the hope you'll stick it out and not abandon the effort midway due to "pulling your hair" in frustration, while not knowing if anyone else really cares. I care :)

@zazamrykh I saw the initial struggles with the branch selection (dev-3) and appreciate the effort you're making to get this PR in shape for dev. Will be incredibly useful to have transformerlens work also with llava (vision) models. Just wanted to show some early appreciation for the effort in the hope you'll stick it out and not abandon the effort midway due to "pulling your hair" in frustration, while not knowing if anyone else really cares. I care :)

I am very glad to hear such a words of supporting) Changes in one function invoke changes in others so I've changed some functions which initially I haven't thought change. Then I've broke generate function, then, as I've thought, I've fixed it, but now some testes are not passing! I'll try to clarify what is the reason of failing tests. If you want to investigate VLM now, I think you can get last commit of feature-llava-support branch from forked repository, but I hope llava support appears soon in transformerlens!


Add buttons to Application Status page

Issue:

https://github.com/bcgov/entity/issues/22656

Description of changes:

Replace p tags with actual buttons
Tech debt cleanup

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the namex license (Apache 2.0).

Temporary Url for review: https://strr-ui-dev--pr-85-8bubya4x.web.app

Passed, after convo with Dima.


Above procedure to turn on nullable reference types for an entire project is incorrect for VS 2019 Preview 2

incorrect for VS 2019 Preview 2 "<NullableReferenceTypes>true</NullableReferenceTypes>"
should be replaced with "<NullableContextOptions>enable</NullableContextOptions>"
more info at https://github.com/dotnet/roslyn/blob/master/docs/features/nullable-reference-types.md#setting-project-level-nullable-context

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b991cba9-efd4-68ae-283a-b575054b4907
Version Independent ID: 793f790b-e2aa-e292-b12b-f4ba9763f172
Content: Design with nullable reference types
Content Source: docs/csharp/tutorials/nullable-reference-types.md
Product: dotnet-csharp
GitHub Login: @BillWagner
Microsoft Alias: wiwagn

This was fixed in #10381
Closing as fixed.


Improvements

Hello Hadi,
I just took some time to isolate the public improvements from the assignments, and pushed them to a fork. Feel free to merge or cherry-pick whatever you think is useful.

Commit
Change

47f967a
Properly delete output file from HDFS. If it is not deleted, a second run of the program raises a Java exception.

a9b10d2
Ignore internal_use/tmp directory

4b860bf
Remove Python -u option. This did not work on Hortonworks VM for me.

365d5ae
Use modern toolchain: hdfs and yarn instead of hadoop

bbeddcc
Don’t hardcode paths in Java code. This failed on a local Hadoop cluster I have access to.

d9611bc
Workarounds to work on Mac OS X: (1) Use md5 (from Openssl) instead of md5sum (from GNU coreutils) is unavailable. (2) Enforce Java 1.7 output (To be more correct, I should also set the bootstrap class path).

0aae948
Improved run.sh script: (1) Check if settings.sh is sourced. (2) -v option (verbose) show commands as they are executed. (3) Allow selective execution of assignments. E.g. run.sh A D to run assignments A and D. The default is to run them all.

Regards,
Freek
PS: it is shocking to see how many people push their assignment solutions to a public repository, thus violating the third clause of the Coursera Honour Code!
PS2: I just realized that none of these commits have a sign-off line. In case you appreciate it: Signed-off-by<EMAIL_ADDRESS>A.k.a.: yeah, I really am to blame me for any bug introduced by these commits ;).

Thanks Freek,
Nice work.
I will merge these issues in the master as soon as course is ended.


feat(dal): only run update_from_prototype_function in DVU

To reduce more chances of conflict, remove all calls to update_from_prototype_function for dependent functions, restricting their use to the DependentValuesUpdate job (except in the case of the function execution call on the customize screen, which we have to decide how best to handle).

bors r+


Change degrees to an absolute value when converting to DMS format to …

…avoid negative numbers. Magnitude in DMS is indicated by cardinal directions as opposed to sign.
Fixes #173

Thanks for this, @mpatzer! Apologies for taking so long to get this merged in. I'm in the process of cutting a new release now.


OSSM-3912 Unify install bookinfo and install/upgrade smcp test

Jira: https://issues.redhat.com/browse/OSSM-3912
I unified test smcp install and upgrade and installation bookinfo test case in only one test case with sub test cases that verify that after upgrade bookinfo application continues working properly.

@luksa please check because the proxy start check time I think could be better, right now I'm only taking the time between pods ready and validation of bookinfo works properly

Ok, I submited some changes and will explain why I deleted the basic install of 2.x smcp. These changes are to improve the testing basic or smoke test group, so in this test we will check:

Install smcp -1 version
Install Bookinfo and verify that works
Upgrade to version
verify that Bookinfo still work
Delete version

We have only one more additional smoke test in the testing group:

TestRequestRouting: this test install de default testing SMCP so the installation of the SMCP is covered by this test, we do not need to retest the install smcp if another test is doing this

In fact all the others test cases in full testing group will fail if we have a issue with the smcp installation so the smoke test covers the basic
@luksa

I have an execution error in the test, checking this

I submit the test working but the last subtest for deletion of the smcp not work because a route is still in istio-system namespace:
route.route.openshift.io/bookinfo-bookinfo-gateway-525eca1d5089dbdc   bookinfo-bookinfo-gateway-525eca1d5089dbdc-istio-system.apps.fh.servicemesh.rhqeaws.com          istio-ingressgateway   http2                 None

I think the reason because this happens it's because we install bookinfo in 2.3 smcp and them do the upgrade and after the upgrade and verification of the bookinfo we delete the app. When the sub test case for deletion of the smcp goes for all the resources I think this resource is not listed and is not deleted
Make any sense to you this @luksa

The error commented before will disappear with the latest build because IOR will be enable by default

/retest

/test lint-2-4

Will drop this PR to fix the merge conflict


Replacing libdice with dicebag

Your old version wasn't working for me, so I frankensteined it with dicebag and made it work. Sorry for the mess, I don't know anything about ruby or github, so I'm guessing I've violated some massive code hygiene laws for both.
Anyway, here you go, if you want it.
Stay Classy

Hey, thanks mate! :) However I like my old die roll interface better, so I'm keeping it.
But you say the old version was not working? Was there a specific technical problem with the old file or did it just not fit your needs? I couldn't find anything wrong.

Like I said, I know nothing of Ruby, but I
 can tell
the issue is somewhere in that msg.print () that prints when you type !roll
xdy, because the help text works fine, and the line works if you write
anything else in.
 In that case, the issue is either in how it's being printed out, or
possibly something in libdice itself. The elsif works just fine. There is
no error that spits out, and the dicebot does not crash, by the way.
I can understand having an attachment to code you've personally written,
and if anything this is a temp fix.
My version of murmur is
 1.2.13.
Closed #3 https://github.com/vaxr/mumble-dicebot/pull/3.
—
Reply to this email directly or view it on GitHub
https://github.com/vaxr/mumble-dicebot/pull/3#event-526103266.

Thanks for your help! Still not sure what's going on though. I thought it might be the Ruby version, but it seems to work fine both with 1.9.3 and 2.1.0. Or maybe I'm doing it wrong.
As for keeping the old code, it is not so much being attached to the code, but being able to mix arbitrary dice with text and have the results printed per die, rather than a total sum:
[11:17 PM] To Root: !roll I roll a d20 to hit and 4d6 damage.
[11:17 PM] (Channel) dicebot: vaxr: I roll a 12 to hit and 3 1 6 4 damage.


Change the type of Samza Runner MaxBundleSize pipeline option to match with Flink Runner

Currently there is an issue that when both Flink and Samza runners are in the same classpath, they have different types of MaxBundleSize pipeline option, i.e. Long v.s. long. This causes problem when creating PipelineOptions. This patch makes  them consistent.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

@yananhao12 : please take a look. Thanks.


diaPASEF module

Notes:

Having .toml files for each module is perhaps overkill, as it implies tool output parsing has to be re-implemented (to some extent) for each module, which is not desired. However, even small changes in datasets used render a .toml useless (for example, runs in the AIF module are differently named than the diaPASEF runs). What's the best way to handle this.

I actually do not have any further comments @rodvrees, maybe ready for review? And I see no reason not to merge.

I have an edge case that needs fixing, maybe you can help @RobbinBouwmeester :
When I open the web interface and do a benchmarking run in the diaPASEF module, it works as desired.
However, when I open the web interface, do a benchmarking run in the AIF module (which works), and then straight after that in the diaPASEF module (without refreshing first). The 'Results new data' tab will show the plot with the data from the AIF json, with the new run. As if the wrong json is loaded (I think it has something to do with below code, where submitted_data is already assigned from earlier and not re-assigned). I can't find it how to solve it immediately, any idea where to look?
https://github.com/Proteobench/ProteoBench/blob/2b328c23c9e4c4a3dbe1fa7a03e2fc53e28cb555/webinterface/pages/base_pages/quant.py#L256C1-L262C14

I know where this is, it is likely in the page variables. It safes the results of diaPASEF in the session state, but the same var name is used to retrieve it from the session state for AIF.
FYI, I am going to work from this branch on another branch to avoid too many merge conflicts... But the sooner we can merge the better ;)

Of course, I forgot to change 1 variable name... We should be good to go now


[Table] onRowSelection return 'none' string on unselect all

@umidbekkarimov Looks like a good feature


Azure AD: access token validation via oidc_jwt_keyfile endpoint

Issue:

Most of IdPs' $oidc_jwt_keyfile endpoint works for access token validation. But Azure AD doesn't work as the following message:
[info] 87#87: *500 JWT RS validation failed kid:"2ZQpJ3UpbjAYXYGaXEJl8lV0TOI" (SSL: error:04091068:rsa routines:int_rsa_verify:bad signature), client: <IP_ADDRESS>, server: nginx.azure.test, request: "GET /v1/api/example HTTP/1.1", host: "nginx.azure.test:15000"

Because, if nonce includes in access token, validating signature with JWT.io or JWT SecurityToken won't success.

To solve the #1, we can add "[CLIENT_ID]/.default" in authorization endpoint of Azure AD.

However, the access token doesn't work for userinfo endpoint as a Bearer in the header of API request although the validation works.
{"error":{"code":"InvalidAuthenticationToken","message":"Access token validation failure. Invalid audience.","innerError":
{"date":"2022-10-15T07:13:25","request-id":"xxxx-xxx-xxx-xxx-xxxx","client-request-id":"xxxx-xxx-xxx-xxx-xxxx"}}}

To successfully get 200 response from the /userinfo endpoint, we either can remove the scope in the authorization endpoint or find to solve invalid audience.

Misc:
 Azure AD Access Token TypesThere are two main types of Azure AD access token, summarised below, and our problem is that we are using the wrong type of token:

Audience
Description

Microsoft APIs
These contain a nonce field in the JWT header and are not designed for custom APIs such as ours to validate

Custom APIs
Custom APIs need to get a token that can be validated, and we are not yet configured to enable this

- If we get a token with a nonce field in the JWT header, then it is intended for Microsoft APIs to validate, and will always fail standard signature based validation.
References:

https://github.com/AzureAD/microsoft-authentication-library-for-js/issues/521#issuecomment-577400515
https://stackoverflow.com/questions/45317152/invalid-signature-while-validating-azure-ad-access-token-but-id-token-works
https://learn.microsoft.com/en-us/azure/active-directory/develop/userinfo
https://github.com/AzureAD/microsoft-authentication-library-for-js/issues/815
https://authguidance.com/azure-active-directory-setup/
https://authguidance.com/azure-ad-troubleshooting/

 Azure AD Access Token TypesThere are two main types of Azure AD access token, summarised below, and our problem is that we are using the wrong type of token:

Audience
Description

Microsoft APIs
These contain a nonce field in the JWT header and are not designed for custom APIs such as ours to validate

Custom APIs
Custom APIs need to get a token that can be validated, and we are not yet configured to enable this

If we get a token with a nonce field in the JWT header, then it is intended for Microsoft APIs to validate, and will always fail standard signature based validation.


[Request-evaluator] Improve debuggability

Improve debuggability of the request-evaluator:

Add pretty stack traces for all requests being evaluated
Add a -debug-cycles frontend option to dump cycles when we encounter them (without affecting type checking behavior)
Improve output of the dependency dumps with color and cycle highlighting

@swift-ci please smoke test and merge


Add npm scripts (dev/deploy).

Adding npm scripts as per https://github.com/MoritzStefaner/ach-ingen-zell/issues/4
To run locally, use npm run dev
To build prod, use npm run deploy
In production, we set NODE_ENV=production, for React (see: https://facebook.github.io/react/downloads.html#npm)
Also, we removed source-map from webpack.config.js and explicitly added it to dev build (assuming you don't want to have source maps in production).

Thanks! Very helpful.


[FLINK-18973][docs-zh] Translate the 'History Server' page of 'Debugging & Monitoring' into Chinese

What is the purpose of the change
Translate the 'History Server' page of 'Debugging & Monitoring' into Chinese
Brief change log
Translate the 'History Server' page of 'Debugging & Monitoring' into Chinese

The page url is https://ci.apache.org/projects/flink/flink-docs-release-1.11/zh/monitoring/historyserver.html
The markdown file is located in flink/docs/monitoring/historyserver.zh.md

Verifying this change
This change is a pure documentation translation.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? no

Hi @klion26
Could you help me to review this PR if you have free time?
Thank you.

@klion26
Thank you very much for your guidance and advices. I made some adjustments according to your suggestions.
If you have time, please have a look. Looking forward to your further suggestions.

Merging...


fix(core/select): clear button

Pull request checklist
Please check if your PR fulfills the following requirements:

[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been reviewed and added / updated if needed (for bug fixes / features)
[x] Build (yarn build) was run locally and any changes were pushed
[ ] Unit tests (yarn test) were run locally and passed
[x] Visual Regression Tests (yarn visual-regression) were run locally and passed
[ ] Linting (npm lint) was run locally and passed

Pull request type

Please check the type of change your PR introduces:

[x] Bug fix
[ ] Feature
[ ] Refactoring (no functional changes, no API changes)
[ ] Build related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

GitHub Issue Number: #456
What is the new behavior?

Clear button now works for Single select and Multiselect
Adjusted the clear button size as designed

Does this introduce a breaking change?

[ ] Yes
[x] No

Testing

Other information

Why does it not show up in VRT?

None of the existing examples had the property to use the clear button.
Added a vrt to test the clear button.


Refactoring and Support New Methods and Documents (V0.2.7)

Code refactoring:

Refactor openmixup.core (instead of openmixup.hooks) and openmixup.models.augments (contains mixup augmentation methods which are originally implemented in openmixup.models.utils). After code refactoring, the macro design of OpenMixup is similar to most projects of MMLab.
Support deployment of ONNX and TorchScript in openmixup.core.export and tools/deployment. We refactored the abstract class BaseModel (implemented in openmixup/models/classifiers/base_model.py) to support forward_inference (for custom inference and visualization). We also refactored openmixup.models.heads and openmixup.models.losses to support forward_inference. You can deploy the classification models in OpenMixup according to deployment tutorials.
Support testing API methods in openmixup/apis/test.py for evaluation and deployment of classification models.
Refactor openmixup.core.optimizers to separate optimizers and builders and support the latest Adan optimizer.

Supporting new features:

Support detailed usage instructions in README of config files for image classification methods in configs/classification, e.g., mixups on ImageNet. We will update READMEs of other methods in configs/selfsup and configs/semisup in a few weeks.
Refine the origianzation of README files according to README-Template.
Support the new mixup augmentation method (AlignMix) and provide the relevant config files in various datasets.

Updating documents:

Update documents of mixup benchmarks on ImageNet in Model_Zoo_sup.md. Update config files for supported mixup methods.
Update formats (figures, introductions and content tables) of awesome lists in Awesome Mixups and Awesome MIM and provide the latest methods.

OpenMixup V0.2.7 is now released!
We will fix reported bugs and update documents of V0.2.7 if new issues are raised.