Graphql schema limitations for using amplify codegen

Which Category is your question related to?
Amplify Codegen
Amplify CLI Version
4.29.3
What AWS Services are you utilizing?
AppSync
Provide additional details e.g. code snippets
I have GraphQL schema which is deployed to AWS AppSync, and trying to use amplify codegen to generate static types for it.
Experimentally it was found out that codegen fails if:

Types implementing the same interface have properties of different type. F.e. something like this

interface Object {
  id: String
}
type Child1 implements Object {
  id: String
  location: Location
}
type Child2 implements Object {
  id: String
  location: String
}

Will produce Fields "location" conflict because they return conflicting types String and Location. Use different aliases on the fields to fetch both if this was intentional.

Subqueries for types with required arguments also throw errors. F.e.:

type Child2 implements Object {
  id: String
  location(city: String!): String
}

Will produce Field "location" argument "city" of type "String!" is required, but it was not provided.
So I was wondering if there are some limitations/rules which amplify codegen imposes on graphql schema, or there's some configuration that can be used to resolve these problems.

Hi I cannot reproduce the issue you mentioned. Did you try the latest version of CLI? If the issue still remains in codegen, could you provide your codegen target languange?(can be found in .graphqlconfig.yml, codegenTarget field)

Hi I cannot reproduce the issue you mentioned. Did you try the latest version of CLI? If the issue still remains in codegen, could you provide your codegen target languange?(can be found in .graphqlconfig.yml, codegenTarget field)


[BE] fix CouponDetailsAPI url regexp

The problem with 404 error was caused because of an invalid regexp

Coverage remained the same at 90.031% when pulling 36a0835a6df250131a7e783d9739f65331dec85c on feature/incorrect-error into 4bf1c1d34b3f58e511cecb01a440708cce0324d6 on feature/coupons.

Coverage remained the same at 90.031% when pulling 36a0835a6df250131a7e783d9739f65331dec85c on feature/incorrect-error into 4bf1c1d34b3f58e511cecb01a440708cce0324d6 on feature/coupons.


Game crash

don't know why, but for some reason there was a crash due to ClearDespawn.
Fabric Loader - 0.12.12
MC Version - 1.18.1
Crash log - https://crashy.net/JsGL5RAcK2Bwr3utGta6

Can you try deleting the config file for this mod and trying again? I think the config file is written wrong.

Hey any updates?


docs: Add nextjs related

📝 Description
The navbar.link caused the page refresh on Next.js, but we do not see this chapter related with Next.js Link in the docs. It spent me a lot of time to find what cause the page refresh.
⛳ Current behavior (updates)
Add the navbar.link Next.js related doc.
🚀 New behaviorNo
Add the navbar.link Next.js related doc.
💣 Is this a breaking change (Yes/No):
No
📝 Additional Information

The code showcase needs some help.
Just use the Next/Link to wrapper the <Navbar.Link /> Component.


FileUtils Transaction  problem

as shown above, in some cases, There is a Transaction Problem(randomAccessFile.writeBytes(System.lineSeparator()) was called,but randomAccessFile.writeBytes(value) did not.  when offset file ends with '\n', FileUtils.readLastLine  will occur a NPE

Coverage decreased (-0.001%) to 24.554% when pulling 9b65be8f500df4374d5060cc82548e078d24e2f3 on IanCao:5.x into 93aeb4970da3f6f7492b0ade69455ba296466c15 on apache:5.x.


failed to set X-Kong-Upstream-Status header while sending to client

Is there an existing issue for this?

[X] I have searched the existing issues

Kong version ($ kong version)
2.8.0
Current Behavior
from kong log, below error can be seen:
2024/04/18 16:35:00 [error] 28#0: *28981717 [lua] handler.lua:1524: before(): failed to set X-Kong-Upstream-Status header while sending to client, client: xxx, server: kong, request: "GET /profiles/list HTTP/1.1", host: "xxx"
and all requests will fail with status 403
Expected Behavior
should have no such error
Steps To Reproduce
deploy kong 2.8.0 and ingress controller 2.3.1
register some upstream services, and routes
keep calling those registered routes in a long running job
Anything else?
No response

Could you upgrade kong to latest 3.6.x and try again? We always suggest our user to use the latest version.


feat(host): Remove MAX_RETRIES

Overview
Removes MAX_RETRIES in kona-host's prefetcher. The op-program just absolutely sends it, and waits for the RPC
it depends on to come back online.

📚 $\text{Stack Overview}$
Pulls submitted in this stack:

#739 👈
main

This comment was automatically generated by st.


[Onboarding] Tests etc

disabled tests around 'logged out state' since we don't do that anymore
there are some failing tests that weren't easy to figure out why (lots of ones that seemed unstubbed?) -> commented those out for now.

          2 Errors

  :no_entry_sign:
  No CHANGELOG changes made

  :no_entry_sign:
  CHANGELOG isn’t valid YAML

          2 Warnings

  :warning:
  Needs testing on a Phone if change is non-trivial

  :warning:
  Detected some Swift building time outliers

Time | Class | Function |
| --- | ----- | ----- |321.0ms | AuctionViewController.swift | @objc dynamic func showRefineTappedAnimated(animated: Bool)
274.5ms | SwiftyJSON.swift | get {}
205.0ms | WebSocket.swift | private func processRawMessage(buffer: UnsafePointer, bufferLen: Int)
102.0ms | AuctionLotMetadataStackScrollView.swift | required init(viewModel: LiveAuctionLotViewModelType, salesPerson: LiveAuctionsSalesPersonType, sideMargin: String)
95.8ms | LiveAuctionLotViewController.swift | @objc override func viewDidLoad()
90.5ms | Observable.swift | private final func nextTokenHash() -> Int
79.4ms | UIViewController+BlurredStatusView.swift | func ar_presentBlurredOverlayWithTitle(title: String, subtitle: String, buttonState: BlurredStatusOverlayViewCloseButtonState = default)
78.3ms | RefinementOptionsViewController+Private.swift | func stackView() -> ORStackView
77.0ms | LiveAuctionsAdminViewController.swift | @objc override func viewDidLoad()
76.9ms | UIViewController+BlurredStatusView.swift | (closure)
62.8ms | LiveAuctionLotViewModel.swift | @objc func addEvents(newEvents: [LiveEvent])

  Generated by :no_entry_sign: danger


Allow signed addKey/addService return type for later submission by other agent

Is your feature request related to a problem? Please describe.
We need to invoke the below methods (using ethr-did-provider) which result in writes to the network:

didManagerAddKey (specifically X25519 and Ed25519 key types); and,
didManagerAddService (e.g. did-comm endpoint).
As these calls invoke eth_sendTransaction which are listed as BLOCKED_RPC_METHODS within MetaMask Snaps (more information here) the calls to these methods fail (see bottom of issue for error example).

Describe the solution you'd like
I am open to suggestions on how best to approach this issue. One approach would be to allow an option/flag that would not submit the transaction to the network, but rather return the prepared (signed) transaction for submission in another context (i.e. via a Veramo agent instantiated in the DApp invoking). The addKey method looks like the below (did-ethr):
  async addKey(
    { identifier, key, options }: { identifier: IIdentifier; key: IKey; options?: TransactionOptions },
    context: IRequiredContext,
  ): Promise<any> {
    const ethrDid = await this.getEthrDidController(identifier, context)
    const usg = key.type === 'X25519' ? 'enc' : 'veriKey'
    const encoding = key.type === 'X25519' ? 'base58' : options?.encoding || 'hex'
    const attrName = `did/pub/${key.type}/${usg}/${encoding}`
    const attrValue = '0x' + key.publicKeyHex
    const ttl = options?.ttl || this.ttl || 86400
    const gasLimit = options?.gasLimit || this.gas || DEFAULT_GAS_LIMIT
    if (options?.metaIdentifierKeyId) {
      const metaHash = await ethrDid.createSetAttributeHash(attrName, attrValue, ttl)
      const canonicalSignature = await EthrDIDProvider.createMetaSignature(context, identifier, metaHash)

      const metaEthrDid = await this.getEthrDidController(identifier, context, options.metaIdentifierKeyId!)
      debug('ethrDid.addKeySigned %o', { attrName, attrValue, ttl, gasLimit })
      delete options.metaIdentifierKeyId
      const txHash = await metaEthrDid.setAttributeSigned(
        attrName,
        attrValue,
        ttl,
        { sigV: canonicalSignature.v, sigR: canonicalSignature.r, sigS: canonicalSignature.s },
        {
          ...options,
          gasLimit,
        },
      )
      debug(`ethrDid.addKeySigned tx = ${txHash}`)
      return txHash
    } else {
      debug('ethrDid.setAttribute %o', { attrName, attrValue, ttl, gasLimit })
      const txHash = await ethrDid.setAttribute(attrName, attrValue, ttl, undefined, {
        ...options,
        gasLimit,
      })
      debug(`ethrDid.addKey tx = ${txHash}`)
      return txHash
    }
  }

The returned (prepared and signed but not submitted) return type would look something like the below:
return {
  attrName,
  attrValue,
  ttl,
  gasLimit,
  { sigV: canonicalSignature.v, sigR: canonicalSignature.r, sigS: canonicalSignature.s },
  options,
}

Describe alternatives you've considered
The approach that I have outlined above would necessitate the implementation of a new function (e.g. submitAddKey) so I am keen to hear other approaches.
Additional context
Any method that invokes the eth_sendTransaction is going to result in this issue with errors corresponding broadly to the below example:
Error#1: could not coalesce error (error={ "code": -32601, "data": { "method": "eth_sendTransaction" }, "message": "The method does not exist / is not available." }, payload={ "id": 7, "jsonrpc": "2.0", "method": "eth_sendTransaction", "params": [ { "data": "0x7ad4b0a4000000000000000000000000ab0207f2084bafd9814fd0cf9a3736adf03f117b6469642f7075622f456432353531392f766572694b65792f686578000000000000000000000000000000000000000000000000000000000000000000000000800000000000000000000000000000000000000000000000000000000000015180000000000000000000000000000000000000000000000000000000000000002007cf5bc93cc16d1559ac91b05e444bc59b554cb39d9f7cc6d1f7f2c6ddde2bd9", "from": "0xab0207f2084bafd9814fd0cf9a3736adf03f117b", "gas": "0x186a0", "to": "0x03d5003bf0e79c5f5223588f347eba39afbc3818" } ] }, code=UNKNOWN_ERROR, version=6.10.0)

Building on the above, the amended addKey method would take an option (on the existing options object of signOnly: boolean. If set to true, then the didManagerAddKey (via the provider addKey method would return an object that looks like the following TypeScript tuple type (which conforms to the function parameters of metaEthrDid.setAttributeSigned):
type TxnParams = [
    attrName: string,
    attrValue: string,
    ttl: number,
    signature: { sigV: number; sigR: string; sigS: string },
    options: Record<string, any>,
];

In the introduced didManagerSubmitTransaction the function signature would then look like the following:
async didManagerSubmitTransaction(args: TxnParams, context: IAgentContext<IKeyManager>): Promise<unknown>;

This method (didManagerSubmitTransaction) can then be called from another context (in an agent not constrained by the snap environment) referencing the options.metaIdentifierKeyId for submission to the network.
async didManagerSubmitTransaction(args: TxnParams, context: IAgentContext<IKeyManager>): Promise<any> {
  // this method invokes a new provider method, which means that the calling agent calls `metaEthrDid.setAttributeSigned`
  // with the provided txnParams (that were returned from the agent call in the first invocation within the MM Snap.
  return provider.submitTransaction(args, context); // additional function made available on `did-ethr-provider`
}

Providing an update for the sake of completeness. In our case, the concern was that the eth_sendRawTransaction method was not supported within the context of the MetaMask Snap, and therefore, a workaround (such as signing the transaction and submitting via a proxy such as Infura, or on a server) was proposed.
However, the eth_sendRawTransaction method has since been unblocked by the MetaMask Snaps developers (refer to this PR), and as such, the above, while still relevant in some scenarios no longer applies in the case of MetaMask Snaps.


dagger : run unit tests

Find is too recursive, maybe we should consider -maxdepth arg, we can find files such as ./core/tests/visual_studio/packages/zlib-vc140-static-<IP_ADDRESS>/lib/native/include/inffixed.h


fix: props not defined in pageLayout component

Remove props which are not defined in the pageLayout component
Description:
For the frontend, yarn run build failed with error closeOrphanNodeDeletionModal, showOrphanNodeDeletionModal, openOrphanNodeDeletionModal not declared in the component PageLayout

Hi @CpEtoile all the type fixes are already pushed to DEV we will merge it to main very soon

Thanks for your reply.


⚠ Bofrost.de has degraded performance

In 9a2796a, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 880 ms

Resolved: Bofrost.de performance has improved in 139480c after 10 minutes.


Cause of artifacts at certain zoom levels

When I zoom the tileset, I see artifacts at various points, little lines at what I think are the edges of the tiles:

I'm guessing this has to do with rounding errors when the scale is set to particular floats?
How would one go about fixing this?

If you are using non-integer scales you can get rounding errors. Not sure there is a good fix for this in WebGL1 unfortunately. Depending on what scale you're wanting to use you may be able to round or clamp the coordinates in the shader, but I'm not sure there is a general fix for all scales.

I'm actually using webgpu, and looking at existing tile renderers to try to find a solution (since webl material is so prevalent) but I think the same concepts apply across both graphics apis.

WGSL has integers, most of the floats in the gl-tiled shaders should be integers (which don't exist in  GLSL ES 1.0). Switching those over would likely help.

hmm, that's interesting. Is there any way of determining what scales would be problematic?

Not sure, I haven't thought about this problem space is ~3 years?

WGSL has integers, most of the floats in the gl-tiled shaders

True, though most of the floats are related to texture sampling, and those have to be floats I assume, right?

Not sure, I haven't thought about this problem space in ~3 years?

Oh...well, sorry.  Definitely not trying to pester you.  Just figured this might be relevant to both of our repos.

Not a pestering for sure, just unfortunately I don't think I have good answers for you :)


rebar3 always recompile modules put in a subdirectory of appdir/src/

This might be unusual but I put a bunch of modules inside appdir/src/db/ and those modules will have updated modtime after every compile despite no source changes to them.

@leoliu can you describe the situation a little better.  A tree representation of your project file structure would be great and I assume you mean the mod times of the beam files is what changes?

Oh! Never mind, I reproduced. It is the fact they are under src/db that causes it. Ok, this is an issue with rebar_erlc_compiler's function for checking if a source file has changed.
I bet it is checking the file src/x.erl for ebin/x.beam even if was actually from src/db/x.erl.
Will work on a fix.

@tsloughter, yes that is the issue I am seeing.

@leoliu did you by chance move the source files from src/ to src/db after having compiled them already? I think I discovered that to be the issue.

As in, if you removed _build/default/lib/appdir/.rebar3/erlcinfo the problem would go away. But obviously it would be best if the compiler was smart enough to notice this change, so I'm still looking into that.

Hi @tsloughter, Sorry for my delay. Yes I think that is what happened.

Great, it should be fixed now.


Publish of WebApplication results in no output due to lack of WebBuildTools workload

https://github.com/Microsoft/dotnet-framework-docker/blob/7636ebaba31bf2cc299fa1ebba0999a18f5cbbc6/4.7.1-windowsservercore-1709/build/Dockerfile#L20
Currently cannot successfully publish a ASP.NET MVC 4 Web Application probably due to missing Web Development Build Tools.
Got it working by creating my own (slightly modified) build-tools-container
# Use the latest Windows Server Core image.
FROM microsoft/windowsservercore

# Download useful tools to C:\Bin.
ADD https://dist.nuget.org/win-x86-commandline/v4.1.0/nuget.exe C:\\Bin\\nuget.exe

# Download the Build Tools bootstrapper outside of the PATH.
ADD https://aka.ms/vs/15/release/vs_buildtools.exe C:\\TEMP\\vs_buildtools.exe

# Add C:\Bin to PATH and install Build Tools excluding workloads and components with known issues.
RUN setx /m PATH "%PATH%;C:\Bin"
RUN C:\TEMP\vs_buildtools.exe --quiet --wait --norestart --nocache --installPath C:\BuildTool \
    --add Microsoft.VisualStudio.Workload.WebBuildTools;includeRecommended \
    --add Microsoft.VisualStudio.Workload.MSBuildTools \
#    --remove Microsoft.VisualStudio.Component.Windows10SDK.10240 \
#    --remove Microsoft.VisualStudio.Component.Windows10SDK.10586 \
#    --remove Microsoft.VisualStudio.Component.Windows10SDK.14393 \
#    --remove Microsoft.VisualStudio.Component.Windows81SDK \
 || IF "%ERRORLEVEL%"=="3010" EXIT 0

ADD https://download.microsoft.com/download/9/0/1/901B684B-659E-4CBD-BEC8-B3F06967C2E7/NDP471-DevPack-ENU.exe C:\\TEMP\\NDP471-DevPack-ENU.exe
RUN C:\TEMP\NDP471-DevPack-ENU.exe /install /quiet /norestart

# SHELL ["C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe", "-command"]

# Start developer command prompt with any other commands specified.
# ENTRYPOINT C:\BuildTools\Common7\Tools\VsDevCmd.bat &&

# Default to PowerShell if no other command specified.
CMD ["powershell.exe", "-NoLogo", "-ExecutionPolicy", "Bypass"]

@riezebosch - do you need all of WebBuildTools or are the web targets and tasks sufficient?  I have been pondering this in a WIP PR.  Curious if the proposed Dockerfile would work for your scenario?

It seems working, a get the same results!

Glad to hear it works - I'm going to close because this work is being tracked under #103.


chore: rewrite lakefile.lean

Reorganize lakefile.lean, and remove unnecessary stuff.

@semorrison It appears the some of the Archive lean files are making use of autoImplicit which is why the build is failing.

This conflicts with #8461, so
bors d+

bors merge


[Bug]: Error: Not connected - is it possible to make connection timeout configurable?

What happened?
Please compare these two outputs from<EMAIL_ADDRESS>Build step done in 716.21s
    Error: Not connected
Error: Process completed with exit code 1.

and
Build step done in 31.32s
- Copying files: Calculating...
✔ Copied 4 files to remote machine
Running: docker compose up -d --remove-orphans --no-build
- Connecting to remote docker socket...
✔ Connected to remote docker socket

The first one is a cold run of my workflow where the build stage took >10m, and succeeded, but artifacts copying failed right after it. The second output is from the same workflow retried - it reused cached images and thus build phase finished in 30s and preevy-up-action didn't have any problems with copying the artifacts to the runner.
I suspect that there is a timeout (ssh connection timeout) that approximately equals 10m somewhere. Wondering if it's possible to make it configurable for docker-compose stacks that require a longer build phase?
Add screenshots
please see the previous section with error details
Steps to reproduce the behavior
My setup:

public tunnel server
deploy runtime: AWS Lightsail
a docker-compose with a service which build:context step takes a long time (10m)
GHA with<EMAIL_ADDRESS>that uses GH builder and GHCR

GHA:
...
      - name: Set up Docker Buildx
        id: buildx_setup
        uses: docker/setup-buildx-action@v3
        with:
          buildkitd-config: .github/buildkitd.toml

      - name: Login to GitHub Container Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ github.token }}

      - uses<EMAIL_ADDRESS>        id: preevy
        with:
          install: gh-release
          profile-url: "${{ vars.PREEVY_PROFILE_URL }}"
          args: --registry ghcr.io/my-org --builder ${{ steps.buildx_setup.outputs.name }}
          docker-compose-yaml-paths: "./docker-compose.yml"
        env:
          GITHUB_TOKEN: ${{ github.token }}

Expected behavior
Avoid Error: Not connected error when the build step takes a long time, i.e. either make the timeout configurable or retry the connection just a few times.
What OS are you seeing the problem on?
Linux
Additional context
No response
Record

[X] I agree to follow this project's Code of Conduct

hmm I noticed the same issue with builds that took "only" ~7 minutes:

Hey @akramarev, can you post some of the logs before the error? Preferably add --debug. I had builds longer than 10m and they did not time out. Usually during the build the ssh connection is active (messages are being sent) and there is no reason for it to time out.
What I did see in the past is builds that overload the machine's resources to the point where the ssh server hangs. You can try running top while the machine is building (connect using preevy ssh). You can also try a larger instance type or offloading the build to the GH action runner. LMK.

Thanks for the reply @royra. I observe this problem only when I offload the build to the GHA runner (please check "My setup" section above for details), if I use the default builder (build happens on the remove Lightsail instance) I don't have this issue.
So my suspicion is while GHA runner is building the image preevy doesn't actively use the opened earlier SSH connection and it's timing out by the moment GHA finishes the build and is ready to upload artifacts.
Attaching logs (build details in the middle manually reducted):
preevy-logs.txt

Sorry I missed the fact that you're already offloading the build.
Can you look at the ssh server logs on the lightsail instance? If there's nothing interesting try changing the LogLevel setting at /etc/ssh/sshd_config

Thank you for your reply @royra and livecycle team for keeping this github issue open.
Noticed in /var/log/auth.log that SSH server restarted right at the moment when preevy action reported that it successfully configured the new lightsail machine:

At the same time /var/log/syslog indicates that cloud-init is the process that restarted sshd.
Sep  7 00:41:01 ip-172-26-8-210 systemd[1]: Starting Execute cloud user/final scripts...
Sep  7 00:41:01 ip-172-26-8-210 dockerd[1011]: time="2024-09-07T00:41:01.470172700Z" level=info msg="API listen on /run/docker.sock"
Sep  7 00:41:01 ip-172-26-8-210 systemd[1]: Starting Update UTMP about System Runlevel Changes...
Sep  7 00:41:01 ip-172-26-8-210 systemd[1]: systemd-update-utmp-runlevel.service: Succeeded.
Sep  7 00:41:01 ip-172-26-8-210 systemd[1]: Finished Update UTMP about System Runlevel Changes.
Sep  7 00:41:01 ip-172-26-8-210 cloud-init[2353]: Lightsail: Starting Instance Initialization.
Sep  7 00:41:01 ip-172-26-8-210 cloud-init[2353]: Lightsail: SSH CA Public Key created.
Sep  7 00:41:01 ip-172-26-8-210 cloud-init[2353]: Lightsail: SSH CA Public Key registered.
Sep  7 00:41:01 ip-172-26-8-210 systemd[1]: Stopping OpenBSD Secure Shell server...
Sep  7 00:41:01 ip-172-26-8-210 systemd[1]: ssh.service: Succeeded.
Sep  7 00:41:01 ip-172-26-8-210 systemd[1]: Stopped OpenBSD Secure Shell server.
Sep  7 00:41:01 ip-172-26-8-210 systemd[1]: Starting OpenBSD Secure Shell server...
Sep  7 00:41:01 ip-172-26-8-210 systemd[1]: Started OpenBSD Secure Shell server.
Sep  7 00:41:01 ip-172-26-8-210 cloud-init[2353]: Lightsail: sshd restarted.
Sep  7 00:41:02 ip-172-26-8-210 cloud-init: #############################################################
Sep  7 00:41:02 ip-172-26-8-210 cloud-init: -----BEGIN SSH HOST KEY FINGERPRINTS-----
-- {redacted}
Sep  7 00:41:02 ip-172-26-8-210 cloud-init: -----END SSH HOST KEY FINGERPRINTS-----
Sep  7 00:41:02 ip-172-26-8-210 cloud-init: #############################################################
Sep  7 00:41:02 ip-172-26-8-210 cloud-init[2353]: Cloud-init v. 23.3.3-0ubuntu0~20.04.1 running 'modules:final' at Sat, 07 Sep 2024 00:41:01 +0000. Up 72.92 seconds.
Sep  7 00:41:02 ip-172-26-8-210 cloud-init[2353]: Cloud-init v. 23.3.3-0ubuntu0~20.04.1 finished at Sat, 07 Sep 2024 00:41:02 +0000. Datasource DataSourceEc2Local.  Up 73.33 seconds
...

As expected any further attempts to restart the job:

doesn't lead to sshd restart
succeeded, i.e. successfully copied context to the lightsail instance and runs compose

Is there anything you can suggest in this situation?


Spring Security OAuth2 social

I have a question about setting up Facebook yml.
The link I refer to is https://spring.io/guides/tutorials/spring-boot-oauth2/.
If you set yml as below, the default profile information will not be passed.
According to the Facebook Document, https://developers.facebook.com/docs/facebook-login/permissions#reference-default
The following fields are provided as standard.

id
first_name
last_name
middle_name
name
name_format
picture
short_name

Why is not the above information coming over?
I solved it by setting yml as below.
facebook:
  client:
    clientId: <>
    clientSecret: <>
    accessTokenUri: https://graph.facebook.com/oauth/access_token
    userAuthorizationUri: https://www.facebook.com/dialog/oauth
    tokenName: oauth_token
    authenticationScheme: query
    clientAuthenticationScheme: form
    scope:
        - email
  resource:
    userInfoUri: https://graph.facebook.com/me?fields=id, name, email, picture

In the case of Google, basic information is passed.
But Facebook does not.
Sorry, I am not very good at english
I am always thanking Spring Open Source.

@cheese10yun I'm not sure why Facebook does not send back the other fields as default. Each provider is different so you really need to read their API docs to see how to correctly configure the client. You are correct though, with Facebook you need to provide the list of fields that you want returned in userInfoUri.
Have you tried the new OAuth 2.0 / OpenID Connect support we've been building since Spring Security 5.0? Here is a link to the sample for Facebook.
I would highly encourage you to try Spring Security 5.x OAuth support as it provides a fairly complete feature list for Client and we're introducing Resource Server support in 5.1, which is scheduled for this Friday.
I'm going to close this issue as answered. If you feel you need more answer(s), we can re-open this.

@jgrandja Thank you for your comment.
Spring Security 5.1 version is expected. I always appreciate Spring Open Source.


feat: refactor i2gw codes and add new features

What type of PR is this?

/kind feature
What this PR does / why we need it:

[ ] refactor code base to have a better extensible translator code structure.
[ ] init basic testing abilities:

[ ] unit test
[ ] coverage test
[ ] e2e test ( TODO )

[ ] add support for version output in cmd
[ ] add support for different resource mode: local and remote, to support translating Ingress from files besides from the clusters.
[ ] add support for filtering different GWAPI Resources in output
[ ] add support for different output type: json and yaml.

Sub of #9
Which issue(s) this PR fixes:
xref: #21
Does this PR introduce a user-facing change?:

xref: https://github.com/kubernetes-sigs/ingress2gateway/issues/21

@Xunzhuo let us know if you want to follow up on these, for now I'm going to close out in favor of more recent PRs.


Remove the bottom bar?

Hello!
I dig through the VideoView but could not find a way to remove the bottom bar/MediaController (the one with the play/stop button and seekbar).
There is any way to do that I am missing?

all media controller's ui handle in DefaultMediaController.java, have a look at this file,if you want remove all the contronller see https://github.com/tcking/GiraffePlayer2/issues/9

There should be an option to hide controls

it will create a MediaController at runtime without fork the lib


Realm: unknown crashing when running pygion examples under jupyter notebook.

When running some pygion examples such as index_launcher, partitions under jupyter notebook, the ipython kernel sometimes crashes.
I have seen the crash on CI test and my Macbook, but not on sapling.
Here is the stack when crashing:
Legion process received signal 11: Segmentation fault
Process 165 on node 85d4af47dd1f is frozen!

#1  0x00007f58b9c5355a in __sleep (seconds=0) at ../sysdeps/posix/sleep.c:55
#2  0x00007f58baa90921 in Realm::realm_freeze (signal=11) at /repo/runtime/realm/runtime_impl.cc:179
#3  <signal handler called>
#4  0x00007f57abad2cc7 in _PyEval_EvalFrameDefault () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#5  0x00007f57abad0a0f in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#6  0x00007f57abad10fc in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#7  0x00007f57abad7303 in _PyEval_EvalFrameDefault () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#8  0x00007f57abacf803 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#9  0x00007f57abad12be in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#10 0x00007f57abad7303 in _PyEval_EvalFrameDefault () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#11 0x00007f57abacf803 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#12 0x00007f57abad12be in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#13 0x00007f57abad7303 in _PyEval_EvalFrameDefault () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#14 0x00007f57abacf803 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#15 0x00007f57abad12be in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#16 0x00007f57abad7303 in _PyEval_EvalFrameDefault () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#17 0x00007f57abacf803 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#18 0x00007f57abad0e25 in _PyFunction_FastCallDict () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#19 0x00007f57abbbbfd1 in _PyObject_FastCallDict () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#20 0x00007f57abbbc6fd in _PyObject_Call_Prepend () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#21 0x00007f57abbbc768 in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#22 0x00007f57abad6225 in _PyEval_EvalFrameDefault () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#23 0x00007f57abacf803 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#24 0x00007f57abad12be in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#25 0x00007f57abad7303 in _PyEval_EvalFrameDefault () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#26 0x00007f57abacf803 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#27 0x00007f57abad12be in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#28 0x00007f57abad7303 in _PyEval_EvalFrameDefault () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#29 0x00007f57abacf803 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#30 0x00007f57abad0e25 in _PyFunction_FastCallDict () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#31 0x00007f57abbbbfd1 in _PyObject_FastCallDict () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#32 0x00007f57abbbc6fd in _PyObject_Call_Prepend () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#33 0x00007f57abbbc768 in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#34 0x00007f57aba41492 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#35 0x00007f57aba871d7 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#36 0x00007f58b95326db in start_thread (arg=0x7f579ee2d700) at pthread_create.c:463
#37 0x00007f58b9c9061f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Here is the top result, legion_python is the child process of jupyter-run
    1 root      20   0   18516   2924   2840 S   0.0  0.0   0:00.07 bash
   16 root      20   0    4636    792    792 S   0.0  0.0   0:00.04 sh
   22 root      20   0   18516   3332   2968 S   0.0  0.0   0:00.03 bash
  100 root      20   0    4636    712    704 S   0.0  0.0   0:00.05 sh
  106 root      20   0   18516   3448   2992 S   0.0  0.0   0:00.02 bash
  158 root      20   0  457192  49564  12860 S   0.0  0.6   0:01.48 jupyter-run
  165 root      20   0 6533524 165904  59368 S   0.0  2.0   0:10.00 legion_python
  226 root      20   0   38724   3104   2648 R   0.0  0.0   0:00.00 top

I have not figured out what causes the crash, so I created a docker image reproducer:

docker pull registry.gitlab.com/eddy16112/legion/jupyter-crash:latest (if you have not used gitlab's registry before, you will need to run docker login registry.gitlab.com -u  -p , please refer to https://docs.gitlab.com/ee/user/packages/container_registry/)
start a container
once inside the container, type
a. bash (optional only if you are not in bash)
b. cd /repo/jupyter_notebook/
c export LD_LIBRARY_PATH=/repo/bindings/python:$LD_LIBRARY_PATH
export PYTHONPATH=/repo/bindings/python:$PYTHONPATH
export LEGION_FREEZE_ON_ERROR=1
d. jupyter run --kernel legion_kernel_nocr ./ci_test.py
Then you should be able to catch the crashing.

@eddy16112 There are threads here that Realm did not make that are calling into the Python interpreter. We need to find out where they are coming from. For example, the thread that segfaulted started like this:
#33 0x00007f34bd346768 in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#34 0x00007f34bd1cb492 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#35 0x00007f34bd2111d7 in ?? () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#36 0x00007f35be83d6db in start_thread (arg=0x7f349ffff700) at pthread_create.c:463
#37 0x00007f35bef9b61f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Realm Python threads on the other hand will look like this:
#137 0x00007f34bd346768 in PyObject_Call () from /usr/lib/x86_64-linux-gnu/libpython3.6m.so
#138 0x00007f35c022f77d in Realm::LocalPythonProcessor::execute_task (this=0x55a7fbdda600, func_id=10, task_args=...) at /repo/runtime/realm/python/python_module.cc:981
#139 0x00007f35bff30c41 in Realm::Task::execute_on_processor (this=0x55a7fbf566e0, p=...) at /repo/runtime/realm/tasks.cc:302
#140 0x00007f35bff34d62 in Realm::KernelThreadTaskScheduler::execute_task (this=0x55a7fbdda900, task=0x55a7fbf566e0) at /repo/runtime/realm/tasks.cc:1366
#141 0x00007f35c022d4e5 in Realm::PythonThreadTaskScheduler::execute_task (this=0x55a7fbdda900, task=0x55a7fbf566e0) at /repo/runtime/realm/python/python_module.cc:326
#142 0x00007f35bff33b0b in Realm::ThreadedTaskScheduler::scheduler_loop (this=0x55a7fbdda900) at /repo/runtime/realm/tasks.cc:1105
#143 0x00007f35c022d6b1 in Realm::PythonThreadTaskScheduler::python_scheduler_loop (this=0x55a7fbdda900) at /repo/runtime/realm/python/python_module.cc:395
#144 0x00007f35c02334fe in Realm::Thread::thread_entry_wrapper<Realm::PythonThreadTaskScheduler, &Realm::PythonThreadTaskScheduler::python_scheduler_loop> (obj=0x55a7fbdda900) at /repo/runtime/realm/threads.inl:97
#145 0x00007f35bff0a630 in Realm::KernelThread::pthread_entry (data=0x55a7fbf34950) at /repo/runtime/realm/threads.cc:774
#146 0x00007f35be83d6db in start_thread (arg=0x7f34afffe800) at pthread_create.c:463
#147 0x00007f35bef9b61f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Side note: you can actually skip logging into the container registry if you're just reading public containers. I started the container with docker run -it registry.gitlab.com/eddy16112/legion/jupyter-crash:latest /bin/bash without logging in.

How do I start the kernel with the -lg:delay option?

How do I start the kernel with the -lg:delay option?

You can add it to /root/.local/share/jupyter/kernels/legion_kernel_nocr/kernel.json

I've got something better. This is the Python backtrace for the thread that segfaulted:
(gdb) py-bt
Traceback (most recent call first):
  File "/usr/lib/python3.6/selectors.py", line 458, in select
    return ready
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1415, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/lib/python3.6/asyncio/base_events.py", line 438, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.6/dist-packages/tornado/platform/asyncio.py", line 199, in start
    self.asyncio_loop.run_forever()
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/iostream.py", line 76, in _thread_main
    self.io_loop.start()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
    self._bootstrap_inner()

The kernel is forking its own threads which is not going to work: with Realm, at least not in a way that is going to call back into the interpreter without Realm being in control.

It looks like there is a second thread that has just bootstrapped itself as well:
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/heartbeat.py", line 100, in run
    zmq.device(zmq.QUEUE, self.socket, self.socket)
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
    self._bootstrap_inner()

The realm thread in the Python interpreter is actually waiting on the GIL (partial backtrace since it is crashing):
(gdb) py-bt
Traceback (most recent call first):
 Waiting for the GIL
 File "/repo/bindings/python/pygion.py", line 414, in get
   c.legion_future_get_void_result(self.handle)
 File "/repo/bindings/python/pygion.py", line 2472, in execution_fence
   f.get()
 File "<ipython-input-1-b04a89c21988>", line 43, in <module>
 <built-in method exec of module object at remote 0x7fc5fb094638>
 File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3343, in run_code
   exec(code_obj, self.user_global_ns, self.user_ns)
 File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3263, in run_ast_nodes
   if (await self.run_code(code, result,  async_=asy)):
 File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3072, in run_cell_async
   interactivity=interactivity, compiler=compiler, result=result)
 <built-in method send of coroutine object at remote 0x7fc4dd123ca8>
 File "/usr/local/lib/python3.6/dist-packages/IPython/core/async_helpers.py", line 68, in _pseudo_sync_runner
   coro.send(None)
 File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2895, in _run_cell
   return runner(coro)
 File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2867, in run_cell
   raw_cell, store_history, silent, shell_futures)
 File "/usr/local/lib/python3.6/dist-packages/ipykernel/zmqshell.py", line 539, in run_cell

@streichler will need to remind us of the details, but I think Realm's Python interpreter manages the GIL in its own way and that is probably going to be problematic for the creation of threads by the kernel.

There are definitely two threads here actively running in the same interpreter, which is very bad since the GIL is supposed to prevent that.

The kernel is forking its own threads which is not going to work: with Realm, at least not in a way that is going to call back into the interpreter without Realm being in control.

Yes, the ipython base kernel uses asyncio module to do asynchronous progressing, so I will not be surprised if asyncio creates other threads.

Here are complete backtraces for all the Python threads:
Thread 7 (Realm Python processor):
(gdb) py-bt
Traceback (most recent call first):
  File "/repo/bindings/python/pygion.py", line 414, in get
    c.legion_future_get_void_result(self.handle)
  File "/repo/bindings/python/pygion.py", line 2472, in execution_fence
    f.get()
  File "<ipython-input-1-b04a89c21988>", line 49, in <module>
  <built-in method exec of module object at remote 0x7f8bd44ca638>
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3263, in run_ast_nodes
    if (await self.run_code(code, result,  async_=asy)):
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3072, in run_cell_async
    interactivity=interactivity, compiler=compiler, result=result)
  <built-in method send of coroutine object at remote 0x7f8ab42cd048>
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/async_helpers.py", line 68, in _pseudo_sync_runner
    coro.send(None)
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2895, in _run_cell
    return runner(coro)
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2867, in run_cell
    raw_cell, store_history, silent, shell_futures)
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/zmqshell.py", line 539, in run_cell
    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/ipkernel.py", line 302, in do_execute
    res = shell.run_cell(code, store_history=store_history, silent=silent)
  <built-in method next of module object at remote 0x7f8bd44ca638>
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 162, in _fake_ctx_run
    return f(*args, **kw)
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 234, in wrapper
    yielded = ctx_run(next, result)
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 541, in execute_request
    user_expressions, allow_stdin,
  <built-in method next of module object at remote 0x7f8bd44ca638>
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 162, in _fake_ctx_run
    return f(*args, **kw)
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 234, in wrapper
    yielded = ctx_run(next, result)
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 261, in dispatch_shell
    yield gen.maybe_future(handler(stream, idents, msg))
  <built-in method next of module object at remote 0x7f8bd44ca638>
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 162, in _fake_ctx_run
    return f(*args, **kw)
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 234, in wrapper
    yielded = ctx_run(next, result)
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelbase.py", line 361, in process_one
    yield gen.maybe_future(dispatch(*args))
  <built-in method send of generator object at remote 0x7f8ab655ce08>
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 775, in run
    yielded = self.gen.send(value)
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 162, in _fake_ctx_run
    return f(*args, **kw)
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 814, in inner
    self.ctx_run(self.run)
  File "/usr/local/lib/python3.6/dist-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/usr/local/lib/python3.6/dist-packages/tornado/ioloop.py", line 688, in <lambda>
    lambda f: self._run_callback(functools.partial(callback, future))
  File "/usr/lib/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1451, in _run_once
    handle._run()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 438, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.6/dist-packages/tornado/platform/asyncio.py", line 199, in start
    self.asyncio_loop.run_forever()
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/kernelapp.py", line 619, in start
    self.io_loop.start()
  File "/usr/local/lib/python3.6/dist-packages/traitlets/config/application.py", line 664, in launch_instance
    app.start()
  File "/root/.local/share/jupyter/kernels/legion_kernel_nocr/legion_kernel_nocr.py", line 79, in <module>
    IPKernelApp.launch_instance(kernel_class=LegionKernelNoCR)
  <built-in method exec of module object at remote 0x7f8bd44ca638>
  File "/repo/bindings/python/legion_top.py", line 222, in run_path
    exec(code, module.__dict__, module.__dict__)
--Type <RET> for more, q to quit, c to continue without paging--
  File "/repo/bindings/python/legion_top.py", line 398, in legion_python_main
    run_path(args[start], run_name='__main__')

Thread 10 (segfault):
(gdb) py-bt
Traceback (most recent call first):
  File "/usr/local/lib/python3.6/dist-packages/zmq/sugar/socket.py", line 729, in recv_multipart
    while self.getsockopt(zmq.RCVMORE):
  File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 596, in _handle_recv
    msg = self.socket.recv_multipart(zmq.NOBLOCK, copy=self._recv_copy)
  File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 577, in _handle_events
    self._handle_recv()
  File "/usr/local/lib/python3.6/dist-packages/zmq/eventloop/zmqstream.py", line 661, in <lambda>
    self.io_loop.add_callback(lambda: self._handle_events(self.socket, 0))
  File "/usr/local/lib/python3.6/dist-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/usr/lib/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1451, in _run_once
    handle._run()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 438, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.6/dist-packages/tornado/platform/asyncio.py", line 199, in start
    self.asyncio_loop.run_forever()
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/iostream.py", line 76, in _thread_main
    self.io_loop.start()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
    self._bootstrap_inner()

Thread 11
(gdb) py-bt
Traceback (most recent call first):
 File "/usr/local/lib/python3.6/dist-packages/ipykernel/heartbeat.py", line 100, in run
   zmq.device(zmq.QUEUE, self.socket, self.socket)
 File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
   self.run()
 File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
   self._bootstrap_inner()

Thread 14:
(gdb) py-bt
Traceback (most recent call first):
 <built-in method acquire of _thread.lock object at remote 0x7f8ab429daa8>
 File "/usr/lib/python3.6/threading.py", line 295, in wait
   waiter.acquire()
 File "/usr/lib/python3.6/threading.py", line 551, in wait
   signaled = self._cond.wait(timeout)
 File "/usr/local/lib/python3.6/dist-packages/IPython/core/history.py", line 829, in run
   self.history_manager.save_flag.wait()
 File "/usr/local/lib/python3.6/dist-packages/IPython/core/history.py", line 58, in needs_sqlite
   return f(self, *a, **kw)
 File "/usr/local/lib/python3.6/dist-packages/decorator.py", line 232, in fun
   return caller(func, *(extras + args), **kw)
 File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
   self.run()
 File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
   self._bootstrap_inner()

Thread 15
(gdb) py-bt
Traceback (most recent call first):
 Waiting for the GIL
 <built-in method sleep of module object at remote 0x7f8acdba6818>
 File "/usr/local/lib/python3.6/dist-packages/ipykernel/parentpoller.py", line 36, in run
   time.sleep(1.0)
 File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
   self.run()
 File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
   self._bootstrap_inner()

Thread 55 (Another Realm thread):
(gdb) py-bt
Traceback (most recent call first):
  Waiting for the GIL
  File "/repo/bindings/python/pygion.py", line 414, in get
    c.legion_future_get_void_result(self.handle)
  File "/repo/bindings/python/examples/future.py", line 73, in main
    print("value of i.get() is %s" % i.get())
  File "/repo/bindings/python/pygion.py", line 1814, in execute_task
    result = self.body(*args)
  File "/repo/bindings/python/pygion.py", line 1738, in __call__
    return self.execute_task(*args, **kwargs)

The suspicious threads are 10, 11, and 14. Threads 7 and 55 have both released the GIL as part of their CFFI call, and are waiting before getting it back. Thread 15 is also being good and waiting for the GIL. The others though all look like they are running concurrently which shouldn't be allowed.

I think thread 14 is safe. It is waiting to acquire a lock, so it is blocked. Threads 10 and 11 appear to be running concurrently.

Have you tried this on my more recent Python versions with updated packages?

@lightsighter What is your most recent python? I tried on python 3.8. It seems like the chance of crashing is decreased (at least sometimes all tests can be passed in docker, which is not possible on python3.6), but it still crashes.

The threads are part of Jupyter, are they not? Why would we expect this to be related to Python version?
I think we need to either (a) figure how to get Jupyter to stop using threads or (b) just figure out how to cope with them being there.

I think it is impossible to let jupyter not using threads, because the base class of all kernels https://github.com/ipython/ipykernel/blob/f03856014d5fdf86779a72b8d7f85f412f446696/ipykernel/kernelbase.py#L70 uses asyncio and zmq module, which have threads, but I could be wrong.

What is your most recent python? I tried on python 3.8. It seems like the chance of crashing is decreased (at least sometimes all tests can be passed in docker, which is not possible on python3.6), but it still crashes.

You built the container with 3.6 and that is pretty old at this point.

The threads are part of Jupyter, are they not? Why would we expect this to be related to Python version?

I looked at the code in the threading module of Python, and it's implementation changed dramatically from 3.7 to 3.8.

I think we need to either (a) figure how to get Jupyter to stop using threads or (b) just figure out how to cope with them being there.

Well, I'm still surprised as to how we get two threads into the interpreter at the same time. Realm is calling PyGILState_Ensure before running every task, so that should be grabbing the GIL:
https://gitlab.com/StanfordLegion/legion/-/blob/master/runtime/realm/python/python_module.cc#L319
Similarly, the threading module backtraces suggest that they are getting the GIL by calling PyEval_AcquireThread:
https://github.com/python/cpython/blob/3.7/Modules/_threadmodule.c#L992
Which should be acquiring the GIL, and yet it seems like there are two threads both going through that path and ending up in the interpreter, which I don't understand at all. Both the threads running concurrently are from the threading module and should both have called PyEval_AcquireThread. The Python GIL should be protecting us but its not.

This part is scary and I hope it is not happening:
Python supports the creation of additional interpreters (using [Py_NewInterpreter()](https://docs.python.org/3/c-api/init.html#c.Py_NewInterpreter)), but mixing multiple interpreters and the PyGILState_* API is unsupported.

https://docs.python.org/3/c-api/init.html#non-python-created-threads

I'm 99% sure that no one uses this feature, because Sean and I were looking at this (before we settled on submodules) and there is global state that this does not get rid of. As best I can tell this is an evolutionary dead end and will likely be made obsolete by the actual remove of the GIL in an (unscheduled, but probably not too distant) future release.

Just as a heads up, there are plans to remove the GIL in Python, so if we do choose to rely on this we'll need to track this work carefully because at some point the GIL may be removed: https://news.ycombinator.com/item?id=31348097

I think they can remove the GIL and it shouldn't impact this. The problem is not the application code, the problem is that we're segfaulting inside the interpreter, so removing the GIL would just make this style of execution legal. What we need to understand is how two threads made by the threading module both managed to think that they are holding the GIL at the same time.

I finally done with the python3.8 image, docker pull eddy16112/jupyter-crash:py38. Somehow, I can not push the image to gitlab, so I pushed it to the docker hub.
By the way, I also tried to un-define the USE_PYGILSTATE_CALLS, and let realm to use PyEval_RestoreThread and PyEval_SaveThread, but it did not help.

https://gitlab.com/StanfordLegion/legion/-/blob/master/runtime/realm/python/python_module.cc#L752
PyThreadState_Get needs to be called after GIL lock, but I do not see where the lock is acquired even though comments says "acquired the GIL on our behalf already"

I accidentally hit the following error:
Fatal Python error: PyEval_SaveThread: the function must be called with the GIL held, but the GIL is released (the current Python thread state is NULL)
Python runtime state: initialized

Thread 0x000000017644b000 (most recent call first):
  File "/Users/weiwu/legion_python/bindings/python/pygion.py", line 407 in get
  File "/Users/weiwu/legion_python/bindings/python/examples/index_launch.py", line 62 in main
  File "/Users/weiwu/legion_python/bindings/python/pygion.py", line 1814 in execute_task
  File "/Users/weiwu/legion_python/bindings/python/pygion.py", line 1738 in __call__

Thread 0x0000000175b3f000 (most recent call first):
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/site-packages/ipykernel/parentpoller.py", line 37 in run
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/threading.py", line 1009 in _bootstrap_inner
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/threading.py", line 966 in _bootstrap

Thread 0x0000000174aa7000 (most recent call first):
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/threading.py", line 320 in wait
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/threading.py", line 600 in wait
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/site-packages/IPython/core/history.py", line 854 in run
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/site-packages/IPython/core/history.py", line 60 in only_when_enabled
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/site-packages/decorator.py", line 232 in fun
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/threading.py", line 1009 in _bootstrap_inner
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/threading.py", line 966 in _bootstrap

Thread 0x0000000173a9b000 (most recent call first):
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/selectors.py", line 562 in select
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/asyncio/base_events.py", line 1860 in _run_once
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/asyncio/base_events.py", line 600 in run_forever
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/site-packages/tornado/platform/asyncio.py", line 199 in starTraceback (most recent call last):
  File "/Users/weiwu/opt/anaconda3/envs/py310/bin/jupyter-run", line 8, in <module>
    sys.exit(RunApp.launch_instance())
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/site-packages/jupyter_core/application.py", line 269, in launch_instance
    return super().launch_instance(argv=argv, **kwargs)
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/site-packages/traitlets/config/application.py", line 976, in launch_instance
    app.start()
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/site-packages/jupyter_client/runapp.py", line 107, in start
    reply = self.kernel_client.execute_interactive(code, timeout=OUTPUT_TIMEOUT)
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/site-packages/jupyter_client/utils.py", line 30, in wrapped
    raise e
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/site-packages/jupyter_client/utils.py", line 27, in wrapped
    return loop.run_until_complete(future)
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/site-packages/nest_asyncio.py", line 89, in run_until_complete
    return f.result()
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/Users/weiwu/opt/anaconda3/envs/py310/lib/python3.10/site-packages/jupyter_client/client.py", line 536, in _async_execute_interactive

so I added the some checks to the realm python module
https://gitlab.com/eddy16112/legion/-/commit/08ead612b159c56b8e714f4a918fd3173c957829
and I caught an error:
legion_python: /repo/runtime/realm/python/python_module.cc:604: virtual void Realm::PythonThreadTaskScheduler::thread_blocking(Realm::Thread*): Assertion `(pyproc->interpreter->api->PyGILState_Check)() == 1' failed.

It tells us when calling PyEval_SaveThread, the GIL is not acquired.
this line https://gitlab.com/StanfordLegion/legion/-/blob/master/runtime/realm/python/python_module.cc#L599
This is a non-deterministic error, I only caught it once, but if we forgot to acquire the GIL, we should hit the assertion all the time, which I do not understand.

https://gitlab.com/StanfordLegion/legion/-/blob/master/runtime/realm/python/python_module.cc#L752
PyThreadState_Get needs to be called after GIL lock, but I do not see where the lock is acquired even though comments says "acquired the GIL on our behalf already"

@streichler I think this might have worked on older versions of Python, but it looks like the semantics of the API have changed in later versions of Python. I'm pretty sure the comment is referring to this line here:
https://gitlab.com/StanfordLegion/legion/-/blob/master/runtime/realm/python/python_module.cc#L181
More recent Python versions though say that this method call is a no-op and does not guarantee that we are holding the GIL:
https://docs.python.org/3/c-api/init.html#c.PyEval_InitThreads
Do you agree that this is an actual bug or is there some other way that we end up getting the GIL?

This is a non-deterministic error, I only caught it once, but if we forgot to acquire the GIL, we should hit the assertion all the time, which I do not understand.

What happens in the cases when you don't hit it? Are we crashing in a different way. All signs are definitely pointing to some lack of GIL discipline somewhere.

I am able to reproduce the GIL assertion on linux with Python3.6. Here is the error message with backtrace. I caught a case that realm and non-realm threads both crash.
legion_python: /repo/runtime/realm/python/python_module.cc:604: virtual void Realm::PythonThreadTaskScheduler::thread_blocking(Realm::Thread*): Assertion `(pyproc->interpreter->api->PyGILState_Check)() == 1' failed.
Legion process received signal 11: Segmentation fault
Legion process received signal 6: Aborted
Process 741 on node 15a9fc1a8355 is frozen!
Process 741 on node 15a9fc1a8355 is frozen!

bt of thread 23 (realm thread)
#2  0x00007f6e4597d921 in Realm::realm_freeze (signal=6) at /repo/runtime/realm/runtime_impl.cc:179
#3  <signal handler called>
#4  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#5  0x00007f6e44a9c7f1 in __GI_abort () at abort.c:79
#6  0x00007f6e44a8c3fa in __assert_fail_base (fmt=0x7f6e44c136c0 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=assertion@entry=0x7f6e46431fc0 "(pyproc->interpreter->api->PyGILState_Check)() == 1",
    file=file@entry=0x7f6e46431e98 "/repo/runtime/realm/python/python_module.cc", line=line@entry=604,
    function=function@entry=0x7f6e46433600 <Realm::PythonThreadTaskScheduler::thread_blocking(Realm::Thread*)::__PRETTY_FUNCTION__> "virtual void Realm::PythonThreadTaskScheduler::thread_blocking(Realm::Thread*)") at assert.c:92
#7  0x00007f6e44a8c472 in __GI___assert_fail (
    assertion=0x7f6e46431fc0 "(pyproc->interpreter->api->PyGILState_Check)() == 1",
    file=0x7f6e46431e98 "/repo/runtime/realm/python/python_module.cc", line=604,
    function=0x7f6e46433600 <Realm::PythonThreadTaskScheduler::thread_blocking(Realm::Thread*)::__PRETTY_FUNCTION__> "virtual void Realm::PythonThreadTaskScheduler::thread_blocking(Realm::Thread*)") at assert.c:101
#8  0x00007f6e45e11235 in Realm::PythonThreadTaskScheduler::thread_blocking (this=0x560f1c23e900,
    thread=0x7f6d2caef2c0) at /repo/runtime/realm/python/python_module.cc:604
#9  0x00007f6e45d011b8 in Realm::Thread::wait_for_condition<Realm::EventTriggeredCondition> (cond=...,
    poisoned=@0x7f6cce37529f: false) at /repo/runtime/realm/threads.inl:218
#10 0x00007f6e45cefdea in Realm::Event::wait_faultaware (this=0x7f6cce3752a0, poisoned=@0x7f6cce37529f: false)
    at /repo/runtime/realm/event_impl.cc:254
#11 0x00007f6e47c06507 in Legion::Internal::LgEvent::wait_faultaware (this=0x7f6cce3752a0,
    poisoned=@0x7f6cce37529f: false) at /repo/runtime/legion/legion_types.h:2486
#12 0x00007f6e48103d9d in Legion::Internal::FutureImpl::get_untyped_result (this=0x7f6d342082e0,
    silence_warnings=false, warning_string=0x0, internal=false, check_size=false, future_size=0)
    at /repo/runtime/legion/runtime.cc:596
#13 0x00007f6e47bf17d1 in Legion::Future::get_untyped_result (this=0x7f6cc4001cc0, silence_warnings=false,
    warning_string=0x0, check_size=false, future_size=0) at /repo/runtime/legion/legion.cc:2236
#14 0x00007f6e47c5b2a8 in Legion::Future::get_untyped_pointer (this=0x7f6cc4001cc0, silence_warnings=false,
    warning_string=0x0) at /repo/runtime/legion/legion.inl:19882
#15 0x00007f6e47c4382f in legion_future_get_untyped_pointer (handle_=...) at /repo/runtime/legion/legion_c.cc:3117
#16 0x00007f6d4194e052 in ffi_call_unix64 ()
   from /usr/local/lib/python3.6/dist-packages/cffi.libs/libffi-d58a691e.so.8.1.0
#17 0x00007f6d4194cc25 in ffi_call_int ()
   from /usr/local/lib/python3.6/dist-packages/cffi.libs/libffi-d58a691e.so.8.1.0
#18 0x00007f6d41b72d86 in cdata_call (cd=cd@entry=0x7f6cce632ee0,
    args=args@entry=(<_cffi_backend.__CDataOwn at remote 0x7f6cc400c1d0>,), kwds=kwds@entry=0x0)
    at c/_cffi_backend.c:3186
#19 0x00007f6d42d23f29 in _PyObject_FastCallDict (func=<_cffi_backend._CDataBase at remote 0x7f6cce632ee0>,
--Type <RET> for more, q to quit, c to continue without paging--
    args=<optimized out>, nargs=1, kwargs=0x0) at ../Objects/abstract.c:2331
#20 0x00007f6d42c38ec8 in call_function (pp_stack=0x7f6cce377660, oparg=<optimized out>, kwnames=0x0)
    at ../Python/ceval.c:4875
#21 0x00007f6d42c3f303 in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>)
    at ../Python/ceval.c:3335
#22 0x00007f6d42c38a0f in _PyEval_EvalCodeWithName (_co=<code at remote 0x7f6d28443ae0>, globals=<optimized out>,
    locals=<optimized out>, args=<optimized out>, argcount=1, kwnames=0x0, kwargs=0x7f6cce586da0, kwcount=0, kwstep=1,
    defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name='get', qualname='Future.get') at ../Python/ceval.c:4166
#23 0x00007f6d42c390fc in fast_function (kwnames=0x0, nargs=1, stack=<optimized out>,
    func=<function at remote 0x7f6cce5edf28>) at ../Python/ceval.c:4992
#24 call_function (pp_stack=0x7f6cce377900, oparg=<optimized out>, kwnames=0x0) at ../Python/ceval.c:4872
#25 0x00007f6d42c3f303 in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>)
    at ../Python/ceval.c:3335
#26 0x00007f6d42c37803 in _PyFunction_FastCall (co=<optimized out>, args=<optimized out>, nargs=nargs@entry=1,
    globals=globals@entry={'__name__': 'pygion', '__doc__': None, '__package__': '', '__loader__': <SourceFileLoader(name='pygion', path='/repo/bindings/python/pygion.py') at remote 0x7f6d28435c88>, '__spec__': <ModuleSpec(name='pygion', loader=<...>, origin='/repo/bindings/python/pygion.py', loader_state=None, submodule_search_locations=None, _set_fileattr=True, _cached='/repo/bindings/python/__pycache__/pygion.cpython-36.pyc', _initializing=False) at remote 0x7f6d28435cc0>, '__file__': '/repo/bindings/python/pygion.py', '__cached__': '/repo/bindings/python/__pycache__/pygion.cpython-36.pyc', '__builtins__': {'__name__': 'builtins', '__doc__': "Built-in functions, exceptions, and other objects.\n\nNoteworthy: None is the `nil' object; Ellipsis represents `...' in slices.", '__package__': '', '__loader__': <type at remote 0x7f6d2c013cf8>, '__spec__': <ModuleSpec(name='builtins', loader=<type at remote 0x7f6d2c013cf8>, origin=None, loader_state=None, submodule_search_locations=None, _set_fileattr=False, _cached=None) at remote 0x7f6e48...(truncated))
    at ../Python/ceval.c:4933
#27 0x00007f6d42c392be in fast_function (kwnames=0x0, nargs=1, stack=<optimized out>,
    func=<function at remote 0x7f6cce581ea0>) at ../Python/ceval.c:4968
#28 call_function (pp_stack=0x7f6cce377ab0, oparg=<optimized out>, kwnames=0x0) at ../Python/ceval.c:4872
#29 0x00007f6d42c3f303 in _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>)
    at ../Python/ceval.c:3335
#30 0x00007f6d42c38a0f in _PyEval_EvalCodeWithName (_co=<code at remote 0x7f6d283f5390>, globals=<optimized out>,
    locals=<optimized out>, args=<optimized out>, argcount=0, kwnames=0x0, kwargs=0x0, kwcount=0, kwstep=2, defs=0x0,
    defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at ../Python/ceval.c:4166
#31 0x00007f6d42c394ce in PyEval_EvalCodeEx (_co=<optimized out>, globals=<optimized out>, locals=<optimized out>,
    args=<optimized out>, argcount=<optimized out>, kws=<optimized out>, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0,
    closure=0x0) at ../Python/ceval.c:4187
#32 0x00007f6d42cf8d6b in function_call (func=<function at remote 0x7f6cce588bf8>, arg=(), kw=0x0)
    at ../Objects/funcobject.c:604
#33 0x00007f6d42d24768 in PyObject_Call (func=<function at remote 0x7f6cce588bf8>, args=<optimized out>,
    kwargs=<optimized out>) at ../Objects/abstract.c:2261
#34 0x00007f6d42c3e225 in do_call_core (kwdict=0x0, callargs=(), func=<function at remote 0x7f6cce588bf8>)
    at ../Python/ceval.c:5120
--Type <RET> for more, q to quit, c to continue without paging--
#35 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:3404
#36 0x00007f6d42c38a0f in _PyEval_EvalCodeWithName (_co=<code at remote 0x7f6d283ee0c0>, globals=<optimized out>,
    locals=<optimized out>, args=<optimized out>, argcount=4, kwnames=0x7f6e48e9f060, kwargs=0x7f6e48e9f068,
    kwcount=0, kwstep=2, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name='execute_task',
    qualname='Task.execute_task') at ../Python/ceval.c:4166
#37 0x00007f6d42c38c8d in _PyFunction_FastCallDict (func=<function at remote 0x7f6cce5810d0>, args=0x7f6cce378080,
    nargs=4, kwargs={}) at ../Python/ceval.c:5084
#38 0x00007f6d42d23fd1 in _PyObject_FastCallDict (func=func@entry=<function at remote 0x7f6cce5810d0>,
    args=args@entry=0x7f6cce378080, nargs=nargs@entry=4, kwargs=kwargs@entry={}) at ../Objects/abstract.c:2310
#39 0x00007f6d42d246fd in _PyObject_Call_Prepend (func=<function at remote 0x7f6cce5810d0>, obj=<optimized out>,
    args=(<bytearray at remote 0x7f6cce59dd88>, <bytearray at remote 0x7f6d4149c458>,<PHONE_NUMBER>099910147), kwargs={})
    at ../Objects/abstract.c:2373
#40 0x00007f6d42d24768 in PyObject_Call (func=<method at remote 0x7f6d28475088>, args=<optimized out>,
    kwargs=<optimized out>) at ../Objects/abstract.c:2261
#41 0x00007f6d42c3e225 in do_call_core (kwdict={},
    callargs=(<bytearray at remote 0x7f6cce59dd88>, <bytearray at remote 0x7f6d4149c458>,<PHONE_NUMBER>099910147),
    func=<method at remote 0x7f6d28475088>) at ../Python/ceval.c:5120
#42 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:3404
#43 0x00007f6d42c38a0f in _PyEval_EvalCodeWithName (_co=<code at remote 0x7f6d2846ce40>, globals=<optimized out>,
    locals=<optimized out>, args=<optimized out>, argcount=4, kwnames=0x0, kwargs=0x0, kwcount=0, kwstep=2, defs=0x0,
    defcount=0, kwdefs=0x0, closure=0x0, name='__call__', qualname='Task.__call__') at ../Python/ceval.c:4166
#44 0x00007f6d42c38d34 in _PyFunction_FastCallDict (func=<function at remote 0x7f6cce5f1ea0>, args=0x7f6cce378460,
    nargs=4, kwargs=<optimized out>) at ../Python/ceval.c:5084
#45 0x00007f6d42d23fd1 in _PyObject_FastCallDict (func=func@entry=<function at remote 0x7f6cce5f1ea0>,
    args=args@entry=0x7f6cce378460, nargs=nargs@entry=4, kwargs=kwargs@entry=0x0) at ../Objects/abstract.c:2310
#46 0x00007f6d42d246fd in _PyObject_Call_Prepend (func=<function at remote 0x7f6cce5f1ea0>, obj=<optimized out>,
    args=(<bytearray at remote 0x7f6cce59dd88>, <bytearray at remote 0x7f6d4149c458>,<PHONE_NUMBER>099910147), kwargs=0x0)
    at ../Objects/abstract.c:2373
#47 0x00007f6d42d24768 in PyObject_Call (func=<method at remote 0x7f6d28475648>, args=<optimized out>,
    kwargs=<optimized out>) at ../Objects/abstract.c:2261
#48 0x00007f6d42cb189d in slot_tp_call (self=<Task at remote 0x7f6d283f8048>,
    args=(<bytearray at remote 0x7f6cce59dd88>, <bytearray at remote 0x7f6d4149c458>,<PHONE_NUMBER>099910147), kwds=0x0)
    at ../Objects/typeobject.c:6207
#49 0x00007f6d42d24768 in PyObject_Call (func=<Task at remote 0x7f6d283f8048>, args=<optimized out>,
    kwargs=<optimized out>) at ../Objects/abstract.c:2261
#50 0x00007f6e45e12fee in Realm::LocalPythonProcessor::execute_task (this=0x560f1c23e600, func_id=26, task_args=...)
    at /repo/runtime/realm/python/python_module.cc:989
#51 0x00007f6e45b12fef in Realm::Task::execute_on_processor (this=0x7f6d2cd56fb0, p=...)
    at /repo/runtime/realm/tasks.cc:302
#52 0x00007f6e45b170fc in Realm::KernelThreadTaskScheduler::execute_task (this=0x560f1c23e900, task=0x7f6d2cd56fb0)
    at /repo/runtime/realm/tasks.cc:1366
--Type <RET> for more, q to quit, c to continue without paging--
#53 0x00007f6e45e10be2 in Realm::PythonThreadTaskScheduler::execute_task (this=0x560f1c23e900, task=0x7f6d2cd56fb0)
    at /repo/runtime/realm/python/python_module.cc:328
#54 0x00007f6e45b15ea4 in Realm::ThreadedTaskScheduler::scheduler_loop (this=0x560f1c23e900)
    at /repo/runtime/realm/tasks.cc:1105
#55 0x00007f6e45e10e65 in Realm::PythonThreadTaskScheduler::python_scheduler_loop (this=0x560f1c23e900)
    at /repo/runtime/realm/python/python_module.cc:400
#56 0x00007f6e45e16ec2 in Realm::Thread::thread_entry_wrapper<Realm::PythonThreadTaskScheduler, &Realm::PythonThreadTaskScheduler::python_scheduler_loop> (obj=0x560f1c23e900) at /repo/runtime/realm/threads.inl:97
#57 0x00007f6e45aec9b6 in Realm::KernelThread::pthread_entry (data=0x7f6d2caef2c0)
    at /repo/runtime/realm/threads.cc:774
#58 0x00007f6e4441f6db in start_thread (arg=0x7f6cce379800) at pthread_create.c:463

py-bt of thread 23 (realm thread)
(gdb) py-bt
Traceback (most recent call first):
  File "/repo/bindings/python/pygion.py", line 407, in get
    value_ptr = c.legion_future_get_untyped_pointer(self.handle)
  File "/repo/bindings/python/examples/region.py", line 74, in main
    print("child task future contains", child_result.get())
  File "/repo/bindings/python/pygion.py", line 1910, in execute_task
    result = self.body(*args)
  File "/repo/bindings/python/pygion.py", line 1803, in __call__
    return self.execute_task(*args, **kwargs)

thread 10 (segfault, non-realm thread)
(gdb) py-bt
Traceback (most recent call first):
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1342, in _add_callback
    assert not isinstance(handle, events.TimerHandle)
  File "/usr/lib/python3.6/asyncio/selector_events.py", line 517, in _process_events
    self._add_callback(reader)
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1416, in _run_once
    self._process_events(event_list)
  File "/usr/lib/python3.6/asyncio/base_events.py", line 438, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.6/dist-packages/tornado/platform/asyncio.py", line 199, in start
    self.asyncio_loop.run_forever()
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/iostream.py", line 76, in _thread_main
    self.io_loop.start()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
    self._bootstrap_inner()

thread 7 (realm python processer thread)
(gdb) py-bt
Traceback (most recent call first):
  File "/repo/bindings/python/pygion.py", line 414, in get
    c.legion_future_get_void_result(self.handle)
  File "/repo/bindings/python/pygion.py", line 2568, in execution_fence
    f.get()
  File "<ipython-input-1-a129c1b45817>", line 100, in <module>
  <built-in method exec of module object at remote 0x7f336e03f638>
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3263, in run_ast_nodes
    if (await self.run_code(code, result,  async_=asy)):
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 3072, in run_cell_async
    interactivity=interactivity, compiler=compiler, result=result)
  <built-in method send of coroutine object at remote 0x7f324c6080f8>
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/async_helpers.py", line 68, in _pseudo_sync_runner
    coro.send(None)
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2895, in _run_cell
    return runner(coro)
  File "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py", line 2867, in run_cell
    raw_cell, store_history, silent, shell_futures)
  File "/usr/local/lib/python3.6/dist-packages/ipykernel/zmqshell.py", line 539, in run_cell
...

The other threads are the same as what @lightsighter found. Looks like Realm did not acquired the GIL, but started working on something.
I put the reproducer in the docker hub eddy16112/jupyter-crash:py36

After apply the patch from Legion meeting. the jupyter notebook works. I have done the CI tests with python3.6, 3.8 and 3.10, and never see the failures. @elliottslaughter @lightsighter @streichler Thanks for your help. I will close the issue now.


How to create a graph of the scrolling_summary.json

Excuse me
When i run the commd : **flutter drive --driver=integration_test/perf_driver.dart --target=integration_test/perf_test.dart --profile --trace-startup --no-dds
**
I get a file ：scrolling_summary.json
How to create a graph of the results？
I try use chrome://tracing the file， but it not work for me

Hi @artem-2024, is this still an issue with the latest versions of flutter?


implement public mentor page

Purpose

The purpose of this PR is to fix #9
Goals

The goal of this PR is to implement public mentor page
Approach

Screenshots

Checklist

[ ] This PR doesn't commit any keys, passwords, tokens, usernames, or other secrets.
[ ] I have read and understood the development best practices guidelines ( http://bit.ly/sef-best-practices )
[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation

Related PRs

Test environment

Learning

functionality wise it working without any issues. but We have to do some changes to the backend endpoint.
Ex: Pagination should be implemented
@anjula-sack

@janithlahirukariyawasam Can you update this? It's been dragging for a while. Let me know if you need any help.

@janithlahirukariyawasam Any updates?

Continued to #91

Fixed with https://github.com/sef-global/scholarx-frontend/pull/91


PostgreSQL Contrib extension uuid-ossp is not available

Thanks for resolving #11.
https://github.com/CircleCI-Public/cimg-postgres/issues/11#issuecomment-959794008:

Images have been updated to include extensions that are part of the PostgreSQL Contrib package.
Can people confirm on your end that the errors went away?

I can confirm that I am getting something different, but it doesn't appear to have entirely resolved the problem. We are now getting the following in our build pipelines:
SQL State  : 58P01
Error Code : 0
Message    : ERROR: could not open extension control file "/usr/lib/postgresql/13/share/extension/uuid-ossp.control": No such file or directory
Statement  : create extension "uuid-ossp"

and in the Postgres image container output, the corresponding:
2021-11-04 02:19:51.485 UTC [1407] ERROR:  could not open extension control file "/usr/lib/postgresql/13/share/extension/uuid-ossp.control": No such file or directory
2021-11-04 02:19:51.485 UTC [1407] STATEMENT:  create extension "uuid-ossp"

For reference, it looks like uuid-ossp might need special configuration, as mentioned in https://www.postgresql.org/docs/13/uuid-ossp.html#id-<IP_ADDRESS>.6

This is blocking us from migrating from the old circleci/postgres-ram images.

Bump. We just tried upgrading to cimg-postgres and experienced the same issue.

Same errors for cimg/postgres:13.4-postgis / cimg/postgres:13.3-postgis when trying to migrate from circleci/postgres:13.2-postgis-ram.

I think this is affecting all images, getting same error on 11.3

@felicianotech: Is this something that's likely to receive attention from CircleCI?

@schuyler yes

Thank to you everyone that has reported. This will be looked into and hopefully resolved by the beginning of the next work week at the latest.

Hey everyone. I made some changed to include uuid-ossp however before I respin all images, I wanted to verify that it's solving the problem.
Only the image tagscimg/postgres:13.4 and cimg/postgres:13.4-postgis have been updated. Can someone who's using that image, or can try it out, confirm that the extension is available? If so, I can then continue to roll it out to the other tags.
cc @exactlyaron since you mentioned this version specifically.

Hey everyone. I made some changed to include uuid-ossp however before I respin all images, I wanted to verify that it's solving the problem.
Only the image tagscimg/postgres:13.4 and cimg/postgres:13.4-postgis have been updated. Can someone who's using that image, or can try it out, confirm that the extension is available? If so, I can then continue to roll it out to the other tags.
cc @exactlyaron since you mentioned this version specifically.

Yes, the cimg/postgres:13.4 has been installed OK with the uuid-ossp extension!
Though, it is giving the WARNING: POSTGRES_HOST_AUTH_METHOD has been set to "trust" despite of specified POSTGRES_PASSWORD

cimg/postgres:13.4 worked for us as well.

@texpert That warning is okay. PostgreSQL changed their default password behavior a couple of years ago. We basically undid that due to a large number of CircleCI users not using passwords on our platform. For a test CI database on a CI platform, that's okay. This warning is just due to that setting change.

I'm happy things seem to be working now. We actually did another update recently which includes even more extensions. Basically, we're trying to mimic the same compile config options that the official Debian package uses for PostgreSQL. That should provide much more compatibility for more use cases of PG.

@felicianotech yeah that's working.     Thanks very much for getting it sorted so quickly.

Great! Is this working on 11.13 as well?

I've confirmed that my build pipeline works when I use cimg/postgres:13.4. At least, the migrations pass correctly and most tests run. We have one test failure that may be due to a time bomb or race condition, and we are currently evaluating that.

Great! Is this working on 11.13 as well?
EDIT: I can confirm that this problem still exists with the 11.13 image

That's to be expected. He stated that it was only for 13 and 9 until he gets confirmation that it works, and it does work. So hopefully we get it on 11 soon

thank you for moving forward on this @felicianotech - we're looking forward for uuid-ossp to work with the 12.7 image

Bumping this issue again as we can't migrate to the next-gen images without it. Thanks!

Even though this issue isn't closed, we got a successful build run on cimg/postgres:11.13. Thanks all.

thanks @edeesis , cimg/postgres:12.7 is building 🚀

We were able to successfully switch to cimg/postgres:11.13 after this change went in, thanks!

Thanks all, I'll close this as it seems to be done.


question about details of parameters

hello, i am  trying to reproduce the result in the paper.I run the scripts/run_epr.sh successfully. i get the em score of about 71 in mrpc which is 75.98 in the paper.Are the settings in run_epr.sh different from the paper? Can you provide the setting parameters in the paper?
Thanks a lot!

Hi, it's weird as the settings in run_epr.sh is the same as that in the paper. Could you check whether you can obtain similar results to the paper for other methods such as Topk-BERT as I'm not sure if it's due to the randomness of running on different machines.

I found the same question, too. I run the script/run_bm25.sh, and got the acc on mrpc validation set is 0.576 which is far lower than in the paper. I just change the num_ice to 27, and leave other parameters unchanged. I also tried to evaluate sst5 ( set num_ice to 27), the acc is 0.296. Can you give me some advice so that I can get the same acc as you claimed in your paper?

I found the same question, too. I run the script/run_bm25.sh, and got the acc on mrpc validation set is 0.576 which is far lower than in the paper. I just change the num_ice to 27, and leave other parameters unchanged. I also tried to evaluate sst5 ( set num_ice to 27), the acc is 0.296. Can you give me some advice so that I can get the same acc as you claimed in your paper?

While replicating the process, I noticed issues with the training code for the encoder. I've rewritten the training code without using a trainer, following the style of Hugging Face, and without the use of accelerate. Additionally, there were some discrepancies between the parameters used in the code and those described in the paper. For instance, the paper mentions using the three samples with the highest and lowest scores as positive and negative examples, respectively, whereas the code only samples one. I have made adjustments to align with the paper. After these modifications, the results on some datasets are close to those reported in the paper. You might want to give it a try.


Custom shape on the front layer

It would be nice to be able to change the shape of the front layer. I have a design with beveled corners like in Shrine Material theme, but now there is no way to change shape.

https://material.io/components/backdrop#theming

@WieFel I am surprised that this is not being implemented!
Should we generalise frontLayerBorderRadius to frontLayerShape ?? Or keep both?
I am in favour of only having frontLayerShape and deprecating frontLayerBorderRadius.

Looks non-trivial to just replace. frontLayerBorderRadius is used to "clip" as well - maybe to show scrim properly.
Check this code:
https://github.com/fluttercommunity/backdrop/blob/e623fb85a904035247e766b24faea80d8f198fac/lib/src/scaffold.dart#L544-L551

Ok, looks like we can do it "clipping" based on ShapeBorder with ShapeBorderClipper

@milkyway044 Request you to review and test against the PR for this feature.

@WieFel I am surprised that this is not being implemented!

Me too actually!

Should we generalise frontLayerBorderRadius to frontLayerShape ??

Yes LGTM

@milkyway044 Request you to review and test against the PR for this feature.

@daadu this is exactly what i need, thank you so much, guys!


refactor(samples): restore stdout to original state in test

googleapis/code-suggester@v1 was failed but that's not related to this changes.


Added load_grammar.find_grammar_errors

@MegaIng No comment?

Sorry I forget this. I will take a look. I couldn't do a deepdive when I first saw this PR.

Also, that function should be added to the stubs.

@MegaIng I addressed all your comments.

Merged. Thanks!


Resolve comments for crd upgrade verify

/lgtm
/approve
/hold
just holding to remove the forced failure that we see evidence of in https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_api/895/pull-ci-openshift-api-master-verify/1382343659604676608

@deads2k I did some more update

prow only has master branch, so the script find the v1beta1  crd from 1 commit before instead of switching branch.
update some other parts and use clusterversionoperator to test...

/lgtm
Thanks for the test.  pull it out and this can merge.

/hold cancel

/lgtm

adding valid bug. this is a verify script, not shipped product code.


Request retry when future exception occurs may cause warning for future being set twice when heartbeat to server is lost

Fixes the future set twice problem during server poweroff. Fixes issue https://github.com/hazelcast/hazelcast-cpp-client/issues/173

Test PASSed.

Test FAILed.

Test FAILed.

Test FAILed.

Test FAILed.

Test FAILed.

Test FAILed.

Test FAILed.

Test FAILed.

Test FAILed.

Test FAILed.

Test FAILed.

Test FAILed.

verify

Test FAILed.

Test FAILed.

Test FAILed.

Test FAILed.

Test PASSed.


Is the wheels.yml example still considered a good template?

Is the https://github.com/wjakob/nanobind_example/blob/master/.github/workflows/wheels.yml still considered a good template to build off of for nanobind projects?
I immediately hit a bunch of different issues on all three platforms, before I get too deep debugging I'd like to know if this was indeed the right starting point or if there's a better example out there.
Thank you!

This should be okay. What were the issues?


Bug: selecting a link entity and replacing it with a new text keeps the first character associated with the link

Do you want to request a feature or report a bug?
Bug
What is the current behavior?
When you create a link entity, then select it (via mouse or cmd+A) and replace it by typing a new text, first character of the new text is still associated with a link. If you delete a link with a backspace though, everything works as expected, so it only happens for selection without explicit removal.
Repro here https://codepen.io/anon/pen/MzyVNy:

Type in a word
Select the whole thing (cmd+A), click 'Add a link'
Add a link, hit Confirm
Select your text or highlight everything (cmd+A)
Start typing a new text so that it replaces your link
See the first character of your new text having an old link entity

What is the expected behavior?
First character of the new text should not have an entity
Which versions of Draft.js, and which browser / OS are affected by this issue? Did this work in previous versions of Draft.js?
Affected version of Draft: 0.10.5. All previous versions of Draft do not remove entity on selection at all, entity is being applied to the new text.
Trying in Chrome 70.0.3538.77 (64-bit)

Having the same issue here.. Any news or solutions?

Still an issue in Draft 0.11.7

Any update on this? I'd like to keep selected entity when typing.


Uploading Data

In the Query Tab when uploading a CSV file users are not shown their uploaded data unless they re-upload the data or click on one of the functions.

https://github.com/posit-dev/r-shinylive/issues/124
Potential fix 'adding tags$script(src = "https://cdn.plot.ly/plotly-2.11.1.min.js") to the UI so that the dependency is loaded from external CDN'

Following a similar approach as the shiny live issue, I added
tags$script(src = "https://cdn.datatables.net/1.11.3/js/jquery.dataTables.min.js") to the UI and this worked in showing all 3 tables at once. Only issue now is if I update the tables to bootstrap4 to fix the bunched up next page function, it only runs for the 3rd table.


Links are not available for adding service principal

Hi,
As Document mentions below command to add service principal
Docs
New-ServicePrincipal -AppId <APPLICATION_ID> -ServiceId <OBJECT_ID> [-Organization <ORGANIZATION_ID>]***
But there is no such service principal even the link given then descriptions are broken for New-ServicePrincipal cmdlet and Get-ServicePrincipal cmdlet

New-ServicePrincial
https://docs.microsoft.com/powershell/module/exchange/new-serviceprincipal
Get-ServicePrincipal
https://docs.microsoft.com/powershell/module/exchange/get-serviceprincipal
Examples:
New-ServicePrincipal -AppId 71487acd-ec93-476d-bd0e-6c8b31831053 -ServiceId 6233fba6-0198-4277-892f-9275bf728bcc
and
Get-ServicePrincipal -Identity ca73fffa-cedb-4b84-860f-d7fb8aa8a6c1 | Format-List


fix(tribler): Add ARCH support (fixes #1217)

Fixes #1217
Tested and working locally.

@flexiondotorg I can confirm this is working but cannot merge as the mandatory test is failing for some curious reason of its own.

@flexiondotorg I can confirm this is working but cannot merge as the mandatory test is failing for some curious reason of its own.

I didn't run any tests locally, just installed the app. Is the test failure due to API rate limiting?

It does look like rate limiting but the test should have an API token (and I've tried in my filen PR to use alternative ways to put the token in the environment without success).  Maybe something has changed with secrets (but not yet made the documentation ...)


fix libdl name src/Pkcs11Interop/Pkcs11Interop/Common/NativeMethods.cs

Hello,
Our library depends on your lib v4.1.1. We have noticed that applications linked with this lib have an error:
Unhandled exception. System.DllNotFoundException: Unable to load shared library 'libdl' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: liblibdl: cannot open shared object file: No such file or directory
   at Net.Pkcs11Interop.Common.NativeMethods.dlopen(String filename, Int32 flag)
   at Net.Pkcs11Interop.Common.UnmanagedLibrary.Load(String fileName)
   at Net.Pkcs11Interop.LowLevelAPI80.Pkcs11..ctor(String libraryPath)
   at Net.Pkcs11Interop.HighLevelAPI80.Pkcs11..ctor(String libraryPath, AppType appType)
   at Net.Pkcs11Interop.HighLevelAPI.Pkcs11..ctor(String libraryPath, AppType appType)
   at Aktiv.RtAdmin.Startup.RegisterServices(String nativeLibraryPath) in /home/lo1ol/rtadmin/Aktiv.RtAdmin/Startup.cs:line 51
   at Aktiv.RtAdmin.Startup.Configure(String logFilePath, String nativeLibraryPath) in /home/lo1ol/rtadmin/Aktiv.RtAdmin/Startup.cs:line 20
   at Aktiv.RtAdmin.RtAdmin.Main(String[] args) in /home/lo1ol/rtadmin/Aktiv.RtAdmin/Program.cs:line 48
Aborted (core dumped)

This problem can be solved by downloading package  libc6-dev or by creating sym link libdl.so to the library libdl.so.2.
Nevertheless, this solution isn't convenient for common users of our application.  We decided to change name of linked library by libdl.so.2. and this help.
I don't know the purpose why the link to libdl.so doesn't exist by default, but inside all tested OSes libdl.so.2 exists always.
If our solution is relevant, could you please  fix the current version of you project and create a new branch v4.1.2 with this fix for our library.

Hey @lo1ol, thanks for the PR. Could you please answer these two additional questions?

On which Linux distributions and versions are you experiencing this problem?

Did you try to use dllmap in app.config of your application?
<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <dllmap dll="libdl">
        <dllentry os="linux" dll="libdl.so.2" />
    </dllmap>
</configuration>

Thank your for your response.

I found this problem in Ubuntu and some Russian OSes based on Debian and Red Hat (Astra Linux, Red OS, Alt Linux and ROSA).

I think that dllmap can solve my problem. I try it tomorrow.

Hello,
We tested your solution with an implementation similar to this.
Firstly we tried set lib name resolver for our application Assemble, but it didn't work. After that we tried set lib name resolver for your Assemble and this is help.
I close MR, thank you for your help.


arm64 docker images

Describe the problem this improvement solves
When using Apples M1 chip I get the following message, since it uses amd64 builds of the docker images.
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Describe the solution you'd like
Have amd64 and arm64 docker images available
For example like this: https://github.com/virtual-vehicle/pointcloudset/blob/master/.github/workflows/docker.yml
Screenshots

Would this apply to our new Kubernetes backend as well?

Would this apply to our new Kubernetes backend as well?

Pretty sure it would still apply, yes. It depends on the architecture of the nodes, for example when running minikube on an M1 Mac.

Having multi-arch images is on our roadmap and is WIP.

Hi, love the tech! Is there a Docker image available for Apple Silicon now?

@daanrongen That is great to hear! At this time we don't ship any arm images. I don't expect this to change any time soon since most Kubernetes clusters use Linux nodes.
Thank you for expressing your interest though! Helps us prioritize :)


Add support for GC and CC on Firefox

We do get the odd bug report from external contributors that comes from Domato. Hopefully this helps shake out more bugs :)

Thanks!


Add a collection to the index

Collection name
a
Maintainer name
b
Maintainer contact link (i.e. repo issues link, email)
c
Repository URL
d
OCI Reference
efg
I acknowledge that this collection provides new functionality, distinct from the existing dev container collections part of this index

[x] Confirm

#sign-off


response

I was slow on this because I missed class yesterday.  I understand what we are doing somewhat, but I ran out of time.

3.2/6


raidboss: DSR P6 P7 some triggers

P6 Entangled Flame / Spreading Flame trigger
P7 Trinity tank swap trigger

oops...


OCPBUGS-20331: manifests: rename API performance dashboard

In previous PR this manifests was labelled as "available only when Console capability enabled". This causes CVO to force enable Console capability when upgrading from baseline 4.13 cluster - as this manifest is present.
In order to avoid this, the manifest needs to be renamed, so that CVO would treat it as a new one (since its applicability has changed)

/cherrypick release-4.14

/jira refresh

I wonder if CVO will correctly remove the previous manifests. Please make sure no new dashboards appear on Observe - Dashboard

I guess Observe - Dashboard would be inaccessible, since no Console installed


Add tests to note state

Codecov Report
Attention: Patch coverage is 99.12281% with 1 line in your changes missing coverage. Please review.
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Files
Coverage Δ

src/dictionary/mod.rs
100.00% <100.00%> (ø)

src/dictionary/tests.rs
100.00% <100.00%> (ø)

src/note_state/mod.rs
100.00% <100.00%> (ø)

src/note_state/tests.rs
100.00% <100.00%> (ø)

src/main.rs
0.00% <0.00%> (ø)


Accessing 'template' in <web@component:async-button::ember472> is deprecated.

I'm getting this deprecation log in reference to async-button...
DEPRECATION: Accessing 'template' in <web@component:async-button::ember472> is deprecated. To determine if a block was specified to <web@component:async-button::ember472> please use '{{#if hasBlock}}' in the components layout.
        at HANDLERS.(anonymous function) (http://localhost:4201/assets/vendor.js:72821:7)
        at raiseOnDeprecation (http://localhost:4201/assets/vendor.js:72907:7)
        at HANDLERS.(anonymous function) (http://localhost:4201/assets/vendor.js:72821:7)
        at handleDeprecationWorkflow (http://localhost:4201/assets/vendor.js:72956:9)
        at HANDLERS.(anonymous function) (http://localhost:4201/assets/vendor.js:72821:7)
        at deprecationCollector (http://localhost:4201/assets/vendor.js:72978:5)
        at HANDLERS.(anonymous function) (http://localhost:4201/assets/vendor.js:72821:7)
        at invoke (http://localhost:4201/assets/vendor.js:72833:7)
        at Object.deprecate (http://localhost:4201/assets/vendor.js:72912:5)

Is this due to my usage, or something within the addon?
thanks!

If you use the master branch, that deprecation is resolved. We have a release pending

Great, thanks!


Super high Memory consumption

I got the pruned version of this to run with super large workers, but the full recipe seems to even blow these workers out of memory:

We previously noted that these files are super heavily compressed, but one file will only be ~20 GB in memory, what is ramping the memory up to past 100GB?

Perhaps in this case turning of gc is not a great solution?

Checking that in #1 (https://github.com/leap-stc/wavewatch3_feedstock/pull/1/commits/2659b1bcaf2eff9d34d6fd2618f59f9a3caee333)

No that was not it. Let me try to drop all but one variable.

Ok this is riduculous. I have given this extremely large workers (800GB RAM) and still see the same pattern:

From the logs I can tell that it is starting to combine every one of the output chunks (87000 time steps/100 time steps perchunk ~800 - and we see those chunks being combined). At the point where the RAM fills up, not a single chunk of the data has been written.

There is something super wrong here, somehow this is neither scaling up the workers, nor streaming the work through the one very large worker.

Ok now it scaled up the workers to 2 (I set a limit of 3 because these could get very expensive).

Yeah this is horse 💩: Maybe If I give enough workers to load the entire dataset into distributed memory it would work, but that is not really how this is supposed to work:


Adding in an implementation for consumer to sign bodies

Added a consumer implementation to signed request bodies, as well as some tests.
Let me know if there are some test cases i've missed that would be good to put in

Changes Unknown when pulling 2f034a947fd52528c51ba189125f919079052ab5 on rkuncewicz:consumer-implementation into ** on omsmith:master**.

Closing this, ended up just implementing it outside of the library in our project