Rate-limited Hydra APIs cause breakage

We've encountered some issues when interacting with a larger number of OAuth2 clients using the provider. Specifically we're seeing the following:

The errors.As in readOAuth2ClientResource panics with a non-nil pointer.
Hydra APIs can be rate-limited (especially for Ory Network provided Hydra, https://www.ory.sh/docs/guides/rate-limits), so an optional mechanism for backing off and retrying HTTP 429 responses from Hydra SDK calls is introduced.

Both issues can be reproduced by creating a sufficient amount of Hydra OAuth2 clients[1] (e.g. 50) against a rate-limited Hydra and then refreshing the state. With the existing provider version, this should cause an error, which causes the panic[2]. After fixing the error.As issue, we then hit the 429 throttling[2], for which the only immediate recourse without touching the provider is setting Terraform parallelism to something low, which reduces the pressure on the API and avoids getting throttled, but which also then makes certain Terraform runs prohibitively length.
I've provided an example of solving these in pull request https://github.com/svrakitin/terraform-provider-hydra/pull/20.
[1] Sample Terraform setup to repro with:
terraform {
  required_providers {
    hydra = {
      source = "svrakitin/hydra"
      version = "0.5.2"
    }
  }
}

provider "hydra" {
  endpoint = "https://PROJECT-SLUG.projects.oryapis.com"

  authentication {
    http_header {
      name  = "authorization"
      value = "bearer ORY_TOKEN"
    }
  }
}

resource "hydra_oauth2_client" "example" {
  count = 50

  client_id   = "test-client-${count.index}"
  client_name = "test-client-${count.index}"

  redirect_uris = ["http://localhost:8080/callback"]

  response_types             = ["code"]
  token_endpoint_auth_method = "none"
}

[2] Example of provider panic when refreshing:
% terraform apply
hydra_oauth2_client.example[19]: Refreshing state... [id=test-client-19]
hydra_oauth2_client.example[17]: Refreshing state... [id=test-client-17]
hydra_oauth2_client.example[25]: Refreshing state... [id=test-client-25]
hydra_oauth2_client.example[3]: Refreshing state... [id=test-client-3]
hydra_oauth2_client.example[42]: Refreshing state... [id=test-client-42]
hydra_oauth2_client.example[8]: Refreshing state... [id=test-client-8]
hydra_oauth2_client.example[1]: Refreshing state... [id=test-client-1]
hydra_oauth2_client.example[29]: Refreshing state... [id=test-client-29]
hydra_oauth2_client.example[5]: Refreshing state... [id=test-client-5]
hydra_oauth2_client.example[40]: Refreshing state... [id=test-client-40]
hydra_oauth2_client.example[35]: Refreshing state... [id=test-client-35]
hydra_oauth2_client.example[23]: Refreshing state... [id=test-client-23]
hydra_oauth2_client.example[2]: Refreshing state... [id=test-client-2]
hydra_oauth2_client.example[27]: Refreshing state... [id=test-client-27]
hydra_oauth2_client.example[22]: Refreshing state... [id=test-client-22]
hydra_oauth2_client.example[13]: Refreshing state... [id=test-client-13]
hydra_oauth2_client.example[28]: Refreshing state... [id=test-client-28]
hydra_oauth2_client.example[46]: Refreshing state... [id=test-client-46]
hydra_oauth2_client.example[31]: Refreshing state... [id=test-client-31]
hydra_oauth2_client.example[0]: Refreshing state... [id=test-client-0]
hydra_oauth2_client.example[4]: Refreshing state... [id=test-client-4]
hydra_oauth2_client.example[14]: Refreshing state... [id=test-client-14]
hydra_oauth2_client.example[48]: Refreshing state... [id=test-client-48]
hydra_oauth2_client.example[26]: Refreshing state... [id=test-client-26]
hydra_oauth2_client.example[21]: Refreshing state... [id=test-client-21]
hydra_oauth2_client.example[38]: Refreshing state... [id=test-client-38]
hydra_oauth2_client.example[49]: Refreshing state... [id=test-client-49]
hydra_oauth2_client.example[39]: Refreshing state... [id=test-client-39]
hydra_oauth2_client.example[11]: Refreshing state... [id=test-client-11]
hydra_oauth2_client.example[33]: Refreshing state... [id=test-client-33]
hydra_oauth2_client.example[47]: Refreshing state... [id=test-client-47]
hydra_oauth2_client.example[18]: Refreshing state... [id=test-client-18]
hydra_oauth2_client.example[32]: Refreshing state... [id=test-client-32]
hydra_oauth2_client.example[12]: Refreshing state... [id=test-client-12]
hydra_oauth2_client.example[7]: Refreshing state... [id=test-client-7]
hydra_oauth2_client.example[37]: Refreshing state... [id=test-client-37]
hydra_oauth2_client.example[44]: Refreshing state... [id=test-client-44]
hydra_oauth2_client.example[15]: Refreshing state... [id=test-client-15]
hydra_oauth2_client.example[20]: Refreshing state... [id=test-client-20]
hydra_oauth2_client.example[45]: Refreshing state... [id=test-client-45]
hydra_oauth2_client.example[41]: Refreshing state... [id=test-client-41]
hydra_oauth2_client.example[10]: Refreshing state... [id=test-client-10]
hydra_oauth2_client.example[34]: Refreshing state... [id=test-client-34]
hydra_oauth2_client.example[36]: Refreshing state... [id=test-client-36]
hydra_oauth2_client.example[24]: Refreshing state... [id=test-client-24]
hydra_oauth2_client.example[9]: Refreshing state... [id=test-client-9]
hydra_oauth2_client.example[43]: Refreshing state... [id=test-client-43]
hydra_oauth2_client.example[6]: Refreshing state... [id=test-client-6]
hydra_oauth2_client.example[30]: Refreshing state... [id=test-client-30]
hydra_oauth2_client.example[16]: Refreshing state... [id=test-client-16]

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Plugin did not respond
│
│   with hydra_oauth2_client.example[6],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The
│ plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│   with hydra_oauth2_client.example[24],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The
│ plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│   with hydra_oauth2_client.example[43],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The
│ plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│   with hydra_oauth2_client.example[16],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The
│ plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│   with hydra_oauth2_client.example[34],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The
│ plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│   with hydra_oauth2_client.example[9],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The
│ plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│   with hydra_oauth2_client.example[30],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The
│ plugin logs may contain more details.
╵
╷
│ Error: Plugin did not respond
│
│   with hydra_oauth2_client.example[36],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ReadResource call. The
│ plugin logs may contain more details.
╵

Stack trace from the terraform-provider-hydra_v0.5.2 plugin:

panic: errors: target must be a non-nil pointer

goroutine 138 [running]:
errors.As({0x1053c5cd8, 0x1400052f700}, {0x1052f36c0?, 0x0})
	errors/wrap.go:103 +0x470
github.com/svrakitin/terraform-provider-hydra/internal/provider.readOAuth2ClientResource({0x1053cccb0, 0x1400062eee0}, 0x14000510c80, {0x105307720?, 0x1400050f260})
	github.com/svrakitin/terraform-provider-hydra/internal/provider/resource_oauth2_client.go:343 +0x188
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0x140000e4c40, {0x1053ccc08, 0x14000739620}, 0xd?, {0x105307720, 0x1400050f260})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.29.0/helper/schema/resource.go:795 +0xe8
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0x140000e4c40, {0x1053ccc08, 0x14000739620}, 0x14000a8cea0, {0x105307720, 0x1400050f260})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.29.0/helper/schema/resource.go:1089 +0x430
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0x14000371bc0, {0x1053ccc08?, 0x140007dfef0?}, 0x1400052eb00)
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.29.0/helper/schema/grpc_provider.go:649 +0x3e4
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0x14000255b80, {0x1053ccc08?, 0x140007df710?}, 0x140003b3560)
	github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/tf5server/server.go:789 +0x390
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0x105392aa0?, 0x14000255b80}, {0x1053ccc08, 0x140007df710}, 0x14000510980, 0x0)
	github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:431 +0x164
google.golang.org/grpc.(*Server).processUnaryRPC(0x1400024a1e0, {0x1053ccc08, 0x140007df5f0}, {0x1053d0db8, 0x1400025d860}, 0x14000566120, 0x14000383980, 0x1058ed9a8, 0x0)
	google.golang.org/grpc@v1.59.0/server.go:1343 +0xb8c
google.golang.org/grpc.(*Server).handleStream(0x1400024a1e0, {0x1053d0db8, 0x1400025d860}, 0x14000566120)
	google.golang.org/grpc@v1.59.0/server.go:1737 +0x990
google.golang.org/grpc.(*Server).serveStreams.func1.1()
	google.golang.org/grpc@v1.59.0/server.go:986 +0x88
created by google.golang.org/grpc.(*Server).serveStreams.func1 in goroutine 13
	google.golang.org/grpc@v1.59.0/server.go:997 +0x160

Error: The terraform-provider-hydra_v0.5.2 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

[3] Example of 429 rate-limiting:
% terraform apply
╷
│ Warning: Provider development overrides are in effect
│
│ The following provider development overrides are set in the CLI configuration:
│  - svrakitin/hydra in /Users/ttj/projects/metaplay/terraform-provider-hydra
│
│ The behavior may therefore not match any released version of the provider and applying changes may cause the state
│ to become incompatible with published releases.
╵
hydra_oauth2_client.example[27]: Refreshing state... [id=test-client-27]
hydra_oauth2_client.example[39]: Refreshing state... [id=test-client-39]
hydra_oauth2_client.example[6]: Refreshing state... [id=test-client-6]
hydra_oauth2_client.example[38]: Refreshing state... [id=test-client-38]
hydra_oauth2_client.example[19]: Refreshing state... [id=test-client-19]
hydra_oauth2_client.example[17]: Refreshing state... [id=test-client-17]
hydra_oauth2_client.example[32]: Refreshing state... [id=test-client-32]
hydra_oauth2_client.example[46]: Refreshing state... [id=test-client-46]
hydra_oauth2_client.example[22]: Refreshing state... [id=test-client-22]
hydra_oauth2_client.example[48]: Refreshing state... [id=test-client-48]
hydra_oauth2_client.example[33]: Refreshing state... [id=test-client-33]
hydra_oauth2_client.example[34]: Refreshing state... [id=test-client-34]
hydra_oauth2_client.example[18]: Refreshing state... [id=test-client-18]
hydra_oauth2_client.example[12]: Refreshing state... [id=test-client-12]
hydra_oauth2_client.example[45]: Refreshing state... [id=test-client-45]
hydra_oauth2_client.example[49]: Refreshing state... [id=test-client-49]
hydra_oauth2_client.example[20]: Refreshing state... [id=test-client-20]
hydra_oauth2_client.example[37]: Refreshing state... [id=test-client-37]
hydra_oauth2_client.example[5]: Refreshing state... [id=test-client-5]
hydra_oauth2_client.example[9]: Refreshing state... [id=test-client-9]
hydra_oauth2_client.example[29]: Refreshing state... [id=test-client-29]
hydra_oauth2_client.example[0]: Refreshing state... [id=test-client-0]
hydra_oauth2_client.example[47]: Refreshing state... [id=test-client-47]
hydra_oauth2_client.example[42]: Refreshing state... [id=test-client-42]
hydra_oauth2_client.example[25]: Refreshing state... [id=test-client-25]
hydra_oauth2_client.example[2]: Refreshing state... [id=test-client-2]
hydra_oauth2_client.example[4]: Refreshing state... [id=test-client-4]
hydra_oauth2_client.example[13]: Refreshing state... [id=test-client-13]
hydra_oauth2_client.example[23]: Refreshing state... [id=test-client-23]
hydra_oauth2_client.example[40]: Refreshing state... [id=test-client-40]
hydra_oauth2_client.example[8]: Refreshing state... [id=test-client-8]
hydra_oauth2_client.example[43]: Refreshing state... [id=test-client-43]
hydra_oauth2_client.example[31]: Refreshing state... [id=test-client-31]
hydra_oauth2_client.example[10]: Refreshing state... [id=test-client-10]
hydra_oauth2_client.example[36]: Refreshing state... [id=test-client-36]
hydra_oauth2_client.example[44]: Refreshing state... [id=test-client-44]
hydra_oauth2_client.example[28]: Refreshing state... [id=test-client-28]
hydra_oauth2_client.example[11]: Refreshing state... [id=test-client-11]
hydra_oauth2_client.example[26]: Refreshing state... [id=test-client-26]
hydra_oauth2_client.example[7]: Refreshing state... [id=test-client-7]
hydra_oauth2_client.example[14]: Refreshing state... [id=test-client-14]
hydra_oauth2_client.example[16]: Refreshing state... [id=test-client-16]
hydra_oauth2_client.example[15]: Refreshing state... [id=test-client-15]
hydra_oauth2_client.example[24]: Refreshing state... [id=test-client-24]
hydra_oauth2_client.example[1]: Refreshing state... [id=test-client-1]
hydra_oauth2_client.example[30]: Refreshing state... [id=test-client-30]
hydra_oauth2_client.example[21]: Refreshing state... [id=test-client-21]
hydra_oauth2_client.example[41]: Refreshing state... [id=test-client-41]
hydra_oauth2_client.example[35]: Refreshing state... [id=test-client-35]
hydra_oauth2_client.example[3]: Refreshing state... [id=test-client-3]

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: 429 Too Many Requests
│
│   with hydra_oauth2_client.example[14],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
╵
╷
│ Error: 429 Too Many Requests
│
│   with hydra_oauth2_client.example[16],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
╵
╷
│ Error: 429 Too Many Requests
│
│   with hydra_oauth2_client.example[35],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
╵
╷
│ Error: 429 Too Many Requests
│
│   with hydra_oauth2_client.example[30],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
╵
╷
│ Error: 429 Too Many Requests
│
│   with hydra_oauth2_client.example[41],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
╵
╷
│ Error: 429 Too Many Requests
│
│   with hydra_oauth2_client.example[3],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
╵
╷
│ Error: 429 Too Many Requests
│
│   with hydra_oauth2_client.example[1],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
╵
╷
│ Error: 429 Too Many Requests
│
│   with hydra_oauth2_client.example[24],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
╵
╷
│ Error: 429 Too Many Requests
│
│   with hydra_oauth2_client.example[15],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
╵
╷
│ Error: 429 Too Many Requests
│
│   with hydra_oauth2_client.example[21],
│   on test.tf line 21, in resource "hydra_oauth2_client" "example":
│   21: resource "hydra_oauth2_client" "example" {
│
╵

@tjorri Thanks for reporting this as I never tested this provider against the managed Hydra. :)

This should now be resolved with https://github.com/svrakitin/terraform-provider-hydra/pull/20. Closing.


snakemake-storage-plugin-gcs not in container image when using remote storage

Getting this error when trying to run --executor kubernetes:
ERROR: Could not find a version that satisfies the requirement snakemake-storage-plugin-gcs (from versions: none)
ERROR: No matching distribution found for snakemake-storage-plugin-gcs
snakemake version: commit 6f345d3183101d1d01c0600d40a470cff7b99094
Does snakemake/snakemake:vget_last_stable_version() need to have snakemake-storage-plugin-gcs installed on it?

Actually not sure if the container is the issue.  A basic workflow that creates a text file seems to work.

Sorry was using the wrong snakemake version.


Flutter Web, script to provide PWA functionality causes errors on profile/release mode when using canvaskit

Steps to Reproduce

Create a default app.
Integrate firebase (add libraries to the index.html)
Run in profile mode (if running with the HTML web-renderer flag works ok)

When running the flutter app in profile mode u can see in the google inspector some binding issues:
BindingError: Expected null or instance of Paragraph, got an instance of Paragraph
js_primitives.dart:47     at BindingError. (https://unpkg.com/canvaskit-wasm@0.25.1/bin/profiling/canvaskit.js:9:130708)
js_primitives.dart:47     at new BindingError (eval at createNamedFunction (https://unpkg.com/canvaskit-wasm@0.25.1/bin/profiling/canvaskit.js:9:130417), :4:34)
js_primitives.dart:47     at throwBindingError (https://unpkg.com/canvaskit-wasm@0.25.1/bin/profiling/canvaskit.js:9:134225)
js_primitives.dart:47     at upcastPointer (https://unpkg.com/canvaskit-wasm@0.25.1/bin/profiling/canvaskit.js:9:140630)
js_primitives.dart:47     at RegisteredPointer.nonConstNoSmartPtrRawPointerToWireType [as toWireType] (https://unpkg.com/canvaskit-wasm@0.25.1/bin/profiling/canvaskit.js:9:143398)
js_primitives.dart:47     at Canvas$drawParagraph [as drawParagraph] (eval at new_ (https://unpkg.com/canvaskit-wasm@0.25.1/bin/profiling/canvaskit.js:9:152523), :8:26)
js_primitives.dart:47     at JavaScriptObject.drawParagraph$3 (http://localhost:62205/main.dart.js:44475:23)
js_primitives.dart:47     at Object.drawParagraph$3$x (http://localhost:62205/main.dart.js:10081:43)
js_primitives.dart:47     at CkCanvas.drawParagraph$2 (http://localhost:62205/main.dart.js:27886:9)
js_primitives.dart:47     at CanvasKitCanvas.drawParagraph$2 (http://localhost:62205/main.dart.js:28349:29)
js_primitives.dart:47 Another exception was thrown: Instance of 'ErrorSummary'
In the index.html if the default script to provide PWA functionality is commented out the app works ok
  <script>
    var serviceWorkerVersion = null;
    var scriptLoaded = false;
    function loadMainDartJs() {
      if (scriptLoaded) {
        return;
      }
      scriptLoaded = true;
      var scriptTag = document.createElement('script');
      scriptTag.src = 'main.dart.js';
      scriptTag.type = 'application/javascript';
      document.body.append(scriptTag);
    }

    if ('serviceWorker' in navigator) {
      // Service workers are supported. Use them.
      window.addEventListener('load', function () {
        // Wait for registration to finish before dropping the <script> tag.
        // Otherwise, the browser will load the script multiple times,
        // potentially different versions.
        var serviceWorkerUrl = 'flutter_service_worker.js?v=' + serviceWorkerVersion;
        navigator.serviceWorker.register(serviceWorkerUrl)
          .then((reg) => {
            function waitForActivation(serviceWorker) {
              serviceWorker.addEventListener('statechange', () => {
                if (serviceWorker.state == 'activated') {
                  console.log('Installed new service worker.');
                  loadMainDartJs();
                }
              });
            }
            if (!reg.active && (reg.installing || reg.waiting)) {
              // No active web worker and we have installed or are installing
              // one for the first time. Simply wait for it to activate.
              waitForActivation(reg.installing ?? reg.waiting);
            } else if (!reg.active.scriptURL.endsWith(serviceWorkerVersion)) {
              // When the app updates the serviceWorkerVersion changes, so we
              // need to ask the service worker to update.
              console.log('New service worker available.');
              reg.update();
              waitForActivation(reg.installing);
            } else {
              // Existing service worker is still good.
              console.log('Loading app from service worker.');
              loadMainDartJs();
            }
          });

        // If service worker doesn't succeed in a reasonable amount of time,
        // fallback to plaint <script> tag.
        setTimeout(() => {
          if (!scriptLoaded) {
            console.warn(
              'Failed to load app from service worker. Falling back to plain <script> tag.',
            );
            loadMainDartJs();
          }
        }, 4000);
      });
    } else {
      // Service workers not supported. Just drop the <script> tag.
      loadMainDartJs();
    }
  </script>

No issues found! (ran in 1.4s)

[√] Flutter (Channel stable, 2.2.2, on Microsoft Windows [Version 10.0.19042.1052], locale es-ES)
    • Flutter version 2.2.2 at C:\src\flutter\flutter
    • Framework revision d79295af24 (3 days ago), 2021-06-11 08:56:01 -0700
    • Engine revision 91c9fc8fe0
    • Dart version 2.13.3

[√] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at C:\Users\cahev\AppData\Local\Android\sdk
    • Platform android-30, build-tools 30.0.3
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b04)
    • All Android licenses accepted.

[√] Chrome - develop for the web
    • Chrome at C:\Program Files (x86)\Google\Chrome\Application\chrome.exe

[√] Visual Studio - develop for Windows (Visual Studio Community 2019 16.7.5)
    • Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
    • Visual Studio Community 2019 version 16.7.30523.141
    • Windows 10 SDK version 10.0.18362.0

[√] Android Studio (version 3.6)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin version 49.0.1
    • Dart plugin version 192.8052
    • Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b04)

[√] IntelliJ IDEA Community Edition (version 2019.2)
    • IntelliJ at C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2019.2.1
    • Flutter plugin version 45.1.1
    • Dart plugin version 192.7761

[√] VS Code (version 1.55.2)
    • VS Code at C:\Users\cahev\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.20.0

[√] Connected device (3 available)
    • Windows (desktop) • windows • windows-x64    • Microsoft Windows [Version 10.0.19042.1052]
    • Chrome (web)      • chrome  • web-javascript • Google Chrome 91.0.4472.77
    • Edge (web)        • edge    • web-javascript • Microsoft Edge 91.0.864.37

• No issues found!

2. Integrate firebase (add libraries to the index.html)

@chessasuke Does the error occur if you don't integrate firebase ?
Also, which script section inside index.html need to comment out which then works ?
Thanks.

The one that I copied in the description. It's the one that has this default comment when we create a new project:

To see the error u need to run in profile or release. Also must be in the default web-renderer canvaskit, if running with the web-renderer HTML flag it wont show.
U don't need actually to integrate any firebase service, the error shows without adding any SDK to the html.

Thanks for the details.  I see the same behavior using latest stable.

browser console log

    Uncaught TypeError: Cannot read property 'toString' of null
    at _PointerAdapter_setup_closure1.call$1 (pointer_binding.dart:488)
    at _PointerAdapter__addPointerEventListener_closure.call$1 (pointer_binding.dart:506)
    at _BaseAdapter_addEventListener_closure.call$1 (pointer_binding.dart:214)
    at invokeClosure (js_helper.dart:1826)
    at js_helper.dart:1858
js_primitives.dart:47 BindingError: Expected null or instance of Paragraph, got an instance of Paragraph
js_primitives.dart:47     at BindingError.<anonymous> (https://unpkg.com/canvaskit-wasm@0.25.1/bin/profiling/canvaskit.js:9:130708)
js_primitives.dart:47     at new BindingError (eval at createNamedFunction (https://unpkg.com/canvaskit-wasm@0.25.1/bin/profiling/canvaskit.js:9:130417), <anonymous>:4:34)
js_primitives.dart:47     at throwBindingError (https://unpkg.com/canvaskit-wasm@0.25.1/bin/profiling/canvaskit.js:9:134225)
js_primitives.dart:47     at upcastPointer (https://unpkg.com/canvaskit-wasm@0.25.1/bin/profiling/canvaskit.js:9:140630)
js_primitives.dart:47     at RegisteredPointer.nonConstNoSmartPtrRawPointerToWireType [as toWireType] (https://unpkg.com/canvaskit-wasm@0.25.1/bin/profiling/canvaskit.js:9:143398)
js_primitives.dart:47     at Canvas$drawParagraph [as drawParagraph] (eval at new_ (https://unpkg.com/canvaskit-wasm@0.25.1/bin/profiling/canvaskit.js:9:152523), <anonymous>:8:26)
js_primitives.dart:47     at JavaScriptObject.drawParagraph$3 (http://localhost:53698/main.dart.js:46222:23)
js_primitives.dart:47     at Object.drawParagraph$3$x (http://localhost:53698/main.dart.js:10244:43)
js_primitives.dart:47     at CkCanvas.drawParagraph$2 (http://localhost:53698/main.dart.js:29347:9)
js_primitives.dart:47     at CanvasKitCanvas.drawParagraph$2 (http://localhost:53698/main.dart.js:29810:29)
4js_primitives.dart:47 Another exception was thrown: Instance of 'ErrorSummary'

flutter doctor -v
/Users/dhs/Documents/Fluttersdk/flutter/bin/flutter doctor --verbose
[✓] Flutter (Channel stable, 2.2.2, on Mac OS X 10.15.4 19E2269 darwin-x64, locale en-GB)
    • Flutter version 2.2.2 at /Users/dhs/Documents/Fluttersdk/flutter
    • Framework revision d79295af24 (5 days ago), 2021-06-11 08:56:01 -0700
    • Engine revision 91c9fc8fe0
    • Dart version 2.13.3

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.56.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (4 available)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 • ios            • iOS 14.4.1
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     • ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    • darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   • web-javascript • Google Chrome 91.0.4472.106

• No issues found!

We've upgraded to CanvasKit 0.27. Does it still happen on the master channel?

I don't see the problem now, thanks!!


libman.so

Hello still got libma.so error though i have alredy checked Player settings a it was mentioned inprevious answers   java.lang.UnsatisfiedLinkError: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.xyz--xcZJ-kO3KCWgDTEKcmJxQ==/base.apk"],nativeLibraryDirectories=[/data/app/com.xyz--xcZJ-kO3KCWgDTEKcmJxQ==/lib/x86, /data/app/com.xyz--xcZJ-kO3KCWgDTEKcmJxQ==/base.apk!/lib/x86, /system/lib]]] couldn't find "libmain.so"

Is it workable on real device . I cannot see if emulator run it/.

@asmadsen
Hi, can you take some time to reply to this question? This library is very valuable to our users

Yep. I have tested on real device and it works but still cannot launch it on emulator.

Hi, what version of 'react native' you are using? I can't start it on the physical device now, and it will crash when I enter 'app'. I use 'react native 0.62.2' and 'unity' version is' 2019.3.5 '

"dependencies": {
"@asmadsen/react-native-unity-view": "0.0.6",
"expo": "~38.0.1",
"expo-splash-screen": "~0.4.0",
"expo-status-bar": "^1.0.0",
"expo-updates": "~0.2.10",
"react": "~16.11.0",
"react-dom": "~16.11.0",
"react-native": "~0.62.2",
"react-native-gesture-handler": "~1.6.1",
"react-native-reanimated": "~1.9.0",
"react-native-screens": "~2.9.0",
"react-native-unimodules": "~0.10.1",
"react-native-web": "~0.11.7"

2019.4.f1

Unity only exports fo arm7 and arm64 architectures. The android emulator running on the desktop pc is x86 (usually). So it is impossible to get the unity exported project to run on android emulator.


Subfeature/merge quickhull revised with new core

you add a couple of models and materials, are they used anywhere? or going to be used?

you add a couple of models and materials, are they used anywhere? or going to be used?

yes those are the test meshes used for quickhull

@LeonBrands whenever you can review again, you are the only thing blocking this PR now


refactor and streamline repository

wis2box at present is built out as follows:

wis2box: Python package with docker setup for entire stack
wis2box-api: separate repository
wis2box-ui: separate repository

To streamline development and better represent functionality:

update wis2box repository into repo of docker components
docker-compose*.yml files at root
each component in its own top level directory (api, ui, auth, etc.)
move wis2box-api and wis2box-ui proper as top level directories into wis2box repo

This will also help in cutting a single version against a single repo as part release management.

I like the idea of restructuring the repository to have component folders at the root level.
For what its worth, I think wis2box-ui and wis2box-api are better left in separate repositories. Their respective directories could be compose file configuration for these images. It is my impression that including the source code for these components encumbers future modular growth.
How is this going to work with nginx configuration? Will python3 wis2box-ctl.py ... still have zero dependencies to run on a fresh install of python3?

Implemented in #362


Create tooling to flag breaking changes in Cadl API definition

We need a breaking changes tool that can can diff a new Cadl file with a previous one and  report any changes that violate Azure’s breaking changes policy.

FYI @maririos

@allenjzhang The tooling team depends on the language teams for any language-specific tooling. As this tool is specific to Cadl the work falls to the Cadl team.


Breaks with Ember 3.1: Cannot read property 'fullName' of undefined

Just upgraded a project to Ember 3.1 and this package throws an error when trying to instantiate - TypeError: Cannot read property 'fullName' of undefined.
It looks like the error stems from this code.
I found a similar issue here, and there's a suggested workaround there.

+1


Eliminate redundant traversal logic of SHAMap:

Only the const_iterator interface remains.

@miguelportilla @scottschurr

The Travis failure is bogus.  It's a build timeout on the gcc non-unity build.  That build successfully got into the npm test phase.

I requested a rebuild (uch, accidentally requested a rebuild of all travis for that branch, not the one failed branch).
We need to make sure we can get everything green before we mark something approved or the whole process becomes too subjective to be useful!
Do we know why we keep timing out?

I spent a fair time examining the new upper_bound method and (unsurprisingly) found no problems.  I'm assuming that the rest of the code base had already been adjusted so the removed SHAMap interfaces were already un-needed.  The result is a very nice narrowing of the SHAMap interface.  Looks good.
It would be great if there were a unit test for upper_bound, but I suspect a good unit test would be tough to build.  So I won't hold my breath for that.
:+1:

@rec, regarding the timeouts Travis says this at the top of the timed-out runs: "This job ran on our legacy infrastructure.  Please read our docs on how to upgrade".  It includes the following link: http://docs.travis-ci.com/user/migrating-from-legacy/?utm_source=legacy-notice&utm_medium=banner&utm_campaign=legacy-upgrade

@rec I was wondering if it is possible to use this iterator for ledger_data
and avoid the use of succ() after each iteration of the loop?

@sublimator Well, yeah, calling succ to iterate through all the items in a ledger is horribly overpriced.

@sublimator Sorry let me clarify. We should not be using SHAMap::const_iterator in ledger_data, because the SHAMap is not exposed (open views have no SHAMaps). Instead we should use the ReadView::sles container and use its iteration facilities (which call into the SHAMap iterator for ledgers):
https://github.com/ripple/rippled/blob/develop/src/ripple/ledger/ReadView.h#L367

sles has facility to use findNext type functionality to skip lots of keys?

@sublimator Well lets look at the implementation of the sles iterator for Ledger derived ReadViews:
https://github.com/ripple/rippled/blob/develop/src/ripple/app/ledger/Ledger.cpp#L64

I can't see anything related to findNext?

@sublimator sles_iter_impl uses SHAMap::const_iterator

Example ?

:+1:
Good changes

@vinniefalco

Using for(auto const& sle : ledger.sles) is idiomatic

But when you're paging through a map, 256 or 2000 items at a time, as you get toward the end of the map, you're needlessly iterating through hundreds of thousands of sles you're already seen.

So I was wondering if there's any method here that allows creating some kind of iterator, that sets the initial position to be after(marker) by following it down into the map.  This would probably be better than walking over all items until you found your marker, or calling succ() in a loop to avoid that.

This sounds like the mission statement for upper_bound that this P/R creates.

@HowardHinnant
Yeah :) !

@HowardHinnant Do you remember which unit tests will call into upper_bound? As Scott mentioned, the current shamap.test doesn't call any of your changes and I would like to checkout some of the other tests.
Beyond this PR... The shamap test seems to lack coverage. It only tests one type of the three types of maps. Is it possible to test the others? The test itself uses up to 8 hashes, why not construct a heavily loaded tree and traverse it deeply?


Refactor README.md

The README has grown long and complex. This makes it hard to digest... Still, key information is missing, such as: detailled explanation of all variables, which roles are required for which usage scenario, mandatory vs. optional roles, dependencies, etc.
To address these issues, I'd suggest the following changes:

Split the examples from the main README.md into multiple documentation files (in a samples/*.md directory), each addressing a particular usage scenario. e.g.:

Different installation methods (repo vs. local archive vs. remote archive vs. directory)
Different inventory formats (YAML vs. JSON)
Different platforms (RHEL/CentOS vs. SLES vs. Ubuntu)
Different components (e.g. Core vs. GUI vs. CES, etc.)

Add a file VARIABLES.md with detailled explanation of each variable (including default value)

I will draft a PR to implement these suggestions. Please let me know if you have further ideas on how to improve the documentation...

As #237 was merged I'm closing this issue.


Merging updates in Master to Develop

We last made changes in 'master' after 'develop' was merged into 'master', so here we are merging the latest changes into 'develop' so it has everything 'master' has and we can use that for the security updates and any other changes to test with on the AWS Develop instance before committing to 'master' and then to the AWS Production instance.

I used the "rebase and merge' Pull Request on 'develop' to better reflect it's status as the prototype (if that makes sense).


Cannot submit a valid operation to IonQ after local simulation of another operation with flow control

To Reproduce

Start IQ#
Simulate an operation that does flow control
Execute another operation that doesn't do flow control on IonQ simulator

Expected behavior
The operation without flow control runs on the IonQ simulator.
System information

OS: Windows 10
Browser: Edge
IQ# Version: 0.17.2105143879

Additional context
Workaround: swap repo steps 2 and 3.
Jupyter notebook to reproduce:
{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e6c348bb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/json": "{\"rows\":[{\"@type\":\"@tuple\",\"Item1\":\"iqsharp\",\"Item2\":\"0.17.2105143879\"},{\"@type\":\"@tuple\",\"Item1\":\"Jupyter Core\",\"Item2\":\"<IP_ADDRESS>\"},{\"@type\":\"@tuple\",\"Item1\":\".NET Runtime\",\"Item2\":\".NETCoreApp,Version=v3.1\"}]}",
      "text/html": [
       "

Component	Version

iqsharp	0.17.2105143879

Jupyter Core	<IP_ADDRESS>

.NET Runtime	.NETCoreApp,Version=v3.1

"
      ],
      "text/plain": [
       "Component    Version\r\n",
       "------------ ------------------------\r\n",
       "iqsharp      0.17.2105143879\r\n",
       "Jupyter Core <IP_ADDRESS>\r\n",
       ".NET Runtime .NETCoreApp,Version=v3.1\r\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7b00e4c9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/json": "[\"Qrng\"]",
      "text/html": [
       "

Qrng

"
      ],
      "text/plain": [
       "Qrng"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "operation Qrng() : Result {\n",
    "    use q = Qubit(); \n",
    "    H(q);\n",
    "    return M(q);\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "eac1244d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/json": "[\"QrngWithControl\"]",
      "text/html": [
       "

QrngWithControl

"
      ],
      "text/plain": [
       "QrngWithControl"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "operation QrngWithControl() : Result {\n",
    "    use q = Qubit(); \n",
    "    H(q);\n",
    "    if (M(q) == One) {\n",
    "        return One;\n",
    "    } else {\n",
    "        return Zero;\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8660abec",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/json": "1",
      "text/plain": [
       "One"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%simulate QrngWithControl"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "97bd4653",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Connected to Azure Quantum workspace AquaPub in location westus.\n",
      "Loading package Microsoft.Quantum.Providers.IonQ and dependencies...\n",
      "Active target is now ionq.simulator\n",
      "Submitting Qrng to target ionq.simulator...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The Q# operation Qrng could not be compiled as an entry point for job execution.\n",
      "C:\\snippet_efc9c24b-c26d-4501-9c55-279c76ac6a64.qs(5,9): error QS5023: The target ionq.simulator does not support comparing measurement results.\n"
     ]
    },
    {
     "data": {
      "application/json": "{\"error_code\":1009,\"error_name\":\"InvalidEntryPoint\",\"error_description\":\"The specified Q# operation cannot be used as an entry point for Azure Quantum job submission.\"}",
      "text/html": [
       "The specified Q# operation cannot be used as an entry point for Azure Quantum job submission."
      ],
      "text/plain": [
       "The specified Q# operation cannot be used as an entry point for Azure Quantum job submission."
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%azure.connect \"/subscriptions/7a67bf47-0427-4db4-b956-1e1f39a954cc/resourceGroups/Playground/providers/Microsoft.Quantum/Workspaces/AquaPub\" location=\"West US\"\n",
    "\n",
    "%azure.target ionq.simulator\n",
    "\n",
    "%azure.execute Qrng"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa4a33f8",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Q#",
   "language": "qsharp",
   "name": "iqsharp"
  },
  "language_info": {
   "file_extension": ".qs",
   "mimetype": "text/x-qsharp",
   "name": "qsharp",
   "version": "0.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}

The issue was that all code was being compiled against the given hardware target, rather than just the specified entry point. The linked PR fixes the bug.


Enhancement on the vertical tabs

Mozilla is bringing vertical tabs for users. The icons with all templates are misaligned.

To reproduce the issue, you can download Firefox Nightly. Go to about:config and set sidebar.revamp to true.

The first issue is that the Flat theme needs a padding for tabs:
./tweaks.sh -f flat

Monterey looks good:

The second issue is that window buttons are missed
Firefox with the default theme

Firefox with the all variants of the WhiteSur theme

now it's ok

If you active this hide-window-buttons.css windows button will gide when  maximized

If you active this hide-window-buttons.css windows button will hide when maximized

hide-window-buttons.css is not enabled and the window is not fully maximized. There are no window buttons.

I'm using the flat theme. Then opened omgubuntu.org in a new container. The container boarder line is not aligned properly.


A question about the deep separable convolutional layer

A very excellent job. May I ask if it is possible to introduce the reason why deep separable convolutional layer was first used in LightM-UNet

A very excellent job. May I ask if it is possible to introduce the reason why deep separable convolutional layer was first used in LightM-UNet

A very excellent job. May I ask if it is possible to introduce the reason why deep separable convolutional layer was first used in LightM-UNet

Thank you for your attention and recognition of LightM-UNet.
In LightM-UNet, we employ depthwise convolution for several reasons.
Existing visual theories suggest that the shallow layers of a network extract shallow features of images, such as color and texture, which do not necessarily require complex global information to capture. In comparison to architectures like Mamba or Transformer, CNNs are better suited for this task.
However, as the network delves deeper and needs to extract deeper features of images, such as semantic features, it requires modeling long-range spatial dependencies. CNNs, constrained by their receptive fields, struggle to model this characteristic effectively.
To minimize the parameter count of the model, we utilize depthwise convolution instead of the naive convolution method.

A very excellent job. May I ask if it is possible to introduce the reason why deep separable convolutional layer was first used in LightM-UNet

Thank you for your attention and recognition of LightM-UNet. In LightM-UNet, we employ depthwise convolution for several reasons. Existing visual theories suggest that the shallow layers of a network extract shallow features of images, such as color and texture, which do not necessarily require complex global information to capture. In comparison to architectures like Mamba or Transformer, CNNs are better suited for this task. However, as the network delves deeper and needs to extract deeper features of images, such as semantic features, it requires modeling long-range spatial dependencies. CNNs, constrained by their receptive fields, struggle to model this characteristic effectively.
To minimize the parameter count of the model, we utilize depthwise convolution instead of the naive convolution method.

Thank you very much for your answer, which has greatly benefited me.
May I ask if there are any papers that can support this viewpoint“the shallow layers of a network extract shallow features of images, such as color and texture, which do not necessarily require complex global information to capture”

A very excellent job. May I ask if it is possible to introduce the reason why deep separable convolutional layer was first used in LightM-UNet

Thank you for your attention and recognition of LightM-UNet. In LightM-UNet, we employ depthwise convolution for several reasons. Existing visual theories suggest that the shallow layers of a network extract shallow features of images, such as color and texture, which do not necessarily require complex global information to capture. In comparison to architectures like Mamba or Transformer, CNNs are better suited for this task. However, as the network delves deeper and needs to extract deeper features of images, such as semantic features, it requires modeling long-range spatial dependencies. CNNs, constrained by their receptive fields, struggle to model this characteristic effectively.
To minimize the parameter count of the model, we utilize depthwise convolution instead of the naive convolution method.

Thank you very much for your answer, which has greatly benefited me. May I ask if there are any papers that can support this viewpoint“the shallow layers of a network extract shallow features of images, such as color and texture, which do not necessarily require complex global information to capture”

Perhaps you can refer to 'SwinIR: Image Restoration Using Swin Transformer' and related works.

A very excellent job. May I ask if it is possible to introduce the reason why deep separable convolutional layer was first used in LightM-UNet

Thank you for your attention and recognition of LightM-UNet. In LightM-UNet, we employ depthwise convolution for several reasons. Existing visual theories suggest that the shallow layers of a network extract shallow features of images, such as color and texture, which do not necessarily require complex global information to capture. In comparison to architectures like Mamba or Transformer, CNNs are better suited for this task. However, as the network delves deeper and needs to extract deeper features of images, such as semantic features, it requires modeling long-range spatial dependencies. CNNs, constrained by their receptive fields, struggle to model this characteristic effectively.
To minimize the parameter count of the model, we utilize depthwise convolution instead of the naive convolution method.

Thank you very much for your answer, which has greatly benefited me. May I ask if there are any papers that can support this viewpoint“the shallow layers of a network extract shallow features of images, such as color and texture, which do not necessarily require complex global information to capture”

Perhaps you can refer to 'SwinIR: Image Restoration Using Swin Transformer' and related works.

Thank you very much for your reply.


restclient lib should communicate its HTTP needs with loose coupling

The restclient lib should communicate its needs to GET and POST to pages with loose coupling.
This is because certain types of clients may need to handle their requests in a manner that cannot be accomplished with a single method call.  E.g. a javascript wrapper should use the standard ajax calls, which are async and have their own callbacks.
I feel like this should be done with two channels, one for input and one for output.  These receive and send simple data-only structs.  E.g.
Output:
{
"method": "POST",
"url": "blah",
"content-type": "application/json",
"data": "blahblahblah"
}
Input:
{
"output": as per above,
"error": null,
"http-response": {
"status": 200
}
}

We will need an ID, as we cannot use content addressable storage on items that are time dependent (e.g. server could be restarted).

Using a fairly standard pattern where the user receives a channel that dispenses a command they should execute, along with another channel for the result (one per command).

Supported with e77df05


CIP-0046? | Merkelised Plutus Scripts

Currently, the hash of a script is simply the hash of its serialisation.
This CIP proposes changing this such that the hash of a script (term)
is a function of its immediate children's hashes, forming a Merkle Tree from the AST.
This allows one to shallowly verify a script's hash, and is useful on Cardano,
because it allows scripts to check that a script hash is an instantiation of a parameterised script.
In addition, a blake2b_224 built-in function must be added.
This is inspired by BIP-144,
but the motivations are very different.
Rendered

This proposal is a low-cost way to allow us to design protocols autonomously making use of staking validators. For that alone it is worth pushing forward on as currently they're not very useful, apart from one fancy technique.
I'm also thinking it may find different uses but I'll sit on it for a while.

Withdrawn my cip in favour of this.

This may be a n00b question and I may have forgotten my PL coursework, but doesn't this make the hash of the script reliant on the parser/lexer now? This is not worse than the way it works for compilation today, I am just wondering if the exact same AST will be generated between different platform bindings (e.g., Plutus/Haskell, Helios, Aiken) or if the writer/deployers will have to be cognizant of the platform choice.

@michaelpj
Use Case:

I was contacted during Cardano Summit 2022 in Lausanne by a member of MartifyLabs team that our current model of off-chain-dapp registry (https://github.com/Cardano-Fans/crfa-offchain-data-registry) doesn't work for parameterised scripts. Basically what Martify Labs folks are doing they are generating NFT market places... on the fly. With such a model each time any param changes, they generate new script hashes. It is not possible currently to identity if script hash in fact belongs to the same root. This in itself does not seem like an issue but it could be a big problem for things like dApp audits / certification where a certain certification firm / company will audit certain script hashes and will provide proof on chain for doing so: see CPS: https://github.com/cardano-foundation/CIPs/pull/393

@ matiwinnetou why can't they use a datum?

For the efficiency argument, could you instead of merklising every node of the AST, make it optionally merkled or the hash of the serialisation of that node.

For the efficiency argument, could you instead of merklising every node of the AST, make it optionally merkled or the hash of the serialisation of each node.

There are two ways of doing this:

Allow the user to choose how to hash the script, this means many hashes for one script (seems problematic),
or, make the choice part of the AST itself. This preserves one hash per script, but makes the scripts bigger.

The structure of the Flat encoding is thankfully quite simple.
Given an ADT, the encoding is essentially a constructor index, then the fields placed sequentially.
Primitive types are handled like ADTs.
Because everything is variable length, you can not know where anything starts and ends without decoding the entire script.
This also makes the trick @micahkendell suggested not useful AFAICT.
One thing I'm wondering about is, will you ever hash a script without also running it? If you will run it, you have to decode it anyway.
Assuming that there are instances where you don't currently have to run them (and thus decode them),
then I see two routes for improving the hashing performance besides raw engineering:

Making the encoding format more amenable to Merkle hashing.
Changing the scheme in the CIP to more closely match the Flat encoding.

For the latter, e.g. rather than hashing an integer the way described in the CIP,
it could simply be the hash of the flat encoding. One issue with this is that it complicates checking the hash
on-chain, because you now have to emulate Flat encoding (which is admittedly simple, so should still be possible,
and also much more uncommon than checking the hash off-chain).
Hashing a Term would involve hashing the index the Flat encoding uses, concatenated to the hash of the arguments to the constructor.
To do this you don't to "decode" the term, but rather determine how long the index is (to hash the rest) and how many arguments there are.
This however still doesn't seem optimal.
Going back to @micahkendall's idea, perhaps we can make it doable:
In addition to Term, have another MerkleTerm that has the same structure,
but at each leaf, has Either Term MerkleTerm. (We could use HKDs for this.)
What we then serialise would be Either Term MerkleTerm.
MerkleTerm wouldn't use the Flat encoding, as that's part of the problem here.
The encoding would be the constant-length (1 byte) constructor index, then the encodings of the fields placed serially.
When the child is a Term, it's prefixed with its length.
Then, when hashing a script, we use Merkle hashing until we reach a Term. Because we know the length of its encoding, we can pass the slice of the input bytestring to the hashing algorithm trivially.
We don't need to know the length of the encodings of the MerkleTerms, because we need to decode those parts anyway.
This scheme is a bit complicated, but AFAICT would keep hashing just as fast for all existing scripts.
For scripts that use MerkleTerm, the hashing would inherently be slower, but you'd only use the MerkleTerm
constructors when you want to be able to prove what arguments you are passing in.
As for hashing the constants, that would probably be kept as described in the CIP for constants inside MerkleTerm,
but the same otherwise.
This scheme however seems more complicated than what's described in this CIP, and in the end I have to run some benchmarks to see in practice how efficient Merkle hashing can get.
I will work on this when I find time, I suppose the CIP is blocked on that until then.

The structure of the Flat encoding is thankfully quite simple.

You may be aware of this already, but FYI there's a very detailed specification of the flat format as it's used for Plutus Core in Appendix E of the draft of the updated Plutus Core specification here.

I've thought about this for a bit, and come to the conclusion that the scheme described in the previous comment is in fact flawed.
The issue is that it needs to account for the worst-case, a user who actively uses as much "Merklisation" as possible.
However, I think the original scheme in the CIP is in fact not problematic.
There are two cases to consider:
When we attach a reference script to a UTXO, the hash verification can never fail, because the hash rather than being verified, can be computed from the attached script. The node can then cache the computation of the hash of this script and store it as part of the UTXO (if this is not already done).
The other case to consider is when we need to pass in a witness/concrete script that matches a script hash.
If the script comes from a reference script, all you need to do is a 32-byte equality check. This is trivial.
The other case to consider is when we pass in the script as part of the transaction itself.
This can fail. What's important to note, is that this is only needed in cases where 1) we need to execute the script and 2) the user chooses to use a reference script.
There are two broad fixes to this:

Disallow not using reference scripts. This would mean passing in the wrong script would cause 1 succeding transaction and 1 phase 1 failing transaction. You still pay fees for the first transaction.
Delay verification of the hash of the scripts included in the transaction to phase 2. Passing in an incorrect script would then cause you to pay the collateral. You can optionally fuse decoding/hashing/execution to further optimise it, but it's not necessary.

With this, AFAICT, performance is no longer a problem. I am fine with either of the above two solutions. 2) might be less disruptive, but 1) seems cleaner.
Thoughts? @kwxm @michaelpj and others

I'm not a huge fan of either of those:

Seems extremely disruptive and basically gives up on the original EUTXO model.
Makes script hashing uniquely special and expensive in a way that's likely to be a big pain.

Plus both of these would be quite a bit of work for the ledger. So it doesn't seem worth it to me.
I wonder if there's a more focussed change that would get you what you want. This changes the entire means of hashing scripts in order to find out whether a script is applied to something. We're not really using all that power! An alternative would be to have something like this:
data PlutusScript = JustScript ActualPlutusScript | AppliedScript ActualPlutusScript ActualPlutusScript

I don't particularly love this either, but I think it's at least worth thinking about alternatives that aren't so invasive.

I don't really know much about hashing so this question is more out of curiosity. Why is merkilisation better than allowing parameterized reference scripts (#354)? It seems that there isn't a consensus on whether the merkilisation can be done efficiently enough. Does anyone have an idea of what the resource cost would be to allowing a custom parameter in reference scripts?

@michaelpj That seems feasible. I don't see any issues with that design, and though it's effectively "1 level" of Merkle hashing, it seems to be powerful enough for almost all use cases? The use case this wouldn't cover, is passing in an arbitrary script on-chain, then applying a parameter to it, because that arbitrary script may already be using 1 level of Merkle hashing.
IMO though, checking the script hash should be phase 2 anyway, and so should anything that doesn't have to be in phase 1 (though this may be due to ignorance on my part).
I am however fine with implementing this proposed solution. It seems simple to implement.
One question that remains is, should the JustScript case pass the hash through? This would maintain backward compatibility.
The AppliedScript case can AFAICT just be hash (hash x <> hash y), since there is only one constructor with 2 fields.
In that case, do we need to increment the Plutus language version? Seems like a hard fork would be enough.

IMO though, checking the script hash should be phase 2 anyway, and so should anything that doesn't have to be in phase 1 (though this may be due to ignorance on my part).

I don't see why. All other hash checking is phase 1. Phase 2 only exists for script evaluation, because it's so expensive.

I am however fine with implementing this proposed solution. It seems simple to implement.

To be clear, that was a straw proposal. I don't really like it either. I just wanted to encourage the search for more ideas.

I had a long talk with @micahkendall , and I believe that what you proposed is more than sufficient, in addition to being the optimal solution.
Informal proof: Given any use of n-deep Merkle hashing for scripts, you can roughly turn it into 1-deep Merkle hashing by putting all added arguments into a list on the RHS in AppliedScript. You do lose some power, but not much. If your goal is to add extra logic, the elements in the applied list can be hashes the refer to other scripts. Of course, if you want to add to the list, you need the entire list in the transaction, but that is still much smaller than having the entire script.
One minor change, however, is that it probably makes sense to have it be
data PlutusScript = JustScript ActualPlutusScript | AppliedScript ActualPlutusScript Data

Morally, this applied argument is similar to redeemers and datums, and hence should be in the same format.
If redeemers and datums are changed to be of some other format, this should be changed to be of the new format too.
Everything in the motivation can still be done with this scheme. What do you think? @michaelpj

@L-as if this submission is not Abandoned then please post here how you intend to develop this idea.


apt::default tricked by old update-success_stamp

apt:default intentionally does not run apt-get update if /var/lib/apt/periodic/update-success-stamp is present.
This can be a bit of a problem if a system is being bootstrapped for the first time and apt-get update has not been run in some time.   This can often be the case with hosted providers such as digital ocean.
$ date
Sun Feb  1 14:34:12 PST 2015
$ cd /var/lib/apt/periodic
$ ls -al update-success-stamp
-rw-r--r-- 1 root root    0 Jan  8 13:33 update-success-stamp

The execute[apt-get-update-periodic] later in the recipe checks the timestamp of the file and should execute apt-get update if the mtime of the file is too old (24 hours by default).
It is not really clear to me why this didn't work for you. Could you possibly provide additional information, including the result of the guards checked for the various resources?

This appears to be WORKS_AS_INTENDED.   I caused the problem.  I configured build-essential to run during compile phase without configuring the apt with compile_time_update.


.NET Core generation support

Whilst in an ideal world, it would be great to both run CppSharp on .NET and generate code for .NET Core I think currently it would suffice to just be able to generate code that is compatible with .NET Core. It would be great to be able to support all the new platforms that are to be added (OS X, Linux, FreeBSD) in the Microsoft framework, meaning that we wouldn't always need Mono there when wanting to use a CppSharp bound library.

+1. What specific issues have you run into?

The primary issue I have noticed is that SuppressUnmanagedSecurity is
emitted.
On Mon, 7 Mar 2016, 16:41 Nathanael Jones<EMAIL_ADDRESS>wrote:

+1. What specific issues have you run into?
—
Reply to this email directly or view it on GitHub
https://github.com/mono/CppSharp/issues/629#issuecomment-193337705.

SuppressUnmanagedSecurity is unfortunately vital to us because it skips certain checks when calling into native code thus significantly increasing performance. In order to solve this, we need more details. For example, is the problem that SuppressUnmanagedSecurity causes some problem with .NET Core or rather that it doesn't exist at all? Is there an alternative to it? In case there is, is there a #define we can place in the generated code to detect .NET Core?

All P/Invoke's on CoreCLR effectively have this
https://github.com/dotnet/coreclr/issues/1605
On Mon, 7 Mar 2016 at 17:13 Dimitar Dobrev<EMAIL_ADDRESS>wrote:

SuppressUnmanagedSecurity is unfortunately vital to us because it skips
certain checks when calling into native code thus significantly increasing
performance. In order to solve this, we need more details. For example, is
the problem that SuppressUnmanagedSecurity causes some problem with .NET
Core or rather that it doesn't exist at all? Is there an alternative to it?
In case there is, is there a #define we can place in the generated code to
detect .NET Core?
—
Reply to this email directly or view it on GitHub
https://github.com/mono/CppSharp/issues/629#issuecomment-193351893.

If that is the only issue on .NET Core then we can easily fix it, though ideally we should also integrate CoreCLR on the CI system to test against it.

can we have a simple tutorial or example using a known c++ library to port it to .NET Core? If such a tutorial already available, please share the links

@JimSEOW: You can use https://github.com/ddobrev/QtSharp as an example, or all the other open source projects linked on the README file. And you can also take a look at our test suite.

Thanks @tritao, after extensive search through internet, I found (especially Japanese sites) offering step by step tutorials using the foo example. Thanks. cppSharp offers a much simpler experience than SWIG.

@benpye we still haven't fixed this issue after quite some time but there is a reason. Please feel free to use our e-mail for support if you would like to discuss this reason in more detail.

@tritao the fix for dotnet/coreclr#22087 should be in Preview 4.

Awesome thanks for the fix! 👍

To use the nuget package under netcore 2.2, it is needed to recompile this project and generate the output as nuget package?


Testing markdown link rename: prompt looks awkward

Testing #164522
When trying to rename something, I get this prompt:

I think that the really long first and third button make it sort of weird to look at (maybe it seems like the buttons have a lot to read?). Is this how we usually show prompts that have an always/never + yes/no question?
Feel free to close this if it's how it's supposed to be.

Duplicate of #164556


MSID: 89319 Version: 1 DOI: 2<IP_ADDRESS>5400

"msas": "Microbiology and Infectious Disease"
"msid": "89319"
"preprintDoi": "10.1101/<IP_ADDRESS>5400"
"Reviewed Preprint posted": "2023-08-21"
"Sent for peer review": "2023-05-26"
"Posted to bioRxiv": "2023-05-15" (link: "Go to bioRxiv": "https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>5400v2/")
[PLACE PDF URL HERE WHEN AVAILABLE] See step 7
Step 1. Inform bioRxiv
Who can help: @QueenKraken, @nlisgo, @scottaubrey

[x] Manuscript is in data hub index (https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v1/by-publisher/elife/get-by-manuscript-id?manuscript_id=89319)

Step 2. Create preview of manuscript
Who can help: @fred-atherden, @nlisgo, @scottaubrey

[ ] Manuscript is available for preview (https://prod--epp.elifesciences.org/preview/89319/v1)

[ ] Trigger import for staging: https://prod--epp.elifesciences.org/import
[ ] Sync data folder with s3://prod-elife-epp-data/data
[ ] Trigger import for staging: https://staging--epp.elifesciences.org/import
[ ] Sync data folder with s3://staging-elife-epp-data/data
[ ] Pull request created to enhanced-preprint-data
[ ] Post link to pull request below

Pull request: [PLACE LINK TO PULL REQUEST HERE]
Detailed instructions: https://github.com/elifesciences/enhanced-preprints-data#add-a-manuscript
Step 3: Awaiting public reviews and QC
Who can help: Production team

[ ] QC passed or QC not needed

Who can help: Editorial team

[x] Public reviews available on sciety (https://sciety.org/articles/activity/10.1101/2<IP_ADDRESS>5400)
[x] Manuscript and editor details at the top of this issue (Supplied by Editorial team)
[x] Reviews returned by EPP - https://prod--epp.elifesciences.org/api/reviewed-preprints/89319/v1/reviews

Example
"msas": "Genetics and Genomics", "Neuroscience"
"msid": "84628"
"version": "1"
"preprintDoi": "10.1101/2<IP_ADDRESS>4241"
"articleType": "Reviewed Preprint"
"status": "Published from the original preprint after peer review and assessment by eLife."

"Reviewed Preprint posted": "2023-01-02"
"Sent for peer review": "2022-10-28"
"Posted to bioRxiv": "2022-11-21" (link: "Go to bioRxiv": "https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>4241v1")

Editors:
Reviewing Editor
Michael B Eisen
University of California, Berkeley, United States
Senior Editor
Michael B Eisen
University of California, Berkeley, United States

Step 4: Modify manuscripts.json (no PDF)

[ ] Reviewed preprint is published to EPP (https://elifesciences.org/reviewed-preprints/89319)

[ ] Pull request created to enhanced-preprint-client
[ ] Pull request switched to "Ready for review"
[ ] Post link to pull request below

[ ] Request that a doi be registered

[ ] Post link to #enhanced-preprint comment below

Pull request: [PLACE LINK TO PULL REQUEST HERE]
#enhanced-preprint comment thread: [PLACE LINK TO COMMENT HERE]

Instructions to modify manuscripts.json

Visit: https://github.com/elifesciences/enhanced-preprints-client/actions/workflows/publish-manuscript.yaml
Click: Run workflow
Complete the form and click "Run workflow"
A successful run should result in a new pull request at https://github.com/elifesciences/enhanced-preprints-client/pulls
Open the pull request and click the "Ready for review" button to trigger tests
Once the tests pass and you are happy with the changes the PR can be merged

Example pull request: https://github.com/elifesciences/enhanced-preprints-client/pull/334/files
Once the pull request is merged in it should be available a few minutes later.

Request that a doi
Post the following in #enhanced-preprint:
@Fred can you register a doi for https://elifesciences.org/reviewed-preprints/89319

Step 5: Awaiting search reindex

[ ] Reviewed preprint is avaliable on journal homepage (https://elifesciences.org)

The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.

Additional info
If needed, the jenkins pipeline to reindex search can be triggered sooner.
https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 6: Published! Request PDF generation

[ ] Post the link to the PDF url at the top of the issue

[ ] Post link to #enhanced-preprint comment below

#enhanced-preprint comment thread: [PLACE LINK TO COMMENT HERE]
Post the following to the #enhanced-preprint on slack:
@Ryan Dix-Peek please can you generate a PDF for https://elifesciences.org/reviewed-preprints/89319

Step 7: Introduce PDF to data folder and git repo

[ ] PDF is avaliable at https://github.com/elifesciences/enhanced-preprints-data/raw/master/data/89319/v1/89319-v1.pdf

Detailed instructions: https://github.com/elifesciences/enhanced-preprints-data#add-a-pdf
Step 8: Add PDF url to manuscripts.json

[ ] Reviewed preprint PDF is available for download (https://prod--epp.elifesciences.org/reviewed-preprints/89319)

[ ] Pull request created to enhanced-preprint-client
[ ] Pull request switched to "Ready for review"
[ ] Post link to pull request below

[PLACE LINK TO PULL REQUEST HERE]

Instructions to add PDF url to manuscripts.json

Visit: https://github.com/elifesciences/enhanced-preprints-client/actions/workflows/add-pdf-url-to-manuscript.yaml
Click: Run workflow
Complete the form and click "Run workflow"
A successful run should result in a new pull request at https://github.com/elifesciences/enhanced-preprints-client/pulls
Open the pull request and click the "Ready for review" button to trigger tests
Once the tests pass and you are happy with the changes the PR can be merged

Example pull request: https://github.com/elifesciences/enhanced-preprints-client/pull/397/files
Once the pull request is merged in it should be available a few minutes later.

Step 9: Done!

[ ] Kettle is on!

The details at the top of this ticket use the msid 89947, @KJT08 is that a mistake, and if so, are the other details all still correct or do they need updating as well?
For 89319, the image quality of the graphical abstract is really poor and broken:

It's the same in the PDF at bioRxiv:

But oddly not the same in the PDF in eJP. We also have the original jpeg image, showing that in addition to poor quality, a lot of the cell images are missing:

@acollings (not urgent - for when you are back), I'm not sure what's gone on here but I suspect we need to go back to the authors and ask them to update the version on bioRxiv before we can proceed with this one. Or would that cause problematic confusion at this stage?

@fred-atherden Apologies, the msid 89947 is wrong it should be 89319, all the other details are correct.

Great - thanks Karen.

Thanks @fred-atherden. If it's only the graphical abstract, and as it was posted like this by the authors, I'd be comfortable proceeding. However, I'd be happy to ask the authors to update their preprint, if it would help, so just let me know! Cheers, Andy

Will do, @fred-atherden!

Many thanks!

From the authors: I confirm that I have uploaded a revised version of the preprint, improving the quality of the graphical abstract.
@fred-atherden FYI : I guess we need to wait for V3 to appear (I can't see it yet!)

Nice one - thanks Andy. I'll check in with this one in the next day or two

V3 full text is available but the MECA package isn't yet:
curl 'https://api.biorxiv.org/meca_index/elife/all/2<IP_ADDRESS>5400' | jq
{
  "status": [
    {
      "count": 1,
      "messages": "ok"
    }
  ],
  "results": [
    {
      "msid": "535400",
      "tdm_doi": "10.1101/2<IP_ADDRESS>5400",
      "ms_version": "2",
      "filedate": "2023-05-15",
      "tdm_path": "s3://transfers-elife/biorxiv_Current_Content/May_2023/16_May_23_Batch_1613/9cc0713b-6c78-1014-bbfa-a46f51a53a35.meca",
      "transfer_type": "api"
    }
  ]
}

If it's not available tomorrow, I'll contact bioRxiv


Add ability to control skipping Deps and Repos separately

Right now we have only one flag which passed to both variables, which isn't flexible as we would like to.
Main problem in our case that we are using Helmfile with envs. In each env we can have different version of remote chart. To apply that configuration in parallel we would like to split repos sync and deps build into 2 stages.
Repos sync can't be executed in parralel because it cause helm locks and context deadline exceeded exceptions.

@kuzaxak this is a valuable feature, why was the PR closed?

this is a valuable feature

I found root cause for the issue we had. Problem with helm dependency cache itself.
We are changing env variable to run helmfile in parallel.
script.dir(PATH) {
      script.withEnv([
        "HELMFILE_ENVIRONMENT=${deployment.directory}",
        "HELM_REPOSITORY_CACHE=${cacheFolder}/cache",
        "HELM_REPOSITORY_CONFIG=${cacheFolder}/config.json",
      ]) {
        sh("helmfile -e ${deployment.name} diff")
      }
    }


Fix deprecation warning.

Fixes #50

Codecov Report

Merging #55 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #55   +/-   ##
=======================================
  Coverage   86.01%   86.01%
=======================================
  Files          19       19
  Lines         944      944
  Branches      175      175
=======================================
  Hits          812      812
  Misses         96       96
  Partials       36       36

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a726be5...ce8877f. Read the comment docs.


EntityHasComponentsCard should be able to show aggregated components

Feature Suggestion
On the EntityPage of a Domain, I would like to use the EntityHasComponentsCard (or some other similar card) to list the Components which are part of the Systems in this Domain.

Possible Implementation
Perhaps the EntityHasComponentsCard could take the relationsType="aggregated" prop like the EntityOwnershipCard does?
Context
When learning about a new Domain, it's nice to be able to browse through the Components easily so you can understand what they do, and thus learn more about what is in the scope of this Domain.
It is possible to click through the Systems individually, but that may take some time.

Think we probably want to mock this up a little bit with some examples. Does it make sense to group them in the table by System?


Please review.


Modify Rosetta to support Expresso Hardfork

Expresso HF modifies the rules of block processing, in particular it introduces the block context on which gas price minimun is only computed at the beginning of the block (vs. after every tx).
Rosetta mimics this behaviour and needs to be modified to handle both options (pre and post hardfork) and have an activation block that should be configurable.
(we can have hard coded config options for alfajores, baklava and mainneet)

After initial research by @YoduYodu we figured there's no need to modify rosetta. only need to upgrade node version on the docker file


Improvements the mouse and keys of the MenuBar.

I think this https://github.com/migueldeicaza/gui.cs/pull/596/commits/ce30a837642cc72aef6433e9322a6c4970c13028 fixes #436.

We probably need a better description of what this change does on the description, it is hard to judge the changes without it.

Without merge with https://github.com/migueldeicaza/gui.cs/pull/596/commits/4a1f5a9309cf10d45d13c3b7d87ba7ceb8303a6a Alt key doesn't work in the Menu.

@BDisp I'm very confused about what this PR does and doesn't do.
It makes a lot of changes that are hard to follow in the diff. Can you try to explain the problem and the fix more clearly?

@BDisp I'm very confused about what this PR does and doesn't do.
It makes a lot of changes that are hard to follow in the diff. Can you try to explain the problem and the fix more clearly?

OnKeyUp and OnKeyDown in reality I reset them for what remains the same because my intention was to activate the Menu right after pressing and not when releasing the key but this is not effective because it activates right when switching between windows and leaves the Menu activated no need. At the mouse level, I activated both ButtonPressed and ButtonDoubleClicked to open and close the Menu more quickly. Now it works alternately, that is, if the Menu is closed both events will open and close it, otherwise. There is another commit that also solves a problem in which if you open the "Text Editor Demo" in the Example project, pressingCtrl + Qwould trigger the Application.Top Quit method that does not belong to the Toplevel current.

@tig here is a video explaining what I was meaning about the Quit that belongs to the Application.Top is running in the Text Editor Demo. If I choose "Ok" to quit after I close the editor the application exits.

You are awesome. Thank you!


ComponentNotFoundException: Failed to load the component 'EntityExtractor'. Cannot find class 'EntityExtractor' in global namespace. Please check that there is no typo in the class name and that you have imported the class into the global namespace. Either your pipeline configuration contains an error or the module you are trying to import is broken

Rasa version: 2.4.0
Rasa SDK version (if used & relevant):2.4.1
Python version: 3.7.0
Operating system (windows, osx, ...):windows
Issue: ComponentNotFoundException: Failed to load the component 'EntityExtractor'. Cannot find class 'EntityExtractor' in global namespace. Please check that there is no typo in the class name and that you have imported the class into the global namespace. Either your pipeline configuration contains an error or the module you are trying to import is broken (e.g. the module is trying to import a package that is not installed). Traceback (most recent call last):
ImportError: Cannot retrieve class from path EntityExtractor.
Error (including full traceback):
Content of configuration file (config.yml) (if relevant):
# https://rasa.com/docs/rasa/nlu/components/
language: en
pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: EntityExtractor
  - name: EntitySynonymMapper
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100

# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
  - name: MemoizationPolicy
  - name: FormPolicy
  - name: TEDPolicy
    max_history: 3
    epochs: 200
  - name: MappingPolicy
  - name: FallbackPolicy
    nlu_threshold: 0.3
    core_threshold: 0.4
    fallback_action_name : "action_Custom_fallback"

@Sara-tagger @tjpalmer @sheldon @fungiboletus

Thanks for raising this issue, @amn41 will get back to you about it soon✨
Please also check out the docs and the forum in case your issue was raised there too 🤗


(analytics) add table records, loaded after initial results, to the results object

This commit fixes an issue where, in the ag-grid, records loaded after the initial results wouldn't be selectable or cannot be added to the collection or any other functionality requiring selection.

There is an issue with this fix: the datasource might request you for records at an arbitrary page (like, go from page 1  directly to 100), so you cannot just append these new records at the end of searchService.results.records, that would be inconsistent.
What is the issue with selection exactly ? Are you using storage = 'id' or storage = 'record' for your selectionOptions? (cf https://github.com/sinequa/sba-angular/blob/c537f46c5c0107efbd30d0a709f6465d6cf866da/projects/components/selection/selection.service.ts#L30)

I see, it makes sense.
The issue is that records are not selected when using the select all button. I am using the storage = 'record'.

The "select all" button is problematic because there is ambiguity about what "all" means: is it the current page of results ? or is it the entire result set (basically any document returned by the WHERE clause). It's hard to do something in between...
In the case of AG Grid, this is all the more an issue because the concept of "results page" is hidden.
One possibility is to just remove the "Select All" button, and let users select documents manually. If there is a need for selecting all the results set, then maybe go for a custom implementation (instead of tracking documents 1 by 1, track the WHERE clause?).

Removing the "Select All" button is not a possibility for us unfortunately. This is one of the specification that was accepted and we have to have a select all option.
Now unfortunately, tracking the WHERE clause would work if my goal was only the download of all documents and we did not care of the visual impact on the interface, but we need to have the documents selected in the grid, which is only possible through the ag-grid, because they are only accessible there.
Now, how about keeping the part of the code I added, but adding a check on the rank of the records before adding them to the results.records object? The rank should give us the position of the records compared to other records, so I think we could use that to order them.

That solution makes sense. My concerns are with the potential sides effects:

By mutating searchService.results, there could be some impacts on other views/components that take the results as an input (I think the risk is reasonably low).
If you allow many documents to be selected and using "storage = 'record'", the list of selected records could grow pretty big in memory (1000 selected docs x 1MB = 1GB...), so watch the performance
When the user scrolls to arbitrary locations, there will be "holes" in the list of records (like if you jump from page 1 to page 10, then pages 2-9 are missing): I wonder if this could cause some modes of selection to fail, like when you hold SHIFT to select a range of docs (ie. does AG Grid allow you to select records from page 1 to page 10 with this method?)

Anyway, let's keep the PR open, and feel free to update it with your solution. I am sure this problem will come up again, so this discussion will be very helpful. If all the risks are cleared, I'd be happy to merge it.

Regarding the points you brought up:

I am not too concerned about other views/components being impacted, but I'll keep an eye open to make sure of it
This might be concerning indeed, I'll run some quick tests to check the memory capacity/performances when using ag-grid
For now I did not notice any difference in the selection between my modified version and the original version. The SHIFT selection mode works the same way in both cases, and the select all only works in the modified version. Although, the SHIFT selection mode is behaving strangely when skipping some pages, in both versions: selecting records before the skipped pages, then trying to select after the skipped pages only works after "reselecting" the first record to start from the SHIFT selection in the pages after the skip. If you want to discuss that I can open a bug report.
It might need clarification at first, it is a good point. I'll make sure to bring this to my team and we'll include this detail in the user manual or bring it up directly with the customer

Alright, I'll keep the change on my side while we try to resolve the points we talked above. If you find anything else that might be shaky with this modification, let me know!

About the burst in memory: AG-Grid only creates DOM elements for the rows/columns that are visible on the screen (even if it has 2000 records loaded in memory), so at most a few dozens of rows and columns. But when you switch to a normal result list, Angular just generates as many DOM elements as there are records, so it's very costly.
Resetting the records when you leave the AG-Grid view is a good idea!

I see, I understand now why I am not seeing any burst while using the AG-Grid.
I was also wondering, rather than forcing this update on everyone using the AG-Grid we could make it optional via an @Input, but this also requires to add a parameter to the constructor of the datasource.ts.

ag-grid-view.component.ts
/** Activates the possibility to select all the records in the AG-Grid using the SelectionService
* Please monitor performances when setting to true. This option saves the newly loaded records
* to the results object, so it can become costly in memory usage */
@Input() activateSelectAll: boolean = false;
...
makeDatasource() : IDatasource {
if(this.results) {
return new SqDatasource(this.results, this.query, this.colDefs, this.searchService, this.appService, this.facetService, this.selectionService, this.activateSelectAll);
}
return {getRows: () => []};
}

datasource.ts
constructor(
public results: Results,
public query: Query | undefined,
public colDefs: ColDef[],
public searchService: SearchService,
public appService: AppService,
public facetService: FacetService,
public selectionService: SelectionService,
private selectAllActive: boolean = false
)

Did you have a chance to take a look at the changes I pushed?

Hi Guillaume,
Sorry for the delay. Your implementation looks fine, but I had some concerns about the user experience, so I am curious to hear your feedback. Did you deploy these changes to users? Are they happy with it and not confused by the behavior of the select all button?
One small implementation detail: rather than completely clearing the results in OnDestroy, I would rather truncate the list to the first pageSize, as it's possible to switch results views without triggering a new search.

Hi Eric,
The change has not been tested by users yet, this is a functionality developed for a project not yet in production. Though, the functionality has been made optional with my last push, and it is off by default.
Oh, you're right indeed, I'll push this fix next week as I am off until Tuesday.

Hi Eric, did you have a chance to get a look at the changes yet?

Hi Guillaume,
Yes, I did and I think your code is fine from a technical point of view. I am just worried about the side-effects we discussed above. If you tell me that you tried it in prod and users are happy, then let's merge this.

When I first pushed the changes for this pull request, everything was done on my master branch, which left me unable to update it since it was always on standby here. I have now created a dedicated branch for the ag-grid changes, and will create a new pull-request once I have sufficient user feedback.


update to run with stable mongo 4.0

@dzunftmeister-evorhei Sorry, I didn't catch this.
I'll check this all out right now and get it merged & pushed.  I'll likely still keep this as a "beta" for the next release until I'm fully confident after running in production.  Currently refactoring current production code-base to use updated dependencies.

@dzunftmeister-evorhei dependencies updated in v2.0.0-beta.10.
The new MongoDB driver has pretty strict typings now, so let me know if you come across any issues and I can try to loosen it on this library's end.


Any way to route specific traffic to a external proxy

Dear Community,
We have got the requirement to route specific traffic - based on the domain name via an additional external proxy.
Example
client request 01: https://my-api.internal.com/hello          --> should be routed directly
client request 02: https://other-api.external.com/hello     --> should be routed via an additional (external) proxy
Is there any way to achieve this with Envoy?
We already tried to use "extensions.filters.network.tcp_proxy.v3.TcpProxy" - but failed as it seems that it allows only one entry (so one proxy for all TCP Traffic - or not proxy at all)

For this you want two virtual hosts, and two static clusters.
vhost 1 is for my-api
vhost 2 is for other-api
You will reach each vhost depending on the Host/Authority header your client sends. You can test this with curl like follows
curl localhost:8099 -H "Host: other-api.internal.com" -v
cluster 1 and 2 is effectively statically routed from each vhost.
This is using cleartext http, converting to tls is left as an exercise for the reader :)
Example config is here:
static_resources:
  listeners:
  - name: http_connect
    address:
      socket_address:
        protocol: TCP
        address: <IP_ADDRESS>
        port_value: 8099
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: "ingress_http"
          route_config:
            name: test
            virtual_hosts:
            - name: my-api
              domains:
              - "my-api.internal.com"
              routes:
              - match:
                  prefix: '/'
                route:
                  cluster: my-api
            - name: other-api
              domains:
              - "other-api.internal.com"
              routes:
              - match:
                  prefix: '/'
                route:
                  cluster: other-api
          http_filters:
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  clusters:
  - name: my-api
    connect_timeout: 0.25s
    type: STATIC
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: my-api
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: <IP_ADDRESS>
                port_value: 80
  - name: other-api
    connect_timeout: 0.25s
    type: STATIC
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: other-api
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: <IP_ADDRESS>
                port_value: 80

Hi @nbaws:
Thanks for the quick reply!  Highly appreciated!
One thing were I still struggle:
At the end of the day I would like to do a 'standard curl' - and thus use the 443 port.
E.g.:  curl https://www.google.com
I tried to add your filter shown above to an exiting 'listeners' (have a standard Istio installation) -
but somehow I failed.
Can you give me a hint where I need to place / add the filter ? (to which listener / to which context?)

Take a look at this set of configurations for TLS: https://www.envoyproxy.io/docs/envoy/latest/start/sandboxes/tls
Your main issue will be how you want envoy to be presented to the client. Either it is presented as a http proxy, or as the real host (with appropriate certificate/s). The previous configuration is only dependent on the Host:/Authority: header being correct when it lands on Envoy. However TLS will complicate this as it will not validate correctly on your client unless the certificate being presented matches what your client is expecting to connect to.

@nbaws:
Thanks a lot!


Better setup of integration testing

We should have a npm command to run integration tests from everywhere

Done by #101


fix: added sanity check to avoid unexpected eval error

Description
There was an unexpected error during evaluation because of a missing config for APIs. Added sanity check to validate prroperty path only if it has config
Fixes #12454
Type of change

Bug fix (non-breaking change which fixes an issue)

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Test coverage results :test_tube:
:red_circle: Total coverage has decreased
// Code coverage diff between base branch:release and head branch: promises-prod-issue

Status
File
% Stmts
% Branch
% Funcs
% Lines

:red_circle:
total
55.72 (0)
37.14 (-0.02)
35.44 (0)
55.98 (0)

:red_circle:
app/client/src/utils/autocomplete/TernServer.ts
52.71 (-0.23)
40.83 (-0.84)
36.21 (0)
56.74 (-0.25)

:red_circle:
app/client/src/workers/DataTreeEvaluator.ts
75.81 (-0.12)
56.37 (-1.1)
73.86 (0)
75.37 (-0.13)

/ok-to-test sha=ff1cb7e

/ok-to-test sha=ff1cb7e


bug auto_planting and double_mob_drops

The items in the monster's hands will be copied, and this should be filtered.
auto_planting
The player's tools will be repaired after startup.
double_mob_drops
The items in the monster's hands will be copied, and this should be filtered.

Please fix these bug


Fix for issue#662

Proposed changes
Added logs in src/wq/core/task.py. fix for issue #662
Types of changes
What types of changes does your code introduce to the project?
Put an x in the boxes that apply

[ ] Bugfix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Maintenance (update of libraries or other dependences)

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[x] I have read the CONTRIBUTING doc
[x] I have signed the CLA
[ ] I have added tests that prove my fix is effective or that my feature works
[x] I have run all the existing tests locally (not just those related to my feature) and there are no errors
[x] After the last push to the PR branch, I have run the lint script locally and there are no changes to the code base
[x] I have updated the RELEASE NOTES
[ ] I have added necessary documentation (if appropriate)
[x] Any dependent changes have been merged and published in downstream modules

Further comments
If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...

Hi, @c0c0n3 @chicco785 when i run the tests locally all tests were passed but in the CircleCi the test_others tests are failing. Sometimes after running so many times, it will pass automatically.

@Necravisaketi see my comment on #661. It seems bizarre, but it looks like adding log statements to WQ causes that error? Can't believe it, but can we try getting to the bottom of this?

@Necravisaketi actually I'm wrong about test_others failures being related to this PR. If you look at #660 you'll see there's no code changes in there, but test_others still failed. I've opened a separate issue about it: #666. Meanwhile we can ignore the failing test here, rebase on master after merging #661, then merge this PR. Thanks alot!

@Necravisaketi can you please run the linter on locally on your branch, see what files it changes and figure out what goes wrong? thanks alot, much appreciated!

@Necravisaketi it looks like this PR is good to go? There's a couple of issues but I don't think they've got anything to do w/ your code:

reporter test failing. we talked about this already, see my other comments. definitely not related to your code change.
linter failing. I saw your latest commit to fix the linter issue and I think the issue is actually fixed, thanks! the problem is the github action can't run cleanly for some reason. I supposed we can ignore this.

Should we merge or would you still like to add more code?

Hi, @c0c0n3 I have done from my side for this issue and there are no conflicts with these changes so we need to merge this pr.

good stuff, thanks alot @Necravisaketi!


[WIP] Add prompt tuning module

Description
Add a prompt tuning module, which provides functions:

System prompt generation (with few-shot examples)
System prompt tuning

Checklist
Please check the following items before code is ready to be reviewed.

[ ]  Code has passed all tests
[ ]  Docstrings have been added/updated in Google Style
[ ]  Documentation has been updated
[ ]  Code is ready for review

A new PR will be added. Closing this for now.


Interpret color based on player's in-game choice.

This is definitely not the best implementation of #17, as I'm not sure how I should share the setup of the database (generateTreeFromDatabase). It would also be great to be able to color the player arrow too, which would involve mutating PlayerArrow too. First pass:

Nevermind that, here's player color too.

@rgbkrk Looks good! For the sharing of the database tree: I'm thinking about ways to optimise this, so it doesn't need regenerate the tree each time it gets accessed. So I not only want to move this step to the reducer, but make some kind of improved diffing.

Sounds good, would love to optimize the sharing of the database tree. Most components are only going to need 1 or 2 fields most of the time (well, top down even more). The pipboy colors should probably be part of the central store.


Query DRY up - one more sweep

coverage: 93.503% (-0.001%) from 93.504%
when pulling e38db469de3baa41a1fd8218aa4e3492a796d05c on standardize-controller-indexes
into aeb1765ed2995f2edf62c1f3ba695fe23c2353dd on main.