Feature req: Partitioning actions to execute on separate nodes

If actions can be partitioned like sensors we can have actions and sensors on separate instance in may be separate n/w zone where it can perform actions on nodes in that zone and report result to main instance in less secure zone where other services like WebUI and Slack are available.

Thanks - yes this is a good feature request and something we have already discussed in the past (runner affinity, would come especially handy for Python runner actions, etc.).
We don't have an ETA for it right now, but I do think it's something we will look into it in the future.

My use case is an action-chain workflow where task1 writes a file and task2 uses that file. Our stackstorm cluster uses 4 nodes so task2 can't see the files created by task1 when the tasks run on different nodes. More concretely, task one calls openssl to write out a key and CSR. Task 2 emails those files to the requester.
A couple workarounds are to (a) use a shared filesystem or (b) pass everything as variables instead of files. Right now I'm trying to rework the workflow for (b) but haven't actually finished it yet.

👍 Need it!

Wanted to add my use case for this and maybe some other features that may follow as a result.
Use Case 1 : Multiple Environments Same Datacenter
In a given datacenter we maybe have N environments present. Currently we are forced to deploying in one of two ways:

A shared deployment where all actionrunner instances have access to all machines in every environment because we don't know which actionrunner will receive the tasks.

Individual/isolated deployments. For some environments where we are not allowed to share infrastructure (usually for security reasons) we have another full StackStorm deployment where the actions are copied.

We would like to minimize the security impact of option 1. by only requiring a small number of instances (maybe 1-2) to have access to specific environments. This also might help convince security that types type infrastructure would be OK to serve some of our more sensitive areas.
Use Case 2 : Multiple Datacenters
When deploying a hybrid cloud or multi-datacenter fashion we currently have the same two options as above.

Have a single StackStorm deployment in an arbitrary datacenter, then perform all actions in other datacenters over the WAN. For datacenters that are geographic close this can be OK, but not ideal. For datacenters that are across country over overseas this incurs a pretty big performance hit by running SSH, WinRM and HTTP calls over the WAN.

Have a isolated StackStorm deployment in every datacenter.  This solves the latency and throughput problem by having resources and workers local to the datacenter where actions need to occur. However it adds a great deal of complexity in terms of workflow design and message routing logic. Now all workflows need to start out by determining where the API call(s) and resources reside, then branching and either running the workflow locally or invoking that workflow on the StackStorm instance in another datacenter. This can compound when part of the workflow needs to be run in Datacenter A and part in Datacenter B then wait for both of those results, then run a final part back in Datacenter A again.

Ideal World
Personally i would like to only manage one StackStorm instance/cluster and spread out components as needed to various datacenters and environments. I know this comes with its own set of challenges for both StackStorm and the components underneath StackStorm and Mistral (rabbitmq, mongodb, postgres, etc).
Maybe if somehow multiple StackStorm instances could "cooperate" with each other in a more seamless fashion that might help a little. However, there is still the drawback that things like the datastore would need to be synchronized across deployments.

Need it too!
I think about a solution reposed on a centralized instanciation of Stackstorm with several "workers" deployed where we want as long as they can join st2api, RabbitMQ and Postgres servers.
My use case is almost the same as use case 2 of @nmaludy.
I actually need to deploy some "workers" on a specific datacenter with specific network access and I don't want to deploy there all Stackstorm services.

Related note - the single-sensor-per-container mode lets you have a form of affinity for sensors. https://docs.stackstorm.com/reference/ha.html#st2sensorcontainer

Need it also.
I have a stackstorm high availability setup, with stackstorm installed on two virtual machine nodes.
My use case is that I want to run a pack install on both HA nodes. I want to install a stackstorm pack on both HA nodes. The problem is that when I run pack install, it puts an execution request on the rabbitmq through the API. Then one of the two HA nodes picks it up and runs it, and not necessarily the node that I want to run it on. This installs the pack (and therefore the pack’s files) on one of the nodes, but not necessarily on both.
I want to be able to put a action on the queue and force stackstorm to make it run on a particular node so that I can force the pack to install correctly on both.

Pack install has a few other things to cover. I think that "want st2 pack install to work in HA environment" should be treated as a separate issue to "I want to be able to run an action on a specific action runner for networking or security reasons"

We are also exploring this option (node affinity based action execution) and would love to have it.
Our use case is similar to the above mentioned ones - we have multiple datacenters and we would ideally want to manage single Stackstorm instance with deployments in the different datacenters, where we can run actions specific to a given datacenter (for latency/performance gains and availability, not security concerns).
I've tried achieving this by "merging" only the mongodb service between the different instances, where I thought that this would give us common configuration (pack config/kv datastore etc.) and history between the instances, but still will allow us to target specific instance by using the API of specific node when running actions/workflows (as RabbitMQ is "local" for the datacenter instance).  In practice though, we've noticed that even if we fire an action through API of instance A, it could eventually be picked up and executed by runners in instance B, which would mean that the initial request for action execution traverses through mongodb - maybe someone with more in depth knowledge around the internal Stackstorm communication between its components can explain why this happens.

and would love to have it

I don't know of anyone else working on this, or anyone planning on doing so. PRs welcome. That's the only way it's going to get implemented. Probably better to open an issue with a proposed design first, before doing too much work. There's a few different ways this could be done, and I'm sure people will have opinions on how to do it.

I've tried achieving this by "merging" only the mongodb service between the different instances, where I thought that this would give us common configuration (pack config/kv datastore etc.) and history between the instances,

I'm fairly sure the internal system design never considered having a combined DB but separate instances of everything else, especially RabbitMQ. Not surprised by the results you got

I would like to participate in a design call here.

On May 2, 2019, at 5:18 PM, Lindsay Hill<EMAIL_ADDRESS>wrote:
and would love to have it
I don't know of anyone else working on this, or anyone planning on doing so. PRs welcome. That's the only way it's going to get implemented. Probably better to open an issue with a proposed design first, before doing too much work. There's a few different ways this could be done, and I'm sure people will have opinions on how to do it.
I've tried achieving this by "merging" only the mongodb service between the different instances, where I thought that this would give us common configuration (pack config/kv datastore etc.) and history between the instances,
I'm fairly sure the internal system design never considered having a combined DB but separate instances of everything else, especially RabbitMQ. Not surprised by the results you got
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Set something up. Write up some proposals, figure out a suitable forum for discussion. If that means a call, figure out a time that works for interested parties.

Been a few years on this topic. Kind of curious if there was any ideas or designs proposed somewhere?

Hello, it's been a few more years on this. Is this in development by anyone and/or has anything been posted regarding it?


Flask app silently fails when querying with a big number of variantsets

More details to come

So after some investigations, it has been found that when there's more than ~50 generators combined with the itertools.chain_from_iterators method, the subsequent conversion to list failed, not sure what exactly was happening here.
The temporary solution for now is to wrap the generator in a list at the time of it being generated, iterators.append(list(paging.VariantsIntervalIterator(request, item))) in the related generators. This works because the itertools.chain actually does combine multiple lists. Ideally we would want to the generators to work properly, but if not then this is not an unacceptable workaround.

Update - seg fault related

Refer to SYS-5.4|0*4-7 for further details

PR merged. Closing issue


⚠ File access (beta) has degraded performance

In ec290bf, File access (beta) (https://beta.fyle.uk/file/fish) experienced degraded performance:

HTTP code: 200
Response time: 3023 ms

Resolved: File access (beta) performance has improved in 9968e4c.


Fix 787

Need PR #668 to merge.
Not sure this will need any unit test ?
Maybe just making sure this is launching ? Is this useful ?

The unit test could be as simple as taking one of the existing unit tests for the median filter and make it accessible from image.

this PR has been modified by the PR  #803


Client Encryption: Adds fix for Patch operation to use the correct encryption policy and modifies operation retry behaviour.

Description
This PR brings in the following changes and fixes

changes the retry logic where few operation(create/read/upsert etc) were retried upon failure due to policy changes. With this PR this retry has been removed and we just refresh the policy. This is to align with and have a uniform bahaviour across since few other operations like batch, feed cannot be retried.

adds fix for patch operation to pick up(and refresh the cache with) the correct encryption policy when the client has cached incorrect/stale policy which can happen when container is deleted and recreated with same id but with a different encryption policy.

Type of change

[] Bug fix (non-breaking change which fixes an issue)

closes #2714

            streamPayload.Position = 0;

Still needed?

Refers to: Microsoft.Azure.Cosmos.Encryption/src/EncryptionContainer.cs:872 in a1cfd32. [](commit_id = a1cfd327a10bbde0522b2f27539a7d69f8cc3d64, deletion_comment = False)


Icon request: Fandom

URL for the brand / site / app
https://www.fandom.com
Logo resources for the icon
https://fandomdesignsystem.com/?path=/story/assets-brand--brand
Proof of the Aegis-compatible 2FA
https://community.fandom.com/wiki/User_blog:DToast/Arrived:_Two-Factor_Authentication_for_Fandom_Accounts
Anything else?
No response
Confirmation

[X] I have read and followed the issue guidelines & issue form.

Filled by #874
https://github.com/aegis-icons/aegis-icons/blob/master/icons/1_Primary/Fandom.svg

:question: How to add icon(s) to Aegis?
:star: Support this project by giving a star! Just adding like to this comment is also appreciated.


Could not verify LLVM version with Emscripten on NixOS

When trying to compile the minimal example from stdweb to emscripten target, I get the following error:
cargo web start --target=wasm32-unknown-emscripten
error: linking with `emcc` failed: exit code: 1
  |
  = note: "emcc" "-s" ...
  = note: CRITICAL:root:Could not verify LLVM version: [Errno 2] No such file or directory
          CRITICAL:root:fastcomp in use, but LLVM has not been built with the JavaScript backend as a target, llc reports:
          ===========================================================================
          (no targets could be identified: [Errno 2] No such file or directory)
          ===========================================================================
          CRITICAL:root:you can fall back to the older (pre-fastcomp) compiler core, although that is not recommended, see http://kripken.github.io/emscripten-site/docs/building_from_source/LLVM-Backend.html
          INFO:root:(Emscripten: Running sanity checks)
          ERROR:root:failing sanity checks due to previous fastcomp failure

(some output elided)
My best guess is that emscripten can't find llvm (It should be discoverable at least through pkg-config).
Additionally, I noticed cargo-web downloaded its own version of emscripten.
I believe providing my own version of emscripten would solve the issue (from my system's packages), how can I do that?

Misc information:
❯ cargo web --version
cargo-web 0.6.26

❯ cargo --version
cargo 1.42.0-nightly (86134e766 2019-12-23)

❯ llc --version # This is from my system, I'm pretty sure it's not found
LLVM (http://llvm.org/):
  LLVM version 7.1.0
  Optimized build.
  Default target: x86_64-unknown-linux-gnu
  Host CPU: sandybridge
...

❯ rustup target list
...
asmjs-unknown-emscripten (installed)
...
wasm32-unknown-emscripten (installed)
wasm32-unknown-unknown (installed)
...
x86_64-unknown-linux-gnu (installed)
...

❯ emcc --version # From my system, I don't think it was found by cargo-web
emcc (Emscripten gcc/clang-like replacement) 1.38.28 ((unknown revision))

❯ uname --all
Linux ... 4.19.91 #1-NixOS SMP Sat Dec 21 09:57:45 UTC 2019 x86_64 GNU/Linux

This error is the same regardless of what I try to compile. I gave the minimal example for illustration and reproducibility, but I can actually reproduce with any other example project and a few other projects.
Also of note is that the wasm32-unknown-unknown target successfully compiled for every example I tried.

You can try to run cargo web build --use-system-emscripten to use your own emscripten version. It's in the help if you check cargo web build --help

Thank you, I was reading the source code as well and I just stumbled on that flag!
I was thinking it should be more prominent in the docs (I don't know why I didn't think of checking the --help)
I'll test that and post back here.

Well it's not the same error so that's progress I guess?
❯ cargo web build --use-system-emscripten --target=wasm32-unknown-emscripten --verbose
...
error: linking with `emcc` failed: exit code: 1
  |
  = note: "emcc" "-s" "DISABLE_EXCEPTION_CATCHING=0" "-L" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.17qvp3wvy3ge36bp.rcgu.o" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.29ezgwktcliapzds.rcgu.o" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.2dixcyq2dznoqf6f.rcgu.o" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.2e3sp2f5hdniw8dq.rcgu.o" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.2stwxpje5swges4g.rcgu.o" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.39faxkgwwrs762t7.rcgu.o" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.3eb8m1razly0qf8f.rcgu.o" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.3tzo732szfa65bgw.rcgu.o" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.443gbpfb1fa8iu4a.rcgu.o" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.5dl1mmeah6wfm5cu.rcgu.o" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.c2srm2fuob3bb6p.rcgu.o" "-o" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.js" "-s" "EXPORTED_FUNCTIONS=[\"_main\",\"_rust_eh_personality\"]" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/minimal.dbnavdkjs80i37r.rcgu.o" "-O0" "--memory-init-file" "0" "-g4" "-s" "DEFAULT_LIBRARY_FUNCS_TO_INCLUDE=[]" "-L" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps" "-L" "/home/user/dev/stdweb/examples/target/debug/deps" "-L" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/libstdweb-7a6929fc6b0490d1.rlib" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/libdiscard-61ecf5c35989b1da.rlib" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/libstdweb_internal_runtime-6eee130e7adc7143.rlib" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/libserde_json-c735cd759ad596de.rlib" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/libryu-c64b79a8ce6acb24.rlib" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/libitoa-17a5036565265a53.rlib" "/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/libserde-db64dd451a129519.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libstd-55787d58fb5ea6fe.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libpanic_unwind-f59a7a55c9d2b3e1.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libhashbrown-dfc95e57308ebe8f.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_std_workspace_alloc-69c81ff795157707.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libbacktrace-977f4d06be3c4d88.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_demangle-0bd9079b201eae64.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libunwind-d452119022cae4dc.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcfg_if-1cada331e1cd4d54.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/liblibc-f5e8192cc8a59183.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/liballoc-b3d1e9d085a55269.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/librustc_std_workspace_core-c13c0ad8f20110e1.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcore-9f7cee0f9a15a27c.rlib" "/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/wasm32-unknown-emscripten/lib/libcompiler_builtins-13394c9769210e34.rlib" "-l" "c" "-s" "NO_EXIT_RUNTIME=1" "-s" "ALLOW_MEMORY_GROWTH=1" "-s" "ERROR_ON_UNDEFINED_SYMBOLS=1" "-s" "ASSERTIONS=1" "-s" "ABORTING_MALLOC=0" "-Wl,--fatal-warnings"
  = note: shared:WARNING: object /run/user/1000/emscripten_temp_RyTEXP_archive_contents/stdweb-7a6929fc6b0490d1.11ivc8q9b1ak0uju.rcgu.o is not valid according to llvm-nm, cannot link
          shared:WARNING: object /run/user/1000/emscripten_temp_RyTEXP_archive_contents/stdweb-7a6929fc6b0490d1.11mnvucp52kumo9.rcgu.o is not valid according to llvm-nm, cannot link
...
532 lines omitted
...
          shared:WARNING: object /run/user/1000/emscripten_temp_fQbSlr_archive_contents/compiler_builtins-13394c9769210e34.compiler_builtins.5m4is7s4-cgu.0.rcgu.o is not valid according to llvm-nm, cannot link
          shared:WARNING: object /run/user/1000/emscripten_temp_fQbSlr_archive_contents/lib.rmeta is not valid according to llvm-nm, cannot link
          shared:WARNING: object /run/user/1000/emscripten_temp_fQbSlr_archive_contents/compiler_builtins-13394c9769210e34.compiler_builtins.5m4is7s4-cgu.0.rcgu.bc.z is not valid according to llvm-nm, cannot link
          shared:ERROR: undefined exported function: "_main"

error: could not compile `minimal`.

Caused by:
  process didn't exit successfully: `rustc --crate-name minimal minimal/src/main.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=618e83af52622a74 --out-dir /home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps --target wasm32-unknown-emscripten -C incremental=/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/incremental -L dependency=/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps -L dependency=/home/user/dev/stdweb/examples/target/debug/deps --extern stdweb=/home/user/dev/stdweb/examples/target/wasm32-unknown-emscripten/debug/deps/libstdweb-7a6929fc6b0490d1.rlib --cfg cargo_web -C link-arg=-s -C link-arg=NO_EXIT_RUNTIME=1 -C link-arg=-s -C link-arg=ALLOW_MEMORY_GROWTH=1` (exit code: 1)
error: aborting due to previous error
error: build failed

Same error for me. What's going on? did emcc bump the llvm version so it doesn't match anymore?

@elichai Are you sure it's the same error?
If you're referring to the Could not verify LLVM version error, in my case I'm pretty sure it's not due to a specific version of LLVM not being found, but rather LLVM not being found at all.

Nope, Don't think it's the same anymore, sorry: https://github.com/koute/cargo-web/issues/233

@elichai Actually, you were right! For my second error it's indeed the rust compiler >=1.40.0 that's at fault!
I have successfully compiled the minimal example with version 1.39.0!
For the record:
❯ cargo --version
cargo 1.39.0 (1c6ec66d5 2019-09-30)

❯ cargo web start --use-system-emscripten --target=wasm32-unknown-emscripten
...
    Finished dev [unoptimized + debuginfo] target(s) in 1m 10s
...


"ERR_REQUIRE_ESM" when running yarn dev

node v14.16.0
yarn 1.22.10
yarn dev is working on 4616d09, but fails with 'ERR_REQUIRE_ESM' on 12a0da9.
arch ~/git/dawn (git)-[bisect/bad~1|bisect] % yarn dev
yarn run v1.22.10
warning<EMAIL_ADDRESS>The engine "ghost" appears to be invalid.
warning<EMAIL_ADDRESS>The engine "ghost-api" appears to be invalid.
$ gulp
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /home/arch/git/dawn/node_modules/beeper/index.js
require() of ES modules is not supported.
require() of /home/arch/git/dawn/node_modules/beeper/index.js from /home/arch/git/dawn/gulpfile.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /home/arch/git/dawn/node_modules/beeper/package.json.

    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1080:13)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)
    at Module.require (internal/modules/cjs/loader.js:952:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (/home/arch/git/dawn/gulpfile.js:10:16)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14) {
  code: 'ERR_REQUIRE_ESM'
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

git bisect start
# bad: [930bd3c9123a22fd4c66aadbfe0a085afaf1965c] Update dependency postcss to v8.2.14
git bisect bad 930bd3c9123a22fd4c66aadbfe0a085afaf1965c
# good: [4616d09efedd24e2669422eeb474c098aa2b99e1] Remove custom routes
git bisect good 4616d09efedd24e2669422eeb474c098aa2b99e1
# bad: [1522c7df0a58a95afc58cab1cd77401afb9b5321] Fix table of content links
git bisect bad 1522c7df0a58a95afc58cab1cd77401afb9b5321
# bad: [12a0da937bd581fbb7178b64469d37221e7fbb42] Update dependency beeper to v3
git bisect bad 12a0da937bd581fbb7178b64469d37221e7fbb42
# first bad commit: [12a0da937bd581fbb7178b64469d37221e7fbb42] Update dependency beeper to v3

Hey @26fiftysix, that was related with updated version (v3) of beeper that is used in the development workflow. We just reverted the package version in https://github.com/TryGhost/Dawn/commit/83c67feec7788389f82cf2e9aacddb25ab7807cf. You'll now be able to run yarn dev.
Thanks for the report :)


Add link to Member Expectations doc in README

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Util; Missing Typed Array - Float64Array

Float64Array
blocked by hxcpp float implementations, from what I am told :]

resolved in future branchs


Certbot Repository has been deprecated

When I do a sudo apt update on the system I get:
The repository 'http://ppa.launchpad.net/certbot/certbot/ubuntu focal Release' does not have a Release file
So I try:
sudo apt-add-repository -r ppa:certbot/certbot
and get:
The PPA has been DEPRECATED.
To get up to date instructions on how to get certbot for your systems, please see https://certbot.eff.org/docs/install.html.
More info: https://launchpad.net/~certbot/+archive/ubuntu/certbot
Press [ENTER] to continue or Ctrl-c to cancel removing it.
As a side note,  Have you thought of using acme.sh instead?

just read that the certbot PPA is Ubuntu 18 only.   20.04 has its own certbot package or you can use Snap to get the latest: https://certbot.eff.org/instructions?ws=nginx&os=ubuntufocal

Thank you.  The script for handling SSL certs was originally written by the Ubiquity forum member PetriR who designed it for Debian and I brought it over to Ubuntu and for the latest Stack file, Ubuntu 20.04.  I didn't have a great way to test it, so I didn't see that it wasn't supported.
I'll keep this open until I have time to look into it, or if you want to look at the file /cloud-init/startup.sh and submit a PR or recommend code changes, I would be happy for some collaboration.

Ok, thanks, @MallocArray I will take a look and see what I can do.
one thing that I am wondering about is how  you are taking care of getting the public ip updated for for the domain so that certbot or acme.sh is able to verify the domain.   I see that you allow for a ddns url in the stack variables.   Is this how it is taken care of?

Same type of situation.  I'm using the PetriR script contents but not entirely sure what the ddns step does.  It looks like it just does a curl -fs to the ddns address, but not sure how that helps, unless the url you provide has all of your authentication information in it.

In my testing, adding an update URL with authentication info does not work.
The problem I think too is that the Compute instance gets assigned an ephemeral ip address at some point during the installation.
The script then has to determine the ip address and send this to update the DDNS server before the lets encrypt script runs.
The question then is:  at one point in the process is the ip address available in the compute instance during the stack build process.  Do you have any idea?

This is supported now, as the Easy Install Script that is called fully supports Lets Encrypt


Repository license

I'm working on an acknowledgements file for an app I'm developing. I noticed that the license file referred to "Firebase, Inc." in the license appendix. Looking at the commit history shows that this was introduced in #6044.
I've got pretty much no idea how the Apache 2.0 license should be applied to an open source repository - i.e. whether or not the appendix at the end should be filled with the name of the copyright holder of the repository. Going off a quick search, the Google open source docs suggest that the appendix shouldn't be filled:

There must also be a file named LICENSE in the top-level directory that contains a copy of the license. For the Apache License, simply copy the official 2.0 license into your LICENSE file verbatim (do not edit the Copyright section in the template, only when you use that template in the header of your source code files.)
Why shouldn't I edit the template in the Apache License? The "LICENSE" file is supposed to be our inclusion of an exact copy of https://www.apache.org/licenses/LICENSE-2.0.txt as required by section 4(a) of the Apache 2 license, so we shouldn't modify it in any way, such as by adding our copyright notice. Also, the Appendix (lines 179-202) represents a notice to third-parties informing them of how they should apply the Apache 2 license to their own works, so inserting our copyright notice in place of the template text renders those instructions incorrect. Many projects insert copyright notices into their LICENSE file at line 190 because this is a common and understandable misinterpretation of the license's instructions. We should attempt to fix this wherever we find it, particularly for first-party projects.

I will happily admit that I don't know much about how to apply the Apache 2.0 license to a piece of work. However, I thought that it was probably something that I should bring to your attention.

@hithomasmorelli Thanks for the issue. Your analysis looks right to me that we shouldn't have merged #6044. I'll send a PR to fix.


[CT-1492] [Feature] Add support for Iceberg Tables

Is this your first time submitting a feature request?

[X] I have read the expectations for open source contributors
[X] I have searched the existing issues, and I could not find an existing issue for this feature
[X] I am requesting a straightforward extension of existing dbt-snowflake functionality, rather than a Big Idea better suited to a discussion

Describe the feature
Snowflake has announced support for Iceberg Tables, an external table format but with full interoperability as a 1st class snowflake table.
Once this feature goes live, dbt-snowflake should support materializing a model as an external iceberg table.
Describe alternatives you've considered
No response
Who will this benefit?

dbt users who want their models available as parquet files in storage they control
dbt users who want an easy way to interoperate with non-dbt systems that use the iceberg table format
dbt users who want direct access and control over the underlying storage of their data

Are you interested in contributing this feature?
yes
Anything else?
Iceberg Tables are in private preview as of 2022-11-10

Any chance to put more light on that? We really need a Snowflake equivalent of https://github.com/dbt-athena/dbt-athena to push forward the adoption of open table format (which, I believe, would be also pretty crucial for dbt to support in the long term).

+1 here! This would be super useful to support for enterprise sas use cases!

+1 here!

+1 here! Looking into Iceberg for enterprise sharing across platforms

+1, Iceberg is now in public preview on Snowflake

+1 here!

+1 here!

+1

+1

+1, will work on this a bit in the coming weeks and, if successful, open a pull request

+1

+1 here. We are adopting Iceberg tables as our data lake format and I would love to generate the tables out of Snowflake with dbt.

+1

+1

+1 :-)

+1

Hey all! Thanks for "upvoting", please continue to do so.
We are wrapping up all the internals work to support 1.8, plus a few more things to do for quality of the plugins and how we test them, and then Iceberg will be at the top of the list!

+1

+1

+1

+1

+1

+1

Any ETA on 1.8 ?

Any ETA on 1.8 ?

According to the documentation, it should be released as of April 2024: https://docs.getdbt.com/docs/dbt-versions/core#latest-releases

i'm getting errors when trying to query a MAP iceberg object from dbt:
  002040 (42601): SQL compilation error:
  Unsupported data type 'map(VARCHAR(16777216), VARCHAR(16777216))'.
  compiled Code at target/run/dbt_snowflake_1/models/L1_iceberg/L1__client_events_iceberg.sql

any idea why it happens? can i somehow solve it?
didn't find a documentation suggesting MAP columns are not supported on dbt_snowflake integration

+1

Is this now LIVE...? Was slated for APRIL release?

+1

Any news on this?

Any update?  We're looking into iceberg tables on Snowflake now as well.

+1

We will be prioritizing this work over the next couple of weeks. Please stay tuned.

+1

+1

I'm drafting out features right now. We have a table prototype going if you want a sneak peek. Fair warning, we have some performance issues to navigate given Snowflake metadata constraints.
We are actively scoping incremental tables out :)
#1170

+1

+1

For those following along, we just merged my linked table materialization PR above into main. We have some demos to do which will ferret out any minor concerns.

Closing this out as we've shipped support:

tables
DTs
incremental models

Docs for these have shipped

@VersusFacit It's my understanding that this is live in cloud today. For core users, is this available in the 1.9 beta series?

@VersusFacit I don't see Iceberg in the docs you linked. Maybe the link incorrect, or the docs have not yet been released?

@VersusFacit I don't see Iceberg in the docs you linked. Maybe the link incorrect, or the docs have not yet been released?

hey @sfc-gh-dsotolongo sorry you only see them if you have the Version="Versionless" in the dropdown. This URL with a query param should take you right there.
@will-sargent-dbtlabs For core users, it's in the 1.9 beta series


Lost temp sensor with acurite sensors.

I have lost just the temperature sensor with the acurite 06045m lightning detector sensor and the temperature sensor for the 00592txr doesn't even show up to add. The only error I can find is on the log page saying "[R82XX] PLL not locked!" It looks like mqtt is receiving something "1649222885: Received PUBLISH from rtl_433-057fffff (d0, q0, r1, m0, 'rtl_433/Acurite-6045M/47/A/temperature_F', ... (4 bytes))" as well as "1649222882: Received PUBLISH from rtl_433-057fffff (d0, q0, r1, m0, 'rtl_433/Acurite-Tower/3912/A/temperature_F', ... (4 bytes))". Any Idea's?

I think I need to disconnect everything and figure out which of these sensors are mine. I just ended up with 4 more and the readings are all over the place. Sorry for jumping the gun here. I think my issues are on my end...


Add Lunar as adopter

Been on this journey with you guys since September 2020

Tugboat has finished building the preview for this pull request!
Link:

https://pr4696-5wywqebk8yhfcyhpqrgwymsq3aew9aej.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/6037845bf285af2d8d9839f8

@JacobValdemar thanks for this! although you need to run prettier on the ADOPTERS.md file 🙈

Tugboat has finished building the preview for this pull request!
Link:

https://pr4696-5wywqebk8yhfcyhpqrgwymsq3aew9aej.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/6037845bf285af2d8d9839f8

Welcome! (?) 😁
Needs a bit of this, or permission to push to your fork :<
diff --git a/.github/styles/vocab.txt b/.github/styles/vocab.txt
index 831df202e..b672405d1 100644
--- a/.github/styles/vocab.txt
+++ b/.github/styles/vocab.txt
@@ -272,6 +272,7 @@ untracked
 upvote
 url
 utils
+Valdemar
 validators
 varchar
 winston

Thanks 😁 So many checks 🙈

Tugboat has finished building the preview for this pull request!
Link:

https://pr4696-5wywqebk8yhfcyhpqrgwymsq3aew9aej.tugboat.qa

Dashboard:

https://dashboard.tugboat.qa/6037845bf285af2d8d9839f8


How to save image as monochrome (black/white, 1 Bit colormap)?

I use the following code with your lib to achieve a monochrome image with just black and white pixels in it (1 Bit colordepth):
$converter = new GDIndexedColorConverter();
$palette = array(
    array(0, 0, 0),
    array(255, 255, 255),
);
$new_image = $converter->convertToIndexedColor($image, $palette, 0.8);
imagepng($new_image, 'bwimage.png', 6);

But the resulting imagefile is written as RGB with 8-Bit colordepth:
root@server /tmp # file bwimage.png
bwimage.png: PNG image data, 76 x 96, 8-bit/color RGB, non-interlaced

I've tried to add filters, like found on the Internet, but they seem to have no effect on the output format:
$converter = new GDIndexedColorConverter();
$palette = array(
    array(0, 0, 0),
    array(255, 255, 255),
);
$new_image = $converter->convertToIndexedColor($image, $palette, 0.8);
imagefilter($new_image, IMG_FILTER_GRAYSCALE);
imagefilter($new_image, IMG_FILTER_CONTRAST, -1000);
imagepng($new_image, 'bwimage.png', 6);

An acceptable solution for me is this:
$bw_img = $converter->convertToIndexedColor($img, $palette, 0.8);
imagefilter($bw_img, IMG_FILTER_NEGATE);
imagetruecolortopalette($bw_img, true, 2);
imagecolorset($bw_img, 0, 255, 255, 255);
imagecolorset($bw_img, 1, 0, 0, 0);


Galley CPU leak

over a 24hr window
Its now at ~100m baseline, but after I had it running for a week it was at 700m, as can be seen in the far end of the graph before I restarted it.

The same can be seen for mixer-policy, suggesting something to do with kube crds

Its the metrics. After just 1 day of running /metrics returns 26mb!

If I look at most expensive metrics with topk(10, count by (__name__)({__name__=~".+"}))

Metric
Number of Series

istio_mcp_message_sizes_bytes_bucket{}
163557

istio_response_bytes_bucket{}
73200

istio_request_bytes_bucket{}
73200

istio_request_duration_milliseconds_bucket{}
59700

envoy_cluster_client_ssl_socket_factory_ssl_context_update_by_sds{}
30590

istio_request_duration_seconds_bucket{}
16200

container_memory_failures_total{}
14088

envoy_cluster_upstream_cx_connect_ms_bucket{}
13000

envoy_cluster_upstream_cx_length_ms_bucket{}
13000

envoy_server_initialization_time_ms_bucket{}
12780

Job
Series

{job="galley"}
194137

{job="kubernetes-cadvisor"}
171092

{job="istio-telemetry"}
59923

{job="kubelet"}
20619

{job="svc05-0"}
9516

{job="svc12-0"}
9436

{job="svc03-0"}
9382

{job="istio-ingressgateway"}
9156

{job="svc14-0"}
9114

{job="svc10-0"}
9047

It is like memory leak in default profile, there is only two pod  to test.

I think this is fixed by #19170


Fails silently when analyzing files containing reserved words

Hi.
I had a source file that declared a local variable like this:
var interface =
{
    Something: function() { ... }
}

This results in Madge skipping the file, failing to detect any dependencies:
precinct could not parse content: Unexpected reserved word 'interface'. (366:8) +32ms
Complete debug output:
MBP:AppKit jemt$ node_modules/madge/bin/cli.js --debug Client/AppKit/Component.js
⠋ Finding files  madge using src paths [ '/Users/jemt/www/AppKit/Client/AppKit/Component.js' ] +0ms
  madge using config { baseDir: null, excludeRegExp: false, fileExtensions: [ 'js' ], includeNpm: false, requireConfig: null, webpackConfig: null, tsConfig: null, rankdir: 'LR', layout: 'dot', fontName: 'Arial', fontSize: '14px', backgroundColor: '#111111', nodeColor: '#c6c5fe', nodeShape: 'box', nodeStyle: 'rounded', noDependencyColor: '#cfffac', cyclicNodeColor: '#ff6c60', edgeColor: '#757575', graphVizOptions: false, graphVizPath: false, dependencyFilter: [Function (anonymous)], debug: true, version: [Function: version] } +3ms
  madge using base directory /Users/jemt/www/AppKit/Client/AppKit +2ms
  tree given filename: /Users/jemt/www/AppKit/Client/AppKit/Component.js +0ms
  tree resolved filename: /Users/jemt/www/AppKit/Client/AppKit/Component.js +0ms
  tree visited:  {} +1ms
  tree traversing /Users/jemt/www/AppKit/Client/AppKit/Component.js +0ms
  precinct paperwork: invoking precinct +0ms
  precinct options given: { includeCore: false } +0ms
  precinct we assume this is JS +1ms
  precinct could not parse content: Unexpected reserved word 'interface'. (366:8) +32ms
  tree extracted 0 dependencies:  [] +33ms
  tree cabinet-resolved all dependencies:  [] +0ms
  tree using filter function to filter out dependencies +0ms
  tree unfiltered number of dependencies: 0 +0ms
  tree filtered number of dependencies: 0 +0ms
  tree traversal complete {} +0ms
  tree deduped list of nonExistent partials:  [] +0ms
  tree object form of results requested +0ms
  tree final tree { '/Users/jemt/www/AppKit/Client/AppKit/Component.js': {} } +0ms
Processed 1 file (481ms)

I suggest that the check fails with an exit code or at least issues a warning (even if --warning is not specified), rather than giving the impression that no dependencies are present.
-- Thanks :-)

I see two problems here:

we should split two types of warning: "user skipped the file"/ "cannot parse file", and the latter should not be affected by --warning cli flag
handle files with reserved word like 'interface' correctly ( I dug a little here and found that interface was actually a JS reserved word: MDN, DOM


Add Angular with Node 14 container.

Signed-off-by: Jaideep Singh Heer<EMAIL_ADDRESS>

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: jaideepheer sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Thanks for the pull request!  A few comments questions:

There's already a definition for Node.js 12 and it definitely makes sense to add one for 14 now that it is out.  However, this definition deviates pretty significantly from the Node 12 version. Any particular reason why?  For example, we include a "vscode" user in all definitions since this is important for Linux users.
Naming wise, it seems like this definition is more typescript-node-14-angluar isn't it? Angular is the library in use, but TypeScript is more the language.
It looks like you are using a multi-stage Dockerfile. Given we're pre-building the Node images to speed up their use, we could tweak this a bit to I think get the same general idea across and also include a "builder" stage.  For example, if we stick with typescript-node-12, it could be something along these lines:

FROM mcr.microsoft.com/vscode/devcontainers/typescript-node:12 AS development

# Build steps go here
FROM development as builder
WORKDIR /app
COPY src/ *.json ./
RUN yarn install \
    && yarn compile \
    #  Just install prod dependencies
    && yarn install --prod

# Actual production environment setup goes here
FROM node:12-slim AS production
WORKDIR /app
COPY --from=builder /app/out/ ./out/
COPY --from=builder /app/node_modules/ ./node_modules/
COPY --from=builder /app/package.json .
EXPOSE 3000
ENTRYPOINT [ "/bin/bash", "-c" ]
CMD [ "npm start" ]


@conda-forge-admin, please add user @ricardoV94

Additional comment:
No response

Hi! This is the friendly automated conda-forge-webservice.
I just wanted to let you know that I added user @ricardoV94 in conda-forge/pymc-bart-feedstock#6.


SwiftFormat removed from macOS 12

Description
SwiftFormat 0.49.7 is on the macos-11 runner: https://github.com/actions/virtual-environments/blob/main/images/macos/macos-11-Readme.md#tools
But it's missing from the macos-12 runner: https://github.com/actions/virtual-environments/blob/main/images/macos/macos-12-Readme.md#tools

Why was it removed? I can't find any news or information on this.
Can it be added back? SwiftFormat is an extremely widespread tool. If macos-12 includes SwiftLint, it should probably also include SwiftFormat.

Virtual environments affected

[ ] Ubuntu 18.04
[ ] Ubuntu 20.04
[ ] Ubuntu 22.04
[ ] macOS 10.15
[ ] macOS 11
[X] macOS 12
[ ] Windows Server 2019
[ ] Windows Server 2022

Image version and build link
n/a
Is it regression?
Yes, macos-11 included SwiftFormat.
Expected behavior
SwiftFormat is included.
Actual behavior
SwiftFormat is not included.
Repro steps
Attempt to run SwiftFormat on macos-12.

Hey @bdrelling,
We will pre-install   SwiftFormat on macOS-12.

@al-cheb Incredibly quick response, really appreciate it!
I'm not sure how to keep up with changes to the runners, do you know when/how this would be added to the image?
In the meantime, it is simple enough to add a brew install swiftformat step for now, so this isn't urgent, just inconvenient.
Thanks again!

The new image has been deployed with swiftformat on board.


Updating the JavaAgent with a best effort to avoid port conflicts dur…

…ing tournaments.

Another day, another blind pull request to accept.


Add container support?

This is probably something that should be posted to the Visual Studio Team... But maybe someone here can shed some light on this?
I have tried to get container support running on a project created with this sdk, but it doesn't seem to work.
(See here  for the official docs on this container support)
I have added a package reference to the container tools:
<PackageReference Include="Microsoft.VisualStudio.Azure.Containers.Tools.Targets" Version="1.10.13" />

I have added a Dockerfile and a .dockerignore file.
I have added a profile to the launchsettings:
"Docker": {
  "commandName": "Docker",
  "launchBrowser": true,
  "launchUrl": "{Scheme}://{ServiceHost}:{ServicePort}",
  "publishAllPorts": true,
  "useSSL": true
}

But when I try to debug or run the project using that profile, I get an error message "the project doesn't know how to run the profile Docker":

Do these steps work successfully if trying to use them on an ASP.NET Core application?  If this is the case, then it's possible that the Microsoft.NET.Sdk.Web contains something that allows the package to work that Microsoft.NET.Sdk (which this project imports) does not, or, perhaps, Visual Studio itself likes to treat Microsoft.NET.Sdk.Web projects special.
If ASP.NET Core applications do not work for you following the steps described, then you likely need to first troubleshoot that problem.  If, however, ASP.NET Core applications do work, then we would need to find out what allows the Microsoft.NET.Sdk.Web SDK to work.  I have a feeling that, if this is the case, it is likely a <ProjectCapability>.

@fretje
Looking through the notes on docker support - remember that you will have to use windows containers with IIS to get this to work as you are running under the Full Framework.
There are some notes in the documentation about

If you are using the full .NET Framework console project template, the supported option is Add Container Orchestrator support after project creation, with options to use Service Fabric or Docker Compose. Adding support at project creation and Add Docker support for a single project without orchestration are not available options.

However, as @TheJayMann says - the Launch Profile option for docker seems likely to be lit up via a <ProjectCapability>
It does seem like the package Microsoft.VisualStudio.Azure.Containers.Tools.Targets does have a couple of these included:
  <ItemGroup>
    <ProjectCapability Include="ContainerTargets" />
  </ItemGroup>
  <PropertyGroup>
    <HasContainerTargets>true</HasContainerTargets>
  </PropertyGroup>

and
  <ItemGroup>
    <ProjectCapability Include="BuildServiceTargets" />
  </ItemGroup>

Unfortunately I couldn't find any more information on how launchSettings commands were resolved - obviously there is Program, IIS Express and Docker but it is not obvious how these work.
It seems as if you can force the issue by creating a folder Container under obj, and adding an empty file called launch.sem
Then if you just Build the project, it seems to create the container and launch it as expected.
Note that the web browser does not launch, but if you look in the Docker Desktop UI, or the Containers window in VS you can see the container running, and browsing to the host port seems to work as expected.
Obviously, this is not debugging the container, but you can do Debug->Attach to Process, select Docker (Windows Container), find the connection target, set the type to Managed 4.x code, show all users and select the w3wp.exe process.
Altogether though it is not a great experience.

@CZEMacLeod

Unfortunately I couldn't find any more information on how launchSettings commands were resolved - obviously there is Program, IIS Express and Docker but it is not obvious how these work.

I believe a package (built-in or add-on) supports a launch command via IDebugProfileLaunchTargetsProvider.SupportsProfile. This component should then be decorated with AppliesTo that determines what project capabilities are required for the component to activate. This can be a complex expression e.g. Cap1&(Cap2|Cap3)
@fretje
Could you try with these?
<ProjectCapability Include="ContainerToolsDebugging" />
<ProjectCapability Include="ContainerTargets" />

This lights up the Add > Docker Support... in the project context menu. Doesn't seem to work properly though.
<PropertyGroup>
    <ProjectTypeGuids>{349c5851-65df-11da-9384-00065b846f21};{fae04ec0-301f-11d3-bf4b-00c04f79efbc}</ProjectTypeGuids>
</PropertyGroup>

I am trying to convert an existing NetFramework4.8 (webapi) project to use this new SDK style. This NetFramework project has docker support enabled, and it is possible to build the docker container and debug the application inside the docker container all from visual studio. I actually enabled that support by specifying it while creating the project, but I think that is the same as not specifying it there and then simply adding it afterwards by right clicking on the project and add > docker support. Which is actually available for NetFramework projects even though the docs might state it otherwise.
I have created a brand new asp.net mvc (netframework) application and enabled docker support from the get go, just to test this out. This is the "Container Tools" output after creating the project:
========== Warming up container(s) for WebApplicationNetFramework ==========
Starting up container(s)...
docker build -f "C:\Users\fretje\source\repos\DockerSdkStyle\WebApplicationNetFramework\Dockerfile" --force-rm -t webapplicationnetframework:dev --build-arg "source=obj\Docker\empty" --label "com.microsoft.created-by=visual-studio" --label "com.microsoft.visual-studio.project-name=WebApplicationNetFramework" "C:\Users\fretje\source\repos\DockerSdkStyle\WebApplicationNetFramework"
Sending build context to Docker daemon  3.584kB

Step 1/6 : FROM mcr.microsoft.com/dotnet/framework/aspnet:4.8-windowsservercore-ltsc2019
 ---> 8558397b56ab
Step 2/6 : ARG source
 ---> Running in 4b390cb85d90
Removing intermediate container 4b390cb85d90
 ---> 23c01119ba3c
Step 3/6 : WORKDIR /inetpub/wwwroot
 ---> Running in 6bffcf2d148f
Removing intermediate container 6bffcf2d148f
 ---> 822a1def8864
Step 4/6 : COPY ${source:-obj/Docker/publish} .
 ---> 31f910c32bcd
Step 5/6 : LABEL com.microsoft.created-by=visual-studio
 ---> Running in 028d2bb7f300
Removing intermediate container 028d2bb7f300
 ---> cccfa21ca4bd
Step 6/6 : LABEL com.microsoft.visual-studio.project-name=WebApplicationNetFramework
 ---> Running in 9549234dec8e
Removing intermediate container 9549234dec8e
 ---> d1873cb1c772
Successfully built d1873cb1c772
Successfully tagged webapplicationnetframework:dev
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
docker run -dt -v "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\Remote Debugger:C:\remote_debugger:ro" -v "C:\Users\fretje\source\repos\DockerSdkStyle\WebApplicationNetFramework:C:\inetpub\wwwroot" -e "DEV_ENVIRONMENT=1" -e "VBCSCOMPILER_TTL=604800" -e "COMPLUS_ForceENC=1" -P --name WebApplicationNetFramework --entrypoint cmd webapplicationnetframework:dev /c "start /B C:\ServiceMonitor.exe w3svc & C:\remote_debugger\x64\msvsmon.exe /noauth /anyuser /silent /nostatus /noclrwarn /nosecuritywarn /nofirewallwarn /nowowwarn /fallbackloadremotemanagedpdbs /timeout:2147483646"
4d05feb6daa35263cc9868cadca0d64f711818c72bdca5f302f8065e7cd0384b
docker exec 4d05feb6daa35263cc9868cadca0d64f711818c72bdca5f302f8065e7cd0384b cmd /c "C:\Windows\System32\inetsrv\appcmd.exe set config -section:system.applicationHost/applicationPools /[name='DefaultAppPool'].processModel.identityType:LocalSystem /commit:apphost & C:\Windows\System32\inetsrv\appcmd.exe set config -section:system.webServer/security/authentication/anonymousAuthentication /userName: /commit:apphost"
Applied configuration changes to section "system.applicationHost/applicationPools" for "MACHINE/WEBROOT/APPHOST" at configuration commit path "MACHINE/WEBROOT/APPHOST"
Applied configuration changes to section "system.webServer/security/authentication/anonymousAuthentication" for "MACHINE/WEBROOT/APPHOST" at configuration commit path "MACHINE/WEBROOT/APPHOST"
Trying to connect to <IP_ADDRESS> for debugging
Trying to connect to <IP_ADDRESS> for debugging
Container started successfully.
========== Finished ==========

Running and debugging the project in Docker works as you would expect.
For comparison, I have done the same for an ASP.NET Core Web project, but there everything comes in the "Build" output. There is no "Container Tools" output:
1>------ Build started: Project: WebApplicationNetCore, Configuration: Debug Any CPU ------
1>WebApplicationNetCore -> C:\Users\fretje\source\repos\DockerSdkStyle\WebApplicationNetCore\bin\Debug\net5.0\WebApplicationNetCore.dll
1>WebApplicationNetCore -> C:\Users\fretje\source\repos\DockerSdkStyle\WebApplicationNetCore\bin\Debug\net5.0\WebApplicationNetCore.Views.dll
1>docker build -f "C:\Users\fretje\source\repos\DockerSdkStyle\WebApplicationNetCore\Dockerfile" --force-rm -t webapplicationnetcore:dev --target base  --label "com.microsoft.created-by=visual-studio" --label "com.microsoft.visual-studio.project-name=WebApplicationNetCore" "C:\Users\fretje\source\repos\DockerSdkStyle"
1>Sending build context to Docker daemon  84.28MB
1>
1>Step 1/6 : FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
1>5.0: Pulling from dotnet/aspnet
...
1>f2250ff2c8dd: Pull complete
1>Digest: sha256:c0cc95b0d87a31401763f8c7b2a25aa106e7b45bfcaa2f302dc9d0ff5ab93fa2
1>Status: Downloaded newer image for mcr.microsoft.com/dotnet/aspnet:5.0
1> ---> 834c20869027
1>Step 2/6 : WORKDIR /app
1> ---> Running in 161b357d7430
1>Removing intermediate container 161b357d7430
1> ---> 58b55130c04c
1>Step 3/6 : EXPOSE 80
1> ---> Running in 7f4c46f6748a
1>Removing intermediate container 7f4c46f6748a
1> ---> 3d4fad8f210e
1>Step 4/6 : EXPOSE 443
1> ---> Running in 66b9461d041d
1>Removing intermediate container 66b9461d041d
1> ---> cc6bd66954a2
1>Step 5/6 : LABEL com.microsoft.created-by=visual-studio
1> ---> Running in c176d9ef762e
1>Removing intermediate container c176d9ef762e
1> ---> 5167d54a976b
1>Step 6/6 : LABEL com.microsoft.visual-studio.project-name=WebApplicationNetCore
1> ---> Running in cc0c4e0c88c0
1>Removing intermediate container cc0c4e0c88c0
1> ---> 6886a6aec2db
1>Successfully built 6886a6aec2db
1>Successfully tagged webapplicationnetcore:dev
1>Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
1>docker run -dt -v "C:\Users\fretje\onecoremsvsmon\16.9.1154.0:C:\remote_debugger:ro" -v "C:\Users\fretje\source\repos\DockerSdkStyle\WebApplicationNetCore:C:\app" -v "C:\Users\fretje\source\repos\DockerSdkStyle:c:\src" -v "C:\Users\fretje\AppData\Roaming\Microsoft\UserSecrets:C:\Users\ContainerUser\AppData\Roaming\Microsoft\UserSecrets:ro" -v "C:\Users\fretje\AppData\Roaming\ASP.NET\Https:C:\Users\ContainerUser\AppData\Roaming\ASP.NET\Https:ro" -v "C:\Users\fretje\.nuget\packages\:c:\.nuget\fallbackpackages2" -v "C:\Program Files\dotnet\sdk\NuGetFallbackFolder:c:\.nuget\fallbackpackages" -e "DOTNET_USE_POLLING_FILE_WATCHER=1" -e "ASPNETCORE_LOGGING__CONSOLE__DISABLECOLORS=true" -e "ASPNETCORE_ENVIRONMENT=Development" -e "ASPNETCORE_URLS=https://+:443;http://+:80" -e "NUGET_PACKAGES=c:\.nuget\fallbackpackages2" -e "NUGET_FALLBACK_PACKAGES=c:\.nuget\fallbackpackages;c:\.nuget\fallbackpackages2" -P --name WebApplicationNetCore --entrypoint C:\remote_debugger\x64\msvsmon.exe webapplicationnetcore:dev /noauth /anyuser /silent /nostatus /noclrwarn /nosecuritywarn /nofirewallwarn /nowowwarn /fallbackloadremotemanagedpdbs /timeout:2147483646 /LogDebuggeeOutputToStdOut
1>0f882de424f9864e76d925c88f8f09c2cb3ac959204662629122bcf49990a182
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

Here also Debugging works fine.
Both projects are available here: https://github.com/fretje/DockerSdkStyle
I'll try and create a similar sdkstyle project in the same repo and see where I get with some of the suggestions here...

Adding
<ItemGroup>
  <ProjectCapability Include="ContainerToolsDebugging" />
</ItemGroup>

Seems to have gotten me further. Now the "Container Tools" output looks promising:
Docker Desktop is running.
========== Verifying Docker OS ==========
Verifying that Docker Desktop's operating system mode matches the project's target operating system...
Docker Desktop's operating system mode matches the project's target operating system.
========== Pulling Required Images ==========
Checking for missing Docker images...
Pulling Docker images. To cancel this download, close the command prompt window.
docker pull mcr.microsoft.com/dotnet/framework/aspnet:4.8-windowsservercore-ltsc2019
Docker images are ready.
========== Warming up container(s) for WebApplicationSystemWeb ==========
Starting up container(s)...
docker run -dt -v "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\Remote Debugger:C:\remote_debugger:ro" -v "C:\Users\fretje\source\repos\DockerSdkStyle\WebApplicationSystemWeb:C:\inetpub\wwwroot" -e "DEV_ENVIRONMENT=1" -e "VBCSCOMPILER_TTL=604800" -e "COMPLUS_ForceENC=1" -P --name WebApplicationSystemWeb --entrypoint cmd webapplicationsystemweb:dev /c "start /B C:\ServiceMonitor.exe w3svc & C:\remote_debugger\x64\msvsmon.exe /noauth /anyuser /silent /nostatus /noclrwarn /nosecuritywarn /nofirewallwarn /nowowwarn /fallbackloadremotemanagedpdbs /timeout:2147483646"
a5a7662449e430689b8e4ba30272009ddf4b6968617d247d26bc0c2a7c4eb9bd
Container started successfully.
========== Finished ==========

Starting debug build output looks like this:
Build started...
1>------ Build started: Project: WebApplicationSystemWeb, Configuration: Debug Any CPU ------
1>
1>Restore operation started...
1>
1>5 libraries restored in 0,5 seconds
1>WebApplicationSystemWeb -> C:\Users\fretje\source\repos\DockerSdkStyle\WebApplicationSystemWeb\bin\WebApplicationSystemWeb.dll
1>docker rm -f a5a7662449e430689b8e4ba30272009ddf4b6968617d247d26bc0c2a7c4eb9bd
1>a5a7662449e430689b8e4ba30272009ddf4b6968617d247d26bc0c2a7c4eb9bd
1>docker run -dt -v "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\Remote Debugger:C:\remote_debugger:ro" -v "C:\Users\fretje\source\repos\DockerSdkStyle\WebApplicationSystemWeb:C:\inetpub\wwwroot" -e "DEV_ENVIRONMENT=1" -e "VBCSCOMPILER_TTL=604800" -e "COMPLUS_ForceENC=1" -P --name WebApplicationSystemWeb --entrypoint cmd webapplicationsystemweb:dev /c "start /B C:\ServiceMonitor.exe w3svc & C:\remote_debugger\x64\msvsmon.exe /noauth /anyuser /silent /nostatus /noclrwarn /nosecuritywarn /nofirewallwarn /nowowwarn /fallbackloadremotemanagedpdbs /timeout:2147483646"
1>806b1dd3f92fc81a012266d39f44291257ea1ee095fbf80d51ac740817dd4e5c
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

But then I get an error Failed to find path of the output assembly for project WebApplicationSystemWeb.:

The project has been added to the test repository...

@fretje Great to have all 3 types in a repo for comparison. I had pretty much got as far as you, with the same error message.
I wonder if we might need to build a specialized IDebugLaunchProvider which works for the MSBuild.SDK.SystemWeb project type.
Either that or maybe we need a friendly MS/VS developer to give us some hints at what is happening inside the VS Docker support code at that point?

I hope not. Writing an IDebugLaunchProvider is no easy task. Most of these errors indicate that some MSBuild property is not set but are expected by the docker package. This could be either because it's in NET.Sdk.Web or some other lightup that would trigger it is still missing.

Tried this with a simple console app, the only difference was the target framework. Works for net5, same error message for net48. Unfortunately it looks like the ContainerBuildAndLaunch task in Microsoft.VisualStudio.Azure.Containers.Tools.Targets has some hardwired behavior for framework vs core. 😞

@bachratyg I've looked through the source of the ContainerBuildAndLaunch task itself, and while it does have differences between the NETFramework and NETCore implementations, I'm not convinced this is where the error is occurring.
Specifically, as the actual build succeeds, and the container is actually launched.
You can actually manually attach the debugger to the container and that bit works.
I think this is actually in the internal VS Extension that handles the IDebugLaunchProvider implementation for Docker.

If I'm mistaken the package is not OSS. Where did you get the source?

I don't think it's developed in the open, but you can download the nupkg from here and extract the target file: https://www.nuget.org/packages/Microsoft.VisualStudio.Azure.Containers.Tools.Targets/

I am looking to try and capture information from the issues into documentation, and while this is an ongoing issue pending some feedback from the docker tools team, I feel it might be useful to capture the current state of the issue as perhaps a known limitation or similar - obviously with a link to this issue.
Documentation?

@fretje / @CZEMacLeod , are there any updates on this?
@CZEMacLeod I see you added the "known limitation" label somewhat recently. Are there any news on this or was that done only because you created the label recently (or forgot to tag it before)?

As far as I can tell, there seems to be some support in the latest docker container targets, but debugging still fails to automatically attach.
If you do debug and get the error, you can then manually attach the debugger successfully and debug inside the container.
Unfortunately, this is all dependent on code inside of VS itself, so is not something that can be dealt with here. Hence the known limitation label.


zepto.js compatibility

It would be nice to be zepto.js compatible. :)

Moved this to the new repo for the js.


1.The first version of STM32_Platform

2.Change folder names to lower-case
3.delete document files.

pls write meaningful log.


Please cut a release to allow using the latest @typescript-eslint/eslint-plugin

I'm on 21.0.1 (the latest version) but am impacted by https://github.com/typescript-eslint/typescript-eslint/issues/3727, fixed in https://github.com/typescript-eslint/typescript-eslint/commit/d97140ecf4aeb0a1f8b391f46a31881f21ad93c3 and out for 5 months. I also see https://github.com/standard/eslint-config-standard-with-typescript/pull/702 has the fix I need, but that update has not yet been released.
Please cut a release so I can remove some eslint-ignores related to that issue.

@Divlo Could you release a new version ?

@QuentinFarizon I don't have the permissions/rights to publish a new version on npm for this package.
Hopefully, @mightyiam might have some time soon to publish a new version.

@Divlo A version on NPM would definitely be better, but what would you think of creating a branch in this repo with

new package version (probably major version 22.0.0 since I ESLInt 7 has been dropped as a peer depenency?)
lib directory NOT git-ignored

That would enable installing the repo via package.json:
    eslint-config-standard-with-typescript: "git+https://github.com/standard/eslint-config-standard-with-typescript.git#v22.0.0"

Or as a workaround for other people, that's what I've done with https://github.com/ChristopherChudzicki/eslint-config-standard-with-typescript/releases/tag/v22.0.0 (Though, having it on the main repo seems better which is why I'm suggesting it).

I won't get to this. If anyone wishes to take over contact me directly.

What is required to move this forward? maybe @Divlo can help? As I am not a contributor I can only offer my future support to whoever will try to get this done :)

I'm happy to take over @mightyiam @feross

@meyfa also offered the same.

If @mightyiam doesn't want to continue to maintain these modules, then I would prefer that we align them with the other modules in this org and try to streamline the release progress and get a more synced up release progress with less duplicate work.
Not sure if that's what @mightyiam is saying though?

Continue discussion in #727 ?

Work on the project is ongoing on a weekly schedule. A release will be cut eventually. See #840 .

It's on a weekly schedule but it's been months :)

@paralin yup. The blocker currently is that I'm not an owner of this organization. A GitHub App needs to be created to serve as part of the implementation of #840 . It seems only organization owners can do that. I'm sure this will be resolved one way or another soon.

Can't a manual release be made one very last time? The release blocks tens of projects for months....
In my opinion #840 could be worked on after a release.

Yes, that's correct, @lusu007 . But... as soon as I get the permission I can launch the CD pipeline. I already managed to get a response from someone who technically can do that. But they referred me to @feross :shrug:

It's been almost 2 weeks since the last sign of life. Did you get the permissions to launch the CD pipeline? @feross are you able to speed up the whole thing?
Nevertheless, I think at the moment everyone would love a manual release. The lack of a release, unlike the CD pipeline, costs many projects real money as a lot of time has to be invested in implementing alternatives.

I've contacted several of the owners of this organization asking them to either grant me greater permissions in the standard organization or create the GitHub app themselves. Either no response or "I don't feel comfortable doing either of those".
So just now I've contacted two more of the owners.
It's funny that you mention money, @lusu007 . I don't see any. And yet I do desire to serve the users best I can. I told myself that the next release will be an automated one. Shall I regress from that for the benefit of the users and make a manual release?

https://www.npmjs.com/package/eslint-config-standard-with-typescript/v/22.0.0

@mightyiam Thank you for your great and tireless efforts.

https://www.npmjs.com/package/eslint-config-standard-with-typescript/v/22.0.0

Works perfectly.


Compute land-atmosphere terrestrial coupling index in key_metrics

All Submissions:

[ y] Have you followed the guidelines in our Contributor's Guide (including the pre-commit check)?
[ y] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

[ ] Does your submission pass tests? -- Not sure
[ ] Have you lint your code locally prior to submission? -- Not sure

Changes to Core Features:

[ ] Have you added an explanation of what your changes do and why you'd like us to include them? -- In conversation, yes.
[y ] Have you successfully tested your changes locally?

Thanks for opening this PR!
A few comments before diving into the nuts and bolts:

Can you please run pre-commit (and set it up to run on all future commits from your CUPiD directory)? Instructions are in the wiki (specifically, in the Contributor's Guide). I would recommend running both pre-commit run --all-files and pre-commit install from (cupid-dev), and then committing additional changes to the branch from that environment as well
There are a lot of empty cells in the notebook that can probably be removed. Also, adding the hide-input tag will hide the input cell when creating the JupyterBook webpage while hide-cell will hide both the input and output; you might want to add one of those or the other to each cell in your notebook to make the webpage a little easier to view
Can you please remove all the output from the notebook in examples/nblibrary/? Just a simple "restart kernel and clear output" before saving will keep it as a blank template.
I see a bunch of /glade/derecho/scratch/mdfowler/tmp/ipykernel_13813/1925081246.py:42: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.), and I think the warnings are pointing to lat_fluxnet[iStation] = siteInfoDF['LOCATION_LAT'].values[indStation] and lon_fluxnet[iStation] = siteInfoDF['LOCATION_LONG'].values[indStation] in [9]; can you figure out the correct indexing to make sure both sides of the equation are scalars and avoid this warning? (I'm happy to help you play with that if you'd like, though I won't have a chance until tomorrow)

Once those bigger updates are made, I'll look more closely at the notebook

I've made most of those changes. For some reason, the tags for hide-input and hide-output aren't working as expected, but otherwise I think it's better!

I'm not sure where the error messages are coming from, but I can confirm that the website looks good now! And things are updated to point to the right file name.

This looks great! One more question before I merge it -- it looks like you added examples/nblibrary/lnd/image.png to the repository. Is that on purpose? I don't think the image is used anywhere.

Nope I didn’t add that on purpose, but I think it auto-generated when I ran the script. I’ll delete it and re-commit :)


refactor(protocol): make sure L1.init called in genesis, not the first block

David, in the previous code, the block.number can be either 0 or 1. With our current way of genesis the L2, the value should always be 0, is it?
@davidtaikocha @adaki2004 could one of you run the genesis test, my docker is broken locally.

I think its intended for unit tests (block.number can be either 0 or 1)? since for genesis generation, we wont need / run a init function, we just put the values in the slots: https://github.com/taikoxyz/taiko-mono/blob/main/packages/protocol/utils/generate_genesis/taikoL2.ts#L343


2D image detection

Its also tells the real face of 2d static image

Because this model's accuracy is limited.


COMP: Fixing autodeduction of type for old clang compiler

Fixes error reported in https://open.cdash.org/viewBuildError.php?buildid=6179213
Related to http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3922.html
See also https://stackoverflow.com/questions/25612262/why-does-auto-x3-deduce-an-initializer-list

PR Checklist

[ ] :no_entry_sign: Makes breaking changes
[ ] :no_entry_sign: Makes design changes
[ ] :no_entry_sign: Adds the License notice to new files.
[ ] :no_entry_sign: Adds Python wrapping.
[ ] :no_entry_sign: Adds tests and baseline comparison (quantitative).
[ ] :no_entry_sign: Adds test data.
[ ] :no_entry_sign: Adds Examples to ITKExamples
[ ] :no_entry_sign: Adds Documentation.

Refer to the ITK Software Guide for
further development details if necessary.

cc @seanm

LGTM


ModuleNotFoundError: No module named 'keyboard'

Hello I'm trying to start with the program but this error appears "ModuleNotFoundError: No module named 'keyboard'"

Did you run the installer?

I got it, just needed to install the pip modules
sorry for the inconvenience


webfonts instead of base64 images

to #525

nice
works for me, however this is the first time we have an external dependency in the main screen, so lets merge this and see if it causes any trouble for anyone


Add Algolia

I was doubting to add requires-approval or not, because default free plan is 10k records, 100k ops; default OSS is 50k / 100k and more needs approval but is definitely possible. Docsearch is also always free for OSS
Thank you for taking the time to work on a PR for Awesome-Open-Source-Supporters!
To ensure your PR is dealt with swiftly please check the following:

[x] Your submissions are formatted according to the guidelines: [Company Name](link) `tag` - description
[x] Your additions are ordered alphabetically.
[x] Your additions are commercial software that offer their services for free to Open Source projects.
[x] Your additions contain any relevant tags: requires-approval
[x] You have searched the repository for any relevant issues or PRs.
[x] Any category you are creating has the minimum requirement of 2 items.

updated @zachflower

Looks good, thanks!

@Haroenv small correction, the Algolia for Open Source plan is 100k records and 200k ops by default.


Feature/attach profile

Draft PR for attaching a profile in a message. Some of this will change as other pieces come (DIDCommV2 messaging handling, threads plugin, etc.)

On-hold for backend changes.

closing as stale, replaced by newer/upcoming PRs


prep for releases

The .github/workflows/release-plz.yml is as created by release-plz init; I didn't alter it at all. It's supposed to open a release PR when anything changes, update it as more changes come in, then actually do a release when I merge the PR. It uses crates.io and GitHub PAT tokens that I've scoped to the repo and the needed permissions.

Add repository link to the crates.io metadata so folks can find where to file an issue and such.

Add badges to the README as also generally expected.

Other fields to consider would be rust-version if you have an expected MSRV.

Make sure to add a documentation field to the Cargo.toml. docs.rs is a fine place to link to, just link to the URL that you expect the docs will appear at once generated.

Is there value in having this if docs.rs works? (And I think it should.) https://crates.io/crates/http-auth has a docs link without my putting anything in https://github.com/scottlamb/http-auth/blob/main/Cargo.toml but maybe something else needs it?

Other fields to consider would be rust-version if you have an expected MSRV.

Good call; will add this shortly.

Is there value in having this if docs.rs works?

Yes, it populates the field which appears on your crates.io page

Is there value in having this if docs.rs works?

I guess not, that certainly didn't used to be the case

A lot of projects support their MSRV by adding a CI job that checks it against that compiler. I'm not going to require it but it's something to consider.

A lot of projects support their MSRV by adding a CI job that checks it against that compiler. I'm not going to require it but it's something to consider.

Yeah, I've done that in other projects. Not sure I'm up for a fight with the workflow format this very second but I'll put it on my mental todo list.


Username disappears when sudo su while ssh-ed in

I've tried looking for this everywhere, I am not sure if it's meant to be this way or not and I couldnt find an issue about it either.
I've got all setup, .zshrc and .oh-my-zsh, in /etc/skel, and on user creation it uses spaceship, my issue is that when i sudo su into a different user, the user disappears from the prompt.
Is this the normal behaviour? if so, is there any way to make it appear? I know nohting about how zsh or theming zsh works, I apologise if the question is irrelevant.

Yes, that's normal behavior. Every user has its own shell and environment.


[Example] Create a dataflow modeling information extraction using an LLM

Write an example dataflow that uses Hamilton to model an information extraction task using an LLM.
For example:

given an output schema
given input text
make a prompt that is sent to an LLM API (pick one)
then write a function to validate the output

For inspiration you can look at Langchain's implementation.
The code for this example should end up under the /examples/LLM_Workflows directory.

Hi @skrawcz , I want to work on this issue. Can you guide me a little on what exactly needs to be done here ?

Hi @skrawcz , I want to work on this issue. Can you guide me a little on what exactly needs to be done here ?

Sure, if the following makes sense to you. If not we can see if there's another issue that would be a fit.

Are you familiar with Hamilton? If not, I suggest going through the tutorial at www.tryhamilton.dev to understand what Hamilton does. That will help you understand the code to write.
Are you familiar with LLM APIs? Do you know what a prompt is?
A use case for this work might look like the following:

I have user feedback data about some product, and I want to extract some information from it. E.g. what the product the review is about, what the sentiment is, etc. Does that make sense?

In terms of example - you can see LangChain's example which shows a schema, i.e. what we want to get back out, and an example sentence, and then the result.
i.e. given
Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.

it outputs:
[{'name': 'Alex', 'height': 5, 'hair_color': 'blonde'},
     {'name': 'Claudia', 'height': 6, 'hair_color': 'brunette'}]

LLMs aren't always good at following directions, so the example should show a check to parse & validate the output returned.
So the task would be to encode the above into a Hamilton dataflow, or DAG. With the first target being the examples/ directory. But longer term we'd ship this as part of the user contributed code library that will ship with Hamilton.

Hi @skrawcz , thanks for explaining the use case. Let me try it on a basic level and get back to you. I am taking inference from Knowledge Retrieval example under LLM workflows and I guess the core concept remains same.

Hi @skrawcz , thanks for suggesting LangChain, we can leverage the same API for this use case. On top of that, we can have a wrapper for encoding it into a Hamilton dataflow. But for implementing LangChain, we need an open_api_key, which I am unable to find. I am really curious where are you setting it to use OpenAi api in other examples. Can you please help me with that ?

thanks for suggesting LangChain, we can leverage the same API for this use case. On top of that, we can have a wrapper for encoding it into a Hamilton dataflow.

Just to be clear. We don't want to wrap langchain. We want to take a "chain" and reimplement it in Hamilton.

we need an open_api_key

Yep. You need to sign up for one. It comes with a few dollars free credit to create the API. E.g. sign up via openai.com. It also doesn't have to be openai if there's an alternative LLM API.

@skrawcz,  can I work on it? Or is it a WIP?

@skrawcz, can I work on it? Or is it a WIP?

Yeah this could be a little bit of work and figuring out how to write code. The output should look similar in style to this text_summarization example. For a quicker task I would look at https://github.com/DAGWorks-Inc/hamilton/issues/284 or #410 .

@skrawcz, I am familiar with both OpenAI´s API and Langchain. Have tried them myself. So I would love to give it a try :)


Converted CC_014 to processing

https://github.com/CodingTrain/website/issues/267
Uses mouse scroll wheel instead of a slider to change angle

Thank you @TheLastDestroyer! 🎉


Performance degradation on v6

We observe a slowdown when running tests on the beta branch, which will eventually become Starknet.js v6.
Examples:

on latest v5 (currently develop branch):  tests/account.test.ts (90.367 s)  (working with RPC v0.5.1)
on latest v6 (currently beta branch): tests/account.test.ts (261.338 s) (working with RPC v0.6)

Tests are running on devnet-rs: https://github.com/0xSpaceShard/starknet-devnet-rs
beta CI example: https://github.com/starknet-io/starknet.js/actions/runs/7627795895/job/20777335303

Happy to explorer and work on that

Hi @ivpavici I would love to work on this

Ok thanks @ivpavici

related https://github.com/0xSpaceShard/starknet-devnet-rs/issues/293

Hey!
Yes, I’m looking into this one. Sry for not writing down here. :/
@ivpavici


Accessibility: screen reader does not read results after typing / in editor

Issue Summary
On pretty much any browser and using the free, open-source screen reader NVDA, the screen reader is unable to read the results when typing / in the editor in order to go through insertable cards.
Rather, the screen reader simply reads the inserted / character, making it seem the cursor is trapped while the arrow keys are actually cycling through the cards.
To Reproduce

Open the new post editor and type a / while using a screen reader.
Observe that visually the up and down arrow keys, when pressed, cycle through cards whereas the screen reader only reports "/".

This is a bug, because it renders the dynamic parts of the Ghost Editor inaccessible to screen reader users. If a card has no Markdown equivalent, it cannot be used.
Technical details:

Presumably 2.2.4, it is the version running on the PaaS
Node Version: Unknown
Browser/OS: Windows 10 1809, Chrome 68
Database: default

This may be on interest to anyone looking to pick this issue up https://engineering.linkedin.com/blog/2018/10/making-ember-applications--ui-transitions-screen-reader-friendly

I'm afraid I don't speak Ember at all, or I'd like to help. Generally though, what you'll likely need to do is populate an invisible div of some kind with an aria-live role with the results of the / command as you arrow up and down through them, this will make the screen reader read them ouut loud automatically. However, if interactions get more tricky than just a list you select an option from, you'll need to somehow add the relevant elements to the DOM and make them focusable somehow. The Monaco editor, the editor component that powers Visual Studio Code, might have some examples to work from although that is not Ember-based I'm pretty sure.
An alternative is to make the cards searchable in some kind of help reference, and to add textual tags to insert them individually the way headings and images are inserted. So, you'd essentially be adding new tags to the Markdown parser at that point. That will probably be more work, but might not require as much specific knowledge/experience.

An alternative is to make the cards searchable in some kind of help reference, and to add textual tags to insert them individually the way headings and images are inserted.

Cards are insertable by using the / command, eg. you can type /embed and press Enter to create an embed card.

That already helps a lot, actually. I just tested the embed example, and the edit field to paste your embed URL seems to be accessible. Is there a list of accepted / commands/


Various inspect links in the glimpse user interface say they have loaded, but nothing happens

Hi,
This is what I have:

Latest Glimpse from NuGet: Glimpse Core 1.5.0 and Glimpse Asp.NET 1.3.1.
IIS 7.5 with .NET 4.5.
ASP.NET Web Forms (but mostly AJAX via ASHX requests)

It runs fine without needing to change any defaults that the NuGet install applies.
I can see basic timelines/requests etc in the Glimpse view when enabled. But I cannot drill down anywhere. For example, in the AJAX tab, if I click an "inspect" link, it load something and goes green, but I don't see any further data anywhere.
Also the trace doesn't list any server side operations (and I know there are quite a few!). Normal trace.axd shows it.
E.g. the various "Inspect" links, in the "Ajax" and "History" tabs are all defined with an href of '#'. But I can see you use data attributes to eventually become querystrings in the URL. But the # remains at the end, e.g.
http://localhost/myApp/Glimpse.axd?n=glimpse_popup&requestId=7ae2b35a-c839-45aa-b8d7-819b5ced2960&hash=c6df3424#
In my app, I follow a common pattern to manage AJAX history (back button support etc), by working with the contents after the #. If there is no content after the hash, my app performs a default action.
When your "inspect" links (and other such links with href="#")  are clicked, these cause extra interactions as a result. Is it possible that when you generate your links that you don't use # in it? Could it be this that is preventing those links from working as well?
Admittedly in Firefox if I use firebug to edit those links and remove the # manually I still can't get the "Inspect" links to work.
Any pointers as to where to look or what further info I can provide? Apologies if this is already answered elsewhere; I missed it when searching around...
Thanks!

Does glimpse work for Ajax calls using EXTJS frameworks. my ajax calls are on a click event, its a single page application, I don't see any log in Ajax tab on glimpse. Am I missing any configuration or glimpse has not yet come to that point to support such requests?

@hbopuri I am assuming this is related to the question you raised over on the other issue. I'm going to keep the conversation over there.


Latest update won't upload files

First off - love the ability to run this docker package! It significantly reduced my physical servers!
I am running this on a Synology DS1819+ with 20GB ram. It has been working great for almost a year and from time to time I need to do an update.  This is the only image running on this Synology.
Here are the steps that I have followed in the past and for this upgrade:
I go into docker and shut down the container and then clear the settings.  This go around, it scans the files but then will not start the upload. Here is a screenshot of where it stops:

After trying a few different things, I went in and cleaned out all directories where the container was and then pulled the latest again. Same results. It scans but then doesn't start the upload.
Any ideas?

I had this problem.  Apparently, you have to enumerate the share directories individually.
Jlesage offered the following  example:
-v /volume1/Documents:/volume1/Documents:ro -v /volume1/Photos:/volume1/Photos:ro
see:
https://github.com/jlesage/docker-crashplan-pro/issues/310
hal

Thanks for the idea.....after a couple of days, it rescanned and did a full maintenance and finished the backup. If it happens again, I'll definitely try your method.

I'm closing the issue, please re-open if needed.

Thank you. And thank you for an awesome container!!!
On Mon, Jul 5, 2021, 7:16 PM Jocelyn Le Sage @.***>
wrote:

Closed #313 https://github.com/jlesage/docker-crashplan-pro/issues/313.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/jlesage/docker-crashplan-pro/issues/313#event-4980487068,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AE4RXSNB3Z4DSJC3Y32KGRDTWJKO3ANCNFSM47NYQ4TA
.


Hydroform custom data path not working on windows

Description
When setting a custom path for the hydroform files on windows, it does not work and the following error is shown:
Error: unable to provision gardener cluster:
Error: No configuration files
Apply requires configuration to be present. Applying without a configuration
would mark everything for destruction, which is normally not what is desired.
If you would like to destroy everything, run 'terraform destroy' instead.

Expected result
Terraform finds the files in the given path
Actual result
Terraform returns the same error as when running on an empty directory, meaning that the given custom path is not handled properly on windows.

After investigation by @kavyakat, there is a solution found here:
https://github.com/hashicorp/terraform/issues/22668


Accessing a TypedDict field has ValidationIterator instead of the original value

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
I would expect when accessing the field of a model for that field to be the field unchanged but am not seeing that behavior. The field in this case (see code below) is overwritten with a ValidationIterator, which causes incorrect downstream behavior where I expect the field to be the original field value.
Example Code
from openai.types.chat import ChatCompletionAssistantMessageParam
from pydantic import BaseModel

class MyModel(BaseModel):
    history: list[ChatCompletionAssistantMessageParam]

history = [
    {
        "content": None,
        "role": "assistant",
        "tool_calls": [
            {
                "id": "id",
                "function": {
                    "arguments": '{"location":"Tokyo, Japan"}',
                    "name": "GetCurrentWeather",
                },
                "type": "function",
            }
        ],
    },
]
my_model = MyModel(history=history)
print(my_model.history)

"""
[{'role': 'assistant', 'content': None, 'tool_calls': ValidatorIterator(index=0, schema=Some(TypedDict(TypedDictValidator { fields: [TypedDictField { name: "id", lookup_key: Simple { key: "id", py_key: Py(0x105500270), path: LookupPath([S("id", Py(0x1055002b0))]) }, name_py: Py(0x1023563b0), required: true, validator: Str(StrValidator { strict: false, coerce_numbers_to_str: false }) }, TypedDictField { name: "function", lookup_key: Simple { key: "function", py_key: Py(0x1055003f0), path: LookupPath([S("function", Py(0x105500430))]) }, name_py: Py(0x1025aa230), required: true, validator: TypedDict(TypedDictValidator { fields: [TypedDictField { name: "arguments", lookup_key: Simple { key: "arguments", py_key: Py(0x1055002f0), path: LookupPath([S("arguments", Py(0x105500330))]) }, name_py: Py(0x10240e4b0), required: true, validator: Str(StrValidator { strict: false, coerce_numbers_to_str: false }) }, TypedDictField { name: "name", lookup_key: Simple { key: "name", py_key: Py(0x105500370), path: LookupPath([S("name", Py(0x1055003b0))]) }, name_py: Py(0x10233d1b0), required: true, validator: Str(StrValidator { strict: false, coerce_numbers_to_str: false }) }], extra_behavior: Ignore, extras_validator: None, strict: false, loc_by_alias: true }) }, TypedDictField { name: "type", lookup_key: Simple { key: "type", py_key: Py(0x105500470), path: LookupPath([S("type", Py(0x1055004b0))]) }, name_py: Py(0x102356ef0), required: true, validator: Literal(LiteralValidator { lookup: LiteralLookup { expected_bool: None, expected_int: None, expected_str: Some({"function": 0}), expected_py_dict: None, expected_py_list: None, values: [Py(0x1025aa230)] }, expected_repr: "'function'", name: "literal['function']" }) }], extra_behavior: Ignore, extras_validator: None, strict: false, loc_by_alias: true })))}]
"""

Python, Pydantic & OS Version
pydantic version: 2.7.1
pydantic-core version: 2.18.2
pydantic-core build: profile=release pgo=false
python version: 3.9.16 (main, Apr  2 2023, 22:08:02)  [Clang 14.0.0 (clang-14<IP_ADDRESS>)]
platform: macOS-14.2.1-arm64-arm-64bit

Hi @willbakst,
Thanks for reporting this. Definitely not desired behavior here - I've added this to our 2.8 milestone!

same issue. any temporary fix?

Not aware of one at the moment :(.

I’ve been using SkipValidation as my workaround while we wait :)

Supporting this. OpenAI's SDK uses TypeDict throughout, and not being able to use them in Pydantic models is a pain! Thanks!

If we just want to return the original value:
[
    {
        "id": "id",
        "function": {
            "arguments": '{"location":"Tokyo, Japan"}',
            "name": "GetCurrentWeather",
        },
        "type": "function",
    }
]

We would just need to change this line in pydantic-core.src.validators.generator(https://github.com/pydantic/pydantic-core/blob/ba8eab4acba8ad04c45f27a58a5d001c8f212361/src/validators/generator.rs#L89C1-L89C35):
impl Validator for GeneratorValidator {
    fn validate<'py>(
        &self,
        py: Python<'py>,
        input: &(impl Input<'py> + ?Sized),
        state: &mut ValidationState<'_, 'py>,
    ) -> ValResult<PyObject> {
       ...
        Ok(v_iterator.into_py(py))
    }

to
{
    ...
    Ok(input.to_object(py))
}

The into_py seems to be calling ValidatorIterator.__repr__() but I haven't figured out how.

Depending on how you're using Pydantic, you can workaround this by manually consuming the validation iterator, for example:
from pydantic import TypeAdapter
from openai.types.chat.completion_create_params import CompletionCreateParams

adapter = TypeAdapter(CompletionCreateParams)
validated = adapter.validate_python(
    dict(
        model="gpt-4o-2024-08-06",
        messages=[
            {"role": "system", "content": "You are a helpful math tutor."},
            {"role": "user", "content": "solve 8x + 31 = 2"},
        ],
        stream=False,
        response_format={"type": "json_schema", "json_schema": {"name": "x", "schema": {}}},
    )
)
messages = validated'messages']
while True:
    try:
        next(messages)
    except StopIteration:
        break

Excited to see this land in 2.10! As you can see in the linked issues, this has been causing problems for folks using the popular openai and anthropic libraries.
Is late October still the expected release date?

I'm curious how the related issue https://github.com/pydantic/pydantic/issues/9266 and it's V3 tag impact this issue and its potential for release with v2.10

For our use case, we have Iterable[T] in our types so that users can pass in any kind of iterable object but ideally after pydantic validation, the user input would be left as-is if that would be possible? e.g. if a list is given, the user should just get a list back and not an iter([])

Will chat with @davidhewitt about this early next week + get back to you folks regarding next steps!

Alright - we're going to work on a fix for this for v2.10, though it might be opt in to preserve backwards compatibility. Still TBD, as the existing behavior is quite unhelpful.

Hey folks! Wondering what the use case is for using Iterable here... in the short term, I think using a more specific type can help to solve your problem, ex: list or tuple or something similar.
Moving forward, I think the right approach here would be to add a flag like lazy_generator_eval which defaults to True (current behavior), but can be set to False to get the behavior you folks are looking for. We could even consider changing the default in V3.
Happy to look into implementation here next week, but would like to hear back re use cases for this as well.

Unfortunately it's not that we are using Iterable but that it's the type returned by e.g. OpenAI when using their API...

Wondering what the use case is for using Iterable here... in the short term, I think using a more specific type can help to solve your problem, ex: list or tuple or something similar.

We use Iterable in our types for request params in the OpenAI API as it's my understanding that it's best to accept as broad of a type as possible.
We can't use list because users would run into annoying typing variance issues, here's an example:
from typing import Iterable

def takes_list(x: list[str | int]) -> None: ...
def takes_iter(x: Iterable[str | int]) -> None: ...

a = ["foo"]
takes_list(a)  # error: Argument of type "list[str]" cannot be assigned to parameter "x" of type "list[str | int]"
takes_iter(a)  # no error!

An alternative would be to switch to Sequence but that would be a breaking change unfortunately.

Moving forward, I think the right approach here would be to add a flag like lazy_generator_eval which defaults to True (current behavior), but can be set to False to get the behavior you folks are looking for. We could even consider changing the default in V3.

Appreciate that changing your default behaviour would also be a breaking change. Would the flag be applied to all nested types or would it need to be set at every layer?
class Inner(TypedDict, total=False):
  content: Iterable[Thing]  # would the flag apply here?

class CompletionCreateParams(TypedDict, total=False):
  inner: Inner

adapter = TypeAdapter(CompletionCreateParams, config={'lazy_generator_eval': False})

If it only needs to be set at the top then that sounds like a good solution!

@RobertCraigie,
Awesome, thanks for your feedback! Understood.

We use Iterable in our types for request params in the OpenAI API as it's my understanding that it's best to accept as broad of a type as possible.

Hmm, my general thought is that it's better to narrow accepted types, but perhaps that's the bias I carry in trying to speed up validation against types (the more specific you can be with your type, the fewer validation branches we need to go down). See these performance docs, for example.

We can't use list because users would run into annoying typing variance issues

Ah yes, that is frustrating.

Appreciate that changing your default behaviour would also be a breaking change. Would the flag be applied to all nested types or would it need to be set at every layer?

The flag would follow our existing pattern for config flags, which is that it doesn't apply to nested models - you might want to do something like this in order to automatically set it at every level.
Will ping you with an update later this week when I get to work on implementing this flag 👍.

Hey folks! A brief update here:
We're pushing this to v2.11. We tried implementing a basic config flag to enable eager eval of generators / iterables, but that solution felt quite hacky, and we want to do this the right way, by creating a new core schema for Iterable types.
In the meantime, you can use this workaround:
from pydantic import BaseModel, field_validator
from typing import Iterable

class Model(BaseModel):
    values: Iterable[int]

    @field_validator('values', mode='after')
    def values_validator(cls, values) -> Iterable:
        validated_values = []
        while True:
            try:
                value = next(values)
                validated_values.append(value)
            except StopIteration:
                break
        return validated_values

m = Model(values=(1, 2, 3))
print(m.values)
# > Without custom validator: ValidatorIterator(index=0, schema=Some(Int(IntValidator { strict: false })))
# > With custom validator: [1, 2, 3]

You can even reuse that custom validator (or something similar) across models, if you'd like to always use it with Iterable type annotations.
Sorry for the delay here. We wanted to get this into v2.10, but it just doesn't quite fit with our release timeline.