Flakiness: VM gets stuck on shutdown because of background compiler?

We have a lot of flakiness on the VM bots because of timeouts that happen on tests that usually don't take that long to run.
Today alone I've seen these fail because of timeouts:

none-vm-checked debug_x64 language/vm/lazy_deopt_vm_test
none-vm-checked debug_x64 corelib/big_integer_huge_mul_vm_test
none-vm release_simarm64 standalone/io/file_read_special_device_test
none-vm release_simarm64 standalone/io/process_set_exit_code_test
none-vm release_x64 co19/LibTest/isolate/Isolate/spawn_A04_t05
none-vm release_ia32 standalone/io/http_server_close_response_after_error_test
precompiler-dart_precompiled release_arm language/assignable_expression_test/none
precompiler-dart_precompiled release_arm corelib/regexp/look-ahead_test
precompiler-dart_precompiled release_arm language/getter_override2_test/03

I saw the stack trace below and thought I'd looked suspicious:

FAILED: none-vm release_simarm64 standalone/io/process_set_exit_code_test
Expected: Pass
Actual: Timeout
CommandOutput[vm]:
diagnostics: PID 24166 - process
TID 24166:
#0  0x00007fd76d359404 pthread_cond_wait@@GLIBC_2.3.2
#1  0x00000000007dfa06 dart::Monitor::WaitMicros(long)
#2  0x00000000008c2ac2 dart::ServiceIsolate::Shutdown()
#3  0x00000000006419c6 dart::Dart::Cleanup()
#4  0x00000000005a65c5 Dart_Cleanup
#5  0x0000000000590f76 dart::bin::main(int, char**)
#6  0x00000000005912a9 main
#7  0x00007fd76c37df45 __libc_start_main
#8  0x00000000005969db _start
TID 24167:
#0  0x00007fd76c456a13 epoll_wait
#1  0x00000000005d4665 dart::bin::EventHandlerImplementation::Poll(unsigned long)
#2  0x00000000005cefe9 dart::bin::ThreadStart(void*)
#3  0x00007fd76d355184 start_thread
#4  0x00007fd76c45637d __clone
TID 24169:
#0  0x00007fd76d359404 pthread_cond_wait@@GLIBC_2.3.2
#1  0x00000000007dfa06 dart::Monitor::WaitMicros(long)
#2  0x00000000006352dd dart::BackgroundCompiler::Run()
#3  0x00000000008fa39d dart::ThreadPool::Worker::Loop()
#4  0x00000000008fa56f dart::ThreadPool::Worker::Main(unsigned long)
#5  0x00000000007dee46 dart::ThreadStart(void*)
#6  0x00007fd76d355184 start_thread
#7  0x00007fd76c45637d __clone

After investigating these timeouts more and landing https://github.com/dart-lang/sdk/commit/8347d26d7e419aff46c646acf44fd6da1cbda2ec which captures stack traces of threads of leaked dart processes (i.e. not tests themselves, but child processes of tests), I encountered this buildbot failure:
FAILED: none-vm release_simarm64 standalone/io/test_extension_fail_test
Expected: Pass
Actual: Timeout
CommandOutput[vm]:
diagnostics: PID 30774 - process
    TID 30774:
    #0  0x00007f9d8bf9c404 pthread_cond_wait@@GLIBC_2.3.2
    #1  0x00000000007e07d6 dart::Monitor::WaitMicros(long)
    #2  0x00000000008c3c82 dart::ServiceIsolate::Shutdown()
    #3  0x0000000000642386 dart::Dart::Cleanup()
    #4  0x00000000005a67c5 Dart_Cleanup
    #5  0x0000000000591176 dart::bin::main(int, char**)
    #6  0x00000000005914a9 main
    #7  0x00007f9d8afc0f45 __libc_start_main
    #8  0x0000000000596bdb _start
    TID 30775:
    #0  0x00007f9d8b099a13 epoll_wait
    #1  0x00000000005d4f65 dart::bin::EventHandlerImplementation::Poll(unsigned long)
    #2  0x00000000005cf299 dart::bin::ThreadStart(void*)
    #3  0x00007f9d8bf98184 start_thread
    #4  0x00007f9d8b09937d __clone
    TID 30777:
    #0  0x00007f9d8bf9c404 pthread_cond_wait@@GLIBC_2.3.2
    #1  0x00000000007e07d6 dart::Monitor::WaitMicros(long)
    #2  0x0000000000635c9d dart::BackgroundCompiler::Run()
    #3  0x00000000008fb67d dart::ThreadPool::Worker::Loop()
    #4  0x00000000008fb84f dart::ThreadPool::Worker::Main(unsigned long)
    #5  0x00000000007dfc16 dart::ThreadStart(void*)
    #6  0x00007f9d8bf98184 start_thread
    #7  0x00007f9d8b09937d __clone

stdout:
ERR: Unhandled exception:
ball
#0      Cat.throwMeTheBall (file:///tmp/dart_test_extension_failjK49Hg/test_extension.dart:19:72)
#1      main.<anonymous closure> (file:///tmp/dart_test_extension_failjK49Hg/test_extension_fail_tester.dart:20:23)
#2      Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:16)
#3      _Timer._runTimers (dart:isolate-patch/timer_impl.dart:385)
#4      _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:414)
#5      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:148)

OUT: * 1057d Setting working directory: /mnt/data/b/build/slave/vm-linux-release-simarm64-be/build/sdk
* 1057d Working directory URI: file:///mnt/data/b/build/slave/vm-linux-release-simarm64-be/build/sdk/
* 1057d Resolving script: /tmp/dart_test_extension_failjK49Hg/test_extension_fail_tester.dart
* 1057d Resolved entry point to: file:///tmp/dart_test_extension_failjK49Hg/test_extension_fail_tester.dart

ERR: Native extension path must be absolute, or simply the file name: extension/test_extension

OUT: * e740b Setting working directory: /mnt/data/b/build/slave/vm-linux-release-simarm64-be/build/sdk
* e740b Working directory URI: file:///mnt/data/b/build/slave/vm-linux-release-simarm64-be/build/sdk/
* e740b Resolving script: /tmp/dart_test_relative_extension_failak8i5X/test_relative_extension_fail_tester.dart
* e740b Resolved entry point to: file:///tmp/dart_test_relative_extension_failak8i5X/test_relative_extension_fail_tester.dart

Command[vm]: DART_CONFIGURATION=ReleaseSIMARM64 out/ReleaseSIMARM64/dart --ignore-unrecognized-flags --packages=/mnt/data/b/build/slave/vm-linux-release-simarm64-be/build/sdk/.packages /mnt/data/b/build/slave/vm-linux-release-simarm64-be/build/sdk/tests/standalone/io/test_extension_fail_test.dart
Took 0:04:30.153256

Short reproduction command (experimental):
    python tools/test.py -mrelease -asimarm64 -t240 standalone/io/test_extension_fail_test

Which looks like what this bug report is about.
But the taskkill step also killed a leaked process, which looks like this:

***************** Killing dart *****************
Hanging process info:
  PID: 30836
  Command line: out/ReleaseSIMARM64/dart --ignore-unrecognized-flags --packages=/mnt/data/b/build/slave/vm-linux-release-simarm64-be/build/sdk/.packages /mnt/data/b/build/slave/vm-linux-release-simarm64-be/build/sdk/tests/standalone/io/test_extension_fail_test.dart
  Stack:
    PID 30836 - process
    TID 30836:
    #0  0x00000000005f9d7c base::internal::SpinLockDelay(int volatile*, int, int)
    #1  0x00000000005f9c3c SpinLock::SlowLock()
    #2  0x00000000005fc368 tcmalloc::CentralFreeList::InsertRange(void*, void*, int)
    #3  0x00000000005f73a2 tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*, unsigned long, int)
    #4  0x00000000005f744a tcmalloc::ThreadCache::ListTooLong(tcmalloc::ThreadCache::FreeList*, unsigned long)
    #5  0x0000000000adbe94 tc_cfree
    #6  0x00007f9d8afdb1d0 __run_exit_handlers
    #7  0x00007f9d8afdb1f5 exit
    #8  0x00000000005db73c dart::bin::Process::TerminateExitCodeHandler()
    #9  0x0000000000591171 dart::bin::main(int, char**)
    #10 0x00000000005914a9 main
    #11 0x00007f9d8afc0f45 __libc_start_main
    #12 0x0000000000596bdb _start
Killed pid: 30836

=> This issue is probably the same as Issue 28246 . Merging this into Issue 28246.


wiki has exclusive info

Someone should not have to open the wiki to play their job nor learn game mechanics.

The Librarian books have been ideaguys'ed  to have this info readily available and printed out as recipes and guides--As well as pushing the librarian into a possible "teacher" role instead of just playing dnd and handing out fliers like they currently do.
An idea that doesn't conflict with this but also stands on it's own is assistants spawning with a tutorial magazine that could help show them the basics while also not having all the information on it and instead being randomized from a list to not overwhelm a new player/let them learn some of the game on their own.
Both of these require proper text formatting to be in the game however, and while I've been trying to learn how to code I've also used some of my free time to instead simply update the wiki which didn't even have a lot of info just a week ago, with food recipes notably lacking; I can try and buckle down in three or so days and knock this out so we can move onto actually making the books themselves?

Is there any chance of getting the wiki content automatically integrated into these books somehow? Rather than risk the in-game books' content lagging far behind the wiki, and having to maintain both.
On the other hand, maybe we should Keep It Simple, Silly!

Info on "overview" being brought down could be good!
It'd have to be formatted a particular way to gel properly but the formatting around the text could also be used to help indicate that it's a book, so the books on the wiki could just lead to the appropriate wiki page--In this case the jobs--when clicked on too.
It also sounds like a pain, considering anyone can edit the wiki where sure, books might need to be maintained along with wiki's but can be done in a way to mitigate that as much as possible by future-proofing the text: Not writing about currently non-roles like Science, Laying out the basics of what to do as Sec, Explaining the functions of the tools in your toolbelt as Engi... The intent is to also make formatting in the game so editing books is as easy as possible.

I'll stop ideaguysing this in GitHub after this, but may I suggest we make these Markdown files in the repo either way? That way it's easy for anyone to edit. If it's possible for UI windows to pull their content from a Markdown filepath, then this would be the way to go in my opinion.

It might be better to have the book info in game and have the wiki pull from that, we already do that with chem recipes

More ideas:

Informational posters, that you can open up in say chemistry and have basic recipes or infographics on which chem heals what.
Technical manuals, how to build the AME that comes in the box with the parts.
Other basic starter "X For Dummies" style books with a basic overview on your job inside the department so a brand new player has some basic direction.

I want to move the entire chemistry and cooking guides in-game if possible. The wiki should be optional.

we've talked about this before and settled on in-game books rendered with markdown basically, for the 'technical manuals' and 'x for dummies' books acruid mentions

Is there any chance of getting the wiki content automatically integrated into these books somehow? Not all of the content in every page obviously, but a specific "overview" section of the content on pages that would have corresponding books (job roles, etc.)
Otherwise, the in-game books' content could lag far behind the wiki, and we'd have to maintain both.
On the other hand, maybe we should Keep It Simple, Silly!

Only method would be webui, mediawiki's language is essentially a markdown built on top of bog standard HTML.

We have in-game books AND we have the guidebook, not sure if this should be closed since the guidebook hasnt quite been completed yet iirc.


S3 bucket name error

Describe the bug
aws_s3_bucket.this[0]: Creating...
╷
│ Error: Error validating S3 bucket name: only lowercase alphanumeric characters and hyphens allowed in "s3_bucket_deployments"
│
│   with aws_s3_bucket.this[0],

You just need to set a valid name for the bucket.

Correct, but module terraform can generate correct name based on the
user/CloudCraft wrong input.
I have fixed all that issues in my files. But it can potentially improve
modules.tf.
I will try create PR.
On Tue, Sep 28, 2021 at 12:33 AM Anton Babenko @.***>
wrote:

You just need to set a valid name for the bucket.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/antonbabenko/modules.tf-lambda/issues/63#issuecomment-928932956,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACGJNZRG3ORHPUCPIQTPGHTUEFVV3ANCNFSM5E4JIVTA
.

It is rather impossible to try to match requirements by AWS with user's input. Some of the examples:

SG name should not start with sg-
S3 bucket name should contain only lowercase alphanumeric characters and hyphens
...

Instead, we require developers to verify/edit and fix the errors.
Maybe we will improve this in the future but not in the near future.

I will try contribute this fix as an regexp.
We need just replace matches with - and make lower case after, also trim
multiple-.
Security group:
import re regex = r"^sg-" test_str = "sg-my-security" result =
re.sub(regex, “”, test_str).lower()
Something like this.
On Tue, Sep 28, 2021 at 12:42 AM Anton Babenko @.***>
wrote:

It is rather impossible to try to match requirements by AWS with user's
input. Some of the examples:

SG name should not start with sg-
S3 bucket name should contain only lowercase alphanumeric characters
and hyphens
...

Instead, we require developers to verify/edit and fix the errors.
Maybe we will improve this in the future but not in the near future.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/antonbabenko/modules.tf-lambda/issues/63#issuecomment-928938668,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACGJNZW3VKS6MBX4VR2O6HLUEFWXPANCNFSM5E4JIVTA
.

I see what the issue. there no real way to change/check the Names because it is Just JSON Mapping...


fix bug with unremoving EventListener

onDocumentClick  and onDocumentClick.bind(this) are different functions...
document.removeEventListener doesn't remove listener (onDocumentClick).
we can fix it by proposed way

That's right Tatyana!


GGML / GGUF formats

The model is pretty amazing and thanks a lot for open sourcing it. Is there a way to size it down and run in hardwares like Apple silicon using ggml ?
GGML
Would this improve the inference times ? For me in Apple M2 it takes 12 seconds to translate 1 sentence. If you can guide me to do this would be willing to help!

Hi @damodharanj
Yes, this should improve the inference time. However, it would require you to write the model definitions in C++ similar to llama.cpp and convert it to ggml.
Currently, we don't have the bandwidth, experience and hardware resources to help you port the models to ggml. Please let us know if there is any progress on this thread.

thanks a lot for the response! Sure will update what I can do from my end once I take this up


Blacklist DFID research outputs from the mirrors

URLs under this path have filenames which are too long, which causes errors in the crawler worker when writing to disk.
Blacklist this path temporarily until the publisher has been fixed to limit URLs to 255 characters.
@tuzz: Please would you add something to the story in your backlog to revert this change once the fix is complete?

Merge away.

Added: https://trello.com/c/PVT5Fnpd/255-truncate-over-length-dfid-urls-medium

Thanks!


Edit search label in embed config

Overview
Update embedded map UI to add a "Text search label" and search sidebar UI to show custom label which is set on the EmbedConfig
Connects #1534
Demo
Embed Config page

Embedded Map

Testing Instructions

Run ./scripts/server and log in as any users with embed permissions (e.g<EMAIL_ADDRESS>Go to the Setting - Embed page to set the search label config

[ ] The info text is descriptive enough for users
[x] When no fields are marked searchable, the custom search box is reset to the default text and disabled.
[x] No lag during typing
[ ] If the text search label is blank, it is replaced with the default text

Go to preview and embed mode map

[x] The search label changes accordingly

Checklist

[ ] fixup! commits have been squashed
[ ] CI passes after rebase
[ ] CHANGELOG.md updated with summary of features or fixes, following Keep a Changelog guidelines

In the issue #1534 description we have

Update API to allow passing text_search_label when updating EmbedConfig
Normalize empty string to null

When I am implementing this change, I realize that whenever text_search_label is empty string or null, it will be updated into the default text. Should this normalization be achieved at API level?

Taking a look now

When I am implementing this change, I realize that whenever text_search_label is empty string or null, it will be updated into the default text. Should this normalization be achieved at API level?

I think it's fine that we're managing the change to default text on the UI only, as it allows us to have a single source of truth for the default text.

This is neat. Thanks for putting this together. I left some comments.
Currently if no "searchable" checboxes are checked, the input is disabled. This is good! Can you also make the entire subsection grayed out? Maybe just set the opacity of it to, say, 60%.

Thanks for making those changes. Few more tweaks requested in some code comments. Almost there!

I made the changes accordingly. Please take another look, thank you!

Thank you both for reviewing!


TOOLS-1974 Add SSL_OP_NO_TLSv1_2 and SSL_OP_NO_TLSv1_1 to openssl

LGTM. But @xdg is the primary on this.

https://evergreen.mongodb.com/version/5aac180b2fbabe08558c1ecb


Mono last version error

NLOG crash using MONO & NLOG last versions.
See log belowe.
NLog version: 4.5.10
Platform: Mono <IP_ADDRESS>
Unhandled Exception:
System.TypeInitializationException: The type initializer for 'APP.Core.Utilities.CommonHelper' threw an exception. ---> System.TypeInitializationException: The type initializer for 'NLog.LogManager' threw an exception. ---> System.TypeInitializationException: The type initializer for 'NLog.LogFactory' threw an exception. ---> System.OverflowException: TimeSpan overflowed because the duration is too long.
at System.TimeSpan.Interval (System.Double value, System.Int32 scale) <0x74a03bb8 + 0x0010c> in :0
at System.TimeSpan.FromSeconds (System.Double value) <0x74a03e88 + 0x0001b> in :0
at NLog.LogFactory..cctor () [0x00000] in <1f7874c9e0b047af9cbf88ebd4c70184>:0
--- End of inner exception stack trace ---
at (wrapper managed-to-native) System.Object.__icall_wrapper_mono_generic_class_init(intptr)
at NLog.LogManager..cctor () [0x00000] in <1f7874c9e0b047af9cbf88ebd4c70184>:0
--- End of inner exception stack trace ---

Please fill in the template.
Also, I think it's a config error in your nlog.config. See "TimeSpan overflowed because the duration is too long." and check the internal log please

@304NotModified No.
I did some test on MONO using this single line of code :
TimeSpan DefaultFlushTimeout = TimeSpan.FromSeconds(15);
you use into the NLog.Factory cntr, and MONO crashes as well.
It's a bug introduced by MONO last version. It's not up to you.
Sorry.

Thanks for the info!
Sounds like a serious bug!

It is!


Set the 1.11 branch UI version to 1.11-dev.

High-level description

Reset the 1.11 branch UI version to 1.11-dev

It was wrongly set to 1.11.9, when the current version of 1.11 is 1.11.11 (https://github.com/dcos/dcos/blob/master/mergebot-config.json#L35)

@mesosphere-mergebot label  ready for review

But 1.11-dev and the likes do not really fit that picture.

I get this. I iterated "unofficially" with various formats, before deciding on making the major.minor branches as major.minor-dev version so that we will have minimal periodic work and sometimes don't have to do at-the-wish, "version updates" in major branches like major.minor, and will have same consistent version for "in-development" branches like 1.11, and 1.12 and 1.13 and master.
When it becomes a release branch of the format major.minor.release, it becomes the only isolated change to do, and doing it only once without being limited by other factors like full CI cycle, or getting blocked by other changes, which will be the situation, if you want to do it in major branches like x.y

In the UI I absolutely would like to see whether I am on 1.13.0-dev vs. 1.13.2-dev. It's a quite relevant difference.

Agree again. I wasn't seeing people concerned about UI versions for the clusters they tried with in-development version of the DC/OS.

I think while we are here we should do "the right thing". We don't really use the two-digit notation elsewhere so it does not seem to be right.

I agree. I will change it. I will have think about another solution (/automation) for quick periodic updates in the major branch now.

@jgehrcke , please take a look again.


Find a name for the Gateway

I'm opening this issue to hold discussion and ideas regarding the name of the "Event Gateway".
Ideas?

eventide though this might be more suited to an Event integrated development environment, and there is an existing project
Event One - a reference to the Big Bang at the start of the universe, "one" indicating that the gateway is first in line for events

Eventgate! and the logo could be a dark room with a flashlight on

Everest - nice mountain for the logo and connection of words, event + rest, so you get the gateway and you can have a rest

I really love the eventide @MikeRalphson suggested but also the Everest words combo @derberg  made here.
Let's see if we can get more genius people into the game!

Eventacle: Word play between event + tentacle. Logo could be something related to octopuses, and i'm thinking on this extremely intelligent being that has a lot of hands(middlewares) that can do many wonderful things
+1 to only calling it "Gate", like "AsyncAPI Gate", it's not something fancy, but for me it's very clear what this product does. And this could open up space to decouple the gate as much as it's possible (Gate would be the core product, and each middleware can have a diff name, and can be used or not, similar to how Golang plugins work)

@emilianozublena those names look cool as well!
I have some doubts regarding the word game on octopuses in particular because of KrakenD.
I think the "AsyncAPI Gate" is now my favorite :)
Btw, it would be awesome if you can share your usecase(s) through this new Issue Template

Agreed, after re-reading my comment, im thinking "ohhh eventacle not so good idea" :P
i'll check the use cases asap. thx!

After some time with not so much activity on this issue, I'm gonna rise this again.
At this moment, I vote for Eventgate as it is easy to remember and to understand.
What others think about it?

Rendezvous - where Data meets !


is there an "automatic creation of indexes" feature?

https://www.npmjs.com/package/loopback-connector-apachecouchdb (github: https://github.com/akat/loopback-couch-connector) has a nice and working automatic creation of indexes. does you connector has this feature as well?
the issue is that the automatic creation of indexes works fine in loopback-couch-connector, but the filtering somehow does not work for me there at all.
yous connector'sfiltering  works, but the automatic creation of indexes i do miss. are you planning to implement it?

We don't have the feature nor the plan (yet). I'd recommend using the views directly (see Readme). Personally I feel the "auto index" and "find by filters" etc are only sugars and over-engineered. But still, you are more than welcome to tell us how it should be implemented or shoot us a PR.

+1


List Language Translations API: support new parameters

Crowdin has recently added new parameters to the List Language Translations API endpoint:

branchId
directoryId

These parameters should also be supported in the Crowdin API client. It would also be good to check if other parameters are supported.
References:

List Language Translations

Hi @andrii-bodnar, can I work on this?

Hi @Riju-bak, sure, thank you!

@andrii-bodnar Here's the PR


📣 [Promote]: Create blog post about SPFx Toolkit manage capabilities

🎯 Aim
The aim is to create a blog post that would describe all of the different SPFx Toolkit management capabilities. So mainly focusing on the first two views of the toolkit

The aim would be to describe:

login experience
environment details - so basic links from tenant and health incidents list
app catalog list - what is present there, what are the manage options we have for each app like enable/disable, remove, upgrade ect.
GitHub Copilot Chat Participant - the /manage chat command to retrieve basic data from tenant

Ideally, the blog post should be added to our community PnP blog: https://pnp.github.io/blog/
🙏 Help
Are you a regular user of SPFx Toolkit and you wanna help out but don't have any skills in developing VS Code extensions then this is a perfect issue for you! Help us out to promote the tool so that we may spread the word and reach more SPFx developers and help them rock 🤩
Tip
Previous marketing things done are all grouped in this wiki page. Adding them here for inspiration

I’ll take this @Adam-it

I’ll take this @Adam-it

All yours.
Please base your blog on the latest (pre) release as some improvements to the to the /magae chat command or to the app catalog list view were merged just a few days ago. Also we could recheck with progress on this issue #350
which is an awesome manage capability and also include it in the blog post once it out.
And of course mention that we are adding more and more in this area
Please tag me over when the blog post is ready 🙏.
Thanks for your help in spreading the news. You Rock 👏🤩


Failed to import - js-untar

Failing module

GitHub: https://github.com/InvokIT/js-untar
npm: https://www.npmjs.com/package/js-untar

import untar from<EMAIL_ADDRESS>
Error message
After running deno run I got this:
Import<EMAIL_ADDRESS>failed: 500 Internal Server Error

Additional info

esm.sh version: v89
Deno version: 1.24.1

Strangely enough https://esm.sh/js-untar@2.0.0?target=es2020 seems to work fine, only Deno builds fail.

This seems to work now. I think b81ea0f66c41a61f6db8666de3748ea0c629919f might have fixed it.


fix type of keywords entry in frontmatter (in /docker-for-windows/ dir)

Describe the proposed changes
in /docker-for-windows/ :
change keywords's type from array to string.
Project version
current stable
Related issue
contributes to #435
Signed-off-by: Gaetan de Villele<EMAIL_ADDRESS>

Deploy preview processing.
Built with commit 738546c4365854cb619906dad06d52976a452d80
https://app.netlify.com/sites/docker-docs/deploys/58228ad0c4d9cc157794cf0c

Thanks, @gdevillele !


Updating Wiki info

I am not able to update the wiki info. A guide on how to call the api will be use full

I sent you an invite to collaborate on the project. Feel free to update the wiki if needed. Over the next couple of days I will tidy up this GitHub project a little.

I will try to solve the api - call counter. Then we can do the beta release. There are going to be some bugs for sure. But dont have time to do all the testing

Api call count is solved.


Update dependencies & add update-grayskull bot

Checklist

[X] Used a personal fork of the feedstock to propose changes
[X] Bumped the build number (if the version is unchanged)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[X] Ensured the license file is being packaged.

Fixes #92

Thanks for this!
Could you juste please enlighten me on what Grayskull is about ?

Grayskull is a package that generates conda recipes.  I had posted a question about if there was a better way to keep conda-forge packages up-to-date with their upstream repos, and I was told that adding this to the conda-forge.yml is one way to do that.


How can i use sdk to trace event and generate full callstack sample.

I want to trace my native code in application without command line, expect to generate some method full callstack and cpu time.
Currently i use sdk sample to trace my method, but the result has no callstack,only trace event.
My question is: does sdk have the feature? And how?

If you are on Android, you can use callstack sampling as described on this page through Perfetto - https://perfetto.dev/docs/quickstart/callstack-sampling.


Add support for environment variables, and files added as link, both in configuration, and in the .ts files

I'm asking for a feature.
In industry, cost plays an important role, and a lot of measures are taken to reduce costs and boost productivity.
For that reason, we only use Visual Studio IDE. And we've built a lot of our company's architecture on the features it provides for us.
Two amazing features VS offers are:

environment variables
add as link

Based on these valuable features, we thought of using TypeScript in our organization-wide architecture.
We include a bunch of TypeScript files in different hierarchies and file-system architecture throughout the entire organization, and we need to be able to reuse them across the entire organization.
For example, we want to have one central tsconfig.json file, and reuse it across all of our projects, because we have a SPL, and all of our projects are like each other in technology stack.
But we tried to include that central tsconfig.json in other projects using "add as link" feature, and TypeScript doesn't recognize it.
Also we would love to use environment variables in our import clauses like:
import DateTools from '%ProjectsRoot%/Framework/Scripts/DateTools'
Please add support for environment variables, and add as link features to make working with TypeScript smoother in Visual Studio for multi-project environments with heavy code reuse.
Thank you so much.

I also saw #18755 and #31648. And I realized that you have design limitations. At least support environment variables. Environment variables are first-class entities throughout the entire software world. They exist in any operating system, any programming language, and almost any IDE.

Also it's correct that you have --preserveSymlinks compiler option, but symlinks can't be managed in TFS (Azure DevOps) client for Visual Studio, which is called Team Explorer. When you get the latest version on another machine, it gets the real file without creating the symbolic link for it. Thus it's not useful in real-world scenarios.

For example, we want to have one central tsconfig.json file, and reuse it across all of our projects, because we have a SPL, and all of our projects are like each other in technology stack.

I'm pretty sure you can do this today, using extends:
{
    "extends": "/path/to/main-tsconfig.json"
}

You can then add any project-specific overrides, if necessary.

@fatcerberus the point is we can't use environment variables in that. We can't also use abslute paths, since it breaks local builds on developers' machines. For example we can't have a tsconfig.json file in C:\Projects\SomeProject that is as follow:
extends: '%FrameworkPath%/TypeScript/CentralTsConfig.json'
And if we have this, then our colleaguea won't be able to work, unless they all follow file-system architectur strictly:
extends: 'C:\\Framework\\TypeScript\\CentralTsConfig.json'
Because for example one of develoers map projects to his D folder, in a different file-system architecture.

Surely you can use relative paths, though.
TS is not here to provide new ways of naming modules - this is a runtime concern

I think he wants the environment variables to be resolved at compile time.

@RyanCavanaugh relative paths break when developers map repositories differently. Relative paths only work when we force a file-system archietcure, which in turn spawns other problems.

@fatcerberus thanks for mentioning goals and non-goals. I fail to see how this is related to an end-to-end build issue. The other feature we mentioned

Use Babel + plugin to achieve this?

@milesj isn't it ridiculous to use two JavaScript compilers/transpilers? Do you write Java in C#? Do you use SQL Server Management Studio for MySQL?

@nefcanto  I type check with TS but compile with Babel because it provides the entire plugin ecosystem. It's the primary reason TS support was added to Babel.
For example, with Babel I can compile __DEV__ constants in my code to process.env.NODE_ENV !== 'production'. The same can be done with your environment variables within import paths.

@milesj what you is a very good and vivid reason TypeScript needs more features. Parallel technologies should not be used in real-world exapmles for a lot of reasons. Higher costs, steeper learning-curves, more painfull upgrades, less hiring chances, more compatibility to look for, less flexibility for innovation, and much more.
Angular is enough, so that you don't need React or Vue with it. IMHO what you said is a big red flag for TypeScript team to consider.

TypeScript is a transpiler that transpiles superset of JavaScript into different flavors of JavaScript. Babel is a transpiler of JavaScript. So, I have to use the handle of this spoon, and the cup of the other spoon because they both lack to be a real spoon?
And your solution won't work for us. Because we need to be able to do so:
import CountryParser = require('%FrameworkProjectsRoot%\TypeScript\Geo\CountryParser');
Using Babel won't help us getting the type-checking and intellisense and more features here. Because it's a design-time feature, not compile-time.

No, it's not a question. It's a feature request.


Add strong typesystem

Add the strong type system that we were talking about when the language was not on github.
Types
1. u8
2. u32
3. u64
4. f32
5. f64
6. i8
7. i32
8. i64
9. String
10. void

Syntax
var wouw: String = "";
var testn: i32 = 100;

function testnf() -> i32 {
    return 100;
}

// Lets also try and make it so that no types is not acceptable.

These are the ones that are currently in my mind. Might have forgotten to mention some other ones.

LOANS FOR 2% PERSONAL LOAN & BUSINESS LOAN OFFER APPLY NOW CITY FINANCING LOAN OFFER APPLY NOW (all location) Apply for a quick and convenient loan to pay off bills and to start a new financing your projects at a cheapest interest rate of 2%. Do contact us today contact via email<EMAIL_ADDRESS>WhatsApp OR CALL +918130061433


Add support for maven wrapper

The maven wrapper project adds a tiny shell script wrapper to maven builds, much like gradlew does for gradle builds. This way users can choose which maven version should be used by specifying it in a special properties files. Maven wrapper will then download the specified maven version to the m2 repository and use it for building the project. The mvnw wrapper script has the same interface as mvn itself.
This PR adds mvnw detection to JVM builds.

Looks good. Could you also document this in https://github.com/travis-ci/docs-travis-ci-com? (https://github.com/travis-ci/docs-travis-ci-com/blob/gh-pages/user/languages/java.md)

@BanzaiMan done, see travis-ci/docs-travis-ci-com#661


Jesse: Watch mukesoft.com

Jesse requests the watch of the watch_keyword mukesoft\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

mukesoft\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Makyen in Charcoal HQ


Springfox/swagger doesn't recognize the ** as query parameter

Hi all,
I'm using Springfox version 2.9.2 and i have a question.
I define a API which have a query parameter which define as ** under the API method which i pull using the HttpServletRequest (** is used for supporting special characters)

when accessing this API from swagger UI not parameter is defined and there is no way to use swagger in order to use this API (No query parameter text field)

Can someone help on how to define this parameter so i can use swagger to test this API?

If you want swagger-ui to respect the wildcard endpoint, you need to describe all the things the UI has access to int the metadata. IOW your expectation that swagger-ui will allow a wildcard entry is not going to work.

Thanks for the answer.
Can you give me an example how to do it? or it just cannot be done?

@AfikGrinstein you could try using @ApiImplicitParams annotations, but its kinda awkward. In the endpoint, you're saying you'll accept anything, but for the purpose of the swagger specification, you'd end up describing the parameters. Why not describe the parameters natively?

Thanks for the tip about @ApiImplicitParams, I used it and it solved my issue :
https://github.com/voyages-sncf-technologies/hesperides/pull/671/files
In my case I had to use a HttpServletRequest because this path parameter can contain slashes...

Hi guys,
My Controller code as below. But when I parse request to get custCode, it returns custCode = ** meanwhile I expect it is custCode value enterted on Swagger-UI.
(I use Swagger 2.8.0)
Can you please help suggest a solution for this.
Thanks
@GetMapping("/rest/**")
@ApiOperation("Get customer by name and code")
@ApiImplicitParams({
@ApiImplicitParam(name = "custCode", required = true, dataType = "string", paramType = "path"),})
public ResponseEntity getCustomer(@PathVariable("custName") final String custName,
HttpServletRequest request)
{
//Some Code
}


Upload Gamecity

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

[ ] Filename in the lowercase format
[ ] Image format: png
[ ] Image size: 256 by 256 px

Added lowercased version here https://github.com/TrustWallet/tokens/commit/92c44810037d0a44e9a4b5e2971d01e9cdc1bd87
Also, you can use these graphics to say that your ERC20 token fully compatible with Trust Wallet (https://trustwalletapp.com/logo-assets). Tokens will show up automatically once the user receives them.
Feel free to share this on social media and tag Trust Wallet:

Twitter: @trustwalletapp

Facebook: @trustwalletapp


[Feature]: Add attendee info on scanning page

New feature you are looking for
Add attendee information to the scanning page
Any additional context or links?
No response

Duplicate of #162


Provider plugin crashes on destroy

What happened?
I dynamically generated a provider from https://github.com/Constellix/terraform-provider-constellix. I can create a resource just fine. When I try to delete the resource, the plugin crashes. I've tried to reproduce this issue with a TF program but it worked without issues, which leads me to believe there is something wrong in the bridge perhaps? Note that the resource is not actually deleted so the plugin seems to crash early. I have to go in to the Constellix console and manually remove the CNAME and refresh the state as a workaround.
error: rpc error: code = Unavailable desc = error reading from server: EOF:

    Stack trace from the terraform-provider-constellix_v0.4.6 plugin:

    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xc442e8]

    goroutine 41 [running]:
    github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).ApplyResourceChange(0xc00000f218, {0xc00021d650, 0x4b4f86}, 0xc00021d650)
        github.com/hashicorp/terraform-plugin-sdk@v1.17.2/internal/helper/plugin/grpc_provider.go:812 +0x1a8
    github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0xe6b320, 0xc00000f218}, {0x1076d50, 0xc0003991a0}, 0xc000361320, 0x0)
        github.com/hashicorp/terraform-plugin-sdk@v1.17.2/internal/tfplugin5/tfplugin5.pb.go:3305 +0x170
    google.golang.org/grpc.(*Server).processUnaryRPC(0xc00021b180, {0x1086918, 0xc000182900}, 0xc000310d00, 0xc0002f65d0, 0x1684ac0, 0x0)
        google.golang.org/grpc@v1.32.0/server.go:1194 +0xc8f
    google.golang.org/grpc.(*Server).handleStream(0xc00021b180, {0x1086918, 0xc000182900}, 0xc000310d00, 0x0)
        google.golang.org/grpc@v1.32.0/server.go:1517 +0xa2a
    google.golang.org/grpc.(*Server).serveStreams.func1.2()
        google.golang.org/grpc@v1.32.0/server.go:859 +0x98
    created by google.golang.org/grpc.(*Server).serveStreams.func1
        google.golang.org/grpc@v1.32.0/server.go:857 +0x294

    Error: The terraform-provider-constellix_v0.4.6 plugin crashed!

    This is always indicative of a bug within the plugin. It would be immensely
    helpful if you could report the crash with the plugin's maintainers so that it
    can be fixed. The output above should help diagnose the issue

Example
import pulumi
import pulumi_constellix as constellix

config = pulumi.Config()

domain = config.require("domain")

constellix_domain = constellix.get_domain_output(name=domain)

cname = constellix.CnameRecord(
    "demo",
    domain_id=constellix_domain.apply(lambda d: d.id),
    host="google.com.",
    ttl=300,
    name="a-demo",
    noanswer=False,
    gtd_region=1,
    record_option="roundRobin",
    source_type="domains",
)

Output of pulumi about
CLI
Version      3.134.0
Go Version   go1.23.1
Go Compiler  gc

Plugins
KIND      NAME                VERSION
resource  azure-native        2.62.0
resource  azuread             4.3.0
resource  kubernetes          4.17.1
language  python              unknown
resource  random              4.16.3
resource  terraform-provider  0.2.0
resource  tls                 4.11.1

Host
OS       ubuntu
Version  20.04
Arch     x86_64

This project is written in python: executable='******' version='3.8.10' <--- Someone else reproduced the issue with 3.12 too.

Dependencies:
NAME                 VERSION
pip                  20.0.2
pkg-resources        0.0.0
pulumi-azure-native  2.62.0
pulumi-azuread       4.3.0
pulumi-constellix    0.4.6
pulumi-kubernetes    4.17.1
pulumi-random        4.16.3
pulumi-tls           4.11.1
setuptools           44.0.0

Additional context
Here's the equivalent TF program I used to check if the same issue occurs in TF.
terraform {
  required_providers {
    constellix = {
      source  = "Constellix/constellix"
    }
  }
}

provider "constellix" {
  apikey    = "xxxxxxx"
  secretkey = "xxxxxxxx"
  insecure  = true
}

data "constellix_domain" "domain1" {
  name = "<DOMAIN NAME>"
}

resource "constellix_cname_record" "demo" {
  name = "a-demo2"
  host = "google.com."
  ttl = 100
  noanswer = false
  gtd_region = 1
  record_option = "roundRobin"
  source_type = "domains"
  domain_id = "${data.constellix_domain.domain1.id}"
}

Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

Hi @praneetloke - thank you for sending this issue.
We'll be taking a look as soon as we can. It is certainly odd that the panic is happening in the terraform plugin; thank you very much for verifying the TF-only version of this.

@guineveresaenger just wondering if there is any other info I can gather to help diagnose this? Not sure if you've been able to reproduce the issue?

Hi @praneetloke. It's possible you could unblock us by giving us a state file that we can import and then run pulumi destroy on, so we could reproduce the panic. Depending on where the panic is, that might work. If we need real credentials, then that won't help.
I think that we need https://github.com/pulumi/pulumi-terraform-provider/issues/37 to gather enough information to act here in the general case. We have that issue scheduled for this iteration.

I've sent the state file privately to @iwahbe. Let me know if you need anything else to help diagnose this.

Had the same issue now while trying to use generated provider from https://registry.terraform.io/providers/signalsciences/sigsci/latest
Diagnostics:
  sigsci:index:SiteRule (rule-test4):
    error: rpc error: code = Unavailable desc = error reading from server: EOF:

    Stack trace from the terraform-provider-sigsci_v3.3.0 plugin:

    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xbc9fd1]

    goroutine 91 [running]:
    github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).ApplyResourceChange(0xc000012128, {0xc0000f7110?, 0x4c05e6?}, 0xc0000f7110)
        github.com/hashicorp/terraform-plugin-sdk@v1.14.0/internal/helper/plugin/grpc_provider.go:802 +0x1b1
    github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ApplyResourceChange_Handler({0xd9c460?, 0xc000012128}, {0xf578f0, 0xc00071a570}, 0xc0005f5020, 0x0)
        github.com/hashicorp/terraform-plugin-sdk@v1.14.0/internal/tfplugin5/tfplugin5.pb.go:3305 +0x170
    google.golang.org/grpc.(*Server).processUnaryRPC(0xc000428480, {0xf5b420, 0xc000328000}, 0xc0006b8d00, 0xc00051d470, 0x14921e0, 0x0)
        google.golang.org/grpc@v1.27.1/server.go:1024 +0xd4f
    google.golang.org/grpc.(*Server).handleStream(0xc000428480, {0xf5b420, 0xc000328000}, 0xc0006b8d00, 0x0)
        google.golang.org/grpc@v1.27.1/server.go:1313 +0xa25
    google.golang.org/grpc.(*Server).serveStreams.func1.1()
        google.golang.org/grpc@v1.27.1/server.go:722 +0x98
    created by google.golang.org/grpc.(*Server).serveStreams.func1
        google.golang.org/grpc@v1.27.1/server.go:720 +0xea

    Error: The terraform-provider-sigsci_v3.3.0 plugin crashed!

    This is always indicative of a bug within the plugin. It would be immensely
    helpful if you could report the crash with the plugin's maintainers so that it
    can be fixed. The output above should help diagnose the issue.


Is it possible to bypass a particular circumstance which is throwing the .imageMapping error?

Hey gang! Running Moya 11.0.2
So I've run into an interesting situation with a value in a json payload called banner_image, which is usually a String with a path to an image, but it can also sometimes be null
eg (roughly) :
{
"foo": 1234,
"bar": false,
"banner_image": "http://baz.com/image.png"
}

and sometimes:
{
"foo": 1234,
"bar": false,
"banner_image": null
}

I'm not actually using the banner_image and not currently mapping the value in the relevant model within the app. When the banner_image has a value, the request is successful and I decode the response data and carry on.
However, when banner_image returns null, switching on the result fails with the .imageMapping error.  EDIT: I found I was still running Moya 9, and after upgrading to Moya 11 it now it returns a parameterMapping error with the message  (more or less) "can't do; corrupt data" ;)
I tried mapping the image value to my model with an optional String to see if that would make it happy, but that didn't work.
Curiously, another endpoint in the app also has a value that is sometimes a String, and sometimes null, but this value is called "status":
{
// other keys etc
"status": "woohoo"
}

{
// other keys etc
"status": null
}

I'm also ignoring this status value when decoding a successful response, however the requests don't fail in this case when status is null.
So I'm wondering if there is some smart detection in the inner workings of either Moya or Alamo that recognizes the word image in a json key and attempts to do something (___ ?) and then fails when the value is null?
Not sure how far or close to accurate my assessment is, but if this makes sense to someone, is there a flag or setting or some way to bypass this behavior?
Thanks so much, love love love Moya!

It was something completely unrelated to my (in retrospect) droolings of a madman above. I'm a close minded fool and was blinded by what I conceptualized "imageMapping" to mean. Sorry for the waste of time my Moya friends!

@mossby Glad you could get this one resolved 👍 No worries, I make the most outrageous mistakes all the time 😆


volare enable remotedisconnected

I was running make of OpenLane in WSL 2 with all prerequisites satisfied including Docker. When it ran ./venv/bin/volare enable after creating a venv and upgrading volare in the venv, it seemed to wait for a few seconds and showed:

[10:09:34] Found version de752ec0ba4da0ecb1fbcd309eeec4993d88f5bc in ./dependencies/tool_metadata.yml.     common.py:226
Traceback (most recent call last):
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.10/http/client.py", line 1374, in getresponse
response.begin()
File "/usr/lib/python3.10/http/client.py", line 318, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.10/http/client.py", line 287, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
resp = conn.urlopen(
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 550, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
raise value.with_traceback(tb)
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
six.raise_from(e, None)
File "", line 3, in raise_from
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.10/http/client.py", line 1374, in getresponse
response.begin()
File "/usr/lib/python3.10/http/client.py", line 318, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.10/http/client.py", line 287, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/mnt/d/Resources/OpenLane/./venv/bin/volare", line 8, in
sys.exit(cli())
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/volare/manage.py", line 320, in enable_cmd
enable(pdk_root=pdk_root, pdk=pdk, version=version)
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/volare/manage.py", line 240, in enable
with requests.get(link, stream=True) as r:
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/mnt/d/Resources/OpenLane/venv/lib/python3.10/site-packages/requests/adapters.py", line 547, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

I tried to run the command in bash but it gives the same error. The connection to GitHub is proved to be of no problem. Are there any ways to know what happened and what caused this?

Hello Liu- not really sure, RemoteDisconnected means something may be up with your connection to GitHub. It could be that if you're sharing your IP with a whole bunch of people, GitHub is throttling connections.
To solve this problem, I would have to somehow make the download more resilient and/or enable people to download the .tar.xz themselves and just have volare import it.
For now, you can try volare build -j$(nproc) instead of volare enable, which will build the PDK from sources.

Thank you donn. It is true that my connection to GitHub was still problematic and very unstable. Now I changed to a completely "normal" network of educational network, and it works.


Injecting a scope variable into rhai's global module

Hi!
I was looking for a way to create constants that can be injected into a script via rhai::Scope while being accessible to any function.
Please checkout the pseudo-code below.
// engine.rs
let engine = rhai::Engine::new();
let mut scope = rhai::Scope::new();

scope.push_constant("STATE", ...);
let ast = engine.compile(script);
engine.call_fn(&mut scope, &ast, "my_func", ()));

// script.rhai

fn print_state() {
  print(STATE.something);
  print(STATE.something_else);
}

// this works.
print(STATE.something);
// this does not.
print_state();

When calling print_state I get:
Variable not found: CTX (line 3, position xx) in call to function print_state < anon$ec0bd323da317e6d
I guess the constant in the scope is not declared as global, and therefore is not accessible by any function.
Is there a way to set a constant via rust's scope in the global module ?

Rhai functions are pure, so they typically cannot access global variables.  That's why they can be taken apart and merged again.
However, this is common enough that you have two alternatives:

https://rhai.rs/book/engine/var.html

https://rhai.rs/book/patterns/constants.html

Ok so, I can't use your first solution because my use of the engine is shared between threads, has well as the ast, and I need to create a scope on events with a certain state in it. (sorry, my message was missing some context)
For the second solution, it is the same as above, I need to inject the constant at "runtime". I can't use a variable resolver.
Same for the third one.
The option I would go for is this: https://rhai.rs/book/patterns/singleton.html
But can you use the singleton in a function without passing it by parameter ? Or is it possible to "re-export" it to the global module ?

Ah.  You want different constants for different runs based on the same AST.
Option 1: This is a typical use of variable resolvers.  Are you sure you cannot use this?
Option 2: You can "re-optimize" an AST using Engine::optimize_ast which is reasonably efficient.  So keep the original AST but re-optimize it based on new constants before each run.  As long as you're not doing this millions of times a second that's OK.
Option 3: Engine::call_fn will use constants in your scope when running the named function.  It doesn't, however, propagate those into sub calls to other functions though - which is probably what you want.  In other words, in your example, accessing STATE within my_func should work, but if my_func calls print_state it won't work.

But can you use the singleton in a function without passing it by parameter ? Or is it possible to "re-export" it to the global module ?

You can definitely use a singleton in a function without passing it by parameter.  Just define a function that returns it.
In fact, this is encouraged because that function can easily be changed to add new functionalities without touching any scripts.
For example, you can do:
fn print_state() {
    const STATE = get_current_state();
    print(STATE.something);
    print(STATE.something_else);
}

In that case, all your custom logic resides in the function get_current_state and you don't have to ever worry about scopes, constants etc.

Option 1: This is a typical use of variable resolvers. Are you sure you cannot use this?

No not really. As I understood a variable resolver does not work in my case because i am working in a multithread environment. I would need to call engine.on_var in every thread, using a mutex on the engine. this is not what I want.

Option 2: You can "re-optimize" an AST using Engine::optimize_ast which is reasonably efficient. So keep the original AST but re-optimize it based on new constants before each run. As long as you're not doing this millions of times a second that's OK.

Re-optimizing the ast in a multi-threaded environment would mean that I have to lock the ast under a mutex right ? If that is the case, this isn't what I want.

Option 3: Engine::call_fn will use constants in your scope when running the named function. It doesn't, however, propagate those into sub calls to other functions though - which is probably what you want. In other words, in your example, accessing STATE within my_func should work, but if my_func calls print_state it won't work.

Ok I did not notice that ! does something like engine.eval propagates the constant into other functions scopes though ?

You can definitely use a singleton in a function without passing it by parameter. Just define a function that returns it.

This seems like the best solution so far ! I'll give it a shot and keep you posted.
Thank you for all your propositions !

Well, actually, I did not find a way to inject a constant injected via a scope in a rhai function, do you known if there is a way to do that with the Singleton Command Object example ?

I could also try this example https://rhai.rs/book/patterns/parallel.html#one-engine-instance-per-call with the variable resolver as you suggested !

Re-optimizing the ast in a multi-threaded environment would mean that I have to lock the ast under a mutex right ? If that is the case, this isn't what I want.

Ok I did not notice that ! does something like engine.eval propagates the constant into other functions scopes though ?

Engine::eval_with_scope does not propagate constants into functions.  In fact, even Engine::call_fn doesn't; it just treats that single function as a whole script.

I'm actually trying to make this work with engine::new_raw & the variable resolver right now, if that does not work, I'll look into this ! i'm just a little bit scared that the ast could be expensive to clone. Thank's again !

In that case, you might really want to think about exposing your API via a set of functions instead of exposing a singleton object.

Unfortunately, I cannot do that. the rhai environment needs to mutate a state that I inject via the scope and that I can get back once the engine evaluation is done.

Or you might want to check out the events handler pattern: which actually I think is what you may be looking for: https://rhai.rs/book/patterns/events.html

Yup I saw that too, I'll take a look into this.

Unfortunately, I cannot do that. the rhai environment needs to mutate a state that I inject via the scope and that I can get back once the engine evaluation is done.

You can easily do that.  If that state is a single (meaning it is Arc<RwLock<...>> wrapped), then simply have a function that returns it and register the API on top of that object.
let state = get_current_state();
state.property = "new string";
state.update(42);

Sometimes people break down mutable properties into sets of functions if the API surface is small enough.  This can be a very simple style for casual users.
set_current_prop("new string");
update_state(42);

There are multiple ways to access a global state from all functions in the script, and I think that's what you're looking for.
As I have outlined above, they fall under the following categories:

Always bind said state to the this pointer

Register a function-based API for each action on that state

Use a function call to get access to a shared version of the state

Okay I see where what you mean, this is exactly what I'm looking for. I'll keep you posted ! Thanks again !

So I've made it all work with the One Engine Instance Per Call pattern & using the Engine::on_var function to inject the data I want from rust in rhai functions like get_current_state and it works perfect!
Unfortunately, I just checked out the 1.7 release and it seems Engine::on_var is deprecated, or at least unstable. Do you known if there is a "stable" way to use the variable resolver in 1.7 ?

It is marked deprecated because it is considered an "advanced" or "volatile" API - meaning that I'm free to change it in the future if the needs arise and you may have to adjust your programs a bit...
Rust doesn't have any way to mark something as "volatile" so "deprecated" is used instead.
on_var will always be there in the future, however its "volatile" status mean that I may add new parameters to the callback in the future if needs arise (although that chance may be rare).
If the API is stable for a long time, I'll probably remove the volatile status.

Okay, that seems logical. I'll adapt the code if you feel that the api needs to change.
Anyway, everything is good in my end, I'll close the issue.
Thanks again for the help, ill ask my questions in discord from now on.
have a great day !


Typings improvement.

I want to suggest some improvement on typings:

Mark the length property as readonly since only getter is defined for this property.
I would use optional arguments syntax, e.g. function foo(argument?: whatever) {} instead on consequencial overloads since it's considered as a best practice. (https://github.com/invertase/denque/blob/master/index.d.ts#L4-L8)

We're willing to accept a PR to improve the types if you're interested?

Sure, i'll open a PR in a couple of days.


Importing gluonnlp giving :'ImportError: cannot import name '_get_repo_url' '

All libraries were upgraded; running on Ubuntu
A simple import of gluonnlp giving the following error
Traceback (most recent call last):
  File "lm.py", line 25, in <module>
    import gluonnlp as nlp
  File "/home/venv/gluon/lib/python3.6/site-packages/gluonnlp/__init__.py", line 23, in <module>
    from . import data
  File "/home/venv/gluon/lib/python3.6/site-packages/gluonnlp/data/__init__.py", line 23, in <module>
    from .utils import *
  File "/home/venv/gluon/lib/python3.6/site-packages/gluonnlp/data/utils.py", line 32, in <module>
    from mxnet.gluon.utils import _get_repo_url, download, check_sha1
ImportError: cannot import name '_get_repo_url'

Currently gluonnlp requires the pre-release version of mxnet. You can try pip install --pre --upgrade mxnet

@Aashit-Sharma let me know if there's still issue. Thanks

Earlier I ran into this issue https://github.com/apache/incubator-mxnet/issues/11911 and then I downgraded by mxnet version to 1.1.0 . After upgrading mxnet again to the solution above you have mentioned @szha  I am getting the the error mentioned @Aashit-Sharma. Following are the logs.
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ pip install mxnet-cu80==1.1.0
Collecting mxnet-cu80==1.1.0
Downloading https://files.pythonhosted.org/packages/9c/55/bcfd26fd408a4bab27bca1ef5dc1df42954509c904699a6c371d5a4c23ab/mxnet_cu80-1.1.0-py2.py3-none-manylinux1_x86_64.whl (274.9MB)
100% |████████████████████████████████| 274.9MB 172kB/s
Collecting requests==2.18.4 (from mxnet-cu80==1.1.0)
Downloading https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl (88kB)
100% |████████████████████████████████| 92kB 4.1MB/s
Collecting graphviz==0.8.1 (from mxnet-cu80==1.1.0)
Downloading https://files.pythonhosted.org/packages/82/cc/668725769a1b184322019b494bbdfbb82bbef5cecc1dd8be6f778cf1422b/graphviz-0.8.1-py2.py3-none-any.whl
Collecting numpy<=1.13.3 (from mxnet-cu80==1.1.0)
Downloading https://files.pythonhosted.org/packages/0d/41/6c224571decd61c2578baedfdb0eec6283617c6679c35b20973f4e68aeaf/numpy-1.13.3-cp35-cp35m-manylinux1_x86_64.whl (16.9MB)
100% |████████████████████████████████| 16.9MB 1.3MB/s
Collecting urllib3<1.23,>=1.21.1 (from requests==2.18.4->mxnet-cu80==1.1.0)
Downloading https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl (132kB)
100% |████████████████████████████████| 133kB 3.2MB/s
Collecting idna<2.7,>=2.5 (from requests==2.18.4->mxnet-cu80==1.1.0)
Downloading https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl (56kB)
100% |████████████████████████████████| 61kB 4.9MB/s
Requirement already satisfied: certifi>=2017.4.17 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests==2.18.4->mxnet-cu80==1.1.0) (2018.11.29)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests==2.18.4->mxnet-cu80==1.1.0) (3.0.4)
Installing collected packages: urllib3, idna, requests, graphviz, numpy, mxnet-cu80
Found existing installation: urllib3 1.24.1
Uninstalling urllib3-1.24.1:
Successfully uninstalled urllib3-1.24.1
Found existing installation: idna 2.7
Uninstalling idna-2.7:
Successfully uninstalled idna-2.7
Found existing installation: requests 2.20.1
Uninstalling requests-2.20.1:
Successfully uninstalled requests-2.20.1
Found existing installation: graphviz 0.8.4
Uninstalling graphviz-0.8.4:
Successfully uninstalled graphviz-0.8.4
Found existing installation: numpy 1.14.6
Uninstalling numpy-1.14.6:
Successfully uninstalled numpy-1.14.6
Successfully installed graphviz-0.8.1 idna-2.6 mxnet-cu80-1.1.0 numpy-1.13.3 requests-2.18.4 urllib3-1.22
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ python train_transformer.py --dataset "/home/user1/nmt_tensorflow_git/nmt/scripts/wmt16_de_en" --src_lang en --tgt_lang de --batch_size 2700 --optimizer adam --num_accumulated 16 --lr 2.0 --warmup_steps 4000 --save_dir "/home/user1/transformer_nmt/ready_to_run_transformer/saved_dir" --epochs 30 --gpus 0,1,2,3,4,5,6,7 --scaled --average_start 5 --num_buckets 20 --bucket_scheme exp --bleu 13a --log_interval 10
Traceback (most recent call last):
File "train_transformer.py", line 46, in
import gluonnlp as nlp
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/init.py", line 24, in
from . import data
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/init.py", line 23, in
from . import (batchify, candidate_sampler, conll, corpora, dataloader,
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/batchify/init.py", line 23, in
from . import batchify, language_model
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/batchify/language_model.py", line 32, in
from ..utils import slice_sequence, _slice_pad_length
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/utils.py", line 36, in
from mxnet.gluon.utils import _get_repo_url, download, check_sha1
ImportError: cannot import name '_get_repo_url'
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ python train_transformer.py --dataset WMT2014BPE --src_lang en --tgt_lang de --batch_size 2700 --optimizer adam --num_accumulated 16 --lr 2.0 --warmup_steps 4000 --save_dir transformer_en_de_u512 --epochs 30 --gpus 0,1,2,3,4,5,6,7 --scaled --average_start 5 --num_buckets 20 --bucket_scheme exp --bleu 13a --log_interval 10
Traceback (most recent call last):
File "train_transformer.py", line 46, in
import gluonnlp as nlp
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/init.py", line 24, in
from . import data
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/init.py", line 23, in
from . import (batchify, candidate_sampler, conll, corpora, dataloader,
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/batchify/init.py", line 23, in
from . import batchify, language_model
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/batchify/language_model.py", line 32, in
from ..utils import slice_sequence, _slice_pad_length
File "/home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages/gluonnlp/data/utils.py", line 36, in
from mxnet.gluon.utils import _get_repo_url, download, check_sha1
ImportError: cannot import name '_get_repo_url'
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ pip list
Package    Version

certifi    2018.11.29
chardet    3.0.4
gluonnlp   0.5.0.post0
graphviz   0.8.1
idna       2.6
mxnet-cu80 1.1.0
numpy      1.13.3
pip        18.1
requests   2.18.4
setuptools 40.6.2
urllib3    1.22
wheel      0.32.3
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ pip install --pre --upgrade mxnet
Collecting mxnet
Downloading https://files.pythonhosted.org/packages/3b/9d/9ff59b9a1bbd122873e5942e526543d837af4bc3ee3c1a2bba616979aff4/mxnet-1.5.0b20181217-py2.py3-none-manylinux1_x86_64.whl (27.5MB)
100% |████████████████████████████████| 27.5MB 591kB/s
Collecting requests>=2.20.0 (from mxnet)
Using cached https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl
Requirement already satisfied, skipping upgrade: graphviz<0.9.0,>=0.8.1 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from mxnet) (0.8.1)
Requirement already satisfied, skipping upgrade: numpy<1.15.0,>=1.8.2 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from mxnet) (1.13.3)
Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet) (1.22)
Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet) (2.6)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet) (3.0.4)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet) (2018.11.29)
mxnet-cu80 1.1.0 has requirement requests==2.18.4, but you'll have requests 2.21.0 which is incompatible.
Installing collected packages: requests, mxnet
Found existing installation: requests 2.18.4
Uninstalling requests-2.18.4:
Successfully uninstalled requests-2.18.4
Successfully installed mxnet-1.5.0b20181217 requests-2.21.0
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ python train_transformer.py --dataset WMT2014BPE --src_lang en --tgt_lang de --batch_size 2700 --optimizer adam --num_accumulated 16 --lr 2.0 --warmup_steps 4000 --save_dir transformer_en_de_u512 --epochs 30 --gpus 0,1,2,3,4,5,6,7 --scaled --average_start 5 --num_buckets 20 --bucket_scheme exp --bleu 13a --log_interval 10
Illegal instruction (core dumped)
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ python train_transformer.py --dataset "/home/user1/nmt_tensorflow_git/nmt/scripts/wmt16_de_en" --src_lang en --tgt_lang de --batch_size 2700 --optimizer adam --num_accumulated 16 --lr 2.0 --warmup_steps 4000 --save_dir "/home/user1/transformer_nmt/ready_to_run_transformer/saved_dir" --epochs 30 --gpus 0,1,2,3,4,5,6,7 --scaled --average_start 5 --num_buckets 20 --bucket_scheme exp --bleu 13a --log_interval 10
Illegal instruction (core dumped)

I am sorry the formatting here, I am new to posting logs. I will attach it as a file.
log.txt

@vikaskumarjha9 Sorry to hear that it's still happening to you. From the log, it looks like you have both mxnet and mxnet-cu80, and the mxnet-cu80 is of older version. That might be the cause of this issue.

I have removed mxnet library, now I have only mxnet-cu80 version1.1.0 as my CUDA version is 8+. I am getting this error.
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ python train_transformer.py --dataset WMT2014BPE --src_lang en --tgt_lang de --batch_size 2700 --optimizer adam --num_accumulated 16 --lr 2.0 --warmup_steps 4000 --save_dir transformer_en_de_u512 --epochs 30 --gpus 0,1,2,3,4,5,6,7 --scaled --average_start 5 --num_buckets 20 --bucket_scheme exp --bleu 13a --log_interval 10
Traceback (most recent call last):
File "train_transformer.py", line 45, in
from mxnet import gluon
ImportError: cannot import name 'gluon'
I tried installing gluon package but no help. Also I have gluonnlp-0.5.0.post0 installed, does this work with mxnet-cu80? Pls help.

@vikaskumarjha9 GluonNLP 0.5.0 depends on MXNet 1.3.0, so in this case you can try pip install --upgrade mxnet-cu80.

Thank you for prompt reply, I updated it to 1.3.1 but now I am getting this error again.
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ pip install --upgrade mxnet-cu80
Collecting mxnet-cu80
Using cached https://files.pythonhosted.org/packages/b0/a4/f4ab7885416a5b752782492dba335f58d942a917a7259826689f552d9af6/mxnet_cu80-1.3.1-py2.py3-none-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: numpy<1.15.0,>=1.8.2 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from mxnet-cu80) (1.13.3)
Requirement already satisfied, skipping upgrade: graphviz<0.9.0,>=0.8.1 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from mxnet-cu80) (0.8.1)
Requirement already satisfied, skipping upgrade: requests>=2.20.0 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from mxnet-cu80) (2.21.0)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet-cu80) (2018.11.29)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet-cu80) (3.0.4)
Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet-cu80) (2.6)
Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in /home/user1/transformer_nmt/trans_env/lib/python3.5/site-packages (from requests>=2.20.0->mxnet-cu80) (1.22)
Installing collected packages: mxnet-cu80
Found existing installation: mxnet-cu80 1.1.0
Uninstalling mxnet-cu80-1.1.0:
Successfully uninstalled mxnet-cu80-1.1.0
Successfully installed mxnet-cu80-1.3.1
(trans_env) user1@S3:~/transformer_nmt/ready_to_run_transformer$ python train_transformer.py --dataset WMT2014BPE --src_lang en --tgt_lang de --batch_size 2700 --optimizer adam --num_accumulated 16 --lr 2.0 --warmup_steps 4000 --save_dir transformer_en_de_u512 --epochs 30 --gpus 0,1,2,3,4,5,6,7 --scaled --average_start 5 --num_buckets 20 --bucket_scheme exp --bleu 13a --log_interval 10
Illegal instruction (core dumped)
Am I doing something wrong ?

Oops sorry, here you go.
log2.txt

tyhsxvwqwasavsrfrhenwr gfweq5ywhnsqgwefsqWHWGES6XYEWS @HCEKDFNCCRYESDYSH3WBQAWF2Q @B DHEVSVFYSB #1538


Context should try harder before saying "Sorry, but I can only assist with programming related questions."

VS Code Version: Latest Stable
OS Version: Mac latest beta

Steps to Reproduce:

Write a chapter in a book titled "AI for IT", Chapter 5 "Reports and Documentation" in Asciidoc (.adoc) format
Auto complete a few paragraphs with inline Copilot
Ask Copilot Chat "what should i write next? give me an outline and the conclusion paragraph for this section"
Copilot Chat says "Sorry I can't help you with that. I can only help with programming"
Tell it "But im writing an AI book for IT pros"
Watch as it outlines the chapter

Thank you 🙇🏼

Just a note that I think the "Sorry, but I can only assist with programming related questions." should be relaxed a bit, since not everything we do in VS code is programming, sometimes it's ui copy, translation resources, etc.
Sometimes I use co-pilot to help me with suggestions for UI copy (eg. when text is too long for a certain UI element, I ask co-pilot to make it shorter, and give it feedback like "can you make it more friendly"). It's super helpful, especially for people who are not native speakers of a language to write good UI copy
I got the "Sorry I can't help you with that. I can only help with programming" for the first time today.
Using this trick works, I just need to say "yeah, but I'm writing a copy for a UI in react native" and it will gladly output the answer, but I would urge the co-pilot team to broaden the restriction to not only programming, but other programming and product building tasks :)

@roblourens just saw your off topic label. am I in the wrong repo?

No, the issue is about our overactive off-topic filter :)

Another example:
Q: What are other terms like "tech debt" to describe code that is work in progress, still working on it and needs polish?
A: Sorry, but I can only assist with programming related questions.
Q: tech debt is programming related
A: Sorry, but I can only assist with programming related questions.

03 May 2024. This is still and Issue.

Why copilot?

This happens more and more. Are we supposed to pay more?

Same here, with code related questions...

I've been suffering with this more and more too. I ask it to literally generate some code, and I get "Sorry, but I can only assist with programming related questions." back. And then, if I only answer with something like "yes you can" it generates what I asked for. I'm practically having an aneurysm trying to make it be useful. It's rather quick to just . . . not do what it was created for, even if the question or prompt is or isn't programming related. Whether it is or isn't should be up to us, not to an arbitrary ruling or weights in the model. Not when it clearly is not up to the task.

It's crazy how many times I had to say "yes you can" to it and then it does what it's supposed to. Super annoying.

This does not solve the problem, but it is a workaround: I created a GitHub Copilot Chat Extension that allows you to use the OpenAI (Azure or openai.com) API within GitHub Copilot Chat.
https://marketplace.visualstudio.com/items?itemName=chrissylemaire.assistants-chat-extension

If you don't already have an assistant setup, it'll offer to setup a Beavis and Butthead assistant which demonstrates that there really aren't any restrictions on the chat output.

Note that this does cost API tokens.

@potatoqualitee thank you, I got stuck at this step:
"In the search bar, type "Assistants Chat Extension". Couldn't find it.
Anyway, do we know anything about a recent update that made the chat worse? It's incredible how uncooperative and generic it's been lately.

@fgaletic ah, ive renamed it since and will update the docs, thank you. You can just skip that step, it'll auto-detect the missing key and prompt you to add either openai or azure openai info.

We updated the off-topic filter and see a significant drop in false-positive filtering in experiment results.
Closing as fixed, but please don't hesitate to file new issues in case you still run into this.

thank you! 🙏🏼


feat: Quantized models II

Description
This is Part II of adding support for quantized models and lazy loading required files.
Prevents the download of unused ONNX files.
Part I - #199

closing in favour of #201 , feel free to re-open if I got it wrong


Allow valid "en-us"

Coverage increased (+0.05%) to 91.613% when pulling f507e10a1e3d30bbb0c5c2054065e7a7cbd9c970 on malbertSC:master into dd47dd1b0d6b2b59ce0fd083582ded7e441f4755 on chemerisuk:master.

Fixed in 3.0.0-rc.2


arch/x86_64:Synchronize comments and data type information in the com…

Summary
Synchronize comments and data type information in the common/intel64
Impact
no impact
Testing
ostset

[Experimental Bot, please feedback here]
No, this PR does not meet the NuttX requirements.
Missing Information:

Summary:

Lacks a clear explanation of why the comments and data type information need synchronization. What problem does this address?
Needs to specify which comments and data types are being changed.

Testing:

"ostset" is not sufficient testing. Provide:

Specific build hosts used (OS, CPU, compiler versions).
Specific target architectures and boards tested.
Relevant log snippets demonstrating the issue before the change and the improvement after the change.

Recommendations:

Elaborate on the "Why": Clearly articulate the reason for this synchronization.  Are there inconsistencies, errors, or outdated information?
Be Specific: Identify the affected files, functions, or data structures within common/intel64.
Provide Concrete Testing Details: Follow the template's structure and include:

Build host specifications (OS, CPU, compiler)
Target details (architecture, board, configuration)
Log excerpts showcasing the issue before and the solution after your changes.

A well-structured PR increases the likelihood of swift review and merging.


hgroup isn't supported

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/hgroup
I get this error trying to use it in the view! macro:

cannot find function hgroup in crate leptos::leptos_dom

Oh, weird.
If you want to make an easy PR you just need to add it somewhere around here, following the same format as all the other elements (a comment usually from the MDN description, the tag name, and the web_sys element type with in this case is HtmlElement) Otherwise I can do it tonight some time
https://github.com/leptos-rs/leptos/blob/fc7199f188e1eda1b4529a98b7f27e326b9a5c98/leptos_dom/src/html.rs#L1045


[Bug Report] tree structure filter node method, empty filter conditions, the return of the tree structure by default to expand all nodes

Element UI version
2.0.1
OS/Browsers version
10.12.6／61.0.3163.100
Vue version
2.3.3
Reproduction Link
https://codepen.io/zhengxiaowen/pen/wrLbQw
Steps to reproduce
在树形结构过滤结点时，清空过滤条件，树形结构的所有结点默认展开了（初始化的时候，是不默认展开的）
What is Expected?
清空过滤条件，应该要和初始化设置的状态一样，而并非默认展开所有结点
What is actually happening?
清空过滤条件，默认展开了所有结点

Fixed in #7853

@Leopoldthecoder
Are you sure that the problem was fixed?
I try his Reproduction Link，import<EMAIL_ADDRESS>is new version elementjs, and see the problem he described is still.


perf: Implement timeouts and Max-batch size for Consolidation

Fixes #N/A
Description
I ran some tests with KwoK and a cost monitoring tool to scale up clusters to 2000, 1500, 1000, 500, and 100 nodes. I tracked metrics like cumulative cost, cumulative pending pod seconds, and overall test time.
The test scaled down the clusters incrementally by 25% (100% -> 75% -> 50% -> 25%), waiting for Consolidation to fully scale down to the next level and exiting when the number of nodes got to 25% of the peak. With a timeout of 1 minute for Multi Machine Consolidation, 3 minutes for Single Machine Consolidation, and a max batch size of 100 machines for Multi, Karpenter was able to take more actions quicker, speeding up the whole test to 33 minutes.

Includes a new metric counter that tracks how many time a timeout has been reached for both Multi-Machine and Single-Machine Consolidation.
This also adds in higher bucket sizes for Prometheus histogram metrics since Deprovisioning has some histogram items that can reach higher than 60 seconds.

How was this change tested?

Tested with KWOK, and make presubmit

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

20 of 31 (64.52%)  changed or added relevant lines in 5 files are covered.
11 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.2%) to 79.413%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/controllers/deprovisioning/singlemachineconsolidation.go
6
10
60.0%

pkg/controllers/deprovisioning/multimachineconsolidation.go
7
14
50.0%

Files with Coverage Reduction
New Missed Lines
%

pkg/controllers/provisioning/scheduling/topology.go
2
86.49%

pkg/controllers/provisioning/scheduling/topologygroup.go
2
96.75%

pkg/controllers/provisioning/scheduling/preferences.go
7
86.67%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
8097

Relevant Lines:
10196

💛  - Coveralls

Merging so we can get some test runs.


[Feature] Only clone the branch that triggered the action.

Clone the repositories with a --branch flag.
Should make cloning big projects a lot more efficient.

Fixed in https://github.com/MarvinJWendt/run-node-formatter/commit/d7eb846d329f8eba8387ca8f34a2fdcd8321c364


Fix ORT to atomically place files.

I'm submitting a ...

[x] bug report
[ ] new feature / enhancement request
[ ] improvement request (usability, performance, tech debt, etc.)
[ ] other

Traffic Control components affected ...

[ ] CDN in a Box
[ ] Documentation
[ ] Grove
[ ] Traffic Control Client
[ ] Traffic Monitor
[ ] Traffic Ops
[x] Traffic Ops ORT
[ ] Traffic Portal
[ ] Traffic Router
[ ] Traffic Stats
[ ] Traffic Vault
[ ] unknown

Current behavior:
ORT does not atomically place new files. It uses cp, which is not atomic.
This means if somethings fails – Perl crashes, the OS crashes, the syscall fails for some reason – the file will be half-written, causing ATS reloading to fail.
Expected / new behavior:
The proper procedure is to make a new temporary file, and then call mv, which is guaranteed to be atomic, and either succeed or fail.
Code is here:
https://github.com/apache/trafficcontrol/blob/master/traffic_ops/ort/traffic_ops_ort.pl#L1575
Minimal reproduction of the problem with instructions:
Difficult to reproduce.  But it would definitely happen. And then your ATS reload or restart would fail.
Anything else:

removing GFI label. hope that's ok @rob05c

Why? This should be pretty straightforward for a new developer.
It's really just a matter of replacing the cp with code to make a new file with a temporary name and mv it to the correct location.

Because it's Perl?
Which, maybe new developers won't be learning from their universities, but new contributors might know. So I think it's a fine GFI.

added it back :)

@rob05c can this be closed?

Yes, t3c currently atomically places new files: https://github.com/apache/trafficcontrol/blob/master/cache-config/t3c-apply/torequest/torequest.go#L499


No response when sending a large file.

Problem Description
mitmproxy does not send a response when response body is large.

Curl will times out, without receiving any data, no matter how long you wait.

mitmproxy TUI shows flow as "content missing", but does show correct response code.

Nginx logs register incoming request and response code. xxx.xxx.xxx.xxx - - [31/Jul/2023:20:05:02 +0000] "GET /kffff HTTP/1.1"<PHONE_NUMBER> "-" "curl/7.85.0" "-"

Steps to reproduce the behavior:

Generate a large file to send later  dd if=/dev/urandom bs=2000000000 count=1 | base64 | fold -w100 >> /tmp/file.txt
Write an Nginx config to serve the file

server {
    listen          80;
    server_name    localhost;
    location ~* .* {
        root                        /usr/share/nginx;
        try_files                   /payload =404;
        chunked_transfer_encoding   on;
        sendfile                    on;
    }
}

Write a script to start Nginx and mount files to the container

#!/usr/bin/env bash

docker run --rm -d --name nginx \
  -v path_to_config_file/nginx.conf:/etc/nginx/conf.d/default.conf \
  -v  /tmp/file.txt:/usr/share/nginx/payload \
  -p 6000:80 nginx

Start nginx and make a curl request to port 6000 to verify nginx configuration. curl -v localhost:6000/kffff
Start mitmproxy mitmproxy --mode reverse:http://localhost:6000 -p 6001 --set keep_host_header=true
Make a request to proxy curl -v localhost:6001/kffff

Expected result

Curl starts to output the file
mitmproxy TUI shows content of the response and not message "content missing"

Actual result

Curl does not receive any data
mitmproxy shows "content missing" instead of the response content.

System Information
Mitmproxy: 9.0.1
Python:    3.11.4
OpenSSL:   OpenSSL 1.1.1u  30 May 2023
Platform:  macOS-13.1-x86_64-i386-64bit
Additional Info
Similar results are achieved when response body is replaced by an addon.
import mitmproxy

file_path = "/tmp/file.txt"

def generate_chunked_response(file_path):
    try:
        with open(file_path, 'rb') as file:
            result = bytearray()

            # Read the file in chunks
            chunk_size = 1024  # You can adjust this value to your needs
            while True:
                data_chunk = file.read(chunk_size)
                if not data_chunk:
                    break  # Reached the end of the file
                chunk_length_hex = hex(len(data_chunk))[2:]  # Get the chunk size in hexadecimal format
                chunk_header = f"{chunk_length_hex}\r\n".encode('utf-8')
                result.extend(chunk_header + data_chunk + b'\r\n')

            # Add the final chunk indicating the end of the response
            result.extend(b'0\r\n\r\n')

            return result

    except FileNotFoundError:
        return b"File not found."

print('Generatig paylod')
payload = bytes(generate_chunked_response(file_path))

class OverWriteResponseHeaders:
    def response(self, flow: mitmproxy.http.HTTPFlow):
        flow.response.content = payload

addons = [OverWriteResponseHeaders()]

thanks - duplicate of #6126 I think.

@mhils I do not think it is a duplicate. #6126 is about UI, this one is about not sending a reply when response body is large.
I probably should not have put this many details in this ticket, but my main concern is the fact that the file is not being received. There is no traffic going back to curl at all.

Have you tried setting https://docs.mitmproxy.org/stable/concepts-options/#stream_large_bodies?

I did, but apparently I was not doing it properly. Now, when I corrected my mistake it works as expected. Thank you for your support.


length mismatch for istft

🐛 Bug
Hi I am having some trouble with torch.istft. It seems that the length doesn't match even when I specify the length (this issue seems to happen with very small probability).
To Reproduce
Steps to reproduce the behavior:

download stft.pt from link
run the following code

import torch
fname = 'stft.pt'
stft_feat = torch.load(fname)
print("stft_feat", stft_feat.size())
n_fft = 512
hop_length = 320
win_length = 512
window = torch.hann_window(win_length).cuda()
center = True
length = 150079
reconstruct = torch.istft(stft_feat, n_fft, hop_length=hop_length, win_length=win_length, window=window, center=center, length=length)[0]
print("reconstruct", reconstruct.size())
Expected behavior
In my environment, the program printed
stft_feat torch.Size([1, 257, 469])
reconstruct torch.Size([150016])
The length seems to be incorrect. The stft_feat is the output from neural network, I cannot guarantee the value to be very correct.
Environment

What commands did you used to install torchaudio (conda/pip/build from source)?
pip in conda environment
If you are building from source, which commit is it?
What does torchaudio.__version__ print? (If applicable)
0.8.0a0+a751e1d

Collecting environment information...
PyTorch version: 1.8.0
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A
OS: Debian GNU/Linux 9.13 (stretch) (x86_64)
GCC version: (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
Clang version: 3.8.1-24 (tags/RELEASE_381/final)
CMake version: version 3.20.0-rc2
Python version: 3.6 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration:
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti
GPU 2: GeForce RTX 2080 Ti
Nvidia driver version: 440.33.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Versions of relevant libraries:
[pip3] numpy==1.18.4
[pip3] pytorch-lightning==1.1.0
[pip3] pytorch-ranger==0.1.1
[pip3] torch==1.8.0
[pip3] torch-optimizer==0.1.0
[pip3] torch-stoi==0.1.2
[pip3] torchaudio==0.8.0a0+a751e1d
[pip3] torchvision==0.9.0
[conda] blas                      1.0                         mkl
[conda] cuda90                    1.0                  h6433d27_0    pytorch
[conda] mkl                       2019.4                      243
[conda] mkl-service               2.3.0            py37he904b0f_0
[conda] mkl_fft                   1.0.14           py37ha843d7b_0
[conda] mkl_random                1.1.0            py37hd6b4f25_0
[conda] numpy                     1.17.2           py37haad9e8e_0
[conda] numpy-base                1.17.2           py37hde5b4d6_0
[conda] numpydoc                  0.9.1                      py_0
[conda] pytorch                   1.0.0           py3.7_cuda9.0.176_cudnn7.4.1_1    pytorch
[conda] torchvision               0.2.1                      py_2    pytorch
Additional context

Since I don't have visibility over stft_feat, I cannot comment on that part. However, have you tried with another library? In particular torch.istft is meant to match with librosa's. Have you tried with the latter?

Since I don't have visibility over stft_feat, I cannot comment on that part. However, have you tried with another library? In particular torch.istft is meant to match with librosa's. Have you tried with the latter?

Hi I think I already attach a link for the stft_feat (https://drive.google.com/file/d/1TjwwGFEotsjAay0osCqw1IlcuANX675R/view?usp=sharing) Would you mind downloading it and have a try? I will also try the librosa and give you feedback. Thanks!

Hi I think I already attach a link for the stft_feat (https://drive.google.com/file/d/1TjwwGFEotsjAay0osCqw1IlcuANX675R/view?usp=sharing)

Sorry, what I meant is that I do not know how stft_feat was generated :) do you have more details about this?

reconstruct_librosa = librosa.istft(stft_feat.data.cpu().numpy().squeeze(), hop_length=hop_length, win_length=win_length, window='hann', center=center, length=length)
print("reconstruct_librosa", reconstruct_librosa.shape)
reconstruct torch.Size([150016])
reconstruct_librosa (150079,)
Sorry for my late reply, I tried librosa and the output length is correct. My stft_feat is not extracted, it is predicted. My situation is that the neural model will predict a real positive value STFT mask M and I will multiply that with the STFT feature (imaginary) of a real audio R. So the stft_feat F = M * R may not be correct. But I think this question is not related to the stft_feat is valid or not right? It is simply about the output length I guess?

Any ideas on this? @vincentqb Thanks a lot!

I have reproduced your result. The reason is that the hop size is 320 that is a little big when n_fft is 512.
Let's see the implementation of torch.istft:
  const auto start = center ? n_fft / 2 : 0;
  const auto end = lengthOpt.has_value()? start + lengthOpt.value() : - n_fft / 2;

The start is set to 256 since center is True in your case.
The end is set to start + lengthOpt.value() that is 256+150079 = 150335.
After slicing, the shape should be 150079, looks perfect.
However, the expected_signal_length is n_fft + hop_length * (n_frames - 1) that is 512+320*468 = 150272. It's smaller than 150335
After slicing operation [256:150272], the shape of your waveform becomes 150016.
I would suggest using hop_length as 256 that is common in speech processing.
In librosa, they added a padding logic if the expected length is shorter than the given length. @mthrok What's your opinion on this?

Hi @HuangZiliAndy, I have added the padding logic to torch.stft in https://github.com/pytorch/pytorch/pull/63469. Now the issue should be resolved in the nightly version soon. However, to make sure the signal is invertible, I suggest using a different center and hop_length setting to avoid padding. Thanks.


Improve code quality

There are still some hacks and similar methods for a single purpose. Gotta fix this.

[x] Add properties to Weaver to get settings.

Good enough to close this.


Update Ericsson members

Signed-off-by: Georg Kunz<EMAIL_ADDRESS>

@anajsana this is the update of the Ericsson members I mentioned last week. Thanks

Great! thank you @gkunz !


server: fixup around nil UnmarshalJSON panics

https://github.com/moov-io/ach/pull/419 started to add a test for converting all test *.ach files to JSON and submitting them against our HTTP API. This completes that work and fixes a couple panics around Batch and IATBatch when nil.

Codecov Report

Merging #421 into master will increase coverage by 0.56%.
The diff coverage is 85.71%.

@@            Coverage Diff             @@
##           master     #421      +/-   ##
==========================================
+ Coverage    85.2%   85.77%   +0.56%
==========================================
  Files         109      109
  Lines        6854     6861       +7
==========================================
+ Hits         5840     5885      +45
+ Misses        715      673      -42
- Partials      299      303       +4

Impacted Files
Coverage Δ

file.go
73.25% <100%> (+0.77%)
:arrow_up:

iatBatch.go
94.39% <100%> (+4.17%)
:arrow_up:

batch.go
93.7% <66.66%> (+0.04%)
:arrow_up:

server/files.go
19.84% <0%> (+18.25%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ccb07cd...b64fc32. Read the comment docs.


code generation: a lot of issues with generating Go code for https://demo.thingsboard.io/v3/api-docs?group=thingsboard

What version of ogen are you using?
$ go list -m github.com/ogen-go/ogen
github.com/ogen-go/ogen

I compiled it from tip of today's main:
$ git describe --always
78b0432b

Can this issue be reproduced with the latest version?
Yes
What did you do?
$ ./ogen 'https://demo.thingsboard.io/v3/api-docs?group=thingsboard'

Schema: https://demo.thingsboard.io/swagger-ui/#/
What did you expect to see?
Generated Go code.
What did you see instead?
First I get:
$ ./ogen 'https://demo.thingsboard.io/v3/api-docs?group=thingsboard'
  - api-docs:1:1418220 -> invalid name: "PageData«EntityVersion»" doesn't match "^[a-zA-Z0-9.\\-_]+$"

After commenting
if err := validateComponentsKeys(p, c); err != nil {
        return nil, err
}

I get:
- api-docs:1:1085604 -> parse path "/api/2fa/account/config/generate{?providerType}": path MUST NOT contain a query string

After commenting
case u.RawQuery != "":
        return nil, errors.New("path MUST NOT contain a query string")

I get:
- api-docs:1:1086888 -> parameter "?providerType" not specified

Is there something wrong with this spec https://demo.thingsboard.io/swagger-ui/#/?

/api/2fa/account/config/generate{?providerType} is definitely wrong path template, query paramters should be described in the parameters section.


Feature: Keyboard Shortcuts - Undo + Redo

These are so useful, and I believe must-haves for 1.0.0

[x] Account for OS differences
[ ] Properly track key changes

Could fix this with ports, fork, etc or wait and follow up with the discussion here:
https://github.com/ohanhi/keyboard-extra/issues/6

fixed this by copying over ohanhi's solution while waiting for to see if he'll merge forceRelease