MIOT device became unavailable

Hi,
Since a week, most of my MIOT devices (like magnetic sensors - lumi.sensor_magnet.v2, isa.magnet.dw2hl, motion sensors - yeelink.light.nl1, lumi.sensor_motion.v2) became very erratic. The status turns to "unavailable" about every minute, or multiple times within a few minutes - then turns to "normal" (open/closed/no motion) a few seconds later.
Tries to get some info from the logs, and I can see a huge number of messages like this:
2024-06-17 15:53:27.270 WARNING (SyncWorker_6) [custom_components.xiaomi_miot.core.xiaomi_cloud] Request xiaomi api: miotspec/prop/get {'params': [{'did': 'lumi.158d0003e80af4', 'siid': 2, 'piid': 1}]} failed, response: None
2024-06-17 15:53:55.990 WARNING (MainThread) [homeassistant.helpers.entity] Update of binary_sensor.lumi_v2_0af4_motion_sensor is taking over 10 seconds
2024-06-17 15:53:56.098 WARNING (SyncWorker_28) [custom_components.xiaomi_miot.core.xiaomi_cloud] Request xiaomi api: miotspec/prop/get {'params': [{'did': 'lumi.158d0003e80af4', 'siid': 2, 'piid': 1}]} failed, response: None
2024-06-17 15:54:25.994 WARNING (MainThread) [homeassistant.helpers.entity] Update of binary_sensor.lumi_v2_09da_motion_sensor is taking over 10 seconds
2024-06-17 15:54:26.301 WARNING (SyncWorker_29) [custom_components.xiaomi_miot.core.xiaomi_cloud] Request xiaomi api: user/get_user_device_data {'did': 'lumi.158d0003e809da', 'key': 'device_log', 'type': 'prop', 'time_start':<PHONE_NUMBER>, 'time_end':<PHONE_NUMBER>, 'limit': 1} failed, response: None
2024-06-17 15:54:39.243 WARNING (SyncWorker_18) [custom_components.xiaomi_miot.core.xiaomi_cloud] Request xiaomi api: v2/message/v2/typelist {} failed, response: None
2024-06-17 15:54:55.432 WARNING (MainThread) [homeassistant.helpers.entity] Update of light.mijia_group2_3216_light is taking over 10 seconds

The log is full with message like above, multiple thousand times in a day.
Reloading or reconfiguring the integration does not help either.

Just adding an info: like a magic, since 26 of august the issues with the sensors disappeared.
I don't k now what happened, but probably we can close this issue.


Is masking in backward RNNs implemented correctly?

I am not really sure, but am wondering if compute_mask() in Recurrent class (recurrent.py) is missing a call to reverse the mask? In the backend K.rnn() reverses the input and mask, before processing it, but the reversed mask is not returned anywhere.
    def compute_mask(self, inputs, mask):
        if self.return_sequences:
            if self.go_backwards:
                return K.reverse(mask, 1)
            else:
                return mask
        else:
            return None

Function compute_mask() in Recurrent class (recurrent.py#L228) basically still just returns the input mask, although the output itself has been reversed (effect of go_backwards=True).
I still think that a if self.go_backwards: mask = K.reverse(mask, 1) is missing.


Update istio control plane and minimize the number of service restarts

(This is used to request new product features, please visit https://discuss.istio.io for questions on using Istio)
Describe the feature request
We are using istio version 1.6.6 in our production environment for a long time.We want to update it to version 1.12.8 by canary.But we have a service has been injected with many devices connected.When we upgrade a version we have to restart all the services for sidecar to update.At this point the concurrent pressure from device reconnections can overwhelm our servers.Update one by one slowly waiting for the device to connect will bring a long downtime.
According to the documentation I need to restart all services at least three times.Can there be some kind of update that only reboots once?
Describe alternatives you've considered
I have tried to update directly to version 1.12.8.But I got these errors:

2022-07-19T01:22:29.994098Z     error   installer       failed to update resource with server-side apply for obj EnvoyFilter/istio-systemats-filter-1.10-1-12-8: admission webhook "validation.istio.io" denied the request: configuration is invalid: 3 errors occurred:
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager

2022-07-19T01:22:30.042608Z     error   installer       failed to update resource with server-side apply for obj EnvoyFilter/istio-systemats-filter-1.11-1-12-8: admission webhook "validation.istio.io" denied the request: configuration is invalid: 3 errors occurred:
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager

2022-07-19T01:22:30.070783Z     error   installer       failed to update resource with server-side apply for obj EnvoyFilter/istio-systemats-filter-1.12-1-12-8: admission webhook "validation.istio.io" denied the request: configuration is invalid: 3 errors occurred:
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager

✘ Istiod encountered an error: failed to update resource with server-side apply for obj EnvoyFilter/istio-systemats-filter-1.10-1-12-8: admission webhook "validation.istio.io" denied the request: configuration is invalid: 3 errors occurred:
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager

failed to update resource with server-side apply for obj EnvoyFilter/istio-systemats-filter-1.11-1-12-8: admission webhook "validation.istio.io" denied the request: configuration is invalid: 3 errors occurred:
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager

failed to update resource with server-side apply for obj EnvoyFilter/istio-systemats-filter-1.12-1-12-8: admission webhook "validation.istio.io" denied the request: configuration is invalid: 3 errors occurred:
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager
* Envoy filter: subfilter match requires filter match with envoy.http_connection_manager

I tried to update to 1.8.6 first and uninstall the control plane of 1.6.6.Then I was able to update to version 1.12.8 and no errors were reported(Not tried in production environment).But this still requires two reboots and may have unknown errors.
So is there a less frequent and safe way to update istio?
Affected product area (please put an X in all that apply)
[X] Docs
[X] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[X] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Affected features (please put an X in all that apply)
[ ] Multi Cluster
[ ] Virtual Machine
[X] Multi Control Plane

maybe you can disabled the webhook during upgrading?

How to disable webhook, will it affect the sidecar of existing services?

maybe you can disabled the webhook during upgrading?

How to disable webhook, will it affect the sidecar of existing services?

find out by running kubectl get MutatingWebhookConfiguration

maybe you can disabled the webhook during upgrading?

How to disable webhook, will it affect the sidecar of existing services?

find out by running kubectl get MutatingWebhookConfiguration

I got these:
agent-injector-webhook-ambassador
cert-manager-webhook
istio-sidecar-injector
pod-identity-webhook
rancher-monitoring-admission
vpc-resource-mutating-webhook
Should I delete one of them? Or should I change the configuration of one of them?

istio-sidecar-injector is the one istio used.

kubectl get MutatingWebhookConfiguratio

Still the same error.
We've decided to update the version in batches. Anyway thanks a lot!


Update to angular RC6  ?

any plans to update angular2-moment to the current Release Candidate?

Hi, is there anything specific not working with RC6?

Hi, is there anything specific not working with RC6?

I updated to RC6 today and now I get this error message:

main.ts:13Error: Template parse errors:
'time' is not a known element:

If 'time' is an Angular component, then verify that it is part of this module.
If 'time' is a Web Component then add "CUSTOM_ELEMENTS_SCHEMA" to the<EMAIL_ADDRESS>of this component to suppress this message.

While digging around I found this comment that says "starting with RC6 Angular2 enforces that tags name are valid." See the comment for more details.
I'm an Angular 2 novice, so I don't have a solution to the problem yet.

Hi, just replace the <time> tag with a <span>, and it should fix the issue:
<span>{{myDate | amTimeAgo:true}}</span>

If you have a few moments to send a pull request for the README with this fix, that would be great!

Thanks, @urish.
I confirmed that replacing the <time> tag withspan works. In fact, span isn't even needed.
I'll send you a PR for the README shortly.
Thanks again.


feat(conditions) Show up the volume, node, disk conditions

https://github.com/longhorn/longhorn/issues/1359
image: wangsiye/longorn-ui:3f5515c
Signed-off-by: Siye Wang<EMAIL_ADDRESS>

Here are the current conditions we're using for:
Node

Ready: Green for true, red for false.
Mount Propagation: Green for true, red for false.
Schedulable: Green for true, red for false.

Disk

Ready: Green for true, red for false.
Scheduable: Green for true, red for false.

Volume

Scheduled: Green for true, red for false.
Restore: Green for true, grey for false if Reason is empty (no restore needed), red for false if the Reason is not empty.

LGTM.


Improve upgrade.md by adding detailed clean up steps

Changes
Fixes #1492
Checklist

[x] An issue to be fixed by this PR is listed above.
[ ] New tests are added to ensure new features are working. Please refer to this page to see how to add a test.
[ ] ./frb_internal precommit --mode slow (or fast) is run (it internal runs code generator, does auto formatting, etc).
[ ] If this PR adds/changes features, documentations (in the ./website folder) are updated.
[ ] CI is passing. Please refer to this page to see how to solve a failed CI.

Good job! I will check a bit more in detail later.


Client Cipher update for OSSM-1667

Updates to align Maistra OpenSSL with Upstream.

I thought we were going to add a feature flag to re-enable the old ciphers?

/retest

/retest


Make Where Do I Belong function name unique

Tested locally. Passed npm run test-challenges. References #7085.

@erictleung Thanks!


[NSwagStudio - TypeScript/Angular2 & C#] Generated Code has Generic Type error

Hi, there is an issue with the generated code which contains generic type. Below is the screenshot for generated TypeScript/Angular2 code:

Note that generated C# code also has the same error with generic type.

Generics are currently not supported. But we should transform this into an allowed name. For now, see https://github.com/NSwag/NSwag/issues/83

Fixed in https://github.com/NJsonSchema/NJsonSchema/commit/79c36913d8c0955178fd2c8e4f5d9f6b537e29ed

NSwag v5 has been released, please check if your problem is solved and reopen the issue if it still remains...

Thank you.


[TF-TRT] Lazy-load trt_convert

The goal of this PR is to avoid running any TF-TRT code when the user doesn't need TF-TRT. In particular, this has the effect that TF-TRT doesn't try to load libnvinfer.so when TF is imported.

Hi @bixia1 Can you please review this PR ? Thank you!

Would you please add a PR description? Is there any way to demonstrate the effect of this PR?

Here are the internal errors, @Nyrio can you please verify ? Thank you!
Traceback (most recent call last):
File "/tensorflow/tools/api/tests/api_compatibility_test.py", line 443, in testAPIBackwardsCompatibility
self._checkBackwardsCompatibility(
File "/tensorflow/tools/api/tests/api_compatibility_test.py", line 420, in _checkBackwardsCompatibility
self._AssertProtoDictEquals(
File "/tensorflow/tools/api/tests/api_compatibility_test.py", line 327, in _AssertProtoDictEquals
self.fail('%d differences found between API and golden.' % diff_count)
AssertionError: 4 differences found between API and golden.

@Nyrio Any update on this PR? Please. Thank you!

@Nyrio Any update on this PR? Please. Thank you!

@Nyrio Any update on this PR? Please. Thank you!

@DEKHTIARJonathan Can I close this one in favor of #58346

@Nyrio yes please


Add dataset-finder link

What
https://trello.com/c/ZEJRw8Qy/1181-update-census-hub-page-to-link-to-new-dataset-finder-page
How to review
DATASET_FINDER_ENABLED=1 ENABLE_GET_DATA_CARD=1 make debug
Who can review
!me

See comment - also, do you think it's worth adding a default flag setting in config.go?

Added.
Though false is default, so I probably wouldn't add it except that it's sort of seems the pattern.


Share Image With Caption

I explored a lot and did not find a way to bound caption with an image while sharing,I did not found it any documentation or sample. Since we are using sendbird and must need this future ASAP. Can anyone please tell me can you fix this issue or if it is already there please point me out.

@IndieAppDevs
We don't support a bound caption on "sendFileMessage".
If you use our product and have a question, you can be supported by our homepage.
https://help.sendbird.com/hc/en-us/requests/new
Thank you for your interest in SendBird SDK :)

@IndieAppDevs If you are looking to add a caption in general, please follow the answer provided here:
https://github.com/smilefam/SendBird-iOS/issues/60

Send a file message with data that has the caption.
If you want to send a file message with an SBDFileMessageParams, set the caption in the data of the SBDFileMessageParams object:
https://docs.sendbird.com/ref/ios/Classes/SBDBaseMessageParams.html#//api/name/data
If you want to use a method to send a file message, use the method that has a data parameter:
https://docs.sendbird.com/ref/ios/Classes/SBDBaseChannel.html#//api/name/sendFileMessageWithBinaryData:filename:type:size:thumbnailSizes:data:customType:progressHandler:completionHandler:
You can see the caption in the data of the file message:
https://docs.sendbird.com/ref/ios/Classes/SBDBaseMessage.html#//api/name/data


Install git-lfs

Since the jenkins git plugin has a LFS option out of the box, this container should support git lfs commands.
Fixes jenkinsci/docker#551

This is a base image that people can extend, it should not include extra plugins or packages, for that you can easily create a new image or use Jenkins agents

What's extra about it? As the image ships now git lfs is supported by the jenkins side, but not the image, it essentially ships with broken functionality.

There are 1300 hundred plugins available for Jenkins, this contained does not (and will not) support each of them "out of the box."
The "Suggest Plugins" are just suggestions by Jenkins, but that doesn't mean that the Docker container will support running code on the master out the box either. Note that there is no Ant, Gradle, or Subversion in this container, despite those being "Suggested.
Generally speaking, this container will not include tools inside the image by default, both to keep the image small, but also to discourage running Pipelines and Jobs on the "master" node, which is very bad for security, as workloads (such as Pipeline from SCM) can access private data within the Jenkins instance.
If you're intent on using Git LFS, and running Pipelines/jobs on your master, then you will need to extend this container and add your tools as necessary

Not having Git LFS in the image presents an interesting challenge.  If we want to be rigorous about a minimal image, then we probably should not include git in the image either, since it probably should not be required on the master agent.
I suspect git is required on the master agent for Blue Ocean and for Pipeline (even with no executors defined on the master).  I haven't checked to confirm it, but I suspect that even with no executors on the master, git commands are executed on the master for Pipeline and for Blue Ocean because the master agent performs "Scan repository"  operations and likely uses command line git to do so.
If I'm correct about the requirement for command line git on the master even with no executors, then as soon as a user defines a Pipeline on a git LFS enabled repository, that Pipeline won't be able to checkout LFS managed files on the master.  That would be a problem if the user included Jenkinsfile as an LFS managed file.  I think it would be a mistake to manage Jenkinsfile as an LFS file, but that's one scenario where this could be a problem.
In case others can benefit, I maintain a docker-lfs repository with an lts-with-plugins branch that includes git-lfs and a set of plugins that are relevant to my testing of the git plugin and the git client plugin.  The Dockerfile changes in the lts and lts-with-plugins branches are useful for me and have helped with plugin testing.


[BUG] how to test the time of response about minicpm-v-2.6 served by VLLM

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

[X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

[X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior
I depoly minicpmv though VLLM, but I want to test the  the time of response.
First, I use OpenAI.chat.completions.create() to access the server. It response the result successfully. Code in the following:

And then, I use post request to access the server. Code is in the following:

The log of server is

the request json
How can I send the request of json to server?
期望行为 | Expected Behavior
No response
复现方法 | Steps To Reproduce
No response
运行环境 | Environment
- OS:
- Python:
- Transformers:
- PyTorch:
- CUDA (`python -c 'import torch; print(torch.version.cuda)'`):

备注 | Anything else?
No response

Download
https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Download
https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file
password: changeme
In the installer menu, select "gcc."

你好，vllm 端点启动可以支持视频吗

请看这个教程https://modelbest.feishu.cn/wiki/LZxLwp4Lzi29vXklYLFchwN5nCf?from=from_copylink


Update swift-build-testing.yml

Summary by CodeRabbit

新功能

修改了Swift项目的构建工作流，以便在“main”分支上直接推送更改时也能触发工作流。

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 13.20%. Comparing base (169830c) to head (ab43d7f).
Report is 16 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #12      +/-   ##
==========================================
+ Coverage   12.91%   13.20%   +0.29%
==========================================
  Files         113      110       -3
  Lines       15513    19050    +3537
==========================================
+ Hits         2003     2516     +513
- Misses      13510    16534    +3024

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Simplify identifier creation in ConcatenatedModules

What kind of change does this PR introduce?

Optimization
Did you add tests for your changes?

No
If relevant, link to documentation update:

Summary

This was implemented in #5997 to fix some memory issues, but I think we
can make this a little more efficient. By using a single forEach instead
of a map, filter, and join, we avoid unnecessary array creations and
iterations.
Does this PR introduce a breaking change?

No
Other information

I did measure it and calling hash.update often is more expensive than calling it with a large string.
As explanation I would guess update calls into node.js C++ + OpenSSL while concatenating strings is very optimized in JIT.

But thanks for the PR. We can optimize webpack by doing the reverse: calling hash.update less often and instead concatenate string.

Thanks for doing the profiling @sokra! I've added a commit to this PR that makes this change.

Thanks

#5718


Move Migration Guides to API ref dropdown

The migration guides currently live in the "Start" section but it was decided that it would be better to add it as a section under the API reference dropdown. At this point we should also reconsider which migration guides we keep, and which are no longer needed and can be removed
Needs further discussion:
we currently have the following migration guides:

https://docs.quantum-computing.ibm.com/start/compare
https://docs.quantum-computing.ibm.com/start/migrate
https://docs.quantum-computing.ibm.com/start/migrate-examples
https://qiskit.org/documentation/migration_guides/algorithms_migration.html (to be migrated to onedocs soon)
https://qiskit.org/documentation/migration_guides/opflow_migration.html  (to be migrated to onedocs soon)
https://qiskit.org/documentation/migration_guides/qi_migration.html  (to be migrated to onedocs soon)

We should definitely keep the algos, opflow and quantum instance migration guides, but there is some debate over which (if any) of the 3 runtime migration guides we should keep. @lerongil @jyu00 @ElePT @blakejohnson please feel free to weigh in so we can come to an agreement on this

I don't know what are the current arguments for/against keeping the runtime migration guides, so I might be repeating arguments, but here are a few thoughts on my side:

A lot of the content of these guides is somewhere between a migration guide and a how-to (except for the provider guide, which is more of a typical migration guide), and I think that it does necessarily have to live in a "migration guides" section. On top of that,  https://docs.quantum-computing.ibm.com/start/compare is more of an informative note than a guide per se. I could see it being merged with other docs page or kept as kind of general/introductory content (should also be checked to make sure that the narrative around backend.run and the primitives is coherent).
I admit that the migration examples in https://docs.quantum-computing.ibm.com/start/migrate-examples are fairly convoluted. This was because the point we were migrating "from" was not very clear, I tried to cover as many paths as possible, and honestly, I am not too satisfied with them. I can also see that my "from" was an algorithms/applications dev perspective (referring to opflow/quantum instance), which is no longer part of the qiskit narrative, and can contribute to the confusion.
Despite what I said in 2, there are important points conveyed by https://docs.quantum-computing.ibm.com/start/migrate-examples that I think are key to cover somewhere in the documentation, be it as a migration guide or somewhere else, those are: a clear explanation of what the different primitives are useful for, how to import the right primitive for your task, and how to change from one primitive to the other.

For this reason, I don't think that simply deleting these guides is a good idea. I think that this content (content that shows the context around Aer primitives, runtime primitives, backend primitives...) needs to live somewhere, and somewhere easy to find, because it is really one of the most commonly asked questions by external users. I understand that, in particular, Aer primitives could be tricky to explain, as they are kind of "external", but at least a clear separation between terra (reference/backend) and runtime ones must be made. People go to the runtime tutorials where most of the example use reference primitives, and they get extremely confused with sessions and what does what. I think if would be extremely helpful to have this compiled in a documentation page that we could easily share with our users when asking about it. So far, the closest thing I am aware we have are these migration examples.


Hackathon 2017

This is the official Hackathon 2017 ticket, which I will update timely as needed.
Steps

[x] Find a city in which to host: Munich

[x] Find a place to stay, the-tent

[x] Find a host: Virtual Identity AG
[x] Settle dates

either on 22 September
second preference is on 15th September

[ ] Virtual Identity will verify the dates to us
[ ] Announce via a blog post
[ ] Make press announcement

social media
maker, hackerspaces via mail or Twitter
on Riot ofc.
(any other effective way to spread about the Hackathon)

The blog post
The blog post should include the following

a call for papers

I'd like to have at least 1, at maximum 5 presentations (given the size of the event)
as in 2015, a presentation about Inexor would be lovely
talk/workshop about the used technology is also very much appreciated, especially flex

the dates for the event
the "package" price, the package should include

the ticket for the-tent
a group ticket / pro-rata for the Munich suburb
(optionally) a "Inexorable" T-Shirt

a_teammate: as a sidenote I would suggest we make it really clear that we are happy for people to just strive by to get things explained without need to commit themselves to the project (thats one of the things I dislike about the word "Hackathon" btw -> I immediately think "nah too much work, don't wanna be there")

I have opened a fresh pad for this at https://hackmd.io/IYDg7MBsAM0GYFoCMBmApgEwQFm2MCAnJBgKwIDGFcSYIkwKGwATEA==

I imagine the schedule to be like this

Friday

Meet at 5pm at Munich Central Station (5pm should be feasible for everyone?)
Drive to the tent
Meet&Greet at 6pm or 6:15, so everyone has time to set up his place
Cooking at 6:30 (we better decide that at the-tent itself, before it gets an outburst in a Github Ticket 🗡 )
Other enjoyable evening activities

Saturday

Breakfast at 10am
Official opening at 11:45am at Virtual Identity
12:00-12:15 Introduction to Inexor (talk? Who want's?)
12:15-13:00 Meet & Greet, Inexor install-party
13:00-14:00 Workshop on Inexor Flex (I strongly recommend this)
14:00 ~ 14:30 or 15:00 Food. I've been told there is a kitchen at VI, but we can order pizza too
15:00 ~ 18:30 "Open Hacking", whatever that means (do music, code, images, textures, maps, story ideas, tickets ...........)
15:00 - 15:45 Administrative work (tickets, projects, votings...)
18:30, open end, Grill & Chill

at Virtual Identity
at the-tent
at the Isar-Park, it's lovely

later, do what you have to do

Sunday

11am - 12am Place for additional workshops or talks
12-am - 2pm Open Hacking (as above)
2pm-2:30pm Closing ceremony at Virtual Identity
3:30pm - 4pm Closing ceremony the-tent
later, get home safely

:+1: for the schedule

[x] @aschaeffer Workshop on Inexor Flex

Following are the points for the administrative part of the Hackathon

release 0.9
write a blog post to the release
check all the tickets concerning the release
define a "Definition of Done" for tickets
talk about standards, and the server-list proposal

if we can find a consens, decide a karma ranking table for the new proposal

answer from the Freiburger CCC:

hi a_teammate,
Sauerbraten #FTW
Terminlich haben wir am 3. Oktober Türöffnertag der Sendung mit der Maus.
am 7.10 eine kleine Geburtstagsfeier und vom 28.10-01.11. die Small Hackerspace Convention.
Somit wäre Wochenende 15.-17. und 20.-22. frei.
Je nach dem könnt ihr euch aber auch einfach der Small Hackerspace Convention anschließen sofern ihr kein Problem mit etwas mehr Trubel habt.
 Ihr fändet mit Sicherheit Testspieler.
Gebt Bescheid wie es euch passt  ich denke wir werden eine Lösung finden.
Habt ihr jemanden aus dem Raum Freiburg im Team? Wäre super mal vorher jemanden Face to Face zu sehen.
Gruß Tau

Let's set this as the new date, fix the location etc

Somit wäre Wochenende 15.-17. und 20.-22. frei.

Meaning that 15-17 is the only weekend they can offer to us? 20-22 would be Wednesday to Friday.

I answered Tau:
I think the date is pretty much settled on the SHC weekend (27-29. 10.).
Our current issue is the accommodation. maybe some "jugendherberge" (^= youth hostel) is giving us a discount.

Since there's not been so much activity recently, here comes my suggestion

I can organize everything in Tuebingen
we can stay either at the University (I have staff access to developer rooms)
or at the FabLab https://www.fablab-neckar-alb.org/ (I was a member there so it'll be very likely we can get any date we want)
we can stay here, http://www.jugendherberge.de/de-de/jugendherbergen/tuebingen113/portraet

Below is a Doodle in which (regardless of the city) please everybody should add his favourite weekend in October.
https://doodle.com/poll/yfm6qz9cmzrrm4v3

What about Freiburg? I was assuming this was pretty much settled already

I wasn't quiet so sure about that. Anyhow my offer is not intended to block the Freiburg event. I just think it'll be most usefull to have an average statistic about what weekend is good in October, so that we can have a more specific look based on the date.

Since @aschaeffer will probably not be able to join us on the weekend, who else is in for a flex workshop? Should I do it? In case I do it, who else would give an introduction to Inexor?

The accomodation is private and probably will be http://tuebinger-lichtenstein.de/

How about we create a flyer that we can use, maybe @aschaeffer or @IAmNotHanni do think that this is possible?
@Croydon mentioned that some parts of the blog post are outdated. Can you fix these?

Since this seems all settled, I will close the ticket.
I will try to print T-Shirts on Wednesday, it will cost < 10 euros. Please let me know via PM or mail if you want one.

https://inexor.org/post/2017/09-28-inexor-hackathon-2017


fix some stroke:#000 occurrences

there are 3 files that still had stroke:#000 instead of stroke:currentColor

Testing done

### Submitter checklist
- [x] Make sure you are opening from a **topic/feature/bugfix branch** (right side) and not your main branch!
- [x] Ensure that the pull request title represents the desired changelog entry
- [x] Please describe what you did
- [ ] Link to relevant issues in GitHub or Jira
- [ ] Link to relevant pull requests, esp. upstream and downstream changes
- [ ] Ensure you have provided tests - that demonstrates feature works or fixes the issue

FWIW the ionic team have suggested we fix this in l:icon itself as thats how they do it via CSS attributes so they don't have to do what we're doing

Not sure what you exactly mean, but if a symbol is more than just some lines in a single color it might have side-effects when applying styles unconditionally.

see https://github.com/ionic-team/ionicons/issues/1129#issuecomment-1295027543.
I wonder if there'd be a way to add it for just ionicons.

afaik all icons in here have class="ionicon"  (to be verified)


feat: Added a display element for the gradients and animation degree

🛠 Fixes #172
👨💻 Changes proposed
added a display element for gradients degree.
caused a change in the structure of the label > span element and styles
✔ Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

📷 Screenshots

You have some merge conflicts to fix✌🏽

Hey @Dun-sin, I am not sure what the conflict is and how to go about it.
I've tried updating my forked branch of the repo to the main branch, but it seems the conflict is still there

Hey @Dun-sin, I am not sure what the conflict is and how to go about it.
I've tried updating my forked branch of the repo to the main branch, but it seems the conflict is still there

You can Google how to fix a merge conflict on github, watch a YouTube video or read an article

Hey @Dun-sin, I am not sure what the conflict is and how to go about it.
I've tried updating my forked branch of the repo to the main branch, but it seems the conflict is still there

You can Google how to fix a merge conflict on github, watch a YouTube video or read an article

I think I figured it out.
Thanks

:tada: This PR is included in version 2.4.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Fix break description.

Not sure I understood the "add #106 to the help" part. And not sure if I came up with a good way to accomplish it.

@deivid-rodriguez Can you help with this failure?
  1) Failure:
Byebug::SteppingRescueTestCase#test_next_steps_over_rescue_when_raising_from_c_method [/home/yuri/_/byebug/test/commands/stepping_test.rb:144]:
--- expected
+++ actual
@@ -1 +1 @@
-9
+26

On second thought, that must be unrelated issue manifesting itself on that particular machine. Where I am now it doesn't fail.

Thanks @x-yuri, the PR was perfect!
The test failure was unrelated indeed, you're probably not using the latest patch level of Ruby on that machine. Upgrade and the test should pass.

I was running ruby-2.1.2 there. In ruby-2.1.3, ruby-2.1.4 and ruby-2.1.5 the tests pass.


Comparing attack effectiveness is done incorrectly

Using the data provided, it is not possible to compare the efficacy of different attacks across models. Imagine we would like to decide whether LLC or ILLC was the stronger attack on the CIFAR-10 dataset.
Superficially, I might look at the “Average” column and see that the average model accuracy under LLC is 39.4% compared to 58.7% accuracy under ILLC. While in general averages in security can be misleading, fortunately, for all models except one, LLC reduces the model accuracy more than ILLC does, often by over twenty percentage points.
A reasonable reader might therefore conclude (incorrectly!) that LLC is the stronger attack. Why is this conclusion incorrect? The LLC attack only succeeded 134 times out of 1000 times on the baseline CIFAR-10 model. Therefore, when the paper writes that the accuracy of PGD adversarial training under LLC is 61.2% what this number means is that 38.8% of adversarial examples that are effective on the baseline model are also effective on the adversarially trained model. How the model would perform on the other 866 examples is not reported. In contrast, when the base model is evaluated on the ILLC attack, the attack succeeded on all 1000 examples. The 83.7 accuracy obtained by adversarial training is inherently incomparable to the the 61.2% value.

Using the data provided, it is not possible to compare the efficacy of different attacks across models. Imagine we would like to decide whether LLC or ILLC was the stronger attack on the CIFAR-10 dataset.
Superficially, I might look at the “Average” column and see that the average model accuracy under LLC is 39.4% compared to 58.7% accuracy under ILLC. While in general averages in security can be misleading, fortunately, for all models except one, LLC reduces the model accuracy more than ILLC does, often by over twenty percentage points.
A reasonable reader might therefore conclude (incorrectly!) that LLC is the stronger attack. Why is this conclusion incorrect? The LLC attack only succeeded 134 times out of 1000 times on the baseline CIFAR-10 model. Therefore, when the paper writes that the accuracy of PGD adversarial training under LLC is 61.2% what this number means is that 38.8% of adversarial examples that are effective on the baseline model are also effective on the adversarially trained model. How the model would perform on the other 866 examples is not reported. In contrast, when the base model is evaluated on the ILLC attack, the attack succeeded on all 1000 examples. The 83.7 accuracy obtained by adversarial training is inherently incomparable to the the 61.2% value.

Comparing different attacks against one model and determining how powerful the attack is evaluated and discussed in Section IV.A where all attacks attack the same model based on the same natural examples. And obviously, LLC is not the stronger attack than ILLC according to the results from Table III.
However, Table V shows the classification accuracy of defense-enhanced models against those adversarial examples that have misclassified by the raw model. That is, 100% minus the classification accuracy does not represent the success rate of attacks. Therefore, the numbers in Table V should be interpreted as the effectiveness of defenses (classification accuracy) against successful adversarial examples.

Right, that's the correct way to interpret these numbers. My concern is that we are going to see someone say "LLC was found to be a stronger attack against defended models than ILLC [cite to this paper]". There's a nice saying that you shouldn't write just to be understood, but write so you can't be misunderstood. The current presentation of the paper encourages this type of misunderstanding. I do agree the data is there to see that LLC is weaker than ILLC on a baseline model, but because this is the only figure that tries to see how well LLC/ILLC works against defended models (the other figure just shows how well LLC/ILLC works on an undefended model), people will take it as such.


Set -fstandalone-debug in clang debug builds

"Clang supports a number of optimizations to reduce the size of debug information in the binary. They work based on the assumption that the debug type information can be spread out over multiple compilation units. For instance, Clang will not emit type definitions for types that are not needed by a module and could be replaced with a forward declaration. Further, Clang will only emit type info for a dynamic C++ class in the module that contains the vtable for the class.
The -fstandalone-debug option turns off these optimizations. This is useful when working with 3rd-party libraries that don’t come with debug information. Note that Clang will never emit type information for types that are not referenced at all by the program."
Without -fstandalone-debug, lldb usage is degraded as debug information of C++ standard library types such strings or containers will be unusable and the types cannot be printed in lldb. Thus set the option by default in clang builds to assist developers unfamiliar with clang / lldb intricacies.

@mxnet-bot run ci [centos-gpu, miscellaneous]


unsupported reserved words, lack of escaping for nodes

I was looking for an alternative to graphviz but I realised quickly that mermaid has some serious limitations that are not documented at all.
graph LR
a-->default

This will not render, probably because of the default node name. I even tried to use double quotes or single ones but with no luck. Also it seems that the online editor gives only a generic "Invalid syntax" which is obviously not helpful at all. A more verbose message, maybe even one with a hyperlink to a documentation would be really desired. Thank you.

I am sorry you had problems with your diagram. For me this worked well:
graph LR
a-->b["default"]

I can see that syntax error might not be so helpful. Good ide with the link to the docs.

b["default"]

This doesn't work in sequence diagrams: http://knsv.github.io/mermaid/live_editor/#/edit/c2VxdWVuY2VEaWFncmFtCiAgICBBLT4-IEJbImItc2VydmljZSJdOiBUZXN0

This is not just node name "default", also affects "go_default". The keything is WHEN this will be fixed?

I confirm that it is still an issue: https://mermaidjs.github.io/mermaid-live-editor/#/edit/c2VxdWVuY2VEaWFncmFtCiAgICBBLT4+IEJbImItc2VydmljZSJdOiBUZXN0/error/UGFyc2UgZXJyb3Igb24gbGluZSAyOgouLi5pYWdyYW0gICAgQS0-PiBCWyJiLXNlcnZpY2UiXTogVGVzdAotLS0tLS0tLS0tLS0tLS0tLS0tLS0tXgpFeHBlY3RpbmcgJ05MJywgJ0FTJywgJywnLCAnU09MSURfT1BFTl9BUlJPVycsICdET1RURURfT1BFTl9BUlJPVycsICdTT0xJRF9BUlJPVycsICdET1RURURfQVJST1cnLCAnU09MSURfQ1JPU1MnLCAnRE9UVEVEX0NST1NTJywgJ1RYVCcsIGdvdCAnLSc
I will take a look when I have some time.


nodePackages: patch node2nix for npm v7+ and switch to building package set with current nodejs

Description of changes
This PR updates the nodePackages set to the latest Node.js LTS of the nodejs package, which is now nodejs-18_x. It does this by importing the fix from svanderburg/node2nix#302 that implements compatbility with NPM v7+ / Node.js 16+
Some notes:

I tried to make sure any packages that were currently building in nodePackages did not regress after this update. Only one package does not build in this PR that built before, nodePackages.mdctl-cli, since it does not support Node.js >14 yet, so I have marked it broken (but it still can be used via nodejs-14_x.pkgs.mdctl-cli as needed)
Some packages did not seem to be fully building during the npm rebuild step before, and while I'm not sure if they were still somehow working properly (I didn't test), I updated their build inputs to allow them to build with newer Node.js
I spot checked a number of CLI packages to make sure they all work as intended, but I could have missed some so feel free to check any packages you personally use

Note: I did not target this towards staging since the rebuild count is not too big, but if it should be targeted towards staging to help breakages get caught there first, I'm happy to retarget this branch
Closes #132456, closes #145432, closes #146440, closes #170722, closes #187337
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[x] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

The manual is failing build on master too so the CI failure seems unrelated to this change (fix for that is in #193338)

CI build failure https://github.com/NixOS/nixpkgs/actions/runs/3144204430/jobs/5109902065
My opinion on this personally is to merge as soon as the CI passes. This is important and it is prone to conflicts. We can fix the small details in subsequent PRs (if you don't mind of course)

CI build failure https://github.com/NixOS/nixpkgs/actions/runs/3144204430/jobs/5109902065 My opinion on this personally is to merge as soon as the CI passes. This is important and it is prone to conflicts. We can fix the small details in subsequent PRs (if you don't mind of course)

I've rebased the PR to pull in the manual fix from #193338  (unlucky timing I guess that I got rebased initially in the 1 hour manual was broken)

There will absolutely be some failures to fix, feel free to update if you find anything odd, happy to help.

There will absolutely be some failures to fix, feel free to update if you find anything odd, happy to help.

Thank you!
I noticed some of the aarch64-linux and -darwin builds failed on ofborg (I don't have a Mac) so I was actually looking through those logs now to see if something broke that shouldn't have
I will additionally watch Hydra for new failures in the next evaluation, in case I missed anything and can trace back any new failures to this PR

I noticed some of the aarch64-linux and -darwin builds failed on ofborg (I don't have a Mac) so I was actually looking through those logs now to see if something broke that shouldn't have

Skimming one of those logs, it looks like it's just timeouts :)

I've fixed all of the Hydra regressions due to this PR from https://hydra.nixos.org/eval/1782541#tabs-now-fail in follow-up PR #193533

I just noticed this PR. Great work! Thanks a lot for working on this!
If you have a ko-fi or such, drop me a link to buy you a coffee

I realise I didn't say this properly, but I'm hugely grateful too. Before this lits of npm packages were simply broken because they deprecated the node 14 APIs. Thanks again !

I realise I didn't say this properly, but I'm hugely grateful too. Before this lits of npm packages were simply broken because they deprecated the node 14 APIs. Thanks again !

I'm glad I could help! I know having nodePackages/node2nix not working with current Node.js versions was starting to get in my way a bunch for packaging stuff (even if I don't use Node much personally), so I implemented this update when I finally had the time and mental bandwidth to

If you have a ko-fi or such, drop me a link to buy you a coffee

I don't have a ko-fi or anything like that and I don't have a good way to accept money right now, but I do really appreciate the offer! ❤


ci_yaml autoroller should not be backfilled

The Linux ci_yaml flutter_roller target on the framework (and probably other repos too) is not guaranteed to run in the order commits appear in the tree, and will be backfilled. This can cause older commits to overwrite the build config of newer commits, leading to hard to debug one-off build failures.
We should either avoid backfilling, or even better, ensure we never push an older config over a newer one.

Actually I guess we should do both. Avoiding backfilling makes sure we always run newer commits. Rebase to TOT guarantees always rolling latest config (related issue: https://github.com/flutter/flutter/issues/112127)

FYI it seems like backfilling is disabled, but we don't run new builds. My expectation in the chart below is the top should be green.
Instead of backfilling, we want a greedy policy where we always run the tip of tree (whenever nothing else is running)

@CaseyHillers That is a good point which we have outlined here: https://github.com/flutter/flutter/issues/131501#issuecomment-1665946645. This issue is specifically for the backfill and the issue you mentioned is already tracked here: https://github.com/flutter/flutter/issues/112127.


EF SQL doc outdated in preview 6

builder.AddSqlServer() doesn't work until reference to Aspire.Hosting.SqlServer package is added. Is this a known change in preview 6, @IEvangelist?
Also there's no longer any hostPort parameter for the .WithEndpoint extension.

Hi @CamSoper - yeah, those hosting packages were all updated in preview 5, detailed here:

https://learn.microsoft.com/fr-fr/dotnet/aspire/whats-new/preview-5?tabs=dotnet-cli#packaging-changes

And preview 6 was a much smaller window (faster iteration), which happened just earlier this week. The hostPort parameter was renamed to just port.

https://learn.microsoft.com/en-us/dotnet/aspire/whats-new/preview-6


Generify ColumnReaderImpl and RecordReader (#1040)

Highly experimental
This is highly experimental, I want to get further fleshing out #171 and #1037 before settling on this. In particular I want to get some numbers about performance. However, I wanted to give some visibility into what I'm doing
Builds on top of #1021
Which issue does this PR close?
Closes #1040.
Rationale for this change
See ticket
What changes are included in this PR?
See ticket
Are there any user-facing changes?
No :grin:

Codecov Report

Merging #1041 (be0fc1b) into master (239cba1) will decrease coverage by 0.02%.
The diff coverage is 80.76%.

@@            Coverage Diff             @@
##           master    #1041      +/-   ##
==========================================
- Coverage   82.31%   82.28%   -0.03%
==========================================
  Files         168      168
  Lines       49031    49047      +16
==========================================
- Hits        40360    40359       -1
- Misses       8671     8688      +17

Impacted Files
Coverage Δ

parquet/src/arrow/array_reader.rs
76.82% <ø> (+0.06%)
:arrow_up:

parquet/src/column/reader.rs
71.23% <76.54%> (-1.10%)
:arrow_down:

parquet/src/arrow/record_reader.rs
93.96% <87.50%> (+0.23%)
:arrow_up:

parquet/src/util/memory.rs
91.12% <100.00%> (+0.08%)
:arrow_up:

arrow/src/datatypes/datatype.rs
65.95% <0.00%> (-0.43%)
:arrow_down:

parquet/src/data_type.rs
76.61% <0.00%> (-0.24%)
:arrow_down:

parquet_derive/src/parquet_field.rs
65.98% <0.00%> (-0.23%)
:arrow_down:

arrow/src/array/transform/mod.rs
85.24% <0.00%> (+0.13%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 239cba1...be0fc1b. Read the comment docs.

Running benchmarks on my local machine I get somewhat erratic results, from which I conclude this has no major impact on performance
arrow_array_reader/read Int32Array, plain encoded, mandatory, no NULLs - old
                        time:   [3.7939 us 3.8031 us 3.8114 us]
                        change: [-3.6579% -3.4154% -3.1951%] (p = 0.00 < 0.05)
                        Performance has improved.
arrow_array_reader/read Int32Array, plain encoded, mandatory, no NULLs - new
                        time:   [2.3030 us 2.3048 us 2.3073 us]
                        change: [+2.5908% +2.7441% +2.9142%] (p = 0.00 < 0.05)
                        Performance has regressed.
arrow_array_reader/read Int32Array, plain encoded, optional, no NULLs - old
                        time:   [59.193 us 59.275 us 59.363 us]
                        change: [-4.2623% -4.1285% -4.0009%] (p = 0.00 < 0.05)
                        Performance has improved.
arrow_array_reader/read Int32Array, plain encoded, optional, no NULLs - new
                        time:   [23.209 us 23.221 us 23.236 us]
                        change: [+32.531% +32.663% +32.835%] (p = 0.00 < 0.05)
                        Performance has regressed.
arrow_array_reader/read Int32Array, plain encoded, optional, half NULLs - old
                        time:   [142.37 us 142.41 us 142.44 us]
                        change: [+5.5942% +6.6789% +7.7376%] (p = 0.00 < 0.05)
                        Performance has regressed.
arrow_array_reader/read Int32Array, plain encoded, optional, half NULLs - new
                        time:   [139.07 us 139.89 us 140.59 us]
                        change: [+0.4422% +0.9960% +1.6028%] (p = 0.00 < 0.05)
                        Change within noise threshold.
arrow_array_reader/read Int32Array, dictionary encoded, mandatory, no NULLs - old
                        time:   [21.919 us 21.923 us 21.927 us]
                        change: [+1.3392% +1.7681% +2.0113%] (p = 0.00 < 0.05)
                        Performance has regressed.
arrow_array_reader/read Int32Array, dictionary encoded, mandatory, no NULLs - new
                        time:   [99.347 us 101.00 us 102.37 us]
                        change: [+5.5715% +6.7636% +8.2107%] (p = 0.00 < 0.05)
                        Performance has regressed.
arrow_array_reader/read Int32Array, dictionary encoded, optional, no NULLs - old
                        time:   [75.648 us 75.663 us 75.681 us]
                        change: [-1.5816% -1.5384% -1.4963%] (p = 0.00 < 0.05)
                        Performance has improved.
arrow_array_reader/read Int32Array, dictionary encoded, optional, no NULLs - new
                        time:   [112.52 us 113.33 us 114.36 us]
                        change: [+5.2751% +7.2166% +9.0108%] (p = 0.00 < 0.05)
                        Performance has regressed.
arrow_array_reader/read Int32Array, dictionary encoded, optional, half NULLs - old
                        time:   [144.77 us 144.80 us 144.83 us]
                        change: [-11.013% -10.318% -9.6258%] (p = 0.00 < 0.05)
                        Performance has improved.
arrow_array_reader/read Int32Array, dictionary encoded, optional, half NULLs - new
                        time:   [191.06 us 191.12 us 191.18 us]
                        change: [+3.4773% +3.5370% +3.5957%] (p = 0.00 < 0.05)
                        Performance has regressed.
arrow_array_reader/read StringArray, plain encoded, mandatory, no NULLs - old
                        time:   [800.06 us 800.19 us 800.32 us]
                        change: [-1.6826% -1.6388% -1.5967%] (p = 0.00 < 0.05)
                        Performance has improved.
arrow_array_reader/read StringArray, plain encoded, mandatory, no NULLs - new
                        time:   [124.84 us 124.86 us 124.88 us]
                        change: [+4.1077% +4.1575% +4.2088%] (p = 0.00 < 0.05)
                        Performance has regressed.
arrow_array_reader/read StringArray, plain encoded, optional, no NULLs - old
                        time:   [846.35 us 846.59 us 846.87 us]
                        change: [+0.8637% +0.9228% +0.9834%] (p = 0.00 < 0.05)
                        Change within noise threshold.
arrow_array_reader/read StringArray, plain encoded, optional, no NULLs - new
                        time:   [143.25 us 143.30 us 143.35 us]
                        change: [+2.6977% +2.7794% +2.8847%] (p = 0.00 < 0.05)
                        Performance has regressed.
arrow_array_reader/read StringArray, plain encoded, optional, half NULLs - old
                        time:   [773.74 us 776.61 us 779.87 us]
                        change: [+3.2218% +3.4681% +3.7063%] (p = 0.00 < 0.05)
                        Performance has regressed.
arrow_array_reader/read StringArray, plain encoded, optional, half NULLs - new
                        time:   [264.22 us 264.80 us 265.57 us]
                        change: [-1.3401% -1.1712% -0.9903%] (p = 0.00 < 0.05)
                        Change within noise threshold.
arrow_array_reader/read StringArray, dictionary encoded, mandatory, no NULLs - old
                        time:   [726.17 us 726.74 us 727.44 us]
                        change: [+1.2812% +1.3725% +1.4618%] (p = 0.00 < 0.05)
                        Performance has regressed.
arrow_array_reader/read StringArray, dictionary encoded, mandatory, no NULLs - new
                        time:   [116.83 us 116.91 us 116.99 us]
                        change: [-3.2217% -3.0893% -2.9282%] (p = 0.00 < 0.05)
                        Performance has improved.
arrow_array_reader/read StringArray, dictionary encoded, optional, no NULLs - old
                        time:   [802.16 us 803.89 us 805.57 us]
                        change: [-0.4055% -0.2549% -0.1073%] (p = 0.00 < 0.05)
                        Change within noise threshold.
arrow_array_reader/read StringArray, dictionary encoded, optional, no NULLs - new
                        time:   [134.39 us 134.43 us 134.48 us]
                        change: [+0.0304% +0.2086% +0.3678%] (p = 0.02 < 0.05)
                        Change within noise threshold.
arrow_array_reader/read StringArray, dictionary encoded, optional, half NULLs - old
                        time:   [742.00 us 742.57 us 743.00 us]
                        change: [+3.4464% +3.6453% +3.8440%] (p = 0.00 < 0.05)
                        Performance has regressed.
arrow_array_reader/read StringArray, dictionary encoded, optional, half NULLs - new
                        time:   [236.67 us 237.14 us 238.07 us]
                        change: [+1.7094% +1.9629% +2.5264%] (p = 0.00 < 0.05)
                        Performance has regressed.

What is strange to me is that this seems to have a consistent ~5% impact on the new StringArrayReader despite this change touching none of the code used by it. I suspect we're in the weeds of the wims of LLVM, which I'm not really sure it makes sense to optimise for at this stage - there's a lot of lower hanging fruit

I've renamed a number of the methods and traits based on the great feedback, and also added a load of doc comments. In particular I took inspiration from std::Vec, in particular Vec::spare_capacity_mut and Vec::set_len which is effectively what is going on here.
I'm happy that this interface is sufficiently flexible for the optimisations I have in mind, many of which I've already got an initial cut of, and so I'm marking this ready for review.
I am aware this is a relatively complex change, to an already complex part of the codebase so if anything isn't clear please let me know.

I've renamed a number of the methods and traits based on the great feedback, and also added a load of doc comments. In particular I took inspiration from std::Vec, in particular Vec::spare_capacity_mut and Vec::set_len which is effectively what is going on here.
I'm happy that this interface is sufficiently flexible for the optimisations I have in mind, many of which I've already got an initial cut of, and so I'm marking this ready for review.
I am aware this is a relatively complex change, to an already complex part of the codebase so if anything isn't clear please let me know.

cc @nevi-me  @sunchao and @jorgecarleitao
Please let us know if anyone else is interested in reviewing this PR. If not I'll plan to merge it in soon


usm: file registery: Introduce already activated callback

What does this PR do?
Introduces a callback from file-registry for every occasion of trying to register an already-registered binary.

Motivation
To support the go-tls unique use-case, in which we need to run a callback everytime a path id needs to be re-registered, we introduce a new callback option.
Currently, using a nop callback

Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

/merge


How can i Lunch Chrome when  lower version of ChromDriver[85] and upper version of chrome?

I have ChromeDriver 85 version in code which is the biggest in the NuGet but my system chrome version is  87. How can i handle this condition please help ?

Hi @sanatlami,
The NuGet package needs to be updated in the test assembly. You can find this by managing the NuGet Packages with the Package manager in Visual Studio. Please refer to this video which includes a demo:

Hi @sanatlami,
The NuGet package needs to be updated in the test assembly. You can find this by managing the NuGet Packages with the Package manager in Visual Studio. Please refer to this video which includes a demo:

Thank you @aliyoussefi for your reply.
i did update that but in my case NuGet has only 85 the latest one but Chrome in Version 87. I have attached the picture of NuGet and Chrome.

Thank you @aliyoussefi for your reply.
i did update that but in my case NuGet has only 85 the latest one but Chrome in Version 87. I have attached the picture of NuGet and Chrome.

for workaround you can try this webdriver.

for workaround you can try this webdriver.

you can use this package as workaround

you can use this package as workaround

Hi @sanatlami Please review @thathi comment as this is the correct package to use moving forward. I will create an issue to update the assembly in the source. Please confirm so we can close this out.

Hi @sanatlami Please review @thathi comment as this is the correct package to use moving forward. I will create an issue to update the assembly in the source. Please confirm so we can close this out.

Hi @sanatlami Please review @thathi comment as this is the correct package to use moving forward. I will create an issue to update the assembly in the source. Please confirm so we can close this out.

https://github.com/microsoft/EasyRepro/issues/1058

Hi @sanatlami Please review @thathi comment as this is the correct package to use moving forward. I will create an issue to update the assembly in the source. Please confirm so we can close this out.

https://github.com/microsoft/EasyRepro/issues/1058

Hi @aliyoussefi yes i am able to lunch the chrome with @thathi way. I am getting some error i dont think that's related to chrome.

Hi @aliyoussefi yes i am able to lunch the chrome with @thathi way. I am getting some error i dont think that's related to chrome.


ChartJS with dates on the X axis not displaying any graph

Hello,
I have a Vue.JS project getting data from a REST API (mine, so I can modify it if needed). This data is formatted for Chart.JS. I am supposed to display a graph with 3 datasets, 2 of type line with the same X values, but 1 of type bar with different X values (that's why I don't want to specify labels). Whatever, all the X values are dates, so I would like only 1 X axis for all the curves.
I am using datasets with (x,y) data format :
x is an ISO8601 date
y is a float
My problem is that NOTHING is displayed at all...could anyone help me please, I don't understand why. I feel like having done things right. I saw somewhere that I needed to include momentjs, but in the official documentation they say that it is not the case. I bet that the problems come from the dates because I tried changing the y values, and the 2 y axis bounds are modified (so y values are understood). I also tried addind the "xAxisID" option, nothing changed.
Here is a sample of my data (normally hundreds of values) :
{
  "type": "line",
  "data": {
    "datasets": [
      {
        "label": "Température",
        "borderColor": "red",
        "backgroundColor": "red",
        "fill": false,
        "data": [
          {
            "x": "2020-07-05T15:38:47.933711",
            "y": 2.8224692
          },
          {
            "x": "2020-07-05T15:48:47.490669",
            "y": 33.63129
          },
          {
            "x": "2020-07-05T15:58:48.182698",
            "y": 40.540405
          },
          {
            "x": "2020-07-05T16:08:47.829882",
            "y": 3.0312533
          },
          {
            "x": "2020-07-05T16:18:47.489026",
            "y": 49.145626
          }
        ],
        "yAxisID": "yAxeTemperature"
      },
      {
        "label": "Humidité",
        "borderColor": "blue",
        "backgroundColor": "blue",
        "fill": false,
        "data": [
          {
            "x": "2020-07-05T15:38:47.933711",
            "y": 33.980587
          },
          {
            "x": "2020-07-05T15:48:47.490669",
            "y": 2.0313625
          },
          {
            "x": "2020-07-05T15:58:48.182698",
            "y": 24.249685
          },
          {
            "x": "2020-07-05T16:08:47.829882",
            "y": 7.4426904
          },
          {
            "x": "2020-07-05T16:18:47.489026",
            "y": 2.6335742
          },
          {
            "x": "2020-07-05T16:28:48.175547",
            "y": 25.92827
          }
        ],
        "yAxisID": "yAxeHumidite"
      }
    ]
  },
  "options": {
    "responsive": true,
    "hoverMode": "index",
    "stacked": false,
    "title": null,
    "scales": {
      "xAxes": [
        {
          "type": "time",
          "display": true,
          "position": "bottom",
          "id": "xAxeTime",
          "scaleLabel": {
            "display": true,
            "labelString": "Temps",
            "fontColor": "black"
          },
          "time": {
            "unit": "minute",
            "parser": "moment.ISO_8601",
            "tooltipFormat": "ll"
          }
        }
      ],
      "yAxes": [
        {
          "type": "linear",
          "display": true,
          "position": "left",
          "id": "yAxeTemperature",
          "scaleLabel": {
            "display": true,
            "labelString": "Température",
            "fontColor": "red"
          }
        },
        {
          "type": "linear",
          "display": true,
          "position": "left",
          "id": "yAxeHumidite",
          "scaleLabel": {
            "display": true,
            "labelString": "Humidité",
            "fontColor": "blue"
          }
        }
      ]
    }
  }
}

Here is how my chart is created (using vue-chartjs and chart.js) :
createChart(chartId : string, chartData : GrapheBean) {
      const ctx = document.getElementById(chartId);
      // @ts-ignore
      const myChart = new Chart(ctx, {
        type: chartData.type,
        data: chartData.data,
        options: chartData.options,
      });
    }

The result is an empty graph :

I am stuck now, even if still trying things with little hope. Thanks a lot in advance for the ones who could help me.

What version of Chart.js?

What version of Chart.js?

2.9.3

moment.ISO_8601 shouldn't be enclosed in quotes

I have just solved the problem.
It came from 2 things :

As you said "moment.ISO_8601" was a problem. It was one of my last desperate trials. But it is not necessary as it is the default option. I removed this option.

I was not importing the bundle version of chart.js. I added this line :
<script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/2.9.3/Chart.bundle.min.js"></script>

And it is working!
Just one thing, this is clearly stated https://www.chartjs.org/docs/latest/getting-started/installation.html#bundled-build but not https://www.chartjs.org/docs/latest/axes/cartesian/time.html#date-adapters in the time part of documentation. It says "By default, Chart.js includes an adapter for Moment.js.", which was confusing for me, not knowing about the 2 versions of Chart.js.
Thanks for your help anyway, and thanks to all the contributors of this great library.


[bug] No machine details showing in "Machines" list

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
None of my newly registered Machines are showing any details. IP addresses, Talos versions, memory, disk. This is on the "aegn" Omni account.

Expected Behavior
I expect to see the details for each of my registered but unassigned machines.
Steps To Reproduce

In the "aegn" Omni account, register a new machine.
Navigate to the Machines page.
Note the absence of machine details for registered machines.

What browsers are you seeing the problem on?
Chrome
Anything else?
No response

was this machine running Talos previously? not being wiped and still running Talos on being joined?

Looks like a delete + a wipe via PXE + a reinstall fixed this.


feat(documentation): new doc create

Summary

Resources

Creating a package docs

this did not generate any preview version


createOptions specified directly in deployment.json spits out validationErrors

[command]/usr/bin/az iot edge deployment create --config-id <REDACTED_OUT> --hub-name <REDACTED_OUT> --content /tmp/deployment_1588690245764.json --target-condition deviceId=<REDACTED_OUT> --priority 10
ERROR: {
  "validationErrors":[
    {
      "description":"{'HostConfig': {'PortBindings': {'51213/tcp': [{'HostPort': '51213'}]}, 'Privileged': True, 'Devices': [{'PathOnHost': '/dev/v4l/by-id/usb-Intel_R__RealSense_TM__Depth_Camera_435i_Intel_R__RealSense_TM__Depth_Camera_435i_911623050407-video-index0', 'PathInContainer': '/dev/v4l/by-id/usb-Intel_R__RealSense_TM__Depth_Camera_435i_Intel_R__RealSense_TM__Depth_Camera_435i_911623050407-video-index0', 'CgroupPermissions': 'rwm'}, {'PathOnHost': '/dev/v4l/by-id/usb-Intel_R__RealSense_TM__Depth_Camera_435i_Intel_R__RealSense_TM__Depth_Camera_435i_911623050407-video-index1', 'PathInContainer': '/dev/v4l/by-id/usb-Intel_R__RealSense_TM__Depth_Camera_435i_Intel_R__RealSense_TM__Depth_Camera_435i_911623050407-video-index1', 'CgroupPermissions': 'rwm'}]}} is not of type 'string'",
      "contentPath":"modulesContent.$edgeAgent.properties.desired.modules.app.settings.createOptions",
      "schemaPath":"properties.modulesContent.properties.$edgeAgent.properties.properties.desired.properties.modules.patternProperties.^[a-zA-Z0-9_-]+$.properties.settings.properties.createOptions.type"
    }
  ]
}

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7e02119b-094f-29ba-dd91-71702627cfa6
Version Independent ID: 48c4e621-56be-593e-a68f-9f74168bf0f5
Content: Use IoT Edge device local storage from a module - Azure IoT Edge
Content Source: articles/iot-edge/how-to-access-host-storage-from-module.md
Service: iot-edge
GitHub Login: @kgremban
Microsoft Alias: kgremban

Thanks for the question! We will review and get back to you shortly.

@asinitson Which OS are you using? Please ensure to Replace  and  with your host and module storage path; both values must be an absolute path.
Are you able to complete the steps through portal?

Have you had a chance to see above response?

Hello, sorry for not making clear this in my request: deployment works, but only when createOptions is specified as a string. Like this:
"createOptions": "{\"HostConfig\":{\"PortBindings\":{\"51213/tcp\":[{\"HostPort\":\"51213\"}]},\"Privileged\":true,\"Devices\":[{\"PathOnHost\":\"/dev/v4l/by-id/usb-Intel_R__RealSense_TM__Depth_Camera_435i_Intel_R__RealSense_TM__Depth_Camera_435i_some_id_here_-video-index1\",\"PathInContainer\":\"/dev/v4l/by-id/usb-Intel_R__RealSense_TM__Depth_Camera_435i_Intel_R__RealSense_TM__Depth_Camera_435i_some_id_here_-video-index1\",\"CgroupPermissions\":\"rwm\"},{\"PathOnHost\":\"/dev/v4l/by-id/usb-Intel_R__RealSense_TM__Depth_Camera_435i_Intel_R__RealSense_TM__D",
"createOptions01": "epth_Camera_435i_some_id_here_-video-index0\",\"PathInContainer\":\"/dev/v4l/by-id/usb-Intel_R__RealSense_TM__Depth_Camera_435i_Intel_R__RealSense_TM__Depth_Camera_435i_some_id_here_-video-index0\",\"CgroupPermissions\":\"rwm\"}]}}"

This is extremely inconvenient to work with: you easily get lost in all those line breaks. Documentation mentions another format:

Or, you can configure the local storage directly in the deployment manifest.

            "createOptions": {
                "HostConfig": {
                    "Binds":["<HostStoragePath>:<ModuleStoragePath>"]
                }
            }

This format from documentation does not work! So:

Documentation (linked to this bug report) is incorrect
or
There is a bug in IoT Edge
Maybe JSON schema is incorrect? See that in my case createOptions is an object, so it complains that object I provide is not a string as schema expects it:is not of type 'string'

You can see longer Error message above, in the first post.

In other words, to make crisply clear:

I have no problem with paths.
I can apply deployment.
I cannot use human-readable createOptions format from documentation.

Thanks for the detailed information. I have assigned the issue to the content author @kgremban  to evaluate and update as appropriate.

@asinitson The Azure IoT Tools extension for VS Code or the Azure IoT Edge tools for Visual Studio extension help a lot with formatting create options. You can work in a deployment.template.json file and write in pretty json with linebreaks. For example, looking at the edgeHub module description in VS Code:

Then, in the file explorer, right-click the template and select Generate IoT Edge Deployment Manifest. This command takes the more easily edited format of the template file and turns it into the format that IoT Edge expects from deployments in new files called deployment.{architecture}.json. So now the same edgeHub description has been condensed:

Does that help?

@kgremban: Thank you! This gets me somewhat closer.
Here is the full story:

I have an "IoT Edge Deployment Manifest".
Now realize I probably should have used "template" instead. Is that correct name for a human readable format?

That file is parameterized with envsubst, so I can inject container version, environment variable values dynamically without modifying template each time or creating copy for each environment.

Then I feed it to Azure IoT Edge task in Azure DevOps Server (on-premises)

Can I feed human readable "template" after envsubst (deployment.template.json) directly to the Azure IoT Edge task? I would like to skip all this manual conversion and let build server do it's job.
That would be ideal, but then I need to manually convert my existing "manifest" back into the "template". Tried to do it quickly, but couldn't, so I probably need to painstakingly copy modules piece by piece into some existing template to get the right format. I would like to get your confirmation that it will indeed work.

If that is not possible, what is the equivalent of "right-click the template and select Generate IoT Edge Deployment Manifest". I want to do it in an automated way, so I need to use Azure CLI or whatever Visual Studio Code is using. I don't see any hint when I run it from the UI:

P.S. What I really want is something like "Helm for IoT Edge", so I can define each module separately and be able to deploy them together or separately and most importantly I can inject parameters through environment variables. With my setup above I am trying to get close to that, but so far developer UX doesn't seem to be very good with all those workarounds.

@shizn can you take a look at these questions about tools/process?

Issue is resolved. Thank you for the help!

Will reply to my own question: yes, it is possible to generate manifest on a Azure DevOps CentOS Linux build agent as a separate task:

Requirements from the build agent (I tried it on CentOS 7.*):

Python (tried it with 3.6)

pip3 install --user iotedgedev
Really don't want to install it globally, so at least I can limit it to the user which runs the buid. I would prefer to use virtual environment (python3 -m venv env), but how would I force Azure IoT Edge task to run inside of it activated?

LC_ALL set to en_US.UTF-8
Otherwise generating deployment manifest fails, this might be poorly configured build agent issue.

Rant/feedback:
This is harder than I would like to be and installing iotedgedev pollutes build agent with ton of Python packages. In the long term I will consider running this conversion inside of a Docker container instead of using "native" tasks to keep agent clean.
Even better would be not having to deal with this conversion at all and have az cli convert everything for me. I don't really understand why should I even know about this manifest, isn't it an implementation detail? That would simplify the workflow and IoT Edge documentation.
For the contrast this is how installing Helm (Kubernetes package manager) looks like:
sudo snap install helm --classic (Ubuntu 18.04)
Alternatively you just download one binary file. I understand it's Go, not Python, but as tool user I don't really care. And this one executable covers environment variable substitution and everything I need to to do a deployment to the cluster.
These thoughts bring me here: Manage Azure IoT Edge deployments with Kubernetes, but there are problems with this approach:

Doesn't seem to be the mainstream (or already abandoned?)
I need Kubernetes cluster, just so I have good deployment experience (that's a big overkill)

So instead of going through all this trouble I am seriously considering KubeEdge.

Thank you for sharing your solution with us, asinitson. I've passed along your feedback to the product team, and am closing this issue as resolved.
#please-close


fix: use kebab-case as underscores are not allowed in edition 2024

The change is compatible. Just prepare for edition 2024 in advance.
See also:

https://github.com/rust-lang/cargo/issues/13629
https://github.com/rust-lang/cargo/pull/13783
https://github.com/rust-lang/cargo/pull/13798
https://github.com/rust-lang/cargo/pull/13804

Thanks!


Investigate sanitizing secret headers from ManagedIdentityCredential live tests

There are instances in tests recordings of ManagedIdentityCredential where the recordings have captured the secret header, such as this example. These secrets are no longer sensitive as they are ephemeral and only valid for the life of the hosting process. However, they do get flagged by credscan and we should update the tests and recordings to sanitize these if possible.

fixed https://github.com/Azure/azure-sdk-for-net/blob/c6c18984cca1210b84286d67c4c144fd36f3b5c1/sdk/identity/Azure.Identity/tests/IdentityRecordedTestBase.cs#L35C11-L35C11


mitmweb: Allow up to 4GB dump to be uploaded

Description
A temporary fix to allow up to 4GB dump to be uploaded by supplying max_buffer_size of the HTTPServer constructor to be 2**32 (4GB).
(Perhaps it is better to be a configurable item, or change the post mechanism to stream instead if there ever is any security concern implied due to usage or hardware limitations)
By default, if no argument is supplied, the maximum buffer size is 100MB, which causes saved dump file to be unreadable through the web interface.
Reference Documentation: Link
Possibly:
Resolves #4651, resolves #5609
Checklist

[x] I have updated tests where applicable.
[ ] I have added an entry to the CHANGELOG.

Just tried the latest version, still happening with a 400 Mb file

My bad, it just takes a really long time to upload it. Sorry for the confusion.


Watch won't work on file changes

Hi all,
"staticjinja build" works fine on my setup. "staticjinja watch" starts OK but brokes right after it detects a single change in templates. The full log is -

staticjinja watch --srcpath=D:\Work\saree.ru\redesign\templates\ --outpath=D:\Work\saree.ru\redesign\www
Rendering K11-mint-tunic/index.html...
Rendering base.html...
Watching 'D:\Work\saree.ru\redesign\templates' for changes...
Press Ctrl+C to stop.
modified D:\Work\saree.ru\redesign\templates\K11-mint-tunic\index.html
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Programs\Python27\lib\threading.py", line 810, in __bootstrap_inner
self.run()
File "C:\Programs\Python27\lib\site-packages\watchdog\observers\api.py", line 199, in run
self.dispatch_events(self.event_queue, self.timeout)
File "C:\Programs\Python27\lib\site-packages\watchdog\observers\api.py", line 368, in dispatch_events
handler.dispatch(event)
File "C:\Programs\Python27\lib\site-packages\watchdog\events.py", line 322, in dispatch
self.on_any_event(event)
File "C:\Programs\Python27\lib\site-packages\easywatch\easywatch.py", line 25, in wrapper
return handler(event.event_type, event.src_path)
File "C:\Programs\Python27\lib\site-packages\staticjinja\reloader.py", line 47, in event_handler
templates = self.site.get_dependencies(filename)
File "C:\Programs\Python27\lib\site-packages\staticjinja\staticjinja.py", line 278, in get_dependencies
return [self.get_template(filename)]
File "C:\Programs\Python27\lib\site-packages\staticjinja\staticjinja.py", line 94, in get_template
return self._env.get_template(template_name)
File "C:\Programs\Python27\lib\site-packages\jinja2\environment.py", line 791, in get_template
return self._load_template(name, self.make_globals(globals))
File "C:\Programs\Python27\lib\site-packages\jinja2\environment.py", line 765, in _load_template
template = self.loader.load(self, name, globals)
File "C:\Programs\Python27\lib\site-packages\jinja2\loaders.py", line 113, in load
source, filename, uptodate = self.get_source(environment, name)
File "C:\Programs\Python27\lib\site-packages\jinja2\loaders.py", line 160, in get_source
pieces = split_template_path(template)
File "C:\Programs\Python27\lib\site-packages\jinja2\loaders.py", line 31, in split_template_path
raise TemplateNotFound(template)
TemplateNotFound: K11-mint-tunic\index.html

How to fix that?

I hadn't seen it until now, but this just popped up for me as well. I might be able to look into this shortly.

Ah. I think I've found it. I'll prepare a pull request. In my case, Vim is creating a shadow file for this one and then immediately deleting it (or something). Adding os.path.isfile(filename) to the and condition in should_handle seems to fix it.

Thanks for the quick turn around. Not needing to maintain a fork of this will really help us out at work. I appreciate the attention :)


Remove item_info from dialog form

This field isn't necessary, as the form does not contain data for an item instance.

@EliJDonahue: Does this look good? https://github.com/ArasLabs/custom-modal-dialog/pull/5

Issue resolved by PR #5. Pull request has been accepted and merged.


Add Harvester File

This is to spawn the harvester it was created for the fd gamemode and would have been merged with the rest of the gamemode, but since it is now needed for the fw gamemode it is now merged early

We still need to figure out how to stop AI meleeing it dont we

Whatever that can be a future PR

I removed some stuff compared to the fd version of this file because i felt they where only relevant for fd. We either add them back in the fd spawn function or with the fd pr into this one


Should "List all Collaborators" require auth

.. to protect access to users' list without auth.
@aslakknutsen  @alexeykazakov  ?

In my opinion, if there isn't any need to keep it open, then let's require auth.

It should be up to org/space/collaborator. IMO we should allow public collaborators so, no auth needed to list public collaborators. But we also need to support private collaborators which we currently don't.

Was thinking the same - but I think we do need to hide user sensitive information like email address for unauthenticated users.

@sbose78 I am going to close this issue cause the question was answered. If we then find a reason to hide sensitive information, we could open another more specific issue.

Sure Hector.
On Jun 29, 2017 6:06 PM, "Hector Fernandez"<EMAIL_ADDRESS>wrote:

@sbose78 https://github.com/sbose78 I am going to close this issue
cause the question was answered. If we then find a reason to hide sensitive
information, we could open another more specific issue.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fabric8-services/fabric8-wit/issues/1333#issuecomment-311953387,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAhSAOe5gH9AMb0ZKPoHIAUBtDiuqb2Oks5sI5pHgaJpZM4NihoZ
.


Feat expect

Adds expect keyword, a chainable truthy assert statement.
closes #142

Coverage increased (+0.02%) to 94.695% when pulling 47428c735af0d3da68e3b22e1857ebdd476baf29 on feat-expect into 6dddefea26282833040b7bcb820ff421527515dc on main.

coverage: 94.695% (+0.02%) from 94.678%
when pulling 18563eb9d7335103930db93be768404b3b7523d6 on feat-expect
into 6dddefea26282833040b7bcb820ff421527515dc on main.


Adds http.route tag

See https://github.com/openzipkin/zipkin/issues/1874
See census's definition https://github.com/census-instrumentation/opencensus-specs/pull/49
See also brave's recent pull request for more docs we might want to import:
https://github.com/openzipkin/brave/pull/602

Thanks @rakyll and @jkschneider for your help defining this

text is copied here https://github.com/openzipkin/zipkin/pull/1924


Module 5 pip install requests fails

When trying to install the requests package during Module 5 the use of pip install fails:
~/environment/MythicalMysfitsStreamingService-Repository (master) $ pip install requests -t .
Traceback (most recent call last):
File "/usr/lib/python3.6/dist-packages/pkg_resources/init.py", line 658, in _build_master
ws.require(requires)
File "/usr/lib/python3.6/dist-packages/pkg_resources/init.py", line 972, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3.6/dist-packages/pkg_resources/init.py", line 863, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (pip 9.0.3 (/usr/lib/python3.6/dist-packages), Requirement.parse('pip==7.1.0'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/pip", line 5, in
from pkg_resources import load_entry_point
File "/usr/lib/python3.6/dist-packages/pkg_resources/init.py", line 3049, in
@_call_aside
File "/usr/lib/python3.6/dist-packages/pkg_resources/init.py", line 3033, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3.6/dist-packages/pkg_resources/init.py", line 3062, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python3.6/dist-packages/pkg_resources/init.py", line 660, in _build_master
return cls._build_from_requirements(requires)
File "/usr/lib/python3.6/dist-packages/pkg_resources/init.py", line 673, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/usr/lib/python3.6/dist-packages/pkg_resources/init.py", line 858, in resolve
raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pip==7.1.0' distribution was not found and is required by the application

I had the same problem.
From following the guide, it looks like the push at Module 4 Step 3A includes dockerfile changes to use python/pip.
The Docker file from Module 3 onwards uses python/pip, but there are no pushes (as part of the guide) so I wouldn't have noticed this.
Solution:
I just reverted the Dockerfile back to using python3/pip3 and it's fixed the build.
Fix:
The dockerfile from module 3 onwards should use python3/pip3


Install with pyodide

I was playing around with jupyterlite and tried to install Dask
import micropip
await micropip.install("dask")

I think that it would be fun if we could get dask and distributed running smoothly.  My guess is that this is doable if we're able to package things up in a pure python way, but that currently we're not in some way.
cc @jakirkham who I suspect understands this space well

 Traceback
Traceback (most recent call last):
  File "<console>", line 2, in <module>
  File "/lib/python3.8/asyncio/futures.py", line 260, in __await__
    yield self  # This tells Task to wait for completion.
  File "/lib/python3.8/asyncio/tasks.py", line 349, in __wakeup
    future.result()
  File "/lib/python3.8/asyncio/futures.py", line 178, in result
    raise self._exception
  File "/lib/python3.8/asyncio/tasks.py", line 282, in __step
    result = coro.throw(exc)
  File "/lib/python3.8/site-packages/micropip.py", line 158, in install
    transaction = await self.gather_requirements(requirements, ctx)
  File "/lib/python3.8/site-packages/micropip.py", line 154, in gather_requirements
    await gather(*requirement_promises)
  File "/lib/python3.8/asyncio/futures.py", line 260, in __await__
    yield self  # This tells Task to wait for completion.
  File "/lib/python3.8/asyncio/tasks.py", line 349, in __wakeup
    future.result()
  File "/lib/python3.8/asyncio/futures.py", line 178, in result
    raise self._exception
  File "/lib/python3.8/asyncio/tasks.py", line 280, in __step
    result = coro.send(None)
  File "/lib/python3.8/site-packages/micropip.py", line 227, in add_requirement
    await self.add_requirement(recurs_req, ctx, transaction)
  File "/lib/python3.8/site-packages/micropip.py", line 222, in add_requirement
    wheel, ver = self.find_wheel(metadata, req)
  File "/lib/python3.8/site-packages/micropip.py", line 238, in find_wheel
    release = releases[str(ver)]
KeyError: '3.9'
PythonError: Traceback (most recent call last):
  File "/lib/python3.8/asyncio/futures.py", line 178, in result
    raise self._exception
  File "/lib/python3.8/asyncio/tasks.py", line 282, in __step
    result = coro.throw(exc)
  File "/lib/python3.8/site-packages/pyolite/interpreter.py", line 23, in run
    result = await self.run_complete
  File "/lib/python3.8/asyncio/futures.py", line 260, in __await__
    yield self  # This tells Task to wait for completion.
  File "/lib/python3.8/asyncio/tasks.py", line 349, in __wakeup
    future.result()
  File "/lib/python3.8/asyncio/futures.py", line 178, in result
    raise self._exception
  File "/lib/python3.8/asyncio/tasks.py", line 282, in __step
    result = coro.throw(exc)
  File "/lib/python3.8/site-packages/pyodide/console.py", line 253, in load_packages_and_run
    raise e
  File "/lib/python3.8/site-packages/pyodide/console.py", line 247, in load_packages_and_run
    result = await eval_code_async(
  File "/lib/python3.8/site-packages/pyodide/_base.py", line 419, in eval_code_async
    return await CodeRunner(
  File "/lib/python3.8/site-packages/pyodide/_base.py", line 278, in run_async
    res = await res
  File "<console>", line 2, in <module>
  File "/lib/python3.8/asyncio/futures.py", line 260, in __await__
    yield self  # This tells Task to wait for completion.
  File "/lib/python3.8/asyncio/tasks.py", line 349, in __wakeup
    future.result()
  File "/lib/python3.8/asyncio/futures.py", line 178, in result
    raise self._exception
  File "/lib/python3.8/asyncio/tasks.py", line 282, in __step
    result = coro.throw(exc)
  File "/lib/python3.8/site-packages/micropip.py", line 158, in install
    transaction = await self.gather_requirements(requirements, ctx)
  File "/lib/python3.8/site-packages/micropip.py", line 154, in gather_requirements
    await gather(*requirement_promises)
  File "/lib/python3.8/asyncio/futures.py", line 260, in __await__
    yield self  # This tells Task to wait for completion.
  File "/lib/python3.8/asyncio/tasks.py", line 349, in __wakeup
    future.result()
  File "/lib/python3.8/asyncio/futures.py", line 178, in result
    raise self._exception
  File "/lib/python3.8/asyncio/tasks.py", line 280, in __step
    result = coro.send(None)
  File "/lib/python3.8/site-packages/micropip.py", line 227, in add_requirement
    await self.add_requirement(recurs_req, ctx, transaction)
  File "/lib/python3.8/site-packages/micropip.py", line 222, in add_requirement
    wheel, ver = self.find_wheel(metadata, req)
  File "/lib/python3.8/site-packages/micropip.py", line 238, in find_wheel
    release = releases[str(ver)]
KeyError: '3.9'

    at new_error (https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pyodide.asm.js:9:175413)
    at wrap_exception (https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pyodide.asm.wasm:wasm-function[297]:0x1a6c94)
    at FutureDoneCallback_call (https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pyodide.asm.wasm:wasm-function[379]:0x1aaab3)
    at byn$fpcast-emu$FutureDoneCallback_call (https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pyodide.asm.wasm:wasm-function[14420]:0x7a257c)
    at _PyObject_MakeTpCall (https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pyodide.asm.wasm:wasm-function[760]:0x1cabed)
    at __static_1198 (https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pyodide.asm.wasm:wasm-function[2906]:0x2c54f1)
    at byn$fpcast-emu$__static_1198 (https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pyodide.asm.wasm:wasm-function[22640]:0x7bf866)
    at __static_678 (https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pyodide.asm.wasm:wasm-function[1621]:0x21220e)
    at byn$fpcast-emu$__static_678 (https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pyodide.asm.wasm:wasm-function[22120]:0x7bdce6)
    at PyVectorcall_Call (https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pyodide.asm.wasm:wasm-function[763]:0x1cb006)

Ah, it looks like this problem goes deeper
import micropip
await micropip.install("tornado")

ValueError: Couldn't find a pure Python 3 wheel for 'tornado'

Also, I'm guessing that micropip is the right way to install, given https://pyodide.org/en/stable/usage/loading-packages.html#micropip but I'm very new to this.

This works
import micropip
await micropip.install("https://files.pythonhosted.org/packages/94/7c/f4b5259130ecdb0d2c7bc12b32eb8465b74e98b52c86f85de8b6ba5b112c/dask-2021.5.1-py3-none-any.whl")

It looks like maybe pyyaml is the blocker here.

As a more theoretical question—what would the point be of using dask in pyodide right now? pyodide doesn't seem to support threads or other forms of concurrency yet https://github.com/pyodide/pyodide/issues/237.
Welcome to the Pyodide terminal emulator 🐍
Python 3.8.2 (default, Apr 21 2021 19:58:37) on WebAssembly VM
Type "help", "copyright", "credits" or "license" for more information.
>>> import concurrent.futures
>>> import time
>>> with concurrent.futures.ThreadPoolExecutor() as exc:
...   _ = exc.map(lambda _: time.sleep(1), range(10))
...
Traceback (most recent call last):
  File "<console>", line 2, in <module>
  File "/lib/python3.8/concurrent/futures/_base.py", line 600, in map
    fs = [self.submit(fn, *args) for args in zip(*iterables)]
  File "/lib/python3.8/concurrent/futures/_base.py", line 600, in <listcomp>
    fs = [self.submit(fn, *args) for args in zip(*iterables)]
  File "/lib/python3.8/concurrent/futures/thread.py", line 188, in submit
    self._adjust_thread_count()
  File "/lib/python3.8/concurrent/futures/thread.py", line 213, in _adjust_thread_count
    t.start()
  File "/lib/python3.8/threading.py", line 852, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread

The WASM threading model is still at the proposal stage, so it'll probably still be some time until that's matured to where making a Python thread in pyodide just works.
So you couldn't use dask for parallel computation in the browser right now. The main cool thing I can imagine would be constructing graphs in the browser, then connecting to a real Python cluster running elsewhere and submitting them.
I agree this will be cool, but let's define what "running smoothly" will actually mean right now.

cc @jakirkham who I suspect understands this space well

Hehe not really 😅

cc @rth (who may know more here)

If all the dependencies are pure Python and have wheels  they would indeed be uninstallable with micropip.

It looks like maybe pyyaml is the blocker here.

Indeed, it should be resolved with https://github.com/pyodide/pyodide/pull/1181
A complication is that

tornado has some C extensions and so would need to be compiled via the Pyodide build system by adding it as a package here
and more importantly sockets aren't supported in the Webassembly VM : that might be a blocker for tornado?

If tornado doesn't work, I imagine, then one would need to find some alternative for communicating between nodes, possibly with WebSockets.
In the WASM VM, multiprocessing is indeed not supported, threading is not supported yet on all browsers (and not enabled in Pyodide), however there is a possibility to use WebWorkers which also allow to create separate threads although with a different API.

As a more theoretical question — what would the point be of using dask in pyodide right now?

One could be to offload expensive calculations to a cluster. Though in that respect one could also just use jupyter notebook directly with a locally running dask.
Another use case would be to be able to send commands to Python in WASM from Python running locally (or the other way around), for instance for testing https://github.com/pytest-dev/pytest-xdist/issues/336

As a more theoretical question—what would the point be of using dask in pyodide right now? pyodide doesn't seem to support threads or other forms of concurrency yet pyodide/pyodide#237.

We could connect to remote Dask clusters

and more importantly sockets aren't supported in the Webassembly VM : that might be a blocker for tornado?

Ah, if sockets are out then yeah, this has limited utility.

So, how not-supported are sockets.  Are there any inter-node communication systems available?  Websockets maybe?

Dask array in local synchronous mode works fine (with a workaround from https://github.com/pyodide/pyodide/issues/1603) on https://pyodide.org/en/latest/console.html
Nice for demos, teaching, trying things out etc.
Welcome to the Pyodide terminal emulator 🐍
Python 3.9.5 (default, Aug 28 2021 07:18:29) on WebAssembly VM
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.modules['_multiprocessing'] = object # see https://github.com/pyodide/pyodide/issues/1603
>>> import micropip
>>> await micropip.install("dask[array]")
>>> import dask
>>> dask.config.set(scheduler='synchronous')
<dask.config.set object at 0xc8ef10>
>>> import numpy # not sure why this is needed, following line fails otherwise
>>> import dask.array as da
>>> x = da.random.random((1000, 1000), chunks=(100, 100))
>>> x.compute()
array([[0.13043454, 0.16097226, 0.12493807, ..., 0.71990046, 0.64542954,
        0.55997499],
       [0.02051068, 0.59918816, 0.25248748, ..., 0.05438529, 0.05219407,
        0.97351945],
       [0.76723807, 0.95366328, 0.10620701, ..., 0.66297666, 0.52551189,
        0.5808868 ],
       ...,
       [0.58133475, 0.16766972, 0.71604937, ..., 0.62901447, 0.9411289 ,
        0.45654889],
       [0.44033403, 0.59576681, 0.81159661, ..., 0.60512902, 0.99332665,
        0.10700775],
       [0.73980408, 0.05269962, 0.079521  , ..., 0.8677161 , 0.25877279,
        0.39054187]])
>>>

@ashokgaire my guess is one would need to build a new package for Pyodide. In any event this is off-topic for this issue. So would recommend following up on the pyodide issue tracker for help getting that started. Thanks! 😄

cc @davidbrochart (for awareness)

The long-term approach is to have conda-forge support a Wasm platform, then we would use conda to build environments as we are used to, instead of having a monolithic "Python + shipped libraries".
CPython can now "officially" be build for Wasm, and we have started working on the conda-forge wasm target. It's going to take some time, but we'll get there!

Huzzah, looks like a bunch of the examples Just Work :tm:  in jupyterlite today:
import piplite
await piplite.install('dask[array,dataframe,diagnostics]')
import dask.array as da

x = da.random.random((1000, 1000), chunks=(100, 100))
x.compute()

And the demo site even has ipycytoscape:
import dask
df = dask.datasets.timeseries()
df.visualize()

Seems like some hot examples in the docs are more than possible now, akin to what numpy, pandas, sympy and ipywidgets are doing.

Sounds like this more-or-less works. Going to close. Please feel free to raise anything else as a new issue. Thanks all! 🙏