Echo Server Epic

Epic for initial launch of Echo Server, at this point it will have the same features as the old Push Server. At this point the code will be placed onto a branch as used for Coursework. Then the entire team can work on the project to improve with the additional features from the doc.
Required

[x] Register Clients
[x] Unregister Clients
[ ] Accept Webhooks
[ ] CI/CD Pipeline

Nice to Have

[x] Working telemetry
[x] Link telemetry to opentelemetry

Milestones suggestion:

sample instance of echo server hosted for walletconnect e.g. echo.walletconnect.com
Kotlin/Swift SDKs integrate it into their sample dapps and integration tests

Milestones have been discussed internally and will be posted in slack.


Ephys example scripts

Add examples scripts and generated JSON files for ephys_session and ephys_rig

@jsiegle almost there - now you need to regenerate your .json files.

Hmm...I regenerated but it's still failing. And the expected vs. actual JSON strings look the same. Can you tell what the difference is?

@jsiegle It's a couple things.

the datetimes don't match because you are using datetime.now().  Can you fix the time like this? https://github.com/AllenNeuralDynamics/aind-data-schema/blob/83de3c9b189e7a5fd425af385bf79b3287ddb74b/examples/subject.py#L6
subject ID should be a string, not an integer.  I'm not sure why that's rendering out as an integer locally and a string in the CI.

Thanks, it's passing now!


Docs: ES6 Installation

The docs (http://reactivex.io/rxjs/manual/installation.html) say to install rxjs-es and import rxjs. This doesn't work as the import needs to match the npm module name. Github suggests installing rxjs. Which is correct?
Also, it might be worth clarifying this bit:

To import what you need and use it with proposed bind operator:

Does that apply if you import the full set of functionality or only if you import what you need?
Happy to PR this when I get time once I have clarification on the above.

The latest document actually already reflects those changes (to not to use rxjs-es) https://github.com/ReactiveX/rxjs#installation-and-usage , but it's not yet published to website unfortunately. We need to update it, just taking some time.

Note, this is still an issue on the website, which also uses the @reactivex namespace (e.g. import {Observable} from<EMAIL_ADDRESS>here).  It's pretty difficult as a new user to find the "correct" way to install and use this library.
I suggest that this issue be reopened due to:

it's not yet published to website unfortunately. We need to update it, just taking some time.

The issue is about the website, and it seems that it isn't fixed yet.


Only floating capsule ?

Hello,
Thanks a lot for this great project, I am sure it will have a big impact in the future of web 3D.
Do you plan to upgrade the character control to any type of colliders ? I mean for the moment it works for floating capsule but what about a car ? That is just an example. Maybe it is already possible to convert the capsule to other collider but I didn't find a way to do it.
Have a great day !

😁 Thanks @gdamou!
The capsule collider is essential for applying controls, but you can add colliders on top of the capsule within the character component. Here's an example: <Ecctrl><CuboidCollider args={[1,0.2,1]} mass={0}/></Ecctrl>
However, if you change the shape and mass of the character, you may need to adjust the springK and dampingC parameters to find the perfect values for floating and autobalancing.


[drenv] Notes on using drenv to run a ramen e2e test case for regional DR with VolumeReplication

The following notes are provided to enable further automation in the env scripts to make it easier to test with a ramen bundle install. Some steps are also common and needed for make deploy based ramen installs.
Intention of this issue would be to convert each of the steps that need to be automated to an issue, and track it here, changing the instructions as they are fixed to reflect a series of steps to take to test using drenv.

Created a drenv using the regional-dr.yaml environment

Installed ramen on the hub using the upstream bundles

Configure ramen

drenv creates a minio instance, the secret and end point to access this s3 end point needs to be further configured once Ramen is installed

Create a secret in the same namespace as ramen, with the s3 secret data in it (example)

Ramen config map needs to be updated with the minio endpoint and reference the above secret

For example
OR, append per-cluster the following s3 details to the ramen-hub-operator-configs3StoreProfiles:
- s3ProfileName: minio-on-<cluster_name>
s3Bucket: bucket
s3CompatibleEndpoint: <minikube minio url on <cluster_name>> # minikube --profile <cluster_name> -n minio service --url minio
s3Region: <fake value>
s3SecretRef:
	name: s3secret
	namespace: ramen-system

Create base ramen resources, example

Add upstream CatalogSource to managed clusters, example

As creation of DRCluster would trigger auto install of ramen on the dr-cluster, which does not rollout the catalog source onto the managed cluster but expects the same to be present, for auto rollout to succeed.

Install missing CRDs on managed clusters (volsync and velero CRDs)
Create DRCluster(s)
Create a DRPolicy

Simple test:

Deploy a DR protected workload

Create the Channel and namespace resource from samples
Create the workload subscription for the sample busybox application from samples

Modify DRPC spec.preferredCluster to dr1|dr2 and then kubectl apply -k subscriptions/busybox

Wait/monitor for the presence/health of the busybox pod in the busybox-sample namespace on the preferredCluster

Failover

Edit DRPC spec.action to Failover, and set spec.failoverCluster to the peer cluster value (dr1|dr2, based on what the preferredCluster was set to)
Wait/monitor for the presence/health of the busybox pod on the failoverCluster, and also ensure the pods are cleaned up on the preferredCluster

Relocate

Edit DRPC spec.action to Relocate
Wait/monitor for the presence/health of the busybox pod in the busybox-sample namespace on the preferredCluster, and ensure failoverCluster is cleaned up of all resources for the same

Undeploy

kubectl delete -k subscriptions/busybox and ensure all resources in the busybox-sampel namespace on the hub and preferredCluster are cleaned up

@ShyamsundarR should we create the secrets as part of the minio deployment, or
this is ramen specifc configuration?
If we have ramen specific configuration, shouldn't it be part of make deploy-*?
If this is kind of one time configuration that need to be done once after creating
the environemnt, we likey need to add a new ramen component preparing the environemnt
for deploying ramen.
My assumption is that deploying ramen is something you would do multiple times with
the same enviroment, for example:

build test env
repeat:

hack ramen code ...
deploy ramen
test ...

@ShyamsundarR should we create the secrets as part of the minio deployment, or this is ramen specifc configuration?

It is ramen specific configuration

If we have ramen specific configuration, shouldn't it be part of make deploy-*?

It is configuration, not part of deployment. IOW, this is post install configuration that needs to be done and is specific to each cluster where ramen is installed in. For example, here we ask for the s3 store details that Ramen should be interacting with, which is not part of the deployment.

If this is kind of one time configuration that need to be done once after creating the environemnt, we likey need to add a new ramen component preparing the environemnt for deploying ramen.

For a given drenv, this is a one-time configuration post install of ramen.

My assumption is that deploying ramen is something you would do multiple times with the same enviroment, for example:
* build test env

* repeat:

  * hack ramen code ...
  * deploy ramen

If we are only changing the ramen operator image (which is >95% of the time) to test and validate it again, we do not need to rebuild the env or re-configure the env. We would just replace the operator image and re-run the test case.
If testing ramen install or items related to auto-deploy of ramen bundles and secrets to dr clusters, we would recreate drenv fully (or uninstall/reinstall just ramen bundles and componsnts).
In both cases configuration automation from drenv for ramen is helpful and needed as it avoids a lot of manual steps, that are env specific.

  * test ...

Based @ShyamsundarR commemnt, we need to have 4 steps:

build ramen image/bundle that work with developer/CI test env
push image/bundle to developer/CI repo or directly to cluster
deploy ramen in test env
configure ramen for using s3 in the test env

Developer should be able to run all of them automatically for testing changes
during edit-test cycle:
make drenv

Developer should be able to run separate steps if needed:
make drenv-image-build
make drenv-image-push
make drenv-ramen-deploy
make drenv-ramen-configure

The makefile will use environment variables with the names of the clusters:
HUB_CLUSTER=hub
MANAGED_CLUSTERS="dr1 dr2"

A developer can modify the makefile defaults to test a different environment, but out
of the box, running make drenv will do the right thing.
Example developer workflow:
cd test
source ~/.venv/ramen/bin/activate
drenv start regional-dr.yaml

In antoher shell, or via editor console:
while true; do
    edit ...
    make drenv
    test ...

@ShyamsundarR what do you think?

@nirs I think we should not use make targets for this purpose, as the work to be done is more expansive. This should be part of drenv (unless you were referring to the drenv Makefile for the targets) as that has the required environment details and can construct the required resources for the same.
The intention is to install and configure ramen in a given dr environment, and towards this I see the following commands (command structure maybe different, but intent is described here):

drenv deploy sources

Ensure bundle based deploy is not active
Run from source directory/branch that needs to be deployed
make docker-build
load built image to cluster
make deploy-hub

Set ramen config to not auto deploy cluster, as we are not using bundles

make deploy-dr-cluster

drenv deploy bundle

Ensure source based deployment is not active
Deploy catalog source to all clusters
Deploy ramen hub bundle
Ability to use a user defined catalog source should be available, instead of the upstream catalog, which would be the default
(optional) build and push image/bundle/catalog before rolling out the same
Note: bundle deploy will not be done again, it would be done once and techincally the bundle version would be incremented to update the cluster. In our case we could uninstall the bundle and then reinstall it again

drenv configure ramen

Optional command, can be folded with one of the above commands
Create s3 secret in ramen namespace on all clusters
Update ramen config map with s3 access points on all clusters
Create DRCluster resource for each managed cluster

Create them as the same region or different regions based on the env (sync/async)

Create a DRPolicy resource at the hub

Default to 1 minute schedule interval (override with --schedule-interval option at the CLI) if env is async
Default to sync in case env is sync

(optional) Create the Channel resource for ocm-ramen-samples for any further testing

drenv undeploy

Undeploy ramen, however it was deployed. Useful when testing bundle based installs to refresh with a new built bundle in the next run

With the possible flow as above, a dev would do the following:

Create a drenv for testing
Edit...
drenv deploy sources
test
Repeat (2/3/4) as needed

A dev interested in testing using bundles would replace 3 with drenv deploy bundle. Assuming a catalog and bundle override is in use to push these images to and deploy them from a user specified catalog source.
A e2e CI run would potentially use drenv deploy bundle with per build bundle catalog and images, and then run the automated e2e tests for the flavor of the environment (regional/metro/recipe/etc.)
A post merge CI run (or release run) would invoke the same as the e2e, except with the published ramen catalog and bundles.

Added some potential test case flow in e2e here. The CI run may start configuring and then running the tests, than starting at a point where it is configured BUT, this may come later than other tests that expect a configured setup, and may need consideration appropriately.


Collisions are ignored for nested models connected to parent models with fixed joint

Environment

OS: Ubuntu
Source: Built from latest bazel branches

Description
A common model composition paradigm used in our org is to model robot arms and end-effectors separately and connect them together with fixed joints, as such:
<model name = "parent_model">
    <link name="parent_link">...</link>
    <include><uri>model://child</uri> <name> child_model </name></include>
    <joint name="parent_joint">
        <parent> parent_model </parent>
        <child> parent_model </child>
    </joint>
</model>

However, it seems like collisions in the child model are being ignored in dartsim since the update to Ignition Edifice. I have attached a simple sdf (test_cylinders.txt) with two cylinder models that shows this issue. cylinder1 on the left is defined with a nested model structure whereas cylinder2 on the right is defined with a flat model structure. When simulated, cylinder2 behaves correctly and falls on to the ground plane whereas cylinder1 simply falls through.
Steps to reproduce
blaze run -c opt ign_tools:ign -- gazebo -r -v 4 test_cylinders.txt

Output
https://user-images.githubusercontent.com/2412842/122488705-e17a5e00-cf92-11eb-9257-2a1e3e251d69.mov
test_cylinders.txt

I believe this is an issue in ign-gazebo. The problem stems from the recent performance improvements we've been making to ign-gazbebo. In https://github.com/ignitionrobotics/ign-gazebo/pull/678, we iterate through only the links whose absolute poses have changed. Consider a scenario where the canonical link has a change in pose but the other links don't. What will happen is that we update the pose of the model that is associated with the canonical link, as well as the pose of the canonical link relative to the model. However we skip the links whose absolute poses haven't changed, but since the components::Pose component in ign-gazebo is always relative to the parent model, it looks like these have moved even though their absolute poses have not changed.
Here's a simplified version of test_cylinders from the issue description that demonstrates the problem.
test_cylinders_simplified.sdf.txt
Here base_link does not have a collision shape so it will fall indefinitely. link0_outer on the other hand does have a collision shape and it should fall until it hits the ground plane and then stop.
Here's what it looks like when I run it in ign-gazebo 5.0.0 and 4.9.1.
https://user-images.githubusercontent.com/206116/122786771-475c3380-d27a-11eb-96a7-364f03bad2a5.mp4
And this is in ign-gazebo 3.8.0
https://user-images.githubusercontent.com/206116/122786817-517e3200-d27a-11eb-8eb6-29d0595bec5b.mp4
/cc @adlarkin

Looks like temporary fixing the issue I described above doesn't resolve the original issue with nested models raised by OP.

Thanks for looking into this @azeey.
I filed a related issue regarding pose updates for nested models here:
https://github.com/ignitionrobotics/ign-gazebo/issues/871
which I am guessing is caused by the optimizations you mentioned above.

This should be partially fixed by #268.


Topic/small fixes

A few changes regarding formatting and content in documentation and doctests.

Codecov Report

Merging #21 into master will decrease coverage by 0.17%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master      #21      +/-   ##
==========================================
- Coverage   54.21%   54.03%   -0.18%
==========================================
  Files           7        7
  Lines        1826     1832       +6
==========================================
  Hits          990      990
- Misses        836      842       +6

Impacted Files
Coverage Δ

src/types.rs
21.96% <0%> (-0.16%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 28200d8...131df83. Read the comment docs.

@na2021, it looks like we now have a conflict introduced by #22. Can you please resolve (i.e. merge my master into this branch, resolve the conflicts, and then push the changes)?


WEAPON DETECTION USING PYTHON

Is your feature request related to a problem? Please describe.
The problem is the lack of an efficient and accurate system for detecting weapons in images or videos, which is a critical task in various applications such as security surveillance, law enforcement, and border control.
Describe the solution you'd like
A weapon detection system using Python that can accurately identify and classify weapons in images or videos. The system should be able to detect various types of weapons, including handguns, rifles, knives, and explosives, and provide a confidence score for each detection.
Describe alternatives you've considered
Using pre-trained object detection models such as YOLO, SSD, or Faster R-CNN, which can be fine-tuned for weapon detection
Implementing a custom convolutional neural network (CNN) architecture for weapon detection
Utilizing transfer learning from existing weapon detection models
Using traditional computer vision techniques such as edge detection, shape analysis, and texture analysis
Approach to be followed (optional)
Data Collection: Collecting a large dataset of images or videos containing weapons and non-weapons
Data Preprocessing: Preprocessing the dataset by resizing images, normalizing pixel values, and data augmentation
Model Selection: Selecting a suitable object detection model or architecture and fine-tuning it for weapon detection
Model Training: Training the model using the preprocessed dataset and evaluating its performance using metrics such as precision, recall, and F1-score
Model Deployment: Deploying the trained model in a Python application for real-time weapon detection
Additional context
The system should be able to detect weapons in various scenarios, including:
Different lighting conditions
Various angles and orientations
Occluded or partially hidden weapons
Weapons in complex backgrounds

I'd like to request ownership of this project. I plan to use YOLO or SSD. I will work on both and see which one works best to differentiate between people with guns, knives, normal people, and pistols. I plan to merge some existing datasets to include guns, explosives, rifles, and knives in dataset. My approach involves data preprocessing, model training, analysing results and building an inference pipeline for real-time detection. Please assign it to me.
Thank you

I would like to work on this, please assign this to me

I am interested in this issue. Please assign me this


3D Buildings Artefacts in new version

I use ArchLinux, Firefox,  double GPU, and I see this:

I thing it  depends on the lightnings, but how I may fix it?

There's another isue on this problem too: #3886
It seems to be linux specific bug in a new version.
I'm also experiencing the same issue on Kubuntu 19.04, Chrome 78.0.3904.97 and Firefox 70.0.1
The worst thing is that this behaviour is also present on homepage of https://deck.gl and causes browser to almost freeze (1 - 3 FPS on live demo there), so I was having trouble navigating there.

This looks like position attributes (or indices) being read as 0. Are either of you getting any errors in the console?

This looks like position attributes (or indices) being read as 0. Are either of you getting any errors in the console?

Nope.


updated setup.cfg to include the missing deps (pytz)

trying to create a python package, the build fails due to the missing library pytz

the fix is no more valid (and no more needed).
I'm closing the pull request. Thanks for the fix.


Updating tests to use dapr/dapr RC6

Description
Updating tests to use dapr/dapr RC6
Please reference the issue this PR will close: #[issue number]
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[x] The quickstart code compiles correctly
[x] You've tested new builds of the quickstart if you changed quickstart code
[x] You've updated the quickstart's README if necessary
[x] If you have changed the steps for a quickstart be sure that you have updated the automated validation accordingly. All of our quickstarts have annotations that allow them to be executed automatically as code. For more information see mechanical-markdown. For user guide with examples see Examples.

Running checks.  then we should merge.

Ready to merge.


Please rename the repository and remove binary files

To be able to package pyocd for distributions, the tarball produced by python3 setup.py sdist needs significant mangling to remove the binary files which have no source and to rename the entire directory to lowercase only.
Please can the pyOCD directory be renamed to lowercase?
Please can the binaries and elf files not be included in the release?

Hi @codehelp. Thanks for raising these issues. And my apologies in advance, I know nearly nothing about Debian packages.
Why can't binaries be included in the package? The binaries are required for the automated functional tests (test/ directory). How do other Python packages that have binaries such as executable or shared libraries within the Python package deal with this?
Renaming the main pyOCD package to all lowercase is already included for discussion in the list of things to possibly fix for cleaning up the API naming conventions. But it would be a big breaking change, and no decision has been made yet. (I'm in favour of it. Good to hear this feedback from the community.)
But I'm curious why an all lowercase name is required when packaging? And how that affects users since code would not be compatible with pyOCD installed via pip, etc.
cc @c1728p9 @sg-

The most important problem, for me, with the binaries in the git source tree is simply this: Why should I trust these binaries? Just because they exist in a git source tree tells me nothing about how safe it is to run those binaries on my system. I have no reason to trust random binaries on github - indeed I have every reason to not trust them simply because it is github - the wild west of source code.
All the files in a binary Debian package in the main archive must be completely built solely from the source code of that package (without going off to the internet to get mystery blobs). Currently, the presence of the binaries means that I have to build packages just for Linaro and I can't use the normal Debian infrastructure to build and distribute the package. The current pyocd package could go into "non-free" in Debian but that isn't a good place to be, packages are not automatically built on upload, some users don't enable it and it doesn't reflect the actual licensing of the pyocd source code.
I'm not aware of other Python packages which have prebuilt binaries within the source code. Normally, a python package which is a binding for a shared library will simply depend on the shared library although pypi is unable to cope with such things. The shared library would already exist, built from it's source code, as a separate source package.
Alternatively, the source package providing the shared library can also provide a python binding, as Guest FS does: https://tracker.debian.org/pkg/libguestfs - python3-guestfs is built from the same source code that builds the libguestfs0 shared library. (Note that python packages which depend on the python bindings from GuestFS also cannot use pypi because pypi has no understanding of non-pypi dependencies. Not all python code can live in pypi.)
Equally, it's not good for contributors to pyocd to come across mysterious binaries in a git source tree. Git isn't efficient at handling binaries, contributors know nothing about the providence of those binaries and there are risks to running binaries for which the source code is not available. pyocd isn't just about developer boxes and developer machines. People are trying to use pyocd in a production environment - admins get very upset with binary files of unknown providence causing an unknowable risk. Does pyocd even check the integrity of the binaries before glibly using them? Just because they exist in a particular filesystem path is no guide to the security or integrity of these mysterious blobs. There is a security hole worthy of a CVE right there, ready and waiting.
There are other ways of running automated tests - as it is the pyocd packages will not run the current tests anyway. I simply cannot trust the binaries in the git tree. A better solution would be to provide these in a Docker with clear traceability of how the original files were built (by providing logs of the build of the binaries, e.g. inside another docker) so that the community can have some assurance of what is intended to be in those binaries. Publish sha256sums of the built files, show that those checksums are checked when the binaries are installed into their own docker and again when the binaries are used by pyocd - everytime they are used by pyocd.
If pyocd wants to retain these binaries, I would suggest:

Build the binaries in a docker image and publish the full build logs as well as the docker image, for reproducibility.
During the build, create sha256sum (MD5sum optionally too) checksums of all binaries which are to be used later. Publish those checksums alongside the docker image.
Cryptographically sign the checksum files (just as distributions sign their checksum files on the mirrors). Publish the fingerprint of the key so that the community can verify the signatures themselves. Otherwise, the checksum files are no more trustworthy than the existing binaries.
Build a second docker image which pulls in these binaries and checks the checksums and verifies the cryptographic signature on the checksum files. This docker is then used for the automated tests and contains at least a README, preferably the complete build log, with links back to the first docker image and instructions on how to rebuild that first docker and how to verify the binaries.
Every time pyocd accesses these binary files, it must verify the files against those checksums and fail noisily if anything is wrong. (Just as admins can verify the files provided from packages by checksums in case of a security intrusion.)
pyocd gets renamed and added to Debian with no binaries in github but with documentation on how to run the automated tests using docker.

As for renaming the binary, this is simply Debian Policy and there is no prospect of changing this for a little package like pyocd. Other packages all use lowercase, even big projects like PostgreSQL and GuestFS. This isn't just Debian, other distributions have similar policies.
Overall, it would be a big change but it is a change which helps community usage of pyocd. For now, pyocd is a pain to handle because I have to both remove the security risk binaries and I have to rename the files in the tarball produced by python3 setup.py sdist or downloaded from a git tag. I then have to build it manually and provide infrastructure to host the binaries outside Debian. That all adds delays to the process of testing and deploying new versions of pyocd in production labs to do CI on IoT devices.
Renaming the directory doesn't affect users that badly, it's not as if pyocd gets imported into other python libraries or scripts - most users simply call the scripts (which are already lowercased) like /usr/bin/pyocd-flashtool.
(I realise now this would have been easier if I'd filed two separate issues).

Please keep in mind that these are firmware binaries that run on a microcontroller dev board, not on a host system. It's a very different context, and there is little to no value from any attack. However, I understand the concerns with binaries that have no provenance. We will discuss internally what mitigations may be possible. (I'll also digest your suggestions more thoroughly once we get past demands related to TechCon.) Thanks.

Btw, you realize that the flash algos for each of the target MCUs (under pyOCD.target.*) are another set of binaries? They happen to be stored as int lists in the Python code, but could just as easily be .bin files accessed as package resources. Same threat scope as the test binaries. But the flash algos are required for pyOCD to be able to perform one of its key functions, programming target flash.
In the near-term, we will be moving to using CMSIS-Packs as the primary target support vehicle. CMSIS-Packs contain flash algos in .FLM files, which are simply binary ELF files with a different extension. Users will download Packs using one of the various Pack Manager tools (Keil, IAR, GNU MCU Eclipse, etc), then pyOCD will read the .FLM file(s) from the Pack.

The flash algos in target/ like target_CC3220SF.py are fine as source code - these are target-specific references and addresses which only make sense as static data. The data is presented in a form which allows modification and that is what matters. These are included in the package. The files in binaries/ is not included in the package, neither are the files in elf_files.
I understand about the firmware binaries being for separate boards, the difference in context is familiar from issues I have with scripts destined to run on a DUT but packaged in LAVA.
I've tried experimenting with simple changes to MANIFEST.in but setuptools insists on always including binaries/ and elf_files/ into the tarball coming from python3 setup.py sdist. Whilst it is acceptable (for distribution & packaging) to remove these binaries from the tarball of the pyocd release, I do see that it would not make sense to remove these entirely or put these into a separate git repo for "pyocd-data" as that would be too much of a barrier for running the pyocd tests upstream. As it is, the pyocd tests won't be able to run on the packaged pyocd. That isn't a problem, it's just inevitable in how pyocd needs to operate. I can set the Debian CI to add the upstream git repo to the packaged code and then run the tests.
So, I'll repackage each released tarball to drop binaries/ and elf_files/ directories, that's an extra step in the packaging but is do-able. I'll drop the request to remove those directories from the released tarball.
It would still be nice to rename the pyOCD directory to pyocd at some point. I'll rename the description of this issue.

Closing, as #433 renamed the pyOCD package to pyocd.


padding inlay hints

We get padding hints in the hover now:
field foo
Type: bool
Offset: 5 bytes
Size: 1 byte (+2 padding)

But few people will actually check those.
I guess it could be handy to have inlay hints on fields with padding so devs are more aware of the wasted space.

I agree, this would be a nice use of inlay hints.

I guess one could extract the padding computation into a function and reuse in a VisitFieldDecl or similar, not sure what an IndirectFieldDecl is.

POC: https://github.com/Trass3r/llvm-project/commit/227d3ee3b3e7877eec7b0d79be60e0e0a6b512d2

In the meantime it's also useful to put the following into <workspace>/.clangd:
CompileFlags:
  Add: [-Wpadded, -Wno-error=padded]


Update MarkerClusterGroup.js

Use this._maxZoom instead map.getMaxZoom() in _zoomOrSpiderfy.
When we use disableClusteringAtZoom in options this._maxZoom not equal map.options.maxZoom.

Hi!
You are right, it previously used map.getMaxZoom() === map.getZoom() and I just re-used map.getMaxZoom().
But since I merged the cases when all markers are at the exact same position and when we are at maxZoom, I accidentally removed spiderfication of markers at the exact same position when disableClusteringAtZoom is used with a lower zoom than map.options.maxZoom, as you point out.
For the case of markers at "slightly different positions", it could still be interesting to zoom rather than spiderfy, because the map has more zoom levels to separate out those markers. But they will not necessarily separate a lot!
So spiderfying with checking this._maxZoom rather than map.options.maxZoom sounds a reasonable choice, especially with the fact that application developer can still tune the behaviour with spiderfyOnMaxZoom option.

Cool :+1:


Fix a crash when no shortcodes are found

If no matches are found, output of .match() will be null (https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String/match). null is not iterable, so for .. of will throw an error. This pull request fixes the issue by checking if shortcodes is null and setting it to [] instead.

Can you give me an example string which causes this crash?

Can you give me an example string which causes this crash?

Here you are. Using 1.2.5 (latest version).

Thank you for your contribution!


Working with iOS7 and 8

Just included a UIView with black background and alpha 0.85 when iOS7 to get similar feeling as the iOS8 version.

Ops, saw that I forgot to remove redColor as the Views backgroundcolor. Should be blackColor...


A little confused as to how to handle Repository return type.

Larastan Version: 0.7.6
--level used: 5

Description
I have a User model which extends Illuminate\Database\Eloquent\Model with an associated UserRepository:
<?php

namespace App\Repositories;

use App\Models\User;

class UserRepository extends BaseRepository
{

    /**
     * User Repository constructor.
     *
     * @param User $model - User Model
     */
    public function __construct(User $model)
    {
        parent::__construct($model);
    }

    /**
     * @return User
     */
    public function whereEmailFirst($email): User
    {
        return $this->model->where('email', $email)->first();
    }
}

The BaseRepository just assigns the $model parameter to the $model property in the constructor.
Phpstan will throw an error at this:
Method App\Repositories\UserRepository::whereEmailFirst() should return App\Models\User but returns Illuminate\Database\Eloquent\Model.
I'm new to Generics, but i've read various articles etc and am still not understanding exactly how I should handle this situation.
I could add something like @phpstan-return Illuminate\Database\Eloquent\Model But this doesn't feel right, as this method will only ever return an Employee.
What would be the correct way of going about this please?

Hi,
I think you can solve this with the generics as follows:
BaseRepository:
/**
 * @template T of Model
 */
class BaseRepository
{
	/** @var T */
	public $model;
}

UserRepository:
/**
 * @extends BaseRepository<User>
 */
class UserRepository extends BaseRepository
{
}

Then PHPStan would know $this->model would refer to User in UserRepository

Wow, it works. Thanks...
Now all I need is to understand how/why.
I think the extends BaseRepository is what I don't fully grasp.
 Appreciate your fast response though. <3

No problem!
You can read these articles:
https://phpstan.org/blog/generics-in-php-using-phpdocs
https://arnaud.le-blanc.net/post/phpstan-generics.html
https://tomasvotruba.com/blog/2021/01/04/phpstan-abstract-parent-generics-dummies/
https://tomasvotruba.com/blog/phpstan-generics-for-dummies-with-two-parents/
I think especially the last two one will help you understand the @extends BaseRepository<User> syntax 👍🏽


Adjusting height for less than 3 items in the command palette

The height of the <div id="wrapper"> is always set to be 100vh. This doesn't work when there's only one or two available options in the palette since the box-shadow of the container still remains at the same height.

PR #6 changes this to property to max-height instead which can accomodate for such cases.
PS. @rohitpotato Sorry for the delay in responding, I look at Open Source projects on the weekends :P

@raghavkanwal You are correct but this decreases the height of the wrapper , I think this needs to affect the command palette div only.
The reason is if we decide to give a background blend (a tint for example), this height will affect that tint too.

I agree, this changes the height of the wrapper, which is the parent element.
Instead, we can add height: fit-content to the class paletteWrapper inside of #wrapper. That doesn't affect the parent element and still has the desired output.
I've created a new PR, #8 for this change.

Merged


non-immediate timer interrupts at voluntary-resched PPs leak 1 instruction each

prob. also visible in pintos where sometimes takes multiple ticks to trigger a switch. note bolded eips advancing:
[PP]            #97: Preempting TID 4 to switch to TID 3
[PP]            Current stack: TID4 at 0xc0022b0e in sema_up , 0xc0022e41 in lock_release , 0xc00306dd in sys_exit , 0xc00308ba in syscall_handler (syscall.c:0), 0xc0021af0 in intr_handler , 0xc0021da3 in intr_entry (interrupt.c:0), 0x0804a15c in
[LANDSLIDE]     dbg @ 0xc0021efc in intr20_stub
...
[LANDSLIDE]     dbg @ 0xc0021db0 in intr_exit
[LANDSLIDE]     dbg @ 0xc0022e41 in lock_release
[X86]           tick! (0xc0022e41)
[LANDSLIDE]     dbg @ 0xc0021efc in intr20_stub
...
[LANDSLIDE]     dbg @ 0xc0021db0 in intr_exit
[LANDSLIDE]     dbg @ 0xc0022e44 in lock_release
[X86]           tick! (0xc0022e44)
[LANDSLIDE]     dbg @ 0xc0021efc in intr20_stub
...
[LANDSLIDE]     dbg @ 0xc0021db0 in intr_exit
[LANDSLIDE]     dbg @ 0xc0022e45 in lock_release
[X86]           tick! (0xc0022e45)
[LANDSLIDE]     dbg @ 0xc0021efc in intr20_stub

sinister false positive use-after-free was reported where repeated leakage of 1 instruction at a VR PP caused one thread to start executing in free() before the PP could be placed, causing landslide to "lose" the fact that mem->in_free had been set.
thoughts in progress
Note, free():
0xc002398c == push %ebp
0xc002398d == mov %esp,%ebp
[SCHEDULE]      switched threads 1 -> 4

[SCHEDULE]      exiting context switch (TID 4)
[MEMORY]        WARNING: ignoring shm by wrong thread(?) (chosen TID 1; current TID 4) to 0x126c00 @ 0xc002398c in free
[MEMORY]        WARNING: ignoring shm by wrong thread(?) (chosen TID 1; current TID 4) to 0x10208c @ 0xc002398c in free
[ARBITER]       a voluntary reschedule: 1cv to 4
[ARBITER]       Available choices: 4 3tc - Figured I'd look at TID 4 next. // XXX1
[PP]            dbg: checkpting kern_mem->in_free = 0 to mem image 0x5e322f0
[PP]            #117: Voluntary reschedule from TID 1 to TID 4 // XXX2
[MEMORY]        Free() chunk 0xc01160cc, in kernel
[MEMORY]        dbg: TID 4 in_free = true (free called from 0xc002f15d in process_exit <unknown assembly>) // XXX3

........
[PP]            #119: Test ends with TID 3.
[EXPLORE]       from #119/tid3, chose tid 3, child of #117/tid1
[PP]            88 MPH: eip 0xc0020189 -> 0xc002398c
[PP]            dbg: restore kern_mem->in_free = 0 (old value 0) (restored from 0x5e322f0
[PP]            ls restored; think h->eip is 0xc002398c but real eip is actually 0xc0020189

[cpu0] cs:0xc002398d p:0x0002398d  mov ebp,esp
[LANDSLIDE]     ls restored; think h->eip *was* 0xc0020189 but real eip is actually 0xc002398d
simics-common: Assertion `ls->kern_mem.in_free' failed.

Note key sequence of events
// FIXME: IDGI. Why doesn't VR get recognized when context_switch "ret" is "about" to get executed?

sched_update() -> recognizes VR just ended when tid4 is about to execute 0xc002398c.
sched_update() -> save_setjmp() -> snapshots mem->in_free = false.
sched_update() -> save_setjmp() -> does simics run command
mem_update() sees 0xc002398c and sets mem->in_free = true;
Instruction finishes, push ebp is executed
Simics run command is run; booqmarkq is set on actual eip 0xc002398d.

Later rewind to 0xc002398d and push ebp is never executed after that, causing mem->in_free to not get set.

FIXME: IDGI. Why doesn't VR get recognized when context_switch "ret" is "about" to get executed?

because tid 4 is in a timer interrupt and interrupts don't come back on until after iret.


chore: Load <1 Week Events, Load More Events btn

thanks cameri! I'm going to close this since my latest update implements something similar (and a lot of the code you updated is obsolete now)


Force to load select2 js first

Fix the ManyToMany field when horizontal=True by making sure the SelectFilter2 is loaded first.

Coverage increased (+0.02%) to 89.831% when pulling b497d652244dd0e4935a16e37a0706f4fda0a94f on leibowitz:fix_bindfields_order into ca55e0bc8ad378dc7a3d1bb5e68e18e8fc51c546 on digi604:master.

@leibowitz this makes sense, merging.

This PR was made when django just released version 2.1. At the time I was still using django 1.11.
I just tested on the latest django (3.0.6), and this change seems to break the with ManyToMany and horizontal=True as it's not loading the data anymore. This seems to be because the .filtered class it not being searched for in bindfields.js
Fix => https://github.com/jazzband/django-smart-selects/pull/298
The remaining issue I can see is horizontal=True is throwing an error (both with or without these two changes, so not related) which I will try to fix as well in another PR
https://github.com/jazzband/django-smart-selects/issues/299


indices should be either on cpu or on the same device as the indexed tensor (cpu)

It happens when I run the intro.ipynb ,
results = searcher.search(query, k=3)

the output is
RuntimeError                              Traceback (most recent call last)
Cell In[19], line 6
      3 print(f"#> {query}")
      5 # Find the top-3 passages for this query
----> 6 results = searcher.search(query, k=3)
      8 # Print out the top-k retrieved passages
      9 for passage_id, passage_rank, passage_score in zip(*results):

File ~/ColBERT/docs/../colbert/searcher.py:61, in Searcher.search(self, text, k, filter_fn)
     59 def search(self, text: str, k=10, filter_fn=None):
     60     Q = self.encode(text)
---> 61     return self.dense_search(Q, k, filter_fn=filter_fn)

File ~/ColBERT/docs/../colbert/searcher.py:108, in Searcher.dense_search(self, Q, k, filter_fn)
    105     if self.config.ndocs is None:
    106         self.configure(ndocs=max(k * 4, 4096))
--> 108 pids, scores = self.ranker.rank(self.config, Q, filter_fn=filter_fn)
    110 return pids[:k], list(range(1, k+1)), scores[:k]

File ~/ColBERT/docs/../colbert/search/index_storage.py:79, in IndexScorer.rank(self, config, Q, filter_fn)
     77 def rank(self, config, Q, filter_fn=None):
     78     with torch.inference_mode():
---> 79         pids, centroid_scores = self.retrieve(config, Q)
     81         if filter_fn is not None:
     82             pids = filter_fn(pids)

File ~/ColBERT/docs/../colbert/search/index_storage.py:69, in IndexScorer.retrieve(self, config, Q)
     67 def retrieve(self, config, Q):
     68     Q = Q[:, :config.query_maxlen]   # NOTE: Candidate generation uses only the query tokens
---> 69     embedding_ids, centroid_scores = self.generate_candidates(config, Q)
     71     return embedding_ids, centroid_scores
File ~/ColBERT/docs/../colbert/search/candidate_generation.py:55, in CandidateGeneration.generate_candidates(self, config, Q)
     52     Q = Q.cuda().half()
     53 assert Q.dim() == 2
---> 55 pids, centroid_scores = self.generate_candidate_pids(Q, ncells)
     57 sorter = pids.sort()
     58 pids = sorter.values

File ~/ColBERT/docs/../colbert/search/candidate_generation.py:34, in CandidateGeneration.generate_candidate_pids(self, Q, ncells)
     31 def generate_candidate_pids(self, Q, ncells):
     32     cells, scores = self.get_cells(Q, ncells)
---> 34     pids, cell_lengths = self.ivf.lookup(cells)
     35     if self.use_gpu:
     36         pids = pids.cuda()

File ~/ColBERT/docs/../colbert/search/strided_tensor.py:75, in StridedTensor.lookup(self, pids, output)
     74 def lookup(self, pids, output='packed'):
---> 75     pids, lengths, offsets = self._prepare_lookup(pids)
     77     if self.use_gpu:
     78         stride = lengths.max().item()

File ~/ColBERT/docs/../colbert/search/strided_tensor.py:67, in StridedTensor._prepare_lookup(self, pids)
     65     pids = pids.cuda()
     66 pids = pids.long()
---> 67 lengths = self.lengths[pids]
     68 if self.use_gpu:
     69     lengths = lengths.cuda()

RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

seems like the device setting got mismatched and i overwrite the prepare_lookup func
def _prepare_lookup(self, pids):
        if isinstance(pids, list):
            pids = torch.tensor(pids)

        assert pids.dim() == 1

        if self.use_gpu:
            pids = pids.cuda()
        pids = pids.long()
        lengths = self.lengths[pids]
        if self.use_gpu:
            lengths = lengths.cuda()
        offsets = self.offsets[pids]

        return pids, lengths, offsets

another device mismatch happened

Looks like self.lengths, self.offsets, and self.views are meant to be in the CPU.
So a easier way to make sure devices are aligned is making pids to live in the CPU.
Here is my workaround and seems to be working
    def _prepare_lookup(self, pids):
        if isinstance(pids, list):
            pids = torch.tensor(pids)

        assert pids.dim() == 1

        pids = pids.long().cpu()
        lengths = self.lengths[pids]
        offsets = self.offsets[pids]

@eugene-yang Thanks for the PR! I worry this will have negative effects on latency, though.
Not sure why the original code isn't working for you both. I believe @santhnm2 is looking into this

I'm running on python 3.8 with pytorch==1.13.1. Hope this helps.
My understanding is that self.lengths is designed to stay in CPU so I'm more surprised that the original code works.
But if those are all meant to be in GPU if possible, then the original code makes sense but should move things into GPU at during init of StridedTensorCore.

Any updates? I am also facing the same issue.

I merged in @eugene-yang's fix for now, but I'm currently testing a fix that might give a slight performance improvement - will post an update here if this fix works.


Update Example state.md with Hooks

i updated only the example state to hooks!!

thanks rachel!!!


fixed issue #5 Show -gimme function and typo

added method gimme()
typo in applications/config/routes.php used double quote instead of single quote on zzz

gimme test comment

pull request for changing default controller from welcome to hogwarts


New ACE Kernels

As a first step towards retiring ACE1 entirely, I'm putting together a new evaluator (currently in a private repo). For now, it does the following:

evaluate radial basis via SVector splines
sphericart spherical harmonics
pooling via P4ML
StaticPolynomials for the n-correlations
import ACE1 potentials into the new format (correctly!)

This next issue is to incorporate this here, provide wrappers for both JuLIP and AtomsBase (Molly), and move then discuss how to move over the entire ecosystem, I'm collecting some todos:

[ ] switch ACE1 over to using splines for the radial basis so that fewer points can be used
[ ] also implement the entire basis sets with the new kernels
[ ] bring the purification to the new setting

The evaluator is ready to incorporate here. I'd like to get feedback on the following question: Should it continue to live in a separate repo, or should I move the code into ACEpotentials.jl with the idea that it will localize the development, maybe simplify maintenance going forward, and be less confusing for outside users?
CC @casv2 @wcwitt @tjjarvinen

I would be happy if we started to localize, but I also don't feel super strongly.

So it has the evaluator and now needs pairlist and atomic structure data input?
To aswer I would need to know a more details on what kind of input it takes etc.
For the start I would not pull it in ACEpotentials. I would first at least do some preliminary implementation for JuLIP and AtomsBase first before PR. Also, how does this stand with JuLIP retire plan?

The new evaluator doesn't need JuLIP but can get a JuLIP wrapper.

Structure and pairlisz can be anything. Needs a wrapper though.

Then ACEmd might be a better place for AtomsBase wrapper.
Does it calculate per atom inputs or does it take a group atoms at once?

at the moment it does per-atom. I want to start experimenting with groups of 16 or 32, ...

per-atom implementation is very easy just change the ACEmd backend functions https://github.com/ACEsuit/ACEmd.jl/blob/main/src/backend.jl and after that you will have fully working AtomsBase implementation.
For groups we need to change more things, but it should not be much.

The wrapper should be easy enough. it's the kernels that concern me.

We can start with per-atom implementation first in ACEmd and see after that how to proceed with group evaluations.

I followed your ACEmd multi-threaded implementation - very easy and very elegant.

So ... while easy and elegant, unfortunately the performance is a bit worse than the naive JuLIP implementation:
[ Info: Forces - Multithreaded - JuLIP
BenchmarkTools.Trial: 122 samples with 1 evaluation.
 Range (min … max):  32.745 ms … 52.820 ms  ┊ GC (min … max): 0.00% … 19.72%
 Time  (median):     41.009 ms              ┊ GC (median):    0.00%
 Time  (mean ± σ):   41.006 ms ±  4.265 ms  ┊ GC (mean ± σ):  1.54% ±  4.71%

         ▃ ▆ ▁  ▃▃ ▁ █   ▆ ▁ ▆▃▆ ▁▃  ▁▆▁▃▆       ▁
  ▄▇▄▁▁▁▄█▇█▇█▁▄██▇█▇█▇▇▄█▇█▄███▇██▁▄█████▇▄▇▁▄▄▄█▁▁▁▁▄▄▁▄▄▄▇ ▄
  32.7 ms         Histogram: frequency by time        50.6 ms <

 Memory estimate: 10.19 MiB, allocs estimate: 211.
[ Info: Forces - Single threaded - new
BenchmarkTools.Trial: 24 samples with 1 evaluation.
 Range (min … max):  207.507 ms … 219.585 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     216.502 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   214.771 ms ±   3.729 ms  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▁                                          ▁ ▄ █
  █▁▁▁▆▆▆▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆▁▁▁▁▁▁▆▆▆▁▁▁▁▁▁█▁█▁█▆▆▆▁▆▁▁▆▁▁▁▁▆ ▁
  208 ms           Histogram: frequency by time          220 ms <

 Memory estimate: 325.33 KiB, allocs estimate: 4.
[ Info: Forces - Multithreaded - new
BenchmarkTools.Trial: 78 samples with 1 evaluation.
 Range (min … max):  35.884 ms … 269.521 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     60.736 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   64.836 ms ±  29.829 ms  ┊ GC (mean ± σ):  0.68% ± 5.02%

   ▁ ▁ ▃▁▁▁ ▆     █     ▆  ▆▁
  ▇█▇█▄████▇█▁▄▇▁▄█▇▄▄▄▄█▇▇██▄▄▇▄▁▄▁▄▁▁▁▁▇▁▄▁▁▁▄▁▁▁▄▁▁▁▁▁▁▁▁▁▄ ▁
  35.9 ms         Histogram: frequency by time          125 ms <

 Memory estimate: 9.86 MiB, allocs estimate: 201.

The main target with the new multithreading implementation was to fix the multithreading bug. For energy evaluation this lead to a faster code, for forces the code is slower - as you found out.
There is an opportunity for improvements here and it has been on my todo list.
The lesson to lear hear is that, there is the reduction has a considerable effect on final perfomance and it should not be overlooked when we try to get the last bit of performance out.

There is quite a lot going on in AtomsBase, AtomsCalculators, and also in EquivariantModels right now. I will therefore keep UF_ACE separate for now, but as soon as things have settled down a bit I will move the new kernels in ACEpotentials.jl precisely for the purpose of localizing the development of mature / established models and help avoid confusion for users who aren't immersed in the Julia environment. Experimental models can still be kept in separate repos.

On the topic of new ACE kernels for the basis rather than the evaluator: If we only need the matrix-vector multiplication A * c then we can have this right now. This would limit us to using iterative methods such as LSQR. Do we think it is worthwhile putting this together - I would need maybe 1/2 day for it.
to get the entire design matrix efficiently requires implementation of jacobians. This could be a lot of additional work. I can't be sure until I think about it.

It's a bit unclear to me now what the role of ACE1 is anymore. The new implementation started in #195 look elegant, fast, and general. I am now thinking of simply having one minor version with both ACE1 and the new kernels to give some time for testing. Then if the new models are at least as efficient and effective we just kill ACE1 and move over entirely.

#196  supercedes this.


[DOCS] Contributing icons docs

Description
Add a file to the docs/contributing folder called contributing-icons.md
In the file give full instructions for how to fork and clone the project, add the icons to the following 3 files:
icons.md
GetIcons.js
links.json
Then add instuctions for how to commit and add the correct files and make a pull request
Screenshots
No response
Additional information
No response

Can I do it? Can I work on this issue?

I've been already working on it

@virtual-designer in the future it’s best to ask to be assigned before making a PR. It prevents 2 or more people working on the same task unnecessarily

Is this issue still open for contribution? thanks.

@asdsyd There's currently an open PR


Code for gcr.io/spark-operator/spark:v3.1.1

Can someone help me find the code for image - gcr.io/spark-operator/spark:v3.1.1?

@roligupt here you go

Thanks! you are pointing me to apache/spark
How do i know how really "gcr.io/spark-operator/spark:v3.1.1" image is built? I mean does it have hadoop binaries or just spark-3.1.1. your link doesnt really tell me anything.


DI Cleanups

The following PR performs some small maintenance/code improvements to DI. Mainly reducing indentation and renaming ivars that refer to release to now refer to destroys.
rdar://31521023

@swift-ci smoke test and merge


[5/n] Format trait v2

Part of #492 . Depends on #505 #507 #521 #524
Current status:

Code done
Snapshot tests pass
Missing updating unit tests

This is now ready to go! @japaric @jonas-schievink

bors r+

This is awesome! It's great to see the InternalFormatter go away. Also thanks for the macros to simplify the primitive implementations.
Something I noticed that doesn't need to be addressed in this PR:
I no longer see a point to the internp! vs intern! distinction because we no longer LEB encode the tag.
Originally, internp! was created to put "primitive" tags like =u8 at the beginning of the .defmt section so they get smaller values (<128). Because we LEB encoded those tags, primitive tags would be encoded as 1 byte on the wire. The rationale was that primitives are more likely to appear behind =? so they'll take less space on the wire but now we always encode the tag as 2 bytes.
So I think internp! and the .defmt.prim input section can probably go away.

Should we track this in an issue? Or note it in the RFC?

There is still some advantage of putting primitives first. Tags less than 256 have one zero byte. When rzCOBS-encoded, these will be 9 bits (8+1) instead of 16 bits.

When rzCOBS-encoded, these will be 9 bits (8+1) instead of 16 bits.

(I have not been following all PRs but) this is not being used in the code base at the moment, right? at least, that didn't seem to be the case from looking at the decoder tests. Would rzCOBS be applied not in the log macros but rather in the transport layer crate, e.g. defmt-rtt?

Yeah, it's not done yet, but coming next. I have a PoC here.
Yes, it's in the Logger impl. It has to be because impls that allow concurrent logging (eg one channel per irq prio level) need to have one rzcobs encoder state for each channel. If it was on the defmt side I don't think there's a way to do that...


README.md

What is wrong?
README.md's first example does not use the config var net = new brain.NeuralNetwork(>>>config<<<);
Where does it happen?
n/a
How do we replicate the issue?
n/a
How important is this (1-5)?
2
Expected behavior (i.e. solution)
Should be documented right, minor confusion for first timers.
Other Comments

Acknowledged, will be fixed soon.


git-node: add git-node-release

Early draft of an automated release flow to abstract away the tedious aspects of preparing and promoting releases.
I think i'd like to split the preparation and promotion aspects of this new flow into two separate PRs for easier review, so this will probably just end up being the prep step and skeleton code for the promotion step.
Opening now to get some early feedback on the high-level approach, but please mind that this is a WIP and thus not super DRY.
TODO:

[ ] Adding specs
[ ] Settling on flow for when the process is aborted midway through
[ ] Figuring out best approach to updating changelogs (it's a bit unwieldy to auto-handle)
[ ] Whatever feedback i get from y'all 😛

@nodejs/releasers

Thanks for working on this!

Just tried to use it and got an error
⠋ Fetching branch-diffTypeError: Cannot read property 'onlyNotableChanges' of undefined
    at ReleasePreparation.getBranchDiff (/Users/mborins/code/node-core-utils/lib/prepare_release.js:352:14)
    at main (/Users/mborins/code/node-core-utils/components/git/release.js:84:22)
    at release (/Users/mborins/code/node-core-utils/components/git/release.js:59:21)
    at Object.handler (/Users/mborins/code/node-core-utils/components/git/release.js:42:16)
    at Object.runCommand (/Users/mborins/code/node-core-utils/node_modules/yargs/lib/command.js:240:40)
    at Object.parseArgs [as _parseArgs] (/Users/mborins/code/node-core-utils/node_modules/yargs/yargs.js:1145:41)
    at Object.get [as argv] (/Users/mborins/code/node-core-utils/node_modules/yargs/yargs.js:1079:21)
    at Object.<anonymous> (/Users/mborins/code/node-core-utils/bin/git-node:20:3)
    at Module._compile (internal/modules/cjs/loader.js:1147:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:10)

Also... for git node release --prepare it would be nice to have a default that determined for you the type of release (patch / minor / major) rather than having to provide the explicit version number.

Some bugs I found playing with this

I provided the version as v13.11.0 and the branch it crearted was vv13.11.0 (and similar duplication across meta data. Should likely not do that 😅 (ignore v)
I accidentally said yes to changing module version number which resulted in reformatting the entire module version data and thus a lot of churn. Might want to make sure that generated module version number information has a consistent style. Honestly the question came as a surprise... and we should maybe only ask it for Semver-Major release (imho)

I also just accidentally clicked through on the "create PR" before I had a chance to audit stuff... would prefer to not have that step 😅

One other bit... while it is nice having the notable commit information in the commit message we likely don't want it as markdown

Automation is missing one of the link references at the top of the version specific changelog
https://github.com/nodejs/node/pull/32218

dogfooded again. Everything worked great until I attempted to push upstream and then it bailed with the following warning
Error: remote:
remote: Create a pull request for 'v13.12.0-proposal' on GitHub by visiting:
remote:      https://github.com/nodejs/node/pull/new/v13.12.0-proposal
remote:
To github.com:nodejs/node.git
 * [new branch]            v13.12.0-proposal -> v13.12.0-proposal

    at exports.runSync (/Users/mborins/code/node-core-utils/lib/run.js:60:11)
    at ReleasePreparation.pushBranch (/Users/mborins/code/node-core-utils/lib/prepare_release.js:247:5)
    at ReleasePreparation.prepare (/Users/mborins/code/node-core-utils/lib/prepare_release.js:160:10)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

wow, this is a great way to deal with release complexity, and it even edits abi_version_registry!
Minor suggestion is to add changelog-maker to the dependencies list, not strictly essential but since you're directly relying on the executable it it may be a good idea to lock the version to the code rather than whatever branch-diff pulls in.

Seems like the href for the prior release is getting replaced rather than appended in CHANGELOG_vX.x
https://github.com/nodejs/node/pull/32376#discussion_r395615448

trying again and the "auto detect" wasn't working for me... it kept trying to make a v14.x release and complaining that i was on v13.x (for a 13.x release)

Rubberstamp LGTM. I'll try it for the next release I prepare.


Lines style migration problem

We are migrating the stroke color of the line to the fill color.

FYI this doesn't happen for actual migrations: creating a map with the Editor and then opening it with the Builder is fine:

I'll review why does it happen at direct creation from dashboard though.

It Works In My Machine™. It works in local development, not in production. Stay tuned 👻

Ok, layersData in production doesn't contain style_properties<= style migration is not being run.

Ok, after a talk with @xavijam we know what's happening here.
TL;DR: "recent" (post-builder-dataset-view-era) datasets canonical visualizations haven't wizard_properties, so maps based on them haven't (default) style properties.
What happened before

You import a dataset.
Editor Dataset page is open and wizard_propertiesare initialized at canonical visualization.
You create a map with that dataset. The map duplicates dataset canonical visualization properties, which have wizard_properties.
The layers get style_properties from wizard_properties migration when you first present the layers for the Builder.

What happens now

You import a dataset.
Builder Dataset page hasn't anything related to styles, so styles are not initialized.
You create a map with that dataset. The map duplicates dataset canonical visualization properties, which haven't wizard_properties.

Solution alternatives:
a) Initialize styles at dashboard (front) on map creation. It's problematic because style initialization logic should be available not only at Builder but also at the dashboard.
b) Initialize styles at the Builder, when you load a visualization that hasn't style_properties. It's problematic because at the first load of a visualization it should initialize style if it hasn't, and make a PUT to sync them.
c) Initialize styles at the back, on map creation. It's problematic because we should code the logic that initializes based on default cartography and default cartocss in a similar fashion that front does, which is duplicated logic.
IMHO, the best tradeoff is b). @xavijam opinion is c).
cc @javisantana @xavijam @gfiorav

The c option should not be that problematic, it would be a simple one. In fact I guess we are doing something similar for initialize new created layers.

@xavijam yep, I agree. My point about c isn't being complicated but not the best because of "partial duplication" (most creation logic will always be un front, picking some of it at back seems arbitrary. But I don't really love c either, both b and c have tradeoffs.

I'd go with b).
IMO we need robustness over speed at this point in time (specially if it's a one time load). We can always to better later to trim a second of load time afterwards or redesign the styles stuff later on. It will take the same effort as doing it now (we're not making future compromises with b).

Take into account that, if we create a map of 10 layers, we will need 10 PUT requests.

Yeah, but that's the nature of creating a 10 layer map: you get 10x the load 😄
I mean, maybe I'm super off with this one but for me it's natural: not worth sacrificing our current robustness.
Besides, this is the type of thing that improves with technology. Everybody is trying to make 10 PUTs faster, but you can't expect them to solve your specific logic duplication issues. I feel it's more future proof to go with straightforward classic solution, unless it meas waaay to much load time (does it?).

More than slowness, our main concern is make PUTs without any user action behind, I feel it is ugly.

I see what you mean.
It is user action if you think of it. The user created a 10 layer map so here come 10 PUTs. I'm cool with any decision but let's really think if this is so bad.

but B) means to do a PUT when opening a visualization. In the past we had a lot of problems, opening a visualization should not do anything. Only user actions should trigger PUT/POST/DELETE actions
For me c) makes sense and it's not that hard, it's only a template https://github.com/CartoDB/cartodb/blob/bc358f7b1c24f81abff554a277e357916af48b77/lib/assets/javascripts/cartodb3/data/camshaft-reference.js#L122

Some comments:

Take into account that, if we create a map of 10 layers, we will need 10 PUT requests.

There aren't that many map creations with many tables at once, and even those shouldn't be a huge problem as it's only on first creation.

I mean, maybe I'm super off with this one but for me it's natural: not worth sacrificing our current robustness.

I think the key of b is not robustness but maintainability. In fact, if robustness were the key I'd do 100% of the map initialization at backend, as it's a more controlled environment than a browser.

More than slowness, our main concern is make PUTs without any user action behind, I feel it is ugly.

Certainly, that's true. It would be better on pure creation (either at back or front), not first initialization.

For me c) makes sense and it's not that hard

Yes, it's not a matter of hardness but duplication.
I'm not too convinced of any option. As I just had an small preference for b over c, IMHO, let's do c, but considering moving more initialization to backend to avoid duplication of concerns. I'm just guessing but, for example, if front assigns a default CartoCSS on empty layer creation, that could be also deferred to back: if a layer creation contains CartoCSS backend should assume that it's not a default one so it should leave it as it is, but if it doesn't contain any, backend should initialize it. This way the initialization concern is only at back. Again, this is only an example, I hope that you get my point. How does it sound?

Agreed offline:
We will port some of the logic for setting default styles based on geometry to the backend. It should be noted that we still think there's no future separating responsibilities, but let's disagree and commit!
Some notes:

Style form definitions (JSON) changes should be notified to back people
Changes in style logic (we know use border radius or whatever) need to be maintained in back as well (fuel for our disagreement)

We all agree there's no beautiful solution that fits our schedule so we're cool with it :)


Increase allowed heap to fix stack overflow

The current windows tensorflow.dll is 769M;  Node by default has a 1.5GB.  The build is doing a join() of all input things in an array, so there is 1 copy and a new copy in the new buffer that is 769M + Other files; this is more than 1.5GB.  I tried setting just --max-old-size=2048 and it still overflowed... I didn't spend a lot of time trying to tune a minimum boundary.
I ran into this issue installing tfjs-node-gpu, which I assume is something of a fork of this for publishing?

This change is

This deserves a deeper look, and is probably not sufficient itself.

I don't see an issue with that; it's only what can maximum be used, it doesn't change the amount used.
Though there is another call to expand that downloaded zip that failed, I'm not sure if this fixes that; RangeError: Array buffer allocation failed  I guess that was a different error from \adm-zip\adm-zip.js


Feature/do not overwrite original select

Overwriting the original select and hacking it back makes using Reactive Programming libraries difficult (Rx.js, Bacon.js).
Better way is to preserve the original select as hidden and triggering a change event when the item is selected.

Great suggestion, thanks!


Feature request: introduce setting to allow vertical scroll from a child scroll to affect parent

Hi,
in issue #1462 there was a feature request to allow "bubbling" of scroll events from a child container to its parent container. Let's look at an example:
<ion-content>
  Here be lots of text
  <ion-scroll direction="x">
    Here be a really long line of text, to allow horizontal scrolling...
  </ion-scroll>
  Here be even more text and some dragons.
</ion-content>

Demo: http://codepen.io/anon/pen/BoGkA
Scrolling the ion-scroll horizontally works fine, but as soon as you (accidentally) start your touchDown/mouseDown inside the ion-scroll, you will no longer be able to vertically scroll the ion-content. Sometimes, you have a screen with a really big horizontal scroll element that also needs to be vertically scrollable, which will make scrolling that screen up and down really difficulty, since you mustn't start your scroll motion inside the horizontal scrolling element. In the aforementioned ticket, the cause for this was found in tap.js Line 154. If I remove the (e.defaultPrevented) condition, I'm able to scroll both the child and the parent at the same time. Since this doesn't seem to cause any other behaviour problems (I tested it on native Android, iOS browser, desktop browser), it would be great if this specific condition could be turned into a flag that I could set either on the general Ionic Framework or (even better) on each child scroll container.
What do you think?
Example: <ion-scroll direction="x" scroll-outside="true">

Unfortunately I run into the same issue. It would be helpful to know whether you plan to fix this issue in the near feature. If so, until when do you think you are going to release the fix?

This one is tricky since we have to determine a direction before we decide to start preventing default. Doing that smoothly can be problematic. We should address it though.
Horizontal scrolls should not break vertical scrolling.

@perrygovier You can apply my PR and improve the smoothness of the future.

@perrygovier @Ti-webdev I was able to make it work as expected. My solution is to lock the scrolling to one direction at a time, horizontal or vertical, no diagonal scrolling. In the doTouchMove function, I simply calculate which move is the greatest between X and Y, and allow scrolling accordingly.
      var mX = Math.abs(moveX * self.options.speedMultiplier),
            mY = Math.abs(moveY * self.options.speedMultiplier);

      if (self.__enableScrollX && mX >= mY) {

        scrollLeft -= moveX * self.options.speedMultiplier;
        var maxScrollLeft = self.__maxScrollLeft;

        if (scrollLeft > maxScrollLeft || scrollLeft < 0) {

          // Slow down on the edges
          if (self.options.bouncing) {

            scrollLeft += (moveX / 2  * self.options.speedMultiplier);

          } else if (scrollLeft > maxScrollLeft) {

            scrollLeft = maxScrollLeft;

          } else {

            scrollLeft = 0;

          }
        }
      }

      // Compute new vertical scroll position
      if (self.__enableScrollY && mY > mX) {
         ...

@pm-desjonqueresg  How do you clear out direction. I developed a similar approach but I couldn't find a better solution than using a timer to clear the direction. It also doesn't make sense to change the direction while you have tracking touches and it starts as a horizontal.

@mkoosej This solution is very simple and is clearly a hack for now as it completely disable diagonal scrolling. But in my application I don't have a use for it, so I'm fine with it. I guess it should check for a flag to see if scrolling is locked or not. I didn't dive much into Ionic code to really understand the whole logic, I can't say if should be done like this, but it works pretty well for me. Not sure why you will need to clear out direction... :)

This bug has already been fixed? I have an ion-content with 5 ion-scroll, so the user can't scroll down.
@bra1n's solution works, but I can't scroll down without accidentally  scroll some ion-scroll to left.
BTW, @bra1n thanks for the temporary solution, you saved me =D

@bra1n
This solution doesn't work for android native scrolling.
is there any solution for that?

If using "ion-scroll" then just set its "Height" as per need and it its "direction" attribute as per convenience it will also work gre8.

Is there any news on this issue ?

+1

+1

any news on this?

+1 .... Please include this feature

+1

+1 @perrygovier Any updates on this :) ?

is the plan to fix this in 1.2? is this why this issue was closed?

+1 Please include this feature, i really need it ...

  Here be lots of text

    Here be a really long line of text, to allow horizontal scrolling...

  Here be even more text and some dragons.

Do we have an update in this issue ?
It seems like a huge hack to always have to edit the ionic.bundle.js file.

It's been added to the milestone 1.2:
https://github.com/driftyco/ionic/milestones/1.2?page=2&q=is%3Aopen+milestone%3A1.2
So, basically it should be fixed in 1.2.

Should be fixed in c5b48cb6e36887bb1ff9daf39d33217e5bc651fd which will be in 1.2 in the new few days. Note: we are moving away from JS scrolling support, so this only works right with native scrolling which is the default in 1.2. Unfortunately, <ion-scroll> did not support native scrolling until that last commit.
Closing, please give it a try and let me know how it works!

Is it a fix for native scroll only? I still have the issue if I enable the js scroll by adding overflow-scroll="false" to <ion-scroll>
I'm using ionic 1.2.1.

This only seems to work when you explicitely set:
$ionicConfigProvider.scrolling.jsScrolling(false);
If you don't add it to your config section it doesn't work
although native scrolling should be default now.

+1

I'm finding that even "with" $ionicConfigProvider.scrolling.jsScrolling(false); on ios I "still" can not scroll horizontally.... "sometimes". It's a really weird temperamental bug. I can happily scroll vertically, but whether the app will let me horizontal scroll seems to be a matter of luck. I haven't figured out yet what I'm doing that allows me to scroll horizontally "sometimes"?

@andmar8 I am having the same problem. Parent vertical scrolling is not working from within child element with horizontal scrolling.
I've prototyped what I'm trying to achieve here: http://play.ionic.io/app/11a919492a31

+1

+1


Feature request: ability (for extensions) to perform a diff outside of source control

I'm aware VS Code can diff any two files you give it if you launch it from the command line, but I don't see a way to do that programmatically.
In my case, I making an extension that essentially performs an advanced find and replace algorithm for pre-determined patterns. I would like to generate a virtual document from the original document and diff it against the original so the user to can decide to accept or reject the changes instead of having to inspect the document themselves after the operation and look for changes by hand. Sort of like a dry-run.

We have a vscode.diff command for this. More info in https://code.visualstudio.com/api/references/commands

Thank you 😄 I missed this page in my searching!

@joaomoreno Semi-related, but is there a way for me to a) check if a given resource is under source control, and b) display the source control diff?
I want to provide a setting for the behavior of my extension:

Don't diff, apply changes right away
Preview changes with diff (diffing a virtual document)
Diff with source control if it is available

check if a given resource is under source control

You can try using the Git extension API. Example.


ICE for wcslen on Windows

Good day folks,
I hit a ICE today. I installed nightly today (and it's reporting 1.81) so I think it's up to date.
This might be related to #2904, but it's not a no_std crate.   I see a shim for wcslen in https://github.com/rust-lang/miri/blob/dc286ac1432932b2a710789de40ae8bc4e7ea7c1/src/shims/foreign_items.rs - but it looks like that shim doesn't work for msvc windows targets at a minimum.
But I'm not an expert on Miri.
windows_core::PCWSTR and windows_core::PWSTR both trivially trigger in via their .len() method - meaning a lot of windows code could benefit.
Edit: repro has been minimized further, see https://github.com/rust-lang/miri/issues/3692#issuecomment-2181830109
// windows-core 0.57.0 is the original source of the struct involved
// I've minimized the example (Trimming unnecessary derive macros, unused functions, et cetera)
// As best I can.
// https://github.com/microsoft/windows-rs/commit/15947886be041bebd0fe670fd4e73f18d95100d0

/// A pointer to a constant null-terminated string of 16-bit Unicode characters.
#[repr(transparent)]
pub struct PCWSTR(pub *const u16);

impl PCWSTR {
    /// Construct a new `PCWSTR` from a raw pointer
    pub const fn from_raw(ptr: *const u16) -> Self {
        Self(ptr)
    }

    /// String length without the trailing 0
    ///
    /// # Safety
    ///
    /// The `PCWSTR`'s pointer needs to be valid for reads up until and including the next `\0`.
    pub unsafe fn len(&self) -> usize {
        //#[cfg(windows)]
        // It's specifically the inclusion of this that makes or breaks the repro
        // The other non-windows one does not trigger the ICE.
        let len = {
            extern "C" {
                fn wcslen(s: *const u16) -> usize;
            }
            wcslen(self.0)
        };

        // #[cfg(not(windows))]
        // let len = {
        //     let mut len = 0;
        //     let mut ptr = self.0;
        //     while ptr.read() != 0 {
        //         len += 1;
        //         ptr = ptr.add(1);
        //     }
        //     len
        // };

        len
    }
}

pub fn does_not_trigger_ice_without_proc_macro()
{
    // This isn't the most portable from endianness perspective. But it'll do.
    let my_str: [u16; 2] = ['a' as u16 , 0];
    let my_pcwstr = PCWSTR::from_raw(my_str.as_ptr());
    // SAFETY: doesn't matter if it's safe, point is that this doesn't trigger an ICE.
    let _my_ice_trigger = unsafe { my_pcwstr.len() };
}

#[cfg(test)]
mod tests
{
    use crate::{does_not_trigger_ice_without_proc_macro, PCWSTR};
    // /// This works as well to trigger it.
    // #[test]
    // pub fn cause_ice2()
    // {
    //     does_not_trigger_ice_without_proc_macro()
    // }

    #[test]
    pub fn cause_ice()
    {
        // This isn't the most portable from endianness perspective. But it'll do.
        let my_str: [u16; 2] = ['a' as u16 , 0];
        let my_pcwstr = PCWSTR::from_raw(my_str.as_ptr());
        // SAFETY: doesn't matter if it's safe, point is that this doesn't trigger an ICE.
        let _my_ice_trigger = unsafe { my_pcwstr.len() };
    }
}

It also triggers on PWSTR. I don't think it repros on strlen though - which makes sense with the backtrace.
wcslen is utf16, not utf32, on Windows.
rustc-ice-2024-06-21T01_12_48-101996.txt

Doing a bit more digging
https://github.com/rust-lang/miri/blob/cc9699dc545e046e8fde9a2da315787ce07c7cb1/src/helpers.rs#L1049
    /// Read a sequence of wchar_t until the first null terminator.
    /// Always returns a `Vec<u32>` no matter the size of `wchar_t`.
    fn read_wchar_t_str(&self, ptr: Pointer) -> InterpResult<'tcx, Vec<u32>> {
        let this = self.eval_context_ref();
        let wchar_t = this.libc_ty_layout("wchar_t");
        self.read_c_str_with_char_size(ptr, wchar_t.size, wchar_t.align.abi)
    }

If it's pulling type layouts from Rust's libc implementation (which seems to make sense, but there's a lot of indirection going on here and I'm not a Rust compiler guru), then I'm really not sure why it doesn't work, because wchar_t should be u16 for windows msvc:
https://github.com/rust-lang/libc/blob/7d7151c51285bf238d2d02dc736c64b3c6dfbcc8/src/windows/mod.rs#L26

PWCSTR from windows-core is explicitly utf-16 / u16.  So I further simplified the repro. THis works fine on
cargo miri test --target x86_64-unknown-linux-gnu
but still ICEs exactly like the previous for cargo miri test --target x86_64-pc-windows-msvc
#[cfg(test)]
mod tests
{
    #[cfg(target_os = "windows")]
    type NativeWchar = u16;
    // This is a hack, but is useful for this repro
    #[cfg(not(target_os = "windows"))]
    type NativeWchar = u32;
    #[test]
    pub fn cause_ice()
    {
        // This isn't the most portable from endianness perspective. But it'll do.
        let my_str: [NativeWchar; 2] = ['a' as NativeWchar  , 0];
        let _len = {
            extern "C" {
                fn wcslen(s: *const NativeWchar) -> usize;
            }
            // SAFETY: doesn't matter if it's safe, point is that this triggers an ICE.
            let _my_ice_trigger = unsafe { wcslen(my_str.as_ptr()) };
        };
    }
}

To add to the mystery, both x86_64-pc-windows-gnullvm and x86_64-pc-windows-gnu both seem to be fine. and I believe gnullvm is using ucrt. So maybe it's related to just the tools, not even the c library in use? I'm not sure what to make of this.

Could you provide the error message? Also please see Creating and highlighting code blocks for how to format code blocks so they're more readable.

Whoops, sorry about the formatting... fixed.
Here's the console output, minus the stack trace that I already attached in my original message
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.56s
     Running unittests src/main.rs (LOCAL_SOURCE_PATH_REDACTED\target\miri\x86_64-pc-windows-msvc\debug\deps\playground01-809fb30126744ceb.exe)

running 1 test
thread 'rustc' panicked at src\tools\miri\src\helpers.rs:254:32:
failed to find required Rust item: ["libc", "wchar_t"]
stack backtrace:
   0:     0x7ff96bb23c3d - std::backtrace_rs::backtrace::dbghelp64::trace
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\..\..\backtrace\src\backtrace\dbghelp64.rs:91
   1:     0x7ff96bb23c3d - std::backtrace_rs::backtrace::trace_unsynchronized
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
   2:     0x7ff96bb23c3d - std::sys::backtrace::_print_fmt
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\sys\backtrace.rs:68
   3:     0x7ff96bb23c3d - std::sys::backtrace::_print::impl$0::fmt
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\sys\backtrace.rs:44
   4:     0x7ff96bb54629 - core::fmt::rt::Argument::fmt
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\core\src\fmt\rt.rs:165
   5:     0x7ff96bb54629 - core::fmt::write
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\core\src\fmt\mod.rs:1168
   6:     0x7ff96bb1a391 - std::io::Write::write_fmt<std::sys::pal::windows::stdio::Stderr>
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\io\mod.rs:1835
   7:     0x7ff96bb23a16 - std::sys::backtrace::print
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\sys\backtrace.rs:34
   8:     0x7ff96bb26bf8 - std::panicking::default_hook::closure$1
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\panicking.rs:265
   9:     0x7ff96bb268a9 - std::panicking::default_hook
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\panicking.rs:292
  10:     0x7ff8622681ac - memchr
  11:     0x7ff96bb27377 - alloc::boxed::impl$50::call
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\alloc\src\boxed.rs:2076
  12:     0x7ff96bb27377 - std::panicking::rust_panic_with_hook
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\panicking.rs:804
  13:     0x7ff96bb27186 - std::panicking::begin_panic_handler::closure$0
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\panicking.rs:670
  14:     0x7ff96bb245af - std::sys::backtrace::__rust_end_short_backtrace<std::panicking::begin_panic_handler::closure_env$0,never$>
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\sys\backtrace.rs:171
  15:     0x7ff96bb26d66 - std::panicking::begin_panic_handler
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\panicking.rs:661
  16:     0x7ff96bb7cfa4 - core::panicking::panic_fmt
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\core\src\panicking.rs:74
  17:     0x7ff618b0597b - <unknown>
  18:     0x7ff618b0d678 - <unknown>
  19:     0x7ff618bb1ca2 - <unknown>
  20:     0x7ff618bad00b - <unknown>
  21:     0x7ff618a8a939 - <unknown>
  22:     0x7ff618af013c - <unknown>
  23:     0x7ff618b01801 - <unknown>
  24:     0x7ff618a198dd - <unknown>
  25:     0x7ff85dfa563b - __ImageBase
  26:     0x7ff85dfa296f - __ImageBase
  27:     0x7ff85dfaa6c9 - __ImageBase
  28:     0x7ff96bb3838d - alloc::boxed::impl$48::call_once
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\alloc\src\boxed.rs:2062
  29:     0x7ff96bb3838d - alloc::boxed::impl$48::call_once
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\alloc\src\boxed.rs:2062
  30:     0x7ff96bb3838d - std::sys::pal::windows::thread::impl$0::new::thread_start
                               at /rustc/d8a38b00024cd7156dea4ce8fd8ae113a2745e7f/library\std\src\sys\pal\windows\thread.rs:52
  31:     0x7ff9e5bd257d - BaseThreadInitThunk
  32:     0x7ff9e69aaf28 - RtlUserThreadStart

error: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/miri/issues/new

note: please make sure that you have updated to the latest nightly

note: please attach the file at `LOCAL_SOURCE_PATH_REDACTED\rustc-ice-2024-06-21T02_01_40-93788.txt` to your bug report

note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental=[REDACTED] -C link-args=/DYNAMICBASE /CETCOMPAT

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack

Miri caused an ICE during evaluation. Here's the interpreter backtrace at the time of the panic:
note: the place in the program where the ICE was triggered
  --> src\rust\playground01\src/main.rs:26:44
   |
26 |             let _my_ice_trigger = unsafe { wcslen(my_str.as_ptr()) };
   |                                            ^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: BACKTRACE on thread `tests::cause_ice`:
   = note: inside `tests::cause_ice` at src\rust\playground01\src/main.rs:26:44: 26:67
note: inside closure
  --> src\rust\playground01\src/main.rs:17:23
   |
16 |     #[test]
   |     ------- in this procedural macro expansion
17 |     pub fn cause_ice()
   |                       ^
   = note: inside `<{closure@src\rust\playground01\src/main.rs:17:5: 28:6} as std::ops::FnOnce<()>>::call_once - shim` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ops\function.rs:250:5: 250:71
   = note: inside `<fn() -> std::result::Result<(), std::string::String> as std::ops::FnOnce<()>>::call_once - shim(fn() -> std::result::Result<(), std::string::String>)` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ops\function.rs:250:5: 250:71
   = note: inside `tests::test::__rust_begin_short_backtrace::<std::result::Result<(), std::string::String>, fn() -> std::result::Result<(), std::string::String>>` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\test\src\lib.rs:625:18: 625:21
   = note: inside `tests::test::types::RunnableTest::run` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\test\src\types.rs:146:40: 146:71
   = note: inside closure at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\test\src\lib.rs:648:60: 648:79
   = note: inside `<std::panic::AssertUnwindSafe<{closure@tests::test::run_test_in_process::{closure#0}}> as std::ops::FnOnce<()>>::call_once` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\panic\unwind_safe.rs:272:9: 272:19
   = note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<{closure@tests::test::run_test_in_process::{closure#0}}>, std::result::Result<(), std::string::String>>` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panicking.rs:553:40: 553:43
   = note: inside `std::panicking::r#try::<std::result::Result<(), std::string::String>, std::panic::AssertUnwindSafe<{closure@tests::test::run_test_in_process::{closure#0}}>>` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panicking.rs:517:19: 517:88
   = note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<{closure@tests::test::run_test_in_process::{closure#0}}>, std::result::Result<(), std::string::String>>` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panic.rs:350:14: 350:33
   = note: inside `tests::test::run_test_in_process` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\test\src\lib.rs:648:27: 648:81
   = note: inside closure at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\test\src\lib.rs:569:43: 577:18
   = note: inside closure at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\test\src\lib.rs:599:41: 599:83
   = note: inside `std::sys::backtrace::__rust_begin_short_backtrace::<{closure@tests::test::run_test::{closure#1}}, ()>` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\sys\backtrace.rs:155:18: 155:21
   = note: inside closure at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\thread\mod.rs:542:17: 542:71
   = note: inside `<std::panic::AssertUnwindSafe<{closure@std::thread::Builder::spawn_unchecked_<'_, '_, {closure@tests::test::run_test::{closure#1}}, ()>::{closure#2}::{closure#0}}> as std::ops::FnOnce<()>>::call_once` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\panic\unwind_safe.rs:272:9: 272:19
   = note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<{closure@std::thread::Builder::spawn_unchecked_<'_, '_, {closure@tests::test::run_test::{closure#1}}, ()>::{closure#2}::{closure#0}}>, ()>` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panicking.rs:553:40: 553:43
   = note: inside `std::panicking::r#try::<(), std::panic::AssertUnwindSafe<{closure@std::thread::Builder::spawn_unchecked_<'_, '_, {closure@tests::test::run_test::{closure#1}}, ()>::{closure#2}::{closure#0}}>>` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panicking.rs:517:19: 517:88
   = note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<{closure@std::thread::Builder::spawn_unchecked_<'_, '_, {closure@tests::test::run_test::{closure#1}}, ()>::{closure#2}::{closure#0}}>, ()>` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\panic.rs:350:14: 350:33
   = note: inside closure at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\thread\mod.rs:541:30: 543:16
   = note: inside `<{closure@std::thread::Builder::spawn_unchecked_<'_, '_, {closure@tests::test::run_test::{closure#1}}, ()>::{closure#2}} as std::ops::FnOnce<()>>::call_once - shim(vtable)` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ops\function.rs:250:5: 250:71
   = note: inside `<std::boxed::Box<dyn std::ops::FnOnce()> as std::ops::FnOnce<()>>::call_once` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\alloc\src\boxed.rs:2062:9: 2062:52
   = note: inside `<std::boxed::Box<std::boxed::Box<dyn std::ops::FnOnce()>> as std::ops::FnOnce<()>>::call_once` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\alloc\src\boxed.rs:2062:9: 2062:52
   = note: inside `std::sys::pal::windows::thread::Thread::new::thread_start` at C:\Users\REDACTED\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\sys\pal\windows\thread.rs:52:13: 52:60
   = note: this note originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)

test tests::cause_ice ... error: test failed, to rerun pass `--bin playground01`

Let me know if there's other information that would help.

Ah, you posted the ICE file earlier but I overlooked it. So the important part is that miri can't find wchar_t in the libc crate. @saethlin could this be because the windows-msvc standard library no longer depends on the libc crate and miri was implicitly relying on it?

Can't blame you for missing it given my crummy formatting 😄.
If it's not related to the libc crate (keeping in mind I'm not familiar with that history), the other theory that comes to mind as I think on it is msvc and msvc library conformance RE: wchar_t. Once upon a time I believe the default was that it just was a typedef, and it's still possible to opt into the non-conformant behavior I believe.
Keeping in mind I'm not clear on whether Miri is relying on Rust type metadata (From libc crate?) or debug info or RTTI info or what. Also keeping in mind I have no idea if it's relevant or not, but thought it was worth mentioning in case it is.
/Zc:wchar_t page
Maybe the lookup needs to be for  __wchar_t?

Miri relies on the Rust types declared in the libc crate.

could this be because the windows-msvc standard library no longer depends on the libc crate and miri was implicitly relying on it?

Yes. Our tests call wcslen via libc to avoid us having to repeat the signature, but that means we do have libc available even on Windows so we are missing the ICE.

FWIW @RalfJung - I did try adding libc as a crate dependency locally after Chris suggested that that might be the issue. It didn't seem to resolve the issue. But I'm guessing that adding libc as a crate dependency for my test crate wouldn't be expected to do it, , as that wouldn't make Miri magically depend on libc/load libc.
Thanks for the fast turnaround on the fix, and for all the awesome work you guys have done on Miri!

Hm, adding it as a dependency should have helped; that's how the test was passing on CI even when it was ICEing for you... maybe you have to also add extern crate libc or so.


BI-13964: Round published at to seconds

This PR rounds the published_at timestamp to the nearest second. The published_at timestamp is sent to CHS Kafka API as part of the Change Resource Event.
BI-13964

CI: Security warnings found!