[Security] Vulnerability in fstream

Security risk found in fstream package
See: https://www.npmjs.com/advisories/886

NPM version: 6.5.0 (yarn v1.15.2)
Node version: 10.13.0
node-sass version: 4.12.0

Output from yarn audit:
┌───────────────┬─────────────────────────────────────────────────────────────
│ high          │ Arbitrary File Overwrite                                     │
├───────────────┼─────────────────────────────────────────────────────────────
│ Package       │ fstream                                                      │
├───────────────┼─────────────────────────────────────────────────────────────
│ Patched in    │ >=1.0.12                                                     │
├───────────────┼─────────────────────────────────────────────────────────────
│ Dependency of │ node-sass                                                    │
├───────────────┼────────────────────────────────────────────────────────────
│ Path          │ node-sass > node-gyp > fstream                               │
├───────────────┼─────────────────────────────────────────────────────────────
│ More info     │ https://www.npmjs.com/advisories/886                         │
└───────────────┴─────────────────────────────────────────────────────────────

Hello I have the same problem , I hope it will been resolved as soon as possible 💃

node-gyp doesn't pin to a particular version of fsstream https://github.com/nodejs/node-gyp/blob/9a404d6d36dcf9c7be2ae9963019c4d89bbb9155/package.json#L25
Clean your node modules/reinstall to get an unnaffected version.
There is nothing to be be done in node-sass

A bump of node-gyp to v4.0.0 will remove fstream dependency, see: https://github.com/sass/node-sass/pull/2681

This PR clearly states why a PR is not required.

For anyone else having this problem:
node-gyp pins to an old version of tar (^2.0.0), which depends on the insecure version of fstream. However, node-tar released a new version of this ancient branch, to fix the vulnerability (2.2.2). Thus, upgrading tar will fix the problem.
However, when I run yarn upgrade tar, it does not upgrade tar to 2.2.2. Probably, yarn finds it cannot upgrade to the latest version 4.x because of dependencies, and then gives up. But, upgrading tar worked when I ran yarn upgrade<EMAIL_ADDRESS>That fixed the security vulnerability for me.


[Istio-1.3.0-rc0] Community Testing, deployment error in Istio Vault CA Integration

Bug description Following the instructions on https://preliminary.istio.io/docs/tasks/security/vault-ca/, at the step "kubectl apply -f sleep-injected.yaml", get an error as follows:
~$ kubectl apply -f sleep-injected.yaml
serviceaccount/sleep created
service/sleep created
error: error validating "sleep-injected.yaml": error validating data: ValidationError(Deployment.spec): missing required field "selector" in io.k8s.ap
i.apps.v1.DeploymentSpec; if you choose to ignore these errors, turn validation off with --validate=false
Affected product area (please put an X in all that apply)
[ ] Configuration Infrastructure
[X ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ X] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Expected behavior
No errors.
Steps to reproduce the bug
Following the instructions on https://preliminary.istio.io/docs/tasks/security/vault-ca/, at the step "kubectl apply -f sleep-injected.yaml", an error is returned.
Version (include the output of istioctl version --remote and kubectl version)
~$ ./bin/istioctl version --remote
client version: 1.3.0-rc.0
citadel version: 1.3.0-rc.0
galley version: 1.3.0-rc.0
ingressgateway version: 1.3.0-rc.0
nodeagent version:
nodeagent version:
nodeagent version:
pilot version: 1.3.0-rc.0
policy version: 1.3.0-rc.0
sidecar-injector version: 1.3.0-rc.0
telemetry version: 1.3.0-rc.0
~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-13T23:16:58Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"12+", GitVersion:"v1.12.8-gke.10", GitCommit:"f53039cc1e5295eed20969a4f10fb6ad99461e37", GitTreeState:"clean", BuildDate:"2019-06-19T20:48:40Z", GoVersion:"go1.10.8b4", Compiler:"gc", Platform:"linux/amd64"}
How was Istio installed?
helm install.
Environment where bug was observed (cloud vendor, OS, etc)
Linux
Additionally, please consider attaching a cluster state archive by attaching
the dump file to this issue.

@geeknoid we need a docs approval on https://github.com/istio/istio/pull/16605 so we can close out this release blocker


python_qt_binding: 0.3.3-1 in 'lunar/distribution.yaml' [bloom]

Increasing version of package(s) in repository python_qt_binding to 0.3.3-1:

upstream repository: https://github.com/ros-visualization/python_qt_binding.git
release repository: https://github.com/ros-gbp/python_qt_binding-release.git
distro file: lunar/distribution.yaml
bloom version: 0.6.1
previous version for package: 0.3.2-2

python_qt_binding
* Prefer qmake-qt5 over qmake when available (#43 <https://github.com/ros-visualization/python_qt_binding/issues/43>)

The debinc uses PySide2 on Zesty, see #16157.

holding until lunar sync

sync is out.
Holding for synchronized release


GitHub Workflows security hardening

This PR adds explicit permissions section to workflows. This is a security best practice because by default workflows run with extended set of permissions (except from on: pull_request from external forks). By specifying any permission explicitly all others are set to none. By using the principle of least privilege the damage a compromised workflow can do (because of an injection or compromised third party tool or action) is restricted.
It is recommended to have most strict permissions on the top level and grant write permissions on job level case by case.

Hello @sashashura,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

E2E tests not automatically triggered, because PR has no approval yet. Please ask a developer to review and then try again to attach the QA label.

Thanks for the PR! This all sounds good to me, but I'll defer to our security team before approving since they're the one who set up this check

Thanks for the review, and thanks again for the PR, @sashashura!
/release-note-none


Switch from a Span trait to a Span struct

Currently rslint_core uses a span trait. This is great for supplying nodes or tokens or ranges, however, it is a bit limiting. I think to allow for more complex span logic we should make a Span struct and then impl From<T> for Span (although no specialization might make this a bit limiting). We can then have logic similar to rustc_errors with things like trim, between, contains, until, etc. This switch should be fairly simple, just add the struct, impl the right traits, and change impl Span to impl Into<Span>.

Closing this in favor of just adding the methods to the trait


Pending form data should be deserialized as it is on the server

Right now, I have to do this:
  const pending = usePendingFormSubmit()
  const newColumnName = pending?.data.get("createColumn.name")
  const deletedColumnId = pending?.data.get("deleteColumn.id")

Which makes sense, as those are the actual names of the  fields that I'm using, but it'd be convenient to have the body on the client in the same object shape as the body on the server, with the nested fields. So, I could do pending?.body.createColumn?.name, and reuse the same body type on the client. We should also keep the actual form data around too, just in case it's needed for whatever reason ¯\_(ツ)_/¯

How about exposing the expand utility?
`formDataToObject(pending?.data).createColumn?.name
I think having both a data and a body prop can be a bit confusing? Or not?


fix: enable consistent_read on default

What does this change?
A clear and concise description of what the changes is.
References

If you have links to other resources, please list them here. (e.g. issue url, related pull request url, documents)

Screenshots
If applicable, add screenshots to help explain your changes.
What can I check for bug fixes?
Please briefly describe how you can confirm the resolution of the bug.

I stop this PR for now.
Because it causes error when enable consistent error with query for GSI.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Validation("Consistent read cannot be true when querying a GSI")', raiden/tests/all/query.rs:133:28


Move away from kubebuilder

Kubebuilder was used to init the project, but it should now be removed. This is to reduce the complexity. It can always be added again later.
Criteria

deploy locally and externally without kubebuilder
test without kubebuilder
develop without kubebuilder
controller file should be split into more files
tests pass

We've decided we actually want to use kubebuilder some more in the future, so will not remove it from the project yet.


Managed resources should be unbound when their claim is deleted

What happened?

The resource claim reconciler supports either dynamically or statically provisioning managed resources. The former involves using a resource class to create a new managed resource, while the latter involves binding to an existing resource.
A resource claim is the Kubernetes owner reference of any managed resource it dynamically provisions, but not the owner reference of any statically provisioned managed resource it later binds to. When a resource claim is deleted its underlying managed resource will only be deleted if it was dynamically provisioned and the user did not supply the --cascade option to kubectl delete.
While it is possible to delete a resource claim without deleting its managed resource, the resource claim reconciler doesn't currently update the managed resource in this scenario. It remains in binding state "bound", with a claim reference to the (now deleted) claim that it was bound to. This is probably not the behaviour we want.
How can we reproduce it?

Statically provision a Crossplane managed resource
Create a claim that specifies the managed resource from step 1 in its .spec.resourceRef
Wait for the claim and resource to bind.
Delete the claim created in step 2
Observe that the managed resource created in step 1 continues to be "bound" to the now deleted resource claim.

What environment did it happen in?
Crossplane version:

Possibly some crossover with https://github.com/crossplaneio/crossplane-runtime/issues/21.


[Swift 5.1] Adopt std lib diffing?

Ordered Collection Diffing:
https://github.com/apple/swift-evolution/blob/master/proposals/0240-ordered-collection-diffing.md
This should land in Swift 5.1. Can we use this instead of a third-party dependency?

Woah

If we decided to do this I would be down to try implementing :)


Multi-client login - custom auth token

This is PR for #30 issue. Pleas do NOT merge it now. This is just POC. We have to figure out a few things before merge.
Summary: with new custom authentication token user have to provide "client" ID when login, but when he log out - we will remove only token created for that client. He will be still "logged in" on other clients.
Remarks about new approach:

there is new Django app with custom token model - djoser.token - I don't know is it the best name for it. Maybe djoer.authtoken? Or djoser.multitoken?
I didn't add any new settings - you just have to install djoser.token and use new djoser.token.authentication.TokenAuthentication as your DEFAULT_AUTHENTICATION_CLASSES. It's possible due to rest_framework.authentication.TokenAuthentication API - it has model field. Please check magic in djoser.utils.get_authentication_token_class and djoser.get_token_model. I don't like it, but it works.
please also review how I handle different login serializer for new token - TokenAuthentication.get_login_serializer_class
probably djoser.utils is not the best place to put those magic methods but I don't have better idea now.
maybe we should generate "client" ID when user didn't provide it?
I think that we should remove LOGIN_AFTER_ACTIVATION / LOGIN_AFTER_REGISTRATION -  this approach doesn't fit well to current solution, and it doesn't make sense at all.
with new API we could add more important fields to our Token model - e.g. expiration date.
of course we have to prepare documentation for this PR, but I think we could do this at the end.

Coverage decreased (-3.42%) to 91.25% when pulling 1a7f35683d3c59ec7529383fc69ad99764ce062d on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-3.42%) to 91.25% when pulling 1a7f35683d3c59ec7529383fc69ad99764ce062d on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

@rugala and @haxoza it will be awesome if you could leave your comments here when you find some spare time.

Coverage decreased (-2.44%) to 92.22% when pulling 25a0c286dd9272313f16ddfb76991530513f3bd9 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-2.44%) to 92.22% when pulling 25a0c286dd9272313f16ddfb76991530513f3bd9 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-2.44%) to 92.22% when pulling 25a0c286dd9272313f16ddfb76991530513f3bd9 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-2.44%) to 92.22% when pulling 25a0c286dd9272313f16ddfb76991530513f3bd9 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-2.44%) to 92.22% when pulling 25a0c286dd9272313f16ddfb76991530513f3bd9 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-3.81%) to 90.86% when pulling 25a0c286dd9272313f16ddfb76991530513f3bd9 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-3.81%) to 90.86% when pulling 25a0c286dd9272313f16ddfb76991530513f3bd9 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-3.81%) to 90.86% when pulling 25a0c286dd9272313f16ddfb76991530513f3bd9 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-3.81%) to 90.86% when pulling 25a0c286dd9272313f16ddfb76991530513f3bd9 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-3.81%) to 90.86% when pulling 25a0c286dd9272313f16ddfb76991530513f3bd9 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-3.81%) to 90.86% when pulling 25a0c286dd9272313f16ddfb76991530513f3bd9 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-3.81%) to 90.86% when pulling 508aa9cd625fc3652ad4b041b311b3818f8e78a5 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-3.81%) to 90.86% when pulling 508aa9cd625fc3652ad4b041b311b3818f8e78a5 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-3.81%) to 90.86% when pulling 508aa9cd625fc3652ad4b041b311b3818f8e78a5 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

Coverage decreased (-3.81%) to 90.86% when pulling 508aa9cd625fc3652ad4b041b311b3818f8e78a5 on custom-auth-token into 7d32d744c9bddf6d2cc75cf6152d8dfc71fb25c2 on master.

It took some time but here are my thoughts.
Client field
According to what we discussed the client field is not required because it's difficult to define what should be put in it. For example on iOS you cannot get any unique device ID. That means that anyway the client needs to generate some random sequence in order to fill client field or it should be randomly filled on server side.
We concluded that client field should be removed. However knowing that we client field was invented to protect against overflowing database with new tokens for the same users we decided to introduce expiration date to the token model. We should also provide a logic for removing expired tokens.
Setting like MAXIMUM_CLIENTS_NUMBER is also a good idea.
The app name
I would opt in for multitoken name. However I would move this django app to seprate project to not mess djoser core functionality.
When we add support for other auth backends (see issues) there is no reason to keep multitoken as a part of djoser. What's more I can imagine that we could provide more features to multitoken app like some storage or caching backends. This will be not connected with djoser directly and it can end up with mess in pull requests and issues on github.
Authentication process
In general it totally makes sense what you did with TokenAuthentication class in this new token app. However considering issues #28 and #40 I would suggest the new way of handling auth backends in djoser. Currently we have a lot of code depending on TokenAuthentication. Probably it's not good idea if we want to support other backends.
That's why I would remove all the code depending on TokenAuthentication, especially UserLoginSerializer and LoginView classes. With the new token app the responsibility how the authentication is handled should rely on this new app, not on djoser. It's handled exactly in the same way by other auth backends:

JWT auth,
Token auth from DRF,
and what's funny BasicAuthentication backend does not require any obtain view.

Other  stuff
I agree we should remove LOGIN_AFTER_ACTIVATION / LOGIN_AFTER_REGISTRATION settings and support.
Further steps
Here is what I suggest to do now:

Extract new token app as a new project
Close this PR and branch.
Create new branch when we add support for other auth backend including new multitoken app.
When the all above works we will merge it to djoser and release new major version.

@konradhalas please deliberate :smile:

Give me a month or so...

any progress update?

@myquant not yet, but there will be really soon :smile: We've just released last djoser version before quite big refactoring which includes finishing this pull request.

I extracted token app created in this branch and pull request to the new project:
https://github.com/sunscrapers/django-rest-multitoken


CI Update

Build.Reason:Manual by Alma Jenks
Build.Url:https://apidrop.visualstudio.com/Content CI/_build/results?buildId=173864&view=results
source_repo.branch:master
source_repo.csvPath:bundlepackages/sqlite-dotnet
source_repo.url:https://apidrop.visualstudio.com/binaries/_git/mrefconfig

Docs Build status updates of commit ba1d496:
:warning: Validation status: warnings

File
Status
Preview URL
Details

dotnet/xml/Microsoft.Data.Sqlite/SqliteConnection.xml
:warning:Warning
View
Details

dotnet/xml/Microsoft.Data.Sqlite/SqliteDataReader.xml
:warning:Warning
View
Details

dotnet/xml/Microsoft.Data.Sqlite/SqliteException.xml
:warning:Warning
View
Details

dotnet/xml/FrameworksIndex/msdata-sqlite-5.0.0.xml
:white_check_mark:Succeeded

dotnet/xml/Microsoft.Data.Sqlite/SqliteBlob.xml
:white_check_mark:Succeeded
View

dotnet/xml/Microsoft.Data.Sqlite/SqliteCacheMode.xml
:white_check_mark:Succeeded
View

dotnet/xml/Microsoft.Data.Sqlite/SqliteCommand.xml
:white_check_mark:Succeeded
View

dotnet/xml/Microsoft.Data.Sqlite/SqliteConnectionStringBuilder.xml
:white_check_mark:Succeeded
View

dotnet/xml/Microsoft.Data.Sqlite/SqliteFactory.xml
:white_check_mark:Succeeded
View

dotnet/xml/Microsoft.Data.Sqlite/SqliteOpenMode.xml
:white_check_mark:Succeeded
View

dotnet/xml/Microsoft.Data.Sqlite/SqliteParameter.xml
:white_check_mark:Succeeded
View

dotnet/xml/Microsoft.Data.Sqlite/SqliteParameterCollection.xml
:white_check_mark:Succeeded
View

dotnet/xml/Microsoft.Data.Sqlite/SqliteTransaction.xml
:white_check_mark:Succeeded
View

dotnet/xml/Microsoft.Data.Sqlite/SqliteType.xml
:white_check_mark:Succeeded
View

dotnet/xml/PackageInformation/msdata-sqlite-1.0.0.json
:white_check_mark:Succeeded

dotnet/xml/PackageInformation/msdata-sqlite-1.1.0.json
:white_check_mark:Succeeded

dotnet/xml/PackageInformation/msdata-sqlite-2.0.0.json
:white_check_mark:Succeeded

dotnet/xml/PackageInformation/msdata-sqlite-2.1.0.json
:white_check_mark:Succeeded

dotnet/xml/PackageInformation/msdata-sqlite-2.2.0.json
:white_check_mark:Succeeded

dotnet/xml/PackageInformation/msdata-sqlite-3.0.0.json
:white_check_mark:Succeeded

dotnet/xml/PackageInformation/msdata-sqlite-3.1.0.json
:white_check_mark:Succeeded

dotnet/xml/PackageInformation/msdata-sqlite-5.0.0.json
:white_check_mark:Succeeded

dotnet/xml/Microsoft.Data.Sqlite/SqliteConnection.xml

Line 0, Column 0: [Warning-xref-not-found] Cross reference not found: 'SQLitePCL.sqlite3'.

dotnet/xml/Microsoft.Data.Sqlite/SqliteDataReader.xml

Line 0, Column 0: [Warning-xref-not-found] Cross reference not found: 'SQLitePCL.sqlite3_stmt'.

dotnet/xml/Microsoft.Data.Sqlite/SqliteException.xml

Line 0, Column 0: [Warning-xref-not-found] Cross reference not found: 'SQLitePCL.sqlite3'.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Is this something new? Earlier it used to have different way of showing applies to which worked at page level rather than overload level. (overload level is better). Just making sure that rendering isn't an accidental bug.

@smitpatel That is a new one to me. I hadn't seen it before. There are constant improvements being made on the back end. This may be one of those.


In auth plugins, support passing information about consumer to backing service

Currently, when using the basicauth and keyauth plugins, a X-Consumer-ID header is added to the request, allowing the backing service to know who authenticated to Kong.  However, unless the service has been told about the consumers through some other channel, it won't know who that consumer is.
One option would be for the service to call back to Kong and do a lookup via the Admin API's GET /consumers/{id} endpoint.  However, the Admin API (at least in its current state) has no security, and isn't really something that you'd want various services calling back to.
A simpler solution would seem to be to add headers to the request to pass along the consumer's username/custom_id (perhaps X-Consumer-Username and X-Consumer-CustomID?)  This would allow the backing service to base further authorization decisions on this information without calling back to Kong.  If desired, this could be an optional feature turned on via the plugin configuration.

+1

+1 - This will save us significant overhead on making additional calls from our API service to translate the consumer id back into the custom id.

Closed with 450f5559b5e8c2c9e000791da97fa99303f47511. It will be available from the next version.

Great!  Thanks, @thefosk!

Awesome thanks @thefosk


DataConstants restructuring to store API keys

-If an API key has not been entered, the user will be prompted and the key will be stored by keyring

Added keyring


6sec hangup when opened by file:// protocol.

Hello!
esdoc is the great tool!
But it seems frozen during about 6 seconds when opened as the local file.
I guess the cause is this.
It's waiting for a timeout of opening file://fonts.googleapis.com.
So I think we can use https://fonts.googleapis.com/css?family=Roboto:400,300,700 instead.
Thanks.
GET file://fonts.googleapis.com/css?family=Roboto:400,300,700 net::ERR_FILE_NOT_FOUND

ESDoc v0.1.1
Node v0.12.2
npm v2.8.4
Windows 7 Pro 64bit

@mysticatea Hello! Thank you for using ESDoc.
The hangup is not occur in my environment(Mac OSX10.10.3 & Chrome42).
However, I think that your proposal is correct.
So I fixed web font protocol and will release the fixing in next version.

Thank you.
I updated esdoc and received the fix!


Adding alert popup whenever user deletes the entities from playlist entity table

Hey, I just made a Pull Request!

:heavy_check_mark: Checklist

[ ] A changeset describing the change and affected packages. (more info)
[ ] All your commits have a Signed-off-by line in the message. (more info)

this PR for https://github.com/backstage/backstage/issues/18836

@kuangp @Rugvip This PR is for https://github.com/backstage/backstage/issues/18836  Please review

@kuangp minor changes done please review

@Rugvip Awaiting for your approval please review.


ERC1155 _mint without data

It is unclear what the ERC1155 _mint data payload is used for.  In many cases it would save gas to include a new internal function to avoid passing this argument:
function _mint (
  address account,
  uint256 tokenId,
  uint256 amount
) internal {
  _mint(account, tokenId, amount, '');
}

@zimmah What is the data payload used for?

Include __internal test in ERC1155Base.ts.

In combination with the "safe/unsafe" and "single/batch" transfer functions, this might require more code than it's worth to be comprehensive and intuitive.


Alchemy API : daily limit

During testing of more advanced labs we came across the message "Alchemy API request error: daily-transaction-limit-exceeded" when sending data to the AlchemyAPI Node-RED node.  This is important if end users of the labs reach this limit.  At present we do not know what the actual limit is and whether Bluenmix is enforcing it or AlchempyAPI team (indirectly).

http://www.alchemyapi.com/company/faq which eventually get you to See https://console.ng.bluemix.net/catalog/alchemy_api/?utm_campaign=Bluemix Pricing&utm_medium=Pricing-Page&utm_source=Web

Not a bug if you use the Alchemy API key you get from the Alchempy API Service in your Bluemix project.  I used the key I obtained from the external web site and this had the limit set on it


Add nix flake support

Build with nix build
Run with nix run
Install with nix profile install <path to flake>


Create LICENSE

This pull request is part of an initiative to make the licensing information about OBO Foundry ontologies more actionable.
While this ontology contains licensing information in its metadata, GitHub is not aware of the license unless you create a LICENSE file with standardized text. This PR creates a LICENSE file with the standard text for the CC BY 4.0 license.
More information about this initiative and further discussion is taking place here: https://github.com/OBOFoundry/OBOFoundry.github.io/issues/1840.

@luanalicata can you please merge this PR so the licensing information about this ontology is actionable?


Open-source CPU targets

Are any of the following open-source CPUs targets for powdr?
Specifically, should I ever expect that powdr will run on, say, the Berkeley/SiFive Rocket rv32ima, etc

[ ] Berkeley/SiFive Rocket - https://github.com/chipsalliance/rocket-chip
[ ] Bluespec Flute - https://github.com/bluespec/Flute
[ ] OpenHardware Group CVA6 - https://github.com/openhwgroup/cva6.git (a.k.a. ETH Zurich Ariane)

Hi @taqtiqa-mark !
I'm personally not familiar with those specific variants, but if you can generate LLVM assembly (AT&T asm syntax) using rv32imac ISAs our powdr-rs crate should be able to compile it to powdr-IR using the asm input.
As I see in the script you linked, it looks like it also generates a rv32ima ELF. Although our ELF frontend only supports rv32im, it's in general possible to remove the atomic instructions by using passes=loweratomic.
It'd be great to have a look at any specific assembly or ELF files you're interested in, otherwise we can have a look ourselves at some point (it'll just take a bit longer).


[XRI3] Eye gaze recording sample is not working

To reproduce
Steps to reproduce the behavior:

Build the Features/XRI3 sample project for HL2
On HL2 go to the EyeTrackingVisualizerExample.unity scene
Click the "Record Heatmap" button
Look at the right panel. Move eyes around all over the right panel (you want see any indication you're looking...that's ok)
Click "Stop Recording"
Click "Play Heatmap"
Result (Bug): No animation plays

Expected behavior
Expected: Heat Map animation to play on the right panel

The problem most likely is that the gaze interactor is not set on the test script...

Setting the gaze interactor on the test script fixes this issue.

This is a "sample only" issue.

Fixed


Benefits of Compressing Unsorted uint64_t

I was playing around with libvbyte. First of all, thank you for the library. It works great on sorted numbers.
Using test.cc as a starting point, I was trying to compress unsorted numbers (dataset attached) but the size of the result set increases compared to the original dataset.

Original dataset (txt): 13.5 MB
Original dataset (binary): 5.4 MB
Encoded dataset (binary): 6.7 MB

I can understand that this could be due to the unsorted nature of the dataset but I am curious what benefits I would get by using libvbyte on unsorted datasets (where the result set size could potentially increase). Or do you recommend that I always sort datasets before using your library?
Thank you!
data.txt

In some cases, the varbyte encoding can require more storage than the unencoded binary number, and this seems to be the case with your data because your numbers are so big. Big numbers require many bytes, and each byte "wastes" one bit for the "continuation bit". Here is an explanation of the algorithm: http://nlp.stanford.edu/IR-book/html/htmledition/variable-byte-codes-1.html
When you sort your numbers then libvbyte only compresses the deltas. These deltas are typically very small and therefore require much less storage.


Dedicated Page for Showdown Syntax

While the demo site is handy, the inclusion of installation instructions makes it a less than optimal choice for linking "Here's how Showdown Markdown works".  Recommend taking the formatting-related content from the demo page, and making a wiki page out of it.
I've already done this on my project site (which implements Showdown), and you're welcome to copy it.
https://github.com/GoogleCloudPlatform/PerfKitExplorer/wiki/Showdown-Markdown-Syntax

I've updated showdown's syntax wiki page.
https://github.com/showdownjs/showdown/wiki/Showdown's-Markdown-syntax
If you feel there's something missing please let me know

I've updated showdown's syntax wiki page.
https://github.com/showdownjs/showdown/wiki/Showdown's-Markdown-syntax
If you feel there's something missing please let me know


Errors with profiles

FYI, not sure if this is a concern - switching back to the production release solved the issue.
glawler@hp-laserjet5 % oo tunnels hb-sb
Starting hb-sb |████████████████████████████████| 100%
Exception ignored in: <function BaseProfile.del at 0x102e16de0>
Traceback (most recent call last):
File "/opt/homebrew/lib/python3.11/site-packages/mozprofile/profile.py", line 77, in del
self.cleanup()
File "/opt/homebrew/lib/python3.11/site-packages/mozprofile/profile.py", line 263, in cleanup
self.permissions.clean_db()
File "/opt/homebrew/lib/python3.11/site-packages/mozprofile/permissions.py", line 417, in clean_db
cursor.execute("DROP TABLE IF EXISTS moz_hosts")
sqlite3.OperationalError: database is locked

This has been solved in #176


Integrate patches into inverse_compression

Unable to integrate into inverse_compression as get_patches requires original time series which is not accessible to inverse_compression. Will keep as separate function to maintain backwards compatibility.


Release preparation for Azure.Core 1.30.0

Changelog and project version update

API change check
API changes are not detected in this pull request.


commit

commit
Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.
List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.
If you had to change anything in the flutter/tests repo, include a link to the migration guide as per the breaking change policy.
Pre-launch Checklist

[ ] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[ ] I read the Tree Hygiene wiki page, which explains my responsibilities.
[ ] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[ ] I signed the CLA.
[ ] I listed at least one issue that this PR fixes in the description above.
[ ] I updated/added relevant documentation (doc comments with ///).
[ ] I added new tests to check the change I am making, or this PR is test-exempt.
[ ] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Hello! It looks like this PR was opened by mistake. If that's not the case, please update the PR description to give more context and reopen it. In the meantime I am going to close this PR. Thanks!


[Feature Suggestion] Mass-Download

Pretty simple suggestion: Add a button that allows you to download an image of all the origins in a datapack all at once, possibly inside of a .zip file.

Great idea, you should now see a "Download All" button at the top right. Thanks


xdg-open unexpected argument '/c'

Issue Description
An error is thrown when loading an ASP.NET Core project. Other project types (Standard, Core) seem to work.
Steps to Reproduce
Open a solution containing an ASP.NET core project in VSCode (only tested on one project)
Expected Behavior
Project would load correctly
Actual Behavior
An error is thrown early on:
<details>[info]: OmniSharp.MSBuild.ProjectManager
        Loading project: /path/to/project.csproj
xdg-open: unexpected argument '/c'
Try 'xdg-open --help' for more information.
[fail]: OmniSharp.MSBuild.ProjectLoader
        Cannot find the specified file

Logs
OmniSharp log

Starting OmniSharp server at 2019-5-12 09:48:10
    Target: /path/to/solution.sln
OmniSharp server started with Mono 5.20.1.
Path: /home/s/.vscode/extensions/ms-vscode.csharp-1.19.1/.omnisharp/1.32.19/omnisharp/OmniSharp.exe
PID: 21793
    Starting OmniSharp on antergos 0.0 (x64)

    DotNetPath set to dotnet

    Located 2 MSBuild instance(s)
        1: Mono 15.0 - "/usr/lib/mono/msbuild/15.0/bin"
        2: StandAlone 15.0 - "/home/s/.vscode/extensions/ms-vscode.csharp-1.19.1/.omnisharp/1.32.19/omnisharp/.msbuild/Current/Bin"

    Registered MSBuild instance: Mono 15.0 - "/usr/lib/mono/msbuild/15.0/bin"
        CscToolExe = csc.exe
        CscToolPath = /home/s/.vscode/extensions/ms-vscode.csharp-1.19.1/.omnisharp/1.32.19/omnisharp/.msbuild/Current/Bin/Roslyn

    Detecting Cake files in '/path/to/repo'.

    Could not find any Cake files

    Project system 'OmniSharp.DotNet.DotNetProjectSystem' is disabled in the configuration.

    Detecting projects in '/path/to/solution.sln'.

    Queue project update for ...

    Detecting CSX files in '/path/to/repo'.

    Could not find any CSX files

    Invoking Workspace Options Provider: OmniSharp.Roslyn.CSharp.Services.CSharpWorkspaceOptionsProvider

    Loading project: /path/to/working.csproj

    Configuration finished.

    Omnisharp server running using Stdio at location '/path/to/repo/' on host 21640.

    Successfully loaded project file '/path/to/working.csproj'.

    Adding project '/path/to/working.csproj''

... more working projects
info: OmniSharp.MSBuild.ProjectManager
Loading project: '/path/to/aspnet.csproj'
xdg-open: unexpected argument '/c'
Try 'xdg-open --help' for more information.
[fail]: OmniSharp.MSBuild.ProjectLoader
Cannot find the specified file
[warn]: OmniSharp.MSBuild.ProjectManager
Failed to load project file '/path/to/aspnet.csproj'
/path/to/aspnet.csproj
/home/s/.nuget/packages/buildwebcompiler/1.12.394/build/BuildWebCompiler.targets(12,9): Error: Cannot find the specified file

Environment information
VSCode version: 1.33.1
C# Extension: 1.19.1
Mono Information
        OmniSharp using global mono :5.20.1
Dotnet Information
.NET Core SDK (reflecting any global.json):
 Version:   2.2.105
 Commit:    7cecb35b92
Runtime Environment:
OS Name:     antergos
OS Version:
OS Platform: Linux
RID:         arch-x64
Base Path:   /opt/dotnet/sdk/2.2.105/
Host (useful for support):
Version: 2.2.3
Commit:  6b8ad509b6
.NET Core SDKs installed:
2.2.105 [/opt/dotnet/sdk]
.NET Core runtimes installed:
Microsoft.NETCore.App 2.2.3 [/opt/dotnet/shared/Microsoft.NETCore.App]
To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download

Visual Studio Code Extensions

Extension
Author
Version

azure-account
ms-vscode
0.8.2

azure-pipelines
ms-azure-devops
1.147.2

azurerm-vscode-tools
msazurermtools
0.6.0

csharp
ms-vscode
1.19.1

Ionide-fsharp
Ionide
3.37.0

Ionide-Paket
Ionide
1.12.0

mssql
ms-mssql
1.6.0

team
ms-vsts
1.149.2

vscode-azureappservice
ms-azuretools
0.14.0

vscode-azurestorage
ms-azuretools
0.7.0

vscode-node-azure-pack
ms-vscode
0.0.6

vscode-nuget-package-manager
jmrog
1.1.6

@spaasis xdg-open is used for launching a browser. You can likely fix this either by disabling browser launch, or by changing the command line. To do either, open launch.json, and customize the 'launchBrowser' element.

Sorry, upon rereading this issue, I realized that I completely misread this bug. Ignore my previous comment.

Possibly related, depending on where that /c parameter is ultimately coming from, to https://github.com/dotnet/corefx/issues/24704, where xdg-open was being used on Linux to provide the equivalent of cmd /c. If the process of opening a .csproj file involves starting an external process, then I'd strongly suspect that https://github.com/dotnet/corefx/issues/24704 is behind the issue. It would be helpful to figure out the full xdg-open command that's being attempted here.

Is there something I can do to help you guys catch this?

CC @rchande @akshita31

Hmm, if I uninstall the WebCompiler (https://github.com/madskristensen/WebCompiler) and restart Omnisharp there's no errors, so I guess this is actually a WebCompiler bug. Should I close this one when I post a new issue over there?

Seems to work fine when I use VSCode on Windows, so it's probably a Linux bug,

As I mentioned in https://github.com/madskristensen/WebCompiler/issues/421#issuecomment-503220197, WebCompiler seems to assume it's running on Windows (calling cmd.exe /c and so on), so it's not designed to work on Linux.
@spaasis I'm curious to know how you're managing to use WebCompiler in VS Code, as it's a Visual Studio extension, not a VS Code extension. Is it a NuGet package you use in your project? When you said that you "uninstalled the WebCompiler" and it worked, what exactly did you do?

@rmunn Yup, that seems to close this case.
So, to clarify;
I was working on a project on Windows, Visual Studio, that utilizes the WebCompiler extension. That extensions seems to automatically install a NuGet package BuildWebCompiler (https://www.nuget.org/packages/BuildWebCompiler) that does.. something (although earlier today I ran a different WebCompiler project without the NuGet package and it compiled the .scss just fine, go figure..).
I tried to open this project on my Linux machine with VSCode and this error popped up. When I said I 'uninstalled WebCompiler', I meant that I removed that BuildWebCompiler NuGet package, nothing fancier than that :)
After removing it, the Omnisharp proceeded without errors, and that's as far as I went with testing.
Thank you guys for helping me out here. I'll close the issue and continue discussing it over at the WebCompiler side.


Unable to go to space 11 via control + - key

After using the following config with ctrl - - switching to space to space 8, 9, 10 stops working. This is also replicable when using ctrl - =.
# Focus : focus space
ctrl - 0 : yabai -m space --focus 10
# Focus : focus space
ctrl - - : yabai -m space --focus 11

Maybe it's not an issue with yabai, but maybe that your key (- is not recognized).
To test it, change the key to something else easier, like
ctrl - a : yabai -m space --focus 11

And try to see if it works.

ctrl - - is not valid syntax; - is a reserved keyword and cannot be used as a literal (because I was lazy when writing the parser). Use the keycode instead as mentioned in the comment above.

Thanks @danymat and @koekeishiya . That resolved the issue for me :)


fix: GraphiQL Explorer Plugin init function usage

The implementation pattern of the GraphiQL Explorer Plugin changed upstream causing the initialization to need to be adjusted.
See: https://github.com/graphql/graphiql/releases/tag/%40graphiql%2Fplugin-explorer%400.3.0

Yes, we should probably pin the version. However, that would mean that we manually need to check for und update to new versions. At least I am not aware of any tools that automatically check for new versions from CDN URLs.

Yes, we should probably pin the version. However, that would mean that we manually need to check for und update to new versions. At least I am not aware of any tools that automatically check for new versions from CDN URLs.

then let's take as bli


🐛 BUG:

Which Cloudflare product(s) does this pertain to?
C3, D1, KV, Pages, Pub/Sub, Queues, R2, Workers for Platforms, Workers/Other, Wrangler
What version of Wrangler are you using?
0.0.0
What operating system are you using?
Mac
Describe the Bug
/ #

I'm going to close this for now, but feel free to add more details and we can investigate your issue


Proxy object destroys inheritance feature

The tests worked great when the objects just had Elasticsearch::Model::Naming methods included, but that's not how Elasticsearch is used in real life. In real life, the index_name and document_name are accessed through a proxy, and the tree search was not considering that. This adds tests to confirm the infinite loop that this caused, and then fixes it by checking against the proxy object's target.

Errors are unrelated to code changes.

Can this get merged soon. We are facing the same problem.

This would be great to get merged!

@jsmestad , @tmandke , @kayakyakr , sorry about the silence here... I still need to focus on other responsibilities currently, but we're scheduling some significant effort for June and onwards to re-focus on the elasticsearch-rails library again, and give it the much needed attention.

@kayakyakr Thanks very much for this pull request. Is there any chance you can create a test that uses the actual library code? I'm afraid that if anything changes in the code base, this test will erroneously continue to pass because of the use of the ::ProxiedModel

I made some comments in the spec why that wasn't possible: testing against the model itself causes inheritance loops. I'm no longer working on a project that uses this, and so no longer have time set aside in my workday to work on finding a different workaround for this.


Migration to Spine 0.8.32-SNAPSHOT

This PR resolves the issues (a compilation errors and a broken tests) occurred after Spine version update.
Summary:

A part of the business logic fixed to pass the Spine validations.
Setup of the tests located under ...agregate.definition package fixed.

@dmytro-grankin PTAL.

@armiol PTAL.


[Suggestion] Let zen mode show all opened files in the top tab

VSCode Version: 1.16.0-Insiders
OS Version: Win10

If I want to switch to another opened file in zen mode, now I have to :

Quit zen mode
Switch to another file
Enter zen mode again

And actually there is lots of space in the top bar of zen mode.
So it may be more convinient if there are all opened files in the top tab of zen mode.

Please use the followign setting
"zenMode.hideTabs": false,


Replace pow with sqrt

Currently the only usage of pow on vector is with a constant argument of 0.5, for sqrt. This is inefficient and increases codesize.
This PR replaces the calls to pow with argument 0.5, with sqrt. Firmware PR coming soon
Proof GCC does not transform pow(x,0.5) into a sqrt instruction: https://godbolt.org/z/Cu7dQh

Coverage remained the same at 100.0% when pulling 96916d800bbdc9f4902e1fefde11170b0e9f5670 on no-pow into 22bf63cb714156eafd8a6d3822b903eba4980a8a on master.

Good find! how much flash did you safe? 😉

Maybe it's worse since more things inline now...


Upgrade to magento 2.0

any way you can upgrade this extension make it available for magento 2.0?

Hi @eclipse3211
We are planning to release a separate Magento 2 extension in the next few months. Stay tuned!

@ngrices saw this was closed yesterday, but are you still planning on testing/releasing the M2 version in the near future? Thanks!

hi @robotarmies see my above comment. We will have a magento 2 extension available to the public in Q2 this year


[TW#23952] esp_pthread.h not C++ compatible

esp_pthread.h is missing the necessary extern "C"{ ... } wrapper to be compatible with C++. The result is link errors when you call its functions from C++ code, because the compiler generates mangled names.
(I'm assuming that IDF headers are all meant to be C++ compatible, since this is the only header I've had to use this workaround with.)
Workaround:
In your C++ source file, use
extern "C" {
#include "esp_pthread.h"
}

Thanks for your suggestion, we have fixed this problem, and it will be available soon.


Type imports cause parse errors

Before You File a Bug Report Please Confirm You Have Done The Following...

[X] I'm using eslint-plugin-svelte. (*.svelte file linting does not work with the parser alone. You should also use eslint-plugin-svelte with it.)
[X] I'm sure the problem is a parser problem. (If you are not sure, search for the issue in eslint-plugin-svelte repo and open the issue in eslint-plugin-svelte repo if there is no solution.
[X] I have tried restarting my IDE and the issue persists.
[X] I have updated to the latest version of the packages.

What version of ESLint are you using?
8.16.0
What version of eslint-plugin-svelte and  svelte-eslint-parser are you using?

<EMAIL_ADDRESS><EMAIL_ADDRESS>
What did you do?

Configuration
root: true
parser: '@typescript-eslint/parser'

extends:
  - eslint:recommended
  - plugin:@typescript-eslint/recommended
  - prettier

plugins:
  - svelte3
  - '@typescript-eslint'

overrides:
  - files: ['*.svelte']
    parser: svelte-eslint-parser

settings:
  svelte3/typescript: () => require('typescript')

rules:
  no-unused-vars:
    - error
    - argsIgnorePattern: '^_'
      varsIgnorePattern: '^_'
      caughtErrorsIgnorePattern: '^_'

  '@typescript-eslint/no-unused-vars':
    - error
    - argsIgnorePattern: '^_'
      varsIgnorePattern: '^_'
      caughtErrorsIgnorePattern: '^_'

parserOptions:
  sourceType: module
  ecmaVersion: 2020

env:
  browser: true
  es2017: true
  node: true

src/routes/index.svelte
<script lang="ts">
  import type { Thing } from '$lib';
</script>

src/lib/index.ts
export type Thing = string;

What did you expect to happen?
The lint should pass.
What actually happened?
$ pnpm lint

><EMAIL_ADDRESS>lint /mcve
> prettier --check --plugin-search-dir=. . && eslint .

Checking formatting...
All matched files use Prettier code style!

src/routes/index.svelte
  2:15  error  Parsing error: Unexpected token {

✖ 1 problem (1 error, 0 warnings)

 ELIFECYCLE  Command failed with exit code 1.

Link to Minimal Reproducible Example
https://github.com/Stonks3141/mcve
Additional comments
If I replace parser: svelte-eslint-parser with processor: svelte3/svelte3 in .eslintrc.yml, ESlint outputs this:
$ pnpm lint

><EMAIL_ADDRESS>lint /mcve
> prettier --check --plugin-search-dir=. . && eslint .

Checking formatting...
All matched files use Prettier code style!

src/routes/index.svelte
  0:0  error  Cannot read properties of undefined (reading 'ESNext')  TypeError

✖ 1 problem (1 error, 0 warnings)

 ELIFECYCLE  Command failed with exit code 1.

This error appears for all .svelte files.
The MCVE was made with pnpm init svelte and selecting Typescript, ESlint, and Prettier.

Don't use eslint-plugin-svelte3, use eslint-plugin-svelte instead.


fix cross compile from darwin arm64 to x86_64

fix #3715

Why hasn't this been merged yet? In 2.4.1 it's still impossible to cross-compile from ARM64 to x86_64.


[Bug] 大佬们，这个函数好像写的有问题，只能解析出来[BEGIN]到[DONE]中间的代码，然而基座模型最先输出的代码不是以[BEGIN]开头的。

Prerequisite

[X] I have searched Issues and Discussions but cannot get the expected help.
[X] The bug has not been fixed in the latest version.

Type
I'm evaluating with the officially supported tasks/models/datasets.
Environment
如题
Reproduces the problem - code/configuration sample
如题
Reproduces the problem - command or script
如题
Reproduces the problem - error message
如题
Other information
如题

Which function do you mean?


Multi context refactor

Signed-off-by: ashish<EMAIL_ADDRESS>Description
This PR fixes #
Notes for Reviewers
Signed commits

[ ] Yes, I signed my commits.

Rerunning build checks - https://github.com/meshery/meshery-linkerd/actions/runs/2477244582


File.ReadAsLinesSync silently does not read the whole of very large file on Linux

Generate a large file, piping something like the following into a file:
import 'dart:io';

main() {
	for (int i = 0; i < 1000 * 1000 * 1000; i++) {
	   print ("$i: Long line of text to pad the file");
	}
}

Run this until the file is ~2.5Gb
Then load the file back
main() {
   var lst = new File('BigListOfNumbers.log').readAsLinesSync();

   print (lst.length);
   print (lst.last);
}

On OS X this will fail with an error message
FileSystemException: readSync failed, path = 'BigListOfNumbers.log' (OS Error: Invalid argument, errno = 22)

On Linux this appears to read only part of the file. The number reported is less than number reported by running 'tail' on the same file.
If there is a size limit on the files that can be read, this should reported as an error, rather than silently failing. This has resulted in a bug that proved very difficult to locate.

The 2.5Gb size suggests a signed 32-bit integer overflow, and Dart doesn't have 32-bit integers, so the problem is likely in the OS glue code or maybe in not using the correct "big file aware" API (if one exists).

Can confirm this issue.

Can confirm this issue.


Compile chess ukernels at test time (rather than downloading)

Copy code from mlir-aie for generating ukernels with chess. Currently placed it in the build_tools/ci, we can consider making it part of core iree-amd-aie (with cmake) at later point (or now if people prefer).
Files copies:
chess-clang : copied directly (except from removal of commented out code)
mm.cc : copied directly
zero.cc : copied directly (I'm not sure we need this though)
xchesscc_wrapper: simplified to only support AIE2, and reduce compile time warnings
This PR also removes ukernels from run_matmul_test.sh, so that ukernels are only tested in cpu_comparison/run_test.sh. This is to reduce code complexity
The one risk I see with this PR is that we become more dependent on chess -- downloading the ukernel meant that we could use ukernels without needing the heavy vitis machinery to start with

CI's vitis doesn't seem to have the required headers. See CI failure message and

@makslevental this isn't a priority, but I'd be interested to hear your thoughts -- can we in theory compile ukernels with whatever vitis/chess there is on CI?

@makslevental this isn't a priority, but I'd be interested to hear your thoughts -- can we in theory compile ukernels with whatever vitis/chess there is on CI?

I made that tarball so I can just remake it with the headers (or I can just upload the headers).

chess-clang : copied directly (except from removal of commented out code)
mm.cc : copied directly
zero.cc : copied directly (I'm not sure we need this though)
xchesscc_wrapper: simplified to only support AIE2, and reduce compile time warnings

Please no bash scripts/wrappers around chess-clang whatever. No shade at you but I think that stuff is junk.
The right way to do this is to handle mm.cc just like we handle chess_intrinsic_wrappers.cpp ie compile it at the xclbin generation step. That moots the chess bash/wrappers junk and also means we can get apples to apples comparisons between Peano/Chess (because, presumably, we would then compile mm.cc with the chosen target compiler instead of always chess).

@makslevental this isn't a priority, but I'd be interested to hear your thoughts -- can we in theory compile ukernels with whatever vitis/chess there is on CI?

I made that tarball so I can just remake it with the headers (or I can just upload the headers).

chess-clang : copied directly (except from removal of commented out code)
mm.cc : copied directly
zero.cc : copied directly (I'm not sure we need this though)
xchesscc_wrapper: simplified to only support AIE2, and reduce compile time warnings

Please no bash scripts/wrappers around chess-clang whatever. No shade at you but I think that stuff is junk.
The right way to do this is to handle mm.cc just like we handle chess_intrinsic_wrappers.cpp ie compile it at the xclbin generation step. That moots the chess bash/wrappers junk and also means we can get apples to apples comparisons between Peano/Chess (because, presumably, we would then compile mm.cc with the chosen target compiler instead of always chess).

Thanks for the feedback, I'd prefer to do this without wrappers but wasn't sure how. I wasn't sure if mm.cc could be compiled with anything other than chess, it has pragmas

Thanks for the feedback, I'd prefer to do this without wrappers but wasn't sure how. I wasn't sure if mm.cc could be compiled with anything other than chess, it has pragmas

peano should be able to compile mm.cc - they null out all the chess specific stuff https://github.com/Xilinx/llvm-aie/blob/aie-public/clang/lib/Headers/aiebase_chess.h

By adding
#include "aiebase_chess.h"
#include "aiebase_typedefs.h"

to the top of mm.cc I got ... didn't get very far
/home/mlevental/dev_projects/iree_plus_amd_aie/llvm-aie/lib/clang/18/include/aiebase_typedefs.h:251:9: error: unknown type name '__acc32'
  251 | typedef __acc32 acc32;
      |         ^
/home/mlevental/dev_projects/iree_plus_amd_aie/llvm-aie/lib/clang/18/include/aiebase_typedefs.h:252:9: error: unknown type name '__acc64'
  252 | typedef __acc64 acc64;
      |         ^
/home/mlevental/dev_projects/iree_plus_amd_aie/llvm-aie/lib/clang/18/include/aiebase_typedefs.h:266:9: error: unknown type name '__accfloat'
  266 | typedef __accfloat accfloat;
      |         ^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:55:3: error: use of undeclared identifier 'event0'
   55 |   event0();
      |   ^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:80:53: error: no member named 'load_v' in namespace 'aie'
   80 |           aie::vector<T_in, MMUL::size_A> A0 = aie::load_v<MMUL::size_A>(pA1);
      |                                                ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:82:53: error: no member named 'load_v' in namespace 'aie'
   82 |           aie::vector<T_in, MMUL::size_A> A1 = aie::load_v<MMUL::size_A>(pA2);
      |                                                ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:84:53: error: no member named 'load_v' in namespace 'aie'
   84 |           aie::vector<T_in, MMUL::size_A> A2 = aie::load_v<MMUL::size_A>(pA3);
      |                                                ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:86:53: error: no member named 'load_v' in namespace 'aie'
   86 |           aie::vector<T_in, MMUL::size_A> A3 = aie::load_v<MMUL::size_A>(pA4);
      |                                                ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:88:53: error: no member named 'load_v' in namespace 'aie'
   88 |           aie::vector<T_in, MMUL::size_B> B0 = aie::load_v<MMUL::size_B>(pB1);
      |                                                ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:90:53: error: no member named 'load_v' in namespace 'aie'
   90 |           aie::vector<T_in, MMUL::size_B> B1 = aie::load_v<MMUL::size_B>(pB2);
      |                                                ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:92:53: error: no member named 'load_v' in namespace 'aie'
   92 |           aie::vector<T_in, MMUL::size_B> B2 = aie::load_v<MMUL::size_B>(pB3);
      |                                                ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:94:53: error: no member named 'load_v' in namespace 'aie'
   94 |           aie::vector<T_in, MMUL::size_B> B3 = aie::load_v<MMUL::size_B>(pB4);
      |                                                ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:98:20: error: no member named 'load_v' in namespace 'aie'
   98 |               aie::load_v<MMUL::size_C>(pC1);
      |               ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:100:20: error: no member named 'load_v' in namespace 'aie'
  100 |               aie::load_v<MMUL::size_C>(pC1 + MMUL::size_C * rowA);
      |               ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:102:20: error: no member named 'load_v' in namespace 'aie'
  102 |               aie::load_v<MMUL::size_C>(pC1 + 2 * MMUL::size_C * rowA);
      |               ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:104:20: error: no member named 'load_v' in namespace 'aie'
  104 |               aie::load_v<MMUL::size_C>(pC1 + 3 * MMUL::size_C * rowA);
      |               ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:107:20: error: no member named 'load_v' in namespace 'aie'
  107 |               aie::load_v<MMUL::size_C>(pC2);
      |               ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:109:20: error: no member named 'load_v' in namespace 'aie'
  109 |               aie::load_v<MMUL::size_C>(pC2 + MMUL::size_C * rowA);
      |               ~~~~~^
/home/mlevental/dev_projects/iree_plus_amd_aie/iree-amd-aie/third_party/mlir-aie/aie_kernels/mm.cc:111:20: error: no member named 'load_v' in namespace 'aie'
  111 |               aie::load_v<MMUL::size_C>(pC2 + 2 * MMUL::size_C * rowA);
      |               ~~~~~^
fatal error: too many errors emitted, stopping now [-ferror-limit=]

but supposedly they (peano) can compile all of mllib (whatever that is - the aieml/aie_api library???) so I think it's doable. It'll just take some fiddling (and some #ifdef __chess__ guards)

@makslevental @Abhishek-Varma
Abhishek needs to change the ukernel in CI, because it currently supports m=n=k=64 only, but for the new objectFifo pipeline we want a m=n=k=32 ukernel (because it aggressively double buffers, it can't fit m=n=k=64, or something like that).
Abhishek could probably "draft a new release":

but this isn't ideal long-term because (1) no one knows where these microkernels come from and (2) version control goes out the window (3) downloading files at test time can cause issues if internet/github down. So we'd like to push on the approach in this PR, where mm.o is created on-the-fly.
So @makslevental we might need your expertise making this happen, as we're not familiar with compiling with chess. Either we can get this PR over the line, or we can use an approach without a wrapper script if it's not too much work (I have gaps in understanding how this would work).

I can see the headers we need in ./Xilinx/Vitis/2023.1/aietools/include but that's not the Vitis path we're using in CI https://github.com/nod-ai/iree-amd-aie/blob/e9448a814190af0be3281a71d7502564ab93ba7c/build_tools/ci/run_matmul_test.sh#L111 (same for cpu_comparison)

The only headers we're missing are aie_api

Close in favor of https://github.com/nod-ai/iree-amd-aie/pull/617


Desktop: 2.4.0 beta1

Please note the changelog is auto-generated from the text changelog in the client repo.
@jnweiger @lefherz Please merge to staging, then let's have a look there

Fixup needed https://github.com/owncloud/owncloud.org/pull/1320


Fixed Exception StackTrace formatting to handle missing StackFrames

Resolves #102

:grey_exclamation: Build Aborted

The PR is not allowed to run in the CI yet

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: [Pull request #117 opened]

Reason: The PR is not allowed to run in the CI yet

Start Time: 2020-11-19T19:32:42.369+0000

Duration: 4 min 16 sec

Commit: 9421302c9e34040b9c5ffec609aff3daf56857c2

Steps errors
Expand to view the steps failures

Load a resource file from a shared library

Took 0 min 0 sec . View more details on here

Error signal

Took 0 min 0 sec . View more details on here
Description: githubPrCheckApproved: The PR is not allowed to run in the CI yet. (Only users with write permission

Log output
Expand to view the last 100 lines of log output

[2020-11-19T19:36:32.660Z] Cleaning workspace
[2020-11-19T19:36:32.685Z] Pruning obsolete local branches
[2020-11-19T19:36:33.289Z] Merging remotes/origin/master commit 9773308ef0743b86010e058b18f8f5a0a70cc5b1 into PR head commit 9421302c9e34040b9c5ffec609aff3daf56857c2
[2020-11-19T19:36:33.429Z] Merge succeeded, producing 9421302c9e34040b9c5ffec609aff3daf56857c2
[2020-11-19T19:36:33.429Z] Checking out Revision 9421302c9e34040b9c5ffec609aff3daf56857c2 (PR-117)
[2020-11-19T19:36:32.621Z]  > git config remote.origin.url git@github.com:elastic/ecs-dotnet.git # timeout=10
[2020-11-19T19:36:32.633Z]  > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
[2020-11-19T19:36:32.648Z]  > git config remote.origin.url git@github.com:elastic/ecs-dotnet.git # timeout=10
[2020-11-19T19:36:32.663Z]  > git rev-parse --verify HEAD # timeout=10
[2020-11-19T19:36:32.672Z] No valid HEAD. Skipping the resetting
[2020-11-19T19:36:32.673Z]  > git clean -fdx # timeout=10
[2020-11-19T19:36:32.691Z] Fetching upstream changes from git@github.com:elastic/ecs-dotnet.git
[2020-11-19T19:36:32.691Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2020-11-19T19:36:32.698Z]  > git fetch --no-tags --progress --prune -- git@github.com:elastic/ecs-dotnet.git +refs/pull/117/head:refs/remotes/origin/PR-117 +refs/heads/master:refs/remotes/origin/master # timeout=15
[2020-11-19T19:36:33.298Z]  > git config core.sparsecheckout # timeout=10
[2020-11-19T19:36:33.309Z]  > git checkout -f 9421302c9e34040b9c5ffec609aff3daf56857c2 # timeout=15
[2020-11-19T19:36:33.374Z]  > git remote # timeout=10
[2020-11-19T19:36:33.384Z]  > git config --get remote.origin.url # timeout=10
[2020-11-19T19:36:33.393Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2020-11-19T19:36:33.401Z]  > git merge 9773308ef0743b86010e058b18f8f5a0a70cc5b1 # timeout=10
[2020-11-19T19:36:33.414Z]  > git rev-parse HEAD^{commit} # timeout=10
[2020-11-19T19:36:33.433Z]  > git config core.sparsecheckout # timeout=10
[2020-11-19T19:36:33.437Z]  > git checkout -f 9421302c9e34040b9c5ffec609aff3daf56857c2 # timeout=15
[2020-11-19T19:36:37.137Z] Commit message: "Elastic.CommonSchema.NLog - Improving documentation"
[2020-11-19T19:36:37.137Z] First time build. Skipping changelog.
[2020-11-19T19:36:37.137Z] Cleaning workspace
[2020-11-19T19:36:37.140Z]  > git rev-parse --verify HEAD # timeout=10
[2020-11-19T19:36:37.147Z] Resetting working tree
[2020-11-19T19:36:37.148Z]  > git reset --hard # timeout=10
[2020-11-19T19:36:37.251Z]  > git clean -fdx # timeout=10
[2020-11-19T19:36:37.282Z]  > git --version # timeout=10
[2020-11-19T19:36:37.288Z]  > git --version # 'git version 2.17.1'
[2020-11-19T19:36:37.297Z] fatal: bad object 1405b404792fb9f8df6252e2e64826b468023313
[2020-11-19T19:36:38.167Z] Masking supported pattern matches of $JOB_GCS_BUCKET or $NOTIFY_TO
[2020-11-19T19:36:38.228Z] Timeout set to expire in 1 hr 0 min
[2020-11-19T19:36:38.254Z] The timestamps step is unnecessary when timestamps are enabled for all Pipeline builds.
[2020-11-19T19:36:38.550Z] [INFO] Number of builds to be searched 10
[2020-11-19T19:36:38.681Z] [INFO] 'shallow' is forced to be disabled when running on PullRequests
[2020-11-19T19:36:38.701Z] Running in /var/lib/jenkins/workspace/ent-dotnet_ecs-dotnet-mbp_PR-117/src/go.elastic.co/apm/ecs-dotnet
[2020-11-19T19:36:38.741Z] [INFO] gitCheckout: Checkout SCM PR-117 with some customisation.
[2020-11-19T19:36:38.769Z] [INFO] Override default checkout
[2020-11-19T19:36:38.823Z] Sleeping for 10 sec
[2020-11-19T19:36:48.955Z] using credential f6c7695a-671e-4f4f-a331-acdce44ff9ba
[2020-11-19T19:36:49.028Z] Cloning the remote Git repository
[2020-11-19T19:36:49.042Z] Cloning repository git@github.com:elastic/ecs-dotnet.git
[2020-11-19T19:36:49.073Z]  > git init /var/lib/jenkins/workspace/ent-dotnet_ecs-dotnet-mbp_PR-117/src/go.elastic.co/apm/ecs-dotnet # timeout=10
[2020-11-19T19:36:49.079Z] Fetching upstream changes from git@github.com:elastic/ecs-dotnet.git
[2020-11-19T19:36:49.079Z]  > git --version # timeout=10
[2020-11-19T19:36:49.091Z]  > git --version # 'git version 2.17.1'
[2020-11-19T19:36:49.092Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2020-11-19T19:36:49.096Z]  > git fetch --tags --progress -- git@github.com:elastic/ecs-dotnet.git +refs/heads/*:refs/remotes/origin/* # timeout=10
[2020-11-19T19:36:50.512Z] Checking out Revision 9421302c9e34040b9c5ffec609aff3daf56857c2 (origin/PR-117)
[2020-11-19T19:36:50.569Z] Commit message: "Elastic.CommonSchema.NLog - Improving documentation"
[2020-11-19T19:36:50.583Z]  > git --version # timeout=10
[2020-11-19T19:36:50.587Z]  > git --version # 'git version 2.17.1'
[2020-11-19T19:36:50.591Z] fatal: bad object 1405b404792fb9f8df6252e2e64826b468023313
[2020-11-19T19:36:49.892Z]  > git config remote.origin.url git@github.com:elastic/ecs-dotnet.git # timeout=10
[2020-11-19T19:36:49.898Z]  > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
[2020-11-19T19:36:49.914Z]  > git config remote.origin.url git@github.com:elastic/ecs-dotnet.git # timeout=10
[2020-11-19T19:36:49.921Z] Fetching upstream changes from git@github.com:elastic/ecs-dotnet.git
[2020-11-19T19:36:49.922Z] using GIT_SSH to set credentials GitHub user @elasticmachine SSH key
[2020-11-19T19:36:49.927Z]  > git fetch --tags --progress -- git@github.com:elastic/ecs-dotnet.git +refs/pull/117/head:refs/remotes/origin/PR-117 +refs/heads/master:refs/remotes/origin/master # timeout=10
[2020-11-19T19:36:50.508Z]  > git rev-parse origin/PR-117^{commit} # timeout=10
[2020-11-19T19:36:50.515Z]  > git config core.sparsecheckout # timeout=10
[2020-11-19T19:36:50.520Z]  > git checkout -f 9421302c9e34040b9c5ffec609aff3daf56857c2 # timeout=10
[2020-11-19T19:36:51.374Z] Masking supported pattern matches of $GIT_USERNAME or $GIT_PASSWORD
[2020-11-19T19:36:51.959Z] + git fetch<EMAIL_ADDRESS>+refs/pull/*/head:refs/remotes/origin/pr/*
[2020-11-19T19:36:52.579Z] Archiving artifacts
[2020-11-19T19:36:53.193Z] + git rev-parse HEAD
[2020-11-19T19:36:53.559Z] + git rev-parse HEAD
[2020-11-19T19:36:53.869Z] + git rev-parse origin/pr/117
[2020-11-19T19:36:53.917Z] [INFO] githubEnv: Found Git Build Cause: pr
[2020-11-19T19:36:54.166Z] Masking supported pattern matches of $GITHUB_TOKEN
[2020-11-19T19:36:55.005Z] [WARN] githubApiCall: The REST API call https://api.github.com/repos/elastic/ecs-dotnet/pulls/117/reviews return 0 elements
[2020-11-19T19:36:55.064Z] [INFO] githubPrCheckApproved: Title: Fixed Exception StackTrace formatting to handle missing StackFrames - User: snakefoot - Author Association: CONTRIBUTOR
[2020-11-19T19:36:55.406Z] ERROR: githubPrCheckApproved: The PR is not allowed to run in the CI yet
[2020-11-19T19:36:55.406Z] ERROR: githubPrCheckApproved: The PR is not allowed to run in the CI yet. (Only users with write permissions can do so.)
[2020-11-19T19:36:55.500Z] [INFO] Let's stop build #1. The PR is not allowed to run in the CI yet
[2020-11-19T19:36:55.527Z] Sleeping for 5 sec
[2020-11-19T19:36:56.740Z] Stage "Sanity checks" skipped due to earlier failure(s)
[2020-11-19T19:36:56.822Z] Stage "Parallel" skipped due to earlier failure(s)
[2020-11-19T19:36:56.910Z] Stage "Linux" skipped due to earlier failure(s)
[2020-11-19T19:36:56.924Z] Stage "Windows" skipped due to earlier failure(s)
[2020-11-19T19:36:56.964Z] Failed in branch Linux
[2020-11-19T19:36:57.008Z] Stage "Windows" skipped due to earlier failure(s)
[2020-11-19T19:36:57.120Z] Failed in branch Windows
[2020-11-19T19:36:57.262Z] Stage "Release" skipped due to earlier failure(s)
[2020-11-19T19:36:57.300Z] Stage "Release" skipped due to earlier failure(s)
[2020-11-19T19:36:57.394Z] Stage "Release" skipped due to earlier failure(s)
[2020-11-19T19:36:57.486Z] Stage "Release" skipped due to earlier failure(s)
[2020-11-19T19:36:57.570Z] Stage "Release" skipped due to earlier failure(s)
[2020-11-19T19:36:57.883Z] Running on Jenkins in /var/lib/jenkins/workspace/ent-dotnet_ecs-dotnet-mbp_PR-117
[2020-11-19T19:36:58.021Z] [INFO] getVaultSecret: Getting secrets
[2020-11-19T19:36:58.191Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-11-19T19:36:59.046Z] + chmod 755 generate-build-data.sh
[2020-11-19T19:36:59.046Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-dotnet/ecs-dotnet-mbp/PR-117/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-dotnet/ecs-dotnet-mbp/PR-117/runs/1 ABORTED 256403
[2020-11-19T19:36:59.296Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-dotnet/ecs-dotnet-mbp/PR-117/runs/1/steps/?limit=10000 -o steps-info.json
[2020-11-19T19:37:00.208Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-dotnet/ecs-dotnet-mbp/PR-117/runs/1/tests/?status=FAILED -o tests-errors.json
[2020-11-19T19:37:00.208Z] Retry 1/3 exited 22, retrying in 1 seconds...
[2020-11-19T19:37:01.119Z] Retry 2/3 exited 22, retrying in 2 seconds...


unexpected inference output 'prob' for model yolov4

the python client works with generating engine file from the example weights file downloaded in the yolov4-triton-client repo.
The python client expects input name inputs and output name detections whereas the c++ client expects data for input and prob for outputs.

error message:
(base) root@wave:/yolov4-triton-tensorrt/clients/c++/build# ./yolov4-triton-cpp-client --video=/Videos/sample_720p.mp4
Server address: localhost:8001
Video name: /Videos/sample_720p.mp4
Protocol:  grpc
Path to labels name:  ../coco.names
Created output prob
failed sending synchronous infer request: unexpected inference output 'prob' for model 'yolov4'
simply changing the names doesn't produce any detections. Has the yolov4.wts file downloaded from the google drive changed?

Yes if you are talking about weights in current yolov4-triton-tensorrt repo release. C++ client needs yolov4 wts weights (and resulting tensorrt binary)generated according to tensorrtx implementation, with following mapping in triton server config.pbtxt related to the deployed model
platform: "tensorrt_plan"
max_batch_size: 1
input {
  name: "data"
  data_type: TYPE_FP32
  dims: 3
  dims: 608
  dims: 608
}
output {
  name: "prob"
  data_type: TYPE_FP32
  dims: 7001
  dims: 1
  dims: 1
}
default_model_filename: "model.plan"

Thanks! @olibartfast one thing was the correct yolov4.wts file and the other was the config file.


feat: support math formula

fix: https://github.com/mckaywrigley/chatbot-ui/issues/1223 https://github.com/mckaywrigley/chatbot-ui/issues/1802

add rehypeKatex plugin to support latex syntax.
replace \(..\) and \[...\] to $...$ and $$...$$.

Due to remark-math only supporting the use of the $ symbol for inline math  and $$ for block math (see discussion at: https://github.com/remarkjs/remark-math/issues/39), we need to convert math notations from \(..\) to $...$ and from \[...\] to $$...$$ .
After:

Before

@usoonees I tried this approach for my project, don't think this is the way to go, there are too many corner-cases.
I think we should write a replacement for remark-math / mdast-util-math to handle them properly.

For example:

LLMs that use the bracket syntax will not escape $ in text, so $100 to $200 will be seen as a formula.
LLMs will produce \[ ... \] without new lines, and use syntax that is only allowed in block math inside. This is parsed as inline math, and gives a rendering error.
Math inside

Deeply

Nested

Lists

requires special handling, as
      $$
      \formula
$$

is not handled correctly.
Another concern, this project is kinda dead, if you consider the number of open PRs and lack of action. I used to host chatbot-ui for myself and friends, but am now working on Congusto Chat, which will soon reach feature parity, and has an ambitious roadmap, including code interpreter and artefacts.


Possible to filter out extensions?

Since people can be quite lazy when tagging their uploads, sometimes searching 'animated' isn't good enough. DanbooruDownloader for example has a feature when batch downloading that allows you to exclude tags.
Typing 'png|jpg|jpeg' in the 'Filter Extension' field with 'Exclude Mode' ticked will only download images without those extensions for example.
Is this also possible with Grabber?

Yes it's possible.
Depending on the source you use, it can even already be do-able from the search field. For example on Danbooru, you can simply search "some_tag -filetype:png" to remove all PNG images from the results.
If your source doesn't support it directly (or if on Danbooru for example you're limited to few search tags), you can use the post-filtering feature which allows to filter the results. Simply add "-filetype:png" to your post-filter and Grabber will remove all results with a PNG extension.

Thanks for the reply, seems to be working. Only problem I'm having now is that it's renaming all extensions to .webm which seems fine for gif and mp4, but it breaks swf for example. Is there a setting that could be causing that?

Moved to #1352


Fix crash when restoring script editor state

Fixes the crash when restoring the script editor state.
closes #28307
closes #28288

Thanks!


Exception wrapping

I've been trying to improve the exception handling of MPF/MC as I work—whenever I get a crash that requires me to scroll through the callstack or look in the logs, I update the Exception chain to try and get the meaningful error into the final callstack message.
I think we should do a 50.4 release soon, so making a PR for the changes I've got so far.

Nice


Avoid writing empty extensions object

Fixed a bug where an empty extensions object would get written if an ExtensibleObject only had unregistered extensions.
Before:

After:

Thanks @lilleyse!


IIS multiple response headers with the same name

I am using IIS Application Request Routing (ARR) with ModSecurity. ModSecurity is installed and configured with the OWASP Core rule set on Windows 2022. I am trying to improve the rule set by incorporating session based logic.
I am using this rule to extract the session cookie:
SecRule RESPONSE_HEADERS:/Set-Cookie2?/ "(?i:(wordpresspass_.*?|j?sessionid|(php)?sessid|(asp|jserv|jw)?session[-_]?(id)?|cf(id|token)|sid)=([^\s]+)\;\s?)" \
	"chain,phase:5,id:'981062',t:none,pass,nolog,capture, \
	setsid:%{TX.6}, \
	setvar:session.sessionid=%{TX.6}, \
	setvar:session.valid=1, \
	expirevar:session.valid=3600, \
	setvar:session.country_name=%{geo.country_name}"
	SecRule UNIQUE_ID "(.*)" "chain,t:none,t:sha1,t:hexEncode,capture, \
		setvar:session.csrf_token=%{TX.1}"
		SecRule REMOTE_ADDR "^(\d{1,3}\.\d{1,3}\.\d{1,3}\.)" "chain,capture,setvar:session.ip_block=%{tx.1}"
			SecRule REQUEST_HEADERS:User-Agent ".*" "t:none,t:sha1,t:hexEncode,setvar:session.ua=%{matched_var}"

The problem is, only the last Set-Cookie set by the application is found. I wrote a rule to help me determine how many Set-Cookie headers ModSecurity thinks I have by :
SecRule &RESPONSE_HEADERS:/Set-Cookie/ "@gt 1" "chain,phase:5,id:'981062',t:none,pass,nolog"
	SecRule RESPONSE_HEADERS:/Set-Cookie2?/ "((?s).*)" \
		"t:none,nolog,capture, \
		setsid:%{TX.1}, \
		setvar:session.sessionid=%{TX.1}, \
		setvar:session.valid=1"

When greater than 1 (“@gt 1”) is set, the session file never gets created, but if it “@gt 0” the last Set-Cookie header is saved.
This https://coreruleset.org/20230717/cve-2023-38199-multiple-content-type-headers/
States:
“This issue was initially reported to the CRS project on 24 March 2023 via the ModSecurity project. We quickly established that the CRS reference platform was not affected (ModSecurity 2.9.x on Apache 2.4). This is because the Apache web server merges identically titled header fields into one, separating any different values with commas (as described in the HTTP standard). The problem is that the situation is different on other platforms. While Apache (our reference platform) merges Content-Type headers together allowing this behavior to be detected, Nginx, for example, will retain each separate, identically named header: no merging occurs.”
This states: https://learn.microsoft.com/en-us/answers/questions/555576/setting-multiple-response-headers-with-same-name-d “IIS doesn't allow custom response headers with the same name.”
Chrome at the end user, does show all 4 Set-Cookie headers in Developer Tools.
What IIS, ARR and/or ModSecurity configuration setting will allow ModSecurity to see all Set-Cookie headers?

Hello @Rtw915 ,
As part of your description, you raise CVE-2023-38199. I don't believe that is relevant to your situation. The use case there is that multiple Content-Type request headers were, in fact, received by ModSecurity, and that the ambiguity could result in the wrong choice for body parsing.
In your case, you appear to be describing a situation where you believe you are sending multiple response headers but that ModSecurity only sees one.
Things you could check to make sure you are fully understanding your situation:

does the client only receive one Set-Cookie header?
does the value of that single response header correspond to only one header (e.g. multiple header values have not been combined into a single value that is a comma-separated list)

My guess is that your issue has nothing to do with ModSecurity, and that you are in fact only sending one Set-Cookie response header. While IIS does seem to enforce some restrictions on duplicate response header names, I would be surprised if there is no way to work around this (for Set-Cookie at least), A quick search found this:https://stackoverflow.com/questions/71348499/adding-and-manipulating-multiple-set-cookie-headers-in-iis  , which might be useful to you.

Thank you for clarifying the situation with CVE-2023-38199.
The client gets all four Set-Cookie headers. It looks to me each one is its own and not combined into a single value.

Can you confirm, does the screenshot look like it's only sending one Set-Cookie response or for individual ones?

I enabled the Failed Request Tracing role and configured it fallowing this guide https://learn.microsoft.com/en-us/troubleshoot/developer/webapps/iis/health-diagnostic-performance/troubleshoot-arr-using-frt-rules .
Failed Request Tracing screenshot:

I think this means IIS is seeing the headers correctly. Is my understanding correct?
In an attempt to rule out regex issues I made this test rule:
SecRule RESPONSE_HEADERS:/Set-Cookie/ "((?s).*)" \
	"chain,phase:5,id:'981062',t:none,pass,nolog,capture, \
	setsid:%{TX.1}, \
	setvar:session.sessionid=%{TX.1}, \
	setvar:session.valid=1, \
	expirevar:session.valid=3600"

This is what the session file contains:

I expected to see all four Set-Cookie headers, but I only see the last one.  Just to be clear, I'm trying to capture ASP.NET_SessionId value.
How can I further troubleshoot ?

Hi @Rtw915 ,
Usually the simplest way to view which response headers ModSecurity sees is to examine part F of the audit log.
A good way to better understand what is happening is to use ModSecurity's debug log (particularly at level 9). The log will provide information about the processing that has occurred for each rule that is executed.
Regarding your most recent 'test rule': Perhaps I've misunderstood you somehow, but you expect the actions of the rule to execute against the content of each of the four header values, right? In that case, note that the 2nd, and subsequent executions are overwriting the previous values. I.e for the active 'session' there is a single 'sessionid' -- and you are replacing the value every time you execute 'setvar'.

Hello @martinhsv,
Thank you for your help!
I enabled audit logging and was able to find the initial response.

I don't understand why the source port in part A is 80.  A source port of 80 seems wrong since it should be an empirical port.

The IP behind the obfuscated source address is my home's public IP (where I'm testing from).
Also, everything should be HTTPS, so why is the destination port 80?

In Part F I don't understand how ModSecurity is getting "HTTP/1.1 500 Internal Server Error," when from a client perspective everything seem to be working.
Lastly, there is only 1 Set-Cookie, so that explains that, but why?

I set the debug log at 9 like you suggested and was able to review. Honestly, I'm having a hard time understanding where to look. I was able to find the rule and this repeats 6 times throughout the log.
[11/Oct/2023:18:03:09.115187 --0400] [ATLPRDDMZARR071/sid#1de51866140][rid#1de52b9e6b0][/][4] Recipe: Invoking rule 1de51aea248; [file "C:\Program Files\ModSecurity IIS\owasp-modsecurity-crs/rules/modsecurity_crs_15_customrules.conf"] [line "56"] [id "981062"].
[11/Oct/2023:18:03:09.115187 --0400] [ATLPRDDMZARR071/sid#1de51866140][rid#1de52b9e6b0][/][5] Rule 1de51aea248: SecRule "RESPONSE_HEADERS:Set-Cookie" "@rx ((?s).*)" "phase:5,auditlog,id:981062,t:none,pass,nolog,capture,setsid:%{TX.1},setvar:session.sessionid=%{TX.1},setvar:session.valid=1,expirevar:session.valid=3600"
[11/Oct/2023:18:03:09.115187 --0400] [ATLPRDDMZARR071/sid#1de51866140][rid#1de52b9e6b0][/][4] Rule returned 0.
[11/Oct/2023:18:03:09.115187 --0400] [ATLPRDDMZARR071/sid#1de51866140][rid#1de52b9e6b0][/][9] No match, not chained -> mode NEXT_RULE.

I'm going to configure a rule like this to help lessen the noise:
SecRule REMOTE_ADDR "@streq <IP_ADDRESS>" \
    phase:1,nolog,pass,ctl:debugLogLevel=9

Hi @Rtw915 ,
If you add two Set-Cookie headers in the web page itself (using the technique that I linked to in my first reply), do you still only see one Set-Cookie header in ModSecurity's part F output?
How about if you use that same technique, but for response headers with a response header names that are not in the 'well known header names' category? E.g. if you add two response headers both named 'MyCustomHeaderName'?
Based on recent inspection of the code, my guess is that in the first case you will still only see one Set-Cookie header, but that you probably will see two 'MyCustomHeaderName' headers.

Hello @martinhsv,
Sorry it took so long, I've been sick for a couple of weeks.
I was able to do as you asked. I added two Set-Cookie headers in the outbound rewrite rules on the origin server and only the second "mycookie2" showed up in ModSecurity's part F output. However, the user browser showed the 4 original cookies and both test cookies.
I then changed "RESPONSE_Set-Cookie" to "RESPONSE_MyCustomHeaderName" and both showed up in the user browser; however, only the second "MyCustomHeaderName" is in ModSecurity's part F output.
We are also stuck still trying to figure out what is causing the 500 logged in part F. The origin server and the ARR proxy server IIS logs and event logs don't show any 500's. Also the user browser dev tools don't show any 500 either.


fix: rename spec-ref-validation to spec-strict-refs

What/Why/How?
Rename spec-ref-validation to spec-strict-refs
Reference
Testing
Screenshots (optional)
Check yourself

[x] Code is linted
[ ] Tested with redoc/reference-docs/workflows
[x] All new/updated code is covered with tests

Security

[x] Security impact of change has been considered
[x] Code follows company security practices and guidelines

The PR title and description is invalid. In code it spec-strict-refs but in descriptions its spec-restrict-refs.

The PR title and description is invalid. In code it spec-strict-refs but in descriptions its spec-restrict-refs.

Fixed, Thank you

It works for me. Thanks!


Share configuration

Currently, we have a bunch of unit tests. Each one of them involves setting up and tearing down the database including initialization of SqlSessionFactory. However, since we use XML configuration heavily. Each time, we initialize SqlSessionFactory, it needs to parse a lot of XML documents and uses a lot of memory. From how XMLConfigBuilder is setup, I see that it would be quite difficult to share the configuration as environment, properties, data source, transaction factory may be different. I wonder whether it is possible to make it such that we can construct a base configuration purely for parsing XML config and then use the result of that to configure SqlSessionFactory instead. That way it does not have to re-parse XML documents again and again at the beginning of every test.
Thanks

I don't know why are you do like this, but I think you can integration Springframework to resolve this problem. Use Springframework's unit-test ability.

This should be fixed in 3.4.2 via #867 .


evaluator: cycle error incorrectly reported for non-concrete field

What version of CUE are you using (cue version)?

$ cue version
cue version v0.0.0-20220825081001-d3c14a684375

       -compiler gc
     CGO_ENABLED 1
          GOARCH arm64
            GOOS linux
             vcs git
    vcs.revision d3c14a68437588b9dbdec9cb3919f7aafec58111
        vcs.time 2022-08-25T08:10:01Z
    vcs.modified false

Does this issue reproduce with the latest release?
Yes
What did you do?
# This should fail with an error about a and b being
# incomplete
! exec cue export x.cue
stderr 'a: incomplete value'
stderr 'b: incomplete value'

-- x.cue --
x: 200
x: y + 100
y: x - 100

a: b + 100
b: a - 100

What did you expect to see?
Passing test.
What did you see instead?
# This should fail with an error about a and b being
# incomplete (0.014s)
> ! exec cue export x.cue
[stderr]
cycle error:
    ./x.cue:5:4
[exit status 1]
> stderr 'a: incomplete value'
FAIL: /tmp/testscript4050841429/repro.txtar/script.txt:4: no match for `a: incomplete value` found in stderr

Related to #1884.
cc @mpvl

Cycle errors (not to be confused with structural cycles) are a kind of incomplete error. So this is correct. The spec mentions that cycles can be broken.
So this is mostly a documentation / presentation thing.

Also porting the example from #1778 which I totally forgot I raised with @rogpeppe before:
What did you do?
# Below, a.cue and b.cue are identical, but for the order
# of _t and r.

Passing test
What did you see instead?
# Below, a.cue and b.cue are identical, but for the order
# of _t and r.
# With _t before r (0.013s)
# With r before _t (0.011s)
> ! exec cue export b.cue
[stderr]
invalid interpolation: cycle error:
    ./b.cue:1:4
[exit status 1]
> stderr 'non-concrete value string'
FAIL: /tmp/testscript937876582/repro.txtar/script.txtar:10: no match for `non-concrete value string` found in stderr

The error message here is also not ideal because it doesn't say exactly what was non-concrete.


Icon and splashscreen - RC3

While building ionic-conference-app , it builds with default icons and splash screen not the one with resources folder. It creates res folder with custom icons while in platform/android/res -- > default cordova icons are there...
I'm facing this in my own app to so does this demo app of yours.
 I am going to close this issue as a duplicate of https://github.com/driftyco/ionic-cli/issues/1683. Thanks!