Jest building runtime can instantiate a module twice when mocks reuse production utilities.

Do you want to request a feature or report a bug?
Bug
What is the current behavior?
I have a module that connects to a database that I mock using jest. I also have a set of production utilities that uses a value exported from the database module. Finally, my mock implementation of the database module provides some test helpers by importing and using the production utilities.
Depending on the order in which I set up my imports I can get jest to build my mock module twice such that my test and my production utilities call two different versions of the mock. When that mock has a singleton instance (e.g. an in memory facade on a database) this means that I have two copies of the database making for very unexpected results (e.g. the utility writes to it's database but my test does not see any data written).
This happens specifically when the mock module is imported before the utilities which causes jest to then import the mock module again. It can be worked around by importing the utility before importing the mock module directly.
**If the current behavior is a bug, please provide the steps to reproduce and
See https://github.com/lewchuk/jest-import-bug. There are two test suites which have the same tests but a different import order. It also includes a console log of the stack trace when the mock module is imported. When it is broken the stack trace shows the double import:
at Object.<anonymous> (/Users/lewchuk/jest-import-bug/src/__mocks__/toBeMocked.js:5:14)
          at Runtime._execModule (/Users/lewchuk/jest-import-bug/node_modules/jest-runtime/build/index.js:448:13)
          at Runtime.requireMock (/Users/lewchuk/jest-import-bug/node_modules/jest-runtime/build/index.js:322:12)
          at Runtime.requireModuleOrMock (/Users/lewchuk/jest-import-bug/node_modules/jest-runtime/build/index.js:335:19)
          at Object.<anonymous> (/Users/lewchuk/jest-import-bug/src/utilUsingMock.js:1:1)
          at Runtime._execModule (/Users/lewchuk/jest-import-bug/node_modules/jest-runtime/build/index.js:448:13)
          at Runtime.requireModule (/Users/lewchuk/jest-import-bug/node_modules/jest-runtime/build/index.js:265:14)
          at Runtime.requireModuleOrMock (/Users/lewchuk/jest-import-bug/node_modules/jest-runtime/build/index.js:337:19)
          at Object.<anonymous> (/Users/lewchuk/jest-import-bug/src/__mocks__/toBeMocked.js:1:1)
          at Runtime._execModule (/Users/lewchuk/jest-import-bug/node_modules/jest-runtime/build/index.js:448:13)
          at Runtime.requireMock (/Users/lewchuk/jest-import-bug/node_modules/jest-runtime/build/index.js:322:12)
          at Runtime.requireModuleOrMock (/Users/lewchuk/jest-import-bug/node_modules/jest-runtime/build/index.js:335:19)
          at Object.<anonymous> (/Users/lewchuk/jest-import-bug/__tests__/testingUtil-broken.js:1:1)

What is the expected behavior?
When jest builds the runtime for a test suite it should only build a single instance of any imported module.
Please provide your exact Jest configuration
See linked github repo.
Run npx envinfo --preset jest in your project directory and paste the
results here
npx: installed 1 in 7.063s

  System:
    OS: macOS Sierra 10.12.6
    CPU: x64 Intel(R) Core(TM) i7-6567U CPU @ 3.30GHz
  Binaries:
    Node: 8.9.4
    Yarn: 1.5.1
    npm: 5.6.0
  npmPackages:
    jest:
      wanted: ^22.4.3
      installed: 22.4.3

I've thought some more about this and understand that this issue is basically a circular dependency created by the mocking process. If this behavior is not something that jest wants to support, I think a reasonable alternative resolution would be for the jest runtime to throw an error when it encounters this situation so that people don't accidentally get into this situation, especially since static analysis tools like linters won't catch this import cycle.


fix: Dark mode borders and scrollbars

This PR fixes the border colours around the top bars, left sidebar and the scrollbar styling when in dark mode.

🚀 Thank you for your contribution! 🚀


Class should be in new line

Hi @pravynandas
could you share an example please?

Closing as there is no details


Slide up and slide down animation

Hello.
After researching for a while I can't discover how to add an animation to slide up and slide down.
Like this accordion: https://codepen.io/anon/pen/mGgRRr
Can you please help me?

Gidday @rgllm,
While we do publish some basic boilerplate styles to help get people started, animation and transitions fall oustide the scope of this project - we deliberately leave these details up to the user to implement. I've not personally attempted this kind of sliding animation with our accordion because of the reflow issues it would likely cause on long webpages. But if I were to try, I would probably use this library in combination with a 'FLIP' animation library (like this one, for example). Also keep in mind that you can leverage the className and hideBodyClassName props in order to apply different styles to your accordion components when they are open or closed.
Sorry I can't be of more help @rgllm. I'm going to close this issue, but I'm happy to discuss further :)


Update the result field after loading the page

I did not test this.

Thanks!  I'm going to close in favor of #6, which does the same thing.


Import Art from React Native Community Art

Fixes oblador/react-native-progress#158
Fixes oblador/react-native-progress#161
Fixes oblador/react-native-progress#127

@felippepuhle Fixed the issue with the tests. I can update the examples too once this change is published to npm...

@oblador can we merge this and release a new version?

It's out in 4.0.0


[pubsub] Use opentelemetry::trace::kIsRootSpanKey when we make the next otel oldest deps upgrade

In the async batch publish, we create a root span using the string instead of the constant, since it does not exist in the oldest deps.
https://github.com/googleapis/google-cloud-cpp/pull/13285/files#diff-3e8aefbb8885925f3b6c48b310f1fe9e031aa24315c084fa04cfc13ea5bd86f6R63-R66
Current:
opentelemetry::context::Context root_context;
  // TODO(https://github.com/googleapis/google-cloud-cpp/issues/13287): Use the constant instead of the string.
  root_context.SetValue(/*opentelemetry::trace::kIsRootSpanKey=*/"is_root_span", true);
  options.parent = root_context;

Suggestion:
  root_context =
      root_context.SetValue(opentelemetry::trace::kIsRootSpanKey, true);
  options.parent = root_context;

[ ] /*sc::kNetworkTransport=*/"network.transport"
[ ] /*sc::kHttpRequestMethod=*/"http.request.method"
[ ] /*sc::kUrlFull=*/"url.full"
[ ] /*sc::kServerAddress=*/"server.address"
[ ] /*sc::kServerPort=*/"server.port"
[ ] /*sc::kClientAddress=*/"client.address"
[ ] /*sc::kClientPort=*/"client.port"

There are a few more of these semantic convention names we would like to update when possible. Can we have this issue encompass all of them? A checklist would be cool.

Yeah we should add the issue in the code too

We could add comments with issue in the code instead?

The code is already ugly, let's not make it fugly. It will be sufficient to have a list of strings to find and replace. I started a checklist for the semantic conventions I reverted in #12992.


Move base map layer creation into a service

cc @thirystheory
this makes it possible to restore the default base map layers again see also: https://github.com/boundlessgeo/sdk-apps/commit/d551ca410b3d654c10311fda9d621287bfa329be

Coverage increased (+0.06%) to 70.372% when pulling 4de8bc94f9f7decec1d6e2a0186a54a80420bb1d on basemap-config into 27f74ae96918df2154d50be9d3009a8c1b80b72b on master.


added rxvt

How about urxvt?  Is is the modern Unicode version or is that no longer accurate: http://software.schmorp.de/pkg/rxvt-unicode.html  I saw some comments on the sourceforge issues list that said the urxvt.sourceforge.net project was the "legacy" one.  Does urxvt have two main versions, in practice, that we should link to both, or just one?

A maintainer of the rxvt project remarked that rxvt is a "legacy" project. It also doesn't seem to have had a release in 14 years. I believe it would be appropriate to mention rxvt-unicode only.

Hi there @ri0t. Users @leighklotz & @kaytwo. Please do consider to make appropriate amendments. Thanks!

Agreed, @kaytwo. I think you can merge this now, @k4m4 :)

Hey, thank you for the amendment. Please resolve conflicts with master. Thanks again!


custom_matcher: Test the custom_match effectively

The existing test is using NeverMatch matcher which always return false for match() method. So, we didn't really test the matching behavior for custom_match field/mechanism. In this PR:

Implement a new custom_matcher and factory to do simple string comparison matching.
Test custom_match with matcher/factory above for the expected result that result.on_match_.has_value() is true

Signed-off-by: Tianyu Xia<EMAIL_ADDRESS>

/assign-from https://github.com/orgs/envoyproxy/teams/first-pass-reviewers

/assign-from @envoyproxy/teams/first-pass-reviewers

/assign-from @envoyproxy/first-pass-reviewers

/assign-from @envoyproxy/envoy-maintainers


Are SBOM affected statuses determined only based on the CSAF files?

Hi guys, this is not to report a bug but more a way of asking a question.
Context:

I upload the sbom  quarkus-bom-3.2.11.Final-redhat-00001.json
Then I Upload the Red Hat Advisory  rhsa-2024-2705.json

As a result if I hit GET /api/v1/vulnerability/CVE-2024-29025 then I get a response that indicates that my sbom  quarkus-bom-3.2.11.Final-redhat-00001.json has a status not_affected and also a set of purls that indicate which are the packages fixed | not_affected. Everything good here.
Question:

I upload the sbom  [quarkus-bom-3.2.11.Final-redhat-00001.json](https://github.com/user-
Then, rather than uploading the red hat CSAF file, i upload the CVE advisory file from https://github.com/CVEProject/cvelistV5/blob/main/cves/2024/29xxx/CVE-2024-29025.json

Finally I hit GET /api/v1/vulnerability/CVE-2024-29025 then I get the following response:
{
    "normative": true,
    "identifier": "CVE-2024-29025",
    "title": "Netty HttpPostRequestDecoder can OOM",
    "description": "Netty is an asynchronous event-driven network application framework for rapid development of maintainable high performance protocol servers & clients. The `HttpPostRequestDecoder` can be tricked to accumulate data. While the decoder can store items on the disk if configured so, there are no limits to the number of fields the form can have, an attacher can send a chunked post consisting of many small fields that will be accumulated in the `bodyListHttpData` list. The decoder cumulates bytes in the `undecodedChunk` buffer until it can decode a field, this field can cumulate data without limits. This vulnerability is fixed in 4.1.108.Final.",
    "published": "2024-03-25T20:09:35.156Z",
    "modified": "2024-08-02T01:03:51.668Z",
    "cwe": "CWE-770",
    "advisories": [
        {
            "uuid": "urn:uuid:380d8884-34dd-49fd-ab72-6d542c311811",
            "identifier": "CVE-2024-29025",
            "hashes": [
                "sha256:0eeff878f4aac32eda2433723aaf2a5e4be8b40e5b2fd0424f48713838e1425e",
                "sha384:89d8687316504fa41638cbb2546a583005bd8ea6a34b336a8fceab655d3c59c51eeee17746786a9c7fe5cdae0c9357ac",
                "sha512:f825b1f3560043924ca06ded08025b65adee01db4691fe3214c1308e94a214b1b8fa5e2f7ab1fac9c4c8e358d61791adbc7b46ebd97934f13013e855fe66c867"
            ],
            "issuer": {
                "id": "b2e53d5f-8371-47a1-b320-a5e0b0054f72",
                "name": "GitHub_M"
            },
            "published": "2024-03-25T20:09:35.156Z",
            "modified": "2024-08-02T01:03:51.668Z",
            "title": "Netty HttpPostRequestDecoder can OOM",
            "labels": {
                "type": "cve"
            },
            "cvss3_scores": [],
            "purls": {},
            "sboms": []
        }
    ]
}

The previous response indicates that there are zero sboms affected by the vulnerability.
Here is the question, is the importing of red hat csaf files the only way of determining sbom statuses and package statuses?
If I import my company sboms and then only import all the advisories from the CVE project, is it expected to have some sboms with statuses (affected, not affected, etc)?
I am asking because in my mind, an external company would not decide to import the red hat csaf files and for me, personally, it should be enough (even if not perfect) to import advisories from the CVE project or other open sources not strongly related to red hat advisories. WDYT guys?

The CVE will generally be very low value, and perhaps no include actual pURLs or CPEs, so difficult to link to thinks.
One step better would be the Netty project producing GHSAs or OSVs that give more/better details.
The RHT CSAF files purely give statuses within the context of the Red Hat product catalog.
So, ingesting a CSAF from Red Hat will tell you nothing of import about netty if you're using it in a vanilla Spring app or such.
The user will need to ingest advisories relevant to their usage. If they use Quarkus, then our CSAFs. If they use something else, then they need some other advisories.
And generally, CVEs aren't the final answer, just a starting point that other people build advisories from.

Recent changes to the CVE ingestioning may help determine affected SBOMs, even with RHT CSAF files.

It may be useful to error on the side of returning potential false positives, to at least alert the user to possibility of the packages being affected. This would be a signal to enable the importers relevant to their SBOM's rather than just the CVE's.
But when you consider in this particular example that the affected packages are vague, it prompts the question of which "netty" packages we should return as "possibly affected"? There are lots of packages with "netty" in their name, and relatively few named "netty", e.g.

netty-common
netty-codec
netty-codec-socks
netty-codec-http2
netty-buffer
netty-resolver
grpc-netty
netty-transport-native-epoll
netty-incubator-transport-native-io_uring
quarkus-netty
quarkus-netty-deployment
netty-i-think-you-get-the-idea

Should we return all of them with "netty" in the name or some subset? By what rule[s]?


chore: Deprecate unused RuntimeResolution field

Fixes https://github.com/argoproj/argo/issues/4102

@dtaniwaki Care to comment?

Thanks for your comments @dtaniwaki

Thanks all, for addressing this


Adds elasticsearch6 for testcontainers tests

see https://github.com/openzipkin/zipkin/issues/1676

will do all the bookeeping (add to quay, dockerhub)


Fiddle with env vars to fix the release to maven central

TL;DR
Fiddle with env vars in the settings.xml and also log the content of the username to confirm that the env vars have been set in the settings.xml
Type

[X] Bug Fix
[ ] Feature
[ ] Plugin

Are all requirements met?

[x] Code completed
[ ] Smoke tested
[ ] Unit tests added
[ ] Code documentation added
[ ] Any pending items have an associated Issue

Complete description
How did you fix the bug, make the feature etc. Link to any design docs etc
Tracking Issue
NA
fixes https://github.com/flyteorg/flyte/issues/
Follow-up issue
NA

I will be merging PR without reviewers until I make it release the artifacts to maven central


Sensitive information shown on the log details of 'LOGIN-USER' operation

As the admin of the DMP, I am now able to see everyone's pwd and totp! 1) pwd shouldn't appear in the Details of LOGIN-USER; 2) should we encrypt the pwd when storing them in the dmp?

@kai87  password is hashed and salted. We dont (never) store plain password at server. Only hash of the password is stored.
When checking password whether the user input is correct we just compare the hash.
This logging showing the plaintext of password is probably due to an implementation issue from @Siyao that he takes the password input from the client-side and directly store it onto log collection. Siyao needs to fix it ASAP.

@wsy19961129 fix it ASAP :dagger:


[Cloud Driver]Huawei CCE node operating system options are mismatched

The operating system options in Huawei CCE cloud driver are EularOS 2.2 and EularOS 2.3 in Rancher UI. But EularOS 2.2 and CentOS 7.4 are shown in Huawei cloud UI. We need to update our options and keep sync with Huawei public cloud.

@orangedeng @loganhz is there a way to avoid hard coding this and being able to pull it directly from  CCE?

Confirmed with @orangedeng . There is no API we can use for now.

@loganhz I do not have testing credentials to validate Huawei. Could we assign to someone with such credentials or provide them to me privately?

Tested with rancher/rancher:master from Apr 16.
This version of Huawei CCE driver node os options are the same as Huawei public cloud.
Confirmed fixed.


A get method to current route with arguments (if any) in NavigatorState class.

Hi, Need a get method to current route with arguments (if any) in NavigatorState class. Valid use case of this maybe while receiving a notification we want to check user in on which screen and want to compare arguments to decide whether to show the notification or not.
Syntax for the required method
Route<dynamic> get currentRoute() {
            return _navigator != null && _navigator._history.last;
}

This method is tried and tested. It works fine but i need to change it in NavigatorState class to use it so if it can be shipped with sdk it will be helpful.

Hi @Faiyyaz
please take a look to the answers in this stackoverflow question,
also consider that some work is in progress for Navigator 2.0.
Does this solve the issue?
thank you

@iapicca No. None of the answers on that post match my requirements. Some of them returns just the current route name but i need to get the arguments passed too.
And some methods just give me that the route isCurrent or not

Hey what is the status on this issue.
Actually i have this use case again for building chat in my app

Changes to the method as per flutter v1.17 release
Route<dynamic> get currentRoute {
            return _navigator != null && _navigator._history.last.route;
}

@chunhtai

You should be able to just do ModalRoute.of(context) and look at the arguments in the settings there.


Function getHostByName returns empty string silently

Hi!
We upgraded Spinnaker recently to a version where Helm is >= 3.11.1 (where --enable-dns was added) and some pipelines started failing because helm renders ip: "". The k8s control plane returns this error for IPs with the empty string value:
spec.template.spec.hostAliases.ip: Invalid value: "": must be valid IP address

This is misleading for us because the actual error is using getHostByName without passing --enable-dns on the helm template command. Could helm panic instead of returning the empty string here?
getHostByName
	// When DNS lookups are not enabled override the sprig function and return
	// an empty string.
	if !e.EnableDNS {
		funcMap["getHostByName"] = func(name string) string {
			return ""
		}
	}

Thanks in advance!

The reason for the --enable-dns flag, is that a malicious chart could use DNS exfiltration to export secrets (upon templating) if DNS queries are allowed.
Helm implemented the above change (the --enable-dns flag) when this security vulnerability was realized: https://github.com/helm/helm/security/advisories/GHSA-pwcw-6f5g-gxf8
I don't think there is a "correct" behavior when getHostByName is disabled like this. Panic, empty string or default/dummy value are all options. And keep in mind, Helm needs users to utilize --enable-dns only if they understand and trust the chart (and e.g. panicing with a "please use --enable-dns" message might prompt users to naively disable this protection).

Thanks, that makes sense. How about a panic with something like this:
DNS lookup is disabled by default. If you understand the security implications, check the documentation for how to enable DNS lookups

At least that would point users in the right direction, which is reviewing their DNS lookup use cases with an eye on the security aspect. That's better than investigating "why is k8s suddenly not accepting the same manifest of yesterday".
I'd be happy to open a PR for that. Would that be ok?

I think a panic would be too much, if you consider that every/any chart would now error upon template. Even if the chart does:
getHostByName("example.com") | default "<IP_ADDRESS>" # default/dummy value for templating

When this change was implemented, maintaining compatibility with as many users as possible would be the primary concern. I vaguely remember it was settled on an empty string to allow users to choose.
I do agree that the behaviour change is suprising. And I could imagine cost some time to debug. A middle ground might be to add a debug/log statement (which would appear with helm template --debug; need to double check that):
	if !e.EnableDNS {
		funcMap["getHostByName"] = func(name string) string {
		        e.Log.Debug("DNS lookup is disabled by default. If you understand the security implications, check the documentation for how to enable DNS lookups")
			return ""
		}
	}

And one more thought. I guess Spinnaker is deploying Helm charts via something like: helm template | kubectl apply -f - (argocd does something similar, iirc). Rather than using Helm natively to manage the Kubernetes API objects.
I'll raise this as a point of discussion, that we have users relying on helm template for production workflows (rather than just e.g. to manually eyeball produced yaml)

And oops, I misremembered. Looking at the code, Helm disables DNS for both template and install/upgrade/rollback. Certainly panicwould not be appropriate forinstall/upgrade/rollback.

@gjenkins8 Do you think we can get this through?


No named parameter with the name 'thumbVisibility'.         thumbVisibility: widget.itemsPerPage == null ? false : true,

Salam @omar-hadek , mar7ba,
Maybe issue #70 can help you with this issue?
Otherwise, maybe I can help myself if you send your flutter, dart and search_choices plugin versions.

@lcuis  Chokran
i've tried the latest version
search_choices: ^2.1.1
--------Flutter ----------------------------------
Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel stable, 2.10.5, on Microsoft Windows [Version 10.0.19044.2006], locale en-US)
[√] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1)
[√] Chrome - develop for the web
[√] Visual Studio - develop for Windows (Visual Studio Community 2022 17.3.1)
[√] Android Studio (version 2021.1)
[√] VS Code (version 1.71.2)
[√] Connected device (3 available)
[√] HTTP Host Availability

Thanks for the details @omar-hadek ,
From #70 , I know there is a flutter version between 2.10.2 and 2.11.0 that forced some abrupt changes with this thumbVisibility parameter. From the issue you raised, it seems that flutter version 2.10.5 also has this issue.
It seems to relate to this commit and probably started with version 2.11.0 :
https://github.com/flutter/flutter/commit/03da339ffe1fa1f45beac136725cb6560514c67e
The options I see for you are the following:
a) Upgrade your flutter version to at least 2.11.0 . I use 3.0.5 with success.
b) Or fix search_choices version to 2.0.17 .
c) Or fork search_choices and change the only use of thumbVisibility in search_choices.dart into isAlwaysShown .
I would recommend option a) and I am sorry for the inconvenience.

Closing. Feel free to reopen if needed.


fixed Vue.js compilation error by updating Vue.js version

Because of this bug in version 2.4.2 of Vue.js the compilation of Vue app was broken. I have fixed it by updating the package.
CLA

[x] I have signed the CLA; if other committers are in the commit history, they have signed the CLA as well

Thanks @Serhiy-Shekhovtsov !


Issue 668/documentation improvements

Closes: https://github.com/oceanprotocol/ocean.js/issues/668

not sure why Travis would run into issues for only markdown file changes. Maybe try another merge of main so we get a new build run?
Other than that this is all way better than what we have right now so let's merge it afterwards.

@kremalicious Thank you


Cilium connectivity test fails if any cilium pod in the cluster is in a non-pending state

Is there an existing issue for this?

[x] I have searched the existing issues

Version
equal or higher than v1.16.4 and lower than v1.17.0
What happened?
Running cilium connectivity test in a larger cluster with frequent node churn is almost impossible b/c it appears the test will fail to start if any of the cilium agent pods are in a pending, initializing, or terminating state:
Unable to detect Cilium version, assuming v1.15.3 for connectivity tests: unable to parse cilium version on pod "cilium-g6zh2": unable to fetch cilium version on pod "cilium-g6zh2": error with exec request (pod=cilium/cilium-g6zh2, container=cilium-agent): error dialing backend: remote error: tls: internal error
connectivity test failed: failed to fetch cilium runtime config: error with exec request (pod=cilium/cilium-g6zh2, container=cilium-agent): error dialing backend: remote error: tls: internal erro

It doesn't seem like the test should have such strict requirements as these prevents the usage of this debugging tool in anything but small or completely stable clusters.
This seems particularly curious as cilium version runs fine even if all the cilium agents are not running.
How can we reproduce the issue?

Install cilum
Create a large cluster with frequent node churn
Run cilium connectivity test

Cilium Version
cilium-cli: 0.16.4 compiled with go1.22.2 on linux/amd64
cilium image (default): v1.15.3
cilium image (stable): v1.16.4
cilium image (running): 1.16.3

Kernel Version
Linux ip-10-0-176-92.us-east-2.compute.internal 6.1.115 #1 SMP Fri Nov 15 19:15:45 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
Kubernetes Version
Client Version: v1.29.4
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.10-eks-7f9249a

Regression
No response
Sysdump
No response
Relevant log output

Anything else?
No response
Cilium Users Document

[x] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

[x] I agree to follow this project's Code of Conduct

Thanks for the report. I agree that I think it makes sense to make the connectivity test resilient to this condition. I would welcome proposals to improve the behaviour in this case.


Fix #69 - as suggested on https://github.com/eth-cscs/manta/issues/69…

…#issuecomment-2116955733
Just added the change suggested by @Masber

LGTM!


Thumbnails for "Continue watching" don't show

Issue description
When i begin watching a series/movie that was added with the themoviedb api it doesn't show a thumbnail in the "continue watching" section. Thumbnails in other sections work fine. Self-added thumbnails show up, too. The server and client are both online, streama doesn't report errors. I set up streama to use a mariadb which appears to work just fine. I don't think this is related to #455 .
Steps to Reproduce

Launch streama
Add a valid themoviedb api key
Add a series/movie with data from themoviedb
Begin watching it (port 8080) and go back to the menu

Expected Behaviour
Streama should show thumbnails for series/movies added with the api
Actual Behaviour
They're missing
Environment Information

Operating System: Arch Linux x64
Streama version: v1.6.0-RC9

Picture

+1. This is also the case for the "Recommended for You" section.

Hey, @f1reflyyyylmao  add me in discord: Wander#2568
for fix

Please send me your discord name#tag please.

I don't have a discord account and, duo to their privacy policy I won't create one. Why don't you write your fix in the wiki or better create a pull request to fix it?

@wanderfyx If you have a fix, discord isn't the best place for it. Please update this issue or create a PR.

I do not know very well how I did the correction, but I remember doing something that out of nothing all miniature came back to appear.

Just try to describe as good as you can what you did to make it work again. If you're uncomfortable with that I'm sure we can find a way to speak and I'll write it up if it works.

After upgrading to the newest streama version this issue is resolved for me, @jguerinet please verify if it works for you too

RC10 fixes the thumbnails in the "Continue Watching" section, but the ones in the "Recommendations for You" are unfortunately still missing.

How can i make streama show the "recommended for you" section? It's not being displayed on my dash..

In the "Hidden dash sections" within the admin settings, do you have recommends? If so, remove that and save.

No, my "hidden dash sections" is empty. However, i just noticed that i get a ton of sql errors because a foreign key restraint appears to fail, so there's probably something broken with my sql database. I'll delete the whole thing and report again when i managed to get it to run without errors

This is the original issue. After looking it up, it looks like the Recommendations section only shows up if you  have chosen a favorite genre (Profile icon in top right corner -> user settings -> Choose a genre that has at least 1 video associated to it).
Make sure to refresh the browser once you do this.

After setting some favorite genres i still don't get any recomendations despite having some movies for most of them in my library

Hm, I'm not sure what is happening then. Maybe @dularion can shed some light on why the recommended section isn't showing up?

I already fixed it in my reply, i guess github just didn't send another email to you? It works now and I can confirm that the thumbnails are missing.

Yeah, I don't believe GH sends emails for edited replies. Good to know that I'm not the only one!


Update readme ruby

Alternative install Ruby for Mac OS

Any problems here?

Any updates @Tony133

Hi, Sorry if I did not answer you but I did not have time these days


College Management System

Prerequisites

[X] I am a GSoC-22 Participant
[X] I checked to make sure that this issue has not already been filed

Description
The idea behind the project as
like we are in college we often see this scenario that how college maintains our data regarding every field.
Project description
We will have an interface in our console with a few options you just need to select. According to which just enter their sr number then this function will get invoked.
Refer this screenshots

Tech Stack
C++

Can assign only 3 issues at a time.

@manisha-27 I have solved all the assign issue plz assign my rest of issues

closed with #698


add stack, dockercloud/haproxy not work

Bug description
i add stack to local endpoint.
version: '2'
services:
  web:
    image: <IP_ADDRESS>:8082/inspiration:master-20
    ports:
      - 80
  lb:
    image: dockercloud/haproxy
    ports:
      - 80:80
    links:
      - web
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:rw

those haproxy log, log  show haproxy cannot links to web INFO:haproxy:Linked container:
INFO:haproxy:==========BEGIN==========
INFO:haproxy:Linked service:
INFO:haproxy:Linked container:
INFO:haproxy:HAProxy configuration:
global
  log <IP_ADDRESS> local0
  log <IP_ADDRESS> local1 notice
  log-send-hostname
  maxconn 4096
  pidfile /var/run/haproxy.pid
  user haproxy

while those from my local exec docker-compose, links is ok there
INFO:haproxy:==========BEGIN==========
INFO:haproxy:Linked service: inspiration_web
INFO:haproxy:Linked container: inspiration_web_1
INFO:haproxy:HAProxy configuration:

Hi there,
Are you deploying your stack via docker swarm or docker compose? As, from what I understand swarm does not support links and you should use network-level aliases instead.
Example: https://github.com/moby/moby/issues/30362#issuecomment-285391732

no swarm , i just deploy via docker compose, i use portainer to manage my local docker environment

Hi there,
Are you deploying your stack via docker swarm or docker compose? As, from what I understand swarm does not support links and you should use network-level aliases instead.
Example: moby/moby#30362 (comment)

no swarm , but local mode, i just use portainer to manage my local docker environment

Just to confirm; you are only having this issue when deploying your stack via Portainer and it works fine through the CLI on the same machine?
Also I would make sure that you have the latest version of docker-compose, as I read here there is an issue with the same symptoms as you, that occurs on older versions


[Network] Design and implement message handling behavior for Cancel

design and implement message handling behavior for Cancel. This is tricky since upon receiving Cancel, the client should abort current attempts of sending the Piece message.
Max's Estimate: ??, 6+h
Note that this is not a necessary component. The client can run fine without handling Cancel.

should be dependent on #10
may also needs a message queue


Machine mac-24 went dead from network monitoring website.

Machine mac-24 went dead from network monitoring website.

mac-24 is up and running fine, I can login remotely from my end.
flutter@flutter-devicelab-mac-24 ~ % uptime
9:13  up 15:27, 2 users, load averages: 1.06 1.14 1.12


Update cancellationex13.cs

The assignment to the field internalToken in the constructor appears to be superfluous.
Summary
I removed an unneeded line in the constructor with the intention to improve clarity. The internalToken field is assigned again in DoWork, where it is actually referenced.

Thank you, BIll


ic-light was completely disabled when updating ComfyUI torch to 2.4.0, now it doesn’t work even in version 1.5 =(

ic-light was completely disabled when updating ComfyUI torch to 2.4.0, now it doesn’t work even in version 1.5 =(
Error occurred when executing LoadAndApplyICLightUnet:

IC-Light: Could not patch calculate_weight

File "X:\ComfyUI_windows_portable\ComfyUI\execution.py", line 317, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\ComfyUI_windows_portable\ComfyUI\execution.py", line 192, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\ComfyUI_windows_portable\ComfyUI\execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)
File "X:\ComfyUI_windows_portable\ComfyUI\execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-IC-Light\nodes.py", line 83, in load
raise Exception("IC-Light: Could not patch calculate_weight")

Is that the whole error? It has nothing to do with torch version though, make sure these nodes are up to date as that error is caused by comfy moving a function, which I have fixed.

Is that the whole error? It has nothing to do with torch version though, make sure these nodes are up to date as that error is caused by comfy moving a function, which I have fixed.

I installed COMFYUI from scratch and downloaded the extension again, the error does not disappear.

All I know is that it works with both the node and ComfyUI being latest versions currently, as I specifically fixed it for that. I tried to make it backwards compatible as well, but haven't tested that.

I apologize, my mistake was corrected this way. You never know, maybe someone didn’t understand what the problem was, but in fact ComfyUI was not updated for me via git pull.
checking existing branches
get branch -a
branch selection
git checkout master
Set up tracking for the local master branch:
git branch --set-upstream-to=origin/master master
Now run git pull:
git pull
P.S and of course the last question of the day, is it expected that ICLight for FLUX would be just a brilliant thing...

I apologize, my mistake was corrected this way. You never know, maybe someone didn’t understand what the problem was, but in fact ComfyUI was not updated for me via git pull.
checking existing branches get branch -a branch selection git checkout master Set up tracking for the local master branch: git branch --set-upstream-to=origin/master master Now run git pull: git pull
P.S and of course the last question of the day, is it expected that ICLight for FLUX would be just a brilliant thing...

The model itself is only available for 1.5, so we are limited to that. You can still use it on Flux outputs of course, just probably drops the quality. There's a node called "Detail Transfer" that I added which will help mitigate the issue of lost sharpness that should help as well.

I apologize, my mistake was corrected this way. You never know, maybe someone didn’t understand what the problem was, but in fact ComfyUI was not updated for me via git pull.
checking existing branches get branch -a branch selection git checkout master Set up tracking for the local master branch: git branch --set-upstream-to=origin/master master Now run git pull: git pull
P.S and of course the last question of the day, is it expected that ICLight for FLUX would be just a brilliant thing...

The model itself is only available for 1.5, so we are limited to that. You can still use it on Flux outputs of course, just probably drops the quality. There's a node called "Detail Transfer" that I added which will help mitigate the issue of lost sharpness that should help as well.

special thanks for Detail transfer I didn’t know about its existence, I should periodically update the examples in the folder, otherwise they tend to become outdated...


Migrate all flashcards (too small, ...), but deactivate some

Unfortunately I am not able to reconstruct the context for


[WIP] Explicit selector for legacy/enveloped transactions

Description
Explicit selector for legacy/enveloped transactions (legacy/Eip1559).  Fixes #1581 .
This change is BREAKING:

Code needs to be recompiled with new wallet core version (as proto buf numbering has changed)
Ethereum SigningInput has to be extended with:
txMode = TransactionMode.Legacy for legacy transactions (this can be omitted as Legacy has value 0)
txMode = TransactionMode.Enveloped for enveloped transactions with EIP1559 fee

Testing instructions
Unit tests, esp. TWAnySignerEthereum.
Types of changes

Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[x] Prefix PR title with [WIP] if necessary.
[x] Add tests to cover changes as needed.
[x] Update documentation as needed.
[x] I have read the guidelines for adding a new blockchain
[x] If there is a related Issue, mention it in the description (e.g. Fixes #<issue_number> ).

make sure we don't have assert when build the tx

make sure we don't have assert when build the tx

As far as I know apps should be built with release mode, and asserts should not be present there.  It's important to clarify this in general
I don't see any assert in Eth tx building, only in ABI building
I don't see an assert that would be triggered by gasPrice=0

That's weird, need to check iOS build script


Merge pull request #1 from aspnet/master

sss
Summary of the changes (Less than 80 chars)

Detail 1
Detail 2

Addresses #bugnumber (in this specific format)

dd


fix optimize kerbero options, remove BE kinit, just use UGI login by newInstanceFromKeytab

Proposed changes

we need  remove BE kinit, and use jni login with keytab, because kinit cannot renew TGT for doris in many complex cases.

This pull requet will support new instance from keytab: https://github.com/apache/doris-thirdparty/pull/173, so now we  won't need kinit cmd, just login with keytab and principal

add kerberos_ccache_path to set kerberos credentials cache path manually.

add max_hdfs_file_handle_cache_time_ms to set hdfs fs handle cache time.

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall


Trigger production bundle release for EKS-A v0.19.10

Trigger production bundle release for EKS-A v0.19.10.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/hold

/lgtm

/unhold

/approve


Instill error: atom

Installing<EMAIL_ADDRESS>failed:

<EMAIL_ADDRESS>install /private/var/folders/8m/nw8n4_sn6ds6l82jf80ss5lc0000gn/T/apm-install-dir-119626-1586-gryeg1.xsnb7/node_modules/julia-client/node_modules/@pfitzseb/node-pty-prebuilt
prebuild-install || node scripts/install.js

npm WARN deprecated<EMAIL_ADDRESS>CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
prebuild-install WARN install No prebuilt binaries found (target=2.0.18 runtime=electron arch=x64 libc= platform=darwin)
gyp info it worked if it ends with ok
gyp verb cli [ '/Applications/Atom copy.app/Contents/Resources/app/apm/bin/node',
gyp verb cli   '/Applications/Atom copy.app/Contents/Resources/app/apm/node_modules/.bin/node-gyp',
gyp verb cli   'rebuild' ]
gyp info using<EMAIL_ADDRESS>gyp info using<EMAIL_ADDRESS>| darwin | x64
gyp verb command rebuild []
gyp verb command clean []
gyp verb clean removing "build" directory
gyp verb command configure []
gyp verb check python checking for Python executable "/Applications/Atom copy.app/Contents/Resources/app/apm/bin/python-interceptor.sh" in the PATH
gyp verb which succeeded /Applications/Atom copy.app/Contents/Resources/app/apm/bin/python-interceptor.sh /Applications/Atom copy.app/Contents/Resources/app/apm/bin/python-interceptor.sh
gyp verb check python version /Applications/Atom copy.app/Contents/Resources/app/apm/bin/python-interceptor.sh -c "import platform; print(platform.python_version());" returned: "3.7.3\n"
gyp ERR! configure error
gyp ERR! stack Error: Python executable "/Applications/Atom copy.app/Contents/Resources/app/apm/bin/python-interceptor.sh" is v3.7.3, which is not supported by gyp.
gyp ERR! stack You can pass the --python switch to point to Python >= v2.5.0 & < 3.0.0.
gyp ERR! stack     at failPythonVersion (/Applications/Atom copy.app/Contents/Resources/app/apm/node_modules/node-gyp/lib/configure.js:454:14)
gyp ERR! stack     at /Applications/Atom copy.app/Contents/Resources/app/apm/node_modules/node-gyp/lib/configure.js:443:9
gyp ERR! stack     at ChildProcess.exithandler (child_process.js:267:7)
gyp ERR! stack     at emitTwo (events.js:126:13)
gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
gyp ERR! stack     at maybeClose (internal/child_process.js:925:16)
gyp ERR! stack     at Socket.stream.socket.on (internal/child_process.js:346:11)
gyp ERR! stack     at emitOne (events.js:116:13)
gyp ERR! stack     at Socket.emit (events.js:211:7)
gyp ERR! stack     at Pipe._handle.close [as _onclose] (net.js:554:12)
gyp ERR! System Darwin 18.7.0
gyp ERR! command "/Applications/Atom copy.app/Contents/Resources/app/apm/bin/node" "/Applications/Atom copy.app/Contents/Resources/app/apm/node_modules/.bin/node-gyp" "rebuild"
gyp ERR! cwd /private/var/folders/8m/nw8n4_sn6ds6l82jf80ss5lc0000gn/T/apm-install-dir-119626-1586-gryeg1.xsnb7/node_modules/julia-client/node_modules/@pfitzseb/node-pty-prebuilt
gyp ERR! node -v v8.9.3
gyp ERR! node-gyp -v v3.4.0
gyp ERR! not ok
npm WARN enoent ENOENT: no such file or directory, open '/private/var/folders/8m/nw8n4_sn6ds6l82jf80ss5lc0000gn/T/apm-install-dir-119626-1586-gryeg1.xsnb7/package.json'
npm WARN apm-install-dir-119626-1586-gryeg1.xsnb7 No description
npm WARN apm-install-dir-119626-1586-gryeg1.xsnb7 No repository field.
npm WARN apm-install-dir-119626-1586-gryeg1.xsnb7 No README data
npm WARN apm-install-dir-119626-1586-gryeg1.xsnb7 No license field.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>install: prebuild-install || node scripts/install.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Should be fixed now.


from where i get this      string oauthBasePath = "[OAUTH_BASE_PATH]"; and  string privateKeyFilename = "[PRIVATE_KEY_FILENAME]"; and  string userId = "[USER_ID]"; // use your userId (guid), not email address

from where i get this      string oauthBasePath = "[OAUTH_BASE_PATH]"; and  string privateKeyFilename = "[PRIVATE_KEY_FILENAME]"; and  string userId = "[USER_ID]"; // use your userId (guid), not email address

Once you create a SandBox account you should be able to create Integrator Keys and RSA Private Keys for JWT Authentication...
Check our test project to see how we use oauth base path and other properties.
https://github.com/docusign/docusign-csharp-client/tree/master/test
Follow Quickstart guide here and let us know if you still have questions.
https://developers.docusign.com/esign-rest-api/guides

@developerpp, Closing this issue, feel free to open a new issue if you have any questions.
Thanks


Preserve transaction signatures.

This fixes transaction hashes being computed incorrectly for signed messages submitted via eth_sendRawTransaction.
This should fix #56 .

@benjamincburns Thoughts on merging?

Hi @mgraczyk - thanks for pinging me - I'd meant to respond sooner.
I haven't had a chance to give this a try yet, however if you could add some tests for the functionality that'll make it much simpler to evaluate.

@benjamincburns I added a test which fails before and passes after the change.

LGTM - thanks for the contribution, @mgraczyk!


fix generating streams parameters (ftp, log_files was broken)

Fixed API:

ftp.upload
ftp.download
log_files.download_log_file

Fixed ftp plugin key

Oops, sorry.

@appetito: sorry for the conflict 😅 . Would you mind updating your branch and running the autogen again? It shouldn't create much conflict, but note that mavsdk/generated disappeared and now plugins are in their own module in mavsdk, e.g. mavsdk/ftp/

@appetito: sorry for the conflict

no problem), i updated my branch


Cannot see flag images in flutter windows

Same here. This is from a mac. Flutter running on Web:

As my findings, windows not support country flags.
but you can add country flags font file to your windows. then it shows the flags.
more info:
https://github.com/perguto/Country-Flag-Emojis-for-Windows

Same here on flutter web


Systemic bug in Stan sampler

Summary:
Systemic bug in Stan inference engine
Description:
As demonstrated in C-K Hur, A. V. Nori, S. K. Rajamani, and S. Samuel, "A Provably Correct Sampler for Probabilistic Programs" (alternate link) Stan does not seem to converge to the correct distribution for the following model:
1: double x, y;
2: x ~ Gaussian(0, 1);
3: if (x > 0) then
4:    y ~ Gaussian(10, 2);
5: else
6:    y ~ Gamma(3, 3);
7: return y;

Reproducible Steps:
See ipython notebook 1 which demonstrates the steps to reproduce the behaviour with the following stan model:
"""
parameters {
    real x;
    real y;
}
model {
    x ~ normal(0, 1);
    if(x > 0) {
        y ~ normal(10, 2);
    } else {
        y ~ gamma(3, 3);
    }
}
"""

Current Output:
See Example 1 in ipython notebook 2. Also refer to that notebook for an alternate formulation which stan infers correctly.
Expected Output:
See the graphs in ipython notebook 2 and ipython notebook 1
Additional Information:
As mentioned in the description, we believe this points to a systemic flaw in Stan's sampler. The paper provides the general solution as well as examples of other problematic models.
Current Version:
v2.9.0

Thanks for the clear report.  This isn't a bug in Stan so much as a defect in our documentation and parser warnings.  If it's a flaw in anything, the responsibility lies with the Hamiltonian Monte Carlo sampler.  So there's nothing to fix in Stan, and I'm going to close this issue.
Stan's HMC and NUTS samplers require continuously differentiable density functions because they take the negative log density to be a potential energy function and follow the Hamiltonian of a fictional particle.  When you introduce a jump, as in jump at 0 in the Stan program listed in the issue, it defeats the ability to simulate the Hamiltonian.  It would fail even more dramatically with a gamma with shape of 1 or less.
We've been meaning to provide a compiler warning to warn users against conditionals that depend on parameters.  Other than in spline or other interpolation models, they almost always introduce discontinuities that HMC can't handle.
If you only wanted to recreate the output you're after in Stan, it could be done in the generated quantities block, which really does work as a random number generator.
The model implied by your program is not only unusual in using a continuous variable on which to condition a mixture, but in mixing distributions of different support (gamma requires positive values).  Assuming you wanted to focus on positive values, you'd write this mixture prior (even mix of half normal and gamma) in Stan as follows:
transformed data {
  real<lower=0, upper=1> lambda;
  lambda <- 0.5;
}
parameters {
  real<lower=0> theta;
}
model {
  // log p(theta | lambda)
  // = log(lambda * normal(theta | 10, 2)
  //       + (1 - lambda) * gamma(theta | 3, 3)
  increment_log_prob(log_sum_exp(log(lambda) + normal_log(theta, 10, 2),
                                 log1m(lambda) + gamma_log(theta, 3, 3)));
}

But that's going to be problematic to fit due to multimodality with a fairly deep trough between the modes.  This makes it hard for the HMC sampler to move around the posterior.  If you run the above, you'll find the Markov chains produced by the sampler getting stuck in one of the two modes.
Was there a particular application you had in mind for this kind of mixture?
P.S.  I could open the Python notebooks, but not the linked paper.  Stan does not follow anything like the approach in the Wingate et al. paper.  It uses HMC (specifically NUTS).  There are plenty of references in the manual.

Thanks, Bob. I was just about to address this and I'm glad you already did.
@suhailshergill, I tried following the paper, but if you really wanted to "fit" that model, you'd just do:
model {
}
generated quantities {
  real x;
  real y;

  x <- normal_rng(0, 1);
  if (x > 0) {
        y <- normal_rng(10, 2);
    } else {
        y <- gamma_rng(3, 3);
    }
}
}

thanks @bob-carpenter for the prompt response. i'm far from an expert stan user, so i'm not sure how clear or not it is to users that such discontinuities are not supported in the model block, but quite a few individuals in our meetup were taken aback by the silent failure. A compiler warning would certainly help greatly in this regard. Is there an issue regarding that?
What doesn't help is the difference in behaviour. Specifically,  while the below fails:
"""
parameters {
    real x;
    real y;
}
transformed parameters {
    real ret;
    ret <- y;
}
model {
    x ~ normal(0, 1);
    if (x > 0) {
        y ~ normal(10, 2);
    } else {
        y ~ gamma(3, 3);
    }
}
"""

the transformed model (which is mathematically equivalent), works:
"""
parameters {
    real x;
    real y;
    real z;
}
transformed parameters {
    real ret;
    if (x > 0) {
      ret <- y;
    } else {
      ret <- z;
    }
}
model {
    x ~ normal(0, 1);
    y ~ normal(10, 2);
    z ~ gamma(3, 3);
}
"""

Perhaps I need to read the documentation regarding the subtle differences between what's supported in model, transformed parameters and generated quantities blocks.

@syclik I am the author of one of the Python notebooks. I tried to run your examples, specifically, I tried to run
model {
}
generated quantities {
    real x;
    real y;
    real ret;
    x <- normal_rng(0, 1);
    if (x > 0) {
        y <- normal_rng(10, 2);
    } else {
        y <- gamma_rng(3, 3);
    }
    ret <- y;
}

(don't mind the ret, it's just for further processing in Python). Since the program doesn't have any parameters, I used the "Fixed_param" algorithm with PyStan. However, this crashes. I'm not sure where, I don't see a strack trace on the command line. I'm on Stan 2.9.0.

The fixed_param sampler doesn't work in PyStan v2.9.0. It does work with
CmdStan v2.9.0 and RStan v2.9.0.
If you want to see it running, change it to:
parameters {
  real<lower = 0, upper = 1> theta;
}
model { }

generated quantities {
    real x;
    real y;
    real ret;
    x <- normal_rng(0, 1);
    if (x > 0) {
        y <- normal_rng(10, 2);
    } else {
        y <- gamma_rng(3, 3);
    }
    ret <- y;
}

Regarding the iPython notebook, if you've run only one chain, you might see
that sort of behavior. If you ran multiple chains, my guess is that R-hat
would signal a problem with convergence. (I don't know if that's how you
ran it.) Although there isn't a compile-time error, there should be a clear
diagnosis that there isn't convergence and the output shouldn't be trusted.
It's one of the reasons why we encourage running multiple chains.
On Mon, Jun 13, 2016 at 10:00 PM, Torsten Scholak<EMAIL_ADDRESS>wrote:

@syclik https://github.com/syclik I am the author of one of the Python
notebooks. I tried to run your examples, specifically, I tried to run
model {
}
generated quantities {
real x;
real y;
real ret;
x <- normal_rng(0, 1);
if (x > 0) {
y <- normal_rng(10, 2);
} else {
y <- gamma_rng(3, 3);
}
ret <- y;
}
(don't mind the ret, it's just for further processing in Python). Since
the program doesn't have any parameters, I used the "Fixed_param" algorithm
with PyStan. However, this crashes. I'm not sure where, I don't see a
strack trace on the command line. I'm on Stan 2.9.0.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/stan/issues/1917#issuecomment-225760228, or mute
the thread
https://github.com/notifications/unsubscribe/AAZ_FwGMWcZXOGOoUkK8bXTOD0UHSPgAks5qLgtVgaJpZM4I02oy
.

@syclik Thanks, I try that then.
I did see problems with convergence. I ended up running 32 chains to get somewhat consistent results.

When you have convergence problems, merely running more chains won't solve the problem. Did running 32 actually get split R-hat down to 1?
Running multiple chains will hopefully tell you if there exists a problem (when you heed the existing convergence diagnostics).

On Jun 13, 2016, at 10:11 PM, Torsten Scholak<EMAIL_ADDRESS>wrote:
@syclik Thanks, I try that then.
I did see problems with convergence. I ended up running 32 chains to get somewhat consistent results.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

I'm getting an R-hat of 1.
example_1 = """
parameters {
    real x;
    real y;
}
transformed parameters {
    real ret;
    ret <- y;
}
model {
    x ~ normal(0, 1);
    if (x > 0) {
        y ~ normal(10, 2);
    } else {
        y ~ gamma(3, 3);
    }
}
"""
sm = pystan.StanModel(model_code=example_1)
fit = sm.sampling(data={},
                  iter=30000,
                  warmup=15000,
                  chains=32)

and the fit gives:
Inference for Stan model: anon_model_8c6f88b7c6812397267935ab183ea884.
32 chains, each with iter=30000; warmup=15000; thin=1;
post-warmup draws per chain=15000, total post-warmup draws=480000.

       mean se_mean     sd   2.5%    25%    50%    75%  97.5%  n_eff   Rhat
x      0.74    0.01   0.65-8.0e-3   0.28   0.64   1.12   2.22 2219.0   1.01
y      9.77    0.08   2.43   2.71   8.52   9.93  11.31  13.89  970.0   1.03
ret    9.77    0.08   2.43   2.71   8.52   9.93  11.31  13.89  970.0   1.03
lp__  -1.06    0.02   1.09  -4.02  -1.45  -0.71  -0.29  -0.02 2079.0   1.02

Samples were drawn using NUTS(diag_e) at Mon Jun 13 22:53:29 2016.
For each parameter, n_eff is a crude measure of effective sample size,
and Rhat is the potential scale reduction factor on split chains (at
convergence, Rhat=1).

But this is all beside the point. These examples are just pathologic models from that Hur et al. paper, designed to make a point about how branches and loops in probabilistic programming languages can cause problems for MCMC-based inference.

I created an issue for a compiler warning here in case people didn't notice it in the threaded view: https://github.com/stan-dev/stan/issues/1918
@tscholak Thanks for running the above.  I think the point is that the mean of x is suposed to be 0.  The densities aren't going to be comparable because Stan drops constants, so you can't write a mixture in that kind of loop.  The real problem is that even if you could, it would imply arbitrarily distinct regimes for y based on what x is, so no smooth way to transition from drawing from one component of the mixture to the other when x crosses zero.

There are lots of ways to generate draws from this, the most efficient being the pure Monte Carlo method with generated quantities.  You can use an auxiliary variable method that mixes, but there's a huge problem (see below):
parameters {
  real x;
  real y1;
  real<lower=0> y2;
}
transformed parameters {
  real y;
  y <- if_else(x < 0, y1, y2);
}
model {
  x ~ normal(0, 1);
  y1 ~ normal(10, 2);
  y2 ~ gamma(3, 3);
}

which yields completely independent x, y1, and y2 draws and returns y as a transformed parameter.  And it all works as expected:
Inference for Stan model: mix.
4 chains, each with iter=2000; warmup=1000; thin=1;
post-warmup draws per chain=1000, total post-warmup draws=4000.

      mean se_mean   sd  2.5%   25%   50%   75% 97.5% n_eff Rhat
x     0.02    0.02 1.00 -1.92 -0.68  0.01  0.67  2.03  2752    1
y1   10.02    0.04 2.05  6.01  8.68 10.02 11.37 14.09  2874    1
y2    1.00    0.01 0.58  0.21  0.57  0.90  1.30  2.46  2500    1
y     5.47    0.08 4.73  0.27  0.91  3.15  9.97 13.59  3355    1

The huge problem alluded to earlier is that the variable y is useless because the entire model includiung y is still not differentiable.  And by useless, I just mean you can't condition things on it, so I couldn't, for example, use y as a mixture prior.

Everyone stand back: computer scientists are trying to do statistics again
and it’s not pretty.  Firstly, the original link sent earlier in the thread doesn’t
seem to work, but you can grab the paper at http://research.microsoft.com/pubs/255950/final.pdf.
Given the typical CS hyperbole, it’s understandable how one could get
confused and think that there are deeper issues here.  But this is largely
a consequence of, to be it delicately, misunderstandings but the authors
of that paper.
For example, in the “loop” example of Figure 3 Stan actually is getting the
right answer.  The problem is that the authors completely misinterpret the
Stan language associating “~” with a sampling operation so that at the loop
somehow recursively defines a convolution.  But as discussed extensively
in the manual, “~” is just syntactic sugar for incrementing the log target
density, so
x ~ normal(x, 3);
literally translates to
log_target += normal_log(x, x, 3);
which evaluates to zero given the cancellation of the x’s.  So the model
actually being defined is just
x ~ normal(0, 1)
which is exactly what Stan reports!
Bob and Dan have already commented on the issues with the second model.
It’s effectively trying to fit a discrete model which Stan is not equipped to
handle.  More importantly, Stan is in no way “silently” failing.  Firstly, there
should be at least a few divergences when both x < 0 and y < 0 and Stan
tries to evaluate the gamma_log density at negative y.  Secondly, you’re
getting only O(1000) effective samples out of O(500,000) transitions, and
for autocorrelations that large you can’t even really trust the effective sample
size estimators (there are nasty feedbacks between the higher-order
autocorrelations and the estimator variance, etc).  This is also why the Rhats
are so bad.
Ultimately what’s happening is that when Stan makes large jumps in x
which requires huge jumps in y to move between the modes of those two
components.  Essentially the chain has to reconverge again and again, but
it’s not given enough time to do so, inducing the large autocorrelations and
untrusty worthy fits.
To conclude, the paper is very misleading and completely mischaracterizes
Stan (I won’t make any comments about their awesome “solution”), so these
aren’t really “pathological” models in the sense that Stan should fit them but
can’t.  In the loop case Stan is doing the right thing and in the mixture case
Stan never claims to be able to fit that well and throws plenty of diagnostics
that something is going wrong.
As Bob notes we can have the compiler flag these warnings, but it’s a tricky
issue in that in some cases parameter-based conditions can be okay.  So
we’re left with another tough decision between facilitating a few power users
or trying to automatically identify potential problems for newer users.
On Jun 14, 2016, at 4:09 AM, Torsten Scholak<EMAIL_ADDRESS>wrote:

But this is all beside the point. These examples are just pathologic models from that Hur et al. paper, designed to make a point about how branches and loops in probabilistic programming languages can cause problems for MCMC-based inference.

I am not a STAN developer. However, I was intrigued by the accusation put forward against STAN. Therefore, I read the entire paper mentioned in the original post. Now I realise that the authors are all coming from a technical field that is not computational statistics, most likely computer science/machine learning. Before trying to clear up some confusion the authors have about MCMC in general and STAN in specific, I earnestly suggest the authors get at least one hardcore computational statistician involved in their team.
First of all, STAN obviously did not perform satisfactorily in the examples given by the authors. However, the cause of this disappointment has nothing to do with STAN and has everything to do with the way STAN was employed. As a complete software, STAN is capable of much more that just performing HMC. As we have seen from the previous posts, the examples given can easily be solved using generated quantity or transformed parameters, which only invoke the necessary random number generators but not HMC. However, the authors intentionally invoke the HMC algorithm to sample from these models. The first example results in a nonsmooth density. The second example in the same paper results in an 11 dimension density, disguised as a one-dimensional density. Both of these examples constitute intentional misuse of HMC. There is nothing wrong with neither the code nor the algorithm inside STAN. And it should be obvious at this point that STAN is not a fool-proof software, i.e. user are expected to use STAN properly with the help of the development team.
Second and more importantly, the authors are utterly confused about the theoretical foundation of MH or HMC or MCMC in general. Using the authors' notation, the program P is to be executed in each iteration to generate a new proposal. This is just wrong. That is not what happens in STAN or any MCMC algorithms. Instead, the program P is necessarily translated into a density function T, which is done automatically in STAN (and manually if we are using other MCMC algorithms that are not supported by STAN). The state in each iteration is generated from the proposal mechanism Q, which can be a density or something else (in the case of HMC, it is the discretized solution of the Hamiltonian equations). Anyway, we do not execute P in any iteration, we only need T and Q. Therefore the whole discussion about how STAN (and R2) going wrong because they are ignorant to the specifics of P is really misleading. From the reading, this seems to be an earnest misunderstanding, or not?
Third, the proposed solution in the paper essentially boils down to automatically expanding the sampling space to the total number of random number generations in the program P. Hence the original target T is preserved as a marginal density. Conventional Metropolis-Hastings and Reversible Jump (Peter Green 1995) already cover the theoretical support for this solution. I can't comprehend the notation given in the theorems/proofs given in the paper. It was truly a nightmare. While being theoretically valid, I fail to see why this is an innovative sampling approach. MH is notoriously known to fail in high dimension. Also, why anyone would run MH algorithm when we can draw i.i.d. samples? Also, how can I use the proposed approach to sample from a likelihood function that is not a standard density? I just don't see how. The given simulations just raise many more suspicions. The only thing results published were the computing time, no specific models, no graph, no data were given. How on earth is that publishable?
I really want to ask for my afternoon back! Truly!

thanks @bob-carpenter for creating the compiler warning issue!
@TRANTANKHOA the notation in the paper is truly a nightmare, and is a consequence from them approaching this from the PL side (denotational semantics etc). i'm not sure whether the errors in their use of stan were a result of misunderstanding or something more, but since that explanation suffices, i personally am ok giving them the benefit of doubt. while i can't give you back your afternoon, i would like to thank you for taking the time to go through the paper and bring your thoughts to light.
@betanalpha agreed that CS folks (of which i'm one) are trying to do stats :) probabilistic programming systems seem to give us plenty of rope, and we seem quite equipped to use that rope to hang ourselves. thanks for shedding light on the matter. The issue, in part, seems to be a notational one indeed. Specifically, ~ when used in CS/ML often stands for sampling, and so it is understandable why the authors made that mistake (in fact I did so too, forgetting as you rightly point out that in Stan ~ is just syntactic sugar for incrementing the log target density).
I'm not sure if the examples given in the paper were reduced versions (for didactic purposes) of models that the authors were having trouble with or something else, but it's illuminating to finally understand the details of why it happens (and what that means).

Well, I am a physicist, and I spent several afternoons on that paper, too.
While I can’t say that I had a jolly good time making it through their notation, I liked the premise enough to warrant the issue some thought.
Btw, exchanging thoughts is really the only reason I am here, and this is getting less and less enjoyable. The tone in this thread has become increasingly heated, and I don’t condone that. In response to @betanalpha’s opening statement, I could equally (and wrongfully) claim that Stan is statisticians trying to do computer science, but these kind of accusations are not only demeaning, they are also absolutely not pertinent to discussing this issue.
Now, returning to the issue, I think the predominant opinion is that this is really a non-issue, because the original authors of that paper (purposely?) misunderstood what Stan does or what its semantics tell you it is doing (e.g., ~ being syntactic sugar for increment_log_prob). Be it as it may, I personally was somewhat disappointed to learn that Stan cannot be used for what the authors set out do: inferring the distribution of variables in an imperative probabilistic program with ifs, elses and loops, which I consider to be an interesting and challenging problem. We don’t need to talk about MH being an abysmal answer to it, I guess that is obvious. However, we should acknowledge that there might be a need for communicating to the user that conditioning on parameters is a bad idea, and I am grateful to @bob-carpenter for taking the initiative on this matter, see https://github.com/stan-dev/stan/issues/1918.
@TRANTANKHOA
I don’t think at all that the paper is saying that the program P is executed in each iteration to generate a new proposal. What is executed is their INFER algorithm that takes a P and uses it do build T by means of EVAL.

Well, here is a quote from the paper: "In order to perform MH sampling on a probabilistic program, we need to run the program P which results in the state being constructed incrementally, as P executes along a path. Along the path, the program encounters several probabilistic assignments to variables, and
to generate a new value for each variable, we require the corresponding old value of the variable from the previous run of the program. Such an association between old and new values is easy if each variable is assigned only once, or if the program is a single path without branches or loops."  So indeed, that is what they say.
Also, STAN can be used for what the authors set out to do: inferring the distribution of variables in an imperative probabilistic program with ifs, elses and loops. The authors just did not do it properly.
Also, don't you notice that what the authors set out to do boils down only to i.i.d. sampling all the time and that is not interesting at all.
I don't think anyone here not having enough respect for computer scientists. However, condemning bad science is necessary too. Maybe Michael was not as politically correct as he should have. But it is just calling out the hard truth. Given that Michael is one of the core developers of this software, I can totally understand why he is offended by the unfounded accusation that STAN is fundamentally flawed. Asking for help solving a problem is one thing. Pointing out a problem for other people while it is not there is another. So I don't think Michael started the fight first. Notice that I'm not a STAN developer myself.

@TRANTANKHOA @suhailshergill @tscholak I really appreciate that you guys are willing to be involved in this discussion at such depth.  This is one of those corners of Stan that's hard to explain properly in part because of ill-informed attacks such as the one laid out in the above-mentioned paper.  I like that we can keep it civil (most of the time?) but could we move further discussion to the users' list where other people might benefit from it and understand/join in?

Hi @sakrejda , as far as contributing to the discussion. I think I already made my points. But people are welcome to carry on to the users' list. Cheers!

@TRANTANKHOA They write "incrementally", not "repeatedly", which is what you first alluded to.
@sakrejda Absolutely. Can a conversation (and its history) migrated to the mailing list?

On Jun 14, 2016, at 8:30 AM, Torsten Scholak<EMAIL_ADDRESS>wrote:
Well, I am a physicist, and I spent several afternoons on that paper, too.
While I can’t say that I had a jolly good time making it through their notation, I liked the premise enough to warrant the issue some thought.
Btw, exchanging thoughts is really the only reason I am here, and this is getting less and less enjoyable. The tone in this thread has become increasingly heated, and I don’t condone that.

Me, either.  Nothing to be gained that way.

In response to @betanalpha’s opening statement, I could equally (and wrongfully) claim that Stan is statisticians trying to do computer science, but these kind of accusations are not only demeaning, they are also absolutely not pertinent to discussing this issue.

I completely agree about the demeaning bit.  We get it a lot
the other way from computer scientists, and it's not pleasant.
And completely uncalled for.
For what it's worth, Stan's mostly built by people with CS backgrounds,
like me and Matt Hoffman, or physics backgrounds, like Michael.  Andrew's
the only proper statistician in the group (as in having a Ph.D. in stats),
and he's not involved in the code-level development.

Now, returning to the issue, I think the predominant opinion is that this is really a non-issue, because the original authors of that paper (purposely?) misunderstood what Stan does or what its semantics tell you it is doing (e.g., ~ being syntactic sugar for increment_log_prob).

I don't think they misunderstood what Stan was doing on purpose.
I think it's up to us to make that clear.  There's an entire
DARPA (U.S. defense research funding) project aimed at building
the kind of probabilistic programming languages that can fit the
kind of model first posed.  Why, we're not so sure.  These types of
models tend not to come up in our applied work, and are not the
kind of thing most of our users would come up with.

Be it as it may, I personally was somewhat disappointed to learn that Stan cannot be used for what the authors set out do: inferring the distribution of variables in an imperative probabilistic program with ifs, elses and loops, which I consider to be an interesting and challenging problem.

Challenging, yes.  Not so sure what it's useful for, but we
try to stay open-minded about interesting use cases and don't
claim that Stan can do everything.
Stan can deal with loops and conditionals, but not if they
introduce discontinuities in the densities.

We don’t need to talk about MH being an abysmal answer to it, I guess that is obvious. However, we should acknowledge that there might be a need for communicating to the user that conditioning on parameters is a bad idea, and I am grateful to @bob-carpenter for taking the initiative on this matter, see #1918.

The issue is my way of agreeing.  I'll fix that and try to write
some clearer doc.

Bob

On Jun 14, 2016, at 7:12 AM, Suhail Shergill<EMAIL_ADDRESS>wrote:
... Specifically, ~ when used in CS/ML often stands for sampling,

It usually doesn't mean that anywhere.  Even in something
like Church, the ~ just specifies the forward model.  But you
can't literally execute it as a random sample and hope to get
anywhere.
That's why we've been considering removing ~ from the language
altogether so that it's clear to users that all a Stan program
does is define a target density on the log scale up to an additive
constant that doesn't depend on parameters.

Bob

@betanalpha:  Thanks for clarifying what's going on so clearly.
Please don't stereotype whole groups of scientists as you do in
the first sentences of the first two paragraphs.  I think everyone's
trying to do the right thing here, and comments like this only serve
to annoy people.   I very much want to maintain our friendly lists,
even if someone comes onto our list looking to start a fight.
To mix family and basketball metaphors, it's like we're older
siblings---if the younger sibling punches us first, we still get
the foul for retaliating.

Bob

On Jun 14, 2016, at 5:13 AM, Michael Betancourt<EMAIL_ADDRESS>wrote:
Everyone stand back: computer scientists are trying to do statistics again
and it’s not pretty. Firstly, the original link sent earlier in the thread doesn’t
seem to work, but you can grab the paper at http://research.microsoft.com/pubs/255950/final.pdf.
Given the typical CS hyperbole, it’s understandable how one could get
confused and think that there are deeper issues here. But this is largely
a consequence of, to be it delicately, misunderstandings but the authors
of that paper.
For example, in the “loop” example of Figure 3 Stan actually is getting the
right answer. The problem is that the authors completely misinterpret the
Stan language associating “~” with a sampling operation so that at the loop
somehow recursively defines a convolution. But as discussed extensively
in the manual, “~” is just syntactic sugar for incrementing the log target
density, so
x ~ normal(x, 3);
literally translates to
log_target += normal_log(x, x, 3);
which evaluates to zero given the cancellation of the x’s. So the model
actually being defined is just
x ~ normal(0, 1)
which is exactly what Stan reports!
Bob and Dan have already commented on the issues with the second model.
It’s effectively trying to fit a discrete model which Stan is not equipped to
handle. More importantly, Stan is in no way “silently” failing. Firstly, there
should be at least a few divergences when both x < 0 and y < 0 and Stan
tries to evaluate the gamma_log density at negative y. Secondly, you’re
getting only O(1000) effective samples out of O(500,000) transitions, and
for autocorrelations that large you can’t even really trust the effective sample
size estimators (there are nasty feedbacks between the higher-order
autocorrelations and the estimator variance, etc). This is also why the Rhats
are so bad.
Ultimately what’s happening is that when Stan makes large jumps in x
which requires huge jumps in y to move between the modes of those two
components. Essentially the chain has to reconverge again and again, but
it’s not given enough time to do so, inducing the large autocorrelations and
untrusty worthy fits.
To conclude, the paper is very misleading and completely mischaracterizes
Stan (I won’t make any comments about their awesome “solution”), so these
aren’t really “pathological” models in the sense that Stan should fit them but
can’t. In the loop case Stan is doing the right thing and in the mixture case
Stan never claims to be able to fit that well and throws plenty of diagnostics
that something is going wrong.
As Bob notes we can have the compiler flag these warnings, but it’s a tricky
issue in that in some cases parameter-based conditions can be okay. So
we’re left with another tough decision between facilitating a few power users
or trying to automatically identify potential problems for newer users.
On Jun 14, 2016, at 4:09 AM, Torsten Scholak<EMAIL_ADDRESS>wrote:

But this is all beside the point. These examples are just pathologic models from that Hur et al. paper, designed to make a point about how branches and loops in probabilistic programming languages can cause problems for MCMC-based inference.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Hi all.  I didn't follow all the details of this discussion, but as we've been taking about the verbose parser that picks up possible errors, it strikes me that this is another thing that we can add:  certain if/else statements that are bad news for HMC.
A

On Jun 14, 2016, at 7:12 AM, Suhail Shergill<EMAIL_ADDRESS>wrote:
thanks @bob-carpenter https://github.com/bob-carpenter for creating the compiler warning issue!
@TRANTANKHOA https://github.com/TRANTANKHOA the notation in the paper is truly a nightmare, and is a consequence from them approaching this from the PL side (denotational semantics etc). i'm not sure whether the errors in their use of stan were a result of misunderstanding or something more, but since that explanation suffices, i personally am ok giving them the benefit of doubt. while i can't give you back your afternoon, i would like to thank you for taking the time to go through the paper and bring your thoughts to light.
@betanalpha https://github.com/betanalpha agreed that CS folks (of which i'm one) are trying to do stats :) probabilistic programming systems seem to give us plenty of rope, and we seem quite equipped to use that rope to hang ourselves. thanks for shedding light on the matter. The issue, in part, seems to be a notational one indeed. Specifically, ~ when used in CS/ML often stands for sampling, and so it is understandable why the authors made that mistake (in fact I did so too, forgetting as you rightly point out that in Stan ~ is just syntactic sugar for incrementing the log target density).
I'm not sure if the examples given in the paper were reduced versions (for didactic purposes) of models that the authors were having trouble with or something else, but it's illuminating to finally understand the details of why it happens (and what that means).
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/stan-dev/stan/issues/1917#issuecomment-225850616, or mute the thread https://github.com/notifications/unsubscribe/ADOBUaO6IDo-_9QArBi4PrbePQdogD1Dks5qLoy6gaJpZM4I02oy.

It usually doesn't mean that anywhere.  Even in something
like Church, the ~ just specifies the forward model.

@bob-carpenter my bad. when i said "sampling" i meant drawing/sampling a value from a distribution in the sense of a generative process (i.e. "forward model").

But you can't literally execute it as a random sample and hope to get
anywhere.

agreed.
thank you for your constructive feedback; taking some of my followup thoughts to #1918

On Jun 14, 2016, at 11:27 PM, Suhail Shergill<EMAIL_ADDRESS>wrote:
It usually doesn't mean that anywhere. Even in something
like Church, the ~ just specifies the forward model.
@bob-carpenter my bad. when i said "sampling" i meant drawing/sampling a value from a distribution in the sense of a generative process (i.e. "forward model").

This is where we got the notation, but misinterpretation
of execution is why we're considering just eliminating ~.
When one writes
y ~ foo(theta)
to specify a graphical/generative model, the implication is
that it contributes the following term to the joint log density
p(y, theta, ... ) += log foo(y | theta) + ...
Stan literally executes the increment when it sees the ~ (and
drops constants in the evaluation of foo).  BUGS
or PyMC build up graphical models, which BUGS executes with Gibbs
(or non-conjugate generalizations thereof), whereas PyMC3 does
what Stan does and converts to a log density for sampling with NUTS.
Execution in something like Church/Venture, etc., is more complicated;
it defines the program structure, but the program structure isn't
just executed by running the forward model.

Bob


KQL Detection vulnerable OpenSSL version

Thank you for your contribution! Some pointers to get it merged quickly:
For contributions in software/:

[x] PR Title: Please use "Add <vendor/product name>" (instead of "Update README.md")
[x] Status: please select a value from the status table at the top
[x] Version: Status Vulnerable / Workaround? -> List vulnerable versions.
Status Fix?                     -> List fixed versions.
[x] Links: make sure you link to a public statement by the developer.
Alternatively, include and link a file in the software/vendor-statements/ directory
[x] Please mind the sorting

M365 Defender and Azure Sentinel KQL query to detect OpenSSL installations on devices with version 3.0*

Checklist done, no addition in software.


Short transcription is not saving for flutter

Log file analysis required to identify the cause of 401 errors post-save added attachment for same @shalini8979 .

Please provide the inputs which you use here while recording

It's resolved.


Change defaults of exe configs to contain the name of the exe

Currently all are set to /path/to. Using just the name of the exe without a path will work if the exe is in the user's path. If not, they will have to set it to the full path. This also gives the user the name of the exe so they can search for it on their computer to see if it exists.
While doing this work, it may be useful to change the names of *_EXE to remove _EXE since it is redundant.

The changes are checked into the develop branch. Leaving this ticket open because I still need to update the documentation to list the _EXE are depreacted and the  variables should be used instead. I want to wait to do this because we need to make changes to the documentation that will go into the bugfix release v2.1.1.

We decided not to add the documentation changes to the bugfix release, so this work is completed and will appear in the 3.0 release.


ILocalizationService.GetLocalizedContentItem does not work with Localized Terms

If the content has a TermPart should have a specific logic in order to find localizations for it, like in ILocalizationService.GetLocalizations(); soon a PR for dev.

Looking a little bit deeper into the code and trying the localization functionality on one of our more complex project, some corner cases are not well managed. So I will send a more wide PR in order to manage them.
some significative cases:

if the MasterContent has been deleted, it cannot translate terms
Translating a content starting from a localized version wrongly clones the master content


Polynomial doc rendering

http://mpmath.org/doc/current/calculus/polynomials.html
Polynomial doc is using inline latex for some strings.
I've changed them to inline literals.

Thanks!


Unexpected 'feature' of the docker containerizer with parameters

If you pass something like:
"parameters": [
    {"key": "e", "value": "somevalue"}
]

Marathon will blow up and not properly run that job. The reason is because it takes every key and prefaces -- and passes that to the docker commandline when it executes it. When debugging problems with job failure in the mesos ui, you can see that the stdout and stderr logs in the sandbox are 0 bytes.
Looking at /var/log/mesos/mesos-slave.WARNING of a slave running the above app shows stuff like:
E0224 17:19:58.340267 21710 slave.cpp:2787] Container '8a47bb39-e032-4278-9adc-d0b53014738b' for executor 'mesos_kafka_test.a6130191-bc7b-11e4-be08-56847afe9799' of framework '20150219-102423-134552526-5050-35126-0005' failed to start: Failed to 'docker run -d -c 1024 -m<PHONE_NUMBER> -e MARATHON_APP_VERSION=2015-02-24T23:19:51.248Z -e HOST=els0-mesosl12.internal -e ZOOKEEPER_INSTANCES=els0-mesosl1:2181,els0-mesosl2:2181,els0-mesosl3:2181 -e SERVICE_NAME=KAFKA -e PORT_9092=31163 -e KAFKA_K0_BROKER_INTERNAL_PORT=9092 -e CONTAINER_NAME=K0 -e CONTAINER_HOST_ADDRESS=localhost -e KAFKA_INSTANCES=K0 -e MESOS_TASK_ID=mesos_kafka_test.a6130191-bc7b-11e4-be08-56847afe9799 -e PORT=31163 -e PORTS=31163 -e KAFKA_K0_HOST=localhost -e MARATHON_APP_ID=/mesos/kafka/test -e PORT0=31163 -e MESOS_SANDBOX=/mnt/mesos/sandbox -v /data/mesos/slave/slaves/20150221-165059-84220878-5050-9992-S14/frameworks/20150219-102423-134552526-5050-35126-0005/executors/mesos_kafka_test.a6130191-bc7b-11e4-be08-56847afe9799/runs/8a47bb39-e032-4278-9adc-d0b53014738b:/mnt/mesos/sandbox --net host --e=CONTAINER_HOST_NAME=$(hostname) --entrypoint /bin/sh --name mesos-8a47bb39-e032-4278-9adc-d0b53014738b internal-registry/mesos/kafka:<IP_ADDRESS> -c /data/kafka/.docker/run.py': exit status = exited with status 2 stderr = flag provided but not defined: --e

Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

Run a command in a new container

  -a, --attach=[]            Attach to STDIN, STDOUT or STDERR.
  --add-host=[]              Add a custom host-to-IP mapping (host:ip)
  -c, --cpu-shares=0         CPU shares (relative weight)
  --cap-add=[]               Add Linux capabilities
  --cap-drop=[]              Drop Linux capabilities
  --cidfile=""               Write the container ID to the file
  --cpuset=""                CPUs in which to allow execution (0-3, 0,1)
  -d, --detach=false         Detached mode: run the container in the background and print the new container ID
  --device=[]                Add a host device to the container (e.g. --device=/dev/sdc:/dev/xvdc)
  --dns=[]                   Set custom DNS servers
  --dns-search=[]            Set custom DNS search domains
  -e, --env=[]               Set environment variables
  --entrypoint=""            Overwrite the default ENTRYPOINT of the image
  --env-file=[]              Read in a line delimited file of environment variables
  --expose=[]                Expose a port from the container without publishing it to your host
  -h, --hostname=""          Container host name
  -i, --interactive=false    Keep STDIN open even if not attached
  --link=[]                  Add link to another container in the form of name:alias
  --lxc-conf=[]              (lxc exec-driver only) Add custom lxc options --lxc-conf="lxc.cgroup.cpuset.cpus = 0,1"
  -m, --memory=""            Memory limit (format: <number><optional unit>, where unit = b, k, m or g)
  --name=""                  Assign a name to the container
 --net="bridge"             Set the Network mode for the container
                               'bridge': creates a new network stack for the container on the docker bridge
                               'none': no networking for this container
                               'container:<name|id>': reuses another container network stack
                               'host': use the host network stack inside the container.  Note: the host mode gives the container full access to local system services such as D-bus and is therefore considered insecure.
  -P, --publish-all=false    Publish all exposed ports to the host interfaces
  -p, --publish=[]           Publish a container's port to the host
                               format: ip:hostPort:containerPort | ip::containerPort | hostPort:containerPort | containerPort
                               (use 'docker port' to see the actual mapping)
  --privileged=false         Give extended privileges to this container
  --restart=""               Restart policy to apply when a container exits (no, on-failure[:max-retry], always)
  --rm=false                 Automatically remove the container when it exits (incompatible with -d)
  --security-opt=[]          Security Options
  --sig-proxy=true           Proxy received signals to the process (even in non-TTY mode). SIGCHLD, SIGSTOP, and SIGKILL are not proxied.
  -t, --tty=false            Allocate a pseudo-TTY
  -u, --user=""              Username or UID
  -v, --volume=[]            Bind mount a volume (e.g., from the host: -v /host:/container, from Docker: -v /container)
  --volumes-from=[]          Mount volumes from the specified container(s)
  -w, --workdir=""           Working directory inside the container
E0224 17:19:58.340558 21710 slave.cpp:2882] Termination of executor 'mesos_kafka_test.a6130191-bc7b-11e4-be08-56847afe9799' of framework '20150219-102423-134552526-5050-35126-0005' failed: Unknown container: 8a47bb39-e032-4278-9adc-d0b53014738b
E0224 17:19:58.340773 21706 slave.cpp:3134] Failed to unmonitor container for executor mesos_kafka_test.a6130191-bc7b-11e4-be08-56847afe9799 of framework 20150219-102423-134552526-5050-35126-0005: Not monitored
W0224 17:19:58.341451 21692 docker.cpp:1183] Ignoring updating unknown container: 8a47bb39-e032-4278-9adc-d0b53014738b
E0224 17:20:34.306603 21680 slave.cpp:2787] Container '0e1f2abf-8357-48b7-83dc-396d0f3be706' for executor 'mesos_kafka_test.bb8d0d97-bc7b-11e4-be08-56847afe9799' of framework '20150219-102423-134552526-5050-35126-0005' failed to start: Failed to 'docker run -d -c 1024 -m<PHONE_NUMBER> -e MARATHON_APP_VERSION=2015-02-24T23:19:51.248Z -e HOST=els0-mesosl12.internal -e ZOOKEEPER_INSTANCES=els0-mesosl1:2181,els0-mesosl2:2181,els0-mesosl3:2181 -e SERVICE_NAME=KAFKA -e PORT_9092=31385 -e KAFKA_K0_BROKER_INTERNAL_PORT=9092 -e CONTAINER_NAME=K0 -e CONTAINER_HOST_ADDRESS=localhost -e KAFKA_INSTANCES=K0 -e MESOS_TASK_ID=mesos_kafka_test.bb8d0d97-bc7b-11e4-be08-56847afe9799 -e PORT=31385 -e PORTS=31385 -e KAFKA_K0_HOST=localhost -e MARATHON_APP_ID=/mesos/kafka/test -e PORT0=31385 -e MESOS_SANDBOX=/mnt/mesos/sandbox -v /data/mesos/slave/slaves/20150221-165059-84220878-5050-9992-S14/frameworks/20150219-102423-134552526-5050-35126-0005/executors/mesos_kafka_test.bb8d0d97-bc7b-11e4-be08-56847afe9799/runs/0e1f2abf-8357-48b7-83dc-396d0f3be706:/mnt/mesos/sandbox --net host --e=CONTAINER_HOST_NAME=$(hostname) --entrypoint /bin/sh --name mesos-0e1f2abf-8357-48b7-83dc-396d0f3be706 internal-registry/mesos/kafka:<IP_ADDRESS> -c /data/kafka/.docker/run.py': exit status = exited with status 2 stderr = flag provided but not defined: --e

My biggest issue here is that the mesos sandbox stdout and stderr logs show 0 bytes on both, so it is very difficult for a user who doesn't have access to the mesos bare metal servers to troubleshoot this.
Marathon is working great to manage applications for our firm's internal stuff, but this makes it harder for some users who might just send bad data in their app json to know they send bad data.

See https://issues.apache.org/jira/browse/MESOS-2020 for mesos sanbox issue.

Does it make sense for marathon to not try to run --e though knowing it will fail? I change e to env and it worked just to do some testing.
I was attempting to do:
{"key": "env", "value": "CONTAINER_HOST_NAME=$(hostname)"}

But sadly, marathon (or perhaps mesos proper) escapes it and the CONTAINER_HOST_NAME env var ends up as the literal string of $(hostname).

Use HOST env variable, it is already set for you.

@bobrick: totally. That is what I did. I just opened this issue as marathon could be enhanced to be smarter about that is all.

The parameters are passed blindly to the Mesos Docker containerizer.  The containerizer is where the values are transformed into parts of the docker run command.  Marathon (or Mesos) purposely avoid attempting to parse these Docker CLI arguments.
The lack of visibility into the app misconfiguration is a real problem though.  There are a number of issues to track that already.  See #951, #965, #1179.

Thanks @ConnorDoyle, closing this as it clearly is already in the backlog of RFEs.


Many errors occur while reproducing your setup

Thank you for your research.
However, in my experience reproducing your setup, I encountered several errors while preparing the dataset before training.
Could you commit to an updated version?

This seems like a problem with cameras. The prepare_data.py script only renames files according to our format.
Nontheless, I am investigating the problems reported and will soon upload the datasets so it resolves any mismatch

please follow the thread in issue #11 for updates.

@lilygoli  Thanks I resolve it! :)


Set grpc ports in docker config template

What changed?
Set grpc ports in docker config template

Why?
Fix GRPC port not configured for service error

How did you test it?
Tested locally

Potential risks

Release notes

Documentation Changes

Pull Request Test Coverage Report for Build a31f1dfd-6802-4d49-987c-2a083acf9cd3

0 of 0   changed or added relevant lines in 0 files are covered.
64 unchanged lines in 9 files lost coverage.
Overall coverage increased (+0.02%) to 67.093%

Files with Coverage Reduction
New Missed Lines
%

common/task/weightedRoundRobinTaskScheduler.go
1
89.64%

common/task/fifoTaskScheduler.go
2
82.47%

service/history/handler.go
2
46.74%

service/matching/taskListManager.go
2
74.72%

common/persistence/cassandra/cassandraPersistence.go
3
52.8%

common/types/history.go
4
24.69%

service/history/historyEngine.go
4
71.19%

service/history/execution/mutable_state_task_refresher.go
15
71.18%

common/persistence/cassandra/cassandraPersistenceUtil.go
31
90.87%

Totals

Change from base Build b9bac8da-8ee8-4688-8871-cac90eb5cbe3:
0.02%

Covered Lines:
98321

Relevant Lines:
146545

💛  - Coveralls


Bibtex support

Could Marked have References support, like Pandoc? It's like searching a Bibtex file and delivering list of references by using a citation style (APA, Chicago, IEEE...)

At this time we are focusing on security vulnerabilities, two specifications, and bug fixes to meet those specifications. Closing as out of scope and stale. See #956


add TextMatchFilterOptimizer to maximally push down text_match filters to Lucene

Motivation:
Query performance against the Lucene index suffers when chaining multiple text_match predicates together. Our users often programmatically generate their queries, which exacerbates the issue as 10s/100s of text_match predicates can be included in a single query.
Because of this, users are required to understand Pinot's Lucene implementation details for them to compose an efficient query. To remove this requirement, this PR adds a TextMatchFilterOptmizer that performs the optimization automatically.
Summary:
This functionality is best understood through the unit testcases. In short:

Merge all AND's and OR's text_match operands when possible, without affecting query accuracy
Push down NOT into Lucene, unless all text_match filters are inversed, then the NOT expression remains in Pinot

Open question:
There is one edge case (that I can think of) where this optimization can hurt performance: if there are a number of text_match OR text_match OR text_match etc, early termination when limit is reached might take longer since the entire merged text_match query must now complete. For this reason, it might be prudent to put this behind a query option (or rather a query option to disable it, since I believe it makes more sense to enable by default).
Ideally, the LuceneDocIdCollector could early terminate (but doesn't currently have the required context).
Testing: unit tests (query performance separately verified via running the optimized vs unoptimized queries). Sample optimization improvements:
count(*) = 500_000_000
optimized: 2950ms
unoptmized: 8350ms
> select count(*) from table where text_match(column, '"message_logtype:storage-event" AND /.*c28487d062.*/')
> select count(*) from table where text_match(column, '"message_logtype:storage-event"') AND text_match(column, '/.*c28487d062.*/')

count(*) = 2
optimized: 12ms
unoptimized: 8400ms
> select count(*) from table where text_match(column, '"message_logtype:storage-event" AND /.*c28487d062.*/ AND "offset:46127612"')
> select count(*) from table where text_match(column, '"message_logtype:storage-event"') AND text_match(column, '/.*c28487d062.*/') AND text_match(column, '"offset:46127612"')

tags: feature, performance (?)

Codecov Report
Attention: 12 lines in your changes are missing coverage. Please review.

Comparison is base (4823802) 61.66% compared to head (c7c3b3f) 61.63%.
Report is 2 commits behind head on master.

Files
Patch %
Lines

...ery/optimizer/filter/TextMatchFilterOptimizer.java
86.66%
5 Missing and 7 partials :warning:

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #12339      +/-   ##
============================================
- Coverage     61.66%   61.63%   -0.03%
  Complexity      207      207
============================================
  Files          2421     2422       +1
  Lines        131852   131962     +110
  Branches      20345    20369      +24
============================================
+ Hits          81303    81340      +37
- Misses        44582    44647      +65
- Partials       5967     5975       +8

Flag
Coverage Δ

custom-integration1
<0.01% <0.00%> (ø)

integration
<0.01% <0.00%> (ø)

integration1
<0.01% <0.00%> (ø)

integration2
0.00% <0.00%> (ø)

java-11
34.90% <86.66%> (-26.69%)
:arrow_down:

java-21
61.50% <86.66%> (-0.04%)
:arrow_down:

skip-bytebuffers-false
61.63% <86.66%> (-0.03%)
:arrow_down:

skip-bytebuffers-true
27.70% <0.00%> (-33.81%)
:arrow_down:

temurin
61.63% <86.66%> (-0.03%)
:arrow_down:

unittests
61.63% <86.66%> (-0.03%)
:arrow_down:

unittests1
46.78% <86.66%> (+0.01%)
:arrow_up:

unittests2
27.70% <0.00%> (-0.05%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@siddharthteotia I'd appreciate your thoughts/review

this looks awesome. is it possible to optimize regexp_like(x) OR regexp_like(y) ?


Convert DATE type to logical type

Description
See #4406 and #4069

@aditi-pandit Aditi kindly offered to help with this change.
CC: @duanmeng

@pramodsatya : Please can you try this refactoring.

@aditi-pandit Any update?

I have started working on this. Should have something out by Monday.

@aditi-pandit Aditi, thank you for the update. Is there a PR available for review now?

https://github.com/facebookincubator/velox/pull/5015