[FEATURE] Add support for number of documents when creating workloads

Is your feature request related to a problem? Please describe.
create-workload subcommand currently generates a workload using all the documents from appointed indices. It would be nice if users could specify a number of documents from the indices they've appointed instead of using all the documents.
Currently, this is what create-workload supports:
usage: opensearch-benchmark create-workload [-h] --workload WORKLOAD --indices INDICES --target-hosts TARGET_HOSTS [--client-options CLIENT_OPTIONS]
                                            [--output-path OUTPUT_PATH] [--quiet] [--offline]

options:
  -h, --help            show this help message and exit
  --workload WORKLOAD   Name of the generated workload
  --indices INDICES     Comma-separated list of indices to include in the workload
  --target-hosts TARGET_HOSTS
                        Comma-separated list of host:port pairs which should be targeted
  --client-options CLIENT_OPTIONS
                        Comma-separated list of client options to use. (default: timeout:60)
  --output-path OUTPUT_PATH
                        Workload output directory (default: workloads/)
  --quiet               Suppress as much as output as possible (default: false).
  --offline             Assume that Benchmark has no connection to the Internet (default: false).

Describe the solution you'd like
There should be an additional parameter that takes in the number of documents of appointed indices to generate workload from.
Describe alternatives you've considered
Additional context
Add any other context or screenshots about the feature request here.

Working on this

Background
When users invoke create-workload with a single index, the code converts the indices specified to a list of strings. For each index, OSB fetches the index settings and writes them to their own <index_name>.json. It returns metadata about the index. After fetching all index settings and metadata, OSB extracts and dumps all documents from the respective index. It returns more metadata about the index and its documents.
Section of code that fetches doc count and uses it to get all documents in index
# extract() in corpus.py
total_docs = client.count(index=index)["count"]
    if total_docs > 0:
        logger.info("[%d] total docs in index [%s].", total_docs, index)
        docs_path = get_doc_outpath(output_path, index)
        dump_documents(client, index, get_doc_outpath(output_path, index, "-1k"), min(total_docs, 1000), " for test mode")
        dump_documents(client, index, docs_path, total_docs)
        return template_vars(index, docs_path, total_docs)
    else:
        logger.info("Skipping corpus extraction fo index [%s] as it contains no documents.", index)
        return None

Proposed Solution

Add a parameter to benchmark.py called --doc-counts. This can take in a list, similar to --indices, or a dictionary like --client-options. See the following examples of how this might look:

# Specifying only count of docs
# Single index
--indices=movies --doc-counts=2000

# Two indices
--indices=movies,actors --doc-counts=2000,1000

#Alternative: Specifying the doc counts as a dictionary like client options
# Single index
--indices=movies --doc-counts=movies:'2000'

# Two indices
--indices=movies,actors --doc-counts=movies:'2000',actors:'1000'

If we were to do the latter (dictionary like --client-options), we would need to create a class like ClientOptions which extends ConnectOptions in opts.py. My only concern with this is that it's a little unwieldy but it might prevent issues. It also looks redundant if users only specify a single index.

Add an additional parameter called doc_count representing doc counts to both extract_mappings_and_corpora() in workload_generator.py and extract() in corpus.py. If parameter is not provided, it will just collect all documents from indices.
Check if doc_count is <= total_docs found by match_all query. If it is, use it. If not, log an error to inform the user that OSB will use the number of docs found by match_all query instead since it's less than the specified amount.

Encountering this edge case where a user has an index that is greater than 1000 docs. They specify a doc-count of less than 1000 docs (for example, 750 documents). Should the documents-1k.json be the same as documents.json or should it remain at 1000 docs and documents.json is less than that. The latter makes sense but right now, when users create workloads off of indices with less than 1000 documents, the documents-1k.json is the same as document.json (and is unusable in test mode)

Using a single index and single doc count
hoangia@3c22fbd0d988 opensearch-benchmark % opensearch-benchmark create-workload --workload=movies-test-with-parameter --target-hosts=<TARGET HOST ENDPOINT> --client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'" --indices=movies-2000 --output-path=~/Desktop/workloads --total-docs=1200

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
[INFO] Connected to OpenSearch cluster [eef3f5a9032ea6572cf946f1fbcec586] version [1.1.0].

Extracting documents for index [movies-2000] for test mode... 1000/1000 docs [100.0% done]
Extracting documents for index [movies-2000]...               1200/1200 docs [100.0% done]

[INFO] Workload movies-test-with-parameter has been created. Run it with: opensearch-benchmark --workload-path=/Users/hoangia/Desktop/workloads/movies-test-with-parameter

-------------------------------
[INFO] SUCCESS (took 3 seconds)
-------------------------------

Use multiple indices and multiple doc counts
hoangia@3c22fbd0d988 opensearch-benchmark % opensearch-benchmark create-workload --workload=movies-test-with-parameter --target-hos<TARGET HOST ENDPOINT> --client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'" --indices=movies-2000,movies-1000 --output-path=~/Desktop/workloads --total-docs=1200,1000

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
[INFO] Connected to OpenSearch cluster [48bd5a55847accbfaebf1c57cfa2e513] version [1.1.0].

Extracting documents for index [movies-2000] for test mode... 1000/1000 docs [100.0% done]
Extracting documents for index [movies-2000]...               1200/1200 docs [100.0% done]
Extracting documents for index [movies-1000] for test mode... 1000/1000 docs [100.0% done]
Extracting documents for index [movies-1000]...               1000/1000 docs [100.0% done]

[INFO] Workload movies-test-with-parameter has been created. Run it with: opensearch-benchmark --workload-path=/Users/hoangia/Desktop/workloads/movies-test-with-parameter

-------------------------------
[INFO] SUCCESS (took 3 seconds)
-------------------------------

When user specifies --total-docs but only provides a fraction of indices total docs count
hoangia@3c22fbd0d988 opensearch-benchmark % opensearch-benchmark create-workload --workload=movies-test-with-parameter --target-hosts=<TARGET HOST ENDPOINT> --client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'" --indices=movies-2000,movies-1000 --output-path=~/Desktop/workloads --total-docs=1200

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
[INFO] Connected to OpenSearch cluster [380d8fd64dd85b5f77c0ad81b0799e1e] version [1.1.0].

Extracting documents for index [movies-2000] for test mode... 1000/1000 docs [100.0% done]
Extracting documents for index [movies-2000]...               1200/1200 docs [100.0% done]
[ERROR] Cannot create-workload. --total-docs missing document counts for corresponding indices.

Getting further help:
*********************
* Check the log files in /Users/hoangia/.benchmark/logs for errors.
* Read the documentation at https://opensearch.org/docs.
* Ask a question on the forum at https://forum.opensearch.org/.
* Raise an issue at https://github.com/opensearch-project/OpenSearch-Benchmark/issues and include the log files in /Users/hoangia/.benchmark/logs.

-------------------------------
[INFO] FAILURE (took 2 seconds)
-------------------------------

If user does not specify doc counts, then it uses all indices in the index.
hoangia@3c22fbd0d988 opensearch-benchmark % opensearch-benchmark create-workload --workload=movies-test-with-parameter --target-hosts=https://search-hoangia-aos-testing-2-xpwa7ykgv7bbc7qqfuuzt2no7i.us-east-1.es.amazonaws.com --client-options="basic_auth_user:'hoangia',basic_auth_password:'Hoangia@123'" --indices=movies-2000,movies-1000 --output-path=~/Desktop/workloads

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
[INFO] Connected to OpenSearch cluster [380d8fd64dd85b5f77c0ad81b0799e1e] version [1.1.0].

Extracting documents for index [movies-2000] for test mode... 1000/1000 docs [100.0% done]
Extracting documents for index [movies-2000]...               2000/2000 docs [100.0% done]
Extracting documents for index [movies-1000] for test mode... 1000/1000 docs [100.0% done]
Extracting documents for index [movies-1000]...               1000/1000 docs [100.0% done]

[INFO] Workload movies-test-with-parameter has been created. Run it with: opensearch-benchmark --workload-path=/Users/hoangia/Desktop/workloads/movies-test-with-parameter

-------------------------------
[INFO] SUCCESS (took 3 seconds)
-------------------------------

Updated --total-docs to --number-of-docs:
Using a single index and single doc count
opensearch-benchmark % opensearch-benchmark create-workload --workload=movies-custom-docs --target-hosts=<TARGET_ENDPOINT> --client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'" --indices="movies-2000" --output-path=~/Desktop/workloads --number-of-docs 1200

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
[INFO] Connected to OpenSearch cluster [48bd5a55847accbfaebf1c57cfa2e513] version [1.1.0].

Extracting documents for index [movies-2000] for test mode... 1000/1000 docs [100.0% done]
Extracting documents for index [movies-2000]...               1200/1200 docs [100.0% done]

[INFO] Workload movies-custom-docs has been created. Run it with: opensearch-benchmark --workload-path=/Users/hoangia/Desktop/workloads/movies-custom-docs

-------------------------------
[INFO] SUCCESS (took 1 seconds)
-------------------------------

Use multiple indices and multiple doc counts
opensearch-benchmark create-workload --workload=movies-custom-docs --target-hosts=<ENDPOINT> --client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'" --indices="movies-2000,movies-1000" --output-path=~/Desktop/workloads --number-of-docs 1200,1000

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
[INFO] Connected to OpenSearch cluster [380d8fd64dd85b5f77c0ad81b0799e1e] version [1.1.0].

Extracting documents for index [movies-2000] for test mode... 1000/1000 docs [100.0% done]
Extracting documents for index [movies-2000]...               1200/1200 docs [100.0% done]
Extracting documents for index [movies-1000] for test mode... 1000/1000 docs [100.0% done]
Extracting documents for index [movies-1000]...               1000/1000 docs [100.0% done]

[INFO] Workload movies-custom-docs has been created. Run it with: opensearch-benchmark --workload-path=/Users/hoangia/Desktop/workloads/movies-custom-docs

Encountered this issue: Occurred because params.py only has a check for one index for search operation. Needed to update the default-query-workload.json.j2 Jinja template to incorporate all the indices for "index" field for workload.json.
osbenchmark.exceptions.InvalidSyntax: 'index' or 'data-stream' is mandatory and is missing for operation 'search'

When user specifies too few total-docs compared to indices
opensearch-benchmark create-workload --workload=movies-custom-docs --target-hosts=<ENDPOINT> --client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'" --indices="movies-2000" --output-path=~/Desktop/workloads --number-of-docs 1500

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
[INFO] Connected to OpenSearch cluster [48bd5a55847accbfaebf1c57cfa2e513] version [1.1.0].

Extracting documents for index [movies-2000] for test mode... 1000/1000 docs [100.0% done]
Extracting documents for index [movies-2000]...               1200/1200 docs [100.0% done]

[INFO] Workload movies-custom-docs has been created. Run it with: opensearch-benchmark --workload-path=/Users/hoangia/Desktop/workloads/movies-custom-docs

-------------------------------
[INFO] SUCCESS (took 1 seconds)
-------------------------------

Using without --number-of-docs parameter and gather all docs in index
opensearch-benchmark create-workload --workload=movies-custom-docs --target-hosts=<ENDPOINT> --client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'" --indices="movies-2000" --output-path=~/Desktop/workloads

   ____                  _____                      __       ____                  __                         __
  / __ \____  ___  ____ / ___/___  ____ ___________/ /_     / __ )___  ____  _____/ /_  ____ ___  ____ ______/ /__
 / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \   / __  / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
/ /_/ / /_/ /  __/ / / /__/ /  __/ /_/ / /  / /__/ / / /  / /_/ /  __/ / / / /__/ / / / / / / / / /_/ / /  / ,<
\____/ .___/\___/_/ /_/____/\___/\__,_/_/   \___/_/ /_/  /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/  /_/|_|
    /_/

[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
[INFO] Connected to OpenSearch cluster [380d8fd64dd85b5f77c0ad81b0799e1e] version [1.1.0].

Extracting documents for index [movies-2000] for test mode... 1000/1000 docs [100.0% done]
Extracting documents for index [movies-2000]...               1200/1200 docs [100.0% done]

[INFO] Workload movies-custom-docs has been created. Run it with: opensearch-benchmark --workload-path=/Users/hoangia/Desktop/workloads/movies-custom-docs

-------------------------------
[INFO] SUCCESS (took 1 seconds)
-------------------------------

Encountered the same error in custom-query-workload.json.j2. Made the same update and now it works.
osbenchmark.exceptions.InvalidSyntax: 'index' or 'data-stream' is mandatory and is missing for operation 'search'

To make things simpler, I added a class called StoreKeyPairAsDict, which is a custom action in Argparse. It takes in a string(s) and converts them into a dictionary.
Instead of this:
--indices=movies-2000,movies-1000 --number-of-docs 1200,1000

Users can now run this:
 --indices=movies-2000,movies-1000 --number-of-docs movies-2000:1100 movies-1000:1000


Remove connect timeouts from Bind

This PR removes the connect timeouts from Bind, as described in https://github.com/runconduit/conduit/pull/483#issuecomment-369380003.
It removes the CONDUIT_PROXY_PUBLIC_CONNECT_TIMEOUT_MS environment variable, but not the CONDUIT_PROXY_PRIVATE_CONNECT_TIMEOUT_MS variable, since this is also used for the TCP connect timeouts. If we want also want to remove the TCP connection timeouts, I can do that as well.
Closes #483.

This is removing the timeout for external ("public") outbound connections too, right? We already have another timeout that properly handles the Destination service not providing us a name, right? If so, it seems OK to remove this timeout for outbound connections since either the application we're proxying has its own timeout already or it doesn't have a timeout and presumably doesn't want one, and we have no good way to configure this timeout for an application that actually wants us to provide it as a feature. However, why did we add it in the first place, @olix0r?

Incidentally, in https://github.com/runconduit/conduit/pull/483#issuecomment-369380003, when I said "this timeout", I meant specifically the one controlled by CONDUIT_PROXY_PRIVATE_CONNECT_TIMEOUT_MS. However, I see that the public timeout is using the same code, so it makes sense to remove it at the same time, unless @olix0r has a reason why we need the public timeout.
I would expect that the public timeout would be more useful than the private timeout because if a particular outbound destination is super slow then we should eventually either give up or try a different outbound destination. But maybe this is better undone for now until the retry and circuit breaking features are properly designed.

@briansmith that's correct. The "connect" timeout is applied in Bind::bind_service, which is shared  between the inbound and outbound halves.There's already a separate timeout configured for the total time until binding a service to the name (the Destination service response time + time in the proxy).

I suppose that if we do think having the public connect timeout is valuable, I could also modify Bind to only apply the timeout if it's configured to do so?

Okay, I figured out what's going on: tower-reconnect only reconnects if the connection was successfully established and then errored. If there was an error while connecting, it propagates the error. The connect timeout occurs during the connection, therefore it does not trigger a reconnect.
So it's not so much a bug in tower-reconnect as it is an interaction between reconnect and a timeout wrapped by the reconnect middleware. I'll open an issue on tower for figuring out the relationship between reconnects and retries.

I've opened tower-rs/tower#60 to track adding a backoff policy on connect errors to the tower-reconnect middleware.

I've also opened issue #491 to track the error fixed by this PR.

I've opened tower-rs/tower#60 to track adding a backoff policy on connect errors to the tower-reconnect middleware.

As far as that might affect Conduit, I think we should probably have a broader discussion of what we're doing with respect to back-off and retry logic. It seems like it's far from clear what we actually want to do and I think we need a design doc that describes what we're going to do immediately and a migration path towards something better.

So, removing the timeout for the private connection seems to make perfect sense and I have no concerns about that.
Let's think critically about whether or not we want the timeout in the public (external) connections. A point in favor of removing the public timeout is the fact that we don't have a good UX for configuring the timeout. A point against removing the public timeout is that we don't (IIUC) have any garbage collection for unused connections, so if the underlying application has a timeout and closes the outbound connection to the proxy, that will have no effect on whether we close the outbound connection to the external service. However, one could argue that the more general problem of not having garbage collection for unused connections is the real problem and having this particular timeout logic is just a band-aid that helps with a special case, and that it's not worthwhile. However, one could also argue that 0.3.1 is a stability and bug fixing release and we've not had any bug reports about the timeout for the timeout for public connections, so we shouldn't change anything w.r.t. them.
This last point is the strongest one for me: If we're not fixing something we know to be broken, and there are downsides to the change, then let's avoid making the change for 0.3.1. So, I'm included to have us keep the public timeout and remove the private timeout, probably using the technique @hawkw suggested.
I think it would be good to hear from others (@olix0r, @seanmonstar, @carllerche, @hawkw) before scrambling to update the PR, to avoid waffling further.

A point against removing the public timeout is that we don't (IIUC) have any garbage collection for unused connections, so if the underlying application has a timeout and closes the outbound connection to the proxy, that will have no effect on whether we close the outbound connection to the external service.

I don't think that removing the public timer has any practical downside right now, though I strongly agree that we need to have a larger discussion about lifecycle/resource management moving forward
Specifically, if the proxy attempts to connect to a peer and that peer neither opens nor resets the connection, then we'll hold that connection attempt open until the endpoint is removed from service discovery, at which point the connecting future is dropped...
Currently, if those connections timeout, we'll just try to reestablish them the next time a request is dispatched to that peer.
So, I think this is, operationally, effectively equivalent behavior.

@hawkw Can you update the pr summary with a clear problem / solution, as it would be committed?

@olix0r how does it look now?

We configure outbound with:
config.public_connect_timeout.unwrap_or_else(||Duration::from_millis(300)),

If I understand correctly, the outcome of this change is that we always use a 300ms connect timeout now, because this cannot be configured?

More generally, I don't think we should enforce connect timeouts on TCP connections -- that's up to the application.

More generally, I don't think we should enforce connect timeouts on TCP connections -- that's up to the application.

Due to the way futures work, we'd need to be careful that the steps between receiving a connection and piping on the other side have polled for read (well, HUP, but we can't ask for that directly at the moment), and then notice that the task was woken up because of the HUP. It's, uh, actually a bit tricky.

@olix0r

We configure outbound with:
config.public_connect_timeout.unwrap_or_else(||Duration::from_millis(300)),

If I understand correctly, the outcome of this change is that we always use a 300ms connect timeout now, because this cannot be configured?

I added the public connect timeout environment variable back in 26f162541b8995facc9469bc577ad78b6e9fe0d5 as @briansmith requested in https://github.com/runconduit/conduit/pull/487#issuecomment-369732280. It should still be configurable.

More generally, I don't think we should enforce connect timeouts on TCP connections -- that's up to the application.

@briansmith suggested the TCP connection timeout should be removed in a follow-up PR (https://github.com/runconduit/conduit/pull/487#discussion_r171751339). If we agree that timeouts should be removed from TCP connections, I'm happy to add that change to this PR if yoi'd prefer that.

Yes, let's remember the scope of the problem that we're trying to solve: We're trying to solve the problem that the private timeout is too short. To solve that problem, we're removing the private timeout. It so happens that removing the private timeout is easier to do if we remove the public timeout for HTTP too, which is the only reason we're touching that timeout here.
If the TCP timeout is problematic for whatever reason, that's a separate problem and should be fixed separately.

More generally, I don't think we should enforce connect timeouts on TCP connections -- that's up to the application.

Due to the way futures work, we'd need to be careful that the steps between receiving a connection and piping on the other side have polled for read (well, HUP, but we can't ask for that directly at the moment), and then notice that the task was woken up because of the HUP. It's, uh, actually a bit tricky.

@seanmonstar, @olix0r, and I discussed this today in the context of a 0.3.1 status discussion. @seanmonstar is going to file a new issue regarding his concerns and suggesting a better way to handle the issue in the long term.

I've filed https://github.com/runconduit/conduit/issues/511 for TCP connect timeouts.


find out why clingo_main returns 10

I think (just guessing) that clasp is not conformant with linux return numbers but with SAT solvers.
This mens 10 (model found), 20 UNSAT, and some other things...

Do you know if this is this somewhere documented?

Couldn't find it in the user guide.
Probably @rkaminsk or @BenKaufmann know more about it.
For sat solver competitions I found this:
http://www.satcompetition.org/2004/format-solvers2004.html

Let's not start the discussion whether such kind exit codes are good or bad. The best documentation you find probably here: https://github.com/potassco/clasp/issues/42

I see thank you.


feat: updated wasmd to v0.42.0 #NTRN-122

todo:

[x] update ibc-go to 7.3.0

https://github.com/neutron-org/neutron-tests/actions/runs/6336798686

https://github.com/neutron-org/neutron-tests/actions/runs/6338152950


Issues in trying to make a Scala environment

I am trying to make a simple container for Scala development. The usual process of installing Scala consists of

having a JVM installed
downloading a script, such as https://github.com/paulp/sbt-extras that in turn
downloads the Jar of SBT (the Scala package manager), that in turn
downloads  the Scala compiler and all the dependencies

This seems a little contrived, but allows to change everything (libraries, the version of Scala, the version of SBT itself) from a simple configuration file per project. The only thing that is needed to bootstrap everything is a small script.
So, here is my tentative vagga.yaml:
containers:
  jvm:
    setup:
    - !UbuntuRelease { version: 15.04 }
    - !UbuntuUniverse
    - !Install [openjdk-8-jre-headless, curl]
  sbt:
    setup:
    - !Container jvm
    - !Sh "curl -s https://raw.githubusercontent.com/paulp/sbt-extras/master/sbt > sbt"
    - !Sh "chmod +x sbt"
    volumes:
      /tmp: !Tmpfs
      size: 100Mi
      mode: 0o1777
commands:
  scala: !Command
    container: sbt
    run: ./sbt -sbt-create console
  sh: !Command
    container: sbt
    run: bash

When I try to run sbt -sbt-create or anything that starts the bootstrap process, the sbt launcher tries to download a few jars inside /tmp, which is why I have the temporary volume. Unfortunately, I get the following error:
mkdir: cannot create directory '/.sbt': Read-only file system
Downloading sbt launcher for 0.13.9:
  From  http://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.9/sbt-launch.jar
    To  /tmp/sbt_extras_launchers.dzPs5H/0.13.9/sbt-launch.jar
java.io.IOException: No such file or directory
	at java.io.UnixFileSystem.createFileExclusively(Native Method)

It probably has to do with the fact that sbt expects to be able to write its configuration under $HOME/.sbt, and packages under $HOME/.ivy. I have tried a few settings, but I was not able to make this work.
Is it possible to bind the home directory in some writable dir?

Yes. I usually "fix" the passwd file:
    - !Sh |
        sed -i '/^root/{s@/root@/work/tmp@;}' /etc/passwd

But alternative is too bind a volume:
     volumes:
       /root: !BindRW /work/tmp

Both settings will put home of the (pseudo-) root user into the "tmp" subdir of your project directory.

mkdir: cannot create directory '/.sbt': Read-only file system
It looks like it tries to crate the folder .sbt into the root /.
sbt expects to be able to write its configuration under $HOME/.sbt - vagga doesn't set HOME by default. So, $HOME expands to nothing and you get /.sbt
Is it possible to bind the home directory in some writable dir? - yes.
What you need is environ
commands:
  scala: !Command
    container: sbt
    environ: { HOME: /root }
    run: ./sbt -sbt-create console

Or if you want to be able to write in root /(which I guess wasn't was you tried):
commands:
  scala: !Command
    container: sbt
    write-mode: transient-hard-link-copy
    run: ./sbt -sbt-create console

@lilianmoraru the problem is that java uses /etc/passwd directly (or maybe via libc) instead of looking in the environment. So changing HOME doesn't help indeed.

Thank you for your help! I tried settings $HOME, but in fact it did not help. Writing /etc/passwd seems to work! Instead, the volume trick seemed to have no effect.
I am now almost done, I have a little issue with SSL, but I will post here a working Scala config as soon as it's ready

A working vagga.yaml for scala is:
containers:
  jvm:
    setup:
    - !UbuntuRelease { version: 15.04 }
    - !UbuntuUniverse
    - !Install [openjdk-8-jre-headless, curl]
    - !Sh "update-ca-certificates -f"
  sbt:
    setup:
    - !Container jvm
    - !Sh "curl -s https://raw.githubusercontent.com/paulp/sbt-extras/master/sbt > sbt"
    - !Sh "chmod +x sbt"
    - !Sh "sed -i '/^root/{s@/root@/work/tmp@;}' /etc/passwd"
    volumes:
      /tmp: !Tmpfs
      size: 100Mi
      mode: 0o1777
commands:
  scala: !Command
    container: sbt
    run: ./sbt -sbt-create console
  sh: !Command
    container: sbt
    run: bash

@andreaferretti nice. Could I steal it to examples? Or maybe you'd like to make a pull request?

@andreaferretti Another way I solve my issues with the certificates is to install the package ca-certificates.
Example:
- !Install [ca-certificates]
# Or only at container build
- !BuildDeps [ca-certificates]

@tailhook Sure, use it however you prefer! If you want me to make a PR, just ask
@lilianmoraru Yeah, I confirm that installing ca-certificates works as well

Closing this, as everything is working fine. The latest version I have looks like this:
containers:
  jvm:
    setup:
    - !UbuntuRelease { version: 15.04 }
    - !UbuntuUniverse
    - !Install [openjdk-8-jre-headless, ca-certificates]
  sbt:
    setup:
    - !Container jvm
    - !Install [curl]
    - !EnsureDir /work/.home/sbt
    - !Download
      url: https://raw.githubusercontent.com/paulp/sbt-extras/master/sbt
      path: /work/.home/sbt/sbt
      mode: 0o744
    - !Sh "sed -i '/^root/{s@/root@/work/.home@;}' /etc/passwd"
    volumes:
      /tmp: !Tmpfs
      size: 100Mi
      mode: 0o1777
    environ:
      PATH: "/work/.home/sbt:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
      HOME: /work/.home
commands:
  scala: !Command
    container: sbt
    run: sbt -sbt-create console
    description: Run a scala console
  sh: !Command
    container: sbt
    run: bash
    description: Enter a shell with sbt available

Thanks everyone for the help!

By the way, a few more examples


Implement Roman Numeral Operation Calculator

Issue Id you have worked upon -
590
Briefly explain your program logic -
The provided JavaScript code defines a React component for a Roman Numeral Operation Calculator. The component utilizes the useState hook to manage state variables, including the input Roman numerals (num1 and num2), the selected operation (operation), and the result of the operation (result). The main logic lies within the performOperation function, which is triggered when the user clicks the "Perform Operation" button. This function attempts to convert the input Roman numerals to Arabic numerals, performs the specified arithmetic operation, and then converts the result back to a Roman numeral. The program also handles potential errors, such as invalid Roman numerals or operations, displaying an error message if needed. The user interface is designed using Material-UI components, featuring input fields for numerals and the operation, a button to initiate the calculation, and a display area for the result. The code integrates the romanize library to facilitate the conversion between Roman and Arabic numerals, contributing to the overall functionality of the Roman Numeral Operation Calculator. State management in React ensures dynamic updates and responsiveness to user interactions.
Screenshots(Attach 2 screenshots of your own input and output) -

By raising this PR I affirm that -

[x] My code follows the guidelines of this project.

[x] I have performed a self-review of my own code.

[x] I have commented my code.

[x] My code gives the correct output.

[x] I affirm that I strictly follow contributing guidelines and code of conduct.

I have revert back the changes in package.json and package-lock.json that lies outside calculators folder.

Congratulations 🎉 @SakshiJSanghavi on your contribution.
Thanks for adding value to this repository. 😊


DO NOT MERGE (QENG-1800) Install MSI during package acceptance

Previously it was not possible to run windows acceptance against FOSS
MSI packages. This commit adds a step to install the puppet-agent MSI. It
depends on https://github.com/puppetlabs/beaker/pull/674
With this commit you can do:
$ bundle exec rake ci:test:packages \
    CONFIG=config/nodes/win2008r2-rubyx64.yaml \
    SHA=<SHA> VERSION=3.7.4-1303

Note the VERSION string must match the filename of the MSI which
includes the output from git describe. The host definition file needs
to specify which ruby_arch it wants to install, either x86 or x64.

Supercedes #3584

Still need to verify that the environment variable matches what the AIO pipeline will pass through. Note it is the SHA/version of the puppet-agent package, not for the puppet component. See https://github.com/puppetlabs/puppet/pull/3584#issuecomment-73562557

Also I'm not sure this was the correct thing to do... This commit adds to the setup step for packages type acceptance, not aio...

CLA signed by all contributors.


Extension Activating on Non-Wolfram Code (Python)

Hello,
First of all, thank you for developing this extension. It's been quite useful for my work with Wolfram language in VSCode.
However, I've encountered an issue where the extension activates on my Python files, causing parts of my code to be unnecessarily rendered in red (semantic token key: function). This is affecting the readability and usability of the editor for my Python projects.
Upon checking the package.json file, I noticed that "activationEvents": [] is left empty. I attempted to fill in "wolfram", but I'm unsure if this is the correct approach to ensure the extension only activates on Wolfram files.
I know this extension does not seem to be activated, but when I disable it, everything looks normal again.
Could you please reproduce and address this issue? Here is a screenshot of the problem for reference:

Extension Version: 0.0.73
VSCode Version: 1.90.0
Commit: 89de5a8d4d6205e5b11647eb6a74844ca23d2573
Date: 2024-06-04T19:34:48.028Z
Electron: 29.4.0
ElectronBuildId: 9593362
Chromium: 122.0.6261.156
Node.js: 20.9.0
V8: 12.2.281.27-electron.0
OS: Darwin arm64 23.5.0

Hello,
Thank you for testing this code and raising this issue.
You are correct that the language activates automatically because of the empty activationEvents.
I will issue a temporary fix soon which is to change it to:
"activationEvents": ["onLanguage:wolfram"]
See explanation here.

Thank you for your timely reply!

Hi, I have tried the latest version. But the same bug still exists?

Hello - Sorry for the semantic highlighting issue.
We did find that the extension enabled semantic highlighting for all languages, and have fixed it.
Just in case, you can also check your json settings and set
"editor.semanticHighlighting.enabled": false
This can also be set true or false on an individual language basis.
Thank you

Hi Qi, a new version has been posted to address an issue related to the theme colors.
Are you having the same issue?
Get Outlook for iOShttps://aka.ms/o0ukef

From: Qi Zhang @.>
Sent: Sunday, June 23, 2024 12:32:47 AM
To: markjet7/wlsp @.>
Cc: markjet7 @.>; Comment @.>
Subject: Re: [markjet7/wlsp] Extension Activating on Non-Wolfram Code (Python) (Issue #3)
Hi, I have tried the latest version. But the same bug still exists?
—
Reply to this email directly, view it on GitHubhttps://github.com/markjet7/wlsp/issues/3#issuecomment-2184600846, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAGCS6M74C25O4G3DKKLVODZIZMX7AVCNFSM6AAAAABI5SMPIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBUGYYDAOBUGY.
You are receiving this because you commented.Message ID: @.***>

Hi @markjet7, setting editor.semanticHighlighting.enabled to false did solve this problem. However, this is different from the default value of VSCode and might affect other extensions I have installed. Would there be a better way of resolving it? Thank you!

Hi Qi - yes, the latest version 0.0.77 has this fixed already. You can install from the marketplace or GitHub.
Get Outlook for iOShttps://aka.ms/o0ukef

From: Qi Zhang @.>
Sent: Sunday, June 23, 2024 6:56:25 PM
To: markjet7/wlsp @.>
Cc: markjet7 @.>; Mention @.>
Subject: Re: [markjet7/wlsp] Extension Activating on Non-Wolfram Code (Python) (Issue #3)
Hi @markjet7https://github.com/markjet7, setting editor.semanticHighlighting.enabled to false did solve this problem. However, this is different from the default value of VSCode and might affect other extensions I have installed. Would there be a better way of resolving it? Thank you!
—
Reply to this email directly, view it on GitHubhttps://github.com/markjet7/wlsp/issues/3#issuecomment-2185366476, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAGCS6KSGB4SFE7JK374H2TZI5OCTAVCNFSM6AAAAABI5SMPIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBVGM3DMNBXGY.
You are receiving this because you were mentioned.Message ID: @.***>

Hi @markjet7, I was trying 0.0.77. But setting editor.semanticHighlighting.enabled to false seems too strong a constraint. Would you mind providing a more flexible solution? Thank you!


Update MartyrologyDef

Description of the issue
As there are many types of celebrations (not from the liturgical POV), we should require different properties for each of them.
I suggest creating a type for each celebration type.
For romcal needs, I propose to define the following celebration types:

personal celebrations (PersonalCelebration);
event celebrations (EventCelebration);
dedication celebrations (DedicationCelebration).

There might be more celebration types, we can add them when we find them. :wink:
Moreover, there needs to be a special type (albeit not a celebration type) that would be used for collective names, i.s. to define what persons are part of a collective (including companions). For the sake of simplicity, from now on when I say celebration type, I also mean collective names in this sense.
Finally, we might want to consider adding temporal celebrations (Proper of Times), so that we have all celebration metadata defined in a single place. For that, it might be a good idea to define the variable dates in a codeless manner (i.e. just using POJO properties).
Organisation point of view
From the organisation point of view, we can keep all celebration types (and collective names) in a single file (martyrology.ts) or we might decide to move each celebration type (as well as collective names) to separate files. I have no strong opinion on this. For the sake of this issue (in its draft form), I presume we want to keep it as a single file.
Proposal
I propose the following definitions for each celebration type (consider these types as drafts):
/**
 * Celebration type
 *
 * @remarks A special division of the celebrations. We also include collectives as a special celebration type.
 */
enum CelebrationType {
  /** Collective name */
  COLLECTIVE = 'collective',

  /** Dedication celebration */
  DEDICATION = 'dedication',

  /** Event celebration */
  EVENT = 'event',

  /** Personal celebration */
  PERSON = 'person',
}

/** Sex of the collective */
enum CollectiveSex {
  /** Purely female collective */
  female = 'FEMALE',

  /** Purely male collective */
  male = 'MALE',

  /** Mixed-sex collective */
  mixed = 'MIXED',
}

/** Collective name */
type CollectiveName = {
  /**
   * Number of persons in the collective
   *
   * @remarks It could be set to `many` if the number is unknown.
   */
  count: SaintCount;

  /** Collective name */
  name: string;

  /**
   * Sex of the collective
   */
  sex: Sex;

  /** Celebration type */
  type: CelebrationType.COLLECTIVE;

  /** Persons who make up the collective */
  persons?: string[]
}

/** Dedication celebration */
type DedicationCelebration = {
  /** Church which dedication is defined in this object */
  church: name

  /** Dedication date */
  date?: SaintDateDef,

  /** Celebration type */
  type: CelebrationType.DEDICATION;
}

/** Event celebration */
type EventCelebration = {
  /** Celebration name */
  name: string;

  /** Celebration type */
  type: CelebrationType.EVENT;
}

/** Personal celebration */
type PersonalCelebration = {
  /** Beatification metadata */
  beatification?: {
     /** Beatification date */
    date?: SaintDateDef,
     /** Beatification place */
    place?: string
  },

  /** Birth metadata */
  birth?: {
    /**
     * Birth date
     *
     * @remarks It could be defined as a number (year), a string (in `YYYY-MM` or `YYYY-MM-DD` formats), or an object describing date range, multiple possible date, or a century.
     */
    date?: SaintDateDef,

    /** Whether the birth date is approximate */
    isApproximate?: boolean,

    /** Birth place */
    place?: string
  },

  /** Canonisation metadata */
  canonisation?: {
     /** Canonisation date */
    date?: SaintDateDef,
     /** Canonisation place */
    place?: string
  },

  /**
   * The canonization level of a person
   */
  canonizationLevel: CanonizationLevel;

  /** Death metadata */
  death?: {
    /**
     * Death date
     *
     * @remarks It could be defined as a number (year), a string (in `YYYY-MM` or `YYYY-MM-DD` formats), or an object describing date range, multiple possible date, or a century.
     */
    date?: SaintDateDef,

    /** Whether the death date is approximate */
    isApproximate?: boolean,

    /** Death place */
    place?: string
  },

  /** Dedication metadata */
  dedication?: {
     /** Dedication date */
    date?: SaintDateDef,
  }

  /**
   * First name
   */
  name: string;

  /**
   * Sex of the person
   */
  sex: Sex;

  /**
   * Last name
   */
  surname?: string;

  /**
   * Titles by which the person is venarated
   *
   * @remarks Not all titles are used at all places/calendars, however, all titles by which the person is verated at least at one place should be listed here.
   */
  titles?: {
    /** Common titles */
    common?: Title[],

    /** Patronage titles */
    patronage?: PatronageTitle[]
  };

  /** Celebration type */
  type: CelebrationType.PERSON;
}

/**
 * Martyrology definition
 *
 * @remarks This type is used for any item in the celebration catalog.
 */
type MartyrologyDef = CollectiveName | DedicationCelebration | EventCelebration | PersonalCelebration

Notes

I think that hideCanonizationLevel is not necessary after we define which properties are required per celebration type, as it was used by non-personal celebrations (basically event celebrations) only. Therefore, I propose to remove hideCanonizationLevel.
I think that hideTitles is not necessary too, as we should always choose (unless inherited via calendar inheritance) which titles should be included in a celebration within a calendar, as some persons are venerated by different common/patronage titles in different particular calendars.
We might also want to move the martyrology item ID to the property itself as an id.
[PersonalCelebration] While name is (according to the TSDoc) deprecated ATM, I think we should not remove it, but we should change what value it should have. IMHO we should split it to name and surname and define new IDs for these. Moreover, name should have an enum type (or deeply readonly variable → const firstNames = {...} as const), but we need to keep in mind that some names in one locale can have multiple forms in the other (and vice versa). Then, we should localise the names and surnames separately (see #154). I expect some issues with this (some names/surnames are not always localised the same way, e.g. Spanish vs Portuguese surnames), but I still think that at least 90 % of time this will help us more than cause troubles.
[PersonalCelebration] I am also considering making surname required. The only persons that do not have surnames are the twelve Apostles, Jesus (we could use Christ), Our Lady (which is defined more than three dozen times). We could define the surnames for these people.
[PersonalCelebration] Jesus, Our Lady, St Joseph and St John the Baptist have multiple celebrations catalogued, but no personal definition. IMHO we should create a single personal celebration definition in martyrology with a property that would list (or link by ID) all the celebrations.
[PersonalCelebration] I also think we should all the (liturgical) titles of all persons, both common (priest, martyr, etc) and patronage titles (see #221, #201, and also some other issue/discussion where Etienne and I discussed the patronage title definition type). IMHO all patronage places (patron of what is a particular person) should be (similarly to name and surname) should be listed in an enum (or deeply readonly variable) and referenced in the items of martyrology.ts.
[PersonalCelebration] I have split the common titles from the patronage titles into subproperties, as IMHO patronage titles should be defined as a property, not as a string like COPATRON_OF_EUROPE. Note that the PatronageTitle type still needs to be defined.
[PersonalCelebration] We might want to remove PersonalCelebration.canonizationLevel, as we can compute it fairly easily by changed whether canonisation property exists (if yes, it is a saint; if not, it is a blessed). However, I am not opposed to keeping it.
[PersonalCelebration] We might want to add additional properties I might have forgotten.
I think we should also list all places in some enums (or deeply readonly variables), e.g. Continents, Subcontinents, Regions, Countries, Dioceses, Cities (including villages if necessary, but only those mentioned in national Calendar Propers or at least in ordos). Then, we should reference them in martyrology.ts and localise them in the l10n files.
I am also thinking about updating the definition of SaintDateDef.
[CollectiveName] How can we type CollectiveName.persons so that only personal celebration IDs from martyrology.ts can be used? :thinking:
[DedicationCelebration] We should list all churches (as buildings) that we mention/use in romcal. Then, localise them. We should also define the metadata of churches, like type (cathedrals, basilicas, domes, churches, chapels), city (and via city also country, subcontinet, continent, and optional region), and maybe some other metadata.
[PersonalCelebration] We also should add metadata objects for the following information (I am not sure whether we should create a single metadata object for deacon/priest/bishop/cardinal/pope metadata or several ones; however, it would be awesome to only allow defining some properties if a respective common title is defined, e.g. priest metadata can be defined only if Title.Priest is defined in PersonalCelebration.titles.common):

deacon metadata:

consecration date and place;

priest metadata:

consecration date and place;

bishop metadata

consecration date and place;
diocese;
motto;

cardinal metadata

level (deacon, priest, bishop);
creation date and place;
resignation date;
titular church (cardinal deacons: deaconry; cardinal priests: title or title _pro hac vice_ (for this occasion); cardinal bishop: suburbicarian diocese`);

pope metadata

election date and place;
inauguration date and place;
[required] papal name;
resignation date;
motto;

religious metadata

consecration date and place;
religious name;
order/congregation/institute (should we name the property as community?):

I think we should create an object with metadata of the religious orders/congregations/institute, like official and unofficial name, founder, establishment year, cancellation year, type (order/congregation/institute);

martyr metadata:

who killed them (prosecutor?);
martyrdom type:
a. in odium fidei: being killed due to hatred of faith;
b. ex aerumnis carceris: dying in prison for the faith;
c. in defensum castitatis: being killed while defending virginal purity;
d. per testimonium caritatis fortis: being killed as a result of heroic charity;
e. ex acertatibus et vexationibusque pro fidei quibus pertulit: dying as a result of physical or moral violence endured for the sake of the faith.

I think we should also add an optional notes property to each celebration type. While it would not be used by romcal, it might be actually useful to add some additional metadata.

I think it would be nice to add another public getter function to the library that could be used by client apps.

/cc @TobiTenno, you might want to create and apply some additional labels and projects to this issue. Thanks. :pray:

This is overly expansive, and should be better subdivided to have individual parts we can discuss. having 17 opinionated pieces that you want to add makes this hard to discuss moving forward on it. Several of these can probably be their own issue, but any new fields should be separated from any fields you want to remove unless there are pairs that are explicitly entertained. I agree in concept, but we need to make these thin vertical slices that we can move forward on instead of more and more pieces that will collectively hold us from putting smaller pieces into place.
i.e. no. 17 doesn't need to be blocked by adding all of the no. 15 annotations.

TODO

rename WEEK_{1,2} and YEAR_{1,2} to WEEK_{I,II} and YEAR_{I,II}, respectively;

automatically update romcal versions in README.md:

dev versions: romcal@3.0.0-dev.7/romcal@dev;
dev versions: romcal@1.3.1-beta.0/romcal@beta;
latest versions: romcal@3.0.0/romcal/romcal@latest;

in README.md: if we are to mention both npm and yarn, how about mentioning pnpm?

in README.md: fix some grammar (not yet finished):

remove the dedication_of_consecrated_churches_on_* l10n keys (see https://github.com/romcal/romcal/issues/136#issuecomment-1701541394);

fix gabriel_taurin_dufresse_bishop key name (see https://github.com/romcal/romcal/pull/458/files#r1331061512);

commit pnpm-lock.yaml:

decide whether we should remove package-lock.json or update it in GitHub CI from pnpm-lock.yaml (a PNPM command);

diff --git a/README.md b/README.md
index 722ab09..5b72849 100644
--- a/README.md
+++ b/README.md
@@ -84,7 +84,7 @@ The correct entry points are already configured in the package.json so there sho

 ### Calendars installation

-The romcal library only include the General Roman Calendar (GRC), and the Proper of Time. By default, there is no other calendars, neither translation (even in English) nor extra martyrology metadata.
+The romcal library only includes the General Roman Calendar (GRC), and the Proper of Time. By default, no other calendar or translation (even in English) nor extra martyrology metadata.

 The complete GRC, and any other particular calendar (for a country, a region or a diocese) are available as separated plugins, that contain a bundle of the calendar data, localizations, and a martyrology catalog (containing extra metadata).

add .idea;

sort package.json, esp scripts;

consider using .yml extension instead of .yaml;

romcal plugins
I want to discuss the way we implement plugins in romcal.
IMO user should be able to add multiple plugins at once and all of them should be considered during the calendar generation.
In the future, I think we should have plugins for the following tasks (some tasks could be done in a single plugin):

define additional items or update existing ones in the celebration catalogue (martyrology.ts):

when we add celebrations in plugins, we need to be able to add items to the celebration catalogue;
this might be useful for debugging purposes (before merging the changes to romcal);

define additional calendar (-s):

use case: parish calendars with the parish patron, parish church patron, dedications of churches and chapels, patrons of the filial churches (note: I am not sure how are filial churches (as buildings) or filias (churches are a community) called in English; maybe @TobiTenno knows);
this might be useful for debugging purposes (before merging the changes to romcal);

define additional celebrations or modify existing celebrations in a calendar:

this might be useful for debugging purposes (before merging the changes to romcal);

add a way to select which celebration will be celebrated on a particular day (feria vs optional memorials):

this would be useful for planning the liturgical celebrations in a particular parish, or even better: define a celebration/formular for a specific Holy Mass (as different Masses on the same date in the same parish might be celebrated Holy Masses with different formular);
don’t forget about votive Masses (incl those to saints/blesseds inscribed in the Martyrologium Romanum on that day):

OT: this is why I think we should have more persons/celebrations defined in the celebration catalogue (martyrology.ts) than only those that are to be used directly by romcal;

generate a calendar that would include a national calendar and either all or only selected dioceses of that country:

this is how ordos do it (at least in Hungary and Slovakia): they list all celebrations celebrated on the national level and if there are different celebrations celebrated on the diocesan level (e.g. dedications or patronages), they are listed as the last celebration of a day;

define additional locales:

this might be useful for debugging purposes (before merging the changes to romcal);

add additional strings or update existing ones in .names locales:

when we add celebrations in plugins, we need to be able to localise them;
this might be useful for debugging purposes (before merging the changes to romcal);

update mart (comment)

This is overly expansive, and should be better subdivided to have individual parts we can discuss.

@TobiTenno, okay, I’ll convert this issue to discussion.
IMHO, we should discuss and try to get some consensus on how this object (an item in the celebration catalogue) should look like in the end (at least from higher POV), then we should try to create some plan/points on how to achieve the expansion of celebration metadata in a more iterative way.

having 17 opinionated pieces that you want to add makes this hard to discuss moving forward on it.

I agree these suggestions are very opinionated, but they come from my experience with romcal and liturgical calendars in general. From localisation point of view, my biggest issue is #154: I wish romcal could generate celebration names (titles) from individual pieces, as ATM there is much duplication and human errors occurred quite often to either add some typos or use a different capitalisation. All of this can be automated, as long as we have the metadata in place first.
And yes, most of the metadata is used only by some locales, other metadata is not used at all (but they might be a nice addition).

Several of these can probably be their own issue, but any new fields should be separated from any fields you want to remove unless there are pairs that are explicitly entertained.

Yeah, we should create issues for each separate problem, so that we could more in a more iterative manner, possibly the discuss them separately

I agree in concept, but we need to make these thin vertical slices that we can move forward on instead of more and more pieces that will collectively hold us from putting smaller pieces into place.

:100: agreed.

i.e. no. 17 doesn't need to be blocked by adding all of the no. 15 annotations.

Yeah, the list was never meant to be ordered in the implementation order. :wink: I agree that the number 17 is a small feature to add (it can be done right away). :wink:


Some customers can't install the app unless deleted the user's info in the database

Expected Behavior
When a user installed the app, it should redirect to the billing page and can use the app normaly
Current Behavior
Some of the users are experiencing "There's no page at this address". password is blank in the database.
Failure Information
We can't reproduce the issue. But some of our users are experiencing this issue and we can see that there is a lot of users which don't have value in their password field. They can successfully use the app if we remove their information from our database and ask them to reinstall the app.
https://ibb.co/WDqcHgn
Steps to Reproduce
We can't reproduce the error but a lot of our customer is facing the issue.

Go to https://apps.shopify.com/copify-copy-products
Install the app

Context

Package Version: ^17.0
Laravel Version: ^8.40
PHP Version: 7.4
Using Digital Ocean, Polaris, Ploi.iio

Failure Logs
Logs from Sentry.io
https://ibb.co/SBxRvgv

@osiset I'm not sure if this is related to this https://github.com/osiset/laravel-shopify/issues/919

Yes. I can confirm it's related to #919 I tried clicking Add App and entered my shopify store URL. Then went back to https://apps.shopify.com/copify-copy-products then click Add App again


SalesChannelEntity and -Definition missmatch

PHP Version
7.4
Shopware Version
6.4.5
Expected behaviour
$salesChannel->getMaintenanceIpWhitelist() returns a list of ip addresses.
Actual behaviour
The type of the "maintenanceIpWhitelist" property is a string in the SalesChannelEntity.
But according to the definition it is an array, so a type mismatch exception is thrown as soon as you try to read or set the IpAddresses with the entity.
How to reproduce
$salesChannel->getMaintenanceIpWhitelist()

I am closing this then in favor of the P


Java Support

💡 Feature request
Overview and problem statement
Currently workers does not seem to support Java from the documentation:
https://developers.cloudflare.com/workers/platform/languages
TeaVM (https://github.com/konsoletyper/teavm) compiled Java to Javascript from the browser does it mean it can be used with Workers?

Java is on our list of languages to look at for adding examples.  There are a lot of Java to JS compilers, without a clear best option.  If you get a TeaVM example working, pull requests to https://github.com/cloudflare/cloudflare-docs are welcome.  In the meantime, I'd suggest trying the Kotlin or Scala examples, as the compile to JS ecosystem is well defined for them.
Closing this since it's not specific to Wrangler.

Java is on our list of languages to look at for adding examples.  There are a lot of Java to JS compilers, without a clear best option.  If you get a TeaVM example working, pull requests to https://github.com/cloudflare/cloudflare-docs are welcome.  In the meantime, I'd suggest trying the Kotlin or Scala examples, as the compile to JS ecosystem is well defined for them.
Closing this since it's not specific to Wrangler.


Feature connection queue

you have rewritten the master history and changed the checksums this way.
you should not use rebase or merge master into feature before mergeing feature to master.
this this makes it hard to merge for others ;)

That's the way, proposed by github.I assume this is the best practice.
What checksums have been changed exactly? I don't see any merge problems.

oh it is fine - I was just confused. ;)
sry


Parallelization problem of the Conda Normaliz distribution version 3.9.1 on MS Windows

I have tested the MS Windows normaliz.exe 3.9.1 in the Conda distribution. Unfortunately it suffers from the same parallelization problem as the MacOS binary 3.9.1 in the Normaliz distribution. (The MS Windows binary and the Linux binary 3.9.1 in the Normaliz distribution are not affected.)
I plan to publish 3.9.2 next week. It will have an extra computation step as a precaution against this problem that I have so far only seen in binaries computed with newer versions of clang. Of course this does not mean that the problem is caused by the compiler. Perhaps we have only been lucky so far.
On Linux I have seen it only with clang++-11. clang++-12 does not show it in GitHub workflows. clang++13 will be tested the next days.

It seems that the cause of the problem has been found. See https://github.com/Normaliz/Normaliz/issues/385.

I assume that this has been fixed in the more recent releases?


checkpoint file

Hi, could you share the checkpoint-0.910.h5 file please?
thanks!

Please check in the README.
Drive link is there.

Hi, the link for checkpoint-0.910.h5 is dead, please share the file.

Hi, the link for checkpoint-0.910.h5 is dead, please share the file.


Unresponsive VSCode after upgrade to 1.26.

Hello!
After installing VSCode 1.26, the GUI is almost completely unresponsive when I open my project. After a few minutes, it can be used. However, the CPU usage will remain at 24% (i have 4 cores). Even after the project or folder is closed.
If I open VSCode without a project or folder, it behaves.
VSCode is responsive if launched with --disable-extensions.
Could it be an extentions that misbehaves?
Here is a --status dump:

Version:          Code 1.26.0 (4e9361845dc28659923a300945f84731393e210d, 2018-08-13T16:29:31.933Z)
OS Version:       Windows_NT x64 10.0.16299
CPUs:             Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz (8 x 2712)
Memory (System):  15.85GB (3.24GB free)
VM:               0%
Screen Reader:    no
Process Argv:     C:\Users\forsbdan\AppData\Local\Programs\Microsoft VS Code\Code.exe
GPU Status:       2d_canvas:                    enabled
checker_imaging:              disabled_off
flash_3d:                     enabled
flash_stage3d:                enabled
flash_stage3d_baseline:       enabled
gpu_compositing:              enabled
multiple_raster_threads:      enabled_on
native_gpu_memory_buffers:    disabled_software
rasterization:                enabled
video_decode:                 enabled
video_encode:                 enabled
webgl:                        enabled
webgl2:                       enabled
CPU %   Mem MB     PID  Process
0      100   34680  code main
0       84   22972     shared-process
0      146   23460     gpu-process
23     1271   28764     window (Program.cs - demo2 - Visual Studio Code)
0        6   26188       winpty-process
0       65   16204         C:\WINDOWS\System32\WindowsPowerShell\v1.0\powershell.exe
0       10   19472         console-window-host (Windows internal process)
0       51   30296       searchService
0       14   31020       electron-crash-reporter
0       11   31384       watcherService
0       10   21716         console-window-host (Windows internal process)
0      191   33784       extensionHost
0       38   22568         searchService
0        3   37816         cmd /s /c "C:\Users\forsbdan.vscode\extensions\ms-vscode.csharp-1.15.2.omnisharp\1.30.1\OmniSharp.exe -s c:\src\demo2\Fagkveld.sln --hostPID 33784 --stdio DotNet:enablePackageRestore=false --encoding utf-8 --loglevel information formattingOptions:useTabs=false formattingOptions:tabSize=4 formattingOptions:indentationSize=4"
0       10    7140           console-window-host (Windows internal process)
0      115   19888           C:\Users\forsbdan.vscode\extensions\ms-vscode.csharp-1.15.2.omnisharp\1.30.1\OmniSharp.exe  -s c:\src\demo2\Fagkveld.sln --hostPID 33784 --stdio DotNet:enablePackageRestore=false --encoding utf-8 --loglevel information formattingOptions:useTabs=false formattingOptions:tabSize=4 formattingOptions:indentationSize=4
Workspace Stats:
|  Window (Program.cs - demo2 - Visual Studio Code)
|    Folder (demo2): 83 files
|      File types: json(15) cshtml(12) cs(8) js(5) ts(5) scss(5) cache(5)
|                  map(3) vue(3) txt(2)
|      Conf files: launch.json(2) tasks.json(2) sln(1) csproj(1)
|                  package.json(1) tsconfig.json(1) webpack.config.js(1)
|                  settings.json(1)
|      Launch Configs: coreclr(2)

experiencing the same issue - for a split second i can click into the text area but within a second the ui is unresponsive and after a minute the entire app show a "this window is no longer responding message"

@ly-cultureiq Thanks for your references.
By the way, I think it isn't cause by the size of python file but syntax color rendering. Could you please have a try on the following code in a new .py file with python mode.
By the way, remember to test it in a new workspace in case of crashing your own workspace.
record = {
    "headers": {k: str(v) for k, v in self.request.META.items() if k.startswith('HTTP_')}
}

@ly-cultureiq @jtr109 please file that against the python extension
@kabforks can you also reproduce this if you open an empty vscode window? Or you only see it if you open a C# project? Do you also see the freeze if you disable the c# extension?

Hi @isidorn:
Thanks for your reply.
I comment with my screenshot in the issue I reported. Unfortunately, the issue I meet can reproduced with python extension disabled.

Forwarding to @aeschli since he is handling all these not repsonsive language issues with the new update

dup of #56377


[Task Tracker] Update all existing settings to work with stardust / new api

We need to make sure all existing settings work and if not update them with new app logic and UI.
General

[x] Theme
[x] Language
[x] Currency
[x] Notifications
[x] Network Status
[x] Change profile name

Security

[ ] Export backup
[ ] Automatic lock
[ ] Change password
[ ] Change pincode
[ ] #3318

Advanced

[ ] #3319
[ ] Deep linking
[ ] #3287
[x] Hidden wallets
[ ] Error log
[ ] Crash reports
[ ] Diagnostics

Help & Information

[ ] Documentation
[ ] FAQ
[x] Discord
[x] Report an issue

I think while we're adjusting the settings, we could potentially break it into five:

General - the app-wide settings, i.e. theme, language, notifications (alt. these could be per-profile), etc.
Profile - per-profile settings like currency (alt. could be general setting since that is more in line with language), network config, lock screen timeout, network statistics, delete profile (alt. could be security setting), hidden wallets
Security - export backup, automatic lock, change pincode / password
Advanced - Network config, deep links, crash reports, error log, balance finder
Help / Info - docs, faq, discord, report an issue


Kyverno PolicyExeptions

Introduce policy exceptions for applications which can't fullfill deployed policy.
i.e. bpdm due to usage of opensearch is unable to adhere to non-root user policy

Adjust policy for current valid issues, than activate the policy hard


Initial support for GTFS Fares v2 - Fare Containers and Payment Methods

Currently, a proposal to add fare containers and payment methods to the GTFS spec is being discussed in https://github.com/google/transit/pull/355.  This is an initial tracking bug to add validation support for the new files and fields introduced in that proposal.

Thanks for opening this @bdferris-v2, I am labeling it as blocked until the PR is merged.

Sounds good.  I actually managed to get the validation logic mostly implemented yesterday.  You can see a preview at:
https://github.com/MobilityData/gtfs-validator/compare/issue/1301/fare_containers?expand=1
I haven't generated a PR yet because the code is blocked by #1302.
For the most part, the validation was pretty straightforward (a good sign to me!), though it did lead to the two comments I added to the spec proposal doc.


'link' buttons for headers with anchors on hover

The 'link to this section' buttons that show up when you hover over headers that have anchors: e.g. in https://github.blog/2019-02-14-introducing-draft-pull-requests/  hover over Tag your work in progress

these can be really nice when trying to share a link / example, and a lot of the docs pages have anchors that are basically never used at the moment because there's no indication they are there unless you read the markup

Looks great. let's add this.


hello_world_user: don't always assume userspace

Print whether or not the code is actually in usermode.

This tiny sample is really useful when bringing up userspace support or
when working on its components, and most effective if executed outside of
twister's constraints for such development tasks.
I contribute this exactly because it happened to me that userspace wasn't
effective when I did expect it to be. This change turned out to be useful
to me so I thought of contributing it to help the next person who might
run into the same kind of issues.
But if this isn't deemed useful to others then I'll simply drop it and
keep the knowledge to myself.

Codecov Report

Merging #44710 (70c4841) into main (c11b785) will decrease coverage by 0.00%.
The diff coverage is 66.66%.

:exclamation: Current head 70c4841 differs from pull request most recent head 8ef7dad. Consider uploading reports for the commit 8ef7dad to get more accurate results

@@            Coverage Diff             @@
##             main   #44710      +/-   ##
==========================================
- Coverage   49.77%   49.77%   -0.01%
==========================================
  Files         642      642
  Lines       79750    79751       +1
  Branches    18730    18730
==========================================
- Hits        39699    39694       -5
  Misses      33487    33487
- Partials     6564     6570       +6

Impacted Files
Coverage Δ

subsys/net/lib/sockets/sockets.c
40.14% <0.00%> (ø)

arch/x86/core/prep_c.c
92.30% <100.00%> (+0.64%)
:arrow_up:

subsys/testsuite/ztest/src/ztest_new.c
66.50% <100.00%> (ø)

subsys/testsuite/ztest/src/ztest.c
54.28% <0.00%> (-5.24%)
:arrow_down:

subsys/net/ip/net_pkt.c
68.23% <0.00%> (-0.29%)
:arrow_down:

arch/x86/core/early_serial.c
70.83% <0.00%> (+29.16%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c11b785...8ef7dad. Read the comment docs.


docker container does not start

/usr/local/bin/teslafi_exporter:4: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  import distutils.util as util
INFO:root:Using TeslaFi API Token: <removed>
Traceback (most recent call last):
  File "/usr/local/bin/teslafi_exporter", line 1021, in <module>
    TESLAFI_COLLECTOR = TeslaFiCollector(teslafi_api_token)
  File "/usr/local/bin/teslafi_exporter", line 32, in __init__
    registry.register(self)
  File "/usr/local/lib/python3.10/site-packages/prometheus_client/registry.py", line 26, in register
    names = self._get_names(collector)
  File "/usr/local/lib/python3.10/site-packages/prometheus_client/registry.py", line 66, in _get_names
    for metric in desc_func():
  File "/usr/local/bin/teslafi_exporter", line 94, in collect
    self.getSetData(teslafi_data, teslafi_data_old, "vin"),
  File "/usr/local/bin/teslafi_exporter", line 63, in getSetData
    if data_old[key] is None or data_old[key] == '':
KeyError: 'vin'

This happens when TeslaFi has not been able to poll any data yet. You can try opening your Tesla app and waiting a few mins so that TeslaFi can try polling data again.


Missing stacktrace when using custom protocol in electron

I use @sentry/electron version 0.16.0 with my electron app. I use custom protocol in my app, it looks like this:
const mainWindow = new BrowserWindow({
    show: false,
    title: 'My app',
    webPreferences: {
        nativeWindowOpen: true,
        webSecurity: false,
        allowRunningInsecureContent: false,
    },
});
mainWindow.loadURL(`protocol://app/index.html`);

When I try to call captureException manually in renderer process I get empty stacktrace in my Sentry instance, all errors tell me that the problem is protocol://app/index.html. Here is an example  gist with json event.
My investigations led me here,to @sentry/browser's tracekit module, function computeStackTraceFromStackProp. When I monkeypatched chrome regexp this way ^\s*at (?:(.*?) ?\()?((?:file|https?|blob|chrome-extension|native|eval|webpack|<anonymous>|[a-z]+:|\/).*?)(?::(\d+))?(?::(\d+))?\)?\s*$ (added plus after <anonymous>|[a-z]) I began getting all stacktraces, so my protocol began to be recognized by tracekit module.
So, what I'm to do with this problem? I'm not quite sure, should I open issue here or in sentry-javascript project. but let me start here since it's eletron-speciefic issue.

Thanks for reporting this!
From a security standpoint, it is now recommended to use a custom protocol instead of file:// in Electron.  file:// can access any files on the users machine and a custom protocol can tightly limit the protocols access to specific directories. For this reason it's going to be important to support this.
@HazAT @kamilogorek what do you thing about this change in @sentry/browser?

@timfish hello. Sorry for bothering you, but my futher investigation brought me to a conclusion that normalizeUrl function in normalize.ts module of @sentry/electron should be also modified in order to support source-maps properly. For now stacktraces are correctly bound to uploaded source files, but according to this docs they are not source-map friendly. Or maybe some misconfiguration from my side =)
Just to make sure my config is ok. My catalog tree looks like this
|-webpack.config.js
|-sentry.properties
|+dist
 |+renderer
  |-index.html
  |+static
   |+js
    |-main.js
    |-main.js.map

My protocol://app points to dist/renderer folder, so protocol://app/index.html resolves to dist/renderer/index.html file and etc. My SentryCliPlugin config is the following:
{
  release: 'release',
  include: path.resolve(__dirname, 'dist', 'renderer', 'static', 'js'),
  configFile: path.resolve(__dirname, 'sentry.properties'),
  urlPrefix: '~/static/js',
}

Here is the example gist of sentry event
So, since my sources and maps are located in the same directory and uploaded with same prefix and  since source files are resolved properly I guess something is wrong with stacktraces. What do you think about that?

@timfish I think we can even be more generic, as protocol is inside non-capturing group anyway and change:
(?:file|https?|blob|chrome-extension|native|eval|webpack|<anonymous>|[a-z]+:|\/)

to just something like:
(?:[a-zA-Z]+|chrome-extension|<anonymous>:\/\/)

Thoughts @HazAT?

The latest sentry-javascript uses the following that fixes this:
(?:file|https?|blob|chrome-extension|native|eval|webpack|<anonymous>|[-a-z]+:|\/)


slickFilter and slickUnfilter methods not implemented yet

Not going to be implemented, the same functionality can be achieved as with dynamic slides. See dynamic slides example.


How to write test case for  ?

this doesnt works for me

You probably need to set deviceWidth not width in the ResponsiveContext.Provider value prop

I've just lost like 3 hours because this is not well documented. Could you please review the documentation around testing in the Readme?
I'm guessing this is not the first time someone gets stuck (and tries mocking matchMedia in their test env). Thanks in advance!

@arturmiglio If there are improvements you'd like to see in the documentation please feel free to make a PR to add more information. I try as best as I can to make the docs informative but it is a large surface area and hard to document every single possible way it could be used.


Disable recursive routes in server-side router

Hi,
I was playing around with this tutorial, and I had trouble getting the posts section to work using the server side rendering.  It would crash each time I attempted to load http://localhost:3030/posts/1.  Client-side rendering worked fine.
I tracked the issue down to recursive route handling in Director.  It looks like both the /posts and /posts:id routes were being applied, causing a mismatch in callback parameters and subsequent crash.
It's hard to say, but this may have been a change on Director's part.  According to their README, server-side routing has 'backwards' recursion turned on by default.  Looking at the code, however, it is set to 'forward.'  And it seems to have been changed in an unrelated commit that didn't update the README.
This pull request just configures the DirectorRouter to disable recursive routes, after which the tutorial works on both client- and server-side loads.
Also, thank you for putting this tutorial together.  It has been a huge help in wrapping my head around this architecture.  Cheers,
-Will

Hi wakamoleguy,
thanks for your investigation, while I was also playing with this tutorial I've discover the same problem. Good to see exactly where the problem came from.


Difference in behavior based on OS

🐛 Bug
I ran into this and I wanted to share with you. Maybe you know if something can be done. There is a difference in the extracted information you get when you treat a document (in my case French ID cards in .jpg) with doctr running on Windows or running in a dockerised Linux environment.
To Reproduce
Steps to reproduce the behavior:

Extract each words from the result of DocumentFile.from_images(doc) on Windows and on a Docker
Compare the extracted box content and position

Windows
word     min_x     min_y  ...  page_id  document_name  label
0        FRANÇAISE  0.535156  0.092773  ...        0              1      O
1       REPUBLIQUE  0.097656  0.106445  ...        0              1      O
2      Nationalité  0.703125  0.177734  ...        0              1      O
3       D'IDENTITE  0.284180  0.184570  ...        0              1      O
4               No  0.394531  0.184570  ...        0              1      O
5        Française  0.807617  0.176758  ...        0              1      O
6   CARTENATIONALE  0.101562  0.185547  ...        0              1      O
8             Nom:  0.335938  0.249023  ...        0              1      O
11       Prénom(s)  0.340820  0.349609  ...        0              1      O
14         Né(elle  0.560547  0.435547  ...        0              1      O
16          Sexe:M  0.333984  0.434570  ...        0              1      O
17               à  0.346680  0.501953  ...        0              1      O
Docker
word     min_x     min_y  ...  page_id  document_name  label
0        FRANÇAISE  0.535156  0.092773  ...        0              1      O
1       REPUBLIQUE  0.097656  0.106445  ...        0              1      O
2      Nationalité  0.702148  0.177734  ...        0              1      O
3       D'IDENTITE  0.286133  0.183594  ...        0              1      O
4        Française  0.807617  0.177734  ...        0              1      O
5              Nos  0.395508  0.183594  ...        0              1      O
6   CARTENATIONALE  0.101562  0.185547  ...        0              1      O
8             Nom:  0.335938  0.249023  ...        0              1      O
11       Prénem(s)  0.341797  0.348633  ...        0              1      O
14            Sexe  0.339844  0.441406  ...        0              1      O
15         Né(e)le  0.560547  0.433594  ...        0              1      O
18               a  0.343750  0.500977  ...        0              1      O
Expected behavior
Results should be the same
Environment
Windows
DocTR version: 0.3.0
TensorFlow version: 2.5.0
PyTorch version: N/A (torchvision N/A)
OpenCV version: 4.5.2
OS: Microsoft Windows 11 Pro
Python version: 3.8
Is CUDA available (TensorFlow): No
Is CUDA available (PyTorch): N/A
CUDA runtime version: 11.0.221
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin\cudnn_ops_train64_8.dll
Docker
DocTR version: 0.3.0
TensorFlow version: 2.5.0
PyTorch version: N/A (torchvision N/A)
OpenCV version: 4.5.2
OS: Debian GNU/Linux 10 (buster)
Python version: 3.8
Is CUDA available (TensorFlow): No
Is CUDA available (PyTorch): N/A
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA

Hey @Rob192,
Thanks for reporting this! That's indeed a strange behaviour consider most of the word predictions are identical :thinking:
It's gonna be tricky to find, but it might indicate that the segmentation --> box conversion, or the cropping functions are behaving differently depending on the OS.
If it was coming from the TF models themselves, none of the output would be the same. Would you mind checking the output of the detection predictor in both cases? :pray:  (or if you can share the image that you used)

Hello @fg-mindee,
I realized that the difference between the two ouput actually came from the preprocessing I am doing before feeding the image to Doctr. For my use case, I use cv2.SIFT_create + cv2.FlannBasedMatcher to match a reference CNI card at a reference position. The image resulting from this transformation is clearly different between the Docker and Windows for whatever reason. Therefore Doctr gives different results as well. If I bypass this preprocessing step, there is no difference between Doctr outputs on Windows or Linux.
Sorry for the confusion, I think we can close this.

Thanks for the clarification @Rob192 :pray:


Mrcjkb/issue#533

This pull request got messed up with one of my other branches.
I will close it and create a new one.


Recover messages

Summary

One paragraph explanation of the feature.
Help me solve my own problems

Motivation
Who's my boyfriend I'm a very one woman person

Why are we doing this? What use cases does it support? What is the expected outcome? To prove that I'm not a cheater

@rosie77 is this real? 😂 If yes, could you please expand a little bit?


This error has surfaced again..

Using version 1.13.4,
Traceback (most recent call last):
  File "testConsumerLoop.py", line 5, in <module>
    import ec_lib_adx_consumer_fxns
  File "../../app/ecs/ec_lib_adx_consumer_fxns.py", line 3, in <module>
    import credstash
  File "/Users/mafisher/creatives-backend/services/eligibility-adx-consumer/QA/tests/credstash.py", line 55, in <module>
    'RIPEMD': hashes.RIPEMD160,
AttributeError: 'module' object has no attribute 'RIPEMD160'

Any suggestions?

See #178.


ArgumentError on interpolated string

On a billing page I have the following code:
<.img<EMAIL_ADDRESS>/>

The formatter errors with the following:
mix format failed for file: lib/heads_down_web/live/subscription_live/edit.html.heex
** (ArgumentError) Invalid inlined elixir function:
<EMAIL_ADDRESS>-- missing interpolation terminator: "}" (for string starting at line 1)
    lib/tailwind_formatter.ex:61: anonymous fn/1 in TailwindFormatter.validate_inline_fns/1
    (elixir 1.14.3) lib/enum.ex:975: Enum."-each/2-lists^foreach/1-0-"/2
    lib/tailwind_formatter.ex:58: TailwindFormatter.validate_inline_fns/1
    lib/tailwind_formatter.ex:19: TailwindFormatter.format/2
    (mix 1.14.3) lib/mix/tasks/format.ex:576: Mix.Tasks.Format.format_file/2
    (elixir 1.14.3) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2
    (elixir 1.14.3) lib/task/supervised.ex:34: Task.Supervised.reply/4
    (stdlib 4.2) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

Thank you so much for this library!

Thanks for the report!
Didn't consider nested interpolation.
Now using negative lookahead within the Regex to account for nesting.
I'll be making a release soon, but if you could try the latest commit on your repo and see if any other issues crop up I would appreciate it.
You'll want to put:
      {:tailwind_formatter, github: "100phlecs/tailwind_formatter", override: true},

in your mix deps.
Let me know if you run into any other issues. The more test cases the better :-)


add kichain testnet to registry

For testnets, please use a different chainname.  Chain names are meant to unique, even between mainnets and testnets.

Can you remove the dash please
We decided to keep chain names as only alphanumeric

(also update the folder name too)

Thanks!

@sunnya97 sorry, for the format of the JSON, I fixed the issues the trailing comas 🤦


Test

Testing needed on app creation, deletion, etc... when pip installed from github
Docstring to add
Readme to write properly

Docstring & readme added, testing to be finalised

All done!


assume not working with chained roles

Hi,
first of all, thanks for this great tool, since I jump around between dozens of AWS accounts each day this is really a great addition!
When trying to use a combination of an assume-role chained with an AWS SSO auth, I get an error:
My ~/.aws/config contains this:
[profile jump-account]
sso_start_url = https://REDACTED.awsapps.com/start/#
sso_region = eu-west-1
sso_account_id = FIRST-ACCOUNT-ID
sso_role_name = some-role
region = eu-west-1
output = json

[profile target-account]
source_profile = jump-account
role_arn =  arn:aws:iam::OTHER-ACCOUNT-ID:role/TargetRole
output = json
region = eu-west-1

When I first do an aws sso login --profile jump-account and then run a command like aws s3 ls --profile target-account, everything works.
When I try to do this using assume, I get:
bsh ❯  assume jump-account

[jump-account](eu-west-1) session credentials will expire 2022-03-15 20:29:39 +0100 CET

bsh ❯ assume target-account
operation error STS: AssumeRole, https response error StatusCode: 400, RequestID: 2aa7685d-04ac-406a-a2a7-ee4f48b196db, api error ValidationError: 1 validation error detected: Value '' at 'tokenCode' failed to satisfy constraint: Member must have length greater than or equal to 6

Some things to add:

the role in the target account (arn:aws:iam::OTHER-ACCOUNT-ID:role/TargetRole) can be assumed by the role setup for AWS SSO, of course
AWS SSO is setup in the jump-account (FIRST-ACCOUNT-ID)

Is this case supposed to work, or just not supported (yet)?
Please let me know if there is anything I can help you with - apparently, this would resolve the one thing left to fully rely on this nice setup!

This looks to be an issue with MFA, based on the tokenCode error.
Do you have MFA setup for this target-account role?
Possibly some default MFA configured?
Out of the box, Granted will work with most role chaining with SSO roles however we haven't tested this with chaining to a role that requires MFA

Hi Joshua!
Thanks for the quick reply. MFA is configured for my AWS SSO user right inside AWS SSO. The target account (or the role therein) does not have any MFA requirements / settings.
Please note that we do not use any IAM users - since these are the entities in IAM where you can attach MFA.
Is MFA supported in general?
Is there anything I can provide - logs, configs - to find out more?

The same issue happens on my system (MacOS) I have a similar chained role setup as @jbl-proglove

Thanks a lot for the fix, works perfectly now!


Add traits ParallelDrainRange and ParallelDrainFull

These define methods of the same name, so only one is expected to be
implemented for any given collection type. Collections that support
ranges can always call par_drain(..) for a full drain.
pub trait ParallelDrainRange<Idx = usize> {
    type Iter: ParallelIterator<Item = Self::Item>;
    type Item: Send;
    fn par_drain<R: RangeBounds<Idx>>(self, range: R) -> Self::Iter;
}

pub trait ParallelDrainFull {
    type Iter: ParallelIterator<Item = Self::Item>;
    type Item: Send;
    fn par_drain(self) -> Self::Iter;
}

These define methods of the same name,

I did this so these traits can be seen as extension traits mirroring par_drain to each collection's inherent drain, but some take an argument (like ParallelDrainRange) and some always drain everything (like ParallelDrainFull).

bors r+


[SuperEditor] Feat: Add ability to add EditRules depending on TextEdigingDelta

Description of the feature
For our company it would be necessary that SuperEditor allows us to customise the experience of the editor.
For example, if the user presses enter (on its software OR hardware keyboard) after a header, the editor will add a ListItemNode afterwards automatically.
Another example would be to split a TaskNode when the user presses enter while the selection is inside a TaskNode. As far as I know this is only available through keyboardActions right now, which are only valid for Hardware Keyboards.
Proposed solution
For this purpose Flutter Quill uses "rules". These rules are executed as a chain of responsibility and each one of these rules have access to the delta text.
Once #870 is merged, I propose to add a new concept called EditorRule, which would look like this:
abstract class EditorRule {
  ExecutionInstruction apply(
    DocumentEditor editor,
    DocumentComposer composer,
    TextEditingDelta delta,
  );
}

class EditorRules {
  const EditorRules({
    this.rules = const [],
  });
  final List<EditorRule> rules;

  ExecutionInstruction apply(DocumentEditor editor, DocumentComposer composer, TextEditingDelta delta) {
    for (final rule in rules) {
      final execution = rule.apply(editor, composer, delta);
      if (execution == ExecutionInstruction.blocked || execution == ExecutionInstruction.haltExecution) {
        return execution;
      }
    }

    return ExecutionInstruction.continueExecution;
  }
}

Then, the DocumentEditor would accept these rules through its constructor, and delegate the execution of these rules to a SoftwareKeyboardHandler or a HardwareKeyboardHandler.
In my case I just added these two lines
      final execution = editor.rules.apply(editor, composer, delta);
      if (execution != ExecutionInstruction.continueExecution) continue;

on the applyDeltas method:
 void applyDeltas(List<TextEditingDelta> textEditingDeltas) {
    editorImeLog.info("Applying ${textEditingDeltas.length} IME deltas to document");

    for (final delta in textEditingDeltas) {
      editorImeLog.info("Applying delta: $delta");

      final execution = editor.rules.apply(editor, composer, delta);
      if (execution != ExecutionInstruction.continueExecution) continue;

      if (delta is TextEditingDeltaInsertion) {
        _applyInsertion(delta);
      } else if (delta is TextEditingDeltaReplacement) {
        _applyReplacement(delta);
      } else if (delta is TextEditingDeltaDeletion) {
        _applyDeletion(delta);
      } else if (delta is TextEditingDeltaNonTextUpdate) {
        _applyNonTextChange(delta);
      } else {
        editorImeLog.shout("Unknown IME delta type: ${delta.runtimeType}");
      }
    }
  }

Usage
Now we are able to add the SplitTaskRequest whenever we detect the user has pressed enter while the selection is in a TaskNode:
class SplitTaskRule implements EditorRule {
  @override
  ExecutionInstruction apply(DocumentEditor editor, DocumentComposer composer, TextEditingDelta delta) {
    if (delta is! TextEditingDeltaInsertion || delta.textInserted != '\n') return ExecutionInstruction.continueExecution;

    final selection = composer.selectionComponent.selection;
    if (selection == null || !selection.isCollapsed) return ExecutionInstruction.continueExecution;

    final nodeId = selection.base.nodeId;
    final node = editor.document.getNodeById(nodeId);
    if (node is! TaskNode) return ExecutionInstruction.continueExecution;

    if (node.text.text.isEmpty) {
      editor.execute(NodeToParagraphRequest(nodeId: nodeId));
    } else {
      editor.execute(
        SplitTaskRequest(
          nodeId: nodeId,
          splitPosition: selection.extent.nodePosition as TextNodePosition,
          newNodeId: DocumentEditor.createNodeId(),
        ),
      );
    }
    return ExecutionInstruction.haltExecution;
  }
}

Opinions

Do you think this feature would make sense?
Do you have some concerns about this approach?
What others approaches would make more sense?

@raulmabe-labhouse can you look at the current state of Super Editor on GitHub and let me know if you're able to accomplish these goals?  We've introduced an edit pipeline, which has requests, commands, and reactions. You still don't have direct access to the IME behaviors, but I think you might have enough access to accomplish the goals you listed in this ticket.

Can you please provide two code examples: First, the current workaround that you have to do with Super Editor that you'd prefer not to do. Second, a hypothetical example demonstrating your ideal setup?

Sounds good, I will as soon as I have some spare time 👍🏼


[nodejs][09.message-routing] Sample crashes after receiving greeting

Sample information

Sample type: \samples\
Sample language: nodejs
Sample name: 09.message-routing

Describe the bug
The sample runs smoothly, responds as it should when a message containing help or cancel is received by the user but when a greeting is received, the sample throw the following error
[onTurnError]: TypeError: Cannot read property 'options' of undefined
(node:6528) UnhandledPromiseRejectionWarning: TypeError: Cannot perform 'set' on a proxy that has been revoked
    at adapter.sendActivities.then (C:\Users\username\Documents\Repos\BotBuilder-Samples\samples\javascript_nodejs\09.message-routing\node_modules\botbuilder-core\lib\turnContext.js:175:36)
    at process._tickCallback (internal/process/next_tick.js:68:7)
(node:6528) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:6528) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

To Reproduce
Steps to reproduce the behavior:

Follow the installation instructions in the README
Run the bot with the Bot Framework Emulator
Send 'Hello' to the bot
See the error [onTurnError]: TypeError: Cannot read property 'options' of undefined

Expected behavior
After sending 'Hello', the bot should ask the user for it's name and city and keep running without interruptions.
[bug]

Fixed with PR.721