Persistent error when attempting to use  tool with the correct input format

The final agent and tool in my crew is having an issue where it will not accept the input from its preceding crew member.  The input is coming in as a JSON object and seems to be formatted properly but the tool will not accept it.  Here is the Output:
Action Input:
{
  "coworker": "Expert Query Analyzer and Classifier",
  "question": "I am encountering a persistent error when attempting to use the Document_Synthesis tool with the correct input format. The error states that the Action
Input is not a valid key, value dictionary, despite the format appearing correct. Can you help me understand why this error is occurring and how to resolve it?",
  "context": "I am trying to synthesize information from a document titled 'The_role_of_human_factors_in_aviation_ground_operation_related_accidents.pdf' using the Document_Synthesis tool. The input provided is a JSON dictionary with the query and the document details, which is the expected format for the tool. However, the tool is
returning an error message stating that the input is not valid. Here is the input I have been using: {\"query\": \"What are some common variables used in studies regarding human error-based aviation accidents\", \"documents\": [{\"title\": \"The_role_of_human_factors_in_aviation_ground_operation_related_accidents.pdf\", \"path\": \"C:/Users/Admin/Desktop/erdcDBFunc/crewAIDocSum/documents/The_role_of_human_factors_in_aviation_ground_operation_related_accidents.pdf\"}]}"
}

Final Answer:
I apologize for the inconvenience, but due to a persistent error with the tools provided, I am unable to access the document and synthesize the information required to answer the query regarding common variables used in studies of human error-based aviation accidents. My goal was to provide a detailed synthesis report that addressed the query comprehensively, but without the ability to analyze the document, I cannot fulfill this task. If you have any further questions or need assistance with a different query, please let me know, and I will do my best to assist you.

Oh that is an interesting one, can you share more of the logs so we can debug a little further?

Yes, I have made a few changes to the tools, and tasks assigned to the various agents. I can share the tools and tasks files and the full output of the thought process.

The problem seems to be the agent is getting data passed into its tool with json formatting for example:
Action Input:
{
  "query": "What are some common variables used in studies regarding human error-based aviation accidents",
  "documents": [
    {
      "title": "The Role of Human Factors in Aviation Ground Operation-Related Accidents",
      "path": "C:/Users/Admin/Desktop/erdcDBFunc/crewAIDocSum/documents/The_role_of_human_factors_in_aviation_ground_operation_related_accidents.pdf"
    }
  ]
}

Thought: It seems there was an error in the action input format. I need to correct the input format to ensure it is a valid key-value dictionary as expected by the Document_Synthesis tool.
How do I prevent this formatting and just pass the inputs as a key value pair?

Weird we should threat for that, can you share a little more of the log surrounding that error?

@P7201, @joaomdmoura The problem is the Action Input is not passed verbatim, rather wrapped inside a ```json...``` block. As a workaround, you may want to specifically ask in the task not to do so - i.e., add the following to the task:
IMPORTANT INSTRUCTIONS:
------------------------------
- Action Input, although a JSON dictionary, must *never* be surrounded by backticks!
  It must be passed verbatim as a JSON dictionary.
- "```json ... ```" is not allowed!!!. This is critical, as the tool will not be able to function correctly if this is done.

I find that the agent doesn't always adhere to these instructions on the first attempt, but corrects itself on the second, once tool execution fails. IMO it is better to handle it in the output parser.

@avri-schneider for me it is like
It appears there is a persistent issue with the format of the input. I will ensure that the format strictly adheres to the expected key-value structure. Let's break down the task and ensure each key-value pair is correctly formatted within the input dictionary.

Action: Delegate work to coworker
Action Input:
{
    "task": "Create an investment decision on the stock SPAL based on the provided financial details and analysis.",
    "context": {
        "stock_name": "SPAL",
        "exchange": "NSE",
        "end_date": "2023-12-31",
        "investment_horizon": "long",
   }
}

Still getting Error: the Action Input is not a valid key, value dictionary.

Can you try the following Action Input?
{
"task": "Create an investment decision on the stock SPAL based on the
provided financial details and analysis.",
"context": "{"stock_name": "SPAL", "exchange": "NSE",
"end_date": "2023-12-31", "investment_horizon": "long"}"
}
On Sun, Jun 23, 2024 at 11:06 PM Ojasv Singh @.***>
wrote:

@avri-schneider https://github.com/avri-schneider for me it is like
It appears there is a persistent issue with the format of the input. I will ensure that the format strictly adheres to the expected key-value structure. Let's break down the task and ensure each key-value pair is correctly formatted within the input dictionary.
Action: Delegate work to coworker
Action Input:
{
"task": "Create an investment decision on the stock SPAL based on the provided financial details and analysis.",
"context": {
"stock_name": "SPAL",
"exchange": "NSE",
"end_date": "2023-12-31",
"investment_horizon": "long",
}
}
Still getting Error: the Action Input is not a valid key, value
dictionary.
—
Reply to this email directly, view it on GitHub
https://github.com/joaomdmoura/crewAI/issues/547#issuecomment-2185309770,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABTYRHMS7BXAUJP6PANEO33ZI4TFNAVCNFSM6AAAAABHCIJJ5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBVGMYDSNZXGA
.
You are receiving this because you were mentioned.Message ID:
@.***>

I finally got this to work by being very specific in the task file:
return Task(
description=f"""
Wait until the document_summary_agent has completed their task. Then, using the Query_and_Document_Summary_Analysis tool analyze
the given user query: {query}
to ascertain the specific information required from the document summaries found here: {summaries_path}.
- Use the provided summaries_path to access and review document summaries.
- The input format for the action should be a Python dictionary
- The input should be a Python dictionary, but it MUST NOT be enclosed in triple backticks or have a JSON label.
- The output will be a list of dictionaries detailing relevant documents, including titles and paths.
- Extract key words, phrases, and underlying questions from the user's query using advanced NLP techniques.
- Match these extracted elements with the information in the document summaries from the summaries loacted in the summaries_path to determine which document(s) could potentially answer the query.
- The process should be meticulous to ensure that all possible documents that could answer the query are considered,
""",
agent=agent,
async_execution=False,
expected_output="""
- Provide a list of dictionaries. Each dictionary should contain:
- 'title': The title of the document.
- 'path': The file path to the document.
- You MUST structure your output in the form of a list of dictionaries.
- The input SHOULD NOT be enclosed in triple backticks
- The input SHOULD NOT have a JSON label.
- The input SHOULD NOT have a python label
The list format will facilitate the subsequent processing tasks without needing further parsing.
"""
)

I finally got this working, I had to be very specific in the task description and expected output:
return Task(
description=f"""
Wait until the document_summary_agent has completed their task. Then, using the Query_and_Document_Summary_Analysis tool analyze
the given user query: {query}
to ascertain the specific information required from the document summaries found here: {summaries_path}.
- Use the provided summaries_path to access and review document summaries.
- The input format for the action should be a Python dictionary
- The input should be a Python dictionary, but it MUST NOT be enclosed in triple backticks or have a JSON label.
- The output will be a list of dictionaries detailing relevant documents, including titles and paths.
- Extract key words, phrases, and underlying questions from the user's query using advanced NLP techniques.
- Match these extracted elements with the information in the document summaries from the summaries loacted in the summaries_path to determine which document(s) could potentially answer the query.
- The process should be meticulous to ensure that all possible documents that could answer the query are considered,
""",
agent=agent,
async_execution=False,
expected_output="""
- Provide a list of dictionaries. Each dictionary should contain:
- 'title': The title of the document.
- 'path': The file path to the document.
- You MUST structure your output in the form of a list of dictionaries.
- The input SHOULD NOT be enclosed in triple backticks
- The input SHOULD NOT have a JSON label.
- The input SHOULD NOT have a python label
The list format will facilitate the subsequent processing tasks without needing further parsing.
"""
)


[ppconsul] Add new port

ppconsul is C++ client library for Consul. Consul is a distributed tool for discovering and configuring services in your infrastructure.

Thanks for the new port!  It looks like it is failing on the uwp triplets, in most new libraries this is by design because uwp is not supported upstream.  Even it it is supported that can be handled by a separate PR later.


feat: add avatarFallback prop to RainbowKitProvider

Adds a new optional avatarFallback prop to <RainbowKitProvider>. The options are "emojis" (default) and "nouns".
"nouns" will generate a unique SVG using the connected address as a seed.
Implementation example:
root.render(
  <React.StrictMode>
    <WagmiConfig client={wagmiClient}>
      <RainbowKitProvider chains={chains} avatarFallback="nouns">
        <App />
      </RainbowKitProvider>
    </WagmiConfig>
  </React.StrictMode>
);

Note: I haven't updated the docs since I'm not entirely sure where it fits. It's not a "Custom Avatar", but also doesn't quite fit in the "Theming" section.

@gskril A neat addition. Would like to merge this after a few tweaks for maintainability


Szechun

Refactor group parsing
Improve Find functionality (able to input part of the name and still be able to show the full name/person) instead of exact name previously

Codecov Report

Merging #39 (55359df) into master (6842871) will decrease coverage by 0.39%.
The diff coverage is 19.04%.

@@             Coverage Diff              @@
##             master      #39      +/-   ##
============================================
- Coverage     59.06%   58.66%   -0.40%
- Complexity      422      423       +1
============================================
  Files            90       90
  Lines          1600     1616      +16
  Branches        172      170       -2
============================================
+ Hits            945      948       +3
- Misses          600      613      +13
  Partials         55       55

Impacted Files
Coverage Δ

...eedu/address/logic/parser/AssignCommandParser.java
0.00% <0.00%> (ø)

.../seedu/address/logic/parser/CreateGroupParser.java
0.00% <0.00%> (ø)

...c/main/java/seedu/address/model/person/Person.java
67.18% <0.00%> (ø)

...a/seedu/address/logic/parser/AddCommandParser.java
51.85% <33.33%> (-2.70%)
:arrow_down:

...in/java/seedu/address/commons/util/StringUtil.java
95.00% <100.00%> (+0.26%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6842871...55359df. Read the comment docs.


Suggestion: semantic html elements

DaisyUI would greatly improve the readability of code if the "component" classes would be HTML tags that can be styled with the utility classes.
Short example:

    I open with focus

  <collapse-title class="text-xl font-medium">
    I open with focus
  </collapse-title>

Long example Instead of this:

    I open with focus

Collapse content reveals with focus. If you add a checkbox, you can control it using checkbox instead of focus. Or you can force-open/force-close using
      collapse-open and
      collapse-close classes.

This
<collapse tabindex="0" class="w-96">
	<collapse-title class="text-xl font-medium">I open with focus</collapse-title>
	<collapse-content class="">

			Collapse content reveals with focus. If you add a checkbox, you can control it using checkbox
			instead of focus. Or you can force-open/force-close using
			<badge class="badge-outline">collapse-open</badge> and
			<badge class="badge-outline">collapse-close</badge> classes.

	</collapse-content>
</collapse>

TL;DR – we finally have a way to create our own HTML elements and use them in any framework we want!
Thanks to the popularity of frameworks such as Angular, Vue, and React, component-driven development has become a part of our every day lives. Components help us encapsulate styles and behaviors into reusable building blocks. They make a lot of sense in terms of design, development, and testing.
Unfortunately, framework-specific components fail us in a number of ways:
You can only use them in the framework they're designed for 🔒
Their lifespan is limited to that of the framework's ⏳
New frameworks/versions can lead to breaking changes, requiring substantial effort to update components 😭
Web components solve these problems. They're supported by all modern browsers, they're framework-agnostic, and they're part of the standard, so we know they'll be supported for many years to come.
TL;DR on web components  from shoelace

@samuelstroschein
Thanks for the detailed comment.
There's one big problem about this approach:
When there's a .btn class, we can use that class everywhere: , <button class="btn">, ,... but when we use <btn> it's useless.
Tailwind CSS provides utility classes and daisyUI adds component classes on top. So we're still using a utility first system.

Adding default styling to semantic HTML tags does not entail removing the component classes. I meant that DaisyUI should apply (maybe opt-in) the component classes by default to those HTML elements.
<btn class="btn-outline">Semantic</btn>

Works as well

This doubles the size of CSS files (in both source and package) but I'm not sure what problem is being fixed here 🤷♂

Why would it require duplicative CSS files? TailwindCSS allows adding base styles to HTML elements. The only additional step that is required is defining the custom HTML element tags such as <collapse-title></collapse-title> etc but they are more less just divs with a semantic name.

I'm not sure what problem is being fixed here 🤷♂

It's a code readability enhancement. Instead of scaffolding meaningless elements like
 all over the markup, that only get meaning with the component class e.g.
, the relatively new introduction of custom HTML tags  would restore semantics e.g. <collapse-title>.
Why is this beneficial?
Look at the code from the cards example. Congrats, everything is a meaningless div. A developer now needs to read the class first to understand which component belongs to which div.

That is not too problematic when using inspect but by just looking at the source code, don't you agree that having a semantic tag name (which is a div under the hood having the component class applied by default) would improve the readability?

I came here to request something similar as I am exploring the examples and wondered why the navigation elements are all 's and why  wasn't being used.


License

Would be great if you could specify the terms of usage for the code in this repo by adding a LICENSE file.

Closed by #10


ImportError: cannot import name 'KeyVaultAuthentication' from 'azure.keyvault' (unknown location) - when running az keyvault certificate import

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az keyvault certificate import
Errors:
The command failed with an unexpected error. Here is the traceback:

cannot import name 'KeyVaultAuthentication' from 'azure.keyvault' (unknown location)
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/knack/cli.py", line 206, in invoke
    cmd_result = self.invocation.execute(args)
  File "/usr/lib/python3/dist-packages/azure/cli/core/commands/__init__.py", line 608, in execute
    raise ex
  File "/usr/lib/python3/dist-packages/azure/cli/core/commands/__init__.py", line 666, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/usr/lib/python3/dist-packages/azure/cli/core/commands/__init__.py", line 659, in _run_job
    six.reraise(*sys.exc_info())
  File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/azure/cli/core/commands/__init__.py", line 636, in _run_job
    result = cmd_copy(params)
  File "/usr/lib/python3/dist-packages/azure/cli/core/commands/__init__.py", line 306, in __call__
    return self.handler(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/azure/cli/command_modules/keyvault/_command_type.py", line 96, in keyvault_command_handler
    client = client_factory(self.command_loader.cli_ctx, command_args)
  File "/usr/lib/python3/dist-packages/azure/cli/command_modules/keyvault/_client_factory.py", line 18, in keyvault_data_plane_factory
    from azure.keyvault import KeyVaultAuthentication, KeyVaultClient
ImportError: cannot import name 'KeyVaultAuthentication' from 'azure.keyvault' (unknown location)

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az keyvault certificate import --file {} --name {} --vault-name {} --password {} {} {} {} {} {}

Expected Behavior
Environment Summary
Linux-5.11.0-1023-azure-x86_64-with-glibc2.29
Python 3.8.10
Shell: bash

azure-cli 2.0.81

Extensions:
azure-devops 0.17.0

Additional Context

keyvault

keyvault

What does that mean?
What is the resolution?

azure-cli 2.0.81 is not our official Azure CLI. See #19640 for details
Please install our CLI following https://docs.microsoft.com/cli/azure/install-azure-cli

resovled


Web is not starting in 6-task-queueing

This is what I get
root@b75e5b6f5b9b:/app# bundle exec foreman start --formation web=1
21:06:12 web.1          | started with pid 117
21:06:14 web.1          | config.eager_load is set to nil. Please update your config/environments/*.rb files accordingly:
21:06:14 web.1          |
21:06:14 web.1          |   * development - set it to false
21:06:14 web.1          |   * test - set it to false (unless you use a tool that preloads your test environment)
21:06:14 web.1          |   * production - set it to true
21:06:14 web.1          |
21:06:14 web.1          | /app/config/initializers/omniauth.rb:7:in `block in <top (required)>': undefined method `[]' for nil:NilClass (NoMethodError)

For some reason config/settings.yml is not loading.
 I am using the datastore backend. Which backend are you using?

I do receive the following error when the workers run. I will continue looking into it
13:26:49 worker.2       | Book lookup request<PHONE_NUMBER>079040)
13:26:49 worker.2       | rake aborted!
13:26:49 worker.2       | NoMethodError: undefined method `find_by_id' for Book:Class
13:26:49 worker.2       | /usr/local/google/home/betterbrent/Code/getting-started-ruby/6-task-queueing/lib/active_job/queue_adapters/pub_sub_queue_adapter.rb:40:in `block in run_worker!'
13:26:49 worker.2       | /usr/local/google/home/betterbrent/Code/getting-started-ruby/6-task-queueing/vendor/bundle/ruby/2.2.0/gems/gcloud-0.4.0/lib/gcloud/pubsub/subscription.rb:376:in `block (2 levels) in listen'
13:26:49 worker.2       | /usr/local/google/home/betterbrent/Code/getting-started-ruby/6-task-queueing/vendor/bundle/ruby/2.2.0/gems/gcloud-0.4.0/lib/gcloud/pubsub/subscription.rb:376:in `each'
13:26:49 worker.2       | /usr/local/google/home/betterbrent/Code/getting-started-ruby/6-task-queueing/vendor/bundle/ruby/2.2.0/gems/gcloud-0.4.0/lib/gcloud/pubsub/subscription.rb:376:in `block in listen'
13:26:49 worker.2       | /usr/local/google/home/betterbrent/Code/getting-started-ruby/6-task-queueing/vendor/bundle/ruby/2.2.0/gems/gcloud-0.4.0/lib/gcloud/pubsub/subscription.rb:373:in `loop'
13:26:49 worker.2       | /usr/local/google/home/betterbrent/Code/getting-started-ruby/6-task-queueing/vendor/bundle/ruby/2.2.0/gems/gcloud-0.4.0/lib/gcloud/pubsub/subscription.rb:373:in `listen'
13:26:49 worker.2       | /usr/local/google/home/betterbrent/Code/getting-started-ruby/6-task-queueing/lib/active_job/queue_adapters/pub_sub_queue_adapter.rb:36:in `run_worker!'
13:26:49 worker.2       | /usr/local/google/home/betterbrent/Code/getting-started-ruby/6-task-queueing/lib/tasks/run_worker.rake:4:in `block in <top (required)>'

interesting, I'm using datastore. I was running the 'foreman start' from within the docker container that was built by container builder during the gae deploy. maybe there is a problem with the base docker image. Can you try 'gcloud preview app gen-config' and then docker build/run and see if that works?


Simulation from bulk RNA-seq

Is it possible to use a bulk RNA-seq dataset as input to scDesign3?

Hi Dr. Ji,
Yes, scDesign3 can simulate bulk RNA-seq data; although this is not included in our paper, my collaborators have used it in simulating bulk data. Conceptually, as long as the input is a count matrix, we can use it as the input. The challenge here is that the bulk sample size is smaller.
Best,
Dongyuan

This is great. Thanks!


LLMBenchmark: hermetic environment for reproducible results

Currently LLMBenchmark runtime environment handling is far from optimal.
Issues:

Container created on each execution from scratch which makes it almost impossible to have reproducible results.
Different model families has different package requirements which may conflict with each other.
No debug logs from container_exec  which makes is hard to debug issues.

This patch set migrate to hermetic container image which build from deterministic Dockerfile.

Migrate to Dockerfile
Add container commands logging to Output/runner.log

Bug fixes:

Meta-Llama-3.1-405B-FP8 use bfloat16, but benchmark.py executed with hardcoded float16
llmbench: fix engines build parameters for larger batch_sizes

@microsoft-github-policy-service agree


Update to support latest googleapis release?

Pending: https://codereview.chromium.org/1334533003/


[Question] How can we configure it to isolate requests from accessing these internal interfaces of shenyu-bootstrap?

Question
As the entry point for traffic following load balancing, the Shenyu gateway directs all subdomains directly to shenyu-bootstrap. However, some API interfaces in shenyu-bootstrap are unauthenticated and can be accessed directly via domain + URI, such as /actuator, /actuator/env, etc. How can we configure it to isolate requests from accessing these internal interfaces of shenyu-bootstrap?

in shenyu bootstrap , you can config the internal interface like this :

you can write a api  start with  /shenyu , such as /shenyu/detail
you can config health path in yaml , shenyu.health.paths

shenyu:
  health:
    enabled: true
    paths:
      - /actuator
      - /health_check


first command run for zsh does not have the right status

updated Insider's and the problem remains

@meganrogge I think this is the same as https://github.com/microsoft/vscode/issues/143766? We will need to always send the exit code and have it get evaluated on the renderer against the extracted command name to fix these problems.

interesting to me that it allows repro on linux/WSL and does so inconsistently on mac

K, let's try understand this a little better then, that shouldn't happen as the initial history id is 0:
https://github.com/microsoft/vscode/blob/5357dcfc04ddfce12661dca80ae9a104640a2c7e/src/vs/workbench/contrib/terminal/browser/media/shellIntegration-rc.zsh#L25

Oh I figured out when it reproes on mac - if you run the same command in a different terminal, then the second and beyond ones don't have an exit code

It would be nice to fix this since zsh is the default shell used

Think I see the problem, PR incoming.


Update tutorials from feedback + chain upgrade

add json_prolog, uri_encoded, open predicates
fix base64 commands
some typos, illustrations, ...

@amimart @jeremylgn The commit 60615bf is not compliant with conventional commit rules.
commit 60615bfbab3e8955ff2579b437a4712be503334e
Author: Anthony<EMAIL_ADDRESS>Date:   Fri Jul 7 16:44:06 2023 +0200

    bring corrections on commands & others from Joachim feedback

However the lint-commits job  was successful. Why? it's problematic because the commit has been merged, even though it doesn't conform.


Theme settings should be stored in user.json

VSCode Version: Version 1.4.0 (1.4.0)
OS Version: OS X/macOS 10.11.6 (15G31)

Steps to Reproduce:

Set theme
Check configuration files

Storing the selected theme in the storage.json is a bad idea for several reasons:

storage.json seem to be meant to store temporary and local state.

The set theme can hardly be seen as neither temporary, not local state.

Trying to make a sensible backup of settings is confusing, unncessarily hard, and error prone

To make a sensible backup, one has to (1) backup /Users/abc/Library/Application Support/Code/storage.json for the theme, (2) $HOME/Library/Application Support/Code/User/* for the actual preferences, (3) ~/.vscode/* for extensions. Spreading these out over multiple places is not the moment of least surprise.
Trying to make a sensible backup of settings can accidentally leak non-disclosable information. storage.json contains many file paths which potentially expose undesired information for the public. There is no easy way to just extract settings from that file that are not "recently used files" or "window locations".

Additionally, workspaceStorage should not be in $HOME/Library/Application Support/Code/User but rather in /Users/abc/Library/Application Support/Code. It follows the same criteria as /Users/abc/Library/Application Support/Code/storage.json.

I agree. @aeschli also fyi
I think as soon as we can programmatically change user.json via API we should consider to store the theme in settings.

Apologies if I sounded hard; I just reread my own post. It was meant as constructive critique for future work to make this the best editor experience possible, not https://xkcd.com/386/

related/duplicate of #7358 ?

correct. That's a dup of #7358


Windows Terminal Dev Build crashes on start as of 5330759c

Environment
Windows build number: 10.0.20206.0
Windows Terminal version (if applicable): dev build (master branch)

Any other software?
Visual Studio 16.7.2

Steps to reproduce

Check out any commit 5330759c..7a03f75e
Build and deploy with Visual Studio 2019 (in Release x64 mode)
Click on "Windows Terminal Dev" in Start Menu or Taskbar

Expected behavior
Windows Terminal starts as before
Actual behavior
A grey window without a titlebar appears briefly, then disappears:

Other notes
An error at WindowsTerminal/main.cpp:132 occurs when running from Visual Studio:

The err.exe report for 0xC000027B does not seem immediately useful:
# for hex 0xc000027b / decimal -1073741189
  STATUS_STOWED_EXCEPTION                                        ntstatus.h
# An application-internal exception has occurred.
# as an HRESULT: Severity: FAILURE (1), FACILITY_NULL (0x0), Code 0x27b
# for hex 0x27b / decimal 635
  ERROR_COMMITMENT_MINIMUM                                       winerror.h
# {Virtual Memory Minimum Too Low}
# Your system is low on virtual memory. Windows is increasing
# the size of your virtual memory paging file.
# During this process, memory requests for some applications
# may be denied. For more information, see Help.
# 2 matches found for "0xC000027B"

git bisect nailed the bad commit down to 5330759c:
git bisect start
# bad: [7a03f75ee9b905bd796d4d6a02f7cff3d5b1e472] Keep degenerate UIA text ranges degenerate after movement (#7530)
git bisect bad 7a03f75ee9b905bd796d4d6a02f7cff3d5b1e472
# good: [6acb9f8c90b1e4bc7837fd83fd60afddb46760bf] schema: swap `closeTabsAfter` and `closeOtherTabs` (#7386)
git bisect good 6acb9f8c90b1e4bc7837fd83fd60afddb46760bf
# good: [7803efa6fef8d2efd0d8fd93333cc958d8431882] Make GlobalAppSettings a WinRT object (#7349)
git bisect good 7803efa6fef8d2efd0d8fd93333cc958d8431882
# good: [97c2ccf08bbd2d2c24a33ccc415506a2fab6ffd2] Merge remote-tracking branch 'openconsole/inbox' into HEAD
git bisect good 97c2ccf08bbd2d2c24a33ccc415506a2fab6ffd2
# bad: [5330759c0f3d98b92209813c48e781a664fe22eb] Update to a newer MUX prerelease; remove workaround for compact sizing (#7447)
git bisect bad 5330759c0f3d98b92209813c48e781a664fe22eb
# good: [9279b7a73d3d6e0783ef38fa9cf2ad4406c14824] Add profiles to the Jumplist (#7515)
git bisect good 9279b7a73d3d6e0783ef38fa9cf2ad4406c14824
# first bad commit: [5330759c0f3d98b92209813c48e781a664fe22eb] Update to a newer MUX prerelease; remove workaround for compact sizing (#7447)
commit 5330759c0f3d98b92209813c48e781a664fe22eb
Author: Marcel Wagner<EMAIL_ADDRESS>Date:   Fri Sep 4 22:00:40 2020 +0200

    Update to a newer MUX prerelease; remove workaround for compact sizing (#7447)

    Update the WinUI version which allows us to remove the workaround.

    Closes #6681

 src/cascadia/CascadiaPackage/CascadiaPackage.wapproj           |  4 ++--
 .../LocalTests_TerminalApp/TerminalApp.LocalTests.vcxproj      |  4 ++--
 .../LocalTests_TerminalApp/TestHostApp/TestHostApp.vcxproj     |  2 +-
 src/cascadia/TerminalApp/TerminalAppLib.vcxproj                |  4 ++--
 src/cascadia/TerminalApp/TerminalPage.cpp                      | 10 ----------
 src/cascadia/TerminalApp/dll/TerminalApp.vcxproj               |  4 ++--
 src/cascadia/TerminalApp/packages.config                       |  2 +-
 src/cascadia/WindowsTerminal/WindowsTerminal.vcxproj           |  4 ++--
 src/cascadia/WindowsTerminal/packages.config                   |  2 +-
 .../WindowsTerminalUniversal/WindowsTerminalUniversal.vcxproj  |  4 ++--
 src/cascadia/ut_app/TerminalApp.UnitTests.vcxproj              |  2 +-
 11 files changed, 16 insertions(+), 26 deletions(-)

It appeared in commit 5330759 (PR #7447) because of a bug in MUX (microsoft/microsoft-ui-xaml#3104). There's already a PR for MUX, which should fix it (microsoft/microsoft-ui-xaml#3118), but it needs to be merged and included in a new (pre)release, before it can be updated in terminal.


Issue with fullscreen api

OS: Windows 8.1
Browser: Google Chrome
API version - 3.16.1
I am using following code to enter and leave fullscreen
    handleFullScreen() {
        if (this.scale.isFullscreen) {
            this.scale.stopFullscreen();
        } else {
            this.scale.startFullscreen();
        }
    }

This function is called on a button click. This works fine and can enter as well as exit fullscreen mode when I use the button but if I use "esc" key to exit fullscreen mode then it no longer works. Debugging it in console gives following error

phaser.min.js:1 Uncaught DOMException: Failed to execute 'insertBefore' on 'Node': The new child element contains the parent.
    at initialize.getFullscreenTarget (http://localhost:8080/test/js/phaser.min.js:1:405372)
    at initialize.startFullscreen (http://localhost:8080/test/js/phaser.min.js:1:404881)
    at TheGame.handleFullScreen (http://localhost:8080/test/js/game.js:150:24)
    at initialize.h.emit (http://localhost:8080/test/js/phaser.min.js:1:10074)
    at initialize.processUpEvents (http://localhost:8080/test/js/phaser.min.js:1:627540)
    at initialize.update (http://localhost:8080/test/js/phaser.min.js:1:622087)
    at initialize.updateInputPlugins (http://localhost:8080/test/js/phaser.min.js:1:429636)
    at initialize.queueMouseUp (http://localhost:8080/test/js/phaser.min.js:1:432154)
    at onMouseUp (http://localhost:8080/test/js/phaser.min.js:1:420581)

Further debugging shows that opening fullscreen mode is inserting a DOM element which is deleted on exit fullscreen mode but if "esc" key is used to exit fullscreen mode, then that element is not getting deleted so next time trying to open fullscreen mode fails.

I see this same behavior on macOS / Safari.

A fix to this is to set a parent property of the scale object in the game config like this:
config = { scale: { parent: 'parent' ... } ... };

I already have parent property set in the game config, still this error occurs.

The scale part of the config needs its own parent option. I can confirm without it, this error happens and with it, it does not when paired with the correct parent.

This is what I have in my game config. Please advise what do you think needs a change here
    let config = {
        type: Phaser.AUTO,
        scale: {
            parent: 'mygame',
            mode: Phaser.Scale.RESIZE,
            width: '100%',
            height: '100%'
        },
        backgroundColor: 0x000000,
        scene: TheGame
    }

    let game = new Phaser.Game(config);

Html body is simply defined as

Thank you for submitting this issue. We have fixed this and the fix has been pushed to the master branch. It will be part of the next release. If you get time to build and test it for yourself we would appreciate that.

The error is gone but there is another issue. "Esc" key works fine in case of FIT mode but in RESIZE mode, it sends wrong width and height (which is the fullscreen width and height) so now the game cuts off in RESIZE mode. Pressing "Esc" key will not resize the RESIZE mode game to new width and height. It works fine in both modes when it is done via a button click.


account.getBalance hangs for some reason on js side

Here: https://github.com/encointer/encointer-wallet-flutter/blob/d9cefe6761aec1221bfe076bb3c3f45b2c4e7d1b/lib/service/substrateApi/apiAssets.dart#L15

It crashes due to  DRR: TypeError: this.digest.logsWith is not a function or its return value is not iterable
most likely a compatibiliyt error of polkadot-api with the chain

Will be fixed when 16aff44cb09bf293feb4a44141d9fa1cf366d735 is merged


How to configure 2 Kafka Binders when using Zipkin Kafka

In my application i have a scenario where I have to consume messages from a Kafka topic and persist them into a DB. I am using Spring Cloud Stream Kafka Binder to listen to messages from the kafka topic.
In the same application, I have Zipkin also configured to send my trace to Zipkin Server and we are planning to send the zipkin messages to another kafka binder and have zipkin read them and display in the Zipkin UI.
But I am not sure how to do because for both the Application to read the business message and Zipkin Kafka are using Spring Cloud Kafka Binder.
Below the configuration in the application.properties. Any help will be helpful.
# Spring Cloud Kafka Binder for Consuming messages from a Business Transaction Topic
spring.cloud.stream.kafka.binder.zkNodes=businessMessageBroker1:2181
spring.cloud.stream.kafka.binder.brokers=businessMessageBroker1:9092
spring.cloud.stream.bindings.output.destination=BunisessTopic

# Spring Cloud Kafka Binder for Posting to Zipkin-Kafka
spring.cloud.stream.kafka.binder.zkNodes=hostname2:2181
spring.cloud.stream.kafka.binder.brokers=hostname2:9092
spring.cloud.stream.bindings.output.destination=sleuth

Can you provide any information about your Sleuth version? If you've migrated to the approach where you're using Zipkin's Kafka connector, you can alter the destination via spring.zipkin.kafka.topic (the default value is zipkin. The kafka sender uses the bootstrap.servers property to retrieve a list of broker addresses.
We describe in the documentation that we no longer support Sleuth with Stream. If you migrate to the approach described here https://cloud.spring.io/spring-cloud-static/Edgware.SR3/single/spring-cloud.html#_span_data_as_messages then I guess your problem should go away since Zipkin's Kafka sender doesn't use Stream.

Spring Cloud Slueth version which we are using is 1.3.4.BUILD-SNAPSHOT.

Ok, what about the elements I pointed in the documentation?

In my application we have used the spring-cloud-sleuth-stream and included the spring-cloud-starter-stream-kafka. I am not able follow the documentation which you have mentioned. Because my understanding is that when we integrate slueth-stream and stream-kafka then we will be able to send the zipkin notifications to a kafka topic.
The problem is how can configure the broker url in application.properties for when two different brokers are present.
Spring Cloud Kafka Binder for Consuming messages from a Business Transaction Topic
spring.cloud.stream.kafka.binder.zkNodes=businessMessageBroker1:2181
spring.cloud.stream.kafka.binder.brokers=businessMessageBroker1:9092
spring.cloud.stream.bindings.output.destination=BunisessTopic
Spring Cloud Kafka Binder for Posting to Zipkin-Kafka
spring.cloud.stream.kafka.binder.zkNodes=hostname2:2181
spring.cloud.stream.kafka.binder.brokers=hostname2:9092
spring.cloud.stream.bindings.output.destination=sleuth

In my application we have used the spring-cloud-sleuth-stream and included the spring-cloud-starter-stream-kafka. I am not able follow the documentation which you have mentioned. Because my understanding is that when we integrate slueth-stream and stream-kafka then we will be able to send the zipkin notifications to a kafka topic.

Your understanding is wrong.  slueth-stream is deprecated and should be migrated as described in the documentation.


Batch issues yet again.

Hi Thanks for great library.
but this issue happened again with some strange scenario
I have two functions like this one
one function work like expect but this one process stalled at first .batch() wiith no error no return just process timeout

` res.setHeader('Content-Type', 'application/json; charset=utf-8')
            StringStream
              .from(async function* () {
                const response = await axios.get(csv_url, {
                  responseType: 'stream'
                })
                yield* response.data.pipe(await stripBomStream())
              }, { maxParallel: 4 })
              .CSVParse({ skipEmptyLines: true, header: true })
              .filter((item: any) => (item.Class_Id && parseInt(item.Class_Id) !== 0 && item.Class_Discount > 0))
              .map(async (item: any) => {

                const master_name = master_subgroup.find(
                  (master_item: any) => (parseInt(master_item.id) === parseInt(item.Class_Id)))

                if (!master_name.name || master_name.name === "") return {}
                all_data_count++
                batch_data_count++
                console.log("pass 3")
                const class_obj = {
                  id: item.Class_Id,
                  name: master_name.name,
                  discount_rate: parseFloat(item.Class_Discount),
                  parent_id: master_name.parent_id,
                  status: "active"
                }

                return class_obj

              })
              .batch(500)  < process Stalled here
              .map(async (items: any) => {
                console.log("All Items is : " + items.length)
                const start_at = (batch_count * round_count) + 1
                round_count += 1
                const end_at = (start_at + items.length) - 1
                const range = start_at + "-" + end_at

                const unique_items = discount.objectToArray(discount.arrayToObject(items, "id"))
                const complete_subgroup = unique_items.filter((item: any) => {
                  return Object.keys(item).length > 0
                })
                console.log("Master group is : " + JSON.stringify(complete_subgroup))

                if (complete_subgroup.length > 0) {
                  const discount_obj = { subgroup: complete_subgroup }
                  console.log(discount_obj)
                  console.log("before send")
                  const update_result = await discount.update(req.params.uid, discount_obj)

                  console.log("before result")
                  console.log(update_result.success)
                  console.log("before return")
                  if (update_result.success) {
                    return JSON.stringify({
                      rows: range,
                      received: items.length,
                      accepted: unique_items.length - 1,
                      status: "import_successful",
                      message: "ข้อมูลได้รับการบันทึกแล้ว"
                    }
                    )
                  } else {
                    return JSON.stringify({
                      rows: range,
                      received: items.length,
                      accepted: 0,
                      status: "problem_with_database",
                      message: "ไม่สามารถบันทึกข้อมูล ช่วงข้อมูลที่: " + range,
                    })
                  }
                } else {
                  return JSON.stringify({
                    rows: range,
                    received: items.length,
                    accepted: 0,
                    status: "data_id_not_exits",
                    message: "ช่วงข้อมูลที่: " + range + " รหัสไม่มีอยู่จริงในระบบ ",
                  })
                }
              })
              .catch((error: any) => {
                console.log("catch block 1" + error.stack)
                if (error.code === "ERR_SCRAMJET_EXTERNAL") {
                  res.statusCode = error.cause.response.status
                  const error_obj = makeErr("can_not_reach_product_api",
                    "ไม่สามารถติดต่อ Product API ได้ กรุณาตรวจสอบ",
                    error.cause.response.status,
                    process.env.PRODUCT_API_URL)
                  res.send(JSON.stringify(error_obj))
                  return
                } else {
                  res.statusCode = 422
                  const error_obj = makeErr(error.code, error.message, 422)
                  res.send(JSON.stringify(error_obj))
                  return
                  //  return JSON.stringify(error_obj)
                }
              })
              .batch(10000)
              .stringify((resp: any) => {
                // console.log(resp)
                res.statusCode = 200
                return "[" + resp + "]"
              })
              .catch((error: any) => {
                console.log("catch block 3")
                if (error.code === "ERR_SCRAMJET_EXTERNAL") {
                  res.statusCode = error.cause.response.status
                  const error_obj = makeErr(error.cause.response.data.code,
                    error.cause.response.data.message,
                    error.cause.response.status)
                  //res.send(JSON.stringify(error_obj))
                  return JSON.stringify(error_obj)
                } else {
                  res.statusCode = 500
                  const error_obj = makeErr(error.code, error.message, 500)
                  // res.send(JSON.stringify(error_obj))
                  return JSON.stringify(error_obj)
                }
              })
              .toStringStream()
              .pipe(res)`

Originally posted by @KwangGan in https://github.com/signicode/scramjet/issues/2#issuecomment-523340771

@KwangGan if you could post a sample CSV link here so we could try to reproduce the problem.
Two things may be important:

The csv should include similar data - like if there's some Thai chars it would be good to provide similar example
The length of the csv - if that's smaller than the batch size we should keep it that way.

Here's the overall bugfixing procedure:

Isolate the test case reproducing the issue
Add the test case to scramjet/test/methods (or if that's multi-method we'll find a better place)
Fix the issue.
Confirm issue resolution on your end.
Bugfix release.

Thank you for reopen issue

I tried to increase and decrease batch size but no lock even 1 or 500  per batch

here my CSV that contain only EN alphabet.
https://ybin.me/p/b2a673d1480d1c94#WGKBUy47K9F/ZaRyg5/xEd581xCW9hy+E6VkQOfmCKE=

here is my code using multer_s3 and express js
https://ybin.me/p/4f62a44fa86cb9a7#W2Asc7kv9vX2JQw2D8TqUikJPgwMRiB4Sjyhz73fUxs=

node 10.15

My solution for now is Downgrade to v4.23  and everything works as expect :D

Ok, good, I'll check the differences over the weekend and I'll try to figure out what went down.
4.23 may rely on a bit older papaparse, but that should not affect the parsing. There may be some things missing around rate and exec, but I don't see you using this.

@KwangGan I have tested batch in your scenario and I don't think it's our case (I've added some special tests that should cover the case).
I believe your code may be affected by a change in pull which in turn handles the function* slightly different. Can you try a slightly different way of getting the stream?
StringStream
              .from(async function () {
                const response = await axios.get(csv_url, {
                  responseType: 'stream'
                })
                return response.data.pipe(stripBomStream())
              }, { maxParallel: 4 })

I'm not sure if this helps but the answer will lead me somewhere...

@KwangGan could you create some kind of a repo (or fork of scramjet) that would have a test for you scenario?


Adidas splash page rejects most user agent's.

If you take a screenshot after a task has started and put in the actual splash page, many of them give an error saying that the current browser is "not currently supported" and to "update to a modern browser". This is due to the user agents being outdated.

Here are some example user agents that I've tested.
UserAgent.txt

Also do not use any mobile device user agents. It adidas refuses connection when it attempts to redirect to m.adidas.com
UserAgen.txt

Hey Conrad!
Thank you for bringing this to my attention.  I'm going to collect some modern user-agents and confirm they are working.
Do you think it would be a better idea to have a single user-agent rather than randomizing the header on each driver?  I'm thinking it might be better to just assign "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36" as the header for each driver.
Best,
Christopher

If you are using proxies I don't see the harm in using the same user agent. I guess its not a bad thing, I just don't think its completely necessary.

This may not be the proper place to ask this... but how would I go about using your bot?

You can use the python module  to have a humongous database of User agents to use


⚠ Bofrost.de has degraded performance

In ebf2bd5, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1755 ms

Resolved: Bofrost.de performance has improved in 8d65d30 after 21 minutes.


translation:module 'tensorflow' has no attribute 'VariableAggregation

I get the "AttributeError: module 'tensorflow' has no attribute 'VariableAggregation'" error when running translate. The program version specified by DOCKFILE is cuda9 and tensorflow-gpu1.9. However, in the TENSORFLOW API documentation, the tensorflow1.9 version does not have the VariableAggregation attribute. . I tried upgrading tensorflow to 1.10 and got another error "TypeError: init() got an unexpected keyword argument 'aggregation'".
tensorflow1.9

INFO:tensorflow:Shuffling records in file processed_data/wmt32k-encoded-train-00095-of-00100
INFO:tensorflow:Shuffling records in file processed_data/wmt32k-encoded-train-00096-of-00100
INFO:tensorflow:Shuffling records in file processed_data/wmt32k-encoded-train-00097-of-00100
INFO:tensorflow:Shuffling records in file processed_data/wmt32k-encoded-train-00098-of-00100
INFO:tensorflow:Shuffling records in file processed_data/wmt32k-encoded-train-00099-of-00100
INFO:tensorflow:Shuffling records in file processed_data/wmt32k-encoded-train-00100-of-00100
STARTING TIMING RUN AT 2019-07-01 08:59:15 AM
Running benchmark with seed 1
:::MLPv0.5.0 transformer<PHONE_NUMBER>.920164585 (transformer/transformer_main.py:297) run_start
INFO:tensorflow:Using default config.
INFO:tensorflow:Using config: {'_evaluation_master': '', '_num_ps_replicas': 0, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f9d523097f0>, '_save_summary_steps': 100, '_keep_checkpoint_max': 5, '_master': '', '_device_fn': None, '_task_type': 'worker', '_tf_random_seed': None, '_log_step_count_steps': 100, '_save_checkpoints_secs': 600, '_train_distribute': None, '_service': None, '_task_id': 0, '_save_checkpoints_steps': None, '_model_dir': 'model', '_global_id_in_cluster': 0, '_session_config': None, '_keep_checkpoint_every_n_hours': 10000, '_is_chief': True, '_num_worker_replicas': 1}
Setting random seed =  1
Training schedule:
1. Train for 1 epochs.
2. Evaluate model.
3. Compute BLEU score.
Repeat above steps until the BLEU score reaches 25.0
:::MLPv0.5.0 transformer<PHONE_NUMBER>.930408955 (transformer/transformer_main.py:265) train_loop
INFO:tensorflow:Calling model_fn.
Starting iteration 1
Traceback (most recent call last):
File "transformer/transformer_main.py", line 440, in
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "transformer/transformer_main.py", line 350, in main
FLAGS.bleu_threshold)
File "transformer/transformer_main.py", line 274, in train_schedule
estimator.train(dataset.train_input_fn, steps=single_iteration_train_steps)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 366, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 1119, in _train_model
return self._train_model_default(input_fn, hooks, saving_listeners)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 1132, in _train_model_default
features, labels, model_fn_lib.ModeKeys.TRAIN, self.config)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 1107, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "transformer/transformer_main.py", line 78, in model_fn
train_op = get_train_op(loss, params)
File "transformer/transformer_main.py", line 111, in get_train_op
every_n=100)
File "/usr/local/lib/python3.5/dist-packages/mlperf_compliance/tf_mlperf_log.py", line 55, in log_deferred
aggregation=tf.VariableAggregation.MEAN)
AttributeError: module 'tensorflow' has no attribute 'VariableAggregation'

tensorflow1.10

INFO:tensorflow:Using default config.
INFO:tensorflow:Using config: {'_evaluation_master': '', '_keep_checkpoint_every_n_hours': 10000, '_task_type': 'worker', '_train_distribute': None, '_master': '', '_save_checkpoints_steps': None, '_device_fn': None, '_global_id_in_cluster': 0, '_service': None, '_session_config': None, '_log_step_count_steps': 100, '_save_summary_steps': 100, '_tf_random_seed': None, '_model_dir': 'model', '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7ff8276480f0>, '_is_chief': True, '_num_worker_replicas': 1, '_save_checkpoints_secs': 600, '_num_ps_replicas': 0, '_task_id': 0, '_keep_checkpoint_max': 5}
Setting random seed =  1
Training schedule:
1. Train for 1 epochs.
2. Evaluate model.
3. Compute BLEU score.
Repeat above steps until the BLEU score reaches 25.0
:::MLPv0.5.0 transformer<PHONE_NUMBER>.480519533 (transformer/transformer_main.py:265) train_loop
INFO:tensorflow:Calling model_fn.
Traceback (most recent call last):
File "transformer/transformer_main.py", line 440, in
Starting iteration 1
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "transformer/transformer_main.py", line 350, in main
FLAGS.bleu_threshold)
File "transformer/transformer_main.py", line 274, in train_schedule
estimator.train(dataset.train_input_fn, steps=single_iteration_train_steps)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 376, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 1145, in _train_model
return self._train_model_default(input_fn, hooks, saving_listeners)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 1170, in _train_model_default
features, labels, model_fn_lib.ModeKeys.TRAIN, self.config)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/estimator/estimator.py", line 1133, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "transformer/transformer_main.py", line 78, in model_fn
train_op = get_train_op(loss, params)
File "transformer/transformer_main.py", line 111, in get_train_op
every_n=100)
File "/usr/local/lib/python3.5/dist-packages/mlperf_compliance/tf_mlperf_log.py", line 55, in log_deferred
aggregation=tf.VariableAggregation.MEAN)
TypeError: init() got an unexpected keyword argument 'aggregation'

@JayjeetAtGithub Hey!
Did you solve this issue?

@JayjeetAtGithub @ehern96 Please refer to https://github.com/mlperf/training/issues/231#issuecomment-467769640


Download button does not work

On: https://engines.nanobox.io/ if you click on 'Download', you'll get a 404.

Thanks @kngu9. We have a list of fixes to make on engines.nanobox.io we'll be pushing up soon. We'll get this one in there too.

@kngu9 thanks for catching this, really appreciate it. Fixed.


Gets rid of all leftover usages of IEyeManager.CurrentMap

About the PR

Title
Why / Balance

Project 0 warning
Technical details

IEyeManager.CurrentMap was marked as obsolete in https://github.com/space-wizards/RobustToolbox/pull/5060, the solution seems to be replacing it with the CurrentEye.Position.MapId or using args.MapId when possible? Need @ElectroJr to review

Media

[x] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

https://github.com/user-attachments/assets/c941c698-be13-4e44-8370-0da86d50b387
Breaking changes

Changelog

eye see you

1st change is fine but the latter 3 are just bulldozing underlying problems and should be reverted.

That's what I kind of assumed. I don't mind attempting to fix the other 3, I just need to find where to get the intended MapId from if not the IEyeManager


Allow all git repos from host (for self-hosted corporate git)

Is your feature request related to a problem? Please describe.
I write Rust for work, and I'd like to allow any Git dependency from my work's self-hosted git server. Right now, the only way I see to do that is to manually add every Git repo I depend on to the allow-git list. If these repos were on Github, I could just trust my work's org, however, we use a private work git server.
Describe the solution you'd like
I can think of two solutions:

A new allow-git-hosts key, of type List, which would trust any Git repo whose path has one of the given hosts.
Allow wildcards * in the entries under allow-git, so that I can add a wildcard in the host or path, so that all repos from my org's git server are trusted.

Describe alternatives you've considered
It's really not too hard to manually add every private repo I want to use.

I think having a private or internal entry under allow-orgs would probably be the most clear since it would only accept a host + optional path that the private repos would be all rooted to.
I haven't used a purely internal git server in years so I completely forgot about this use case, thanks for posting the issue!

Yeah, that sounds great, that would meet my needs nicely!


Improvements to animation speed slider

Also, lowers the volume by default

Also, allows focus with options menu for keyboard/controller


insert image via clipboard

Category
feature request
What is the desired behavior?
Would it be possible to allow inserting images via the clipboard? At the moment my workflow is severly limited by the fact that i have to create an image file on disk manually before being able to choose it for insertion.
I often make screenshots which i'd like to insert with just CTRL+V. After pressing the shortcut i'd be asked for a filename and it would be saved next to the file i'm inserting to.

+1 on this. In my case, supporting paste from clipboard would make it much easier for those documents where a user has to be led step-by-step through a user interface. In my case, I have no preference regarding the specific filenames, other than they be stored in a stored in folder separate from the text content.
As a reference here is a site that shows how it is done: https://codepen.io/netsi1964/pen/IoJbg (works in Chrome and Microsoft Edge)

also i just noticed that this works perfectly with issues on github that accept any image from the clipboard via ctrl+V!

+1, when I want to insert a image, I must do these things:
save image from clipboard to desktop;
click upload button;
find image and double click it;
wait for upload;
and then I can edit my passage again.
we need a image paste function, seriously.

This is implemented in the major new version we've been working on (beta.gitbook.com) along with many other editor improvements (and general product improvements).
It's going public end of January.


[RHOAIENG-1101] Cannot remove additional storage from the workbench configuration

https://issues.redhat.com/browse/RHOAIENG-1101
Description
Workbench create/edit now has a table view to allow for multiple PVC instances to be created, where currently only 1 row is creatable/editable. The buttons above the table are not actionable yet and are placeholders for now. Some logic has been updated as a part of the submission process to attach multiple volumes (from PVC creations) for the notebook when available.
Default table data on creation

Detach modal

Empty state when all are detached

Edit modal on creation

NOTE: this edit modal is a starting point and meant to be expanded upon with a separate jira; https://issues.redhat.com/browse/RHOAIENG-7681

(cc @xianli123)
How Has This Been Tested?
When creating a workbench, verify the set of fields that were previously visible are now replaced with the editable table view. The Edit modal itself is a part of a different story, but one was made here as a starting point, where mainly the mount path and some show/hide of alerts will change (along with some implementation).
Test Impact
Updated mocked cypress tests to simply verify table data for now as a part of creating/editing a workbench. Since this feature has other parts, e2e mock testing will be dependent on their completion.
Request review criteria:
Self checklist (all need to be checked):

[x] The developer has manually tested the changes and verified that the changes work
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[x] The developer has added tests or explained why testing cannot be added (unit or cypress tests for related changes)

If you have UI changes:

[x] Included any necessary screenshots or gifs if it was a UI change.
[x] Included tags to the UX team if it was a UI/UX change.

After the PR is posted & before it merges:

[ ] The developer has tested their solution on a cluster by using the image produced by the PR to main

Thanks @jpuzz0 ! Overall LGTM!
As a tip (it may be outside the scope of https://issues.redhat.com/browse/RHOAIENG-1101), when creating a workbench or storage, the storage size should display "Max XXGi" instead of "XXGi."

Thanks @jpuzz0 ! Overall LGTM! As a tip (it may be outside the scope of https://issues.redhat.com/browse/RHOAIENG-1101), when creating a workbench or storage, the storage size should display "Max XXGi" instead of "XXGi."

@xianli123 I've made this change. Thanks for calling it out!

Thanks @jpuzz0 . Once it's changed, it will look good!

Hi @jpuzz0 , Thank you for updating the description above and adding screenshots.
I have some suggestions for the empty state you shared above.

As Katie recently suggested in Figma, the description of this empty state will be To save your project's data, attach cluster storage to your workbench. Your data will not persist without storage.. We can follow this latest content.

As for the empty state style, we can remove the table's header. PF has a guideline for when we can use a table empty state. I quoted it here: "Empty states are useful in a table when a filter returns no results, while data is loading, or when any type of error or exception condition occurs." In our case, the empty state should be a common empty state instead of a table empty state. WDYT?

@jpuzz0 Thanks for doing that!

Hi, I don't see creation or attach existing PVC button on my spawner page, is there anything wrong with my settings?

@DaoDaoNoCode
I commented out the buttons with a comment for now since they're not hooked up to anything yet and could lead to confusion.

Looks good, just found one issue, not sure if it should be fixed in this PR. When there are 2 storages that have the same mount path, the spawner page won't throw any error.

While the workbench couldn't start because the mount path needs to be unique.

@DaoDaoNoCode The edit modal is not finalized and there is no form validation yet, so this is not an expectation for this PR. There is a JIRA dedicated to just this edit modal. Again, as stated here and in the description of this PR, this is just a starting point.

Ok so this needs @dpanshug PR to make it able to merge. BC as of right now there is no way to attach an existing PVC and create a new one after one is detached

Ok so this needs @dpanshug PR to make it able to merge. BC as of right now there is no way to attach an existing PVC and create a new one after one is detached

@Gkrumbach07
We could temporarily comment out the detach-ability. I'm ok with leaving this open though if that's what you'd prefer.

it seems like the edit modal does not save the changes
edit modal opens to custom path every time and parses paths incorrectly sometimes
validation seems incorrect. it allows you to submit plain spaces. this is only for attach storage
you can submit empty for standard path but only after you switch between custom and standard once
cannot detach attached storage or existing storage
names are allowed to be the same
this can happen if you attach an existing storage with the same name
or create a new one with the same name
storage that has already been created does not disable storage class dropdown

on edit of a root mount path, it wont let me save because it thinks that the path is taken, but that path is used by the one i am editing

some bugs i found this iteration

created storage, either first initial one or newly by pressing create storage, will result in a blank path
you can add the same pvc multiple times
duplicate storage path is not being validated with all paths in the table


Make use of new useCheckedLegendItems hook

This was delayed to avoid further merge mayhem. Should be fairly simple to swap out the relevant code for the new hooks in all the visualizations with legends (not mosaic). The major viz work has now been merged into main so it's good to do this any time.

Duplicate of #1370


PATCH method support in typescript

In typescript, Using xhr with PATCH method raises the following compile error.

TS2322: Type '{ url: string; method: "PATCH"; ... }' is not assignable to type 'XhrUrlConfig'.

Adding 'PATCH' on XhrUrlConfig.method will fix this issue.

A PR would be less work than writing the issue ;)
I'm busy organizing a conference and might not have time to do it soon. But I will unless you beat me to it.


Knock Knock, Help! 🚪

Our client is a very awkward person; she needs some help breaking the ice, and would like a quick guide.

[] Create a new file in doc called jokes.md
[] Add whatever jokes you'd like to the md file.
[] Merge the file into master
[] Add a comment to your PR, explaining the original problem and your fix.

Please submit your PR with a checklist, showing that you have done these things.
BONUS: Squash your commits.

We'd be happy to help out with this issue if no one else has claimed it!

Thank you @KatSaldivar !


The f..ck is this?

https://onelinerhub.com/ubuntu/how-to-change-timezone-in-cli

Do we need to create pull request to solve this bug ?
Can you please explain the process to solve the bug

Well, this code is wrong and I was too lazy to describe everything in detail, sorry :)
This code shows timezone rather than allows to change it, so we need correct code here.
One way is to directly fix the file and create pull request. Other way - is to write the correct solution right in this issue comment and I'll deploy that to the right place.
Thanks!

Fixed: https://onelinerhub.com/ubuntu/how-to-change-timezone-in-cli


[d3d9] Add extended swapchain interfaces

Is not a good idea to add to "d3d11_swapchain.cpp" too?

@SirSAC D3D11 already supports all of this.

These fixed the super sharp edges of shadow of the torches on the night on Heaven Benchmark, without these changes the shadow looks anime instead of real life.

@SirSAC That's completely impossible. Not a single game uses this code and even if it did, it has nothing to do with what you're describing.

@SirSAC That's completely impossible. Not a single game uses this code and even if it did, it has nothing to do with what you're describing.

maybe it depends by hardware i have, it just did.

maybe it depends by hardware i have, it just did.

No. Code that is never executed cannot make any difference. Just placebo.

maybe it depends by hardware i have, it just did.

No. Code that is never executed cannot make any difference. Just placebo.

what is placebo? is some kind of imagination or illusion?
english is not my mother tongue

https://ro.wikipedia.org/wiki/Placebo

https://ro.wikipedia.org/wiki/Placebo

how did u know im romanian?

It's tagged on your GitHub profile. Anyway enough communing about that.

It's tagged on your GitHub profile. Anyway enough commenting about that so the PR comments doesn't get too far off track.

PR?

Short for Pull Request. When someone want's you to pull your changes into your git project (like this one here). On Gitlab the same thing is called a MR which is short for Merge Request.

Short for Pull Request. When someone want's you to pull their changes into your git project (like this one here). On Gitlab the same thing is called a MR which is short for Merge Request.

ok, thanks for explaining the terms


Compile.js regression

Then IMHO API Viewer needs some review - it was always the case that compile.json was loaded and passed to compile.js, so that compile.js is able to augment/process the compile.json.  I have made this change before when I fixed the compile.js, so I guess from your comment that you must have reverted this in a recent PR and I missed that's what had happened.
The issue here is that compile.js/json act as a pair (so .json is read and passed to .js), and then the result is a configuration structure which can be parsed and normalised, so that it can subsequently be used to configure the compiler API.
But the reason that this was a regression (and why I fixed it, this time and before) is that compile.js is being ignored completely, unless it also happens to be in a library.
In the current master, this will lead to unexpected side effects - for example, if you run qx compile in your app, then the API viewer's compile.js will be run with the application's compile.json -- but if you run the same command in the API viewer compile.js will be run with the API viewer's .json.  This means that the API viewer's compile.js will be run with completely different (and undefined) source data.
The other reason that the current master is broken is because the compile.js needs to be able to modify the structure before _mergeArguments is called, because _mergeArguments normalises the configuration as well as integrating command line arguments - for example, it makes sure that there is a target, etc.  However, libraries are not known until after _mergeArguments has normalised the configuration.
AFAICT the problem which API viewer's compile.js is solving is that it needs to augment the entry for it's application, by updating the application's environment entry; for the API viewer this needs to be done each time the target fires "checkEnvironment", and it just so happens that API viewer ignores the inputData so API viewer is fine with the way that master works.
IMHO in order to allow libraries to contribute to applications (eg API Viewer) and allow processing of compilation, the API needs to be improved so that there are phases:
(1) compile.json has been processed (if it exists) and normalisation of the structure has not yet begun;
(2) normalisation complete, each library's compile.js is asked to contribute
This will mean that if a library has it's own compile.js used for building it's own demo/test apps, then running qx compile in the library directory will cause both phases to be fired on that library's compile.js

but crucially, if a compilation takes place outside the library (an application that just uses the library), then the library will only see the second phase.

So is the problem that the apiviewer depends on code in compile.js which isn't run when this line is removed?
This means we need so kind of "hook" for packages that act as "plugins", don't we?

Something strage here - the PR could be merged even though we don't have two approvals...

Then IMHO API Viewer needs some review - it was always the case that compile.json was loaded and passed to compile.js, so that compile.js is able to augment/process the compile.json.  I have made this change before when I fixed the compile.js, so I guess from your comment that you must have reverted this in a recent PR and I missed that's what had happened.
The issue here is that compile.js/json act as a pair (so .json is read and passed to .js), and then the result is a configuration structure which can be parsed and normalised, so that it can subsequently be used to configure the compiler API.
But the reason that this was a regression (and why I fixed it, this time and before) is that compile.js is being ignored completely, unless it also happens to be in a library.

But "." path - the current project is a libray.

In the current master, this will lead to unexpected side effects - for example, if you run qx compile in your app, then the API viewer's compile.js will be run with the application's compile.json -- but if you run the same command in the API viewer compile.js will be run with the API viewer's .json.  This means that the API viewer's compile.js will be run with completely different (and undefined) source data.

That should be. e.g. apiviewer needs information from the current compile.json:     let excludeFromAPIViewer = env.excludeFromAPIViewer;     let includeToAPIViewer = env.includeToAPIViewer;

The other reason that the current master is broken is because the compile.js needs to be able to modify the structure before _mergeArguments is called, because _mergeArguments normalises the configuration as well as integrating command line arguments - for example, it makes sure that there is a target, etc.  However, libraries are not known until after _mergeArguments has normalised the configuration.
AFAICT the problem which API viewer's compile.js is solving is that it needs to augment the entry for it's application, by updating the application's environment entry; for the API viewer this needs to be done each time the target fires "checkEnvironment", and it just so happens that API viewer ignores the inputData so API viewer is fine with the way that master works.
IMHO in order to allow libraries to contribute to applications (eg API Viewer) and allow processing of compilation, the API needs to be improved so that there are phases:
(1) compile.json has been processed (if it exists) and normalisation of the structure has not yet begun;
(2) normalisation complete, each library's compile.js is asked to contribute
This will mean that if a library has it's own compile.js used for building it's own demo/test apps, then running qx compile in the library directory will cause both phases to be fired on that library's compile.js

Let's do it in 2 phases:
        if (await fs.existsAsync("compile.js")) {
         config = await this.__loadJs("compile.js", config);
        }

...
    this._mergeArguments(parsedArgs, config, lockfile_content);
...

      if (config.libraries) {
        for (const aPath of config.libraries) {
		  // do not process current compile.js again:
		  if (aPath !== '.') {
             await this.__loadJs(path.join(aPath, "compile.js"), config);
		  }
        }
      }

Something strage here - the PR could be merged even though we don't have two approvals...

We do not have any restrictions yet....

But "." path - the current project is a libray.

Only if the current directory happens to be a library (it doesn't have to be), and even if it does it's for a different purpose.

That should be. e.g. apiviewer needs information from the current compile.json: let excludeFromAPIViewer = env.excludeFromAPIViewer; let includeToAPIViewer = env.includeToAPIViewer;

That's the API viewer adding to the application - but the compile.json that is in compiler.inputData is different depending on where it is being called from.  So in the current design, the API viewer's compile.js cannot rely only anything being in inputData, even though that was a kind of a feature.

Let's do it in 2 phases:

I've been thinking over lunch - the principal has been that compile.json is parsed and fed into compile.js, and so it would follow that the library's compile.json is parsed and fed into it's compile.js, in order to merge into the application's configuration .... but that's getting quite complicated and doesn't feel right.
So - how about this: there are two distinct types of configuration, ApplicationConfiguration and LibraryConfiguration; these can be classes, one or both of which can be exported from the compile.js module.  This could be quite neat because starts to build an actual OO-style API around the pluggable nature of libraries; for example this would lay the groundwork for qx package install ... to allow the library to customise the installation (such as when adding an application entry into the compile.json)
IMHO it would imply that the compiler ignores compile.json if there is a compile.js because the ApplicationConfiguration/LibraryConfiguration implementations in compile.js would determines what, and when, to load external data.

So is the problem that the apiviewer depends on code in compile.js which isn't run when this line is removed?

Yes - and it's this line that breaks "normal" configation

This means we need so kind of "hook" for packages that act as "plugins", don't we?
👍

So - how about this: there are two distinct types of configuration, ApplicationConfiguration and LibraryConfiguration; these can be classes, one or both of which can be exported from the compile.js module. This could be quite neat because starts to build an actual OO-style API around the pluggable nature of libraries; for example this would lay the groundwork for qx package install ... to allow the library to customise the installation (such as when adding an application entry into the compile.json)
IMHO it would imply that the compiler ignores compile.json if there is a compile.js because the ApplicationConfiguration/LibraryConfiguration implementations in compile.js would determines what, and when, to load external data.

Sounds reasonable. To make this BC, the compiler would distinguish between what is imported from compile.js? I.E. function -> current behavior, object having "application" , "library" as members -> new behavior?

To make this BC, the compiler would distinguish between what is imported from compile.js? I.E. function -> current behavior, object having "application" , "library" as members -> new behavior?

not quite - because the current version is broken I'd propose that we do not support processing libraries in this way at all.
OTOH a couple of weeks ago, the only projects on GitHub that were using compile.js was our code, and we are pre-beta still I think?  We could consider just reporting an error if the new form is not implemented

We will break all projects which use apiviewer because apiviewers compile.hjs depends on inputdata from the current (!) project.

Including compiler which is using the apiviewer in this way.

@hkollmann but if we update compiler and apiviewer together, then they don't break - since we now can specify the dependency. We just need to make sure we develop the solution together.

you are right - but we have to find another solution than just do not support processing libraries as John suggested.

@hkollmann  I'd sugget you create a final v0.1 Version of the the API Viewer and tie it to a working compiler version, hen bump the version up to 1.0.0-beta and adapt to whatever we implement now.

sure, i'll do both compiler and API viewer

Merging, thanks @johnspackman !

eek, I thought that the merging would go on hold!  I'm working on it already and aiming to get the PR done this afternoon

Sorry, I thought you wanted to create a new PR. I have created a "do not merge" label for next time.

I like the idea of APIfying the compile step. I think it would make it much easier to work with if the  object imported by compile.js provides a bunch of specialized methods (such as getPkgCompileData() and getAppCompileData()) rather than having to work with the compiler object, which might overwhelm people. It could work like a sandbox object which filters access to the compilation process.

This API could then be extended to provide access to a (virtualiuzed) filesystem, which is important if we want to port the compiler to run in the client.


ES: use Elasticsearch credentials from env

Coverage decreased (-0.03%) to 79.116% when pulling dde1ca54a9668c2aa397116438bf50add650a10e on adericbourg:es_credentials_from_env into 07da226a29eeab0aaa15cc33ac4c5b5ed68625b9 on criteo:master.

Coverage decreased (-0.03%) to 79.116% when pulling dde1ca54a9668c2aa397116438bf50add650a10e on adericbourg:es_credentials_from_env into 07da226a29eeab0aaa15cc33ac4c5b5ed68625b9 on criteo:master.


Non-finite values of hypergeom.logsf and slow evaluation

For some small survival function values for the hypergeom distribution are evaluated as -inf. For example:
from scipy.stats import hypergeom

k = 1e4
M = 1e7
n = 1e6
N = 5e4

hypergeom.logsf(k, M, n, N)
-inf

hypergeom.sf(k, M, n, N)
0.0

R properly calculates the value (using the same inputs):
phyper(k, n, M-n, N, lower.tail=FALSE, log.p=TRUE)
-2239.771

phyper(k, n, M-n, N, lower.tail=FALSE, log.p=FALSE)
0

The scipy method is also slow compared with the R version. Scipy takes 8.8 ms while R takes 5.7 micro seconds - 100 times faster.
In Scipy, the logsf is calculated by first calculating the sf and then taking the log. The non-log output from Scipy is 0 so taking the log of this is just going to give -inf.
I am not sure how R calculates the value, but a more direct approach might be the following: the log survival function can be calculated via (see: https://en.wikipedia.org/Hypergeometric_distribution - note that the symbols have different meanings in the wiki page):

Which equals:
gammaln(1+M-n) + gammaln(1+n) + gammaln(1+M-N) + gammaln(1+N) - (gammaln(2+k) + gammaln(1+M) + gammaln(-k+n) + gammaln(2+k+M-n-N) + gammaln(-k+N)) +

All that would be required is to implement the hypergeometric series 3F2. The package mpmath have done this and there method is described here: http://fredrikj.net/blog/2014/06/easy-hypergeometric-series-at-unity/
I could implement it if you think this is the right direction
print scipy.__version__
0.15.1

Here's another thread related to numerical issues with the hypergeometric distribution: https://github.com/scipy/scipy/issues/2238.

I wonder how the approach for 3F2(..., 1) fares in double precision.

First thing to check for the numerical precision question: for each
summation for-loop in the code, compare the absolute value of the
largest term to the absolute value of the sum. If
abs(sum)<<abs(largest_term), you have a big numerical problem.

Stepping through the _sf function, it creates all pmf values and then sums them. So for the above example it creates an array representing the range of k-values under the tail from k=10001 to k=50000. It then calculates the pmf, however all these values evaluate to 0. So the sum is 0 and the largest term is 0

Yes. However, I meant the general case in the evaluation of 3F2 via the
method suggested above.

Which sum are you referring to: the sum required to evaluate the 3F2
function or the sum of all the gamma functions plus the 3F2?
On 22 June 2015 at 16:27, Pauli Virtanen<EMAIL_ADDRESS>wrote:

Yes. However, I meant the general case in the evaluation of 3F2 via the
method suggested above.
—
Reply to this email directly or view it on GitHub
https://github.com/scipy/scipy/issues/4982#issuecomment-114154135.

--
Josh Levy-Kramer
Data Scientist @ Starcount
[image: starcount-logo]
*UK Office *+44 (0)203 770 7554 |  *Mobile *0781 7970 736 | *Address:  *Henry
Wood House, 2 Riding House Street, London, W1W 7FA
*Singapore: Office +65 6595 6254 | Address: *The Nomad Offices, Suntec
City 9 Temasek Boulevard 09-01 Suntec Tower Two Singapore 038989
www.starcount.com http://www.starcount.com/
Confidentiality
The information contained in this e-mail is confidential, may be
privileged and is intended solely for the use of the named addressee.
Access to this e-mail by any other person is not authorised. If you are not
the intended recipient, you should not disclose, copy, distribute, take any
action or rely on it and you should please notify the sender by reply. Any
opinions expressed are not necessarily those of the company.
We may monitor all incoming and outgoing emails in line with current
legislation. We have taken steps to ensure that this email and attachments
are free from any virus, but it remains your responsibility to ensure that
viruses do not adversely affect you.

Ok I will have a look

All of the sums appearing in hyper1, gammaprod_series and
exp_series on Fredrik's page. Checking numerical stability on already
on a coarse level may require some work as this should be done
analytically (in order to cover the parameter ranges necessary).

Sorry do you mind elaborating on that as I don't think I totally understand
what you mean
On 22 June 2015 at 16:50, Pauli Virtanen<EMAIL_ADDRESS>wrote:

All of the sums appearing in hyper1, gammaprod_series and
exp_series on Fredrik's page. Checking numerical stability on already
on a coarse level may require some work as this should be done
analytically (in order to cover the parameter ranges necessary).
—
Reply to this email directly or view it on GitHub
https://github.com/scipy/scipy/issues/4982#issuecomment-114161265.

--
Josh Levy-Kramer
Data Scientist @ Starcount
[image: starcount-logo]
*UK Office *+44 (0)203 770 7554 |  *Mobile *0781 7970 736 | *Address:  *Henry
Wood House, 2 Riding House Street, London, W1W 7FA
*Singapore: Office +65 6595 6254 | Address: *The Nomad Offices, Suntec
City 9 Temasek Boulevard 09-01 Suntec Tower Two Singapore 038989
www.starcount.com http://www.starcount.com/
Confidentiality
The information contained in this e-mail is confidential, may be
privileged and is intended solely for the use of the named addressee.
Access to this e-mail by any other person is not authorised. If you are not
the intended recipient, you should not disclose, copy, distribute, take any
action or rely on it and you should please notify the sender by reply. Any
opinions expressed are not necessarily those of the company.
We may monitor all incoming and outgoing emails in line with current
legislation. We have taken steps to ensure that this email and attachments
are free from any virus, but it remains your responsibility to ensure that
viruses do not adversely affect you.

The error estimate for relative rounding error in numerically evaluated
sums is roughly
.
err = eps * abs(max_term) / abs(sum_value)
.
where eps is the machine epsilon (1e-16). For sums where the resulting
value is much smaller than the maximum term, i.e. sums with alternating
signs, all precision can be lost and the result become garbage even if
the expressions are in principle mathematically exact.
.
Now, the question is whether this can happen in the proposed way to
evaluate 3F2. mpmath can work around issues like this just by increasing
the calculation precision, and this is what its hypergeom implementation
does also in practice AFAIK (which is why I'm asking), but it is not
possible to do this in floating point implementation. So to be sure
whether the algorithm is OK on a basic level for fp implementation,
you'd need to check whether you have problematic summations in it. If
you have many parameters, it is not very easy to check for this only by
trying different parameter combinations, but looking at the structure of
the terms summed may give some information.

An easier way would be just to adapt the _sf code to compute _logpmf and then use logsumexp:
from scipy.stats import hypergeom
from scipy.misc import logsumexp
import numpy as np

k = 1e4
M = 1e7
n = 1e6
N = 5e4

# Adapted from:
# scipy.stats._discrete_distns.hypergeom_gen._sf
quant, tot, good, draw = k, M, n, N
k2 = np.arange(quant + 1, draw + 1)
logpmf = hypergeom._logpmf(k2, tot, good, draw)   # Evaluate the log-pmf instead
logsf = logsumexp(logpmf)
print logsf
-2239.77124993

In the example above it gives the same answer to R.

Should I make these changes and commit to git?

@joshlk: yes, that would be helpful. (Please see here and here in case you didn't do it before.)

The logsf precision loss has been fixed in gh-5010:
>>> from scipy.stats import hypergeom
>>> k, M, n, N = 1e4, 1e7, 1e6, 5e4
>>> hypergeom.logsf(k, M, n, N)
-2239.771249927684

Performance -wise: this python loop can surely be refactored into something more numpy-like.

Patch claiming to fix this issue was merged and sample code now produces correct result

If the patch was recent (for upcoming release), please do add the milestone to the closed issue(s).

@tylerjereddy I'll try to pay attention, in this case the PR was https://github.com/scipy/scipy/pull/10091 submitted ~10 months ago
I'm doing a triage over the oldest updated bugs so it's unlikely I'll add milestones to many of them.

yeah, thanks, I noticed; if it looks recent maybe just do a quick--it saves me time :)


Conda IOError when installing any package

(Perhaps this is an upstream issue but I'm not sure.)
I was trying to install IJulia after moving my ~/.julia/v0.4 directory out of the way and it errorred out when Conda was trying to install some packages.  In trying to narrow down the issue, if I run conda from the command line /u/hertz/.julia/v0.4/Conda/deps/usr/bin/conda install -y pip its output is:
Fetching package metadata: ....
Solving package specifications: .
Package plan for installation in environment /u/hertz/.julia/v0.4/Conda/deps/usr:

The following NEW packages will be INSTALLED:

    pip:        7.1.2-py27_0
    setuptools: 18.8.1-py27_0
    wheel:      0.26.0-py27_1

The following packages will be UPDATED:

    conda:      3.9.1-py27_0 --> 3.18.9-py27_0
    conda-env:  2.1.3-py27_0 --> 2.4.5-py27_0
    openssl:    1.0.1k-0     --> 1.0.2d-0
    python:     2.7.9-1      --> 2.7.11-0
    pyyaml:     3.11-py27_0  --> 3.11-py27_1
    requests:   2.5.3-py27_0 --> 2.9.0-py27_0
    sqlite:     <IP_ADDRESS>-0    --> <IP_ADDRESS>-1
    tk:         8.5.15-0     --> 8.5.18-0
    yaml:       0.1.4-0      --> 0.1.6-0

Extracting packages ...
An unexpected error has occurred, please consider sending the                                                                              |  12%
following traceback to the conda GitHub issue tracker at:

    https://github.com/conda/conda/issues

Include the output of the command 'conda info' in your report.

Traceback (most recent call last):
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/bin/conda", line 5, in <module>
    sys.exit(main())
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/lib/python2.7/site-packages/conda/cli/main.py", line 202, in main
    args_func(args, p)
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/lib/python2.7/site-packages/conda/cli/main.py", line 207, in args_func
    args.func(args, p)
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/lib/python2.7/site-packages/conda/cli/main_install.py", line 46, in execute
    install.install(args, parser, 'install')
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/lib/python2.7/site-packages/conda/cli/install.py", line 420, in install
    plan.execute_actions(actions, index, verbose=not args.quiet)
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/lib/python2.7/site-packages/conda/plan.py", line 502, in execute_actions
    inst.execute_instructions(plan, index, verbose)
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/lib/python2.7/site-packages/conda/instructions.py", line 140, in execute_instructions
    cmd(state, arg)
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/lib/python2.7/site-packages/conda/instructions.py", line 55, in EXTRACT_CMD
    install.extract(config.pkgs_dirs[0], arg)
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/lib/python2.7/site-packages/conda/install.py", line 448, in extract
    t.extractall(path=path)
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/lib/python2.7/tarfile.py", line 2070, in extractall
    self.extract(tarinfo, path)
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/lib/python2.7/tarfile.py", line 2107, in extract
    self._extract_member(tarinfo, os.path.join(path, tarinfo.name))
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/lib/python2.7/tarfile.py", line 2183, in _extract_member
    self.makefile(tarinfo, targetpath)
  File "/u/hertz/.julia/v0.4/Conda/deps/usr/lib/python2.7/tarfile.py", line 2224, in makefile
    copyfileobj(source, target)
IOError: [Errno 5] Input/output error

Also, /u/hertz/.julia/v0.4/Conda/deps/usr/bin/conda info returns:
Current conda install:

             platform : linux-64
        conda version : 3.9.1
  conda-build version : not installed
       python version : 2.7.9.final.0
     requests version : 2.5.3
     root environment : /u/hertz/.julia/v0.4/Conda/deps/usr  (writable)
  default environment : /u/hertz/.julia/v0.4/Conda/deps/usr
     envs directories : /u/hertz/.julia/v0.4/Conda/deps/usr/envs
        package cache : /u/hertz/.julia/v0.4/Conda/deps/usr/pkgs
         channel URLs : http://repo.continuum.io/pkgs/free/linux-64/
                        http://repo.continuum.io/pkgs/free/noarch/
                        http://repo.continuum.io/pkgs/pro/linux-64/
                        http://repo.continuum.io/pkgs/pro/noarch/
          config file : None
    is foreign system : False

versioninfo() reports:
Julia Version 0.4.2
Commit bb73f34 (2015-12-06 21:47 UTC)
Platform Info:
  System: Linux (x86_64-unknown-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz
  WORD_SIZE: 64
           "Red Hat Enterprise Linux Workstation release 6.4 (Santiago)"
  uname: Linux 2.6.32-358.el6.x86_64 #1 SMP Tue Jan 29 11:47:41 EST 2013 x86_64 x86_64
Memory:<PHONE_NUMBER>2021484 GB<PHONE_NUMBER>5625 MB free)
Uptime: 733885.0 sec
Load Avg:  0.35791015625  0.16845703125  0.11279296875
Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz:
          speed         user         nice          sys         idle          irq
#1-20  2992 MHz   58099163 s         20 s    3562866 s<PHONE_NUMBER> s        127 s

  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.3
Package Directory: /u/hertz/.julia/v0.4
2 required packages:
 - Compat                        0.7.8
 - IJulia                        1.1.8
7 additional packages:
 - BinDeps                       0.3.20
 - Conda                         0.1.8
 - JSON                          0.5.0
 - Nettle                        0.2.0
 - SHA                           0.1.2
 - URIParser                     0.1.1
 - ZMQ                           0.3.1

If this is an upstream issue or if there is anything I can try please let me know.

IOError: [Errno 5] Input/output error

This error number is for generic I/O error. So this could be something with your home directory/hard drive. Does this error persist with another user?
You can also try to rm -rf /u/hertz/.julia/v0.4/Conda and then add the Conda.jl package again.

You can also try to install the new conda package one at the time to check which one is failling. Specifically, runing /u/hertz/.julia/v0.4/Conda/deps/usr/bin/conda install --no-deps requests, /u/hertz/.julia/v0.4/Conda/deps/usr/bin/conda install --no-deps conda, ...

Thanks.  I was out of space on the disk and removing some files fixed the issue.


Shell add getopt function

@carlescufi: This implementation is based on FreeBSD. I've changed it a bit to support multi-instance usage.
However, because of that, it is not 1:1 usage as regular getopt. Can we live with that?

Error: * scan/lib/getopt/getopt.c is not apache-2.0 licensed: bsd-new

@nashif @MaureenHelm @galak in-tree usage of a pre-approved license like BSD is acceptable, especially when it comes to a single file, is this right?

Error: * scan/lib/getopt/getopt.c is not apache-2.0 licensed: bsd-new

@nashif @MaureenHelm @galak in-tree usage of a pre-approved license like BSD is acceptable, especially when it comes to a single file, is this right?

Error: * scan/lib/getopt/getopt.c is not apache-2.0 licensed: bsd-new

@nashif @MaureenHelm @galak in-tree usage of a pre-approved license like BSD is acceptable, especially when it comes to a single file, is this right? lib/fnmatch set that precedent.

After a brief discussion, I think we should follow the procedure here:
https://docs.zephyrproject.org/latest/contribute/index.html?highlight=license#submission-and-review-process
@jakub-uC could you please add a README as described in the doc above?
I can then initiate the process to get TSC and board approval.

Error: * scan/lib/getopt/getopt.c is not apache-2.0 licensed: bsd-new

@nashif @MaureenHelm @galak in-tree usage of a pre-approved license like BSD is acceptable, especially when it comes to a single file, is this right? lib/fnmatch set that precedent.

After a brief discussion, I think we should follow the procedure here:
https://docs.zephyrproject.org/latest/contribute/index.html?highlight=license#submission-and-review-process
@jakub-uC could you please add a README as described in the doc above?
I can then initiate the process to get TSC and board approval.

@carlescufi I've added the README file.

@carlescufi I've added the README file.

@nashif Should I create something like lib/util and put there fnmatch and getopt?

I can then initiate the process to get TSC and board approval.

The board didn't raise any objections, we can now move forward with this.

@nashif can you take another look please? lib/util is vague, but I cannot come up with anything better myself.


urbdrc: fix some usb devices redirection failed

If DeviceDescription get by control_query_device_text contains
extra L'\0', the returned bufferSize do not match the actual value.
This will cause the USB device redirection to fail.

@freerdp-bot test

@akallabeth now fix in libusb_udev_control_query_device_text.

@nanpuyue what exactly is the difference of ret to wcslen *  2 ? this looks like a quick hack without knowing what is wrong, so could you check if this is a off by one or similar?

@nanpuyue checked the values in the debugger, that is not really fixing anything (despite using unsafe string functions on possible non \0 terminated buffers.

It should be *BufferSize = _wcsnlen((WCHAR*)Buffer, ret / 2) * 2;.

@nanpuyue you still fail to explain what the difference here is. I've run this with a check for multiple devices and ret/2==wcslen for all requests so far.

The device is 096e:0209 Feitian Technologies, Inc. ROCKEY ARM:
https://www.ftsafe.com.cn/products/rockey/ROCKEY-ARM/ROCKEY-ARM

could you check the contents of the first 2 bytes of the buffer?
The returned data should consist of the data outlined here: https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/usbspec/ns-usbspec-_usb_string_descriptor

@akallabeth

bLength = 30

@nanpuyue thanks, I´ve added the check to the pr I´ve just opened (a rewrite of the usb redirection) here #5745


i can't close it

"nuxt": "^2.15.3",

Hi dear @ericzzhou. It is strange but might be also valid issue with IDE integration. If for any reason you cannot content with interactive prompts, please use one of the alternative opt-in/out methods mentioned here: https://github.com/nuxt/telemetry#opting-out


make resolve errors non-fatal

Fixes #107
This currently prints a summary of errors after commands that run WorkSpace.resolve.
Note that this currently slightly decreases functionality. A following PR (with this PR as its base branch) will:

implement error details that will be only printed after wit inspect
after error summaries (everywhere there is an error except wit inspect), instruct the user to run wit inspect --tree for more information

Note that this currently slightly decreases functionality. A following PR (with this PR as its base branch) will...

Change of plans: that sounded confusing, so I'm putting all the code in this PR.
Sample output of Conflicting sources error for wit inspect --tree:
--- ERROR ---
Packager rocket-chip::50de8a34 has multiple conflicting sources:
git@github.com:chipsalliance/rocket-chip.git
  foo::4bb8e80
  bar::1caf8a9
git@github.com:freechipsproject/rocket-chip.git
  fizz::8d2948f


FIBOModule and FIBODomain (subclasses of sm:Module)

@ElisaKendall - Currently, under http://www.omg.org/techprocess/ab/SpecificationMetadata/Module we have a flat list of FIBO domains and modules. I'd like to suggest we have two subclasses of http://www.omg.org/techprocess/ab/SpecificationMetadata/Module, namely: FIBOModule and FIBODomain. Are there any contraindications?

I think that makes sense @trypuz - the viewer should display the contents of the metadata for that domain/module, and at the FND domain level, that information is contained in the MetadataFND/FNDDomain element.


fix: Wait for task queue flush before switching a bot

Description

wait lu/lg worker and task queue flush before switching a bot.
add a loading modal when open pending.

Task Item
refs #3101

Screenshots

this haven't include "extend the task queue to cover settings read\write" right?