fix const value return by emplace iterator

with const this code will not compile
tsl::sparse_map<uint32_t,uint32_t> spp;
const auto it=spp.emplace(1,1);
it.first->second=1;

Casting a std::pair<K, V>&  to a std::pair<const K, V>& is an undefined behaviour. See https://stackoverflow.com/questions/8305793/is-it-possible-to-cast-a-pairkey-value-to-a-pairconst-key-value for more information.

It is not  undefined behaviour, because they have the same memory layout.
I use this modified version in my project. And it passed the clang && gcc
UBsan！
Thibaut Goetghebuer-Planchon @.***> 于2023年10月10日周二
03:25写道：

Casting a std::pair<K, V>& to a std::pair<const K, V>& is an undefined
behaviour. See
https://stackoverflow.com/questions/8305793/is-it-possible-to-cast-a-pairkey-value-to-a-pairconst-key-value
for more information.
—
Reply to this email directly, view it on GitHub
https://github.com/Tessil/sparse-map/pull/24#issuecomment-1753571067,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAK6QJWABV22VUVRPYKJM2LX6RFSHAVCNFSM6AAAAAAYKU2DOGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJTGU3TCMBWG4
.
You are receiving this because you authored the thread.Message ID:
@.***>

The new standard shows that it is layout compatible.
https://godbolt.org/z/dKn99YTeM


Introduce an extension registration phase during bootstrapping.

Extensions can be registered at any time - we should consider registering within a specific phase of bootstrapping to ensure there's a point when no further extensions will be registered.
For example, external media libraries can disable the global Media button, but if no library is registered, the button should show. Because there's no point when we can be sure that a media library won't still be registered, we have to show the button by default, so it's visible for a bit before the media library hides it.
Ideally all registrations would complete while the CMS loading screen is active.
This isn't really possible when Netlify CMS is used via CDN - we'll likely need to allow the netlify-cms convenience package to bypass any such phase and allow registration any time as we do currently.

Closing as stale and/or outdated.


 Open file lattice.txt
... Fixing
... Insert segment from 0.00, to 1.00
... Insert segment from 6.00, to 20.00
... Insert segment from 24.00, to 25.00
...
The lattice format is our internal format, and it is not documented, sorry about that.
The format of the word line is: is it the main flow, from, to, word. The lattice with overlapping time in the main flow is not valid.
Is it a bug with fix.lattice.time - NO
Could fix.lattice.time work more correctly and return failure in your case - YES

Things to add to enhancement; now also fix.segments are incompatible with overlapped speech (segments gets truncated and under certain conditions badly), i.e. currently it's incompatible with pyannote diarization (at least it should be documented, IMHO), which produces overlapping segments.


update: containerize translations into directories

With this change, translations are more flexible since there doesn't need to be only one file per language.
Signed-off-by: Vicente J. Jiménez Miras<EMAIL_ADDRESS>
What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind user-interface

/kind content

/kind translation
Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area blog
/area documentation

/area videos

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: vjjmiras
Once this PR has been reviewed and has the lgtm label, please assign jasondellaluce for approval by writing /assign @jasondellaluce in a comment. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 2d4f67cf05fa5c7a0d9c1e5679006caf1d510bbd

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leogr, vjjmiras
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


[$500] Request money - Merchant field is cleared after changing receipt in confirmation page

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: <IP_ADDRESS>
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
Action Performed:

Navigate to staging.new.expensify.com
Go to FAB > Request money > Manual.
Enter amount > choose a participant > proceed to confirmation page.
On the confirmation page, click three-dot menu > Add a receipt.
Click Merchant and enter a merchant.
Click three-dot menu > Add a receipt.

Expected Result:
The entered Merchant is preserved.
Actual Result:
The entered Merchant is cleared.
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence
https://github.com/Expensify/App/assets/93399543/385c0e53-ce1c-4f70-a568-3d8f9d97d80b
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01bcba32334ea8b85b
Upwork Job ID:<PHONE_NUMBER>706914304
Last Price Increase: 2023-11-28

Proposal
Please re-state the problem that we are trying to solve in this issue.
Request money - Merchant field is cleared after changing receipt in confirmation page
What is the root cause of that problem?
We are resetting the merchant here
https://github.com/Expensify/App/blob/70813dbf3e2f73a73e52f880312bc198b39f4dec/src/libs/actions/IOU.js#L2901-L2902
What changes do you think we should make in order to solve the problem?

We shouldn't reset merchant the merchant here
https://github.com/Expensify/App/blob/70813dbf3e2f73a73e52f880312bc198b39f4dec/src/libs/actions/IOU.js#L2901-L2902
What alternative solutions did you explore? (Optional)

Updated proposal

I don't have any context regarding the reason we reset the merchant value, but it seems important according to the PR https://github.com/Expensify/App/pull/25924.
Adding a new param for setMoneyRequestReceipt gives more flexibility to determine whether we need to reset the merchant value, but for now, I don't think it's necessary.
@Gonals @mountiny Tagging you guys here from https://github.com/Expensify/App/pull/25924. Could you guys help give more context on the reason we reset the merchant field for receipt requests?

Asking in Slack, its probably because the smartscaning feature expects no merchant provided

@mollfpr no reason I think then, lets make sure the merchant is not cleared when receipt is added

@mountiny Is there a problem with intelligent scanning with the merchant having a value?

lets make sure the merchant is not cleared when receipt is added

Can we safely revert the removed merchant here?
https://github.com/Expensify/App/blob/56a368b07233a78d0a3603b652798d1d86821885/src/libs/actions/IOU.js#L2939-L2941
I notice that from #25924 there's a comment "For the SmartScan to run successfully, we need to pass the merchant field empty to the API", but it's not there anymore.

@mollfpr @peterdbarkerUK to confirm, the merchant should not change when receipt is changed

to confirm, the merchant should not change when receipt is changed

@mountiny Does this mean the SmartScanning will work fine with merchant field fill?

We are working on making the merchant field required for group policies/ requests in workspace https://github.com/Expensify/App/pull/32486 so it would not be set to Request thats
cc @trjExpensify to help with the expected behaviour here.
If the merchant is passed we would not smartscan
If the user inputs some merchant, then we assume they dont want smartscan, if they upload different receipt at the confirmation page though, it should not reset automatically the merchant previously inserted. If the receipt is different and the merchant should be too, they can update it again.

Not overdue

@tienifr Do you need to update your proposal based on the https://github.com/Expensify/App/issues/32123#issuecomment-1849092288?

@mollfpr It is not reproducible anymore after the recent money request refactor.

Thanks @FitseTLT. I just tested on the latest main, and the issue is fixed.
https://github.com/Expensify/App/assets/25520267/a4369183-a712-4308-a251-20dce6ab8b85

Same, I can't reproduce this anymore either.

Issue not reproducible during KI retests. (First week)

@peterdbarkerUK We can close this issue.

Sorry for this but happy this got resolved!


Catch and report error when IBL map fails to load

This came up on the forum. If you take the old_venetian_crossroads_2k_ibl in this zip, and try to load it in the IBL Sandcastle it fails to load, and the only indication you get is that the models are just completely black.
EnvironmentMap.zip
I eventually realized CesiumJS was failing to load that KTX file because it had an unsupported glInternalFormat of R11F_G11F_B10F. This PR gets OctahedralProjectedCubeMap to reject its readyPromise when it fails to load the environment map, and gets Model.js to listen on that and console.error it out. I also added a test to make sure the promise is rejected when it tries to load an invalid URL.
This KTX file was generated from cmgen 1.4.1, but it seems to have this unsupported format because of some recent change. I tested with cmgen 1.2.0 and it works in CesiumJS (included in the above zip).

Thanks for the pull request @OmarShehata!

:heavy_check_mark: Signed CLA found.
:grey_question: CHANGES.md was not updated.

If this change updates the public API in any way, please add a bullet point to CHANGES.md.

Reviewers, don't forget to make sure that:

[ ] Cesium Viewer works.
[ ] Works in 2D/CV.
[ ] Works (or fails gracefully) in IE11.

Thanks again for your contribution @OmarShehata!
No one has commented on this pull request in 30 days. Maintainers, can you review, merge or close to keep things tidy?
I'm going to re-bump this in 30 days. If you'd like me to stop, just comment with @cesium-concierge stop. If you want me to start again, just delete the comment.

Congratulations on closing the issue! I found these Cesium forum links in the comments above:
https://groups.google.com/d/msg/cesium-dev/6C8O1faCEFc/CvcK83ZyDgAJ
If this issue affects any of these threads, please post a comment like the following:

The issue at https://github.com/AnalyticalGraphicsInc/cesium/pull/8303 has just been closed and may resolve your issue. Look for the change in the next stable release of Cesium or get it now in the master branch on GitHub https://github.com/AnalyticalGraphicsInc/cesium.

Thanks @emackey !


internal/runner: add option to store last stdout in $__

Adds field store_last_output to ExecuteRequest, which stores last stdout in the __ variable of its session.
Currently not worried about non-gRPC support (local runner scenario).
To test this out, make sure to run runme run/runme tui in server mode.

Tested with the server-attached and got the output. However, do we add an extra newline at the end?
❯ runme run hello-world
hello world
❯ runme run echo-
hello world

~/Projects/stateful/oss/runme mxsdev/reference-last *11 !2 ?1 ❯

Also, @mxsdev, will this feature ultimately be available in the TUI?

However, do we add an extra newline at the end?

echo prints a new line at the end, you'll have to use -n to suppress that.

Also, @mxsdev, will this feature ultimately be available in the TUI?

Should be working in the TUI currently, but you need session-strategy set to recent, so that the environment vars persist.

Also, @mxsdev, will this feature ultimately be available in the TUI?

Theoretically should be working in the TUI currently. (With gRPC server)

What the reason for the TUI not persisting ENV across runs by default? Please refresh my mind, I can't recall right now.

@sourishkrout

What the reason for the TUI not persisting ENV across runs by default? Please refresh my mind, I can't recall right now.

This does happen without the server, but as per the main post:

Currently not worried about non-gRPC support (local runner scenario).

In other words, I've only worried about gRPC implementation. If you want it in local runner let me know, its just more work.

@sourishkrout

What the reason for the TUI not persisting ENV across runs by default? Please refresh my mind, I can't recall right now.

Envs are persisted without the server, but as per the main post:

Currently not worried about non-gRPC support (local runner scenario).

In other words, I've only done the gRPC implementation. If you think tui + local runner scenario is important let me know, it can be done.

👍 gotcha. Makes sense. Let's merge what we have and add local-runner-support in a follow-up PR.


Error selenium [winerror 2]

Hi! I'm quite new to programming and i get the following error:
Could anyone please help to fix this :)?
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Users\nlkroe03\Python\lib\site-packages\selenium\webdriver\common\service.py", line 74, in start
stdout=self.log_file, stderr=self.log_file)
File "C:\Users\nlkroe03\Python\lib\subprocess.py", line 709, in init
restore_signals, start_new_session)
File "C:\Users\nlkroe03\Python\lib\subprocess.py", line 997, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\nlkroe03\Python\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "C:\Users\nlkroe03\Python\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "C:/Users/nlkroe03/Desktop/Afstuderen/python/Twitter old tweets/TwSearchEnabled-master/tweetsearch.py", line 39, in getTweet
browser = webdriver.Firefox()
File "C:\Users\nlkroe03\Python\lib\site-packages\selenium\webdriver\firefox\webdriver.py", line 144, in init
self.service.start()
File "C:\Users\nlkroe03\Python\lib\site-packages\selenium\webdriver\common\service.py", line 81, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.

@dk2010
Can you refer to this issue, sorry didn't respond to you.
https://stackoverflow.com/questions/40208051/selenium-using-python-geckodriver-executable-needs-to-be-in-path


[BUG] Cannot import torch when installation torch extension packages

setuptools version
setuptools==69.1.0
Python version
Python 3.12.2
OS
Windows
Additional environment information
This is only happening in "Python 3.12.2 + Windows + setuptools==69.1.0"
Description
python3 -m pip install --verbose --upgrade git+https://github.com/microsoft/tutel@main

My testing in different environment:
Python 3.12.0 + Windows + setuptools==69.1.0:  Installation Passed ✓
Python 3.12.2 + Windows + setuptools==69.1.0:  Installation Failed: × ModuleNotFoundError: No module named 'torch' (I ensure I have installed torch package for Python3.12.2)
Python 3.12.0 + Ubuntu + setuptools==69.1.0:  Installation Passed ✓
Python 3.8.6 + Ubuntu + setuptools==69.1.0:  Installation Passed ✓
Under Linux, I don't find any issues using any versions of Pytorch.
Expected behavior
Python 3.12.2 + Windows + setuptools==69.1.0 should work normally like other platforms and python versions.
How to Reproduce
Under Windows + Python3.12.2 + Torch::
python3 -m pip install --verbose --upgrade git+https://github.com/microsoft/tutel@main

Above command should detect and import package correctly.
Output
This is the error output using Python3.12.2:
  Running command Getting requirements to build wheel
  Traceback (most recent call last):
    File "C:\Users\develop\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
      main()
    File "C:\Users\develop\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\develop\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
             ^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\develop\AppData\Local\Temp\pip-build-env-ijd5_0va\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\develop\AppData\Local\Temp\pip-build-env-ijd5_0va\overlay\Lib\site-packages\setuptools\build_meta.py", line 295, in _get_build_requires
      self.run_setup()
    File "C:\Users\develop\AppData\Local\Temp\pip-build-env-ijd5_0va\overlay\Lib\site-packages\setuptools\build_meta.py", line 480, in run_setup
      super().run_setup(setup_script=setup_script)
    File "C:\Users\develop\AppData\Local\Temp\pip-build-env-ijd5_0va\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
      exec(code, locals())
    File "<string>", line 19, in <module>
  ModuleNotFoundError: No module named 'torch'
  error: subprocess-exited-with-error

Hi @ghostplant, by the looks of the logs it seems that the setup.py script is trying to import the torch package during the build, but it is not installed in the virtual environment[^1] where the build process is running.
There might be a plethora of reasons for the fact it works in some environment combinations and not in others.
The most likely is that https://github.com/microsoft/tutel does not have a pyproject.toml and does not list torch as part of the requires field in the [build-system] table.
Also note that because the absence of pyproject.toml, pip will use its internal heuristics to choose whether or not to rely on build isolation. In turn, these heuristics depend on the state of the virtual environment[^1], and therefore might seem inconsistent between different attempts.
Other combinations of OSs/Python versions might be working because torch is installed there, and/or because of transitive conditional dependencies on that package, and/or a different choice made by pip whether or not the build isolation feature is used.
See docs on Basic Usage for an example of pyproject.toml and where to list build-time dependencies.
Alternatively, if you want to disable "build isolaiton" you can run pip install with the --no-build-isolation and/or --no-pep517 flags (if you do that you need to ensure all the build dependencies are installed).
I do recommend having a pyproject.toml though, as it improves reproducibility and make things more deterministic.
[^1]: Or "global Python environment" related to the installation that is being used.

Thanks, I add that configuration in this branch: https://github.com/ghostplant/tutel/blob/main/pyproject.toml
However, I didn't get any luck by installing with: python3 -m pip install --verbose --upgrade git+https://github.com/ghostplant/tutel@main
A worse thing is that, that file makes all other platforms failed to import torch package as well.

Other options also don't work:
$ pip install --verbose --upgrade --no-build-isolation git+https://github.com/microsoft/tutel@main

  ...
  Resolved https://github.com/microsoft/tutel to commit b440e018c347205d3292a7c1e7947981e8fe46ad
  Running command git rev-parse HEAD
  b440e018c347205d3292a7c1e7947981e8fe46ad
  Running command Preparing metadata (pyproject.toml)
  running dist_info
  creating C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info
  writing C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\PKG-INFO
  writing dependency_links to C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\dependency_links.txt
  writing requirements to C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\requires.txt
  writing top-level names to C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\top_level.txt
  writing manifest file 'C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\SOURCES.txt'
  C:\Users\devel\AppData\Local\Programs\Python\Python312\Lib\site-packages\torch\utils\cpp_extension.py:500: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  reading manifest file 'C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\SOURCES.txt'
  adding license file 'LICENSE'
  writing manifest file 'C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\SOURCES.txt'
  creating 'C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel-0.3.dist-info'
  Try installing without NCCL extension..
  running dist_info
  writing C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\PKG-INFO
  writing dependency_links to C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\dependency_links.txt
  writing requirements to C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\requires.txt
  writing top-level names to C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\top_level.txt
  reading manifest file 'C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\SOURCES.txt'
  adding license file 'LICENSE'
  writing manifest file 'C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\SOURCES.txt'
  creating 'C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel-0.3.dist-info'
  Try installing without CUDA extension..
  running dist_info
  writing C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\PKG-INFO
  writing dependency_links to C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\dependency_links.txt
  writing requirements to C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\requires.txt
  writing top-level names to C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\top_level.txt
  reading manifest file 'C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\SOURCES.txt'
  adding license file 'LICENSE'
  writing manifest file 'C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel.egg-info\SOURCES.txt'
  creating 'C:\Users\devel\AppData\Local\Temp\pip-modern-metadata-fb2ve584\tutel-0.3.dist-info'
  error: invalid command 'bdist_wheel'
  error: subprocess-exited-with-error

$ pip install --verbose --upgrade --no-use-pep517 git+https://github.com/microsoft/tutel@main

  ...

Usage:
  pip install [options] <requirement specifier> [package-index-options] ...
  pip install [options] -r <requirements file> [package-index-options] ...
  pip install [options] [-e] <vcs project url> ...
  pip install [options] [-e] <local project path> ...
  pip install [options] <archive url/path> ...

--no-use-pep517 error: It is not possible to use --no-use-pep517
without setuptools and wheel installed.

I have to unzip the online files to local, and manually run setup.py:
$ git clone https://github.com/microsoft/tutel
$ cd tutel && ./setup.py
  ...
  creating c:\Users\devel\appdata\local\programs\python\python312\lib\site-packages\tutel-0.3-py3.12-win-amd64.egg
  Extracting tutel-0.3-py3.12-win-amd64.egg to c:\Users\devel\appdata\local\programs\python\python312\lib\site-packages
  Adding tutel 0.3 to easy-install.pth file
  detected new path './tutel-0.3-py3.12-win-amd64.egg'

  Installed c:\Users\devel\appdata\local\programs\python\python312\lib\site-packages\tutel-0.3-py3.12-win-amd64.egg
  Processing dependencies for tutel==0.3
  Searching for numpy==1.26.4
  Best match: numpy 1.26.4
  Adding numpy 1.26.4 to easy-install.pth file
  Installing f2py-script.py script to C:\Users\devel\AppData\Local\Programs\Python\Python312\Scripts
  Installing f2py.exe script to C:\Users\devel\AppData\Local\Programs\Python\Python312\Scripts

  Using c:\Users\devel\appdata\local\programs\python\python312\lib\site-packages
  Finished processing dependencies for tutel==0.3

Looks like online/zipped-file installation is problematic in Win11 + Py3.12.2, since running setup.py is always working, even without pyproject.toml configured.
In other platforms like Linux, I don't see any differences between online/zipped-file installation and running ./setup.py manually.

Is it possible to fix --no-build-isolation error above and keep it as the same behavior as running local ./setup.py?
I think configure "torch" dependencies in pyproject.toml isn't a good choice in my case, since torch is a huge package with diverse dependencies from special extra index URL. It is not only expensive to download and install in a fresh isolated sandbox, on the other hand, it is also not likely possible for Tutel (or any other Torch applications) to detect which version, backend-type and index-URL that local torch was ever installed with, so it may result in a wrong torch installed and thus introducing other CXX_ABI errors or backend-mismatch errors.

Thanks, I tried adding that configuration in this branch: https://github.com/ghostplant/tutel/blob/main/pyproject.toml
However, I didn't get any luck by installing with:
python3 -m pip install --verbose --upgrade git+https://github.com/ghostplant/tutel@main

A worse thing is that, that file makes all other platforms failed to import torch package as well.

I think you ended up adding the [project] table (which is not necessary) instead of the [build-system] which allows you to list all the build dependencies. Please have  a look on https://setuptools.pypa.io/en/latest/userguide/quickstart.html#basic-use and don't forget to add all the build-time dependencies to the requires field. I actually recommend completely removing the [project] table for now, because that is a completely different topic.

Other options also don't work:
$ pip install --verbose --upgrade --no-build-isolation git+https://github.com/microsoft/tutel@main

Please note that if you do that you need to install all build dependencies. I have prévios mentioned that

Alternatively, if you want to disable "build isolation" you can run pip install with the --no-build-isolation and/or --no-use-pep517 flags (if you do that you need to ensure all the build dependencies are installed, including setuptools and wheel).

By the looks of the logs, it seems like wheel was not installed.

Is it possible to fix --no-build-isolation error above and keep it as the same behavior as running local ./setup.py? Supporting isolation_safe=False instead of an explicit --no-build-isolation option would be even greater.
I think configure "torch" dependencies in pyproject.toml isn't a good choice in my case, since torch is a huge package with diverse dependencies (e.g. nvidia-ptx/..) from special extra index URL. It is not only expensive to download and install a new torch package in a fresh isolated sandbox, it is also not likely possible for Tutel (or any other Torch applications) to smartly detect which index-URL that local torch was ever installed with, so it may result in installing a wrong torch target in the sandbox and further introducing other CXX_ABI errors or backend-mismatch errors.

I am afraid that is not up to setuptools. The installation process is handled by pip and they have their own way of deciding when "buil isolation" will be triggered.As far as I know, the way to achieve consistency is by installing all the build dependencies manually and then explicitly passing the flag.

Other options also don't work:
$ pip install --verbose --upgrade --no-build-isolation git+https://github.com/microsoft/tutel@main

Please note that if you do that you need to install all build dependencies. I have previuosly mentioned:

Alternatively, if you want to disable "build isolation" you can run pip install with the --no-build-isolation and/or --no-use-pep517 flags (if you do that you need to ensure all the build dependencies are installed, including setuptools and wheel).

By the looks of the logs, it seems like wheel was not installed.

That works! It indeed solves the online installation problem after pip install wheel. However, I think the error message error: invalid command 'bdist_wheel' is very confusing. If reporting Exception ModuleNotFound: wheel instead, it would be much clearer for user ensure wheel package before reinstalling user package again.

BTW, after installing wheel package in advance, I don't even need to set --no-build-isolation to install user packages, which is great. One more question: in order to avoid other environment/users to get into the same trouble, if it possible to set pre-dependency keys in setup.py to ensure pip to install wheel package first and then continuing user package installation. pyproject.toml is a great thing to set pre-dependency in an isolated environment, but how about pre-dependency for non-isolated environment?

If reporting Exception ModuleNotFound: wheel instead, it would be much clearer for user ensure wheel package before reinstalling user package again.

Setuptools does not know exactly which package is providing the bdist_wheel command either to be honest. It uses entry-points and discovery to decouple the implementation of the commands. wheel provides a reference implementation for bdist_wheel, but many projects do customise that.

BTW, after installing wheel package in advance, I don't even need to set --no-build-isolation to install user packages, which is great.

Please be mindful however that this behaviour depends on the internal heuristics used by pip to decide whether to trigger build isolation[^1]. I am not sure how stable are these heuristics and how often they are planned to change. I don't think pip make any promises on that front either. To be fully in control of what is happening and improve reproducibility, I would recommend being explicit with the flags.
[^1]: We can see that one of the parameters used by pip is the presence of wheel installed in the (virtual) environment.

in order to avoid other environment/users to get into the same trouble, if it possible to set pre-dependency keys in setup.py to ensure pip to install wheel package first and then continuing user package installation.

As far as I know there is no API implemented in pip that allows setuptools to tell which packages to pre-install in the case of "no-isolation"[^2]. By the time that setup.py runs it is too late: the imports are already parsed and trigger import errors. If "no-isolation" is needed, the only means I know to minimise errors is to document all the steps in the "Installation" section of your package description.
[^2]: The pyproject.toml and related APIs are only defined for the "isolation" use case. When using "isolation", setuptools uses the build APIs to tell pip to pre-install wheel, but nothing similar exist for "no-isolation"

OK, the online setup has now been changed to multiple stages like this, which runs much more stable on all platforms I have tested:
Line-1:      $ python3 -m pip install setuptools wheel
Line-2:      $ python3 -m pip install -v -U --no-build-isolation git+https://github.com/microsoft/tutel@main

Since it is late for setuptools to analyze build-time dependencies, while most users may still easily forget running Line-1 above, I plan to add some check in setupy.py to check whether it is already in an isolated environment, if so, I need to stop further installation and print useful errors that is not ambiguous like torch is not found.
import setuptools
..

if <I-am-in-an-isolated-environment>:
  raise Exception('Please install wheel and setuptools..')

# Run original scripts
setup(
 ..
)

Is it convient to do <I-am-in-an-isolated-environment> decently?

Hi @ghostplant, it might be useful to also suggest pre-installing torch in Line-1, because it is also a build dependency, isn't it? (It is imported in setup.py, so if the user does not have it installed, Line-2 will fail with an exception).
Regarding the check you mentioned:

I don't know a way to check specifically for <I-am-in-an-isolated-environment>...
You might try to check for the presence of torch or wheel. That might be easy to achieve.
If the (virtual) environment does not contain setuptools installed, the exception will happen when import setuptools run.

It might be worth to keep torch in [build-system] requires anyway, because I guess some users will still oversee the installation instructions and simply attempt to run the install command without the --no-build-isolation. While that is sub-optimal and will take longer, at least they get something...
 [build-system]
+requires = ["setuptools", "wheel", "torch"]
-requires = ["setuptools", "wheel"]
 build-backend = "setuptools.build_meta"

, because I guess some users will still oversee the installation instructions and simply attempt to run the install command without

Thanks for the suggestion. I have tried adding pyproject.toml that but seems like it damages most platform's default installation, because that file makes pip always thinking of creating an isolated environment, and thus triggers to download a new torch from PyPi repo instead of Torch's official repo, which eventually produces pyDLL with an improper ABI and backend setting which is not able to be imported successfully by the Pytorch installed at host site.

Ah, I see: there is also a pip x conda background to this issue.
Yeah that makes sense. I suppose that in this use case it is better to focus on the way conda distributes packages and/or custom detailed installation instructions instead of trying to follow the recommendations we would normally give for a pip-centric process.
I will go ahead and close this issue for now: it seems to me that all the details we have discussed so far are related to how the build environment is set, which happens before setuptools itself is even imported in the first place.
The setting up of the build environment in a modern pip-centric installation process is the responsibility of what we call build-frontend (e.g. pip or build), but setuptools is a build-backend. In the last half-decade PyPA's tools have improved a lot for isolated builds, and in general it is a good idea to use isolation. I understand that there are use cases (e.g. very large dependencies and/or integration with conda) that are probably better if treated differently and the tool chain has not matured enough for that. But this is probably something that has to be discussed in a broader context via https://discuss.python.org/c/packaging/14, since this is not part of the scope of setuptools itself.

Thanks. Windows is more common to see this issue since a fresh Python installation usually excludes setuptools and wheel which make pip force to decide isolation.


update node-libs-browser to ^1.0.0

I've hit some issues using webpack with node-libs-browser previous to 1.0.0 due to it using a older version of the Buffer module (3.X), which is incompatible with the new 4.X.
This PR solves that issue by upgrading the node-libs-browser inside webpack

Fixed in 186d0da5ce1bddc73df8d1337985b6ff92ca5cfb

Thank you :)


LXD service fails to start with "failed to handle BEGIN request: FSM out of sync: timed out enqueuing operation"

Required information

Distribution: Ubuntu
Distribution version: 18.04 beta amd64
Kernel version: at least 4.15 through 4.17
LXC version: 3.0.0
LXD version: 3.0.0
Storage backend in use: btrfs 4.15.1 (separate disk from root FS)

Issue description
The LXD service fails to start (on boot, after a package update, or when started manually for some reason) more often than not, due to a database timeout:
failed to handle BEGIN request: FSM out of sync: timed out enqueuing operation
When observing the daemon behavior, there is high random write activity (averaging ~10MB/s to the SSD) for tens of seconds before the daemon fails with the error above.
The lxd waitready process continues to wait until the 600 second limit expires, then systemd tries to start the lxd daemon, usually succeeding after 2-4 attempts.
This started to happen after upgrading from Ubuntu 16.04 to 18.04 beta.
Information to attach

[X] Any relevant kernel output (dmesg) journal.txt
[ ] Container log (lxc info NAME --show-log)
[ ] Container configuration (lxc config show NAME --expanded)
[ ] Main daemon log (at /var/log/lxd/lxd.log or /var/snap/lxd/common/lxd/logs/lxd.log)
[ ] Output of the client with --debug
[ ] Output of the daemon with --debug (alternatively output of lxc monitor while reproducing the issue)
(Note that some container names have been replaced with "SOME_CONTAINERNAME" in the log above for privacy reasons. I also killed the waitready process after the daemon had failed to speed up the restart.)

What does du -sch /var/snap/lxd/common/lxd/raft show?

I'd normally expect the DB to be tiny so the 10MB/s for a while seems very weird.

That's going to be one for @freeekanayaka as I'm not aware of what those contain exactly and what's safe to cleanup... Unfortunately he's off until Friday so unless he pops in, it may take a little while...

Hi,
the error that you've seen (failed to handle BEGIN request: FSM out of sync: timed out enqueuing operation) happens if a database transaction can't be started within a minute, which is the (for now) hard-coded timeout for waiting for the database raft-based engine to replay the raft logs in the store (which is logs.db file plus the most recent snapshot in the snapshots/ dir).
I've tried to locally replay your logs using the raft.tar.gz file that you attached and it takes less then 300ms on my laptop, so clearly there is something else going on.
If you could, I'd suggest to:

Migrate from the LXD deb package to the LXD snap. That will get you a more recent build that would rule out some fixed issues that are not yet included in the 3.0.0-0ubuntu4 deb from Ubuntu 18.04. The snap will also keep LXD binary you more up-to-date than the deb, since it's generally released more frequently and faster (without loss of stability).

Stop the LXD systemd service (systemd stop lxd; systemd stop lxd.socket), make sure that the LXD daemon process is really gone (ps aux | grep lxd) and then start the LXD daemon from the console in debug mode with sudo lxd --group lxd --verbose --debug". Try to do that a few times to see if it gets stuck (once started you can check that is functional by just running lxc listfrom another console. If hit any problems, please attach the output of the daemon to this issue. If you don't hit anything and it works all the times, it most probably means that there is a bug somewhere in the interaction betweenlxd waitready`` and the daemon. If you are not depending on the lxd daemon to be fully ready when the systemd unit is started (e.g. you don't have systemd-triggered scripts that run lxc commands after the lxd unit), the best workaround for now would be probably to create a custom systemd service file for LXD that does not run waitready: just copy /lib/systemd/system/lxd.service to /etc/systemd/system/lxd.service and remove the waitready line. On a normal system the LXD daemon should be ready after a few milliseconds after the systemd unit is started, and so blocking on waitready might not be an issue for you.

I'll need to figure out what using that snap implies for the rest of the system - additional btrfs subvolumes might confuse the backup solution of that system, so I'm not sure if I can use snaps without screwing something up.
Maybe I can reproduce this on a box where that doesn't matter, I'll have a look.
This is not just an interaction between LXD and waitready, waitready is just what's delaying the auto-restart 10 minutes after LXD failed.
LXD keeps hammering that database file for a minute and then dies, leaving the waitready behind.
Why is that logs.db file so huge in the first place? What is it even storing?

There still could be some bad behavior in the daemon that is triggered by waitready and wouldn't be triggered otherwise, so trying without it (first by hand from the console and then perhaps tweaking the systemd unit) would still be worth.
Re the size of logs.db, it shouldn't really be a concern, it's a boltdb store which allocates buckets beforehand (and doubles allocation when short of buckets), it's used to store raft logs which in any case will be compacted at regular intervals (although boltdb won't shrink, it will just mark those buckets as unused). What matters mostly is the size of bin.db, which is just a SQLite database generated from the raft logs (and this is why we suffixed it with ".bin", to signal that you should not touch it).

Ok, so for anyone who's stuck, those are temporary instructions to get you back to a running state.
Note that this will not persist on reboot and that this will no longer be needed after you run a fix LXD.
set -eu

# Find the database
for db in /var/snap/lxd/common/lxd/raft /var/snap/lxd/common/lxd/database/global /var/lib/lxd/raft /var/lib/lxd/database/global; do
    [ ! -e "${db}" ] && continue
    DB="${db}"
    break
done

[ -z "${DB}" ] && echo "Couldn't find database" && exit 1
echo "Database found at ${DB}"

# Safety check
if mountpoint -q "${DB}" || mountpoint -q "${DB}.mnt"; then
    echo "temporary mounts already setup, aborting"
    exit 1
fi

# Stop LXD
systemctl is-enabled -q lxd 2>/dev/null && systemctl stop lxd lxd.socket
systemctl is-enabled -q snap.lxd.daemon 2>/dev/null && systemctl stop snap.lxd.daemon

# Backup
rm -rf "${DB}.orig"
cp -r "${DB}" "${DB}.orig"

# Setup temporary mount
[ ! -e "${DB}.mnt" ] && mkdir "${DB}.mnt"
mount --bind "${DB}" "${DB}.mnt"
mount --make-private "${DB}.mnt"

# Cleanup unused files
rm -f "${DB}"/db.bin*

# Setup tmpfs
mount -t tmpfs tmpfs "${DB}" -o mode=0750
[ -e "${DB}.mnt/snapshots" ] && mkdir "${DB}/snapshots" && mount --bind "${DB}.mnt/snapshots" "${DB}/snapshots"
touch "${DB}/logs.db" && mount --bind "${DB}.mnt/logs.db" "${DB}/logs.db"

# Start LXD
systemctl is-enabled -q lxd 2>/dev/null && systemctl start lxd lxd.socket
systemctl is-enabled -q snap.lxd.daemon 2>/dev/null && systemctl start snap.lxd.daemon

echo "Done"

You probably want to put that in a file and then run it with "sh". This needs to be run as root.

@TheSeven when I posted earlier that it only took 300ms on my laptop to replay the raft logs you attached, I was misled by the fact that I was performing the test under /tmp, which of course is tmpfs here. I tried again this morning and it actually to ~25s on non-tmpfs. The workaround in #4518 should solve the issue for now, although we'll also want to fix the db engine in a better way.

The timestamps of those snapshots are before my "run lxd manually" tests, and during the first manual start of lxd I have also observed a phase with heavy reading (1MB/s from HDDs) for tens of seconds. That just goes away once it's cached.

I see, then there might be a read problem too. We'll have to do more testing with non-SSD storage, since I didn't see any problem with the read part here, even on non-tmpfs. The PR I linked above should solve the heavy read part on startup too, tho.

Going to close this as the main fix is in place now in master and the rest of the fix will come through dqlite rather than LXD.


white page

Describe the bug
server not work it's just a white page i don't know why !
Link to the Bolt URL that caused the error
https://bolt.new/~/sb1-txu6fr
Steps to reproduce
123
Expected behavior
123
Screen Recording / Screenshot
No response
Platform
Browser name = Chrome
Full version = <IP_ADDRESS>
Major version = 130
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36 Edg/<IP_ADDRESS>
performance.memory = {
  "totalJSHeapSize": 131617487,
  "usedJSHeapSize": 123698779,
  "jsHeapSizeLimit":<PHONE_NUMBER>
}
Username = jps1990
Chat ID = 1e0f083de6b1

Additional context
No response

Same issue

Hi all,
Appreciate the feedback here. We are tracking this issue: #1791. You can follow there for updates. If you continue to have this problem, posting the requested details on that issue page would be helpful. Thanks!


Axial attention

What is the meaning of qkv_channels?
https://github.com/The-AI-Summer/self-attention-cv/blob/5246e550ecb674f60df76a6c1011fde30ded7f44/self_attention_cv/axial_attention_deeplab/axial_attention.py#L32

Hi @Jayden9912
This is because, as far as I can recall from my implementation, the dimensionality of V (Value representation) is different than the dimensionality of K and Q (keys and values).
The projection is being done with a 1d , 1 by 1 conv instead of a nn.Linear layer.
Does that answer your question?


Remove deadlink in thread_local_store file's comment

Signed-off-by: Nguyen Hung Phuong<EMAIL_ADDRESS>The comment in thread_local_store file is not correct so I remove it.

Also per comments on the other PRs, please pick one of these and merge all the link changes into it!


why do we need to declare the maximum number of elements the index can store beforehand?

from the docs:
# Initializing index - the maximum number of elements should be known beforehand
p.init_index(max_elements = num_elements, ef_construction = 200, M = 16)

why do we need to declare the maximum number of elements the index can store beforehand? i don't see any constraint like this in the algorithm described in the hnswlib paper. thanks.

This is due to flat table design of the data/link storage. There is a resize_index method in python which can be called manually.
We can potentially call it automatically once there is not enough space, but it is a O(N) operation which blocks additions for a while an without explicit user call (that also requires an additional global lock, which is a much simpler problem). So we assume that the user takes care of resizes (also doable with a simple python wrapper)

thanks for the response. i have noticed this parameter is not required in FAISS's implementation of HNSW. see this for example. could you explain me why?

I think faiss is fine with O(N) time complexity during insertion. In the code they use std::vector to store the array with pushes can lead to O(N) complexity on rare random occurrences, which I guess is not an issue if you don't do many small inserts (say, a single huge batch insert, which already has even higher complexity).
hnswlib, on the other hand, is optimized for a ability to make small searches in small insertions with low latency.

Thanks Yuri. I looked at the code and tried to understand it.
data_level0_memory_ = (char *) malloc(max_elements_ * size_data_per_element_);

is this also the reason why we cannot delete vectors from the index? If we were using linked lists instead of static array then would these two limitations go away?

have to declare in advance the number of vectors we will be storing in the index
cannot delete vectors from the index. by delete I mean a real delete leading to freeing up of some memory.

I see that FAISS implementation does not require 1. but it does have 2nd limitation. any idea why?
did you do any perf tests comparing the perf impact of using LL instead of the static memory block? curious to see if you have any results of the same.

The problem with simple removal of elements is that the graph structure deteriorate over time. I discuss it a bit in https://www.youtube.com/watch?v=4PsyNdFlxmk&t=2247s
hnswlib does support deletions, though the index RAM consumption does not shrink in size(but does not grow as well)

thanks Yury. Another question I have is over here:
template <bool has_deletions, bool collect_metrics = false>
    std::priority_queue<std::pair<dist_t, tableint>, std::vector<std::pair<dist_t, tableint>>, CompareByFirst>
    searchBaseLayerST(tableint ep_id, const void *data_point, size_t ef, BaseFilterFunctor* isIdAllowed = nullptr) const {

could you explain me why are you using a template for bool? wouldn't it be better to write this function as:
std::priority_queue<std::pair<dist_t, tableint>, std::vector<std::pair<dist_t, tableint>>, CompareByFirst>
    searchBaseLayerST(tableint ep_id, const void *data_point, size_t ef,
bool has_deletions, bool collect_metrics, BaseFilterFunctor* isIdAllowed = nullptr) const {

I don't know much about C++ so asking.

Templates are faster because they allow not doing checks in critical loops by removing them at compile time (collect_metrics=true and collect_metrics=false lead to different binary code).


[terra-application-layout] update to consume the terra-icon upgrades

Summary
Modified the code to use the new a11yLabel where ariaLabel was used in _ApplicationHeader.jsx
This change was tested using:

[ ] WDIO
[X] Jest
[X] Visual testing (please attach a screenshot or recording)
[ ] Other (please describe below)
[ ] No tests are needed

Reviews
In addition to engineering reviews, this PR needs:

[X] UX review
[X] Accessibility review

[ ] Functional review

This PR resolves:
UXPLATFORM-8835

Thank you for contributing to Terra.
@cerner/terra

Since you're using a feature from a newer version of icon, can you update the version of icon in terra-application-layout/package.json?

PR was closed and the brench was deleted as it turns out that no changes are needed for this Jira. IconMenu found in _ApplicationHeader is contained within a button that already has an aria-label attribute. In this scenario no a11yLabel should be added to the icon (as it may enter in conflict with the aria-label attribute).


Add clarity to 3P spacecraft onboarding requests

The last two bullets of the prerequisites section should be indented under the second bullet.
Do we have a link to KSAT, similar to ViaSat, or some external reference?
What information must we have to proceed forward with a 3P integration request? The goal is to sway everyday citizen users away from opening incorrect support tickets but allow real customers with actual contracts the ability to request and step through the 3P authorization process. Should we allude to some of those steps -- technical exchange, key/certificate exchange, etc?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: e42bf9eb-6600-f552-0a66-b6152a7a66f0
Version Independent ID: 03b9ced5-63d1-5dc5-2f88-4b0d432e5a54
Content: Integrate partner network ground stations into your Azure Orbital Ground Station solution
Content Source: articles/orbital/partner-network-integration.md
Service: orbital
GitHub Login: @apoorvanori
Microsoft Alias: apoorvanori

@altrivis
Thanks for your feedback! We will investigate and update as appropriate.

Ack. Added bullet/link updates and clarified contract prereq in a couple places. Will update again once I get more info about 3P contracts/keys from Apoorva.

#please-close
Will work with Apoorva to create more detailed doc page specific to KSAT and Viasat


Dbatiste/perf opt

@dlockhart : this change essentially updates d2l-icon to not wrap iron-icon.  It contains the same logic plus the focusable IE fix, and our styles for icons.  Not wrapping also means we have easy access to the svg, so it's not necessary to walk the composed DOM looking for it.  The following are the perf numbers (250 icons) associated with the change:
Chrome
110.2ms  ->  66.6ms  (39.6% imp)

Safari
129.6ms  ->  70.5ms  (45.6% imp)

FF
321.4ms  ->  167.7ms  (47.8% imp)

IE
568.3ms  ->  325.2ms  (42.8% imp)

I am on the fence about whether or not we should fork iron-icon and apply our changes there or keep this d2l-icon code here.  Currently we have our d2l-icon included in this repo along with the other components (d2l-icon-button, d2l-icon-link, etc.) and the icon-sets.  As is, it's detached from the original.  Thoughts?

@dbatiste that's an interesting question. If we fork and include the fork, I'm guessing your performance gains which are quite significant will be greatly reduced. On the flip side, if we embed a copy we get the larger performance gains but over time the code is going to deviate to the point where we can't take advantage of any innovations iron-icon comes up with. We could mitigate that by diligently applying patches, or just not worry about it.

I just ran this through Galen. Things are still lined up correctly in Chrome and Firefox.

@awikkerink : thanks for running that!  :)
@dlockhart : if we fork we'll still get the performance gains - they come mainly from not having the extra  layer and all the jaz that comes with that.  Instead of:
<d2l-icon>
  <iron-icon>
    <svg></svg>
  </iron-icon>
</d2l-icon>

We have:
<d2l-icon or iron-icon>
  <svg></svg>
</d2l-icon or iron-icon>

We have a mix of d2l-icon and iron-icon throughout our stuff, so if we fork, our changes would either involve switching the component name to d2l-icon (meaning we never intend to merge back), or we need to go through everything else and switch to iron-icon.  Using a fork would also mean pointing all those bower dependencies at a GitHub repo.


Add Shy away by Twenty One Pilots

Song Name
Shy away
Artist Name
Twenty One Pilots
Source
Youtube
Youtube Link
https://www.youtube.com/watch?v=niR2qJ3mGEE&pp=ygUIc2h5IGF3YXk%3D
Direct File Link
No response
Song ID
1249318
Extra
Created from Auto Nong.
Created for level: 106409739

accept


Enable forcing langugae with imdb_lookup

Enable forcing return language in imdb_lookup.
Use by setting language code in config: http://www.science.co.il/Language/Locale-codes.asp
imdb_lookup: es-mx

Isn't caching going to mess this up royally? Every cached item will need to know what language it is, correct?

I can't say that I'm familiar with exactly how caching works, but I suspect it will be irrelevant. Since the user will probably not run several lookups with different languages, but just stick to the one, whatever is cached will remain relevant, won't it?

@liiight Say an user has 2 tasks they want different language lookups for. If the cache isn't made language aware the results will be a seemingly random language depending on which task looked up a given movie first.

I see. I can change lookup to be based on original title. If true try to match title and if fails save title with a relation to query language. Something like that sounds good?

If we want selectable language support, I think each cached item needs a language column, and lookups only grab results from the requested language.

Actually you'd want more than a column. Each selected language can affect more than the title, but also plot air date, etc. I think movie ID will need to be mapped one to many with a MovieData table that will hold the language column.


Application registration error

I am running the BeaconList tutorial and I am getting this error while running it -
Application HelloBeacon has not been registered. This is either due to a require() error during initialization or failure to call AppRegistry.registerComponent.
I have dragged the RNBeacon.xcodeproj to my HelloBeacon project in XCode, changed Info.plist as required and tried deploying it on iPad mini (iOS 9.2.1).
Any clues whats wrong here?

Closing this at it was some issue related to linking where the RNBeacon project had run into node-gyp build issues as I had missed to see when installing the react-native-ibeacon.


feat: add --ignore-engines option

https://github.com/antongolub/yarn-audit-fix/pull/165/checks?check_run_id=3294325391
Preparing temp assets...
Patching yarn.lock with audit data...
invoke yarn audit --json --level moderate
Upgraded deps<EMAIL_ADDRESS>Installing deps update...
invoke yarn install --update-checksums
[1/4] Resolving packages...
[2/4] Fetching packages...
error<EMAIL_ADDRESS>The engine "node" is incompatible with this module. Expected version ">=14.8". Got "12.22.4"
error Found incompatible module.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Failure!
{
  status: 1,
  signal: null,
  output: [ null, null, null ],
  pid: 1663,
  stdout: null,
  stderr: null
}

:tada: This issue has been resolved in version 6.4.0 :tada:
The release is available on:

v6.4.0
npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Could not find JS function u'ur'

Please follow the guide below

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.06.23. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[X] I've verified and I assure that I'm running youtube-dl 2016.06.23

Before submitting an issue make sure you have:

[X] At least skimmed through README and most notably FAQ and BUGS sections
[X] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[X] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

$ /usr/local/bin/youtube-dl -v  https://www.youtube.com/watch?v=QL2CCeWZWQI
[debug] System config: []
[debug] User config: [u'-f', u'bestvideo[ext=mp4]+bestaudio[ext=m4a]/mp4']
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=QL2CCeWZWQI']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.06.23
[debug] Python version 2.7.10 - Darwin-15.5.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 2.8.6, ffprobe 2.8.6
[debug] Proxy map: {}
[youtube] QL2CCeWZWQI: Downloading webpage
[youtube] QL2CCeWZWQI: Downloading video info webpage
[youtube] QL2CCeWZWQI: Extracting video information
[youtube] {22} signature length 40.43, html5 player en_US-vflkX1eCa
[youtube] QL2CCeWZWQI: Downloading player https://s.ytimg.com/yts/jsbin/player-en_US-vflkX1eCa/base.js
ERROR: Signature extraction failed: Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 997, in _decrypt_signature
    video_id, player_url, s
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 912, in _extract_signature_function
    res = self._parse_sig_js(code)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 975, in _parse_sig_js
    initial_function = jsi.extract_function(funcname)
  File "/usr/local/bin/youtube-dl/youtube_dl/jsinterp.py", line 241, in extract_function
    raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
 (caused by ExtractorError(u"Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.",)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 997, in _decrypt_signature
    video_id, player_url, s
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 912, in _extract_signature_function
    res = self._parse_sig_js(code)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 975, in _parse_sig_js
    initial_function = jsi.extract_function(funcname)
  File "/usr/local/bin/youtube-dl/youtube_dl/jsinterp.py", line 241, in extract_function
    raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 676, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 343, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 1570, in _real_extract
    encrypted_sig, video_id, player_url, age_gate)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 1007, in _decrypt_signature
    'Signature extraction failed: ' + tb, cause=e)
ExtractorError: Signature extraction failed: Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 997, in _decrypt_signature
    video_id, player_url, s
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 912, in _extract_signature_function
    res = self._parse_sig_js(code)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 975, in _parse_sig_js
    initial_function = jsi.extract_function(funcname)
  File "/usr/local/bin/youtube-dl/youtube_dl/jsinterp.py", line 241, in extract_function
    raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
 (caused by ExtractorError(u"Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.",)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
...

---
### Description of your *issue*, suggested solution and other information

Tried to download this video: https://www.youtube.com/watch?v=QL2CCeWZWQI and always get the "Could not find JS function u'ur'" error. I've updated to the latest version and that has not resolved anything. Other videos are downloading fine.

Also have this same issue:
/usr/local/bin/youtube-dl -v --verbose --socket-timeout 10 --max-filesize 500m -o '/mnt/ephemeral/workspace/youtubeJ5dYZ6HuOV' -f 'bestvideo[ext=mp4][height<=240]+bestaudio[ext=m4a]' https://www.youtube.com/watch?v=QxsmWxxouIM  Mem: 5.5 MB
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'--verbose', u'--socket-timeout', u'10', u'--max-filesize', u'500m', u'-o', u'/mnt/ephemeral/workspace/youtubeJ5dYZ6HuOV', u'-f', u'bestvideo[ext=mp4][height<=240]+bestaudio[ext=m4a]', u'https://www.youtube.com/watch?v=QxsmWxxouIM']
[debug] Encodings: locale UTF-8, fs UTF-8, out None, pref UTF-8
[debug] youtube-dl version 2016.05.01
[debug] Python version 2.6.6 - Linux-2.6.32-358.14.1.el6.x86_64-x86_64-with-redhat-6.7-Santiago
[debug] exe versions: avprobe v12_dev0-2090-g4d05e93, ffmpeg git-2016-05-10-6c44696, ffprobe git-2016-05-10-6c44696
[debug] Proxy map: {}
ERROR: Signature extraction failed: Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 995, in _decrypt_signature
video_id, player_url, s
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 910, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 973, in _parse_sig_js
initial_function = jsi.extract_function(funcname)
File "/usr/local/bin/youtube-dl/youtube_dl/jsinterp.py", line 241, in extract_function
raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
(caused by ExtractorError(u"Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.",)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 995, in _decrypt_signature
video_id, player_url, s
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 910, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 973, in _parse_sig_js
initial_function = jsi.extract_function(funcname)
File "/usr/local/bin/youtube-dl/youtube_dl/jsinterp.py", line 241, in extract_function
raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 673, in extract_info
ie_result = ie.extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 341, in extract
return self._real_extract(url)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 1568, in _real_extract
encrypted_sig, video_id, player_url, age_gate)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 1005, in _decrypt_signature
'Signature extraction failed: ' + tb, cause=e)
ExtractorError: Signature extraction failed: Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 995, in _decrypt_signature
video_id, player_url, s
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 910, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 973, in _parse_sig_js
initial_function = jsi.extract_function(funcname)
File "/usr/local/bin/youtube-dl/youtube_dl/jsinterp.py", line 241, in extract_function
raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
(caused by ExtractorError(u"Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.",)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Samething around here.

[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--metadata-from-title', '%(title)s', '--extract-audio', '--audio-format', 'mp3', '--verbose', 'http://www.youtube.com/watch?v=wXVghiHgUI0']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.06.23
[debug] Python version 3.5.0+ - Linux-4.2.0-35-generic-x86_64-with-Ubuntu-15.10-wily
[debug] exe versions: avconv 2.7.6-0ubuntu<IP_ADDRESS>, avprobe 2.7.6-0ubuntu<IP_ADDRESS>, ffmpeg 2.7.6-0ubuntu<IP_ADDRESS>, ffprobe 2.7.6-0ubuntu<IP_ADDRESS>
[debug] Proxy map: {}
[youtube] wXVghiHgUI0: Downloading webpage
[youtube] wXVghiHgUI0: Downloading video info webpage
[youtube] wXVghiHgUI0: Extracting video information
[youtube] {22} signature length 40.43, html5 player en_US-vflkX1eCa
[youtube] wXVghiHgUI0: Downloading player https://s.ytimg.com/yts/jsbin/player-en_US-vflkX1eCa/base.js
ERROR: Signature extraction failed: Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 997, in _decrypt_signature
video_id, player_url, s
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 912, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 975, in _parse_sig_js
initial_function = jsi.extract_function(funcname)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/jsinterp.py", line 241, in extract_function
raise ExtractorError('Could not find JS function %r' % funcname)
youtube_dl.utils.ExtractorError: Could not find JS function 'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
(caused by ExtractorError("Could not find JS function 'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.",)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 997, in _decrypt_signature
video_id, player_url, s
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 912, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 975, in _parse_sig_js
initial_function = jsi.extract_function(funcname)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/jsinterp.py", line 241, in extract_function
raise ExtractorError('Could not find JS function %r' % funcname)
youtube_dl.utils.ExtractorError: Could not find JS function 'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 997, in _decrypt_signature
video_id, player_url, s
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 912, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 975, in _parse_sig_js
initial_function = jsi.extract_function(funcname)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/jsinterp.py", line 241, in extract_function
raise ExtractorError('Could not find JS function %r' % funcname)
youtube_dl.utils.ExtractorError: Could not find JS function 'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/YoutubeDL.py", line 676, in extract_info
ie_result = ie.extract(url)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/common.py", line 343, in extract
return self._real_extract(url)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 1570, in _real_extract
encrypted_sig, video_id, player_url, age_gate)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 1007, in _decrypt_signature
'Signature extraction failed: ' + tb, cause=e)
youtube_dl.utils.ExtractorError: Signature extraction failed: Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 997, in _decrypt_signature
video_id, player_url, s
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 912, in _extract_signature_function
res = self._parse_sig_js(code)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/extractor/youtube.py", line 975, in _parse_sig_js
initial_function = jsi.extract_function(funcname)
File "/usr/local/lib/python3.5/dist-packages/youtube_dl/jsinterp.py", line 241, in extract_function
raise ExtractorError('Could not find JS function %r' % funcname)
youtube_dl.utils.ExtractorError: Could not find JS function 'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
(caused by ExtractorError("Could not find JS function 'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.",)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Same here.

Same problem here, started only a couple of hours ago it seems.

Same here, although it seems to be happening only for YouTube videos
$ youtube-dl --version
2016.06.23

$ youtube-dl https://www.instagram.com/p/BCrZZ4YkC9J
[Instagram] BCrZZ4YkC9J: Downloading webpage
[download] Destination: Video by jussondrums-BCrZZ4YkC9J.mp4
[download] 100% of 894.76KiB in 00:00
$
$ youtube-dl https://gifs.com/gif/Krm0EG
[generic] Krm0EG: Requesting header
WARNING: Falling back on generic information extractor.
[generic] Krm0EG: Downloading webpage
[generic] Krm0EG: Extracting information
[download] Destination: Bill Gates DAB!-Krm0EG.mp4
[download] 100% of 234.17KiB in 00:00
$
$ youtube-dl https://www.youtube.com/watch?v=b-J95fYuVz4
[youtube] b-J95fYuVz4: Downloading webpage
[youtube] b-J95fYuVz4: Downloading video info webpage
[youtube] b-J95fYuVz4: Extracting video information
[youtube] b-J95fYuVz4: Downloading js player en_US-vflkX1eCa
ERROR: Signature extraction failed: Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 997, in _decrypt_signature
    video_id, player_url, s
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 912, in _extract_signature_function
    res = self._parse_sig_js(code)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 975, in _parse_sig_js
    initial_function = jsi.extract_function(funcname)
  File "/usr/local/bin/youtube-dl/youtube_dl/jsinterp.py", line 241, in extract_function
    raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
 (caused by ExtractorError(u"Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.",)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Emmanuels-MBP:mediaconcat emmanuelodeke$ youtube-dl version
ERROR: u'version' is not a valid URL. Set --default-search "ytsearch" (or run  youtube-dl "ytsearch:version" ) to search YouTube
$
$ youtube-dl https://www.youtube.com/watch?v=GXiyHhbRRds
[youtube] GXiyHhbRRds: Downloading webpage
[youtube] GXiyHhbRRds: Downloading video info webpage
[youtube] GXiyHhbRRds: Extracting video information
[youtube] GXiyHhbRRds: Downloading js player en_US-vflkX1eCa
ERROR: Signature extraction failed: Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 997, in _decrypt_signature
    video_id, player_url, s
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 912, in _extract_signature_function
    res = self._parse_sig_js(code)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 975, in _parse_sig_js
    initial_function = jsi.extract_function(funcname)
  File "/usr/local/bin/youtube-dl/youtube_dl/jsinterp.py", line 241, in extract_function
    raise ExtractorError('Could not find JS function %r' % funcname)
ExtractorError: Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
 (caused by ExtractorError(u"Could not find JS function u'ur'; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.",)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

I had same issues, however, I could solved this situation by upgrading by pip install
sudo apt-get install youtube-dl
Reading package lists... Done
Building dependency tree
Reading state information... Done
youtube-dl is already the newest version (2016.02.22-1).
0 upgraded, 0 newly installed, 0 to remove and 38 not upgraded.

$ sudo pip install --upgrade youtube-dl
The directory '/home/vanp/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/vanp/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting youtube-dl
  Downloading youtube_dl-2016.9.24-py2.py3-none-any.whl (1.4MB)
    100% |████████████████████████████████| 1.4MB 540kB/s
Installing collected packages: youtube-dl
  Found existing installation: youtube-dl 2016.2.22
    DEPRECATION: Uninstalling a distutils installed project (youtube-dl) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling youtube-dl-2016.2.22:
      Successfully uninstalled youtube-dl-2016.2.22
Successfully installed youtube-dl-2016.9.24

and This works for me

In a word: don't use apt-get. Its youtube-dl is always horribly out-dated.


Underscore in kong.response headers

Summary
It looks like kong transform underscore into dash while setting a new response header
Steps To Reproduce
function Test:header_filter(conf)
  kong.response.set_header('service_name', 'test')
end

results with service-name
Additional Details & Logs

Kong version - 2.3.2

Thanks for reporting this. We're investigating this problem.

There is this option:
https://github.com/openresty/lua-nginx-module#lua_transform_underscores_in_response_headers
KONG_NGINX_HTTP_LUA_TRANSFOR_UNDERSCORES_IN_RESPONSE_HEADERS=off

(but I am not sure if it breaks anything)

Prometheus plugin for example has code like this:
ngx.header.content_type = "text/plain"

Hello @Asafb26, as @bungle is pointing out, some plugins do rely on the automatic conversion of underscores into slashes, so we can't "fix" this without breaking someone else's code.
A possible workaround is just setting lua_transform_underscores_in_response_headers to off. The tradeoff is that doing so might "break" plugins that rely on the underscore transformation, like Prometheus (I don't think this would "break" the Prometheus plugin, but it might make it miss the Content-Type header on some responses).
Palliative mesures:

I am documenting this behavior officially (plus workaround) in #7018
I have opened a PR to the Prometheus Plugin (https://github.com/Kong/kong-plugin-prometheus/pull/124) to stop using underscores, so it is fully impervious to lua_transform_underscores_in_response_headers
I have reviewed all the bundled plugins and the Kong source itself. I did not find another place in which we used underscored header names.

Hi @kikito , Thanks 🙏
I guess that we may change this behavior in the near future? as a Prometheus user, I'm thinking that toggle off the lua_transform_underscores_in_response_headers can be lead to some issues

I guess that we may change this behavior in the near future? as a Prometheus user, I'm thinking that toggle off the lua_transform_underscores_in_response_headers can be lead to some issues

The change in the prometheus plugin seems fairly small. The way I see it your options are:

Just wait until we release a new version of the Prometheus plugin (this can happen independently from Kong releases). Then you can use that version in your Kong rockspec and activate lua_transorm_underscores_in_response_headers
Manually modify the code of the Prometheus plugin yourself. How to implement this depends greatly on your infrastructure setup. The patch to apply is very small (just two lines) but the files can be placed in lots of locations depending on Linux distros etc.
Or you could fork the plugin, release your own version of the rockspec to luarocks, and use that while we release ours.

I would use option 1 if I could wait and option 3 if I couldn't wait.
For now, since this issue is "addressed" in 2.4, I will close the issue.


Fix vm deletion with --cascade=orphan

What this PR does / why we need it:
With https://github.com/kubevirt/kubevirt/pull/8937 a foreground finalizer has been added to the vm.
If --cascade=orphan is passed to a vm delete command the controller should not delete the associated vmi.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes https://github.com/kubevirt/kubevirt/issues/9223
Special notes for your reviewer:
Release note:

NONE

/cc @xpivarc @maya-r

/lgtm
thanks!

/approve

/retest-required

/retest-required

/cherrypick release-0.59


*: Fix embedded mutexes leaking methods

This pull request cleans up all exported structs that were leaking the Lock/Unlock methods from an embedded mutex.

@s111 is there a linter/tool for detecting this?

I've written a quick extension of golint for my own use: https://github.com/s111/lint.

@s111 any way that patch can get into golint upstream or some other linter? If it's a style problem worth fixing I'd like to be able to catch it as part of CI...

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@747993d). Click here to learn what that means.
The diff coverage is 91.42%.

@@            Coverage Diff            @@
##             master    #7833   +/-   ##
=========================================
  Coverage          ?   75.91%
=========================================
  Files             ?      332
  Lines             ?    26177
  Branches          ?        0
=========================================
  Hits              ?    19871
  Misses            ?     4887
  Partials          ?     1419

Impacted Files
Coverage Δ

pkg/testutil/recorder.go
70.37% <100%> (ø)

raft/storage.go
87.96% <100%> (ø)

etcdserver/stats/leader.go
79.48% <75%> (ø)

etcdserver/stats/server.go
71.66% <75%> (ø)

etcdserver/membership/cluster.go
86.98% <93.33%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 747993d...38d0ebe. Read the comment docs.

@sebschrader If your patch https://github.com/s111/lint/commit/d196a16cdcfe62812e210d72d908d3aa81660ef0 goes in upstream, it would be great.
lgtm. Thanks!

Hey @gyuho, you seem to have erroneously mentioned me here.

@sebschrader Apology! I meant to mention @s111
Thanks!


r2pm -i error

r2pm -i retdec-r2plugin                                                                              git clone https://github.com/radareorg/radare2-pm
/data/data/com.termux/files/home/.local/share/radare2/r2pm/git/radare2-pm                                 [r2pm] Updating package database /data/data/com.termux/files/home/.local/share/radare2/r2pm/db ...        Already up to date.                                                                                HEAD is now at 03b6ec1 Update ghidra package
Updating 03b6ec1..4fc22c0                                                                                Fast-forward                                                                                db/ghidra          | 2 +-                                                                                db/r2ghidra-sleigh | 7 +++++--
2 files changed, 6 insertions(+), 3 deletions(-)                                                         [r2pm] Updating package database /data/data/com.termux/files/home/.local/share/radare2/r2pm/db ...        Already up to date.                                                                                error: pathspec 'r2-5.5.0' did not match any file(s) known to git
Already up to date.                                                                                Install Done For retdec-r2plugin                                                                          remote: Enumerating objects: 1439, done.                                                                  remote: Counting objects: 100% (1439/1439), done.
remote: Compressing objects: 100% (534/534), done.                                                        remote: Total 1394 (delta 832), reused 1316 (delta 755), pack-reused 0                                    Receiving objects: 100% (1394/1394), 205.73 KiB | 55.00 KiB/s, done.                                      Resolving deltas: 100% (832/832), completed with 30 local objects.
From https://github.com/trufae/retdec-r2plugin                                                             * branch            fix-build  -> FETCH_HEAD                                                             hint: You have divergent branches and need to specify how to reconcile them.                              hint: You can do so by running one of the following commands sometime before
hint: your next pull:                                                                                hint:                                                                                hint:   git config pull.rebase false  # merge (the default strategy)                                      hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only                                                 hint:                                                                                hint: You can replace "git config" with "git config --global" to set a default                            hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per                             hint: invocation.                                                                                fatal: Need to specify how to reconcile divergent branches.                                               make: *** No rule to make target 'install'.  Stop.
anyone can help ?

compile it by yourself. r2pm is kinda suck for now i think.

Builds again, retdec fixed their bugs silently


"File clean" result for any file

The project runs clean and good,  but it gives a "File clean" result for any file. I tried scanning the EICAR sample file. When scanning the same file with clamscan.exe I get the expected result. I checked the clamd.conf file, I have specified the database as C:\Program Files\ClamAV\db, which is the same what I use for clamscan.
I understand that this query is a bit irrelevant to your project but I would really appreciate your help. Thanks.

Can you provide a code sample?

Sir it is tge same example code you have provided in thw read me file

I re-installed the service, build the config file again, and it is working fine now :+1: Thanks a lot for this project and for your concern Sir.


docs: remove width from cilium arch image

This allows a browser to properly scale the image.
Signed-off-by: Martynas Pumputis<EMAIL_ADDRESS>

This change is

test-docs-please


Spring cloud config server for vault not supporting profiles

Spring cloud config when using vault server is not picking up the correct backend based on profiles for the application.
E.g I have backends secret/ttt-microservice, secret/ttt-microservice/default
If i give the profile=default and develop being the label
http://localhost:1010/ttt-microservice/default/develop
I see the backend for secret/ttt-microservice being mapped but it doesn't pick secret/ttt-microservice/default. Is is correct to deduce spring cloud config for vault doesn't support profiles. Is it something to expect in future. thanks aakash

It is not the case. Default is a special profile in spring

its only with the vault it doesnt map the vault source correctly with secret/ttt-microservice/{profile}. For git it picks up correctly as we have composite property sources. If i change the profile to dev  and create vault backend secret/microservicename/dev it still maps to secret/ttt-microservice

versions <spring.boot.version>2.0.0.M3</spring.boot.version>
<spring.cloud.version>Finchley.M1</spring.cloud.version>

@spencergibb  @mp911de can you please confirm will help to move ahead. thanks

By default vault looks for profiles with a comma, so ttt-microservice,default. You can change it with spring.cloud.config.server.vault.profile-separator.

thanks it seems to work with ,


Two memory management improvements

I created https://northerntech.atlassian.net/browse/MEN-7618 for the unnecessary spike in memory usage for header.tar.


Support Request: Need help/examples for the mqtt part

Describe Your Problem
I don't know what i can put in the two option fields, and how
@banboobee as far as I understand, you wrote the mqtt part. Please could you provide a little guide or help page in the wiki?
MQTT Url
The field hint just shows an IP but you need to give a full URL with mqtt://192.168.x.x. Does it work when using tls by mqtts: instead?
MQTT Options
What are the possible options and how to place more than one? I think one can put username, password, psk and other mqtt-stuff, or?
MQTT Pub Options
Are there any other options other than retain?
Device & Model
meter (EU)
Homebridge SwitchBot Plugin Version
2.5.2
Operating System
Raspbian Bullseye

OPTIONS can be specified in a JSON form of {"a": x, "b"; y}, and ,as described in the PR, following repository details the OPTIONS,
https://github.com/mqttjs/MQTT.js
More preciously, for mqttOptions,
https://github.com/mqttjs/MQTT.js#mqttclientstreambuilder-options
and for mqttPubOptions,
https://github.com/mqttjs/MQTT.js#mqttclientpublishtopic-message-options-callback
also for URL scheme,
https://github.com/mqttjs/MQTT.js#mqttconnecturl-options

OPTIONS can be specified in a JSON form of {"a": x, "b": y}, and ,as described in the PR, following repository details the OPTIONS,
https://github.com/mqttjs/MQTT.js
More precisely, for mqttOptions,
https://github.com/mqttjs/MQTT.js#mqttclientstreambuilder-options
and for mqttPubOptions,
https://github.com/mqttjs/MQTT.js#mqttclientpublishtopic-message-options-callback
also for URL scheme,
https://github.com/mqttjs/MQTT.js#mqttconnecturl-options

Tanks for your help @banboobee. With these informations I could manage all I need.
But that does not explain how I would put this options into the input boxes of the plugin's UI. I need to put that directly to the JSON config.
So I ask you again, if you want to make a little wiki article for that in your PR or the issues.
BTW: is there any chance we can get a battery level exposed as well?

Sorry @banboobee but I think there is a bug. In the UI there are three fields for MQTT URL, MQTT Options and MQTT Pub Options.
But both of the option fields do not work. If I try to put a JSON-object to that fields all " are escaped by \". It worked, when I put the obejects directly to the plugins JSON config. But if I now change anything else in the plugin config via UI, all mqtt option objects are stripped from the config.
No I ended up with an error code in the log, even if I remove all mqtt things. I think I had to re-setup the complete switchbot plugin now. It's just 4 object, so no problem on my site, but others maybe won't like to do.
Config example:
{
  "deviceId": "123456789012",
  "configDeviceName": "MeterPlus",
  "configDeviceType": "MeterPlus",
  "mqttURL": "mqtts://mqtt.example.com",
  "mqttOptions": {
      "port": 8883,
      "username": "hb-switchbot",
      "password": "very-secret"
  },
  "mqttPubOptions": {
      "retain": true
  },
  "refreshRate": 30
},

The Error in Log:
[SwitchBot] API Request: TypeError: Cannot read properties of undefined (reading 'map'), Submit Bugs Here: https://tinyurl.com/SwitchBotBug


Sal 5336 fix dnd bug

https://spotonteam.atlassian.net/browse/SAL-5336
fix bug with adding component on hover
Loom

it seems possible, but not easy. Should I dig more?))

I think it's worth it 👍
if we fix this bug, we can make a pull-request against the head https://github.com/Kiho/react-form-builder , as I'm sure plenty of others would like it


SBatch dont generate output file

I do sbatch on a basic script
#!/bin/bash
#SBATCH --job-name=JOB_ID_NAME
#SBATCH --ntasks=4
#SBATCH --output=/test.out
echo a
Work appears as COMPLETED but can't find the output file anywhere

I try with #SBATCH  -o ../../../../../../test.out to force to put it on / directory and still the same problem

Its could be because output of Docker is stdout? I dont think so but could be

Please need some help, i pass one day trying to solve this issue

Hi @kikegarcia28 ,
The problem is that you trying to write the output file to the path that is accessible for the current node only. The output file must be shared between all compute nodes, the / path of axc-headnode is not.
PARTITION AVAIL  TIMELIMIT  NODES  STATE NODELIST
compute*     up 1-00:00:00      4   idle axc-compute-[01-04]```

As you can see, only `axc-compute-[01-04]` are included under the slurm control as compute nodes.

If you want to write output to that path anyway, you may create a new partition that will contain the head node only. Or, include the head node to the existing "compute" partition.

For example:

* Add this to the bottom of the `/etc/slurm/slurm.conf`

PartitionName=Debug Nodes=axc-headnode Default=No MaxTime=00:30:00 State=UP OverSubscribe=Yes

* Restart the project: `docker-compose restart`
* Add `--#SBATCH --partition debug` to your job script
* Remove `#SBATCH --ntasks=4`
* Run the job: `sbatch /test.sh`

In that way, you will see the `/test.out` file for sure.

SOLVED, thanks you!


⚠ Bofrost.nl has degraded performance

In f633006, Bofrost.nl (https://www.bofrost.nl) experienced degraded performance:

HTTP code: 200
Response time: 1298 ms

Resolved: Bofrost.nl performance has improved in 88e9078.


tiledbsoma 1.5.0rc0

Following
https://github.com/single-cell-data/TileDB-SOMA/wiki/Branches-and-releases
but with an rc step.
This is essentially @Shelnutt2 's https://github.com/TileDB-Inc/tiledbsoma-feedstock/tree/ss/1.5.0-rc0 branch

@jdblischak good to merge?

good to merge?

Looks good to me 🚀


Merge

Summary
Describe the goal of this pull request. Mention any related issue numbers from slackapi/bolt-js/issues.
Requirements (place an x in each [ ])

[x] I've read and understood the Contributing Guidelines and have done my best effort to follow them.
[x] I've read and agree to the Code of Conduct.

hi, @mac2flip! I noticed this PR committed some of your Slack app and other app credentials. For the security of your apps, I strongly recommend regenerating all of the exposed credentials in this PR's .env!

Hi!  I really had no idea what i was doing as i was just learning how to make slack apps, and it was just also my first time learning how to code.  So i really wasn't even sure what i was trying to commit at that time.  Anyways, i know more about things now than i did then.  Thanks for the suggestions.

Marc