KeyError : 'Assets'

Getting this error when searching for a song :
C:\Users\Octave\Desktop\spotdl>spotdl https://open.spotify.com/track/6gBFPUFcJLzWGx4lenP6h2
Fetching Song...
  0%|                                                                                |ETA: ?, ~min/songTraceback (most recent call last):
  File "C:\Users\Octave\AppData\Local\Programs\Python\Python38\Scripts\spotdl-script.py", line 11, in <module>
    load_entry_point('spotdl==3.2.0rc0', 'console_scripts', 'spotdl')()
  File "c:\users\octave\appdata\local\programs\python\python38\lib\site-packages\spotdl\__main__.py", line 88, in console_entry_point
    downloader.download_single_song(song)
  File "c:\users\octave\appdata\local\programs\python\python38\lib\site-packages\spotdl\download\downloader.py", line 283, in download_single_song
    download_song(songObj, self.displayManager, self.downloadTracker)
  File "c:\users\octave\appdata\local\programs\python\python38\lib\site-packages\spotdl\download\downloader.py", line 103, in download_song
    youtubeHandler = YouTube(
  File "c:\users\octave\appdata\local\programs\python\python38\lib\site-packages\pytube\__main__.py", line 91, in __init__
    self.prefetch()
  File "c:\users\octave\appdata\local\programs\python\python38\lib\site-packages\pytube\__main__.py", line 183, in prefetch
    self.js_url = extract.js_url(self.watch_html)
  File "c:\users\octave\appdata\local\programs\python\python38\lib\site-packages\pytube\extract.py", line 143, in js_url
    base_js = get_ytplayer_config(html)["assets"]["js"]
KeyError: 'assets'

The whole process left me with a goosebumps.spotdlTrackingFile file.

I'm getting the exact same error
spotdl https://open.spotify.com/track/7bCfHiRcfUjG0YVVNUL7Ve Fetching Song...   0%|                                                                                |ETA: ?, ~min/songTraceback (most recent call last):   File "C:\Users\admin\AppData\Local\Programs\Python\Python38\Scripts\spotdl-script.py", line 11, in <module>     load_entry_point('spotdl==3.2.0rc0', 'console_scripts', 'spotdl')()   File "c:\users\admin\appdata\local\programs\python\python38\lib\site-packages\spotdl\__main__.py", line 88, in console_entry_point     downloader.download_single_song(song)   File "c:\users\admin\appdata\local\programs\python\python38\lib\site-packages\spotdl\download\downloader.py", line 283, in download_single_song     download_song(songObj, self.displayManager, self.downloadTracker)   File "c:\users\admin\appdata\local\programs\python\python38\lib\site-packages\spotdl\download\downloader.py", line 103, in download_song     youtubeHandler = YouTube(   File "c:\users\admin\appdata\local\programs\python\python38\lib\site-packages\pytube\__main__.py", line 91, in __init__     self.prefetch()   File "c:\users\admin\appdata\local\programs\python\python38\lib\site-packages\pytube\__main__.py", line 183, in prefetch     self.js_url = extract.js_url(self.watch_html)   File "c:\users\admin\appdata\local\programs\python\python38\lib\site-packages\pytube\extract.py", line 143, in js_url     base_js = get_ytplayer_config(html)["assets"]["js"] KeyError: 'assets'

Upgrade to the latest version

same here...
`Fetching Playlist...
0%|                                                         |ETA: ?, ~min/songmultiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 47, in starmapstar
return list(itertools.starmap(args[0], args[1]))
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spotdl/download/downloader.py", line 104, in download_song
on_progress_callback = displayManager.pytube_progress_hook
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/main.py", line 91, in init
self.prefetch()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/main.py", line 183, in prefetch
self.js_url = extract.js_url(self.watch_html)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/extract.py", line 143, in js_url
base_js = get_ytplayer_config(html)["assets"]["js"]
KeyError: 'assets'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/bin/spotdl", line 8, in
sys.exit(console_entry_point())
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spotdl/main.py", line 116, in console_entry_point
downloader.download_multiple_songs(songObjList)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spotdl/download/downloader.py", line 297, in download_multiple_songs
for song in songObjList
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 276, in starmap
return self._map_async(func, iterable, starmapstar, chunksize).get()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 657, in get
raise self._value
KeyError: 'assets'`

No change with new version.

Unfortunately i have the same issue, was not able to proceed to further testing for the issue Audio Distortion#905

I have the same issue, it happens when I try downloading a song/playlist it brings up this message. I can't use anything.

$ spotdl https://open.spotify.com/track/2lCWfOtBzJRI2jakdXfdQv?si=KG6huv9XTe6SS2qFfv8h_A
Fetching Song...
0%|
(most recent call last):
File "c:\users\user_name\appdata\local\programs\python\python39\lib\runpy.py", line 197, in run_module_as_main
return run_code(code, main_globals, None,
File "c:\users\user_name\appdata\local\programs\python\python39\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\George\AppData\Local\Programs\Python\Python39\Scripts\spotdl.exe_main.py", line 7, in
File "c:\users\user_name\appdata\local\programs\python\python39\lib\site-packages\spotdl_main.py", line 100, in console_entry_point
downloader.download_single_song(song)
File "c:\users\user_name\appdata\local\programs\python\python39\lib\site-packages\spotdl\download\downloader.py", line 274, in download_single_song
download_song(songObj, self.displayManager, self.downloadTracker)
File "c:\users\user_name\appdata\local\programs\python\python39\lib\site-packages\spotdl\download\downloader.py", line 102, in download_song
youtubeHandler = YouTube(
File "c:\users\user_name\appdata\local\programs\python\python39\lib\site-packages\pytube_main.py", line 91, in init
self.prefetch()
File "c:\users\user_name\appdata\local\programs\python\python39\lib\site-packages\pytube_main_.py", line 183, in prefetch
self.js_url = extract.js_url(self.watch_html)
File "c:\users\user_name\appdata\local\programs\python\python39\lib\site-packages\pytube\extract.py", line 143, in js_url
base_js = get_ytplayer_config(html)["assets"]["js"]
KeyError: 'assets'

Maybe it's because of ffmpeg, what file path does it need going in exactly?

same here...
`Fetching Playlist...
0%| |ETA: ?, ~min/songmultiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 47, in starmapstar
return list(itertools.starmap(args[0], args[1]))
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spotdl/download/downloader.py", line 104, in download_song
on_progress_callback = displayManager.pytube_progress_hook
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/main.py", line 91, in init
self.prefetch()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/main.py", line 183, in prefetch
self.js_url = extract.js_url(self.watch_html)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pytube/extract.py", line 143, in js_url
base_js = get_ytplayer_config(html)["assets"]["js"]
KeyError: 'assets'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/bin/spotdl", line 8, in
sys.exit(console_entry_point())
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spotdl/main.py", line 116, in console_entry_point
downloader.download_multiple_songs(songObjList)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/spotdl/download/downloader.py", line 297, in download_multiple_songs
for song in songObjList
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 276, in starmap
return self._map_async(func, iterable, starmapstar, chunksize).get()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 657, in get
raise self._value
KeyError: 'assets'`

My mistake, I didn’t check the list of issues carefully, upgrading pytube can solve my problem.
~ pip install -U pytube Collecting pytube   Downloading pytube-10.0.0-py3-none-any.whl (40 kB)      |████████████████████████████████| 40 kB 579 kB/s  Requirement already satisfied, skipping upgrade: typing-extensions in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pytube) (<IP_ADDRESS>) Installing collected packages: pytube Successfully installed pytube-10.0.0 ➜  ~  ➜  ~  ➜  ~ spotdl https://open.spotify.com/playlist/** Fetching Playlist...                                                             100%|█████████████████████████████████████████████████▊|ETA: 00:03, 1.23min/song/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tqdm/std.py:528: TqdmWarning: clamping frac to range [0, 1]   colour=colour) 100%|██████████████████████████████████████████████████|ETA: 00:00, 0.94min/song 100%|█████████████████████████████████████████████████████|ETA: 00:00, ~min/song

Very well, apparently they fixed there problem in there upstream branch. A patch will be put up by evening (hopefully).

Fixed in v.3.2.1

Fixed in v.3.2.1


CI job update 2018-03-01 04:24

Update from CI

:x: Validation status: errors

File
Status
Preview URL
Details

:x:Error

Details

[Error] Cannot sync git repo to specified state. Please make sure the pull request is mergeable. Details: Git merge results in conflict, repository remote url: https://github.com/Azure/azure-docs-sdk-dotnet, working directory: W:\xvf5-s, target branch: master, target branch commit: a4c12cb26a32832acca5058a5ae62e5ced3890a9, source branch: 887dbcc04c6ceff76a50289d8f6f708d/master, source branch commit: .

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:x: Validation status: errors

File
Status
Preview URL
Details

:x:Error

Details

[Error] Cannot sync git repo to specified state. Please make sure the pull request is mergeable. Details: Git merge results in conflict, repository remote url: https://github.com/Azure/azure-docs-sdk-dotnet, working directory: W:\dszk-s, target branch: master, target branch commit: a4c12cb26a32832acca5058a5ae62e5ced3890a9, source branch: 887dbcc04c6ceff76a50289d8f6f708d/master, source branch commit: .

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:x: Validation status: errors

File
Status
Preview URL
Details

:x:Error

Details

:x:Error

Details

[Error] Failed to load 1 files, aborting...
[Error] System.IO.FileNotFoundException: File not found.
File name: 'W:\hovh-log-8816a9cf\change-list-with-redirection.mapped.tsv'
at Microsoft.DocAsCode.Build.Engine.Incrementals.ChangeList.Parse(String tsvFile, String baseDir)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument(BuildJsonConfig config, TemplateManager templateManager, String baseDirectory, String outputDirectory, String pluginDirectory, String templateDirectory)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument()
Line 80: [Warning] Failed to run script W:\hovh-s_dependentPackages\ECMA2Yaml\tools\Run.ps1 with exit code -1

[Error] Failed to load 1 files, aborting...
[Error] System.IO.FileNotFoundException: File not found.
File name: 'W:\hovh-log-8816a9cf\change-list-with-redirection.mapped.tsv'
at Microsoft.DocAsCode.Build.Engine.Incrementals.ChangeList.Parse(String tsvFile, String baseDir)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument(BuildJsonConfig config, TemplateManager templateManager, String baseDirectory, String outputDirectory, String pluginDirectory, String templateDirectory)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument()
Line 80: [Warning] Failed to run script W:\hovh-s_dependentPackages\ECMA2Yaml\tools\Run.ps1 with exit code -1

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:x: Validation status: errors

File
Status
Preview URL
Details

:x:Error

Details

:x:Error

Details

[Error] Failed to load 1 files, aborting...
[Error] System.IO.FileNotFoundException: File not found.
File name: 'W:\nrd5-log-179712e6\change-list-with-redirection.mapped.tsv'
at Microsoft.DocAsCode.Build.Engine.Incrementals.ChangeList.Parse(String tsvFile, String baseDir)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument(BuildJsonConfig config, TemplateManager templateManager, String baseDirectory, String outputDirectory, String pluginDirectory, String templateDirectory)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument()
Line 80: [Warning] Failed to run script W:\nrd5-s_dependentPackages\ECMA2Yaml\tools\Run.ps1 with exit code -1

[Error] Failed to load 1 files, aborting...
[Error] System.IO.FileNotFoundException: File not found.
File name: 'W:\nrd5-log-179712e6\change-list-with-redirection.mapped.tsv'
at Microsoft.DocAsCode.Build.Engine.Incrementals.ChangeList.Parse(String tsvFile, String baseDir)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument(BuildJsonConfig config, TemplateManager templateManager, String baseDirectory, String outputDirectory, String pluginDirectory, String templateDirectory)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument()
Line 80: [Warning] Failed to run script W:\nrd5-s_dependentPackages\ECMA2Yaml\tools\Run.ps1 with exit code -1

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:x: Validation status: errors

File
Status
Preview URL
Details

:x:Error

Details

:x:Error

Details

[Error] Failed to load 1 files, aborting...
[Error] System.IO.FileNotFoundException: File not found.
File name: 'W:\5phm-log-f05211af\change-list-with-redirection.mapped.tsv'
at Microsoft.DocAsCode.Build.Engine.Incrementals.ChangeList.Parse(String tsvFile, String baseDir)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument(BuildJsonConfig config, TemplateManager templateManager, String baseDirectory, String outputDirectory, String pluginDirectory, String templateDirectory)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument()
Line 80: [Warning] Failed to run script W:\5phm-s_dependentPackages\ECMA2Yaml\tools\Run.ps1 with exit code -1

[Error] Failed to load 1 files, aborting...
[Error] System.IO.FileNotFoundException: File not found.
File name: 'W:\5phm-log-f05211af\change-list-with-redirection.mapped.tsv'
at Microsoft.DocAsCode.Build.Engine.Incrementals.ChangeList.Parse(String tsvFile, String baseDir)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument(BuildJsonConfig config, TemplateManager templateManager, String baseDirectory, String outputDirectory, String pluginDirectory, String templateDirectory)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument()
Line 80: [Warning] Failed to run script W:\5phm-s_dependentPackages\ECMA2Yaml\tools\Run.ps1 with exit code -1

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

:x: Validation status: errors

File
Status
Preview URL
Details

:x:Error

Details

:x:Error

Details

[Error] Failed to load 1 files, aborting...
[Error] System.IO.FileNotFoundException: File not found.
File name: 'W:\1kcb-log-e61bb78f\change-list-with-redirection.mapped.tsv'
at Microsoft.DocAsCode.Build.Engine.Incrementals.ChangeList.Parse(String tsvFile, String baseDir)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument(BuildJsonConfig config, TemplateManager templateManager, String baseDirectory, String outputDirectory, String pluginDirectory, String templateDirectory)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument()
Line 80: [Warning] Failed to run script W:\1kcb-s_dependentPackages\ECMA2Yaml\tools\Run.ps1 with exit code -1

[Error] Failed to load 1 files, aborting...
[Error] System.IO.FileNotFoundException: File not found.
File name: 'W:\1kcb-log-e61bb78f\change-list-with-redirection.mapped.tsv'
at Microsoft.DocAsCode.Build.Engine.Incrementals.ChangeList.Parse(String tsvFile, String baseDir)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument(BuildJsonConfig config, TemplateManager templateManager, String baseDirectory, String outputDirectory, String pluginDirectory, String templateDirectory)
at Microsoft.DocAsCode.SubCommands.DocumentBuilderWrapper.BuildDocument()
Line 80: [Warning] Failed to run script W:\1kcb-s_dependentPackages\ECMA2Yaml\tools\Run.ps1 with exit code -1

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Update generator to reject plugin names containing disallowed chars

[x] This contribution adheres to CONTRIBUTING.md.

What does this Pull Request accomplish?
Addresses #168

Refactors generator's Valid Measurement Service Name.vi to explicitly reject names that contain disallowed characters.

Utility VI Get Measurement Library.vi has been adjusted in order to be utilized by (and thus reduce duplicate code in) Valid Measurement Service Name.vi

Updates Create Measurement Service.vi to automatically correct user-provided names which might happen to have (disallowed) leading or trailing whitespace.  Also, improves the message shown if a user enters an invalid name.
Adds new G tests to ensure proper support for both allowed and disallowed measurement plugin names.

Why should this Pull Request be merged?

User of generator dialog benefits from better handling of & response to the name they've entered.
G tests reduce manual test burden are substantially more exhaustive, in terms of coverage, than what we could expect from typical testing by-hand.

What testing has been done?

New G tests pass.
Existing G tests still pass (these tests cover other naming issues: collisions & reserved names).
Exercised dialog by-hand for several allowed & disallowed requested names; all behaved as expected.

Additional Notes

Previously, there was no explicit check for disallowed characters in pending measurement plugin names.  Rather, the generator attempted create a library on disk having the requested file name, and the name would be rejected at that point if it were disallowed by the OS.  To be consistent with what has shipped already, the disallowed character set that's been adopted in this change are consistent with the set disallowed in Windows filenames.

Changes to Create Measurement Plugin.vi
This is what the generator now shows when the user has entered an invalid name:

Diagram Changes

Assigned new text constant.  Changed to a boolean glyph indicator (rather than string with red text) for aesthetics.

Auto-correct for leading & trailing whitespace in the event-handling case upon name value changed.


Add time buffer to persistence test for slower machines

For slower machines, issues arise with stopping the server and immediately running the same instance. I added a line to let the process sleep for 100 milliseconds before restarting the server. 100 ms should be plenty of time, mine worked down to around 8.

I added the awaitStop to Server.java starter code. Let me know if you don't think this will work. Since I can't repo this I am relying on you to test it for me.

I've tested it and it works! That's definitely a much cleaner solution than sleeping. I had tried awaitStop() previously but my problem was that I had put it in place of stop() rather than after it.


Fix typo in docs/usage/exceptions.md

Fix minor typo in the documentation docs/usage/exceptions.md

Merged

Merged


Add workspaceText config option

Closes https://github.com/LeonardSSH/coc-discord-rpc/issues/12

Awesome.

@all-contributors please add @Vendicated for code


Enable BytesWarning during test and fix discovered case

The Python command line argument -b causes Python to emit a warning when
bytes and str usage is mixed. This is generally considered bad practice
as either one or the other is required. Enabling this feature during
tests helps catch them before reaching production.
The warning appeared as:
tests/test_scripting.py::TestScripting::test_eval_msgpack_pipeline_error_in_lua
  .../redis-py/redis/client.py:3967: BytesWarning: str() on a bytes instance
    cmd = ' '.join(imap(safe_unicode, command))

Pull Request check-list
Please make sure to review and check all of these items:

[x] Does $ tox pass with this change (including linting)?
[x] Does travis tests pass with this change (enable it first in your forked repo and wait for the travis build to finish)?
[x] Is the new or changed code fully tested?
[ ] Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?

looks good to me. thanks!


website not working

Thanks for spotting! - it looks like something is not going well in our deploy pipeline. Rolled back to a prior version of the site int he mean time.


Ability to access the entire data of the currently active textResult node (title, tags, raw body)

I was wondering if it would be even more beneficial to allow bondage.js to return the entire contents of the current node, rather than simply the text or the command or whatnot - as an object.
It would be much better to also get all the tags and even the title of the node, as well as its type.
So this is more of a request to get it to return the entire contents, rather than changing the parser/lexer.
That would add another level of flexibility to how it can be integrated in other projects

Anyways, here goes another pull :)
https://github.com/jhayley/bondage.js/pull/39
I noticed that you already have tags in this.yarnNodes[node.title], but you dont yield that. Also they get returned as a string, but should be an array

Just as example use - this data getting returned with textresults would allow to use the titles and tags to trigger things to happen in the game:

change of avatars
triggering of animations in cut scenes
trigger of sound effects and other events
storing of variables from visited text nodes

Apart of being used in a game, this can also help me with adding more data to Yarn's electron version story tester

closed via https://github.com/jhayley/bondage.js/pull/39


Article request: Streaming 103

#3 可以做一个开始，其实就是讲讲关键的概念及其实现。还有近年来我对 streaming 认识的变化。
你觉得 103 应该有啥呢?

closed as unclear


Optimization: removed level of indirection for SubRuleContext.

Description of changes:
Instead of Double id use SubRuleContext directly everywhere. SubRuleContext id is only used internally for backward compatibility in equals and hashCode (not sure if this is actually needed, but tests check that). The benchmarks are not stable (need JMH with forking, proper warmup and multiple longer iterations for individual benchmarks), but overall can see 3-8% in throughput improvement on some benchmarks with this easy fix.
Using long as id instead of double because long is typically faster and actually can produce more unique 64-bit patterns than double because for long every 64-bit pattern is valid while for double it is not true (NaN).
Changed only types but did not rename variables all over the place because it would be harder to review, can be done later as needed.
Benchmark / Performance (for source code changes):
Read 213068 events
Finding Rules...
Lots: 10000
Lots: 20000
Lots: 30000
Lots: 40000
Lots: 50000
Lots: 60000
Lots: 70000
Lots: 80000
Lots: 90000
Lots: 100000
Lots: 110000
Lots: 120000
Lots: 130000
Lots: 140000
Lots: 150000
Lots: 160000
Lots: 170000
Lots: 180000
Lots: 190000
Lots: 200000
Lots: 210000
Lines: 213068, Msec: 11459
Events/sec: 18593.9
 Rules/sec: 130157.6
Reading citylots2
Read 213068 events
EXACT events/sec: 274218.8
WILDCARD events/sec: 176526.9
PREFIX events/sec: 271078.9
PREFIX_EQUALS_IGNORE_CASE_RULES events/sec: 272465.5
SUFFIX events/sec: 260474.3
SUFFIX_EQUALS_IGNORE_CASE_RULES events/sec: 266002.5
EQUALS_IGNORE_CASE events/sec: 233883.6
NUMERIC events/sec: 145937.0
ANYTHING-BUT events/sec: 129919.5
ANYTHING-BUT-IGNORE-CASE events/sec: 142902.7
ANYTHING-BUT-PREFIX events/sec: 153507.2
ANYTHING-BUT-SUFFIX events/sec: 150365.6
ANYTHING-BUT-WILDCARD events/sec: 160322.0
COMPLEX_ARRAYS events/sec: 32078.9
PARTIAL_COMBO events/sec: 49037.5
COMBO events/sec: 20434.3

Overall the becnhmark results are not conclusive across diffrent jvm versions: the same benchmark can be noticeably better or worse, from what I see from main branch history they are quite flaky. Will try to improve benhcmarks in a separate PR.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

hey @svladykin  is this ready to review? its' been in draft state which has me confused on the state of the PR.

@baldawar I did not like the Benchmark numbers for this PR, thus submitted a second one with JMH benchmarks, assuming that I will be able to get more meaningful perfromance numbers here when the second PR is merged. This is why it is still in draft status.

Rebased on top of main to check JMH throughput numbers.

odd, perf tests are a bit of mixed bag when compared to https://github.com/aws/event-ruler/actions/runs/10805542552/job/30018080464 . Is it because of most of the time is spent in other parts of the code (my recent tests showed ByteMachine.getTransitionOn and code tied to exists matcher to be quite expensive)

Yes, I don't see any throughput improvement as well. We can either consider this patch to be just a code cleanup or just close it and stop spending time on it. I'm good with either option.
I also noticed that json parsing takes around 60% of the time for simple rules, which makes me think that using custom binary format instead of json could help a lot.

FWIW, Quamina has a custom hand crafted JSON parser for events, and the benefits of that were huge. But didn't bother for rule parsing.

useful bookmark for the custom JSON parser :

https://www.tbray.org/ongoing/When/202x/2022/06/10/Quamina-Optimizing
https://github.com/timbray/quamina/blob/main/flatten_json.go

For this change, I think its worth merging if we can add a test that shows SubRuleContext is now faster. Benchmark would be over kill but unit test would be good enough with bunch of for-loops similar to ComparableNumberTest.java is good enough IMO.

@svladykin let me know if you're still working on this.

Yes, was a bit busy lately, will catch up this week.

Added a simple performance test which takes ~2500ms on the old code and ~1400ms on the new code.

Thanks @svladykin


request ID not available in middleware context

request ID needs to be present in middleware logger string
probably related to #13

opened up pull request that adds this feature #15

Bump, can any one look at this pull request?

Merged.


Create Post “what-do-termites-look-like”

Automatically generated by Netlify CMS

👷 Deploy Preview for chic-shortbread-66a66b processing.

Name
Link

🔨 Latest commit
a6535c41cb2ca8f9701fc22b4f420617a80c4197

🔍 Latest deploy log
https://app.netlify.com/sites/chic-shortbread-66a66b/deploys/654ef4bd0ef1ea0008bb212e


Window.fetch on macOS

Hello, on macOS we have an error:

Failed to fetch.

ups... my error
sorry


⚠ SceneTime has degraded performance

In 57fa1a9, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 1445 ms

Resolved: SceneTime performance has improved in 916134d after 6 minutes.


serkan-ozal's 3rd submission with some minor improvements

faster merge by ignoring empty entries in the map
enable CDS for faster startup (added prepare_serkan-ozal.sh to generate CDS archive in advance)
some tweaks with JVM options
optimized result printing

Check List:

[x] Tests pass (./test.sh <username> shows no differences between expected and actual outputs)
[x] All formatting changes by the build are committed
[x] Your launch script is named calculate_average_<username>.sh (make sure to match casing of your GH user name) and is executable
[x] Output matches that of calculate_average_baseline.sh
[x] For new entries, or after substantial changes: When implementing custom hash structures, please point to where you deal with hash collisions (line number): 604

Execution time: 3.4s (It was 3.6s in the 2nd submission in my testing env)
Execution time of reference implementation: 5m 54.1s

Hi @gunnarmorling,
I have sent my 3rd submission. Even though I don't see performance improvement in my testing env, it still might perform better in the eval machine as there are some major changes. Let's see how it will be in the eval machine.
Thanks in advance.

Yepp, slower indeed:
Benchmark 1: timeout -v 300 ./calculate_average_serkan-ozal.sh 2>&1
  Time (mean ± σ):      2.648 s ±  0.041 s    [User: 0.003 s, System: 0.006 s]
  Range (min … max):    2.605 s …  2.710 s    5 runs

Summary
  serkan-ozal: trimmed mean 2.6423087684, raw times 2.6048885754,2.7095721204000003,2.6545993044,2.6181676104,2.6541593904000003

Leaderboard
grep: ./src/main/java*/dev/morling/onebrc/CalculateAverage_serkan-ozal.java: No such file or directory

| # | Result (m:s.ms) | Implementation     | JDK | Submitter     | Notes     |
|---|-----------------|--------------------|-----|---------------|-----------|
|   | 00:02.642 | [link](https://github.com/gunnarmorling/1brc/blob/main/src/main/java/dev/morling/onebrc/CalculateAverage_serkan-ozal.java)| 21.0.1-open | [Serkan ÖZAL](https://github.com/serkan-ozal) |  |

Hi @gunnarmorling,
I have updated the optimizations:

faster merge by ignoring empty entries in the map
enable CDS for faster startup (added prepare_serkan-ozal.sh to generate CDS archive in advance)
some tweaks with JVM options

In my testing environment, it has improved the performance by 200ms. Will see how much it will be in the eval machine.
Could you please rerun it?

Yepp, makes for small improvement: 00:02.440 now.


Implement Client-side Handling for Error Responses in Append Transaction Executions

Why
Currently, the client-side implementation does not adequately handle various error responses from the AppendTransactionExecutionsResponse. This can lead to incomplete or failed operations without proper feedback to the user. Implementing comprehensive error handling on the client side will ensure better user experience and robustness of the client application.

We've shelved raft for now, closing.


React 18 not support material ui styles

makestyles unable to use in react 18. need to check

Material UI we can use styled from @mui/system alternative solution


Documentation

We need a spec for documentation comments.
@ark-lang/owners

Could just go with the typical /** */ or rust style ///

That's not really what I'm asking, I mean more of how we're going to label things. (like do we use @param and stuff, how will that work)

Oh, I see. I'm not too sure..

Ok, I've started working on this.
First, I wrote up some draft docs: https://github.com/ark-lang/ark-docs/commit/e5df9a1c586a0ed5f3c19e2fe9d3a6f831cf2240#diff-eb120e74eba90da6132666ceb0405bf9R598
Now I just finished starting the doc generation in the compiler: https://github.com/ark-lang/ark/commit/f33e3974492a85da1e4566a403aeb982ecbbabcc
Run ark tests/big.ark --docgen to try it out, then check the docgen/ directory. Right now it's only bare-bones, because modules need to be implemented before work can carry on.

We should pick one: /** */ and ///.

:+1:  for both /** */ and ///

@raoulvdberge I mean we should only use /** */ or just ///... unless you know that and you want both? I dunno, i put and instead of or in my comment so I thought it may have confused you

@felixangell I'm not sure if only supporting /// is a good idea, usually, documentation can be quite long and typing for every line /// is kinda meh.

@raoulvdberge That's a fair point

@felixangell +1 on styles please
Also, /** */ and /// are both needed. /// for quick descriptions above vars etc, and /** */ for big descriptions about how to use a module (eg https://golang.org/pkg/text/template/)

Sure


Read-only transaction leads to "too much contention error"

In my project I have implemented a get or create function which uses a transaction to first select the document and if it exists returns it right away and otherwise creates it.
However in the case of just returning the document the transaction seems to leave some kind of lock on the document because when I try to delete if afterwards I get the following error message:
thread 'main' panicked at src/main.rs:58:10:
called `Result::unwrap()` on an `Err` value: DatabaseError(FirestoreDatabaseError { public: FirestoreErrorPublicGenericDetails { code: "Aborted" }, details: "status: Aborted, message: \"Too much contention on these documents. Please try again.\", details: [], metadata: MetadataMap { headers: {\"content-type\": \"application/grpc\", \"date\": \"Thu, 23 Nov 2023 09:46:09 GMT\", \"alt-svc\": \"h3=\\\":443\\\"; ma=2592000,h3-29=\\\":443\\\"; ma=2592000\"} }", retry_possible: true })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I've created a minimal example to showcase the problem (and the error message above is also created by this example):
use firestore::FirestoreDb;
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Clone, Debug)]
#[serde(rename_all = "camelCase")]
pub struct TestStruct {
    pub prop: String,
}

const COLLECTION_NAME: &str = "test_col";

#[tokio::main]
async fn main() {
    let db = FirestoreDb::new("gcp-project-id").await.unwrap();
    let doc_id = "testid";

    db.fluent()
        .insert()
        .into(COLLECTION_NAME)
        .document_id(doc_id)
        .object(&TestStruct {
            prop: "test".to_owned(),
        })
        .execute::<TestStruct>()
        .await
        .unwrap();

    db.run_transaction(|db, _transaction| {
        Box::pin(async move {
            let test: Option<TestStruct> = db
                .fluent()
                .select()
                .by_id_in(COLLECTION_NAME)
                .obj()
                .one(doc_id)
                .await?;

            // uncomment to make the error go away
            // db.fluent()
            //     .update()
            //     .in_col(COLLECTION_NAME)
            //     .document_id(doc_id)
            //     .object(&test)
            //     .add_to_transaction(_transaction)?;

            Ok(test)
        })
    })
    .await
    .unwrap();

    db.fluent()
        .delete()
        .from(COLLECTION_NAME)
        .document_id(doc_id)
        .execute()
        .await
        .unwrap();
}

When a write happens the problem vanishes.
Thank you for your consideration.

Hey, thanks for the detailed bug report.
The fix is baked: https://github.com/abdolence/firestore-rs/pull/141

Released in https://github.com/abdolence/firestore-rs/releases/tag/v0.37.3

Thank you very much for the fix and the crazy fast response :)


Engineering electrical assessment career but sustainability

URL
https://dotnet.microsoft.com/en-us/learn/aspnet/blazor-tutorial/run
Operating system
linux
More information about your system
Engineering
Description
electrical assessment career but sustainability
dotnet --info
Engineering electrical assessment career but sustainability

Engineering electrical assessment career but sustainability

Engineering electrical assessment career but sustainability

Seems unrelated to the topic at hand. Closing.

Hello good like


add 3 text svgs (#82)

This reverts commit e44c8fda1c80d6c519d59fdbc99b06e536d5a09d.

Build failed because of:
No cached entry, and OpenEMS has issues with logo: openems-text.svg, ENOENT: no such file or directory, open '/opt/build/repo/hosted_logos/openems-text.svg'
No cached entry, and GridLAB-D has issues with logo: gridlab-d-text.svg, ENOENT: no such file or directory, open '/opt/build/repo/hosted_logos/gridlab-d-text.svg'
No cached entry, and The Energy Web Chain has issues with logo: the-energy-web-chain-text.svg, ENOENT: no such file or directory, open '/opt/build/repo/hosted_logos/the-energy-web-chain-text.svg'