Last.fm -> Maloja stuck processing recent

I just wanted to thank you for this project, it's been incredibly useful!
Describe the bug
I noticed some behavior recently and I am not sure if it's due to a misconfiguration on my part or a bug. I have multi-scrobbler set up to scrobble from last.fm -> Maloja. Recently, the multi-scrobbler queue will get stuck where Queued Scrobbles continues to go up while Tracks Scrobbled and Failed Scrobbles never increase.
From the logs, it seems like backlogged or recently played tracks from the source (last.fm), which have already been scrobbled, are failing to scrobble to Maloja as they are duplicates. My understanding is that would be expected behavior (I wouldn't want it to actually scrobble the duplicate). However, they appear to be causing the queue to get stuck instead of being ignored or showing up as Failed Scrobbles.
If I delete the recent tracks from Maloja and restart multi-scrobbler, it successfully scrobbles them again and the queue continues to be processed from that point. However, as soon as multi-scrobbler is restarted it grabs a few recent tracks from last.fm, which have obviously already been scrobbled, and immediately gets stuck again.

Expected behavior
I would expect the previously scrobbled tracks to either be ignored (not attempted to scrobble them at all), or the duplicate error from Maloja would be handled and the track ignored.
Logs
Below are the logs for one of these events at startup. I included the maloja error as well (I have them in same systemd service)
multi-scrobbler-1  | [2024-08-25 19:10:23.433 -0700] DEBUG  : [App] [Scrobblers] [Maloja - Maloja] <artist> - <track> @ 2024-08-23T17:32:01-07:00 (S) => No Match because no recent scrobbles returned from API
maloja-1           | [database] Incoming scrobble [Client: multi-scrobbler | API: native/v1]: {'track_artists': ['<artist>'], 'track_title': '<track>', 'album_title': '<album>', 'scrobble_time':<PHONE_NUMBER>}
maloja-1           | [sqldb] 1 Scrobbles have not been written to database (already exist)
maloja-1           | Traceback (most recent call last):
maloja-1           |   File "/venv/lib/python3.11/site-packages/maloja/apis/native_v1.py", line 126, in protector
maloja-1           |     return func(*args,**kwargs)
maloja-1           |            ^^^^^^^^^^^^^^^^^^^^
maloja-1           |   File "/venv/lib/python3.11/site-packages/maloja/apis/native_v1.py", line 631, in post_scrobble
maloja-1           |     result = database.incoming_scrobble(
maloja-1           |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
maloja-1           |   File "/venv/lib/python3.11/site-packages/maloja/database/__init__.py", line 115, in incoming_scrobble
maloja-1           |     sqldb.add_scrobble(scrobbledict,update_album=albumupdate,dbconn=dbconn)
maloja-1           |   File "/venv/lib/python3.11/site-packages/maloja/database/sqldb.py", line 159, in wrapper
maloja-1           |     return func(*args,**kwargs)
maloja-1           |            ^^^^^^^^^^^^^^^^^^^^
maloja-1           |   File "/venv/lib/python3.11/site-packages/maloja/database/sqldb.py", line 364, in add_scrobble
maloja-1           |     raise exc.DuplicateScrobble(scrobble=scrobbledict)
maloja-1           | maloja.database.exceptions.DuplicateScrobble
maloja-1           |
multi-scrobbler-1  | [2024-08-25 19:10:23.594 -0700] WARN   : [App] [Scrobblers] [Maloja - Maloja] Maloja Warning: This scrobble exists in the database (same timestamp and track). The submitted scrobble was not added. => null
multi-scrobbler-1  | [2024-08-25 19:10:23.596 -0700] ERROR  : [App] [Scrobblers] [Maloja - Maloja] Scrobble Error (Backlog)
multi-scrobbler-1  | [2024-08-25 19:10:23.598 -0700] DEBUG  : [App] [Scrobblers] [Maloja - Maloja] Raw Payload:
multi-scrobbler-1  | [2024-08-25 19:10:23.598 -0700] ERROR  : [App] [Scrobblers] [Maloja - Maloja] Scrobble processing interrupted
multi-scrobbler-1  | [2024-08-25 19:10:23.599 -0700] ERROR  : [App] [Scrobblers] [Maloja - Maloja] Error occurred while trying to scrobble
multi-scrobbler-1  | Error: Error occurred while trying to scrobble
multi-scrobbler-1  |     at MalojaScrobbler.doProcessing (CWD/src/backend/scrobblers/AbstractScrobbleClient.ts:598:39)
multi-scrobbler-1  |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
multi-scrobbler-1  |     at MalojaScrobbler.startScrobbling (CWD/src/backend/scrobblers/AbstractScrobbleClient.ts:515:27)
multi-scrobbler-1  | caused by: TypeError: Cannot read properties of undefined (reading 'reduce')
multi-scrobbler-1  |     at Function.formatPlayObj (CWD/src/backend/scrobblers/MalojaScrobbler.ts:99:39)
multi-scrobbler-1  |     at MalojaScrobbler.formatPlayObj (CWD/src/backend/scrobblers/MalojaScrobbler.ts:128:90)
multi-scrobbler-1  |     at MalojaScrobbler.doScrobble (CWD/src/backend/scrobblers/MalojaScrobbler.ts:465:38)
multi-scrobbler-1  |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
multi-scrobbler-1  |     at MalojaScrobbler.doProcessing (CWD/src/backend/scrobblers/AbstractScrobbleClient.ts:589:51)
multi-scrobbler-1  |     at MalojaScrobbler.startScrobbling (CWD/src/backend/scrobblers/AbstractScrobbleClient.ts:515:27)
multi-scrobbler-1  | [2024-08-25 19:10:23.611 -0700] INFO   : [App] [Scrobblers] [Maloja - Maloja] Scrobble processing retries (0) less than max processing retries (5), restarting processing after 3 second delay...

One other interesting part of the logs I noticed was this:
[Scrobblers] [Maloja - Maloja] <artist> - <track> @ 2024-08-25T11:51:08-07:00 (S) => No Match because no recent scrobbles returned from API
Is there a chance this is caused by failing to get the list of recently scrobbled tracks from the Maloja api? The API appears to be working when querying it manually, at least.
I can include more logs, if that would be helpful.
Versions (please complete the following information):

multi-scrobbler: foxxmd/multi-scrobbler:latest docker image (master-4fdd346)
maloja: 3.2.2

Thanks for the detailed writeup and logs.
This is partially an issue with Maloja and partially an issue with MS.
When Maloja 2.3.2 (current version) detects a duplicate the response is returned as "OK" with a "warning" for duplicate but the actual response data does not contain a scrobble. This should be an error since the scrobble was not successful and has since been fixed in Maloja but the fix has not been published to a new release yet. MS handles this scenario correctly when using Maloja with the fix but that's not useful for anyone using official releases, of course. I'm not sure if/when the fix was get published so I'll go ahead and compensate for that now.
There's an also issue where MS's parsing of Maloja scrobbles is not defensive enough (caused by the issue above) and that is causing the actual error Cannot read properties of undefined (reading 'reduce') which causes the scrobble process to fail and get stuck rather than push scrobbles to failed queue. I'll address that as well.

No Match because no recent scrobbles returned from API
Is there a chance this is caused by failing to get the list of recently scrobbled tracks from the Maloja api?

This is interesting...did this message appear when first starting up multi-scrobbler or after it was running for a bit? Additionally, when first starting Maloja do you see something like this in logs:
[2024-08-26 11:17:48.949 -0400] DEBUG  : [App] [Scrobblers] [Maloja - default] Attempting to initialize...
[2024-08-26 11:17:48.950 -0400] VERBOSE: [App] [Scrobblers] [Maloja - default] Building required data init succeeded
[2024-08-26 11:17:48.963 -0400] INFO   : [App] [Scrobblers] [Maloja - default] Communication test succeeded.
[2024-08-26 11:17:48.963 -0400] INFO   : [App] [Scrobblers] [Maloja - default] Maloja Server Version: 3.2.2
[2024-08-26 11:17:48.965 -0400] VERBOSE: [App] [Scrobblers] [Maloja - default] Connection check succeeded
[2024-08-26 11:17:48.967 -0400] INFO   : [App] [Scrobblers] [Maloja - default] Auth test passed!
[2024-08-26 11:17:48.968 -0400] INFO   : [App] [Scrobblers] [Maloja - default] Fully Initialized!
[2024-08-26 11:17:48.968 -0400] VERBOSE: [App] [Scrobblers] [Maloja - default] Fetching up to 100 initial scrobbles...
[2024-08-26 11:17:48.968 -0400] DEBUG  : [App] [Scrobblers] [Maloja - default] Refreshing recent scrobbles
[2024-08-26 11:17:48.981 -0400] DEBUG  : [App] [Scrobblers] [Maloja - default] Found 100 recent scrobbles
[2024-08-26 11:17:48.982 -0400] INFO   : [App] [Scrobblers] [Maloja - default] Scrobble processing started

Specifically the Found 100 recent scrobbles line?

Please try using foxxmd/multi-scrobbler:pr-181 docker image to see if this fixes duplicate error problems.

When Maloja 2.3.2 (current version) detects a duplicate the response is returned as "OK" with a "warning" for duplicate but the actual response data does not contain a scrobble. This should be an error since the scrobble was not successful and has since been fixed in Maloja but the fix has not been published to a new release yet. MS handles this scenario correctly when using Maloja with the fix but that's not useful for anyone using official releases, of course. I'm not sure if/when the fix will get published so I'll go ahead and compensate for that now.

Awesome, thanks for taking a look!
Sorry, it looks like I actually misspoke when noting the version of Maloja. That is the reported version on the web-ui, but I had forgotten that I was actually using a locally built image from the latest master (I presume that would include the change you mentioned).

This is interesting...did this message appear when first starting up multi-scrobbler or after it was running for a bit? Additionally, when first starting Maloja do you see something like this in logs:

It appears when first starting up. After the logs you mentioned, directly after the last.fm source discovers recently played tracks. Here are a bit more of the logs starting from startup to the error.
multi-scrobbler-1  | [2024-08-26 13:04:00.939 -0700] DEBUG  : [App] [Scrobblers] Constructing maloja (Maloja) client...
multi-scrobbler-1  | [2024-08-26 13:04:00.940 -0700] WARN   : [App] [Scrobblers] [Maloja - Maloja] Setting verbose matching may produce noisy logs! Use with care.
multi-scrobbler-1  | [2024-08-26 13:04:00.940 -0700] INFO   : [App] [Scrobblers] [Maloja - Maloja] Client Added
multi-scrobbler-1  | [2024-08-26 13:04:00.941 -0700] INFO   : [App] Starting scrobble clients...
multi-scrobbler-1  | [2024-08-26 13:04:00.942 -0700] DEBUG  : [App] [Scrobblers] [Maloja - Maloja] Attempting to initialize...
multi-scrobbler-1  | [2024-08-26 13:04:00.942 -0700] VERBOSE: [App] [Scrobblers] [Maloja - Maloja] Building required data init succeeded
multi-scrobbler-1  | [2024-08-26 13:04:00.952 -0700] INFO   : [App] [Scrobblers] [Maloja - Maloja] Communication test succeeded.
multi-scrobbler-1  | [2024-08-26 13:04:00.952 -0700] INFO   : [App] [Scrobblers] [Maloja - Maloja] Maloja Server Version: 3.2.2
multi-scrobbler-1  | [2024-08-26 13:04:00.956 -0700] VERBOSE: [App] [Scrobblers] [Maloja - Maloja] Connection check succeeded
multi-scrobbler-1  | [2024-08-26 13:04:00.959 -0700] INFO   : [App] [Scrobblers] [Maloja - Maloja] Auth test passed!
multi-scrobbler-1  | [2024-08-26 13:04:00.959 -0700] INFO   : [App] [Scrobblers] [Maloja - Maloja] Fully Initialized!
multi-scrobbler-1  | [2024-08-26 13:04:00.960 -0700] VERBOSE: [App] [Scrobblers] [Maloja - Maloja] Fetching up to 100 initial scrobbles...
multi-scrobbler-1  | [2024-08-26 13:04:00.960 -0700] DEBUG  : [App] [Scrobblers] [Maloja - Maloja] Refreshing recent scrobbles
multi-scrobbler-1  | [2024-08-26 13:04:00.979 -0700] DEBUG  : [App] [Scrobblers] [Maloja - Maloja] Found 100 recent scrobbles
multi-scrobbler-1  | [2024-08-26 13:04:00.979 -0700] INFO   : [App] [Scrobblers] [Maloja - Maloja] Scrobble processing started
multi-scrobbler-1  | [2024-08-26 13:04:01.058 -0700] DEBUG  : [App] [Sources] (last.fm - Source) Constructing lastfm source
multi-scrobbler-1  | [2024-08-26 13:04:01.060 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Note: The player for this source is an analogue for the 'Now Playing' status exposed by lastfm which is NOT used for scrobbling. Instead, the 'recently played' or 'history' information provided by this source is used for scrobbles.
multi-scrobbler-1  | [2024-08-26 13:04:01.061 -0700] DEBUG  : [App] [Sources] [Lastfm - last.fm - Source] Attempting to initialize...
multi-scrobbler-1  | [2024-08-26 13:04:01.063 -0700] VERBOSE: [App] [Sources] [Lastfm - last.fm - Source] Building required data init succeeded
multi-scrobbler-1  | [2024-08-26 13:04:01.133 -0700] VERBOSE: [App] [Sources] [Lastfm - last.fm - Source] Connection check succeeded
multi-scrobbler-1  | [2024-08-26 13:04:01.531 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] [API - Lastfm - last.fm - Source] Client authorized for user <user>
multi-scrobbler-1  | [2024-08-26 13:04:01.534 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Fully Initialized!
multi-scrobbler-1  | [2024-08-26 13:04:03.038 -0700] INFO   : [App] Scheduler started.
multi-scrobbler-1  | [2024-08-26 13:04:03.040 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovering backlogged tracks from recently played API...
multi-scrobbler-1  | [2024-08-26 13:04:03.042 -0700] VERBOSE: [App] [Sources] [Lastfm - last.fm - Source] Fetching the last 10 listens to check for backlogging...
multi-scrobbler-1  | [2024-08-26 13:04:03.216 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 1> @ 2024-08-25T18:58:10-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.217 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 2> @ 2024-08-25T19:01:17-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.217 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 3> @ 2024-08-25T19:07:54-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.217 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 4> @ 2024-08-25T19:10:13-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.218 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 5> @ 2024-08-25T19:12:49-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.218 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 5> @ 2024-08-25T19:15:05-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.219 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 6> @ 2024-08-25T19:26:37-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.219 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 7> @ 2024-08-25T19:30:42-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.220 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 8> @ 2024-08-25T19:34:45-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.221 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 9> @ 2024-08-25T19:38:29-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.221 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Backlog scrobbling is disabled by config, skipping...
multi-scrobbler-1  | [2024-08-26 13:04:03.222 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Polling started
multi-scrobbler-1  | [2024-08-26 13:04:03.222 -0700] DEBUG  : [App] [Sources] [Lastfm - last.fm - Source] Refreshing recently played
multi-scrobbler-1  | [2024-08-26 13:04:03.442 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 10> @ 2024-08-25T12:01:14-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.445 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 11> @ 2024-08-25T12:04:26-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.448 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 11> @ 2024-08-25T12:09:05-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.451 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 12> @ 2024-08-25T14:57:28-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.451 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 13> @ 2024-08-25T15:02:54-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.452 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 14> @ 2024-08-25T15:07:43-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.453 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 15> @ 2024-08-25T15:14:16-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.454 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 16> @ 2024-08-25T18:51:21-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.455 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 17> @ 2024-08-25T18:53:06-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.455 -0700] INFO   : [App] [Sources] [Lastfm - last.fm - Source] Discovered => <artist> - <track 18> @ 2024-08-25T18:56:15-07:00 (S)
multi-scrobbler-1  | [2024-08-26 13:04:03.463 -0700] DEBUG  : [App] [Sources] [Lastfm - last.fm - Source] Last activity was at 2024-08-26T13:04:03-07:00 | Next check interval: 15.00s
multi-scrobbler-1  | [2024-08-26 13:04:04.984 -0700] DEBUG  : [App] [Scrobblers] [Maloja - Maloja] Scrobble refresh not needed
multi-scrobbler-1  | [2024-08-26 13:04:04.987 -0700] DEBUG  : [App] [Scrobblers] [Maloja - Maloja] <artist> - <track 10> @ 2024-08-25T12:01:14-07:00 (S) => No Match because no recent scrobbles returned from API
maloja-1           | [database] Incoming scrobble [Client: multi-scrobbler | API: native/v1]: {'track_artists': ['<artist>'], 'track_title': '<track 10>', 'album_title': '<album>', 'scrobble_time':<PHONE_NUMBER>}
maloja-1           | [sqldb] 1 Scrobbles have not been written to database (already exist)
maloja-1           | Traceback (most recent call last):
maloja-1           |   File "/venv/lib/python3.11/site-packages/maloja/apis/native_v1.py", line 126, in protector
maloja-1           |     return func(*args,**kwargs)
maloja-1           |            ^^^^^^^^^^^^^^^^^^^^
maloja-1           |   File "/venv/lib/python3.11/site-packages/maloja/apis/native_v1.py", line 631, in post_scrobble
maloja-1           |     result = database.incoming_scrobble(
maloja-1           |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
maloja-1           |   File "/venv/lib/python3.11/site-packages/maloja/database/__init__.py", line 115, in incoming_scrobble
maloja-1           |     sqldb.add_scrobble(scrobbledict,update_album=albumupdate,dbconn=dbconn)
maloja-1           |   File "/venv/lib/python3.11/site-packages/maloja/database/sqldb.py", line 159, in wrapper
maloja-1           |     return func(*args,**kwargs)
maloja-1           |            ^^^^^^^^^^^^^^^^^^^^
maloja-1           |   File "/venv/lib/python3.11/site-packages/maloja/database/sqldb.py", line 364, in add_scrobble
maloja-1           |     raise exc.DuplicateScrobble(scrobble=scrobbledict)
maloja-1           | maloja.database.exceptions.DuplicateScrobble
maloja-1           |
multi-scrobbler-1  | [2024-08-26 13:04:05.220 -0700] WARN   : [App] [Scrobblers] [Maloja - Maloja] Maloja Warning: This scrobble exists in the database (same timestamp and track). The submitted scrobble was not added. => null
multi-scrobbler-1  | [2024-08-26 13:04:05.220 -0700] ERROR  : [App] [Scrobblers] [Maloja - Maloja] Scrobble Error (Backlog)
multi-scrobbler-1  | [2024-08-26 13:04:05.221 -0700] DEBUG  : [App] [Scrobblers] [Maloja - Maloja] Raw Payload:
multi-scrobbler-1  | [2024-08-26 13:04:05.221 -0700] ERROR  : [App] [Scrobblers] [Maloja - Maloja] Scrobble processing interrupted
multi-scrobbler-1  | [2024-08-26 13:04:05.222 -0700] ERROR  : [App] [Scrobblers] [Maloja - Maloja] Error occurred while trying to scrobble
multi-scrobbler-1  | Error: Error occurred while trying to scrobble
multi-scrobbler-1  |     at MalojaScrobbler.doProcessing (CWD/src/backend/scrobblers/AbstractScrobbleClient.ts:598:39)
multi-scrobbler-1  |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
multi-scrobbler-1  |     at MalojaScrobbler.startScrobbling (CWD/src/backend/scrobblers/AbstractScrobbleClient.ts:515:27)
multi-scrobbler-1  | caused by: TypeError: Cannot read properties of undefined (reading 'reduce')
multi-scrobbler-1  |     at Function.formatPlayObj (CWD/src/backend/scrobblers/MalojaScrobbler.ts:99:39)
multi-scrobbler-1  |     at MalojaScrobbler.formatPlayObj (CWD/src/backend/scrobblers/MalojaScrobbler.ts:128:90)
multi-scrobbler-1  |     at MalojaScrobbler.doScrobble (CWD/src/backend/scrobblers/MalojaScrobbler.ts:465:38)
multi-scrobbler-1  |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
multi-scrobbler-1  |     at MalojaScrobbler.doProcessing (CWD/src/backend/scrobblers/AbstractScrobbleClient.ts:589:51)
multi-scrobbler-1  |     at MalojaScrobbler.startScrobbling (CWD/src/backend/scrobblers/AbstractScrobbleClient.ts:515:27)
multi-scrobbler-1  | [2024-08-26 13:04:05.227 -0700] INFO   : [App] [Scrobblers] [Maloja - Maloja] Scrobble processing retries (0) less than max processing retries (5), restarting processing after 3 second delay...
multi-scrobbler-1  | [2024-08-26 13:04:08.227 -0700] INFO   : [App] [Scrobblers] [Maloja - Maloja] Scrobble processing started
multi-scrobbler-1  | [2024-08-26 13:04:08.227 -0700] DEBUG  : [App] [Scrobblers] [Maloja - Maloja] Scrobble refresh not needed

Please try using foxxmd/multi-scrobbler:pr-181 docker image to see if this fixes duplicate error problems.

I just tried this out and it appears to fix the queue getting stuck!
multi-scrobbler-1  | caused by: Error: Maloja returned a warning but MS treating as error: (scrobble_exists) This scrobble exists in the database (same timestamp and track). The submitted scrobble was not added.
It looks like it is properly marking them as failed now and the queue continues to progress.
However, now every time it starts up it attempts to process and fails these 200 recent duplicate tracks every time. It's way less of an issue since it doesn't block it from processing any longer, but I am wondering if I have something misconfigured to cause this.

Then again, after restarting it processes 200 tracks and fails them all (but then continues successfully)

@doot please update to the newest image for foxxmd/multi-scrobbler:pr-181 and let me know if that fixes everything for you.

Awesome! Just tried that out and it looks like it's fixed all of the issues I was seeing! The queue is no longer getting stuck and it's not failing to scrobble already scrobbled tracks at startup.

Fixed in v0.8.3


AssertAsync does not escalate Exceptions outside of async context

Found in ".\vendor\tc39\test262\test\language\statements\async-function\dflt-params-trailing-comma.js"

Done in 7e7410f9ca1b22ba6d8ac4f9197dc5b4b2b7e7cf


Adds from_env method and change _store to be a DynaBox

Ref #218
Switching working environments
You can switch between existing environments using:

from_env: (recommended) Will create a new settings instance pointing to defined env.
setenv: Will set the existing instance to defined env.
using_env: Context manager that will have defined env only inside its scope.

from_env

New in 2.0.5

Return a new isolated settings object pointing to specified env.
Example of settings.toml::
[development]
message = 'This is in dev'
foo = 1
[other]
message = 'this is in other env'
bar = 2

Program::
>>> from dynaconf import settings
>>> print(settings.MESSAGE)
'This is in dev'
>>> print(settings.FOO)
1
>>> print(settings.BAR)
AttributeError: settings object has no attribute 'BAR'

Then you can use from_env:
>>> print(settings.from_env('other').MESSAGE)
'This is in other env'
>>> print(settings.from_env('other').BAR)
2
>>> print(settings.from_env('other').FOO)
AttributeError: settings object has no attribute 'FOO'

The existing settings object remains the same.
>>> print(settings.MESSAGE)
'This is in dev'

You can assign new settings objects to different envs like:
development_settings = settings.from_env('development')
other_settings = settings.from_env('other')

And you can choose if the variables from different envs will be chained and overridden in a sequence:
all_settings = settings.from_env('development', keep=True).from_env('other', keep=True)

>>> print(all_settings.MESSAGE)
'This is in other env'
>>> print(all_settings.FOO)
1
>>> print(all_settings.BAR)
2

The variables from [development] are loaded keeping pre-loaded values, then the variables from [other] are loaded keeping pre-loaded from [development] and overriding it.
setenv
Will change in_place the env for the existing object.
from dynaconf import settings

settings.setenv('other')
# now values comes from [other] section of config
assert settings.MESSAGE == 'This is in other env'

settings.setenv()
# now working env are back to previous

using_env
Using context manager
from dynaconf import settings

with settings.using_env('other'):
    # now values comes from [other] section of config
    assert settings.MESSAGE == 'This is in other env'

# existing settings back to normal after the context manager scope
assert settings.MESSAGE == 'This is in dev'

@austinnichols101

@kedark3 I guess this from_env will be useful for your use case.


Typo "@source_object"→"@source_object"

bold with escape characters
https://docs.microsoft.com/en-us/sql/relational-databases/system-stored-procedures/sp-addmergearticle-transact-sql?view=sql-server-2017

@hyoshioka0128 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Verified the update.
@hyoshioka0128 Thanks for making this fix.


Migrate newControlDefaults-manual from nimble-docs to nimble repo

To merge this branch:

Merge devel into newControlLists (so the UserManual exists on that branch)
Merge this PR
Run cd UserManual ; make and commit the updated .pdf

@fritzo Merged and built manual successfully.  Thanks!


Write tests for all the Manager methods

I think PR has done it - https://github.com/openclimatefix/nowcasting_dataset/pull/363

I would like to see the coverage of the tests, to see if we need to write more

Need to write a test for the new Manager.configure_loggers() function.  I've added TODOs to manager.py and test_manager.py.


WebApp:az webapp up DefaultErrorResponseException import error

I followed tutorial from https://docs.microsoft.com/fr-fr/azure/app-service/quickstart-python?tabs=bash&pivots=python-framework-flask but command az webapp up --sku F1 --name test-hello-world is raising an import error.
Command Name
az webapp up
Errors:
cannot import name 'DefaultErrorResponseException' from 'azure.mgmt.web.models' (/usr/lib/python3/dist-packages/azure/mgmt/web/models.py)
Traceback (most recent call last):
python3/dist-packages/knack/cli.py, ln 215, in invoke
    cmd_result = self.invocation.execute(args)
cli/core/commands/__init__.py, ln 558, in execute
    self.commands_loader.load_arguments(command)
azure/cli/core/__init__.py, ln 478, in load_arguments
    self.command_table[command].load_arguments()  # this loads the arguments via reflection
...
cli/command_modules/appservice/custom.py, ln 39, in <module>
    from azure.mgmt.web.models import KeyInfo, DefaultErrorResponseException
ImportError: cannot import name 'DefaultErrorResponseException' from 'azure.mgmt.web.models' (/usr/lib/python3/dist-packages/azure/mgmt/web/models.py)

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az webapp up --sku {} --name {}

Expected Behavior
Environment Summary
Linux-5.10.0-1-amd64-x86_64-with-glibc2.31
Python 3.9.1
Installer: PIP

azure-cli 2.17.1

Extensions:
azure-devops 0.19.0

Additional Context

webapp

webapp

@calvinsID this is concerning can you take a look.

@calvinsID this is concerning can you take a look.

Hi @panzergame, I was unable to repro this. Can you please try reinstalling Azure CLI and letting me know if issue still persists?

Hi @panzergame, I was unable to repro this. Can you please try reinstalling Azure CLI and letting me know if issue still persists?

Hi @calvinsID, I reinstalled Azure CLI and now it seems to work nicely, thanks for testing. Should I let you close the issue ?

Hi @calvinsID, I reinstalled Azure CLI and now it seems to work nicely, thanks for testing. Should I let you close the issue ?


fix(wallet-connect): allow the first chain id to be optional during connector init

Description
Resolves: #469
Additional Information
If dapp supports multiple networks, it is not correct to assume that the first one is required. Therefore, I decided to define chains parameter (required networks for connection) only if there is one network. Otherwise it should be undefined in order to accept connections with wallets that support other networks except the first one from the list of supported networks by the dapp.
Thanks in advance!

[x] I read the contributing docs (if this is your first contribution)

So far LGTM, adding two comments:

connect function needs refactor for this as well, since chains are handled there.
IMO let's wait for some major wallets to support this config (e.g. MetaMask, Rainbow, AlphaWallet between others).

2. IMO let's wait for some major wallets to support this config (e.g. MetaMask, Rainbow, AlphaWallet between others).

Smart contract wallet users, like Safe users, are not able to use any dApp that relies on wagmi when using WalletConnect until this fix is implemented. Except on the specific case that they try to use it on the chain that happen to be the defaultChain
More context on the fix:
https://github.com/WalletConnect/walletconnect-monorepo/pull/2781

@dasanra Wagmi is not UI ready and also WalletConnectConnector offers an option to set selected chains as required on the connect function, still think it's an issue that needs to be addressed asap though

If this is merged, this will break a lot of wallets as not all wallets support optional chains (and may also require at least one required chain).
For example, MetaMask does not support Optional Chains and they also require at least one required chain.

It's great to see so much attention to this issue, guys :pray:
In my opinion, it's not correct to assume that 1st item from the list of passed networks during initialization of connector must be required. If some wallets don't support empty required chain - we should allow devs at least to decide what networks should be required.
Existing implementation on Wagmi side already breaks some dapps. Therefore, we cannot just wait.
@glitch-txs sorry, probably I'm missing something... What exactly do you wanna to refactor in connect method?

@ElvisKrop if you don't pass a chainId into connect function it still gets populated and used as required chain if I'm not mistaken.

As per https://github.com/wagmi-dev/references/pull/470#issuecomment-1673051758, we will need to wait.
We will get in touch with WalletConnect to see what we can do.

Once enough popular wallets support this, this PR can be re-opened on the Wagmi repo. Connectors are moving over there.


link to "Exercise 3: Machine Learning with Jupyter" broken

In workshop readme, link to "Exercise 3: Machine Learning with Jupyter" is broken.

I could not find the broken link. From the https://app.gitbook.com/@ibm-developer/s/cloudpakfordata-telco-workshop/ page I clicked the table of contents,
under Telco Workshop -> Machine Learning with Jupyter and it worked to link to https://app.gitbook.com/@ibm-developer/s/cloudpakfordata-telco-workshop/telco-workshop/machine-learning-in-jupyter-notebook
Also, we have not referred to Exercise 3 in a while. Perhaps you are using an old version of the workshop?

Its broken on the Introduction page. For some reason its linking to the github readme instead of the gitbook page. Will fix in a bit.

Filed #163 for broader updates to workshop/README.md and the out-of-date Agenda.


Fix regression in name resolution in parameter

This ensures we don't overwrite associatedDeclarationForContainingInitializerOrBindingName in resolveNameHelper so we can be sure we have the correct associated declaration.
Fixes #38243

@typescript-bot cherry-pick this to release-3.9


New package: Microsoft.ServiceFabricSDK version 7.0.1816

[x] Have you signed the Contributor License Agreement?
[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?

[x] Does your manifest conform to the 1.5 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

@williamoconnorme please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
@microsoft-github-policy-service agree [company="{your company}"]

Options:

(default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.

@microsoft-github-policy-service agree

(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.

@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree

This PR is blocked due to the requirement of package Microsoft.ServiceFabricRuntime being installed before the SDK. When winget 1.6 introduces dependencies, I'll update this PR to the latest manifest schema

Automatic Validation ended with:

Installation failed with exit code -1978335226
2023-09-20 08:05:56.271 [CLI ] ShellExecute installer failed: 1603 Property(S): ErrorDialog = ErrorDlg
MSI (s) (94:A4) [08:05:56:036]: Product: Microsoft Azure Service Fabric SDK -- Installation failed.
MSI (s) (94:A4) [08:05:56:036]: Windows Installer installed the product. Product Name: Microsoft Azure Service Fabric SDK. Product Version: 7.0.1816.9590. Product Language: 1033. Manufacturer: Microsoft Corporation. Installation success or error status: 1603.


Disconnection without notice, requiring app restart to send messages

Summary
Randomly the Android app will silently disconnect, meaning that it isn't connected but there is no message indicating that it isn't connected. When this occurs, I'm unable to post messages even after several attempts. To resolve it, I need to restart the app, but doing so causes any messages I sent that didn't successfully post to disappear, so I need to retype them.
Environment Information

Device Name: OnePlus 5T
OS Version: 8.0.0
Mattermost App Version: 1.8.0
Mattermost Server Version: 4.9.1

Steps to reproduce
I believe it may occur while switching cell towers or networks. You could go for a long walk while using Mattermost.
Expected behavior
Stay connected or indicate that a disconnection has occurred. Save unsent message after app restart so that it can be resent.
Observed behavior
See summary above.
Possible fixes
Not sure.

Same happened with me. I am also not receiving any notification from mattermost on my Andiod app. Like I have to check the app to see if I have any notification/message.

Hi @iGeekFace @zoobiaashfaq43
Thanks for the report,
I've created a ticket here for us to investigate and try to reproduce...

I'll close this issue off here for now and we can track via the above-linked ticket.

@lindy65 Above ticket is closed, so writing here.
Issue is actual. Mattermost mobile app on android disconnects when it is minimized or in standby mode in a background. If you need more information I can provide

Hi @etursunbaev :)
Thank you for the report!  As this issue is old and we weren't able to reproduce it on our team when it was opened last year (see ticket) would you please open a new issue so we can reproduce it with what you're seeing now?
Please let us know which version on the Android App you are using as well as which Mattermost server you are running.  You can find this by going to 3-dot menu > Settings > About Mattermost.
Please also give us the exact steps of how you reproduce this so we can try to reproduce it in our team.
Thank you!


[wip] Add policy versions

Codecov Report

Merging #87 (a0af1b5) into main (8c1f35f) will increase coverage by 5.04%.
The diff coverage is 99.41%.

:exclamation: Current head a0af1b5 differs from pull request most recent head 23542fb. Consider uploading reports for the commit 23542fb to get more accurate results

@@            Coverage Diff             @@
##             main      #87      +/-   ##
==========================================
+ Coverage   87.04%   92.08%   +5.04%
==========================================
  Files           8        9       +1
  Lines        1011     1061      +50
==========================================
+ Hits          880      977      +97
+ Misses         91       62      -29
+ Partials       40       22      -18

Impacted Files
Coverage Δ

opa/opa.go
67.77% <ø> (ø)

pkg/policy/manager.go
99.40% <99.40%> (ø)

server/server.go
86.60% <100.00%> (+1.82%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8c1f35f...23542fb. Read the comment docs.


Keyscaler rendering issues

Key chart is not big enough
Black box is too wide and not tall enough

This is fixed but it's ugly. The graph portion of a KeyScaler doesn't appear to be vertically scalable. At least not that I could find. It's fixed at 128 pixels. I put a constant in there to exempt 128 pixels from all vertical scaling of CKeyScalerView tags, and that seems to do the trick.
It still doesn't fit perfectly, but it'll do.


Fix provisionedNamespaces status field of ClusterExternalSecret keeps getting updated non-stop

Fix provisionedNamespaces field in Status of ClusterExternalSecret keeps getting updated non-stop.
We've noticed this odd behavior in all of our ClusterExternalSecret objects that are replicated to hundreds of other namespaces. The resourceVersion keeps jumping a few dozen every second and provisionedNamespaces keeps getting reordered despite the items are exactly the same.
We've tested this patch in our cluster and successfully stopped the forever updating.

/ok-to-test sha=f97908a

@moolen thank you! Looks like this is good to go?


Even more bash tips

https://github.com/Idnan/bash-guide

Nice to know, but closing as it's just informational.


Some refactors to Multipart-Upload

Great work!

Oh shoot, those commits are not correct - I'm gonna fix that.


View error

when viewing an assignment from the View button a red error box appears, even when there is no error

@telferha Is that when viewing as a student or instructor?


feat(parameters): Allow settings boto3.client() arguments (mini version)

A mini pr, just focusing on solving for #1079

Issue number:

#1079 (closes)
#1096 (original pr with mypy typing)

Summary
Changes

Please provide a summary of what's being changed

Same as #1096 minus the typing
User experience

Please share what the user experience looks like before and after this change

Same as #1096 minus the typing
Checklist
If your change doesn't seem to apply, please leave them unchecked.

[x] Meet tenets criteria
[x] I have performed a self-review of my this change
[x] Changes are tested
[x] Changes are documented
[x] PR title follows conventional commit semantics

Is this a breaking change?
RFC issue number:
Checklist:

[ ] Migration process documented
[ ] Implement warnings (if it can live side by side)

Acknowledgment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

View rendered docs/utilities/parameters.md

Codecov Report

Merging #1197 (d2673d8) into develop (8ca082f) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff            @@
##           develop    #1197   +/-   ##
========================================
  Coverage    99.88%   99.88%
========================================
  Files          119      119
  Lines         5414     5420    +6
  Branches       616      619    +3
========================================
+ Hits          5408     5414    +6
  Misses           2        2
  Partials         4        4

Impacted Files
Coverage Δ

...ambda_powertools/utilities/parameters/appconfig.py
100.00% <100.00%> (ø)

..._lambda_powertools/utilities/parameters/secrets.py
100.00% <100.00%> (ø)

aws_lambda_powertools/utilities/parameters/ssm.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8ca082f...d2673d8. Read the comment docs.

In an effort to prevent you from wasting time, you should wait for issues to prioritised, this won't be merged for a long time until we're ready to add new features.

@heitorlessa mentioned that this could be merged as an exception here: https://github.com/awslabs/aws-lambda-powertools-python/issues/1079#issuecomment-1088469397

In an effort to prevent you from wasting time, you should wait for issues to prioritised, this won't be merged for a long time until we're ready to add new features.

"so if anyone could create a PR to accept a Client I'll happily work with you to get it merged as an exception."
in order to help customers, I have reduced the scope and size of the original PR, to make it easier to review and merge.

@heitorlessa FYI, here is a simpler working change for allowing private endpoints, with a relavant example.


Creating Docker image for Chatbot

Created test-transcripts folder
Changed code to read transcripts from folder
Fixed up some import declarations to remove any possible name collisions
Added dockerfile
Moved requirements.txt to root
note the dockerfile is not building for me yet.

This is now out of date.  New PR is posted from the main branch.


WPF documentation is not available anymore (sub https://docs.microsoft.com/en-us/dotnet/framework/wpf )

Issue description
When using bing to get to the documentation for wpf with wpf docs microsoft as keywords, a nice summary with a couple of links is provided.
all those lead to sub-pages of https://docs.microsoft.com/en-us/dotnet/framework/wpf
but that page isn't found anymore.
this leads to people thinking wpf being (less /un)supported, which is not great.
I wasn't sure whether to create the issue here or at the docs.microsoft.com repo...
Target framework

[ ] .NET Core
[X] .NET Framework
[ ] .NET Standard

Tagging @adegeo

This was fixed yesterday. Sorry for the confusion.


Document how the new GCP pipeline works

We could use the Wiki section of this repo to maintain the docs. It should document how the pipeline works and act as a playbook to handle unexpected issues.
wishlist

[ ] playbook (e.g. how to start the crawl)
[ ] expand on open issues / edge cases (e.g. what happens when a table is deleted mid crawl)
[x] go back through history and document issues encountered with Beam Python SDK

@tunetheweb heads up, I would love to get some fresh eyes on the process to run the pipeline and improve the documentation to cover everything someone new might need. I'll reach out again once we get closer to "completing" the pipeline so we can work on this together and you can poke holes in everything!

Closing it as this pipeline was deprecated.
See the new pipeline and docs here: https://github.com/HTTPArchive/dataform?tab=readme-ov-file


Align names with those produced by the dataflow runner harness.

These will be unused once the runner harness produces the correct
transform payloads.
Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Make sure there is a JIRA issue filed for the change (usually before you start working on it).  Trivial changes like typos do not require a JIRA issue.  Your pull request should address just this issue, without pulling in other changes.
[ ] Each commit in the pull request should have a meaningful subject line and body.
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Run mvn clean verify to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

@kennknowles

R: @kennknowles
Jenkins: retest this please

Coverage increased (+0.04%) to 69.569% when pulling 29fea883637d809ec75d4dc65f5faeed8eb04210 on robertwb:runner-harness-fixes into 9379ca289a00cae169075728e6230a1d4a317659 on apache:master.

Coverage increased (+0.04%) to 69.576% when pulling 29fea883637d809ec75d4dc65f5faeed8eb04210 on robertwb:runner-harness-fixes into 9379ca289a00cae169075728e6230a1d4a317659 on apache:master.

Coverage increased (+0.04%) to 69.576% when pulling 00ee6800571e7e14d916d6379950255805f3a080 on robertwb:runner-harness-fixes into 9379ca289a00cae169075728e6230a1d4a317659 on apache:master.

Jenkins: retest this please

Coverage increased (+0.05%) to 69.585% when pulling 00ee6800571e7e14d916d6379950255805f3a080 on robertwb:runner-harness-fixes into 9379ca289a00cae169075728e6230a1d4a317659 on apache:master.

On the other hand, the title should be "the Dataflow runner harness" so this is a move away from the SDK using the URNs that it should, and moving towards hardcoding for Dataflow.

Do you register both the OLD_RUNNER_HARNESS_PAR_DO_URN as well as the actual PAR_DO_URN?

Yes, the actual PAR_DO_URN has been registered for a while (against code that actually expects the ParDo Payload).

SGTM

Do we need this change in the release branch supporting python streaming on dataflow with the 2.2.0 release?


Identifiers validation failed for dataset EFC - Herbário Escola de Florestas Curitiba

Identifier validation failed for the dataset EFC - Herbário Escola de Florestas Curitiba:

Crawler attempt: 444
Publishing organization: Universidade Federal do Paraná
Publishing country: BRAZIL
Installation: speciesLink Network / INCT-HVFF IPT
Number of occurrences indexed for this dataset at the time of the issue: 23954
Cause: GBIF ID problems exceed 50% threshold: 100% duplicates; 23954 total records; 23954 absent records
Pipelines execution steps: http://api.gbif.org/v1/pipelines/history/execution/4165309/step

New IDs sample:

Old IDs sample:

BRA:UFPR:EFC:23666
BRA:UFPR:EFC:23667
BRA:UFPR:EFC:23668
BRA:UFPR:EFC:23669
BRA:UFPR:EFC:23670
BRA:UFPR:EFC:23671
BRA:UFPR:EFC:23672
BRA:UFPR:EFC:23673
BRA:UFPR:EFC:23674
BRA:UFPR:EFC:23675

Publisher email
Hello,
I am contacting you from the GBIF Secretariat about a dataset published by the EFC - Herbário Escola de Florestas Curitiba : https://doi.org/10.15468/ucgdyq.
We noticed that the occurrenceIDs were changed. We have temporarily paused the ingestions of this dataset.
As you might already know, when an occurrence record has a new occurrenceID for a given dataset, our system considers it to be a new occurrence.
This means that it will be given a new gbifid and a new occurrence URL (like this one: https://www.gbif.org/occurrence/1252968762) and the old gbifid and URL will be deprecated.
In this case, this means that the occurrence URLs would be deprecated when ingesting the newest versions of these datasets.
We would like to check with you if those changes were intentional. Do you know if this is the case?
Please let us know, thanks! We are happy to resume the dataset ingestion.
Note that some users rely on those occurrence URLs and gbifids (like https://bionomia.net for example). In an attempt to improve
the stability of the occurrence URLs and gbifids, we have implemented a warning system to detect these type of changes in datasets (see this news item).
If the data publisher can provide us with a list of old and new occurrenceIDs per record, we can avoid the identifier and URL changes. Could that be an option?
Please let us know if you have any question. Thanks!
All the best,

You can skip/fix identifier validation using the registry UI.

Identifier validation still  failing for the dataset EFC - Herbário Escola de Florestas Curitiba:

Crawler attempt: 445
Publishing organization: Universidade Federal do Paraná
Publishing country: BRAZIL
Installation: speciesLink Network / INCT-HVFF IPT
Number of occurrences indexed for this dataset at the time of the issue: 23954
Cause: GBIF ID problems exceed 50% threshold: 100% duplicates; 23954 total records; 23954 absent records
Pipelines execution steps: http://api.gbif.org/v1/pipelines/history/execution/4184517/step

New IDs sample:

Old IDs sample:

BRA:UFPR:EFC:23666
BRA:UFPR:EFC:23667
BRA:UFPR:EFC:23668
BRA:UFPR:EFC:23669
BRA:UFPR:EFC:23670
BRA:UFPR:EFC:23671
BRA:UFPR:EFC:23672
BRA:UFPR:EFC:23673
BRA:UFPR:EFC:23674
BRA:UFPR:EFC:23675

Publisher email
Hello,
I am contacting you from the GBIF Secretariat about a dataset published by the EFC - Herbário Escola de Florestas Curitiba : https://doi.org/10.15468/ucgdyq.
We noticed that the occurrenceIDs were changed. We have temporarily paused the ingestions of this dataset.
As you might already know, when an occurrence record has a new occurrenceID for a given dataset, our system considers it to be a new occurrence.
This means that it will be given a new gbifid and a new occurrence URL (like this one: https://www.gbif.org/occurrence/1252968762) and the old gbifid and URL will be deprecated.
In this case, this means that the occurrence URLs would be deprecated when ingesting the newest versions of these datasets.
We would like to check with you if those changes were intentional. Do you know if this is the case?
Please let us know, thanks! We are happy to resume the dataset ingestion.
Note that some users rely on those occurrence URLs and gbifids (like https://bionomia.net for example). In an attempt to improve
the stability of the occurrence URLs and gbifids, we have implemented a warning system to detect these type of changes in datasets (see this news item).
If the data publisher can provide us with a list of old and new occurrenceIDs per record, we can avoid the identifier and URL changes. Could that be an option?
Please let us know if you have any question. Thanks!
All the best,

You can skip/fix identifier validation using the registry UI.


Use ethereum and local signer from keep-common

Depends on https://github.com/keep-network/keep-common/pull/61
EthereumSigner provides functions to sign a message and verify a
signature using Ethereum-specific signature format. It also provides
functions for the conversion of a public key to address.
So far, this code existed only in keep-core and could not be reused
from keep-ecdsa. In #61 the code has been extracted from keep-core
with slight modifications that will allow it to be imported both from
keep-core and keep-ecdsa.
The code is being extracted to allow us to decouple ethereum-specific
implementation functions we use in generic code of keep-ecdsa client,
especially in node.go.
In #61 also the local chain implementation of keep-core's localSigner
has been extracted to keep-common. This way, we'll reuse the same
code from keep-core and keep-ecdsa instead of duplicating it.
Here, we remove the keep-core implementation and import the one from
keep-ecdsa.

Once 61 is merged, we'd need to update mod dependencies. I have no more comments here.

Once 61 is merged, we'd need to update mod dependencies. I have no more comments here.

We can wait with updating dependencies for another PR. I was thinking about releasing new keep-commons version today or later this week.


WIP transaction context

just a copy of https://github.com/latticexyz/mud/pull/2413 for creating a snapshot release

closing with original PR


GPU Support broken in 1.25

Summary
NVidia gpu support is no longer working in microk8s 1.25 when using microk8s enable gpu. The gpu-operator has errors relating to the node.k8s.io/v1beta1 api and no further gpu related pods are launched. microK8s 1.24 works perfectly on the same machine.
What Should Happen Instead?
NVidia GPU support should function in the same way as microk8s 1.24
Reproduction Steps

Install microk8s 1.25
Enable GPU support with microk8s enable gpu
Observe the logs for the gpu-operator pod

Introspection Report
Can you suggest a fix?
The node.k8s.io/v1beta1 api was removed in Kubernetes 1.25 see here:
https://kubernetes.io/docs/reference/using-api/deprecation-guide/#runtimeclass-v125
It should be replaced with node.k8s.io/v1 API
Are you interested in contributing with a fix?
Happy to test a fix

Hi @dkibble, thank you for pointing this out. The nvidia operator is not yet ready for k8s v1.25 because of the API issue you are reporting. The team maintaining the operator is aware and a patch is already ready available [1, 2]. The next release of the gpu operator should address the problem. As soon as the new gpu operator is released we will update the gpu addon to include it.
As k8s 1.25 is relatively new it is somewhat expected that parts of the ecosystem are not yet up to date. We appreciate your understanding.
[1] https://github.com/NVIDIA/gpu-operator/issues/401#issuecomment-1245932303
[2] https://github.com/NVIDIA/gpu-operator/commit/5e73a0b291c8063918a3de9ca918d629e42bd23e

Hi Konstantinos
That's great to hear, I really look forward to the update.
Thanks to you and the team for such a great product and support.
Dave
On 17/09/2022 18:22, Konstantinos Tsakalozos wrote:

Hi @dkibble [1], thank you for pointing this out. The nvidia operator
is not yet ready for k8s v1.25 because of the API issue you are
reporting. The team maintaining the operator is aware and a patch is
already ready available [1, 2]. The next release of the gpu operator
should address the problem. As soon as the new gpu operator is released
we will update the gpu addon to include it.
As k8s 1.25 is relatively new it is somewhat expected that parts of the
ecosystem are not yet up to date. We appreciate your understanding.
[1] NVIDIA/gpu-operator#401 (comment) [2]
[2] @.*** [3]
--
Reply to this email directly, view it on GitHub [4], or unsubscribe
[5].
You are receiving this because you were mentioned.Message ID:
@.***>

Links:
[1] https://github.com/dkibble
[2]
https://github.com/NVIDIA/gpu-operator/issues/401#issuecomment-1245932303
[3]
https://github.com/NVIDIA/gpu-operator/commit/5e73a0b291c8063918a3de9ca918d629e42bd23e
[4]
https://github.com/canonical/microk8s/issues/3452#issuecomment-1250109223
[5]
https://github.com/notifications/unsubscribe-auth/A3EALLFQOE2WOHRYNI4BFTTV6X44XANCNFSM6AAAAAAQO64NF4

Hi @dkibble
GPU operator just released v22.9.0 which fixes support for Kubernetes 1.25.
You can try it out already with
microk8s enable gpu --version v22.9.0

There is also a PR to install v22.9.0 by default when enabling the GPU addon https://github.com/canonical/microk8s-core-addons/pull/109, which will be released soon after CI passes.

Hi AngelosThank you for letting me know. Will give this a go ASAP.DaveSent from my mobile, so please excuse typos
-------- Original message --------From: Angelos Kolaitis @.> Date: 04/10/2022  07:29  (GMT+00:00) To: canonical/microk8s @.> Cc: dkibble @.>, Mention @.> Subject: Re: [canonical/microk8s] GPU Support broken in 1.25 (Issue #3452)
Hi @dkibble
GPU operator just released v22.9.0 which fixes support for Kubernetes 1.25.
You can try it out already with
microk8s enable gpu --version v22.9.0
There is also a PR to install v22.9.0 by default when enabling the GPU addon canonical/microk8s-core-addons#109, which will be released soon after CI passes.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>


Azurite doesn't support blob tags

Error Description:
Azurite doesn't support blob tags.
To Reproduce:
Please run the demo code as following:
import { BlobServiceClient, StorageSharedKeyCredential } from "@azure/storage-blob";
import { assert } from "chai";

async function main() {
    const blobServiceClient = new BlobServiceClient("<Azurite-https-endpoint>", new StorageSharedKeyCredential("<account-name>", "<account-key>"));
    const containerClient = blobServiceClient.getContainerClient("<container-name>");
    await containerClient.create();
    const blockBlobClient = containerClient.getBlockBlobClient("blob-name");
    const content = "Hello World";
    await blockBlobClient.upload(content, content.length);

    const tags = {
        tag1: "val1",
        tag2: "val2"
    };
    await blockBlobClient.setTags(tags);
}

main().catch((err)=>{
    assert.ok(err);
});

Expected Behavior:
Azurite supports blob tags.
@jongio for notification.

@zzhxiaofeng
Blob Tag is already in not supported features list of Azurite, see https://github.com/Azure/Azurite#support-matrix
So this is already in out radar, and we have other issues to track it.
So I will close this issue.
Feel free to contact us if you have any comments/concern.

@blueww  Sorry, I don't know that I don't need to report an issue if it is in the not supported features list of Azurite. I will close the issue and not report similar issue.

@zzhxiaofeng
Thanks for your understanding!

This is where it's being tracked: https://github.com/Azure/Azurite/issues/647


make on eve/build fails

Hi,
I'm trying to compile the latest alpha version you released and get the following message when doing a make on the eve/build path
make: *** No rule to make target compiler.o', needed by eve'.  Stop.
I'm using the latest master branch. My distro is Ubuntu 14.04 LTS with latest updates (kernel <IP_ADDRESS>-generic)

if you run make from the root dir what happens?

I get this when running Eve from the root dir
:~/Eve$ make
cd build ; make
make[1]: Entering directory /Eve/build'
make[1]: *** No rule to make target 'compiler.o', needed by 'eve'.  Stop.
make[1]: Leaving directory '/Eve/build'
make: *** [all] Error 2

@convolvatron any thoughts?

sorry, didn't see this. i think its very likely that there isn't a luajit shaped exactly like
what its looking for.
if you change the top line of build/Makefile from /usr/local to ./lua
it will attempt to build a private copy from the luajit repo

If this helps, this is the listing of my /usr/loca/bin dir
:/usr/local/bin$ ls bower               cg_annotate  luajit          pip     rustdoc    tsserver            vgdb callgrind_annotate  cg_diff      ms_print        pip2    rust-gdb   valgrind callgrind_control   cg_merge     multirust       pip2.7  rustup.sh  valgrind-di-server cargo               composer     multirustproxy  rustc   tsc        valgrind-listener :/usr/local/bin$

Same problem here.
Eve compiled successfully after I adjust LUAJIT_INSTALL in build/Makefile to where LuaJIT installed.

@dram, thanks for the tip. Managed to get past the previous error, but now I get a different one. See below
:~/Eve/build$ make
cc -std=gnu99 -I../csrc/unix -I../csrc/core -I../csrc -I. -I/usr/local/include/luajit-2.1 -g -c ../csrc/http/json.c
In file included from ../csrc/core/core.h:36:0,
                 from ../csrc/runtime.h:4,
                 from ../csrc/http/json.c:1:
../csrc/core/timer.h: In function ‘seconds’:
../csrc/core/timer.h:15:5: warning: left shift count >= width of type [enabled by default]
     return(((u64)n)<<32);
     ^
../csrc/core/timer.h: In function ‘milliseconds’:
../csrc/core/timer.h:20:5: warning: left shift count >= width of type [enabled by default]
     return((((u64)n)<<32)/1000ull);
     ^
../csrc/core/timer.h: In function ‘rdtsc’:
../csrc/core/timer.h:31:5: warning: left shift count >= width of type [enabled by default]
     return (((ticks)a) | (((ticks)d) << 32));
     ^
../csrc/http/json.c: In function ‘print_value_json’:
../csrc/http/json.c:7:5: warning: large integer implicitly truncated to unsigned type [-Woverflow]
     case uuid_space:
     ^
../csrc/http/json.c:10:5: warning: large integer implicitly truncated to unsigned type [-Woverflow]
     case float_space:
     ^
../csrc/http/json.c:10:5: error: duplicate case value
../csrc/http/json.c:7:5: error: previously used here
     case uuid_space:
     ^
../csrc/http/json.c:13:5: warning: large integer implicitly truncated to unsigned type [-Woverflow]
     case estring_space:
     ^
../csrc/http/json.c:13:5: error: duplicate case value
../csrc/http/json.c:7:5: error: previously used here
     case uuid_space:
     ^
make: *** [json.o] Error 1

Adjusting $LUAJIT, $LIBLUAJIT, and $LUAJIT_INCLUDES at the beginning of build/Makefile worked for me. (I have luajit installed with homebrew on OSX; with eve at commit 7060081c0360)

@code-recipes what platform are you compiling on? this is almost certainly due to
using a 32 bit environment. this is totally fixable, but i'll have to bring one up to work
through the issues. can you confirm thats the case?

@code-recipes perfect, i'll try to set up and environment and commit a patch

@code-recipes i checked in a clean 32 bit build on clang into master. it'll take me a bit longer
to actually test it on 32 bits, but if you feel like giving it a whirl

oh, my. i installed a 32 bit vm, installed the tools, went through the process of building eve, and
realized far too late that the way the system is structured, it really does need a 64 bit address space. while this is convenient it isn't strictly necessary.
so unfortunately its going to take a bit more investment of time before i can put in shims so that it will run on a 32 bit machine.
if you're curious, we used a very old technique where instead of adding type tags to objects, we used  the virtual memory system to encode the type directly in the pointer to the object. so, different types live in different parts of the address space. this exploits the fact that 64 bit address have so much room that we can reserve bits for types and have plenty of room left over.
a 32 bit space (really closer to 30 after the runtime and the loader are finished) doesn't really tolerate such a profligate waste, so i have some slightly slower support I can add for dynamic type tags optionally compiled in...but it won't be until next week that i will have time to bring it up and test it on both platforms

@convolvatron thank you for taking the time to look into this ...

Don't need to make in 0.2


Add new functionality to cordon and drain nodes in an ASG

This introduces a new command kubergrunt eks drain which can be used to cordon and drain all the instances in an ASG. This is useful when doing a blue-green replacement of the worker pools.

NOTE: Because this is related to EKS, there is no testing in this repo. Instead, we have integration testing in the terraform-aws-eks repo. Here is the PR: https://github.com/gruntwork-io/terraform-aws-eks/pull/319

Thanks for review! going to merge this in now!


Issue 216: end-to-end CI (DRAFT)

This PR will close #216.
Steps:

[x] hewr function to generate count data from times series of exponential rates $r_t$.
[x] pipeline function to use above to create dummy data with this schema https://gist.github.com/damonbayer/63bfeab36514806896cd06dc0235d1ea @damonbayer
[ ] gh actions basically as per (closed) #172

Blocked by an error

raise ValueError("Data frame appears to have date gaps")

Which is unexpected because the data is generated in sequence.
I'll parse this tomorrow.

That error was due to differing names of disease across different data sets. Onto creating eval data.

This seems to be running but noting that without the __pycache__/ dir it takes a while to compile.

I've tried more "minimalistic" settings e.g. fewer look back points, but occasionally you get fails due to bad sampling propagating something unexpected along the pipeline. At this moment I'm using 90 days data for inference and drawing a single 500/500 chain.
Exploring the fastest option that 1) doesn't fail stochastically, 2) still covers the pipeline sufficiently to avoid false CI passes would be a good issue.

I think this is now erroring because epipredict and epiprocess are not loaded.

@SamuelBrand1 After I added the quarto action, all checks are passing. On Teams you alluded to a potential other issue, so holding off on approval for now.

Nice! Thanks @damonbayer . The other issue was using deps that need to be installed with pak e.g. epipredict; I hadn't got to the Quarto problem yet.
This should be good to review now.


[HTK Parity] Spatial Understanding

HoloToolkit shipped with the Spatial Understanding feature that has not yet been ported to MRTK v2.
This feature took the spatial mapping mesh and performed some analysis on the enviornment to create a play space similar to what is done in the Robo Raid application.
We have received numerous requests to port Spatial Understanding (ex: #6243)

Additional context: since MRTK v2 need to support HoloLens 1st Gen devices, Plane Finder/Spatial Understanding is needed. It will unblock the customers migrating from HTK to MRTK v2.

Is this being considered for MRTK 2.2.0 ?

would like to bring this up again. this is really missing in the MRTK for some types of applications

26.06.2020... still here.

We've opted not to do this. It's been replaced by scene understanding released in MRTK 2.6.


#27 [Add] pin assign for de0_blinkL

Goal
Check the character L blinks on the 7-segment LED at Terasic DE0 with write bitstream.

Synthesis was successful.


Add an overview for rapid test partners

Feature description
As of version 2.1 Antigen-Rapid-Test results can be obtained by the CWA. Even though there are only few partners at the moment an overview on the website (and maybe in the app) of all current partners.

Problem and motivation
The problem is that people don't know wether their local test center is included in the CWA.
People don't want to check the press releases on a regular basis and prefer to have a comprehensive overview.

Is this something you're interested in working on
Unfortunately my coding skills don't allow me to help on this feature.

Internal Tracking ID: EXPOSUREAPP-7043
Internal Tracking ID: EXPOSUREAPP-7342

@kannicht Thanks for the suggestion. Internal Tracking ID: EXPOSUREAPP-7043

Corona-Warn-App Open Source Team

May be it should be moved to wishlist. I think it's also a nice feature for both apps.

It would probably be better to leave this request here on the website and add a separate item on the wishlist to have the list of Rapid Antigen Test partners available in the app. That way the two types of requests (website and app) are decoupled from each other.

@MikeMcC399 agreed.

It would probably be better to[...] add a separate item on the wishlist

Done: https://github.com/corona-warn-app/cwa-wishlist/issues/498

This had been resolved in #1203 and #1205.

@DerVogel2020

This had been resolved in #1203 and #1205.

According to https://www.coronawarn.app/en/blog/2021-05-02-corona-warn-app-version-2-1/ there are currently 8 partners offering integrated rapid COVID-19 tests and a further 125 requests for integration have been received.
If anything like the number of 125 is reached, then a separate place to list all these organizations would be helpful.

One more partner seems to be integrated now: https://landing.ticket.io/news/corona-warn-app-ticket-io-kooperieren-im-kampf-gegen-corona

@Ein-Tim Thanks. Cross-Checking.

@dsarkar
Can confirm that. Had a Test yesterday at testing site which uses their backend.
Even though I was unable to scan the QR-Code when I got the result from their webpage I had the possibility to add the result in the CWA.

@kannicht Thank you very much for the info!

One more partner seems to be integrated now: https://landing.ticket.io/news/corona-warn-app-ticket-io-kooperieren-im-kampf-gegen-corona

If all testing sites using their backend will be listed, the list would be very long since there are very much partners, using the ticket.io-backend for integrating with CWA.
See https://www.google.com/search?q=ticket.io schnelltest .

And one more: https://www.presseportal.de/pm/156361/4932908 (https://www.covisa.de)

Is this what you have been asking for: https://map.schnelltestportal.de

Could this been closed because of www.schnelltestportal.de?

@dsarkar
It does seem that the request has been fulfilled through https://map.schnelltestportal.de/ which is linked to by:

https://www.coronawarn.app/en/faq/#where_can_i_get_tested "Where can I get tested?"
https://www.coronawarn.app/de/faq/#where_can_i_get_tested "Wo kann ich mich testen lassen?"

@dsarkar I suggest to close this issue now.

@dsarkar Please close this issue.


MultiApi for EventHub

Need to support latest stack profile

https://pypi.org/project/azure-mgmt-eventhub/2.4.0/


[@rancher/components] Create a Utility Library

@rak-phillip has some work done to demo this, so it's getting buy-in for a potential merge window early in a release cycle.


Add another explicit page zeroing when MADV_DONTNEED is not enough

Needless to say that clearing the whole heap has some performance implications :(

Do we really need to clear the whole heap?
Can we have non-zeroed pages after alloc.heap_accessible_size?

We don't ever shrink the heap, so I think we only need to zero up until the accessible size.
You'll also likely need to add similar code in the implementation of reset.
I'd also appreciate if you could keep that commented eprintln! around; it's useful to have for debugging things like this.

Thanks Adam. The commented out debugging statement has been re-added.
Explicit zeroing was already done in reset().


Recompile plugin with steampipe-plugin-sdk v4.1.13

Example query results

Results
Add example SQL query results here (please include the input queries as well)

Closing the PR in favor of https://github.com/turbot/steampipe-plugin-azuread/pull/103


Ddl2cpp empty table check

goal is:

to remove hardcoded number of command line options
supprot -enable_feature -no-enable_feature
some minor PEP related issues
add error/warning instead of silent generation of "empty/useless" header
add help message with options descriptions

examples:
ddl2cpp -no-timestamp-warning  db.sql db db #disable timestamp warning
ddl2cpp -no-timestamp-warning -no-warn-on-parse db.sql db db #no warn on parsing errors (enabled by default)
ddl2cpp -no-timestamp-warning -warn-on-parse db.sql db db #warn on parsing error, but continue
ddl2cpp -no-timestamp-warning -fail-on-parse db.sql db db #exit on parsing failure
ddl2cpp:
    Usage: ddl2cpp <path to ddl> <path to target (without extension, e.g. /tmp/MyTable)> <namespace>            ddl2cpp -help
ddl2cpp -help
Usage: ddl2cpp [-help]

 OPTIONAL ARGUMENTS:

 [-[no]-fail-on-parse]:      abort instead of silent genereation of unusable headers
 [-[no]-warn-on-parse]:      warn about unusable headers, but continue
 [-[no]-timestamp-warning]:  show warning about mysql timestamp data type

    <path to ddl> <path to target> <namespace>

 <path to ddl>    path to your SQL database/table definitions (SHOW CREATE TABLE SomeTable)
 <path to target> path to a generated C++ header file. Without extension (no *.h).
 <namespace>      namespace you want. Usually a project/database name

I'll add tests in another PR, I guess

Coverage remained the same at 94.836% when pulling de2409dd7eeece942c8ed84d4bed8ede7d35e1fd on digitalist:ddl2cppEmptyTableCheck into 3e6e16f1f02387b64c684b9c661299b0ecf226bd on rbock:develop.

Coverage remained the same at 94.836% when pulling ec6c36d541f3dda20df7cfd24e106f27875f0426 on digitalist:ddl2cppEmptyTableCheck into 3e6e16f1f02387b64c684b9c661299b0ecf226bd on rbock:develop.

Coverage remained the same at 94.836% when pulling ec6c36d541f3dda20df7cfd24e106f27875f0426 on digitalist:ddl2cppEmptyTableCheck into 3e6e16f1f02387b64c684b9c661299b0ecf226bd on rbock:develop.

Played with it a bit, very nice! Thanks :-)


ghc, ghcjs: Get rid of extraneous alex, happy, and hscolour args for 18.09

Motivation for this change
Backport #47303
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Fits CONTRIBUTING.md.

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: ghc
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-darwin (full log)
Attempted: ghc
Partial log (click to expand)
/nix/store/4rk1ny3wg3rf7bfa8wwc2vmpgdciqpff-ghc-8.4.3

Success on x86_64-linux (full log)
Attempted: ghc
Partial log (click to expand)
/nix/store/n5i1zdpmk2b1s3z96649xh8f9kr3g96s-ghc-8.4.3


"title" is missing ??!!

        bulma.title.h4Is4 "This is a Title 4"

Feliz.Engine.Bulma.1.0.0-beta-006 released with fix


Frame Drop

I record video with 60 second splitter. Some time its complete 60 sec video file
and sometimes its only 53,54,56 second
Some seconds accelerate with frames skipped
Using pi 2 and cam version 1.3
It also happens with motion detection videos
Nothing in logs No errors

If you look in scheduler log you can see when capture started and stopped. If that shows that these are 60 seconds apart but videos are shorter then it does sound that frames are getting dropped.
This should not happen even with Pi Zeros. Have you checked camera cables? Also ensure power supply is good and any USB cable used is short and thick to avoid voltage drop.
Do you have any other heavy duty software running? Run top in a console to see CPU usage.
Other thing to check is GPU memory allocation. This should be 128MB at least.

Hi
"you can see when capture started and stopped"
Yes they are 60 seconds appart
"Have you checked camera cables"  check what ?
Hi there is no red led blinking or any voltage normalized in dmesg
"Do you have any other heavy duty software running? Run top in a console to see CPU usage."
No nothing
I have even disable MP4Box conversion
Yes its 256 Mb "Other thing to check is GPU memory allocation. This should be 128MB at least."

Strangely i ran
And i get perfect 60 seconds videos with no drop at all or any accelerated frames
for i in {1..10};do raspivid -t 60000 -o vi$(date +%H_%M_%S).h264;done
So iam certain its not hardware issue

Strange.
By camera cable I meant check it is inserted well at both ends and also the tiny ribbon on the module is well pressed down.
But if raspivid is working OK it is unlikely to be a cable problem.
I just did a test with split and got twenty files OK, no frame drop. I haven't had this issue arise before.
Have you got any buffering turned on under camera settings? Try it with 0 if you have.

Yes
3000 buffer
"Try it with 0 if you have."
Ok i will

Same issue with buffer 0
Which rpi version and kernel you tested with  ?

The one I ran the test on was raspbian Stretch (9), kernel 4.9.80-v7+ #1098. I have run this on a lot of different versions right through from wheezy, through jessie etc.
There is a simple logging facility for the video callback routine which handles the data from the camera. One enables this by changing the mmal_log config setting to say //dev/shm/mjpeg/mmallogfile This can be done either in the main /etc/raspimjpeg config file or by putting an entry in uconfig in the web folder

I have 4.14
Got with rpi-update

OK. 4.9.80 was the version stretch was released with.
I'll run some tests on various kernel upgrades since then to see if the problem arises from that.
You can use rpi-update to revert the kernel if you wanted to try that.

I'm not seeing an immediate problem after rpi-update but I need to do some more detailed tests.
Where are you getting your duration numbers from? The times on the thumbnails will not be accurate as they are based on file time stamps.
I may also try building against latest videoCore library as that is one of the things that can change when kernel is updated. That may take a couple of days to do.

No
I download file and play with mpv player on ubuntu

Also i have time annotation
and i see skipped seconds or fast frames

Have you tried playing the video directly in the web browser?
Note you can also put %k in annotation string to get the video frame count (0-24), making it easier to see drop frames.

Yes its same in browser
I also tried in b+ (512 mb)
Still same issue

Couple of things.
Are you recording to the local SD memory or some other location? If the latter then you should use the boxing_path facility so the h264 files are recorded locally and then boxed to mp4 to the final location. Details in wiki.
Can you get a mmallogfile for me to check out. This is obtained by editing the /etc/raspimjpeg file and setting
mmal_logfile /dev/shm/mjpeg/mmallogfile
This records some extra debug data from every callback requesting camera video data to be handled.
Stop and start the camera system and then let video record for a few 60 second cycles (till you think there has been frames dropped.
Then turn off that logging by setting the value back to
mmal_logfile
Grab the /dev/shm/mjpeg/mmallogfile, zip it and post here.
Thanks

Its class 10 card both things are done locally
I even tested with boxing off and generate only h264 files and
then download h264   to Ubuntu system and then box them
Even then i get same results
I had 2 cameras having issues with both of them
mmallogfile.zip

OK. Thanks for the log. It will take a little while to analyse

I see something like
{2018/10/27 20:36:53} Capturing with split of 30 seconds
{2018/10/27 20:36:53} Capturing started
{2018/10/27 20:37:23} Stopping video from timer
{2018/10/27 20:37:23} Capturing stopped
{2018/10/27 20:37:23} Restarting next split of 30 seconds
{2018/10/27 20:37:23} Capturing started
[2018/10/27 20:37:31] Logged in user::
[2018/10/27 20:37:31] UserLevel 6
[2018/10/27 20:37:36] Logged in user::
[2018/10/27 20:37:36] UserLevel 6
{2018/10/27 20:37:53} Stopping video from timer
{2018/10/27 20:37:53} Capturing stopped
{2018/10/27 20:37:53} DEBUG 3
{2018/10/27 20:37:53} Restarting next split of 30 seconds
{2018/10/27 20:37:53} Capturing started
[2018/10/27 20:38:17] Logged in user::
[2018/10/27 20:38:17] UserLevel 6
{2018/10/27 20:38:23} Stopping video from timer
{2018/10/27 20:38:23} Capturing stopped
{2018/10/27 20:38:23} DEBUG 3
{2018/10/27 20:38:24} Restarting next split of 30 seconds
{2018/10/27 20:38:24} Capturing started
[2018/10/27 20:38:24] Logged in user::
[2018/10/27 20:38:24] UserLevel 6
{2018/10/27 20:38:54} Stopping video from timer
{2018/10/27 20:38:54} Capturing stopped
{2018/10/27 20:38:54} DEBUG 3
{2018/10/27 20:38:54} Restarting next split of 30 seconds
{2018/10/27 20:38:54} Capturing started
This is like some thing "{2018/10/27 20:38:54} DEBUG 3"


Add C solution for 4 Problems : 121-Best-Time-To-Buy-And-Sell-Stock.c, 100-Same-Tree.c, 235-Lowest-Common-Ancestor-Of-A-Binary-Search-Tree.c, 572-Subtree-Of-Another-Tree.c

File(s) Modified: 121-Best-Time-To-Buy-And-Sell-Stock.c, 100-Same-Tree.c, 235-Lowest-Common-Ancestor-Of-A-Binary-Search-Tree.c, 572-Subtree-Of-Another-Tree.c
Language(s) Used: C
Submission URL: https://leetcode.com/submissions/detail/763949155/, https://leetcode.com/submissions/detail/781206447/, https://leetcode.com/submissions/detail/781204900/, https://leetcode.com/submissions/detail/780945964/

Thanks, @ThBlitz!


feat(node): cleanup output file

:tada: This PR is included in version 1.26.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Update readme to link to new notebook.

Just a simple update in the link so the latest notebook for hyper graph is linked.

Thanks for catching this!


Hyper-V host?

What is meant by "Hyper-V Requirements: A hypervisor has been detected. Features required for Hyper-V are not be displayed." Does it mean Hyper-V is still enabled and working.
I have this message.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 604d834f-6f94-0a37-12b6-a10dcf7c9fa3
Version Independent ID: 4bd4ce05-bdde-f450-ee68-9fcd2cbca48e
Content: Windows 10 Hyper-V System Requirements
Content Source: virtualization/hyper-v-on-windows/reference/hyper-v-requirements.md
Service: windows-10-hyperv
Product: windows-10-hyperv
GitHub Login: @scooley

I am curious as well. I have the same message.

It turned out my system dint support Hyper-V , I had to search a lot in in the haystack for this.
The message it shows is misleading. We need to have some hyper-v tools on the device; if the PC is compatible it downloads that when we turn on this feature. Apperently docs havent mentioned it clearly I suppose. Only when we have

In the right it doesnt show Hyper-V tools , and thats the hint that my machine wont support this

@scooley or someone else at Microsoft take care of this. Its unassigned after 2 months still.

This message is really ambiguous

the message is unclear. what is that supposed to mean?!

It is supposed to mean exactly what it says. A hyperviosr has been detected, based on the information provided by CPU ID function. This most commonly means that you're running inside a virtual machine. It's typically not possible to provide more details—e.g. to know for sure what hyperviosr it is, how it's configured, etc. Hence the message is intentionally vague.
If you believe there should be no hypervisor, i.e. you're running on a bare metal PC, then chances are there's some bug or misconfiguration. This can be troubleshoot on case by case basis, but more info would be needed. Anyhow, this would not be a documentation issue. A recommended approach is to open a support case.

Submitted PR #1311 with clearer documentation and screenshot. Hope it gets approved

#1311 merged . Changes are available


[TASK][MEDIUM] Spark engine query results support reading from HDFS

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Search before creating

[X] I have searched in the task list and found no similar tasks.

Mentor

[ ] I have sufficient knowledge and experience of this task, and I volunteer to be the mentor of this task to guide contributors to complete the task.

Skill requirements

Basic knowledge on Scala Programing Language
Familiar with Apache Spark

Background and Goals
The client's SQL query may cause the Spark engine to fail because the query data result is too large, and the driver to fail due to OOM.
Although the number of query results can be limited by configuring kyuubi.operation.result.max.rows, if the amount of data in one row is too large, it will still cause OOM.
If it can support writing query results to HDFS or other storage systems, when the client needs to obtain the results, the engine will obtain the results from HDFS, which can avoid the OOM problem.
Implementation steps

Distinguish execution plans with query results
Estimate the output size of the execution plan
org.apache.spark.sql.catalyst.plans.logical.statsEstimation.EstimationUtils#getSizePerRow
Use df.write api to write the query results of the execution plan to HDFS
Implement an iterator to read data from HDFS and return it to the client

Additional context
Original reporter is @cxzl25

@cxzl25 this task is missing "implementation steps" before adding into https://github.com/orgs/apache/projects/296

Hi , I am interested in this task, can you assign it to me ? Thanks.

Use df.write api to write the query results of the execution plan to HDFS

directly call df.write will introduce an extra shuffle for the outermost limit, and hurt performance


API framework demoed with some bookmarks SQL work

Ready for review.
The bookmarks table is all wrong as per discussion with @dethe, but it serves the purpose of testing the integration w/ Travis, especially the creation of Postgres tables as part of the continuous integration work.
There are really not that many files in the PR, most of the churn is because I moved the API-documentation code away from the API itself.

The JSON files you refer to are generated by the swagger API doc tool. A
bug, I guess.
On Fri, Feb 3, 2017 at 6:00 PM Dethe Elza<EMAIL_ADDRESS>wrote:

@dethe commented on this pull request.
In README.md https://github.com/participedia/api/pull/12:

The code expects a file .env to be located in the root directory of the folder,
defining the following environment variables:
-AUTH0_CLIENT_ID and AUTH0_CLIENT_SECRET as per the Auth0 application which
-is managing the users, and AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
-corresponding to the AWS account with permissions to access the ElasticSearch
-cluster.

AUTH0_CLIENT_ID and AUTH0_CLIENT_SECRET as per the Auth0 application which
+is managing the users.  The AUTH0_CLIENT_SECRET doesn't have to be the real one for dev environments, it can be anything (e.g. 'thisisjunk')

BEARER_TOKEN so you can run npm test with an authenticated user.  To generate

a bearer token, go to jwt.io, put in the AUTH0_CLIENT_SECRET you picked above (e.g. 'thisisjunk') in the 'verify signature' spot for it, and adjust the payload to look like:

👍 even better!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/participedia/api/pull/12, or mute the thread
https://github.com/notifications/unsubscribe-auth/AADITrOcCB0C53sB-YyJK43Vj7vXqvpfks5rY9uzgaJpZM4L2-T8
.


Allow final trailing space to be an identifier

Fixes #6

Codecov Report

Merging #7 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master       #7   +/-   ##
=======================================
  Coverage   87.78%   87.78%
=======================================
  Files           1        1
  Lines         131      131
=======================================
  Hits          115      115
  Misses          9        9
  Partials        7        7

Impacted Files
Coverage Δ

jsonpointer.go
87.78% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d8f9cf0...a7a9d78. Read the comment docs.

Thank you


✍ Feature request: Add a copy button in the top right corner of code examples

Example can be seen here: https://docs.coil.com/api-versioning

agreed. will aim for next release version.

Probably obvious, but we should make this configurable in the same way search can be disabled.

Sure. Any suggestions for a good setting name? code_clipboard_copy?
The implementation of this would be essentially:

Add new JS file to sources/javascripts that has one function, that when run, iterates through all div.highlight and for each, attach a clipboard button.
Attach event listener to created button that hooks into the Selection API (there's also the older execCommand but I don't see the need to support super old versions of browsers) to copy text contents of pre.highlight > code.
Run the function at the bottom of the layout.erb file, with an if statement gating it using setting described above.

Can play around with exactly what we attach to if we get complaints about it being too narrow, but not sure that's going to be such a problem.
This should safely gate the copy button to only appear on

Done in #1258 and will be in 2.7.0 release.

@MasterOdin
Is there any way to add the copy button to the <blockquote> container but have it copy the text from the <pre> container below?
Like this:


Suggestion: Add more "motivation" in Readme.md of why using spring physics is a good idea vs. tweening. Also make it very clear no durations are used

This is a suggestion from a first time user.
When I found out this library I thought it simply was a tweening library (and "spring" was just a cool/fitting name). I also immediately started looking for instructions on where to configure the duration of an animation :)
Now I totally understand why such a concept does not exist escpecially after looking at the react-motion authors react europe video.
My suggestions would be to add this to Readme.md

Add a better "motivation" text. I know there are some under the "What is it?" section, but I think react-motion explains it better:

For 95% of use-cases of animating components, we don't have to resort to using hard-coded easing curves and duration. Set up a stiffness and damping for your UI element, and let the magic of physics take care of the rest. This way, you don't have to worry about petty situations such as interrupted animation behavior. It also greatly simplifies the API.

Make it very clear no durations are used since it's physics based.

Just some suggestions :D Thanks!

Could you start a PR? I'm in the process of creating real docs currently and would take away most of the text on the GH front, it would be great to concentrate on the strengths then instead of going into coding technicalities like right now.

Yes i'll see what I can do!

Added a new section for it: http://react-spring.surge.sh/gotchas


⚠ Samsung has degraded performance

In 0d001fd, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2552 ms

Resolved: Samsung performance has improved in 3125590.


Distance Matrix arguments did not match any overloaded cal

Here's what I did
Distance Matrix was running ok and after the update it's showing me this error:
          **TypeError: arguments did not match any overloaded call:
          post(self, QNetworkRequest, QIODevice): argument 2 has unexpected type 'dict'
          post(self, QNetworkRequest, Union[QByteArray, bytes, bytearray]): argument 2 has unexpected type 'dict'
          post(self, QNetworkRequest, QHttpMultiPart): argument 2 has unexpected type 'dict'**

Probably related with networkaccessmanager.py

just uploaded the fix to the repository. should be available within the day.


[native aot] Incompatible

I see it just opens the browser and can't seem to connect to the browser

Please follow the issue template.

Please follow the issue template.

Unhandled Exception: PuppeteerSharp.ProcessException: Failed to create connection
---> PuppeteerSharp.PuppeteerException: Failed to fetch browser webSocket url from http://<IP_ADDRESS>:9222.
---> Newtonsoft.Json.JsonSerializationException: Unable to find a constructor to use for type PuppeteerSharp.Messaging.WSEndpointResponse. A class should either have a default constructor, one constructor with arguments or a constructor marked with the JsonConstructor attribute. Path 'Browser', line 2, position 13.
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewObject(JsonReader, JsonObjectContract, JsonProperty, JsonProperty, String, Boolean&) + 0x1bb
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader, Type, JsonContract, JsonProperty, JsonContainerContract, JsonProperty, Object) + 0x2ae
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader, Type, JsonContract, JsonProperty, JsonContainerContract, JsonProperty, Object) + 0xa4
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader, Type, Boolean) + 0x272
at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader, Type) + 0xf3
at Newtonsoft.Json.JsonConvert.DeserializeObject(String, Type, JsonSerializerSettings) + 0x8e
at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String, JsonSerializerSettings) + 0x2b
at PuppeteerSharp.Launcher.d__10.MoveNext() + 0x1be
--- End of inner exception stack trace ---
at PuppeteerSharp.Launcher.d__10.MoveNext() + 0x380
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x20
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0xb2
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x4b
at PuppeteerSharp.Launcher.d__9.MoveNext() + 0x1c0
--- End of inner exception stack trace ---
at PuppeteerSharp.Launcher.d__9.MoveNext() + 0x643
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x20
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0xb2
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x4b
at Program.d__0.MoveNext() + 0x156
--- End of stack trace from previous location ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x20
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0xb2
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task, ConfigureAwaitOptions) + 0x4b
at Program.() + 0x24
at pupputeer!+0x7b896f

I think we can add some logs around that, to see what's coming from the browser.

I think we can add some logs around that, to see what's coming from the browser.

I connect with --remote-debugging-port, it work with jit but not work with aot, i will can because json

you can try PuppeteerAot ，this library is support for aot

you can try PuppeteerAot ，this library is support for aot

thanks

@NCLnclNCL AOT support coming in v19 :)


FIX #2271 don't build 32bits binaries for linux/osx

Signed-off-by: David Gageot<EMAIL_ADDRESS>

LGTM


Add task to Tree

Hello,
With this PR, we are able to add task again to the tree based on recent refactoring.
Thanks,

Woohoo!!! Great job sir!


Issue converting owlv2 model to ONNX format

Hi Team,
I hope this message finds you well.
I've been working with the owlv2 model and have encountered an issue while attempting to convert it into ONNX format using the provided command:
! optimum-cli export onnx -m google/owlv2-base-patch16 --task 'zero-shot-object-detection' --framework 'pt' owlv2_onnx
Unfortunately, I'm facing the following error:
ValueError: Trying to export a owlv2 model, that is a custom or unsupported architecture, but no custom onnx configuration was passed as custom_onnx_configs.
As I am relatively new to this process, I'm unsure about the necessity and usage of custom ONNX configuration. Could you please provide some guidance on how to address this issue? Any assistance or insights would be greatly appreciated.
Thank you for your attention to this matter.

Hi @n9s8a the message ValueError: Trying to export a owlv2 model, that is a custom or unsupported architecture, but no custom onnx configuration was passed as custom_onnx_configs. Please refer to https://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/export_a_model#custom-export-of-transformers-models for an example on how to export custom models. Please open an issue at https://github.com/huggingface/optimum/issues if you would like the model type owlv2 to be supported natively in the ONNX export.
means that owlv2 ONNX export is not yet supported in Optimum. I'll add it to my todo

Hi @fxmarty, Thankyou for your response and for clarifying the situation regarding the owlv2 ONNX export. I'm using owlv2 model in my project. Could you please provide an estimated timeline for when this support might be available?
Additionally, in the meantime, would it be feasible to use any external methods or workarounds to achieve ONNX conversion for the owlv2 model?
Your insights on these matters would be greatly appreciated.

Hi @n9s8a, https://github.com/huggingface/optimum/pull/1689 is merged. I also recommend you to use Transformers from source for now as https://github.com/huggingface/transformers/pull/29326 fixed a bug in the ONNX export of owlvit/owlv2 and is not yet in a release.
optimum-cli export onnx -m google/owlv2-base-patch16 --task 'zero-shot-object-detection' --framework 'pt' owlv2_onnx

now works.

(Where) bound to different types (tensor(double) and tensor(float) in node (/class_head/Where).
I faced this issue


Add new folder for experiment

What type of PR is this?

Uncomment only one  /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:
/kind new feature
/kind bug fix
/kind cleanup
/kind revert change
/kind design
/kind documentation
/kind enhancement

What this PR does / why we need it:
Adding placeholder for QA/Experimental/multicloud resources/scripts
Which issue(s) this PR fixes:

Fixes #
Test Report Added?:

Uncomment only one  /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:
/kind TESTED
/kind NOT-TESTED

Test Report:

Special notes for your reviewer:
This is just adding the directories and readme file

Can we close this?

@kumarashit

Not required. Service already fulfilled by other PR


update istio.io/pkg to latest version

Updates to most recent version of the pkg libraries.

@douglas-reid: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
46d22acce15e1f99aaec230307935f234563f44d
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


[Fix] Fix incorrect logging in gltf-exporter error reporting

Fixes https://github.com/playcanvas/engine/issues/5949

duplicate of https://github.com/playcanvas/engine/pull/5950


Drag and scroll with middle button

OS
macOS 13.5.2
LinearMouse
0.9.5
Is your feature request related to a problem?
I would like to switch from Smooze to your open-source solution 😃
There is only one feature I am missing: drag and scroll using the middle button (wheel button).
Describe the solution you'd like
Implement an option to enable drag and scroll with the middle button, like in Smooze and Smart Scroll
Describe alternatives you've considered
No response
Additional context
No response

+1 for this! I found Smooze Pro that does it, but since that's the only feature I would use, the price did not seem worth it.
Any pointers on how this would be implemented? I would like to give it a try, although I haven't done swift development before.

Another +1!

+1

+1, would absolutely love this feature.