Ensure cache gets properly updated with concurrent access for action with attachments

@tysonnorris notices with concurrent action invocation scenario that same attachment is being fetched multiple times in case of cache miss.
Description
If multiple request try to load an action with attachment which is not present in cache then currently it leads to multiple request to load the same attachment from backing store. This PR adds a new test which demonstrate this behavior. This test would try to concurrently read action (5 threads). Without fix you would see logs like below which show that attachment is being read multiple times
db logs [2018-12-17T08:50:49.800Z] [DEBUG] [#tid_7] [LocalEntitlementProvider] checking user 'anon-nKWrAIa7elnkH4XMmEEsb75ZB9v' has privilege 'READ' for 'actions/anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2'
[2018-12-17T08:50:49.800Z] [DEBUG] [#tid_6] [LocalEntitlementProvider] checking user 'anon-nKWrAIa7elnkH4XMmEEsb75ZB9v' has privilege 'READ' for 'actions/anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2'
[2018-12-17T08:50:49.800Z] [DEBUG] [#tid_4] [LocalEntitlementProvider] checking user 'anon-nKWrAIa7elnkH4XMmEEsb75ZB9v' has privilege 'READ' for 'actions/anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2'
[2018-12-17T08:50:49.800Z] [DEBUG] [#tid_3] [LocalEntitlementProvider] checking user 'anon-nKWrAIa7elnkH4XMmEEsb75ZB9v' has privilege 'READ' for 'actions/anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2'
[2018-12-17T08:50:49.800Z] [DEBUG] [#tid_5] [LocalEntitlementProvider] checking user 'anon-nKWrAIa7elnkH4XMmEEsb75ZB9v' has privilege 'READ' for 'actions/anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2'
[2018-12-17T08:50:49.801Z] [DEBUG] [#tid_3] [LocalEntitlementProvider] authorized
[2018-12-17T08:50:49.801Z] [DEBUG] [#tid_7] [LocalEntitlementProvider] authorized
[2018-12-17T08:50:49.801Z] [DEBUG] [#tid_4] [LocalEntitlementProvider] authorized
[2018-12-17T08:50:49.801Z] [DEBUG] [#tid_5] [LocalEntitlementProvider] authorized
[2018-12-17T08:50:49.802Z] [DEBUG] [#tid_6] [LocalEntitlementProvider] authorized
[2018-12-17T08:50:49.803Z] [DEBUG] [#tid_5] [WhiskAction] read initiated
[2018-12-17T08:50:49.803Z] [INFO] [#tid_5] [WhiskAction] [GET] serving from datastore: CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2) [marker:database_cacheMiss_count:5]
[2018-12-17T08:50:49.803Z] [INFO] [#tid_5] [CouchDbRestStore] [GET] 'whisk_local_whisks' finding document: 'id: anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2' [marker:database_getDocument_start:5]
[2018-12-17T08:50:49.804Z] [DEBUG] [#tid_4] [WhiskAction] coalesced read
[2018-12-17T08:50:49.804Z] [DEBUG] [#tid_7] [WhiskAction] coalesced read
[2018-12-17T08:50:49.805Z] [DEBUG] [#tid_6] [WhiskAction] coalesced read
[2018-12-17T08:50:49.805Z] [INFO] [#tid_4] [WhiskAction] [GET] serving from cache: CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2) [marker:database_cacheHit_count:7]
[2018-12-17T08:50:49.805Z] [INFO] [#tid_7] [WhiskAction] [GET] serving from cache: CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2) [marker:database_cacheHit_count:7]
[2018-12-17T08:50:49.805Z] [INFO] [#tid_6] [WhiskAction] [GET] serving from cache: CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2) [marker:database_cacheHit_count:7]
[2018-12-17T08:50:49.806Z] [DEBUG] [#tid_3] [WhiskAction] coalesced read
[2018-12-17T08:50:49.806Z] [INFO] [#tid_3] [WhiskAction] [GET] serving from cache: CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2) [marker:database_cacheHit_count:8]
[2018-12-17T08:50:49.814Z] [INFO] [#tid_5] [CouchDbRestStore] [marker:database_getDocument_finish:16:11]
[2018-12-17T08:50:49.824Z] [DEBUG] [#tid_5] [WhiskAction] read backend part done, now marking cache entry as done
[2018-12-17T08:50:49.825Z] [DEBUG] [#tid_5] [Entry] read finished
[2018-12-17T08:50:49.828Z] [INFO] [#tid_6] [CouchDbRestStore] [ATT_GET] 'whisk_local_whisks' finding attachment '75ee1112-fcdb-4043-ae11-12fcdbb043e7' of document 'id: anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2, rev: 2-d43fa9eb21253e04d440162630082ce9' [marker:database_getDocumentAttachment_start:30]
[2018-12-17T08:50:49.830Z] [INFO] [#tid_4] [CouchDbRestStore] [ATT_GET] 'whisk_local_whisks' finding attachment '75ee1112-fcdb-4043-ae11-12fcdbb043e7' of document 'id: anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2, rev: 2-d43fa9eb21253e04d440162630082ce9' [marker:database_getDocumentAttachment_start:32]
[2018-12-17T08:50:49.831Z] [INFO] [#tid_5] [CouchDbRestStore] [ATT_GET] 'whisk_local_whisks' finding attachment '75ee1112-fcdb-4043-ae11-12fcdbb043e7' of document 'id: anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2, rev: 2-d43fa9eb21253e04d440162630082ce9' [marker:database_getDocumentAttachment_start:33]
[2018-12-17T08:50:49.831Z] [INFO] [#tid_7] [CouchDbRestStore] [ATT_GET] 'whisk_local_whisks' finding attachment '75ee1112-fcdb-4043-ae11-12fcdbb043e7' of document 'id: anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2, rev: 2-d43fa9eb21253e04d440162630082ce9' [marker:database_getDocumentAttachment_start:33]
[2018-12-17T08:50:49.832Z] [INFO] [#tid_3] [CouchDbRestStore] [ATT_GET] 'whisk_local_whisks' finding attachment '75ee1112-fcdb-4043-ae11-12fcdbb043e7' of document 'id: anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2, rev: 2-d43fa9eb21253e04d440162630082ce9' [marker:database_getDocumentAttachment_start:34]
[2018-12-17T08:50:49.872Z] [INFO] [#tid_6] [CouchDbRestStore] [marker:database_getDocumentAttachment_finish:74:44]
[2018-12-17T08:50:49.875Z] [INFO] [#tid_6] [WhiskAction] write initiated on new cache entry
[2018-12-17T08:50:49.875Z] [DEBUG] [#tid_6] [ActionsApiTests] [GET] entity success
[2018-12-17T08:50:49.875Z] [DEBUG] [#tid_6] [WhiskAction] write backend part done, now marking cache entry as done
[2018-12-17T08:50:49.875Z] [DEBUG] [#tid_6] [Entry] write finished
[2018-12-17T08:50:49.876Z] [INFO] [#tid_6] [WhiskAction] write all done, caching CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2) Cached
[2018-12-17T08:50:49.890Z] [INFO] [#tid_4] [CouchDbRestStore] [marker:database_getDocumentAttachment_finish:92:59]
[2018-12-17T08:50:49.891Z] [DEBUG] [#tid_4] [ActionsApiTests] [GET] entity success
[2018-12-17T08:50:49.891Z] [INFO] [#tid_4] [WhiskAction] write initiated on existing cache entry, invalidating CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2), tid 4, state WriteInProgress
[2018-12-17T08:50:49.892Z] [DEBUG] [#tid_4] [WhiskAction] write backend part done, now marking cache entry as done
[2018-12-17T08:50:49.892Z] [INFO] [#tid_5] [CouchDbRestStore] [marker:database_getDocumentAttachment_finish:94:61]
[2018-12-17T08:50:49.892Z] [DEBUG] [#tid_4] [Entry] write finished
[2018-12-17T08:50:49.892Z] [INFO] [#tid_4] [WhiskAction] write all done, caching CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2) Cached
[2018-12-17T08:50:49.892Z] [DEBUG] [#tid_5] [ActionsApiTests] [GET] entity success
[2018-12-17T08:50:49.892Z] [INFO] [#tid_5] [WhiskAction] write initiated on existing cache entry, invalidating CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2), tid 5, state WriteInProgress
[2018-12-17T08:50:49.893Z] [DEBUG] [#tid_5] [WhiskAction] write backend part done, now marking cache entry as done
[2018-12-17T08:50:49.893Z] [DEBUG] [#tid_5] [Entry] write finished
[2018-12-17T08:50:49.893Z] [INFO] [#tid_5] [WhiskAction] write all done, caching CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2) Cached
[2018-12-17T08:50:49.914Z] [INFO] [#tid_3] [CouchDbRestStore] [marker:database_getDocumentAttachment_finish:116:82]
[2018-12-17T08:50:49.914Z] [INFO] [#tid_3] [WhiskAction] write initiated on existing cache entry, invalidating CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2), tid 3, state WriteInProgress
[2018-12-17T08:50:49.915Z] [DEBUG] [#tid_3] [ActionsApiTests] [GET] entity success
[2018-12-17T08:50:49.915Z] [DEBUG] [#tid_3] [WhiskAction] write backend part done, now marking cache entry as done
[2018-12-17T08:50:49.915Z] [DEBUG] [#tid_3] [Entry] write finished
[2018-12-17T08:50:49.915Z] [INFO] [#tid_3] [WhiskAction] write all done, caching CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2) Cached
[2018-12-17T08:50:49.921Z] [INFO] [#tid_7] [CouchDbRestStore] [marker:database_getDocumentAttachment_finish:123:90]
[2018-12-17T08:50:49.921Z] [INFO] [#tid_7] [WhiskAction] write initiated on existing cache entry, invalidating CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2), tid 7, state WriteInProgress
[2018-12-17T08:50:49.921Z] [DEBUG] [#tid_7] [ActionsApiTests] [GET] entity success
[2018-12-17T08:50:49.922Z] [DEBUG] [#tid_7] [WhiskAction] write backend part done, now marking cache entry as done
[2018-12-17T08:50:49.922Z] [DEBUG] [#tid_7] [Entry] write finished
[2018-12-17T08:50:49.922Z] [INFO] [#tid_7] [WhiskAction] write all done, caching CacheKey(anon-nKWrAIa7elnkH4XMmEEsb75ZB9v/action_tests_name2) Cached
5 was not equal to 1
ScalaTestFailureLocation: org.apache.openwhisk.core.controller.test.ActionsApiTests at (ActionsApiTests.scala:1079)

As a fix now the cache update operation also includes the inline logic. So a cache update would be considered complete only when the action code is also inlined. This ensures even with concurrent reader threads only one thread performs the db access and others make use of the cache
Related issue and scope

[ ] I opened an issue to propose and discuss this change (#????)

My changes affect the following components

[ ] API
[ ] Controller
[ ] Message Bus (e.g., Kafka)
[ ] Loadbalancer
[ ] Invoker
[ ] Intrinsic actions (e.g., sequences, conductors)
[ ] Data stores (e.g., CouchDB)
[ ] Tests
[ ] Deployment
[ ] CLI
[ ] General tooling
[ ] Documentation

Types of changes

[ ] Bug fix (generally a non-breaking change which closes an issue).
[ ] Enhancement or new feature (adds new functionality).
[ ] Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

[ ] I signed an Apache CLA.
[ ] I reviewed the style guides and followed the recommendations (Travis CI will check :).
[ ] I added tests to cover my changes.
[ ] My changes require further changes to the documentation.
[ ] I updated the documentation where necessary.

Codecov Report

Merging #4183 into master will decrease coverage by 4.84%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #4183      +/-   ##
==========================================
- Coverage   86.03%   81.19%   -4.85%
==========================================
  Files         152      152
  Lines        7292     7275      -17
  Branches      472      477       +5
==========================================
- Hits         6274     5907     -367
- Misses       1018     1368     +350

Impacted Files
Coverage Δ

...org/apache/openwhisk/core/entity/WhiskAction.scala
86.74% <100%> (+0.49%)
:arrow_up:

...ache/openwhisk/core/database/DocumentFactory.scala
100% <100%> (+10.52%)
:arrow_up:

...core/database/cosmosdb/RxObservableImplicits.scala
0% <0%> (-100%)
:arrow_down:

...core/database/cosmosdb/CosmosDBArtifactStore.scala
0% <0%> (-95.54%)
:arrow_down:

...sk/core/database/cosmosdb/CosmosDBViewMapper.scala
0% <0%> (-92.6%)
:arrow_down:

...whisk/core/database/cosmosdb/CosmosDBSupport.scala
0% <0%> (-83.34%)
:arrow_down:

...abase/cosmosdb/CosmosDBArtifactStoreProvider.scala
0% <0%> (-62.5%)
:arrow_down:

...in/scala/org/apache/openwhisk/common/Counter.scala
40% <0%> (-20%)
:arrow_down:

...rg/apache/openwhisk/common/ForcibleSemaphore.scala
88.46% <0%> (-7.7%)
:arrow_down:

...penwhisk/core/database/cosmosdb/CosmosDBUtil.scala
92% <0%> (-4%)
:arrow_down:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 780c98c...7a91803. Read the comment docs.


observeConnectionStateChanges

I am facing issue, the observeConnectionStateChanges seems to not emit anything when the screen is off. I am using rxBle in a service so the device is not necessarily active.
Is it a wanted behaviour by devteam ?
version : last 1.6 snapshot rx2

There is no code related to the UI in the library. Are you sure that the app does not get suspended and/or service killed?

Yes i am sure, my routine advertise scan is still on, the connection is effective.
If i deploy from AndroidStudio device On -> it's works
if i deploy from AS device screen off -> no connection events emitted but everything else works well

Could you share a sample code which I could copy-paste to recreate the issue?
Also please write your setup (phone model/ os/ etc)

Sorry i was busy, it's to difficult to extract this part of the code, but after many tests, the issue come from my code cause it appears randomly even if app is waked up.
You can close the ticket :)


Chemistry Template New Scenes

For the Chemistry Template we need

[ ] @stathisnikolaidis : One new type of scene named as "Chemistry 2D naming puzzle".
[ ] @stathisnikolaidis :  One new type of scene named as "Chemistry 3D construction puzzle".
[ ] @tpapaz : In the front-end a new page should be made that allows to modify these scenes. 3D editor is not needed. We need a drag'n'drop form that allows to position assets in the scene (the scene can be represented as a box) in a certain order. Both 2D and 3D puzzle will be modified with the same page.
[ ] @stathisnikolaidis : When a new Chemistry game project is made then it should also have a new Scene named as "2D naming puzzle" and a
[ ] @stathisnikolaidis : new scene named as "3D construction puzzle"


Update to edition 2018

Update mdbook to 0.3.5
Command:
sed -i '/extern crate/ {N;s/\n$//}' src/.md; sed -i 's/extern crate error_chain;/use error_chain::error_chain;/; s/extern crate lazy_static;/use lazy_static::lazy_static;/; s/extern crate bitflags;/use bitflags::bitflags;/; s/extern crate serde_json;/ use serde_json::json;/; s/extern crate serde_derive;/use serde::{Serialize, Deserialize};/; /macro_use/d; /extern crate/ d; s/rust/rust,edition2018/; s/bail!/error_chain::&/; s/(debug|info|warn|error)!/log::&/;' src/.md
Fix #530
fixes #ISSUE_ID
:tada: Hi and welcome! Please read the text below and remove it - Thank you! :tada:
No worries if anything in these lists is unclear. Just submit the PR and ask away! :+1:

Things to check before submitting a PR

[x] the tests are passing locally with cargo test
[x] cookbook renders correctly in mdbook serve -o
[x] commits are squashed into one and rebased to latest master
[x] PR contains correct "fixes #ISSUE_ID" clause to autoclose the issue on PR merge

if issue does not exist consider creating it or remove the clause

[x] spell check runs without errors ./ci/spellcheck.sh
[ ] link check runs without errors link-checker ./book
[x] non rendered items are in sorted order (links, reference, identifiers, Cargo.toml)
[x] links to docs.rs have wildcard version https://docs.rs/tar/*/tar/struct.Entry.html
[x] example has standard error handling
[x] code identifiers in description are in hyperlinked backticks

[`Entry::unpack`]: https://docs.rs/tar/*/tar/struct.Entry.html#method.unpack

Things to do after submitting PR

[ ] check if CI is happy with your PR

Thank you for reading, you may now delete this text! Thank you! :smile:

@AndyGauge Interested to review?

Yeah, I was reading it and I'm impressed.  One thing I'm concerned about is if there are any extern crate definitions that the text doesn't include a reference.  One thing nice about having those extern crate statements is it makes it easy to set up the recipe's Cargo.toml.  I'll build it locally and peruse.

duh, there's crate badges right above the recipes.  That should be instructive enough.  I've peeked at it and from 30000 ft it looks good.

@AndyGauge I think the notable changes are index.hbs, the google analytics were removed I guess. Not sure if we need it?

@AndyGauge What should I do next for this?

I'm going to try and get it compiled and uploaded to review it in book format next.

OK, I've published it here http://www.yetanother.site/rust-cookbook/ and am reviewing

OK, so I'm now seeing new errors... it looks like error-chain does not like its result type being ignored.
error[E0107]: wrong number of type arguments: expected 2, found 1
--> /tmp/rust-skeptic.HOf7zmrK9Y0r/test.rs:12:14
|
12 | fn main() -> Result<()> {
|              ^^^^^^^^^^ expected 2 type arguments
error: aborting due to 2 previous errors
For more information about this error, try rustc --explain E0107.
test about_sect_a_note_about_error_handling_line_150 ... FAILED
error: cannot find macro error_chain_processing in this scope
--> /tmp/rust-skeptic.qfSON6UfVh6g/test.rs:6:1
|
6  | / error_chain! {
7  | |     foreign_links {
8  | |         Utf8(std::str::Utf8Error);
9  | |         AddrParse(std::net::AddrParseError);
10 | |     }
11 | | }
| |_^

looks like we need to do this https://doc.rust-lang.org/edition-guide/rust-2018/macros/macro-changes.html
use error-chain::Result

In order to make Travis useful, I had to fix #569 to figure out what errors are caused by this change.

OK, here's the problem.  error-chain never was updated for Rust 2018 🤕 .  Now I can do that, but it looks like here https://github.com/rust-lang/rust/issues/57013 the way to fix error-chain for now is to use all of the internal macros.
use error_chain::{error_chain, error_chain_processing, impl_error_chain_kind, impl_error_chain_processed, impl_extract_backtrace, ChainedError};

It is really hard to come up with consensus on error handling.  I'm biased, because I maintain error-chain (picked up both projects after Brian left Mozilla).  I think I would use thiserror in my own library at this point though.  Feel free to address error-handling in #502 or #370
The primary reason we don't switch crates is there is no ecosystem stability right now.  The cookbook is all about curated, stable libraries that the library team originally approved.  Adding crates to the list should be a hard thing.  In fact if you want to showcase anyhow as a recipe I'd probably approve it, and throw a thiserror in there too, but for blanket switching all recipes I think that would give me an aneurysm.

@AndyGauge What do we need to do for this pull request?

We've discovered a major pain point using error-chain with 2018.  Either we need to use the internal macros, fix error-chain, or change the error handling library.  I believe I wrote those in order of complexity.
I doubt I will be working on this error-chain change immediately, but I think that would be a good solution.  If you would rather rewrite the error handling story of the cookbook let me know if you have that capacity.

Interestingly I discovered that error-chain 0.12 included the 2018 fix.  cargo downloads both error-chain 0.11 and 0.12, and incorrectly was using the 0.11 version.
I feel like this is a skeptic problem.
I just recompiled this branch with rustc 1.45-nightly and had every single test fail.

OK, let's party like its 2018!

I doubt I will be working on this error-chain change immediately, but I think that would be a good solution. If you would rather rewrite the error handling story of the cookbook let me know if you have that capacity.

Yes, I could rewrite the rust error handling story to use anyhow or thiserror.
Also, I would like change the global data story to use once_cell.
But I may not have time recently but should have some time to work this out in few weeks.

Turns out error chain was updated and the problem was with the dependency
graph. At this time we shouldnt rewrite the error handling. Thanks!
On Sun, May 31, 2020, 6:26 PM Ivan Tham<EMAIL_ADDRESS>wrote:

I doubt I will be working on this error-chain change immediately, but I
think that would be a good solution. If you would rather rewrite the error
handling story of the cookbook let me know if you have that capacity.
Yes, I could rewrite the rust error handling story to use anyhow or
thiserror.
Also, I would like change the global data story to use once_cell.
But I may not have time recently but should have some time to work this
out in few weeks.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/rust-lang-nursery/rust-cookbook/pull/565#issuecomment-636567316,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABRZBXXCHJJNUCSL5ENULS3RUL7VZANCNFSM4J43T6RA
.

How about lazy_static? once_cell looks like a better alternative without using macro. https://docs.rs/once_cell/1.4.0/once_cell/#faq

Funny story.  @matklad was helping me with a pre-rfc when once_cell was born.  https://internals.rust-lang.org/t/pre-rfc-lazy-static-move-to-std/7993/36 https://internals.rust-lang.org/t/pre-rfc-lazy-static-move-to-std/7993/36
I’m fine with a recipe on it.  How about highlighting the regex use case?  I know @burtsushi describes that as a big part of using either of these, and creating a regex is a good introductory problem.  Like https://docs.rs/once_cell/1.4.0/once_cell/#building-block https://docs.rs/once_cell/1.4.0/once_cell/#building-block without creating a macro?
Andrew

On May 31, 2020, at 8:23 PM, Ivan Tham<EMAIL_ADDRESS>wrote:
How about lazy_static? once_cell looks like a better alternative without using macro. https://docs.rs/once_cell/1.4.0/once_cell/#faq https://docs.rs/once_cell/1.4.0/once_cell/#faq
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub https://github.com/rust-lang-nursery/rust-cookbook/pull/565#issuecomment-636592304, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABRZBXTPZJUYGA4XNFN5P2TRUMNMBANCNFSM4J43T6RA.

I’m fine with a recipe on it.  How about highlighting the regex use case?  I know @burtsushi describes that as a big part of using either of these, and creating a regex is a good introductory problem.  Like https://docs.rs/once_cell/1.4.0/once_cell/#building-block https://docs.rs/once_cell/1.4.0/once_cell/#building-block without creating a macro?

https://docs.rs/once_cell/1.4.0/once_cell/#building-block is creating a macro itself and regex already uses lazy_static, we would have to replace it with once_cell and can also explain the benefits of using once_cell such that one could do lazy initialization without macro.


fix: msan stdc++

draft

Benchmark results
Metrics with a significant change:

note_trial_decrypting_time_in_ms (8): 95.4 (+1667%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Values are compared against data from master at commit ede7a47f and shown if the difference exceeds 1%.
L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
8 txs
32 txs
128 txs

l1_rollup_calldata_size_in_bytes
45,444
179,588
716,132

l1_rollup_calldata_gas
222,960
868,124
3,447,740

l1_rollup_execution_gas
842,047
3,595,232
22,203,109

l2_block_processing_time_in_ms
2,006 (-1%)
7,599 (-1%)
29,837 (-1%)

note_successful_decrypting_time_in_ms
294 (-3%)
868 (-2%)
3,174 (-1%)

note_trial_decrypting_time_in_ms
:warning: 95.4 (+1667%)
76.2 (-21%)
134 (+2%)

l2_block_building_time_in_ms
13,461
53,448
212,317 (-1%)

l2_block_rollup_simulation_time_in_ms
12,102
48,159
191,259 (-1%)

l2_block_public_tx_process_time_in_ms
1,318 (-1%)
5,155 (-2%)
20,567 (-2%)

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric
5 blocks
10 blocks

node_history_sync_time_in_ms
23,360 (+7%)
45,850 (+8%)

note_history_successful_decrypting_time_in_ms
2,097 (+2%)
4,327 (+9%)

note_history_trial_decrypting_time_in_ms
121 (+1%)
150 (+3%)

node_database_size_in_bytes
1,629,107
1,095,912

pxe_database_size_in_bytes
29,748
59,307

Circuits stats
Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit
circuit_simulation_time_in_ms
circuit_input_size_in_bytes
circuit_output_size_in_bytes

private-kernel-init
772
61,697
18,905

private-kernel-ordering
127 (+1%)
24,297
8,153

base-rollup
1,840 (+3%)
656,311
814

root-rollup
172 (+2%)
4,072
1,097

private-kernel-inner
798 (+1%)
81,568
18,905

public-kernel-private-input
42.7 (+3%)
41,519
18,841

public-kernel-non-first-iteration
26.7 (+2%)
41,497
18,841

merge-rollup
15.4 (+1%)
2,592
873

Miscellaneous
Transaction sizes based on how many contracts are deployed in the tx.

Metric
0 deployed contracts
1 deployed contracts

tx_size_in_bytes
8,787
27,547

Stale


Fix error in check_entry

I would guess this line is not covered by the test because it always produces a string with ≡ as first character so get getindex fails with 2 since it's in the middle of that multibyte character

@Azzaare bump :)

@Azzaare kind reminder :)


add support for custom templates of gantt elements

Would be great to define template for gantt element with binding to attributes of object.
var tasks = [{
    "startDate": new Date("Sun Dec 09 01:36:45 EST 2012"),
    "endDate": new Date("Sun Dec 10 02:36:45 EST 2012"),
    "taskName": "E Job",
    "status": "FAILED",
    "type": "Type A"
}]

could render on chart into something like:

Type A

Failed

09.12
E Job
10.12

Hmm I am not sure if I know what your trying to say.


Implement startup code injector

The manually managed startup code needs to be replaced with real managed one.
https://github.com/dotnet/corert/pull/468

Before we start working on this, do we want to make GC static section and string literals lazily initialized?
(To keep the stuff the startup code needs to run minimal.)

If this is tracking what the .NET Native for UWP startup code injector does, I think the only thing that's left is static constructors on the <Module> type (nice to have, but we could live without it for a while).
Or is this tracking complete removal of the C++ bootstrapper code?

I agree - we should have more specific issues on what is left. (#657 tracks the module constructors.)


@alias comments aren't visible

How are you using the lua-language-server?
Visual Studio Code Extension (sumneko.lua)
Which OS are you using?
Linux
What is the issue affecting?
Annotations, Hover
Expected Behaviour
a comment to an alias type should be shown as regular comment to a parameter would be shown (at least if there is no comment to the @param annotation itself). The hover message should read:
function f(arg: string)
@param arg — comment from alias

Unused functions.Lua Diagnostics.(unused-function)
Unused local `f`.Lua Diagnostics.(unused-local)

Actual Behaviour
the comment to the alias type isn't shown. the hover message is:
function f(arg: string)
Unused functions.Lua Diagnostics.(unused-function)
Unused local `f`.Lua Diagnostics.(unused-local)

Reproduction steps

cat << EOF > file.lua
---@alias t_arg string comment from alias
---@param arg t_arg
local function f(arg)
end
EOF
code file.lua
hover over the function name and read message

This would be a great quality of life improvement. @sumneko has mentioned in the docs that @alias comments are ignored, and that you would have to manually spell out the parameters of a function each time in order to capture their comments.
What I've found myself doing is writing a string above each function which uses aliases, which outlines the parameters in a one-liner ( arg1, arg2 [, optionalArg3, optionalArg4] ) with comments. But it is a LOT of boilerplate.


docker image

would be nice to have a docker image that can bring up an instance quickly

This is like post-0.2.0 unless there's a PR for it.

This is done!


I have got problem with Flight::register(...)

This is my index.php:

in this file I registered connection to the database.
When i tried to use this class in another file

, it is said that

Sry but I'am just beginner with Flight :|

You can't call start before setting everything up.

So, when I change it, I seen an error :


fix: Don't show suggestions after accepting

Its annoying that suggestions pop back up after running the EE. This makes it so you have to type another character first. Also refactor a little bit to simplify the code.

Closing in favor of identical https://github.com/wandb/weave/pull/616


Lazy code generation should be based on differences in build-version (incl. annotations) of 9ML model not timestamps

Instead of checking the timestamp of the source file, a copy of the build-modified version of the 9ML model being generated should be saved along side the build including all build options as annotations. Lazy code generation should then check for differences with this build-version before deciding to rebuild the model.
PyPe9 version information could also be saved as annotations in the build-version 9ML to force a rebuild when PyPe9 is updated.

Probably should hash the whole build component class including + build options and append that the NEST-node/NMODL-mech name to avoid naming conflicts on load

This has been implemented now


Upgrade httpclient from 4.5.2 to 4.5.3+

There is a vulnerability in httpclient.  Please upgrade to 4.5.3+.  Latest is 4.5.6
https://issues.apache.org/jira/browse/HTTPCLIENT-1803

@WayneMyerAvalara Hey there! Is there any update on this?

This was taken care in release 22.5.0
Please update to the latest version of SDK and let us know if you see an issue.


Depicted code hash is not the contract code hash

The value which is shown as Code Hash is not actually the code hash, but instead the contract's Wasm bytecode.
This is what the UI shows:

This is how the contract bundle looks:
$ cat target/ink/delegator.contract | jq .
{
  "metadataVersion": "0.1.0",
  "source": {
    "hash": "0xfa28de816b7ca333f5258a60b5ee37404174609eb629265cfcf770f0d5c0688a",
    "language": "ink! 3.0.0-rc3",
    "compiler": "rustc 1.53.0-nightly",
    "wasm": "0x0061736d…",
  },
  …
}

So using source.hash instead of source.wasm would be the fix.
(Note: In an ideal world the Canvas UI would compute the hash source.wasm by itself and assert hash(source.wasm) == source.hash.)

this happens in polkadot.js/api-contract and apparently has been always the case. should we change it for Canvas or in general?

Could be that this field was not always available, but now it is, so yeah we should in general change it to always display the actual contract hash and not an excerpt of the contracts Wasm bytecode.


Cannot put wiki repo under organization.

Describe the bug
Currently the wiki mode require the repo to be owned by the login user directly:
https://github.com/lostintangent/gistpad/blob/01a3ac2ae3a25f86d78eb09d7aeda3aeeba8ea97/src/repos/store/index.ts#L174
But I would like to have the wiki mode repo put under an organization.
Is it possible to relax this restriction?

Probably the logic can be changed to:
If there is only one repo "isWiki" then use it regardless the name. The name condition is only used for more than one repo case.

Yep this proposal makes sense! I'll make this change in the next extension update (later this week).


Can't install correctly smb-flexvol

What happened:
I intend to install smb-flexvol using th efollowing command :
kubectl apply -f https://raw.githubusercontent.com/Azure/kubernetes-volume-drivers/master/flexvolume/smb/deployment/smb-flexvol-installer.yaml
I have 4 pods created one is in bad state :

Can you help me?

looks like there is sth wrong with your cluster. And we suggest using https://github.com/kubernetes-csi/csi-driver-smb

Thanks Andy,
Using the plugin I was able to create the PV and the PVC 👍

I am not able to use the volume within a deployment :
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/name: publishing-orchestrator-job
    app.kubernetes.io/version: "{version}"
    app.kubernetes.io/component: web
    app.kubernetes.io/environment: beta
  name: publishing-orchestrator-job-deployment
  namespace: beta
spec:
  replicas: 1
  template:
    metadata:
      name: publishing-orchestrator-job
      labels:
        app: publishing-orchestrator-job
        name: publishing-orchestrator-job
    spec:
      containers:
      - name: publishing-orchestrator-job
        image: tracepartsdockerrepo.azurecr.io/moon-publishingorchestrator.job-beta:{version}
        imagePullPolicy: "Always"
        ports:
          - containerPort: 80
            name: pod-port
        env:
        - name: ASPNETCORE_ENVIRONMENT
          value: Beta
        - name: DLL_NAME
          value: PublishingOrchestrator.job.dll
        volumeMounts:
          - name: test
            mountPath: /data
      volumes:
        - name: test
          flexVolume:
            driver: "microsoft.com/smb"
            secretRef:
              name: smbcreds
            options:
              source: "//<IP_ADDRESS>/publishingstudio$$"
              mountoptions: "vers=2.0,dir_mode=0777,file_mode=0777"

I get the following message
MountVolume.MountDevice failed for volume "pv-smb" : rpc error: code = Internal desc = volume(publishingstudio_smb) mount "//<IP_ADDRESS>/publishingstudio$" on "/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pv-smb/globalmount" failed with mount failed: exit status 32 Mounting command: mount Mounting arguments: -t cifs -o dir_mode=0777,file_mode=0777,vers=2.0,domain=<IP_ADDRESS>, //<IP_ADDRESS>/publishingstudio$ /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pv-smb/globalmount Output: mount error(11): Resource temporarily unavailable Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
Could you help me to understand what is wrong?
Regards,
Sébastien

to narrow down the issue, could you ssh to one agent node, and run
mkdir /tmp/cifs
mount -t cifs -o dir_mode=0777,file_mode=0777,vers=2.0,domain=<IP_ADDRESS>, //<IP_ADDRESS>/publishingstudio ... /tmp/cifs

Looks like there is sth wrong with your smb server.

also you can use this way to create a smb server on k8s cluster: https://github.com/kubernetes-csi/csi-driver-smb/tree/master/deploy/example/smb-provisioner, that may narrow down the issue.


Fix a bug where a non-base WebClient does not respect the scheme of URI.

Motivation:
The following example sends a reqeust with HTTP protocol, not user defined scheme
var client = WebClient.of();
// send to "http://example.com/path"
client.get("h1c://example.com/path")
client.get("h1://example.com/path")
client.get("h2://example.com/path")
client.get("h2c://example.com/path")

Modifications:

Use the scheme of the specfied URI if the client is non-based URI.

Result:
A non-base WebClient respects user defined schedme correctly.

Checkstyle violations 😉


error - dial tcp: lookup dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com: no such host

What is the issue?
Hi team im getting this error below -
C:\Windows\System32>ollama run gemma
pulling manifest
Error: Head "https://dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/sha256/ef/ef311de6af9db043d51ca4b1e766c28e0a1ac41d60420fed5e001dc470c64b77/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=66040c77ac1b787c3af820529859349a%!F(MISSING)20240411%!F(MISSING)auto%!F(MISSING)s3%!F(MISSING)aws4_request&X-Amz-Date=20240411T115747Z&X-Amz-Expires=1200&X-Amz-SignedHeaders=host&X-Amz-Signature=1f76e44610af72e8b0d79dab1389991616ef08be32f9747fe7eb1feae6711eff": dial tcp: lookup dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com: no such host
I tried a lot many ways to download this please help me to fix this error.
What did you expect to see?
No response
Steps to reproduce
No response
Are there any recent changes that introduced the issue?
No response
OS
Windows
Architecture
amd64
Platform
No response
Ollama version
0.1.31
GPU
Nvidia
GPU info
No response
CPU
AMD
Other software
No response

Hi @Ajay-Engineer , it looks like your DNS is broken. Are you behind a firewall?

Hi @Ajay-Engineer access to cloudflare is required to download models via ollama pull. In the future this will be behind a custom domain such as models.ollama.com which should help in environments with firewalls

i am also getting the same error. please let me know if you resolved it

Hello! @Ajay-Engineer, refer this youtube video given below to resolve your issue...
https://youtu.be/CkDEZQuk0bU?si=VlMsIb6FyBWYyomF

Hello, Everyone!
At Ollama we're working on a solution to this issue, and have been seeing some positive results!
Now we need your help testing in your enviroments as well!
How to help:

Run a test pull through our staging server
From the list below, pick one (or many) of the models that you have not pulled already, and perform a pull.
ollama pull issue1736.ollama.dev/library/llama3:8b
ollama pull issue1736.ollama.dev/library/gemma:2b
ollama pull issue1736.ollama.dev/library/mistral
ollama pull issue1736.ollama.dev/library/dolphin-mistral
ollama pull issue1736.ollama.dev/library/wizardlm2
ollama pull issue1736.ollama.dev/library/llava-phi3
ollama pull issue1736.ollama.dev/library/llava-llama3
ollama pull issue1736.ollama.dev/library/dolphin-phi
ollama pull issue1736.ollama.dev/library/nomic-embed-text
ollama pull issue1736.ollama.dev/library/phi3
ollama pull issue1736.ollama.dev/library/orca-mini

Remove and retry 2 or 3 more times
ollama rm issue1736.ollama.dev/library/<model>[:<tag>]
ollama pull issue1736.ollama.dev/library/<model>[:<tag>]

Report back!
Please respond here answering these questions to the best of your ability:

What was the full ollama pull command you ran including model?
What OS are you running the ollama server on?
What speed range did you see? (e.g. 30-50 MB/s)
What version of Ollama are you using?
What region of the world is your ollama running?
What is the top speed of your internet connection?
Was it faster, slower, the same as a normal ollama pull <model> for the same model(s)?

Thank you all so much in advance. We look forward to hearing back from you.

Hello! @Ajay-Engineer, refer this youtube video given below to resolve your issue... https://youtu.be/CkDEZQuk0bU?si=VlMsIb6FyBWYyomF

It worked! Thanks a lot for sharing!!

Please respond here answering these questions to the best of your ability:

What was the full ollama pull command you ran including model?
ollama pull issue1736.ollama.dev/library/phi3
ollama pull issue1736.ollama.dev/library/llava-llama3
ollama pull issue1736.ollama.dev/library/wizardlm2
ollama pull issue1736.ollama.dev/library/gemma:2b
ollama run phi3
What OS are you running the ollama server on?
OS: Apple M1 Pro, 16GB
What speed range did you see? (e.g. 30-50 MB/s)
100.7: Mbps download
79.0: Mbps upload
What version of Ollama are you using?
ollama version is 0.1.38
What region of the world is your ollama running?
India, Maharashtra
What is the top speed of your internet connection?
100.7: Mbps download
79.0: Mbps upload
Was it faster, slower, the same as a normal ollama pull <model> for the same model(s)?
It was working fine for llama and code-llama few days back but has stopped working since last 2 days.

Even after multiple attempts it did not work. Still getting the error "dial tcp: lookup dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com: no such host error"

Please respond here answering these questions to the best of your ability:

What was the full ollama pull command you ran including model?
ollama pull issue1736.ollama.dev/library/phi3
ollama pull issue1736.ollama.dev/library/llava-llama3
ollama pull issue1736.ollama.dev/library/wizardlm2
ollama pull issue1736.ollama.dev/library/gemma:2b
ollama run phi3
What OS are you running the ollama server on?
OS: Apple M1 Pro, 16GB
What speed range did you see? (e.g. 30-50 MB/s)
100.7: Mbps download
79.0: Mbps upload
What version of Ollama are you using?
ollama version is 0.1.38
What region of the world is your ollama running?
India, Maharashtra
What is the top speed of your internet connection?
100.7: Mbps download
79.0: Mbps upload
Was it faster, slower, the same as a normal ollama pull <model> for the same model(s)?
It was working fine for llama and code-llama few days back but has stopped working since last 2 days.

Even after multiple attempts it did not work. Still getting the error "dial tcp: lookup dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com: no such host error"

same here, Did you found any solution?

I am trying on my macbook and getting the same error. After using a VPN connection I was able to download the same!

Hello! @Ajay-Engineer, refer this youtube video given below to resolve your issue... https://youtu.be/CkDEZQuk0bU?si=VlMsIb6FyBWYyomF

this worked , thanks a lot for sharing


Is this correct?

At the top of the page I see "Cluster autoscaler should only be tested in preview on AKS clusters with a single node pool." I believe the kubernetes system pods require a linux node pool so, if I want to run windows containers, I'll have to have two node pools minimum (one linux for kubernetes-system and one windows for my application). Am I expected to be able to use the autoscaler together the multiple node pools I believe are required for running windows containers?
I've been trying to overload my cluster to make it autoscale but so far it's not doing anything. I wonder if it's related to this: "During preview, you can't set a higher minimum node count than is currently set for the cluster. For example, if you currently have min count set to 1, you can't update the min count to 3." but I don't know how to configure a node count for my cluster so I'm not sure how to check. Thanks!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 077f1bb5-2ba9-490c-2c6e-0de292266539
Version Independent ID: 0fd87b4f-88a4-c2eb-3ebc-f2b113ac203c
Content: Use the cluster autoscaler in Azure Kubernetes Service (AKS)
Content Source: articles/aks/cluster-autoscaler.md
Service: container-service
GitHub Login: @mlearned
Microsoft Alias: mlearned

Thanks for the question. We are currently investigating and will update you shortly.

@sirdank take a look at the Limitations Documentation we have for running Windows Nodes
https://docs.microsoft.com/en-us/azure/aks/windows-node-limitations
We don't currently support AutoScaler and Windows Nodes at the moment

@sirdank I will close this out. If you have further questions just let us know and we can always reopen and continue the discussion.


Elementary OpenAPI check in e2e tests

Description
Elementary OpenAPI check in e2e tests
Type of change

[ ] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library
[x] e2e tests

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 91.53%. Comparing base (fccd088) to head (c96affb).
Report is 24 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #773      +/-   ##
==========================================
- Coverage   94.72%   91.53%   -3.19%
==========================================
  Files          53       53
  Lines        1896     1914      +18
==========================================
- Hits         1796     1752      -44
- Misses        100      162      +62

see 5 files with indirect coverage changes

/retest

/retest

/retest

/approve
/lgtm


Query about the evaluation method codes

Thanks for your nice work. Would you mind upload the evaluation method codes?  Then we can reproduce the results in your paper using the uploaded anomaly scores. Thanks for your help.

Thanks for your interest. I've been thinking of how to do that, but taking out the evaluation code was a bit tricky as we evaluate through many visualizations and the detection accuracy.
The calculation of detection accuracy should be straightforward and easy to implement, while the visualizations may not. Our paper got a minor revision recently, and hoping that it'd get accepted soon, and we'll publish the codes asap.

good news. Our paper has been just accepted at the Pattern Recognition journal. The code will be made available soon.


Fix key error in scaler

Only the changes for assemblyline_core/scaler/scaler_server.py are actually from this ticket, the rest are just syncing up change records between dev and master, those changes should actually already be in dev.


Vue is not work by use “”

My code is as follows：

{{message}}

Because I used the iframe tag,The value of the span tag is not turned into “hello world” .
where is the problem?

you should use  to close the iframe tag

@yanzhenzeng you are right!

thanks . it's work @yanzhenzeng


Updated sortition-pools dependency to the most recent one

Refs https://github.com/keep-network/keep-core/issues/2715
See https://github.com/keep-network/sortition-pools/pull/135
See https://github.com/keep-network/sortition-pools/pull/136
See https://github.com/keep-network/sortition-pools/pull/137
This is a preparation for T staking contract integration but will also allow
to use selectGroup for challenges given that in the most recent version this
function is not protected by onlyOwner modifier.
Sortition pool does not longer expose isEligibleFunction. Everyone who
has the minimum required authorization for Random Beacon on T staking contract
will be able to join the pool. Removed RandomBeacon.isEligible.
I have also updated RandomBeacon.registerOperator and
RandomBeacon.updateOperatorStatus. SortitionPool contract does no longer
work with staking contract and it requires the contract owner to pass the
current authorization amount. These functions will be reworked during the
integration with T staking contract. For now, I am just making everything
to compile and work with the most recent sortition-pools version.

Rebased after https://github.com/keep-network/keep-core/pull/2740 landed in main. Linter build should be green now 🤞🏻


BigQuery authorized views

Description & motivation

Add docs for BigQuery authorized views (aka grant_access_to config), which were added in v0.18.0 (https://github.com/fishtown-analytics/dbt/pull/2517)

Pre-release docs
Is this change related to an unreleased version of dbt?

[ ] Yes: please

update the base branch to next
add Changelog components: <Changelog>[New/Changed] in v0.x.0</Changelog>
add links to the "New and changed documentation" section of the latest Migration Guide

[x] No: please ensure the base branch is current
[ ] Unsure: we'll let you know!

@azhard Would you mind giving this a once-over, if you have a moment? Want to make sure I'm saying roughly the right things :)

@azhard Would you mind giving this a once-over, if you have a moment? Want to make sure I'm saying roughly the right things :)

LGTM!

Hey @jtcohen6 hate to do this post merge but realized a slight problem with the SQL config, the project and dataset keys need to be quoted like so:
{{ config(
    grant_access_to=[
+      {'project': 'project_1', 'dataset: 'dataset_1'},
+      {'project': 'project_2', 'dataset': 'dataset_2'}
-      {project: 'project_1', dataset: 'dataset_1'},
-      {project: 'project_2', dataset: 'dataset_2'}
    ]
) }}

Apologies for not catching that the first time!

Hey @jtcohen6 hate to do this post merge but realized a slight problem with the SQL config, the project and dataset keys need to be quoted like so:
{{ config(
    grant_access_to=[
+      {'project': 'project_1', 'dataset: 'dataset_1'},
+      {'project': 'project_2', 'dataset': 'dataset_2'}
-      {project: 'project_1', dataset: 'dataset_1'},
-      {project: 'project_2', dataset: 'dataset_2'}
    ]
) }}

Apologies for not catching that the first time!

@azhard All good, thank you so much for catching!


Update to Node Template v2.0.0-rc6+1

Backport substrate-developer-hub/substrate-node-template#81

Was that really the only place we referenced the tag?

We could go crazy and change it everywhere (e.g. https://github.com/substrate-developer-hub/substrate-developer-hub.github.io/blob/source/docs/tutorials/build-a-dapp/prepare.md) but that didn't seem necessary? Do you disagree?

FYI, this is how I checked for what to change
substrate.dev: grep -r 'git clone -b v2.0.0-rc6' ./docs
./docs/tutorials/create-a-pallet.md:git clone -b v2.0.0-rc6 https://github.com/substrate-developer-hub/substrate-pallet-template test-pallet
./docs/tutorials/create-your-first-substrate-chain/setup.md:    git clone -b v2.0.0-rc6+1 --depth 1 https://github.com/substrate-developer-hub/substrate-node-template
./docs/tutorials/create-your-first-substrate-chain/setup.md:git clone -b v2.0.0-rc6 --depth 1 https://github.com/substrate-developer-hub/substrate-front-end-template

thanks @danforbes

Build a dApp tutorial: You should already have version v2.0.0-rc6 from the first chain tutorial
User: Wait, what? I just did the first chain tutorial and I have version v2.0.0-rc6+1. Are they the same?
That would be my only concern, but it isn't that big of a concern. I'm fine not updating it everywhere. Already approved.

Updated per your suggestion, @JoshOrndorff...great point.


How to read css style property, if the properties are added in css file?

Locator strategy!!!
Based on user Action, a DOM object css property is changed 'display:none;' and this is added in a css file and exposed as class.
We need to assert the element is not visible in the DOM.
Any advice?

Hi there!
Your question is better suited for StackOverflow or Gitter. Please ask a question there with the 'protractor' tag or post in the Gitter Channel to get help.
From the the getting help section of the README:

Please ask usage and debugging questions on StackOverflow (use the "protractor" tag) or in the Angular discussion group. (Please do not ask support questions here on Github.)

Thanks!


Loading sam_checkpoint problem

The app keeps freezing and not responding when I tried to load the sam_checkpoint
I don't know if this issue is because SAM doesn't support m1 "mps" or not
Please add the option to use CPU if the app working on mac m1 or m2 till SAM supports mps
best regards

RuntimeError                              Traceback (most recent call last)
File ~/anaconda3/envs/ai/lib/python3.10/site-packages/napari_sam/_widget.py:329, in SamWidget._load_model(self=<napari_sam._widget.SamWidget object>)
327 model_types = list(sam_model_registry.keys())
328 model_type = model_types[self.cb_model_type.currentIndex()]
--> 329 self.sam_model = sam_model_registry[model_type](
model_type = 'default'
self.sam_model = None
self = <napari_sam._widget.SamWidget object at 0x7fdf34145d80>
sam_model_registry[model_type] = <function build_sam_vit_h at 0x7fdf341465f0>
sam_model_registry = {'default': <function build_sam_vit_h at 0x7fdf341465f0>, 'vit_h': <function build_sam_vit_h at 0x7fdf341465f0>, 'vit_l': <function build_sam_vit_l at 0x7fdf3659f7f0>, 'vit_b': <function build_sam_vit_b at 0x7fdf3667d2d0>}
330     get_weights_path(model_type)
331 )
332 self.sam_model.to(self.device)
333 self.sam_predictor = SamPredictor(self.sam_model)
File ~/anaconda3/envs/ai/lib/python3.10/site-packages/segment_anything/build_sam.py:15, in build_sam_vit_h(checkpoint=PosixPath('/Users/ahmedshaheen/.cache/napari-segment-anything/sam_vit_h_4b8939.pth'))
14 def build_sam_vit_h(checkpoint=None):
---> 15     return _build_sam(
checkpoint = PosixPath('/Users/ahmedshaheen/.cache/napari-segment-anything/sam_vit_h_4b8939.pth')
16         encoder_embed_dim=1280,
17         encoder_depth=32,
18         encoder_num_heads=16,
19         encoder_global_attn_indexes=[7, 15, 23, 31],
20         checkpoint=checkpoint,
21     )
File ~/anaconda3/envs/ai/lib/python3.10/site-packages/segment_anything/build_sam.py:105, in _build_sam(encoder_embed_dim=1280, encoder_depth=32, encoder_num_heads=16, encoder_global_attn_indexes=[7, 15, 23, 31], checkpoint=PosixPath('/Users/ahmedshaheen/.cache/napari-segment-anything/sam_vit_h_4b8939.pth'))
103 if checkpoint is not None:
104     with open(checkpoint, "rb") as f:
--> 105         state_dict = torch.load(f)
f = <_io.BufferedReader name='/Users/ahmedshaheen/.cache/napari-segment-anything/sam_vit_h_4b8939.pth'>
106     sam.load_state_dict(state_dict)
107 return sam
File ~/anaconda3/envs/ai/lib/python3.10/site-packages/torch/serialization.py:797, in load(f=<_io.BufferedReader name='/Users/ahmedshaheen/.cache/napari-segment-anything/sam_vit_h_4b8939.pth'>, map_location=None, pickle_module=<module 'pickle' from '/Users/ahmedshaheen/anaconda3/envs/ai/lib/python3.10/pickle.py'>, weights_only=False, **pickle_load_args={'encoding': 'utf-8'})
792 if _is_zipfile(opened_file):
793     # The zipfile reader is going to advance the current file position.
794     # If we want to actually tail call to torch.jit.load, we need to
795     # reset back to the original position.
796     orig_position = opened_file.tell()
--> 797     with _open_zipfile_reader(opened_file) as opened_zipfile:
opened_file = <_io.BufferedReader name='/Users/ahmedshaheen/.cache/napari-segment-anything/sam_vit_h_4b8939.pth'>
798         if _is_torchscript_zip(opened_zipfile):
799             warnings.warn("'torch.load' received a zip file that looks like a TorchScript archive"
800                           " dispatching to 'torch.jit.load' (call 'torch.jit.load' directly to"
801                           " silence this warning)", UserWarning)
File ~/anaconda3/envs/ai/lib/python3.10/site-packages/torch/serialization.py:283, in _open_zipfile_reader.init(self=<torch.serialization._open_zipfile_reader object>, name_or_buffer=<_io.BufferedReader name='/Users/ahmedshaheen/.cache/napari-segment-anything/sam_vit_h_4b8939.pth'>)
282 def init(self, name_or_buffer) -> None:
--> 283     super().init(torch._C.PyTorchFileReader(name_or_buffer))
torch._C.PyTorchFileReader = <class 'torch.PyTorchFileReader'>
name_or_buffer = <_io.BufferedReader name='/Users/ahmedshaheen/.cache/napari-segment-anything/sam_vit_h_4b8939.pth'>
torch._C = <module 'torch._C' from '/Users/ahmedshaheen/anaconda3/envs/ai/lib/python3.10/site-packages/torch/_C.cpython-310-darwin.so'>
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

I do not have a mac so it won't be possible to reproduce or fix the problem, sorry. Have you installed the pytorch cpu only version?

I did so and it worked, thank you so much

I do not have a mac so it won't be possible to reproduce or fix the problem, sorry. Have you installed the Pytorch CPU only version?

I installed the mps Pytorch version, the problem was due to corrupted check point files and it was solved after manually downloading and replacing the file

Good to know! :)


example galleries

fixes #142
this introduces the idea of the oranda project gallery- which is composed of 2 markdown files and a github workflow:

project_gallery/README.md: table of examples contained in the project_gallery dir
project_gallery/COMMUNITY.md: table of examples from the community
gallery.yml: github workflow to build and publish every example to cloudflare

@shadows-withal i'm also not sure! let's chat about it today on discord- i've been thinking a bit more on the "dual goals" (marketing/usecase gallery, and ci/crater-style integration testing) of this feature and how it might be better to separate them, etc


Errors in structures of isonocardicin A compounds

Hi Gareth and Adnan,
I have realised that the structures for the isonocardicin A compounds (CHEBI:16483, CHEBI:57788 and CHEBI:77633) are wrong concerning the stereoconfiguration of C-9' and C-5. The correct structure can be found in pmid:15252031 . You can also compare with CHEBI:81022,  isonocardicin C.
I have attached the corrected structures.
Please also add a cross reference from CHEBI:57788 to MetaCyc ISONOCARDICIN-A

I just tried to attach the new structures, but I don't think it worked.
I will send them to you in a mail.
Cheers,
Kristian

Now done.
Best Regards,
Adnan


what are the valid values for region?

I had 'KE' as my region but the video was still not available in my region making me think that maybe you have a certain format for specifying regions I don't know about.

Hello, @brayo-pip !
Thanks for the report.
Unfortunately YouTube makes a lot of changes to its site, so this might not work anymore... Your format is okay, KE stands for Kenya, and as far as I know the language spoken there is English, right?
First answer to these questions:

Is your video Unlisted on YouTube
Can you access this video on YouTube

May you send me the simple reproducible sample, and which video you are searching for?

I resolved the problem... It wasn't from your library's end, thanks anyway


antctl Support Bundle Should Collect Logs from Crashed Agent and Controller Containers

Describe the bug
When antctl runs in remote mode, it needs to connect to remote agent or controller APIServer to get support bundle. If remote agent or controller fails, antctl writes the failed node name in support bundle. I'd like to suggest an improvement. If antctl fails to connect to agent/controller APIServer, it can alternatively get logs from K8s API. For example:
logOption := &v1.PodLogOptions{Container: containerName, SinceSeconds: ...}
logs := k8sClientset.CoreV1().Pods("kube-system").GetLogs(podName, logOption)
logStream, err := logs.Stream(context.TODO())
...

Another improvement is that support bundle function controllerRemoteRunE should run getClusterInfo even if there is failure in previous steps. This is to deal with a corner case (or maybe we can see it as a user error) that incompatible antctl is used to get agent and controller logs, then antctl may fail in creating clients for AntreaAgentInfo and AntreaControllerInfo, and it collects nothing. If in this case it can at least run getClusterInfo, we will be able to see the Antrea version from collected Pod manifests and get a clue.
These improvements are minor, but they'll help to save some communication round-trips.
To Reproduce
Edit antrea-controller Deployment and change antrea-controller command to "false".
$ antctl supportbundle --controller-only -d test-bundle
Controller Info Failed Reason: Post "https://<IP_ADDRESS>:10349/apis/system.antrea.io/v1beta1/supportbundles": dial tcp <IP_ADDRESS>:10349: connect: connection refused
Finish [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 200ms
Error: no data was collected: Post "https://<IP_ADDRESS>:10349/apis/system.antrea.io/v1beta1/supportbundles": dial tcp <IP_ADDRESS>:10349: connect: connection refused;

Expected
It's better to also request log stream from K8s API.
Actual behavior
antctl only reports an error.
Versions:
Please provide the following information:

Antrea version: v1.5.2, v1.6.0

working on this


Add customFields to missing targets

Signed-off-by: Frank Jogeleit<EMAIL_ADDRESS>

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

26 of 44 (59.09%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.4%) to 94.189%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/target/kinesis/kinesis.go
2
11
18.18%

pkg/target/s3/s3.go
2
11
18.18%

Totals

Change from base Build<PHONE_NUMBER>:
-0.4%

Covered Lines:
3712

Relevant Lines:
3941

💛  - Coveralls


Close doc preview on release merge

Tested with https://github.com/ajm188/website/pull/17
Some relevant log lines from the testing:
{"level":"info","github_event_type":"pull_request","github_delivery_id":"e6283e60-bf75-11ee-958a-25db88c0abee","github_installation_id":42461852,"github_repository_owner":"ajm188","github_repository_name":"vitess","github_pr_num":14,"time":"2024-01-30T09:54:15-05:00","message":"No cobradocs changed for PR https://github.com/ajm188/vitess.git/vitess#14 at base release-18.0.

The log Skipping first commit ... was for the first commit on your PR that is just a merge?

The first commit (i.e. base branch) of the vitess PR, which would either be main or a release- branch


Background tasks pops up on all App Engine tasks

This happens even if there are no long running Cloud SDK related background tasks:

@ivanporty can we limit this to happen only when there is a Cloud SDK task to run?

Sure, we can do that. I think this is a duplicate of #2077?

oops. definitely a dup. closing


feat: devcontainer DynamoDB and Redis services

Summary
Update the devcontainer so that it includes a local Redis and DynamoDB service.  This is being done for cases where a LocalStack Pro license is not available.
Related

https://github.com/cds-snc/platform-forms-client/issues/4215

Closing as I'll be able to run the same setup as the rest of the team soon.


Change %taintsSection.detail% text

Version 2.2. (Master) 3/25/19
What kind of request is this (question/bug/enhancement/feature request):
Bug
Steps to reproduce (least amount of steps as possible):

Go to the node details page. Look at the taints seciton subtext.

Result:
This text is a bit confusing. Perhaps we should change it to something closer to what the k8s docs describe https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
We may need to ask someone for a better description per the k8s docs as the current text is confusing. Taints seem to affect nodes not necessarily pods, pods just need to match the toleration?

Please let me know the text. I will submit a PR to fix it.

@deniseschannon do you have any suggestions? or perhaps @vincent99 ?
I was thinking it could be worded better per the issue description (see k8s docs) it was a little confusing to me but I'm not great with k8s and am not sure the best text for this.

Ensure workloads only run on appropriate nodes by adding taints to nodes and tolerations to workloads?

I think that makes sense and is pretty clear ^^ @vincent99 @loganhz
How about we use that @loganhz ?

Version: Master (4/23/19)
Confirmed this is resolved. The suggested text is in place:


Bug: MQTT not reconnecting if HASS is not available on startup

Hi,
I have just figured out that HASS.Agent does not tying to reconnect MQTT when workstation was waked from hibernation, but Home Assistant API wasn't available at that moment.
The expected behavior would be to re-try connection after some time. But right now, if connection goes to error state, it would not restore again.
Can you please take a look? This re-connection functionality shouldn't be too complex for implementation.
Thanks!
[2023-07-07] HASS.Agent_20230709.log

Hello,
Please use the "Bug" template and update the issue and also include the logs from after the computer has been woken up from hibernation - this will speed up the investigation a lot.

I have uploaded the log file to initial message, hope this one can help (it's not yet with enabled extended option, I will upload one when this will happen again).
>> MQTT connection does not require any HA API to be available, unless you're using the Home Assistant Supervised/OS and using the MQTT Addon?
Yep, I'm using HASS VM with MQTT addon. HASS VM was offline in its entirety when my workstation have woken up from hibernation.


Overlay Error has returned

🐛 Summary
Website is operational but has a red overlay showing JavaScript errors when present.
To reproduce
Run the latest master branch
Expected behavior
No red overlay
Any helpful log output or screenshots

Fixed by https://github.com/cisagov/crossfeed/pull/1935


Bug fix for example_parsing_ops_test

Delay tensor allocation.
All tensor allocation should be after main() is entered.
https://isocpp.org/wiki/faq/ctors#static-init-order

Can one of the admins verify this patch?

Conflict resolved.
@mrry , can you review this?

Jenkins, test this please.


Add log for completion http request in logs

When the loglevel has been set to trace like so:
xray.Configure(xray.Config{LogLevel: "trace"})

One would expect information about when the request is actually sent to be added to the logs. However, the last thing that gets logged is the json representation of the trace. What I would like to see is additional information such as the ip address the xray client sdk tried to send information to and whether the response was successful or failure. Something as brief as "Connection to X.X.X.X with protocol udp/tcp was successful/unsuccessful" would be great! The usecase is to just have more debugging information available if the xray client sdk is misconfigured. Right now everything just works, but if I were to say misconfigure my firewall/security group/iptables or apply too strict of a selinux policy, it would be great to have the additional xray sdk client logs to help troubleshoot this.

Hello @jason-riddle , thank you for your suggestions. Currently, XRay Go SDK doesn't catch connection error so that we cannot log such information like you mentioned above. However, we should catch errors or exceptions if there is a udp connection error. I will add this feature into our backlog and make sure we will address it soon. Please stay tuned.

Hi @jason-riddle , we do have info level log statement if the udp is successfully dialed: https://github.com/aws/aws-xray-sdk-go/blob/master/xray/emitter.go#L43-L46
The SDK tries to resolve the daemon address during configuration and the config method return all the errors https://github.com/aws/aws-xray-sdk-go/blob/master/xray/config.go#L149 occurred during configuration. The caller can handle errors by logging it (and chose log level) or by panic and it is up to the user. Do you think this solution works better for you?

we do have info level log statement if the udp is successfully dialed: https://github.com/aws/aws-xray-sdk-go/blob/master/xray/emitter.go#L43-L46

There isn't a line 43-46 in this file. Also, I don't see a log statement anywhere in this file.

The SDK tries to resolve the daemon address during configuration and the config method return all the errors https://github.com/aws/aws-xray-sdk-go/blob/master/xray/config.go#L149

Line 149 links to a new line. I'm guessing you meant this line: https://github.com/aws/aws-xray-sdk-go/blob/9779f16ea01fdfc11c54971ed8547dbaa442bb42/xray/config.go#L144. I've added the commit hash just to be sure. That's nice that it returns all of the errors.

The caller can handle errors by logging it (and chose log level) or by panic and it is up to the user. Do you think this solution works better for you?

Are you asking if one approach is preferred over the other? If that's the case, I'd prefer logging as it's backwards compatible and the user doesn't have to add new code to handle the panic.


Assertion failed when running flow type-at-pos

Input file: a.js
$ flow --version
Flow, a static type checker for JavaScript, version 0.14.0
$ flow type-at-pos ./a.js 23 15
File "/private/tmp/frantic-lAdtia/a.js", line 23, characters 15-15:
File "src/typing/flow_js.ml", line 664, characters 9-15: Assertion failed

src/typing/flow_js.ml, line 664: | _ -> assert false (** TODO **)
Looks like this happens when the type of the variable I'm trying to inspect has a custom type (defined with type x = ...).

this is sorta-fixed in the upcoming 0.15. it will say (unknown) instead of crashing. tracking fixing the (unknown) in #463

^ fixed the (unknown) issue


Introduce method .hasCutHole()

## Description
With cut holes being introduced in 2019 (Samsung S10) it would be great to have a .hasCutHole() method that would return an Array with where the cut holes are located or an empty array if no cut holes are present. However, it might be more appropriate to maybe add this to the .hasNotch method. Returning an object describing if it is a cut hole or a notch and if it is a cut hole, specifying its location.
if left and right cut holes -> ['left', 'right'] // Mi Mix 4 patents show dual cut holes
if only left -> ['left']
no cut holes -> []
## Use Case
Some of my Apps hide the Status Bar and the newly introduced cut holes now interfere with some critical buttons.

Huh - I hadn't even seen that this was a thing :-). But it is, just a little non-displayable-area at the top right on the S10
This whole "your display is a rectangle" to "your display is a rectangle with 0 or more parts missing" seems like it needs a more generic solution than "notch", "punch-hole" (what the press is calling the S10 thing), or "multiple punch-holes" (like the maybe-Mi Mix 4)
Seems like the phrase "sensor areas" would describe what they are, then there should be a lookup takes a device brand/model and returns an array of sensor areas as well as their locations. Seems like people might want the actual pixel locations of the bounding box describing the sensor area as well as a vague description of "top-left" and "top-right" and "top-center" etc?
Then the existing hasNotch API could easily be mapped to it if we wanted backwards compatibility but whatever complicated thing Samsung or Mi Mix 4 wanted to do, we could handle easily?
Am I overthinking this?

No, I think its definitely in the right direction!
Sensor area sounds about right. And yes, I would love to have the width and height of the punch holes but since hasNotch only returned true or false, this was enough for me to correctly handle my app layout.
However, I am still a bit confused about how backward compatibility would work on .hasNotch without changing the return value?
Do you mean if hasNotch returns true a new method sensorAreas should describe the sensor areas on the phone?
Here you can see the layouts Xiaomi is considering.

I'm also discovering the thing, wow :D
About BC: we will be releasing 1.0.0 soon (#307) and adopting a strict semantical release after that.
So no harm would be done if we raise to 2.0
But I've got a feeling that after notch, cut hole, punch-hole, something else will soon emerge.
I also don't like at all how hasNotch() is implemented right now: a growing list of devices, sometimes it's the same, but spelled differently. I'd hate to see this replicated for punch hole.
Maybe it's time to rethink the static data structure a bit, and be able to handle hasNotch, hasCutHole, and all future derivatives from a single point.
const devicesFeatures = {
  `oneplus`: [
      {
          // one of the declination, easy to understand for us
          model: '6',
          // could be enhanced by dynamically removing the brand
          aliases: ['ONEPLUS A6000', 'ONEPLUS A6013', 'A6000']
          // hardcoded features, absence means false
          notch: true,
          foldable: true,
          punch: false,  // <- could be skipped
      },
      // ..
   ],
  // ..
}

WDYT?

I like that you already included the foldable prop 😃. I am definitely for it.

foldable, ha! But yes
That data structure or similar was what I was thinking yes. Then what I meant by "mapping" the previous API to it, is just to say that you could have "hasNotch" look in the data structure and if it sees notch: true, it could return true, to maintain that API.
I am not saying that backwards-compatibility is some specific goal - I don't personally care at this point as long as it respects semantic versioning, I was just thinking out loud that it should be possible if desired

There is upstream work for this - there's a PR that might land soon that would allow for layout that respects sensor clusters
https://github.com/facebook/react-native/pull/20999

This looks like it might be the way to go - it is a small lib focused on returning the set of display insets for a device https://github.com/th3rdwave/react-native-safe-area-context
I have not tested it but it may support punch-holes ('displayCutout' in Android API parlance https://developer.android.com/reference/android/view/Display#getCutout()) out of the box just not labeled specifically
@janicduplessis we have been thinking of a way to support our existing hasNotch() API here and extend to cover punch-holes and the like and I wonder if we're just duplicating your effort. Would you be interested in collaborating with the APIs in your repo or over here? Your implementation is far superior at the moment at minimum (actual device inspection + events vs a simple lookup table - no contest)

Hmm we could expose info about display cutouts in rn-safe-area-context. https://developer.android.com/reference/android/view/DisplayCutout.html#getBoundingRects() seems to be what we want. What is punch holes? Is it a different API?

@janicduplessis a 'punch hole' is what the tech press is calling these cutouts that are fully standalone with display between the edge and the cutout - the samsung galaxy s10 is an example https://image-us.samsung.com/us/smartphones/galaxy-s10/v2/design/images/galaxy-s10_design_amoled.jpg - I think the Android OS must handle them the same as any cutout though? As in they are not safe to display over but the display can sort of be stretched to fill them (for background) if desired? Because there's no fresh API to handle them in the Android APIs (I just looked prior to my comment before)
And yes - right now rn-safe-area-context is all about generating the safe area to draw on but if there was also sort of an inspection ability where you could ask for the list of EdgeInsets (I think that is your object name?) then it would cover everything I think?

This api I'm thinking of would actually return a list of Rect objects (top, left, width, height). This would be like DisplayCutout.getBoundingRects. I guess the use case for this is to draw around the cutouts?

The current react-native-device-info use case is mostly just to answer the question hasNotch()? (currently answered via a lookup table) https://github.com/react-native-community/react-native-device-info#hasNotch - it is very simplistic but important to people
A tangential requested use case is to support answering that question in emulators (which hints that the implementation should really not be a lookup table...) #732
Those two seem like easy-to-understand easy-to-handle APIs from react-native-safe-area-context
The request in this specific issue is to support '.hasCutHole()' but I think the real urge is just that they are named slightly differently so it's not clear that the underlying system doesn't care what you call them, they are all just boundedareas you can't safely draw on to near edges except backgrounds (at your option), right?
So if I understand that right then yes it's just getting the BoundingRects for any/all cutouts, and the use case is to alter layout and/or draw around them yes

Any updates on this?

The beauty of open source being that it is obvious when there are no updates ;-)
Propose a PR, ideally to react-native-safe-area-context (or see if there are updates there?) - but nothing happening here


Looking at Dynatest data

While writing the email, I was trying to give specific examples (to make sure that we would get a useful answer) and I find things that I was not sure how to describe/include in the email. Here are questions that you might be able to help me with:

Did you find a way to use the .fis and .srv files? A quick search in the LCMS_DataAnalyser_Manual.pdf indicates that it might be related, respectively, to the raw data and survey data. But these are binary files and I didn't find a way to open them...
We have the MFV-RSP-Manual.pdf and it describes the data contained in the .RSP files (.RSP probably means Road Surface Profilers) and also part of the datasets structure (page 170 and after), did you have a chance to look the data in the .RSP files?
Still for the .RSP it seems like there are some GPS positions but, as for the .GPX files, I didn't find ways to link it back to a xml files

I haven't sent any emails yet, so we can still add other questions in the email :-)

So sorry for the delay in response.

I have downloaded MATLAB and installed the corresponding package (fuzzy logic toolbox). However, when I try to open the .fis file, it still appears as garbled characters or blank. I'm unsure why this is happening. Based on the LCMS_DataAnalyser_Manual PDF, it seems that .fis and .srv files are primarily used with the provided functions for data analysis. However, since we already have the results of data analysis in the files we received, I guess these files are not needed for our website. Nevertheless, I am still working on trying to open them. I have come across some other software options for opening .fis  files, but they seem to be Windows-specific, so I'm still working on it.

Regarding the .rsp files, the PDF mentions, "The RspWin program produces files that are directly 'Importable' to most spreadsheet software and easily readable by dedicated software." So, I think it can be utilized. Within the .rsp files, I found data such as Date and Time / IRI that seems usable, but further research may be needed to apply it to the database.

Concerning the GPX files, there seems to be plenty of information online on how to directly display GPX file data on a map using React, as shown here (https://www.manuelkruisz.com/blog/posts/gpx-maps-react). Since we have GPX files available, I am not sure it's possible that we can directly use the data from GPX files without linking it back to XML files.

Thanks for all your hard work on this project! :-)