Analyzing spec algorithms

Context: algorithms extraction
Reffy now extracts algorithms from specs, see algorithms extracts in Webref. Extraction is far from perfect (a number of limitations are noted as TODO comments in the extraction logic) but the results are still good enough to start analyzing potential problems in algorithms, see study-algorithms.js.
Also see @dontcallmeDOM's initial algorithms explorer based on the extracts.
Analysis: parallel steps that should call "queue a task"
First thing I analyzed are steps that run in parallel because I always forget the need to queue a task within parallel steps before resolving/rejecting a Promise or firing an event. @jyasskin noted the same problem back in 2022 in https://github.com/whatwg/html/issues/8569. What's new here is that we can now compute that information automatically, giving us more weapons to play whack-a-mole.
The results show that about 50% of the specs (56/113) that define algorithms with "in parallel" steps need fixing:

Specs with steps in parallel that miss a call to "queue a task" (56 specs)

Accelerated Shape Detection in Images

The BarcodeDetector/getSupportedFormats() algorithm has a parallel step that resolves/rejects a promise directly

Audio Output Devices API

The HTMLMediaElement/setSinkId() algorithm has a parallel step that resolves/rejects a promise directly
The MediaDevices/selectAudioOutput() algorithm has a parallel step that resolves/rejects a promise directly

Background Fetch

The create record objects algorithm has a parallel step that resolves/rejects a promise directly
The get(id) algorithm has a parallel step that resolves/rejects a promise directly
The getIds() algorithm has a parallel step that resolves/rejects a promise directly
The abort() algorithm has a parallel step that resolves/rejects a promise directly
The updateUI(options) algorithm has a parallel step that resolves/rejects a promise directly

Contact Picker API

The getProperties() algorithm has a parallel step that resolves/rejects a promise directly

Content Index

The add(description) algorithm has a parallel step that resolves/rejects a promise directly
The delete(id) algorithm has a parallel step that resolves/rejects a promise directly
The getAll() algorithm has a parallel step that resolves/rejects a promise directly

Cookie Store API

The CookieStore/get(name) algorithm has a parallel step that resolves/rejects a promise directly
The CookieStore/get(options) algorithm has a parallel step that resolves/rejects a promise directly
The CookieStore/getAll(name) algorithm has a parallel step that resolves/rejects a promise directly
The CookieStore/getAll(options) algorithm has a parallel step that resolves/rejects a promise directly
The CookieStore/set(name, value) algorithm has a parallel step that resolves/rejects a promise directly
The CookieStore/set(options) algorithm has a parallel step that resolves/rejects a promise directly
The CookieStore/delete(name) algorithm has a parallel step that resolves/rejects a promise directly
The CookieStore/delete(options) algorithm has a parallel step that resolves/rejects a promise directly
The CookieStoreManager/subscribe(subscriptions) algorithm has a parallel step that resolves/rejects a promise directly
The CookieStoreManager/getSubscriptions() algorithm has a parallel step that resolves/rejects a promise directly
The CookieStoreManager/unsubscribe(subscriptions) algorithm has a parallel step that resolves/rejects a promise directly

Credential Management Level 1

The Request a Credential algorithm has a parallel step that resolves/rejects a promise directly
The Create a Credential algorithm has a parallel step that resolves/rejects a promise directly
The Prevent Silent Access algorithm has a parallel step that resolves/rejects a promise directly

CSS Object Model (CSSOM)

The CSSStyleSheet/replace(text) algorithm has a parallel step that resolves/rejects a promise directly

Element Capture

The BrowserCaptureMediaStreamTrack/restrictTo() algorithm has a parallel step that resolves/rejects a promise directly

Encrypted Media Extensions

The Navigator/requestMediaKeySystemAccess() algorithm has a parallel step that resolves/rejects a promise directly
The MediaKeySystemAccess/createMediaKeys() algorithm has a parallel step that resolves/rejects a promise directly
The MediaKeys/setServerCertificate() algorithm has a parallel step that resolves/rejects a promise directly
The MediaKeySession/generateRequest() algorithm has a parallel step that resolves/rejects a promise directly
The MediaKeySession/load() algorithm has a parallel step that resolves/rejects a promise directly
The MediaKeySession/update() algorithm has a parallel step that resolves/rejects a promise directly
The HTMLMediaElement/setMediaKeys() algorithm has a parallel step that resolves/rejects a promise directly

EyeDropper API

The EyeDropper/open() algorithm has a parallel step that resolves/rejects a promise directly

Federated Credential Management API

The attempt to disconnect algorithm has a parallel step that resolves/rejects a promise directly
The getUserInfo algorithm has a parallel step that resolves/rejects a promise directly

File System Access

The FileSystemHandle/queryPermission(descriptor) algorithm has a parallel step that resolves/rejects a promise directly
The FileSystemHandle/requestPermission(descriptor) algorithm has a parallel step that resolves/rejects a promise directly
The Window/showOpenFilePicker(options) algorithm has a parallel step that resolves/rejects a promise directly
The Window/showSaveFilePicker(options) algorithm has a parallel step that resolves/rejects a promise directly
The Window/showDirectoryPicker(options) algorithm has a parallel step that resolves/rejects a promise directly
The DataTransferItem/getAsFileSystemHandle() algorithm has a parallel step that resolves/rejects a promise directly

Fullscreen API Standard

The requestFullscreen(options) algorithm has a parallel step that resolves/rejects a promise directly
The exit fullscreen algorithm has a parallel step that resolves/rejects a promise directly

Gamepad

The GamepadHapticActuator/reset() algorithm has a parallel step that resolves/rejects a promise directly

Geolocation Sensor

The read algorithm has a parallel step that resolves/rejects a promise directly

Get Installed Related Apps API

The getInstalledRelatedApps() algorithm has a parallel step that resolves/rejects a promise directly

Handwriting Recognition API

The navigator-query-handwriting-recognizer algorithm has a parallel step that resolves/rejects a promise directly
The navigator-create-handwriting-recognizer algorithm has a parallel step that resolves/rejects a promise directly
The handwriting-drawing-get-prediction algorithm has a parallel step that resolves/rejects a promise directly

HTML Standard

The HTMLImageElement/decode() algorithm has a parallel step that resolves/rejects a promise directly
The WindowOrWorkerGlobalScope/createImageBitmap(image, options) algorithm has a parallel step that resolves/rejects a promise directly

Idle Detection API

The IdleDetector/requestPermission() algorithm has a parallel step that resolves/rejects a promise directly

Indexed Database API 3.0

The IDBFactory/databases() algorithm has a parallel step that resolves/rejects a promise directly

JS Self-Profiling API

The algorithm that starts with "Stops the profiler and returns a trace. This method MUST run these steps:" has a parallel step that resolves/rejects a promise directly

Keyboard Map

The keyboard-getlayoutmap algorithm has a parallel step that resolves/rejects a promise directly

Local Font Access API

The Window/queryLocalFonts(options) algorithm has a parallel step that resolves/rejects a promise directly
The FontData/blob() algorithm has a parallel step that resolves/rejects a promise directly

Managed Configuration API

The NavigatorManagedData/getManagedConfiguration(keys) algorithm has a parallel step that resolves/rejects a promise directly

Media Capabilities

The algorithm that starts with "The decodingInfo() method method MUST run the following steps:" has a parallel step that resolves/rejects a promise directly
The algorithm that starts with "The encodingInfo() method MUST run the following steps:" has a parallel step that resolves/rejects a promise directly

Media Capture and Streams

The MediaDevices/enumerateDevices() algorithm has a parallel step that resolves/rejects a promise directly
The MediaDevices/getUserMedia() algorithm has a parallel step that resolves/rejects a promise directly

Media Session

The update capture state algorithm algorithm has a parallel step that resolves/rejects a promise directly

MediaStream Image Capture

The ImageCapture/takePhoto(photoSettings) algorithm has a parallel step that resolves/rejects a promise directly
The ImageCapture/getPhotoCapabilities() algorithm has a parallel step that resolves/rejects a promise directly
The ImageCapture/getPhotoSettings() algorithm has a parallel step that resolves/rejects a promise directly
The ImageCapture/grabFrame() algorithm has a parallel step that resolves/rejects a promise directly

Payment Handler API

The algorithm that starts with "Upon receiving a PaymentRequest by way of PaymentRequest.show() and subsequent user selection of a payment handler, the user agent MUST run the following steps:" has a parallel step that resolves/rejects a promise directly
An algorithm has a parallel step that resolves/rejects a promise directly

Payment Request API

The algorithm that starts with "The show(optional detailsPromise) method MUST act as follows:" has a parallel step that resolves/rejects a promise directly
The algorithm that starts with "The complete() method MUST act as follows:" has a parallel step that resolves/rejects a promise directly
The can make payment algorithm algorithm has a parallel step that resolves/rejects a promise directly

Picture-in-Picture

The algorithm that starts with "The requestPictureInPicture() method, when invoked, MUST return a new promise promise and run the following steps in parallel:" has a parallel step that resolves/rejects a promise directly
The algorithm that starts with "The exitPictureInPicture() method, when invoked, MUST return a new promise promise and run the following steps in parallel:" has a parallel step that resolves/rejects a promise directly

Presentation API

The PresentationRequest/start() algorithm has a parallel step that resolves/rejects a promise directly
The PresentationRequest/reconnect() algorithm has a parallel step that resolves/rejects a promise directly
The PresentationRequest/getAvailability() algorithm has a parallel step that resolves/rejects a promise directly
The monitoring incoming presentation connections algorithm has a parallel step that resolves/rejects a promise directly

Region Capture

The BrowserCaptureMediaStreamTrack/cropTo() algorithm has a parallel step that resolves/rejects a promise directly

Remote Playback API

The RemotePlayback/prompt() algorithm has a parallel step that resolves/rejects a promise directly

Screen Capture

The MediaDevices/getDisplayMedia() algorithm has a parallel step that resolves/rejects a promise directly

Screen Orientation

The apply orientation lock algorithm has a parallel step that resolves/rejects a promise directly

Service Workers Nightly

The navigator-service-worker-getRegistration algorithm has a parallel step that resolves/rejects a promise directly
The navigation-preload-manager-enable algorithm has a parallel step that resolves/rejects a promise directly
The navigation-preload-manager-disable algorithm has a parallel step that resolves/rejects a promise directly
The navigation-preload-manager-setheadervalue algorithm has a parallel step that resolves/rejects a promise directly
The navigation-preload-manager-getstate algorithm has a parallel step that resolves/rejects a promise directly
The service-worker-global-scope-skipwaiting algorithm has a parallel step that resolves/rejects a promise directly
The client-postmessage-options algorithm has a parallel step that fires an event directly
The clients-get algorithm has a parallel step that resolves/rejects a promise directly
The clients-claim algorithm has a parallel step that resolves/rejects a promise directly
The cache-match algorithm has a parallel step that resolves/rejects a promise directly
The cache-matchall algorithm has a parallel step that resolves/rejects a promise directly
The cache-storage-match algorithm has a parallel step that resolves/rejects a promise directly
The cache-storage-has algorithm has a parallel step that resolves/rejects a promise directly
The cache-storage-open algorithm has a parallel step that resolves/rejects a promise directly
The cache-storage-delete algorithm has a parallel step that resolves/rejects a promise directly
The cache-storage-keys algorithm has a parallel step that resolves/rejects a promise directly
The Handle Fetch algorithm has a parallel step that resolves/rejects a promise directly

Test Utils Standard

The algorithm that starts with "The gc() method must run these steps:" has a parallel step that resolves/rejects a promise directly

The Capture-Handle Actions Mechanism

The MediaStreamTrack/sendCaptureAction() algorithm has a parallel step that resolves/rejects a promise directly

Viewport Capture

The MediaDevices/getViewportMedia() algorithm has a parallel step that resolves/rejects a promise directly

VirtualKeyboard API

The VirtualKeyboard/show() algorithm has a parallel step that fires an event directly
The VirtualKeyboard/hide() algorithm has a parallel step that fires an event directly

Web Background Synchronization

The SyncManager/register(tag) algorithm has a parallel step that resolves/rejects a promise directly
The SyncManager/getTags() algorithm has a parallel step that resolves/rejects a promise directly
The fire a sync event algorithm has a parallel step that fires an event directly

Web Bluetooth

The getDevice invocation algorithm has a parallel step that resolves/rejects a promise directly
The requestDevice invocation algorithm has a parallel step that resolves/rejects a promise directly
The query Bluetooth cache algorithm has a parallel step that resolves/rejects a promise directly
The BluetoothRemoteGATTServer connect algorithm has a parallel step that resolves/rejects a promise directly
The BluetoothRemoteGATTService construction algorithm has a parallel step that resolves/rejects a promise directly
The BluetoothRemoteGATTCharacteristic constructor algorithm has a parallel step that resolves/rejects a promise directly
The BluetoothRemoteGATTCharacteristic readValue() algorithm has a parallel step that resolves/rejects a promise directly
The Write Characteristic value algorithm has a parallel step that resolves/rejects a promise directly
The BluetoothRemoteGATTCharacteristic startNotifications algorithm has a parallel step that resolves/rejects a promise directly
The BluetoothCharacteristicProperties constructor algorithm has a parallel step that resolves/rejects a promise directly
The BluetoothRemoteGATTDescriptor constructor algorithm has a parallel step that resolves/rejects a promise directly
The BluetoothRemoteGATTDescriptor readValue algorithm has a parallel step that resolves/rejects a promise directly
The BluetoothRemoteGATTDescriptor writeValue algorithm has a parallel step that resolves/rejects a promise directly

Web Bluetooth Scanning

The Bluetooth/requestLEScan(options) algorithm has a parallel step that resolves/rejects a promise directly

Web Cryptography API

The SubtleCrypto/encrypt() algorithm has a parallel step that resolves/rejects a promise directly
The SubtleCrypto/sign() algorithm has a parallel step that resolves/rejects a promise directly
The SubtleCrypto/digest() algorithm has a parallel step that resolves/rejects a promise directly
The SubtleCrypto/generateKey() algorithm has a parallel step that resolves/rejects a promise directly
The SubtleCrypto/deriveKey() algorithm has a parallel step that resolves/rejects a promise directly
The SubtleCrypto/deriveBits() algorithm has a parallel step that resolves/rejects a promise directly
The SubtleCrypto/importKey() algorithm has a parallel step that resolves/rejects a promise directly
The SubtleCrypto/exportKey() algorithm has a parallel step that resolves/rejects a promise directly
The SubtleCrypto/wrapKey() algorithm has a parallel step that resolves/rejects a promise directly
The SubtleCrypto/unwrapKey() algorithm has a parallel step that resolves/rejects a promise directly

Web NFC

The NDEFReader/write() algorithm has a parallel step that resolves/rejects a promise directly
The NDEFReader/makeReadOnly() algorithm has a parallel step that resolves/rejects a promise directly
The clean up the pending scan algorithm has a parallel step that resolves/rejects a promise directly

Web Periodic Background Synchronization

The Process periodic sync registrations algorithm has a parallel step that fires an event directly

WebCodecs

The ImageDecoder/isTypeSupported(type) algorithm has a parallel step that resolves/rejects a promise directly

WebDriver

The algorithm that starts with "The remote end steps, given session, URL variables and parameters are:" has a parallel step that resolves/rejects a promise directly
The algorithm that starts with "The remote end steps, given session, URL variables and parameters are:" has a parallel step that resolves/rejects a promise directly

WebHID API

The algorithm that starts with "The forget() method steps are:" has a parallel step that resolves/rejects a promise directly

WebRTC Encoded Transform

The algorithm that starts with "The SFrame transform algorithm, given sframe as a SFrameTransform object and frame, runs these steps:" has a parallel step that resolves/rejects a promise directly
The generate key frame algorithm algorithm has a parallel step that resolves/rejects a promise directly
The send request key frame algorithm algorithm has a parallel step that resolves/rejects a promise directly

WebRTC: Real-Time Communication in Browsers

The RTCPeerConnection/generateCertificate() algorithm has a parallel step that resolves/rejects a promise directly
The RTCRtpSender/replaceTrack() algorithm has a parallel step that resolves/rejects a promise directly
The RTCRtpSender/getStats() algorithm has a parallel step that resolves/rejects a promise directly
The RTCRtpReceiver/getStats() algorithm has a parallel step that resolves/rejects a promise directly
The RTCPeerConnection/getStats() algorithm has a parallel step that resolves/rejects a promise directly

WebTransport

The WebTransport/getStats() algorithm has a parallel step that resolves/rejects a promise directly
The WebTransport/createBidirectionalStream(options) algorithm has a parallel step that resolves/rejects a promise directly
The WebTransport/createUnidirectionalStream(options) algorithm has a parallel step that resolves/rejects a promise directly

WebUSB API

The algorithm that starts with "The getDevices() method, when invoked, MUST return a new Promise and run the following steps in parallel:" has a parallel step that resolves/rejects a promise directly
The request the "usb" permission algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.open() algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.close() algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.forget() algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.selectConfiguration(configurationValue) algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.claimInterface(interfaceNumber) algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.releaseInterface(interfaceNumber) algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.selectAlternateInterface(interfaceNumber, alternateSetting) algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.controlTransferIn(setup, length) algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.controlTransferOut(setup, data) algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.clearHalt(direction, endpointNumber) algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.transferIn(endpointNumber, length) algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.transferOut(endpointNumber, data) algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.isochronousTransferIn(endpointNumber, packetLengths) algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.isochronousTransferOut(endpointNumber, data, packetLengths) algorithm has a parallel step that resolves/rejects a promise directly
The USBDevice.reset() algorithm has a parallel step that resolves/rejects a promise directly

WebXR Device API

The session-supported algorithm has a parallel step that resolves/rejects a promise directly
The request-reference-space algorithm has a parallel step that resolves/rejects a promise directly

To run the analysis and refresh the above results, retrieve Webref's latest crawl results data locally and run:
npx strudy inspect [pathto]/webref --what algorithms --structure type/spec --sort default/title > res.md

On top of these problems (Edit: both examples have been fixed):

The example on using AbortController and AbortSignal objects in the DOM spec seems wrong: step 3.2 should queue a task to resolve p with amazingResult (but note step 2.3.2 seems correct, because abort steps run within a global task)
(Edit: reported in https://github.com/whatwg/dom/issues/1300)
The parallel queue example in the Parallelism section in HTML seems wrong as well: steps 2.4 in both the "incorrect" and "correct" versions should also queue a task, the former because the step explicitly runs "in parallel" in step 2, the latter because steps enqueued to a parallel queue also run in parallel, I think.
(Edit: reported in https://github.com/whatwg/html/issues/10535)

We could start reporting these problems to spec authors on a semi-automated basis (after review) as we do for broken links. Now, as noted by @jyasskin in https://github.com/whatwg/html/issues/8569, "queue a task" means selecting a task source, which is also something that people get confused about (looking into it, I realize that I raised https://github.com/w3ctag/design-principles/issues/38 on this topic back in 2016), and often choose to ignore.
Given the number of specs that get it wrong, two questions that I'm wondering about:

Would it be better to consider less error prone ways to write such steps first?
Do these problems materialize in interoperability bugs? Should we rather keep that in the back burner if it does not have practical implications?

Further analyses
Additional analyses that could be done:

Look into task sources as well. Whether a task source is defined is a bit harder to evaluate automatically because various specs have a blanket "use the foo task source for all tasks in this specification" statement; but the name of the task source is usually wrapped in a <dfn>, which should be easy to detect.
Report terms in steps that should link to their formal definition (e.g., "in parallel", "queue a task") and that don't.
Look into phrasing conventions when defining an algorithm and calling an algorithm, to progressively converge on a similar pseudo-language for algorithms. See how to define an algorithm in Infra for the definition part.
Start tracking algorithm variables, inputs and outputs. That probably requires settling down on stricter conventions first.
Analyze the list of step operations. In the extraction logic, the list is used to assess that a list item is indeed part of an algorithm. It reveals the many verbs that specs use to tell implementers what needs to be done. Some have formal definitions. The meaning of others is more fuzzy. Ideally, the meaning of all operations would be normatively defined somewhere.

Any other interesting or important analyses that could be worth looking into?
How to write/flag algorithms
More broadly speaking, algorithms are currently written with semi-formal structures. There are opportunities to converge on a more formal structure if that seems useful. Examples:

Infra describes how to define an algorithm. Specs use other variants here and there, such as "When the foo() method is invoked, run the following steps...", or algorithms sections where the sub-heading's title is the name of the algorithm.
Specs and spec authoring tools follow different conventions with regards to flagging these algorithms, e.g., with a class="algorithm" attribute. This makes it harder to extract the relevant prose, algorithm name, and actual steps.
Algorithm steps may be more or less atomic. Sometimes, a single step performs multiple operations (e.g., if/then/else all in one sentence). Related discussions about "inlining" steps in HTML in https://github.com/whatwg/html/issues/10049

On top of us raising issues afterwards, spec authoring tools could perhaps better guide spec authors at the authoring step. Additional classes could also help make algorithms visually more readable, as attempted with flowchart symbols in the algorithms explorer.
In any case, convergence would make it easier to extract the algorithms and run further analyses.

Status update:

Strudy was re-written to ease semi-automatic reporting of various types of anomalies.
This made it possible to file "missing "ueue a task" issues against specifications, which we've now started to do. That may take a bit of time but anomalies reported above should end up being reported.
I sent a message to spec-prod to initiate a larger discussion about algorithms


Get stage info

First pass at getting stage info
NB that this only parses "major" stage, i.e. I, II, III, IV, not subtypes (e.g. IA, IB)

Add different terms for IA, IB, etc

Done

@pnrobinson I think we can merge this if it looks okay to you


how to solder pro micro.

Hi, thanks for this great project, i am trying to build it, but being noob in electrical stuff wouldn't be able to figure out how to solder the pro micro, as it is solder on the back of the pcb, with switches on on top it(on front side), can share some pictures of give some suggestion on how to do it.

Hello! Hope this helps you: https://imgur.com/gallery/7m69tiw
If you have any questions, you can write to me at the contacts in my profile.
Don't forget to share a photo of the result 😉


Added make command to format all/changed files

Description:
Added the formatting commands (format and format_diff) to use the ruff formatting tool.
Issue: https://github.com/daxa-ai/pebblo/issues/341
Sample output:

cc: @srics


auth: more granular access control

Enable more granular access control for roles.

This is no longer valid with v3.  Closing.


[modify] twitter post when merge branch

Coverage decreased (-15.2%) to 63.108% when pulling f9378b09eb15a6201ca9c65131b939702296864d on itowtips:fix-sns-poster into 1131f3f9626c98c541ac823c869e9a64500d4341 on shirasagi:master.


python3Packages.tatsu: 5.5.0 -> 5.6.0

Motivation for this change
build is broken on staging-next
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 117223 at d8646dae run on x86_64-linux 1

4 packages built successfully:

python38Packages.ics
python38Packages.tatsu
python39Packages.ics
python39Packages.tatsu


Add support for promises in route functions

This enables users to work with promises in route function handlers, e.g.:
this.get('/users', () => {
  return new Promise(resolve => {
    resolve(new Response(200, { 'Content-Type': 'text/csv' }, 'firstname,lastname\nbob,dylan'));
  });
});

Bumped min pretender to 1.4.1 which does not throw errors on overlapping pretender servers.

Looks great!


build(version): bump version to v2.0.1

PR Checklist

[ ] Unit Tests have been added for new changes
[ ] API tests have been updated if applicable
[ ] All commented code has been removed
[ ] If you've added a dependency, you've ensured license is compatible with Apache 2.0 and clearly outlined the added dependency.

What are you changing?

Anything the reviewer should know when reviewing this PR?
If the there are associated PRs in other repositories, please link them here (i.e. open-amt-cloud-toolkit/repo#365 )

Codecov Report

Merging #377 (43fd3f3) into main (37a47a6) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #377   +/-   ##
=======================================
  Coverage   58.14%   58.14%
=======================================
  Files          45       45
  Lines        2602     2602
  Branches      605      605
=======================================
  Hits         1513     1513
  Misses        918      918
  Partials      171      171

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 37a47a6...43fd3f3. Read the comment docs.


Adding a unit test suite

Adds a unit test suite
Closes https://github.com/beeware/briefcase-template/issues/19
This PR:

[x] Adds a tox.ini configuration file
[x] Adds a GitHub Actions config to run CI tests
[x] Tests that pyproject.toml can be parsed without error
[x] Tests that there are no flake8 errors raised on any of the generated python code
[x] Tests that all the generated python files can be compiled using py_compile
[x] Tests that the filename tree structure of the generated app matches what we expect

STILL TO DO:

[x] Expand to test all GUI frameworks (PySide, PursuedPyBear, etc), not just the default Done!
[x] Replace os.path functions with pathlib where appropriate

I had assumed adding support for all the GUI frameworks would be as easy as adding a couple more fixtures and @pytest.mark.parametrize to all the tests -- but it seems like you can't pass fixtures into parametrize like that. Any suggestions?

I've addressed all the points you made, thank you for the review!
Note that:

I didn't add licenses other than the default to our test cases, primarily because all that changes is the contents of a single text file & there is no test checking that content currently. So as things stand now, it's a little meaningless.
The fixture has a function-scope. Maybe that's slightly inefficient but on the whole it's fine because the 15 tests run in under 13 seconds. (That does bug me a little, I wanted to make it class scoped then pack the three tests into a single class & parameterize that. But I couldn't get that working nicely with the argument passing,)


Possible dev env bug, and some questions

secret_sdk version: Main
Python version: 3.10.2
Operating System: Manjaro Linux (Arch based)

Intro
Hello, I've reviewed and tested this library and I'm finding the code quality and developer experience to be quite good. You guys are off to a great start on this.
Description
I had to upgrade uvloop and run poetry lock to get the deps to install.

This might be a bug that needs a PR

What I Did

git clone (secret-sdk-python)
poetry install
uvloop error traceback
LMK if you want me to try things to reproduce but, I've solved the problem by upgrading the project to the latest uvloop stable version.

Additionally
It appears this library is not Secret specific. It's possibly a superset.  Which is actually a great feature. I was able to early exit this function and get the client to connect to an osmosis node. Presumably because all IBC interfaces are the same (similar?). I'm working on a project that requires interacting with multiple cosmos chains. This client appears to have enough of what I need to do so.
Can you tell me more about what this function does? It doesn't appear be required to make requests to the scrt network. Can we make it optional so that I can use this lib as a generic cosmos client in addition to the scrt specific features it offers?
Thanks,
Nick

Thanks for the notice on uvloop.
As for the function, you can read up on the encryption details here.
As of now, we have no interest in this being a generic cosmos client, but feel free to fork the repo and make your own edits locally.

Works for me. Glad I could provide some help on the UV loop issue.
Closing this.

Also, great job on this.


Two trix_editor fields onto the same model?

Can you successfully scaffold two trix_editor fields onto the same model and have them both work? (Reported by @davidcolby.)

The specific issue here is multiple trix editors on the same page in different forms for the same resource. Our example was a Post index view, where each Post has a Comment form that includes a trix_editor field for the comment's body attribute. Regardless of which comment form you submit, the value of the body param in the form submission is pulled from the first trix editor on the page.
You can fix this by assigning a unique id to each instance of the body editors on the page.

Hey @DavidColby, I just want to make sure I understand this one correctly.
It sounds like the Comment form setup wasn't directly from the vanilla Bullet Train bin/super-scaffold crud command, I'm guessing you're using a custom nested form after initially scaffolding the models? So for Post on the index page you're just showing the body (as opposed to the trix editor) and then have a trix editor to create a Comment attached to each post on the page, is that correct?
Can I ask how you set up the form? Did you use anything like accepts_nested_attributes_for? Would love to write a fix, just haven't wrapped my head around this one yet.

@andrewculver I tried adding  two trix_editor fields onto the same model using super-scaffold and it is working perfectly.

Hey @DavidColby, I just want to make sure I understand this one correctly.
It sounds like the Comment form setup wasn't directly from the vanilla Bullet Train bin/super-scaffold crud command, I'm guessing you're using a custom nested form after initially scaffolding the models? So for Post on the index page you're just showing the body (as opposed to the trix editor) and then have a trix editor to create a Comment attached to each post on the page, is that correct?
Can I ask how you set up the form? Did you use anything like accepts_nested_attributes_for? Would love to write a fix, just haven't wrapped my head around this one yet.

@DavidColby Can you help us to reproduce this error by replying this message? Your help would be highly appreciated


Properly close open file handlers with PYVIPS image backend

Fixes #237

The PR looks good to me. Properly closing the self._image in the PyVipsSlide backend fixes the issue. The PyVipsSlide does not have proper tests in tests/test_backend.py. If we want PyVipsSlide as the main backend for dlup, the tests should be similar to the tests on OpenSlideSlide and TifffileSlide. I do recall that there were some issues with testing of the PyVipsSlide backend before. It would be interesting to discuss here and to hear your input.

There is a 73% coverage for the pyvips backend through other tests. There was a difficulty testing because the tests were still written for OpenSlide. Now this has been improved. To get 100% test coverage we would need to monkey-patch some parts. The major important parts are tested.

Thanks for the review Bart


Error when editing entities.

Describe the bug
It's a simple KOA server with TypeORM entities.
When I edit any resource, an insert command instead of an update runs a give an error:
Installed libraries and their versions
"admin-bro": "^3.3.1",
"@admin-bro/koa": "^1.1.1",
"@admin-bro/typeorm": "^1.4.0",
To Reproduce
Steps to reproduce the behavior:

Go to any edit and change any data
Click on save
See error and on terminal de wrong SQL that has run.

Expected behavior
Edit ends with success
Terminal
19:05:44 DEBUG koa Request POST /admin/api/resources/Organization/records/1/edit
query failed: INSERT INTO organization(id, version, created_at, created_by, updated_by, updated_at, deleted_at, deleted_by, type, name) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) -- PARAMETERS: ["1",1,"2020-12-15T22:04:05.617Z",1,0,"2020-12-15T22:04:05.617Z",null,0,"sponsor","Samsung1"]
error: Error: Duplicate entry '1' for key 'PRIMARY'
at Packet.asError (/Users/wagner-lagoasoft/dev/mmw/monorepo/node_modules/mysql2/lib/packets/packet.js:712:17)
at Query.execute (/Users/wagner-lagoasoft/dev/mmw/monorepo/node_modules/mysql2/lib/commands/command.js:28:26)
at PoolConnection.handlePacket (/Users/wagner-lagoasoft/dev/mmw/monorepo/node_modules/mysql2/lib/connection.js:425:32)
at PacketParser.onPacket (/Users/wagner-lagoasoft/dev/mmw/monorepo/node_modules/mysql2/lib/connection.js:75:12)
at PacketParser.executeStart (/Users/wagner-lagoasoft/dev/mmw/monorepo/node_modules/mysql2/lib/packet_parser.js:75:16)
at Socket. (/Users/wagner-lagoasoft/dev/mmw/monorepo/node_modules/mysql2/lib/connection.js:82:25)
at Socket.emit (events.js:315:20)
at addChunk (_stream_readable.js:295:12)
at readableAddChunk (_stream_readable.js:271:9)
at Socket.Readable.push (_stream_readable.js:212:10)
at TCP.onStreamRead (internal/stream_base_commons.js:186:23) {
code: 'ER_DUP_ENTRY',
errno: 1062,
sqlState: '23000',
sqlMessage: "Duplicate entry '1' for key 'PRIMARY'"
}
AdminBroOptions with schema
branding: {
companyName: 'Company',
},

I have this problem with nest.
My libraries:
"admin-bro": "^ 3.3.1",
"@ admin-bro / nestjs": "^ 1.0.0",
"@ admin-bro / typeorm": "^ 1.4.0",

I have this problem with nest.
My libraries:
"admin-bro": "^ 3.3.1",
"@ admin-bro / nestjs": "^ 1.0.0",
"@ admin-bro / typeorm": "^ 1.4.0",

i have same issue...
but i think that this is not admin-bro issue
If you are using the primary column as a bigint, you should apply the correct transformer.
https://github.com/typeorm/typeorm/issues/2400
export const bigint: ValueTransformer = {
to: (entityValue: number) => entityValue,
from: (databaseValue: string): number => parseInt(databaseValue, 10)
}
@Entity()
export class User {
@Generated('increment')
@PrimaryColumn('bigint', { transformer: [bigint] })
id: number;
}

I have same issue
  @PrimaryGeneratedColumn({
    type: 'bigint',
    name: 'id',
    comment: 'ID',
    unsigned: true,
  })
  id: string;

while editing , i got a error
{
// some other info
    "errors": {
      "undefined": {
        "type": "QueryFailedError",
        "message": "ER_DUP_ENTRY: Duplicate entry '8' for key 'PRIMARY'"
      }
    },
    "id": "8",
}

Closing because the issue is old. If you still need help, please reopen.

This is still an issue today. Can't save an edit for any entity with a bigint.
A sequelize column specified as this will produce a validation error:
@AllowNull(false)
@Column(DataType.BIGINT)
public_id!: bigint;```


Fix condition order

Hi to all 👋
I've found a bug when trying to replace a condition (putResponse).
The problem is that the responses are sorted by their score and then by equality.
If they have the same score, there is no other property that helps to sort the items.
Imagine you have three responses:
val fixture = FixtureDispatcher
with(PathQueryConditionFactory("/")) {
    fixture.putResponse(withPathSuffix("login"), "login_success")
    fixture.putResponse(withPathSuffix("me"), "me_success")
    fixture.putResponse(withPathSuffix("favourites"), "favourites_success")
}

If then we want to replace the login path with
fixture.putResponse(
    PathQueryConditionFactory("/").withPathSuffix("login"),
    "login_failure"
)

what happens is that another login path item is added at the end of the constantResponses of the FixtureDispatcher.
Solution: by adding these changes, the key is always found when using put in the TreeMap.
Maybe you could add some tests to check that replacing a response is done successfully.
PD: Thanks for this lib! 🙂

Could add a unit test for that?
Look at the existing tests for reference.

Could add a unit test for that?
Look at the existing tests for reference.

Sure, done! Let me know if it's good to you 🙂


Declutter global scope

This PR declutters global scope a bit more by adding a function to get the prognostic state.

bors r+


MultiConnection not usable for queries containing Selectable or boxed queries.

Setup
Versions

Rust:
Diesel: e548447
Database:
Operating System

Feature Flags

diesel: sqlite postgres

Problem Description
What are you trying to accomplish?
I've tried the following code:
use diesel::prelude::*;

table! {
    users {
        id -> Integer,
        name -> Text,
    }
}

#[derive(Selectable, Queryable)]
struct User {
    id: i32,
}

#[derive(diesel::MultiConnection)]
enum DbConnection {
    Sqlite(SqliteConnection),
    Pg(PgConnection)
}

fn main() {
    let conn = &mut DbConnection::establish("…").unwrap();

    let u = users::table.into_boxed().load::<(i32, String)>(conn);

    let u = users::table
        .select(User::as_select())
        .load(conn);
}

What is the expected output?
Code compiles
What is the actual output?
Code does not compile:
error[E0277]: the trait bound `BoxedSelectStatement<'_, (diesel::sql_types::Integer, diesel::sql_types::Text), FromClause<users::table>, multi_connection_impl::backend::MultiBackend>: QueryFragment<Sqlite>` is not satisfied
    --> src/main.rs:24:61
     |
24   |     let u = users::table.into_boxed().load::<(i32, String)>(conn);
     |                                       ----                  ^^^^ the trait `QueryFragment<Sqlite>` is not implemented for `BoxedSelectStatement<'_, (diesel::sql_types::Integer, diesel::sql_types::Text), FromClause<users::table>, multi_connection_impl::backend::MultiBackend>`
     |                                       |
     |                                       required by a bound introduced by this call
     |
     = help: the following other types implement trait `QueryFragment<DB, SP>`:
               <BoxedSelectStatement<'a, ST, QS, DB, GB> as QueryFragment<DB, AnsiSqlSelectStatement>>
               <BoxedSelectStatement<'a, ST, QS, DB, GB> as QueryFragment<DB>>
               <BoxedSelectStatement<'a, ST, QS, multi_connection_impl::backend::MultiBackend, GB> as QueryFragment<multi_connection_impl::backend::MultiBackend, MultiSelectStatementSyntax>>
note: required for `BoxedSelectStatement<'_, (Integer, Text), FromClause<table>, MultiBackend>` to implement `QueryFragment<multi_connection_impl::backend::MultiBackend, MultiSelectStatementSyntax>`
    --> src/main.rs:15:10
     |
15   | #[derive(diesel::MultiConnection)]
     |          ^^^^^^^^^^^^^^^^^^^^^^^ unsatisfied trait bound introduced in this `derive` macro
     = note: the full type name has been written to '/tmp/diesel_test/target/debug/deps/diesel_test-e9839cd4beffff40.long-type-6862964700214725501.txt'
     = note: 1 redundant requirement hidden
     = note: required for `BoxedSelectStatement<'_, (Integer, Text), FromClause<table>, MultiBackend>` to implement `QueryFragment<multi_connection_impl::backend::MultiBackend>`
     = note: the full type name has been written to '/tmp/diesel_test/target/debug/deps/diesel_test-e9839cd4beffff40.long-type-17859806514065189679.txt'
     = note: required for `BoxedSelectStatement<'_, (Integer, Text), FromClause<table>, MultiBackend>` to implement `LoadQuery<'_, DbConnection, (i32, String)>`
     = note: the full type name has been written to '/tmp/diesel_test/target/debug/deps/diesel_test-e9839cd4beffff40.long-type-17859806514065189679.txt'
note: required by a bound in `diesel::RunQueryDsl::load`
    --> /home/weiznich/Documents/rust/diesel/diesel/src/query_dsl/mod.rs:1543:15
     |
1543 |         Self: LoadQuery<'query, Conn, U>,
     |               ^^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `RunQueryDsl::load`
     = note: this error originates in the derive macro `diesel::MultiConnection` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0277]: the trait bound `BoxedSelectStatement<'_, (diesel::sql_types::Integer, diesel::sql_types::Text), FromClause<users::table>, multi_connection_impl::backend::MultiBackend>: QueryFragment<Pg>` is not satisfied
    --> src/main.rs:24:61
     |
24   |     let u = users::table.into_boxed().load::<(i32, String)>(conn);
     |                                       ----                  ^^^^ the trait `QueryFragment<Pg>` is not implemented for `BoxedSelectStatement<'_, (diesel::sql_types::Integer, diesel::sql_types::Text), FromClause<users::table>, multi_connection_impl::backend::MultiBackend>`
     |                                       |
     |                                       required by a bound introduced by this call
     |
     = help: the following other types implement trait `QueryFragment<DB, SP>`:
               <BoxedSelectStatement<'a, ST, QS, DB, GB> as QueryFragment<DB, AnsiSqlSelectStatement>>
               <BoxedSelectStatement<'a, ST, QS, DB, GB> as QueryFragment<DB>>
               <BoxedSelectStatement<'a, ST, QS, multi_connection_impl::backend::MultiBackend, GB> as QueryFragment<multi_connection_impl::backend::MultiBackend, MultiSelectStatementSyntax>>
note: required for `BoxedSelectStatement<'_, (Integer, Text), FromClause<table>, MultiBackend>` to implement `QueryFragment<multi_connection_impl::backend::MultiBackend, MultiSelectStatementSyntax>`
    --> src/main.rs:15:10
     |
15   | #[derive(diesel::MultiConnection)]
     |          ^^^^^^^^^^^^^^^^^^^^^^^ unsatisfied trait bound introduced in this `derive` macro
     = note: the full type name has been written to '/tmp/diesel_test/target/debug/deps/diesel_test-e9839cd4beffff40.long-type-6862964700214725501.txt'
     = note: 1 redundant requirement hidden
     = note: required for `BoxedSelectStatement<'_, (Integer, Text), FromClause<table>, MultiBackend>` to implement `QueryFragment<multi_connection_impl::backend::MultiBackend>`
     = note: the full type name has been written to '/tmp/diesel_test/target/debug/deps/diesel_test-e9839cd4beffff40.long-type-17859806514065189679.txt'
     = note: required for `BoxedSelectStatement<'_, (Integer, Text), FromClause<table>, MultiBackend>` to implement `LoadQuery<'_, DbConnection, (i32, String)>`
     = note: the full type name has been written to '/tmp/diesel_test/target/debug/deps/diesel_test-e9839cd4beffff40.long-type-17859806514065189679.txt'
note: required by a bound in `diesel::RunQueryDsl::load`
    --> /home/weiznich/Documents/rust/diesel/diesel/src/query_dsl/mod.rs:1543:15
     |
1543 |         Self: LoadQuery<'query, Conn, U>,
     |               ^^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `RunQueryDsl::load`
     = note: this error originates in the derive macro `diesel::MultiConnection` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0277]: the trait bound `LimitOffsetClause<diesel::query_builder::limit_clause::NoLimitClause, diesel::query_builder::offset_clause::NoOffsetClause>: IntoBoxedClause<'_, multi_connection_impl::backend::MultiBackend>` is not satisfied
  --> src/main.rs:24:26
   |
24 |     let u = users::table.into_boxed().load::<(i32, String)>(conn);
   |                          ^^^^^^^^^^ the trait `IntoBoxedClause<'_, multi_connection_impl::backend::MultiBackend>` is not implemented for `LimitOffsetClause<diesel::query_builder::limit_clause::NoLimitClause, diesel::query_builder::offset_clause::NoOffsetClause>`
   |
   = help: the trait `IntoBoxedClause<'a, Sqlite>` is implemented for `LimitOffsetClause<diesel::query_builder::limit_clause::NoLimitClause, diesel::query_builder::offset_clause::NoOffsetClause>`
   = note: required for `SelectStatement<FromClause<users::table>>` to implement `BoxedDsl<'_, multi_connection_impl::backend::MultiBackend>`

error[E0277]: the trait bound `LimitOffsetClause<diesel::query_builder::limit_clause::NoLimitClause, diesel::query_builder::offset_clause::NoOffsetClause>: IntoBoxedClause<'_, multi_connection_impl::backend::MultiBackend>` is not satisfied
    --> src/main.rs:24:26
     |
24   |     let u = users::table.into_boxed().load::<(i32, String)>(conn);
     |                          ^^^^^^^^^^ the trait `IntoBoxedClause<'_, multi_connection_impl::backend::MultiBackend>` is not implemented for `LimitOffsetClause<diesel::query_builder::limit_clause::NoLimitClause, diesel::query_builder::offset_clause::NoOffsetClause>`
     |
     = help: the trait `IntoBoxedClause<'a, Sqlite>` is implemented for `LimitOffsetClause<diesel::query_builder::limit_clause::NoLimitClause, diesel::query_builder::offset_clause::NoOffsetClause>`
     = note: required for `SelectStatement<FromClause<users::table>>` to implement `BoxedDsl<'_, multi_connection_impl::backend::MultiBackend>`
     = note: 1 redundant requirement hidden
     = note: required for `users::table` to implement `BoxedDsl<'_, multi_connection_impl::backend::MultiBackend>`
note: required by a bound in `diesel::QueryDsl::into_boxed`
    --> /home/weiznich/Documents/rust/diesel/diesel/src/query_dsl/mod.rs:1299:15
     |
1299 |         Self: methods::BoxedDsl<'a, DB>,
     |               ^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `QueryDsl::into_boxed`

error[E0277]: the trait bound `SelectBy<User, multi_connection_impl::backend::MultiBackend>: QueryFragment<Sqlite>` is not satisfied
    --> src/main.rs:28:15
     |
28   |         .load(conn);
     |          ---- ^^^^ the trait `QueryFragment<Sqlite>` is not implemented for `SelectBy<User, multi_connection_impl::backend::MultiBackend>`
     |          |
     |          required by a bound introduced by this call
     |
     = help: the trait `QueryFragment<DB>` is implemented for `SelectBy<T, DB>`
     = note: required for `diesel::query_builder::select_clause::SelectClause<SelectBy<User, multi_connection_impl::backend::MultiBackend>>` to implement `QueryFragment<Sqlite>`
     = note: 4 redundant requirements hidden
     = note: required for `SelectStatement<FromClause<table>, SelectClause<SelectBy<User, MultiBackend>>>` to implement `QueryFragment<multi_connection_impl::backend::MultiBackend>`
     = note: the full type name has been written to '/tmp/diesel_test/target/debug/deps/diesel_test-e9839cd4beffff40.long-type-13019818345722010715.txt'
     = note: required for `SelectStatement<FromClause<table>, SelectClause<SelectBy<User, MultiBackend>>>` to implement `LoadQuery<'_, DbConnection, User>`
     = note: the full type name has been written to '/tmp/diesel_test/target/debug/deps/diesel_test-e9839cd4beffff40.long-type-13019818345722010715.txt'
note: required by a bound in `diesel::RunQueryDsl::load`
    --> /home/weiznich/Documents/rust/diesel/diesel/src/query_dsl/mod.rs:1543:15
     |
1543 |         Self: LoadQuery<'query, Conn, U>,
     |               ^^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `RunQueryDsl::load`

error[E0277]: the trait bound `SelectBy<User, multi_connection_impl::backend::MultiBackend>: QueryFragment<Pg>` is not satisfied
    --> src/main.rs:28:15
     |
28   |         .load(conn);
     |          ---- ^^^^ the trait `QueryFragment<Pg>` is not implemented for `SelectBy<User, multi_connection_impl::backend::MultiBackend>`
     |          |
     |          required by a bound introduced by this call
     |
     = help: the trait `QueryFragment<DB>` is implemented for `SelectBy<T, DB>`
     = note: required for `diesel::query_builder::select_clause::SelectClause<SelectBy<User, multi_connection_impl::backend::MultiBackend>>` to implement `QueryFragment<Pg>`
     = note: 4 redundant requirements hidden
     = note: required for `SelectStatement<FromClause<table>, SelectClause<SelectBy<User, MultiBackend>>>` to implement `QueryFragment<multi_connection_impl::backend::MultiBackend>`
     = note: the full type name has been written to '/tmp/diesel_test/target/debug/deps/diesel_test-e9839cd4beffff40.long-type-13019818345722010715.txt'
     = note: required for `SelectStatement<FromClause<table>, SelectClause<SelectBy<User, MultiBackend>>>` to implement `LoadQuery<'_, DbConnection, User>`
     = note: the full type name has been written to '/tmp/diesel_test/target/debug/deps/diesel_test-e9839cd4beffff40.long-type-13019818345722010715.txt'
note: required by a bound in `diesel::RunQueryDsl::load`
    --> /home/weiznich/Documents/rust/diesel/diesel/src/query_dsl/mod.rs:1543:15
     |
1543 |         Self: LoadQuery<'query, Conn, U>,
     |               ^^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `RunQueryDsl::load`

(That sounds that some queryfragment impls in diesel are to restrictive for using selectable and/or boxed queries with the new MultiConnection derive.
Are you seeing any additional errors?
Checklist

[x] I have already looked over the issue tracker  and the disussion forum  for similar possible closed issues.

[x] This issue can be reproduced on Rust's stable channel. (Your issue will be
closed if this is not the case)
[x] This issue can be reproduced without requiring a third party crate

I did some work on this and it seems like that the problem with SelectBy is rather easy to solve by introducing the following trait:
    pub trait CompatibleBackend<DB>: Backend {}
    impl<DB> CompatibleBackend<DB> for DB where DB: Backend {}

then we can provide the corresponding additional impl for the generated multibackend type and restrict the QueryFragment impl for SelectBy on that.
Unfortunately that will not work for the boxed select statement case, due to the fact that a boxed select statement internally stores boxed query fragment trait objects for a specific backend type and we cannot simply convert them to something else. At that point I'm not even sure if that ever can be supported :(


[Bug Report] datePicker close dropdown menu when selecting a date (even if hide-on-click is false)

Element UI version
2.9.1
OS/Browsers version
Windows 10 / Chrome version 74.0.3729.169
Vue version
2.6.10
Reproduction Link
https://codepen.io/anon/pen/YbbqpN
Steps to reproduce
Select date in datePicker when that component is nested in the dropdown component (with  hide-on-click=false)
What is Expected?
Dropdown should not close itself (if hide-on-click is false) after selecting date in datePicker
What is actually happening?
Dropdown close itself after selecting date in datePicker

How can I solve the same problem?

tofix this bug

You should set attribute popper-append-to-body of that nested select component to false. Problem solved. 😃


CORTX-31329: Remove submodule linking of cortx-management-portal

Signed-off-by: Mukul Malhotra<EMAIL_ADDRESS>Describe your changes in brief
Remove submodule linking of cortx-management-portal
Changes

[ ] Why is this change required? What problem does it solve?
[ ] If proposing a new change then please raise an issue first

How Has This Been Tested? (Optional)

[ ] Please describe in detail how you tested your changes.
[ ] Include details of your testing environment, and the tests you ran to
[ ] How your change affects other areas of the code, etc.

Screenshots (if appropriate)
Checklist

[ ] tested locally
[ ] added new dependencies
[ ] updated the docs
[ ] added a test

Duplicate of PR#1517


Are single and double quotes guidelines allowed by PEP 8?

It depends on character escaping. When a string contains single quote character, you must use double quotes and vice versa.

@jollheef I'm aware of this recommendation of course.
The most probable case when this guidelines and PEP 8 contradict are "double quotes (\") in string for human". Strings for machine rarely contain quotes and other punctuation at all.
For me it's valuable to listen to opinions on how this guideline could become better regarding string quotes. Honestly, I want to leave this rule as it is now: single quotes for machines, double quotes for humans.


Handle VARBINARYs that arent actually binary

I recently discovered internally a situation where we weren't properly converting bytes to string. For us this was for any column that used COLLATE ascii_bin. Digging into it I discovered that MySQL has 2 methods for determining whether something is "binary":

the binary flag (on the flags bitmap with value 128)
the binary collation (collationIndex = 63, returned per field)

If Vitess sees the binary flag, it converts a VARCHAR to VARBINARY before sending to JDBC. The oracle JDBC does similar when talking directly to MySQL, however it has an additional check of the collationIndex to determine if the encoding is binary or not. We weren't doing that.
Also, I simplified the check to remove the LONGVARBINARY case. It's impossible to get this from Vitess. It's possible to get remapped to it above, but only if the collation is binary -- so the check is irrelevant.

Thanks @ashudeep-sharma  -- fixed.  @sougou can you merge when you get a chance?

LGTM


There is a lot of blank space between English and Chinese when converting to PDF

Excuse me!When I use Asciidoc, I use space between Chinese and English words to make my layout more beautiful, but when I convert ADOC to PDF, there is a lot of space between Chinese and English because I add a space (sometimes this happens, sometimes it doesn't). See pictures I posted). Is this related to text alignment? If not, what should I do?

Let me make a few points

I'm using the ADOc plugin for IDEA to convert PDF, but they use asciidoctor-pdf.jar
My preferred font to use is the recommended KaiGenGothicCN
This happens in Figure 2 and figure 3. Sometimes there is a lot of white space, sometimes just a normal blank space

You need to set -a scripts=cjk so that break opportunities are inserted between the Chinese characters. This is covered in the docs. See https://github.com/asciidoctor/asciidoctor-pdf#support-for-non-latin-languages
In the future, please ask usage questions in the #users/asciidoctor-pdf stream in the project chat at https://chat.asciidoctor.org. Thanks!


File Extensions don't show for known file types in Home-Recent Files when set to show

Description
File Extensions don't show for known file types in Home-Recent Files when they are set to show in Settings-Folders. They do appear everywhere else, but not there.
Steps To Reproduce

Go to Settings
Click the Folders tab
Under Display, set Show extensions for known file types to On, then exit Settings.
Go to Home.
None of the files listed under Recent Files list show their file extensions.

Steps to fix
unknown
Files Version
Version: <IP_ADDRESS> OS Version: 10.0.22621.1105
Windows Version
Windows 11 Pro Version	22H2 Installed on	12/19/2022 OS build	22621.1105 Serial number	PW02G088 Experience	Windows Feature Experience Pack 1000.22638.1000.0
Relevant Assets/Logs
debug.log

Merging with #5888 (this should cover recent items as well)


Go to Defintion for component attributes doesn't work if they're bound

Given a component like this:

@code
{
    [Parameter]
    public string? Value { get; set; }

    [Parameter]
    public EventCallback<string?> ValueChanged { get; set; }
}

If we use that component normally, like below, then Go To Definition on the Value attribute works, and navigates to the property.
<Component1 Value="value"></Component1>

If instead we bind the Value attribute to something, like the below, then Go To Definition no longer works.
<Component1 @bind-Value="value"></Component1>

@code{
    string? value = "";
}

We should be able to reason about the fact that a binding is present, and what the property name would be. At the same time as fixing this, we should validate that @bind-Value:after or @bind-Value:event also works.

@DustinCampbell do you want to pick this one up?

@DustinCampbell do you want to pick this one up?

Sure thing!


Create a fully executable WAR

This should partly fix the issue on how one should run OSIAM in
production. A fully executable WAR can be easily started on a command
line. It also supports fundamental SysVinit operations, like start,
stop. See [1] for more details.
[1] http://docs.spring.io/spring-boot/docs/current/reference/html/deployment-install.html#deployment-install

Works like a charm! Thank you!


enh(package): update latest stable dep

Coverage remained the same at 100.0% when pulling e394e9dfb4b54fb20a8c2d7dfbab1b9e1138c5e4 on update-dep into ac48b8ac54d8c9d67334fc3b012f48b6cadf1a1f on master.


changes locale to el

Thank you for the gem! I believe the standard language naming for Greek in Rails is :el. At least that's how I have found it to be in various i18n gems for Rails. I right now use your gem from my repo with these changes.

I also checked it out, you're right, tnx for the PR 👍


iMX8 support

Hi,
We are able to use the WPE engine on iMX6 with the info in the wiki section.  Is WPE backend supported on iMX8 as well? We are planning to use on iMX8M and wanted to see if its supported or would be supported soon?

That doesn't depend on meta-webkit. The hardware support depends on your hardware-BSP layers. More specifically, for i.MX8 you need to use the meta-freescale layers with version sumo or newer.
WPEWebkit itself should build and run fine on any Aarch64 (ARM64) hardware.
And regarding the WPE backends, the same backends that support the i.MX6 Vivante GPU should also do for to the i.MX8 Vivante one (AFAIK the graphic drivers are pretty much the same, aren't they?)

Just a quick note here.  While the MX8M still supports galcore and Vivante's userspace binaries it is no longer using a fbdev interface.  Instead it is using drm/kms and gbm for display management and passing around buffers.  This may work with WPEBackend-mesa, but I haven't had the time to test yet.  I need to take some time and do some benchmarks of Vivante binaries vs drm-etnaviv.

Hi @appcodr
I try to get meta-webkit running on a MX6 as well but had no luck till now. Which way did you configure your image? the wiki mentions multiple with and without wayland.
Could you take a look at my issues? Maybe you see whats wrong -> #61 for fdo and #90 for rdk
If you could share your configuration and layer versions for the imx6 board that would also be great!

Hi @appcodr  and all,
I'm very curious on WPE + iMX8M. Do you have any status update on this? Thank you so much.

I have etnaviv, Weston (6), WPEBackend-FDO and Cog (in Weston with the FDO platform plugin and in FB with the DRM platform plugin) working on iMX8M.
Do you have a specific question about this?

Hi @philn ,
It is great! I'm going to build WPE on TechNexion pico pi imx8mq. But there yocto repo latest is sumo and only have wiki for i.MX6.
So it is very helpful for me if you please share your configuration, information about your working WPE on iMX8M. Thank you :)

You can follow the same instructions as in the i.MX6 wiki page. the meta-webkit git master branch should build for Yocto Zeus (if not, please file issues :)).


Add missing bnonce prose

We have a prose description of bleader in the Shelley spec, but nowhere is bnonce ever described.
Notably, bleader is described as "indicates a possible leader for this slot", but it's not clear how a similar definition would apply to bnonce and if bnonce is the same thing as the nonce_vrf from the binary spec of previous eras

The accessor functions bleader and bnonce are defined in "Figure 53: Block Definitions", right below the definition of BHBody.
The section that you are referring to, "12.9 Verifiable Random Function", is about the VRF checks. The first bullet point in that section describes that both the leader and the nonce VRF values are checked against their proofs in the block header body. The third bullet point describes an additional check involving the leader value, but there is no corresponding check for the nonce value.
The way that the VRF nonce value is used is described in the section "12.5 Update Nonce Transition" and in "Figure 60: Update Nonce rule".
The variable nonce_vrf in the CDDL specification does refer to to VRF nonce value. The accessor function bnonce retrieves the VRF nonce value from the block header body.
Does that clear everything up?

Yes, the fact that we have an accessor is fine. The idea that we have some nonce that updates based off some deterministic function is fine. It doesn't explain why we need to run a 2nd VRF on this nonce though. If bleader "indicates a possible leader for this slot", then what is bnonce for?
Notably, I would expect something like this
bnonce is a second invocation of the vrf on the same key(?) and epoch nonce as bleader, but using a different seed(?) as input, which contributes to the epoch nonce for the next epoch. This is required because *key missing insight here*, which helps avoid stake grinding attacks.

I see. Section 12.1 (which is where I think this should live) is very sparse on prose, nothing in the block header body is explained.

on the same key(?)

yes

but using a different seed(?)

yes, see mkSeed and vrfChecks in the "VRF helper function" table (which sadly isn't numbered for some reason).

required because key missing insight here, which helps avoid stake grinding attacks.

The epoch nonce, and how it evolves, is the heart and soul of Ouroboros Praos. We could mention this, but without going into all the details. It's where the name "ouroboros" comes from, the snake eating its own tail is a reference to this entropy eating its own previous entropy.


Console colors shouldn't assume a dark background

Beets has hard-coded console colors which assume a dark background. This isn't friendly to users with a light background. It would be nice if beets supported configurable colors, or at least a "dark" vs. "light" option.

Resurrected in #1238.


Clarify license for Hyper Admin Theme

Description
I really like this bundle and would like to use it with some of my projects.
I'm concerned about the license, though, as it's based on this theme:
https://themes.getbootstrap.com/product/hyper-responsive-admin-dashboard-template/
But it appears that the license doesn't cover embedding the theme in a framework:
https://themes.getbootstrap.com/licenses/
Can you please clarify?
Example
No response

What do you think about replacing hyper-theme with tabler (https://tabler.io/)?  It's MIT licensed.

Hello,
I have asked question to their support.
I will look at an other bootsrap5 template (tabler ?) if i have to remove theme.

@kevinpapst has a rudimentary bundle based on this theme:
https://github.com/kevinpapst/TablerBundle
Gut feeling is that you should switch to tabler, since it's Open Source any changes that are related to the front end can be made there, and you can focus on the integration.
Thanks.

I'll move this to a new thread, since it's a major change.

Did you hear back from them?  What are your thoughts on moving to Tabler, or some other MIT-licensed Bootstrap 5 theme?

Hi,
Yeah, they answer me. Hyper licence doesn't allow bundling their theme into an open-source project.
I have tried Tabler.io but i have finally chosen AdminKit (Mit license for limited version).
I am working on a new version.
Branch : 4.x.
Demo : https://release.umbrella-corp.dev/
I will publish this version soon.

Terrific!
I like AdminKit, I even bought a license for the pro version.  But then
they gutted some features (like nested sidebar menus) that were in the free
version one day then suddenly they were gone.  This disrupted all the
projects that used them.
Since it's MIT-licensed, though, you can augment your bundle to support
nested and collapsible sidebar menus, yes?  For an admin theme, the sidebar
menu seems to be of paramount importance.
I also like the right-panel settings menu and the mega-menu that are common
in admin themes, but aren't directly supported by AdminKit.
Anyway, awesome work!  Thanks.
On Mon, Dec 6, 2021 at 4:50 PM Adrien Cantepie @.***>
wrote:

Hi,
Yeah, they answer me. Hyper licence doesn't allow bundling their theme
into an open-source project.
I have tried Tabler.io but i have finally chosen AdminKit
https://demo-basic.adminkit.io/ (Mit license for limited version).
I am working on a new version.
Branch : 4.x https://github.com/acantepie/umbrella/tree/4.x.
Demo : https://release.umbrella-corp.dev/
I will publish this version soon.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/acantepie/umbrella/issues/11#issuecomment-987265299,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEXIQOVER5BPPFAUT6KUJ3UPUV3HANCNFSM5H2OLDVA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

I will publish this version soon.

When I install version 4 (composer require umbrella2/adminbundle:^4) I get an error from core about hasLink not being a method.  I imagine this will be fixed when you publish v4.

Terrific!
I like AdminKit, I even bought a license for the pro version.  But then
they gutted some features (like nested sidebar menus) that were in the free
version one day then suddenly they were gone.  This disrupted all the
projects that used them.
Since it's MIT-licensed, though, you can augment your bundle to support
nested and collapsible sidebar menus, yes?  For an admin theme, the sidebar
menu seems to be of paramount importance.
I also like the right-panel settings menu and the mega-menu that are common
in admin themes, but aren't directly supported by AdminKit.
Anyway, awesome work!  Thanks.
I have developed some features missing on free version like nest items on sidebar (quite easy because nest items are bs5 collapse).
Purchase a pro license doesn't allow me to redistribute theme under less restrictive license, so i won't upgrade.

When I install version 4 (composer require umbrella2/adminbundle:^4) I get an error from core about hasLink not being a method. I imagine this will be fixed when you publish v4.

I check

When I install version 4 (composer require umbrella2/adminbundle:^4) I get an error from core about hasLink not being a method. I imagine this will be fixed when you publish v4.

To test vendor from an untagged branch, you shave to specify all vendors version :
composer require umbrella2/corebundle:dev-master
composer require umbrella2/adminbundle:dev-master


Datagrid

Description
Datagrid SelectedItem cannot be unselected by setting the bound property to null.
To Reproduce

Bind Blazority Datagrid SelectedItem to a property.
After populating data and selecting the row the SelectedItem property is set properly.
If you set the SelectedItem property to NULL on a Button click event the SelectedItem property appears to be called in an infinite loop from then on and the page is no longer responsive.

Expected behavior
Row should be deselected.
Actual behavior
Row is not deselected and page becomes unresponsive.
Environment

OS: Windows 10
Browser: Chrome
Blazority Version: 0.5.0
.NET Core Version : 5

Project with modified Index.razor showing behavior.
TestBlazority.zip

Fixed in v0.6.0


Python3

This fixes python3 compatibility and DDBenchmark run method.

Please ignore this PR, it was made against master branch. Use #2


Follow-up on moving out of the SES sandbox to production

Case pending with Amazon support.

@fayez-nazzal note that until this is closed, you will always get 500 error when subscribing to notifications because the backend is trying to send you a confirmation email which gets rejected by SES due to sandbox. If you retry the subscription, you will get "Already subscribed" which means you subscribed but the requests failed while trying to send you an email.
I just added your 2 emails to verified identities, please click on the email link to get them verified and start receiving emails on both (and sending from too if you have access keys in my account).

Once we merge #22, I will generate the static site and send them another followup so they see the real use case.

Just got approved for production!
14 messages per second, 50,000 per day!


Support PHP 7.4 and 8.0

Q
A

Bug fix?
yes

New feature?
yes

BC breaks?
no

Deprecations?
no

License
MIT

EDIT: Passing https://github.com/GrahamCampbell/mock-client/actions/runs/167569254.

// cc @dbu ;0


Migration Console to AmazonLinux Base

Description
Migration Console to AmazonLinux2023 Base.
To support Opensearch-Benchmark on ARM, required building yappi and hdf5 from source.
Updated to python 3.11 because that's what's available on AL2023

Category: New Feature
Why these changes are required? Secure base images to prep for publish to dockerhub and ecr
What is the old behavior before changes and new behavior after changes? Migration console was built on ubuntu, now it's on AL2023

Issues Resolved
MIGRATIONS-1880
Is this a backport? If so, please add backport PR # and/or commits #
Testing
Building locally with arm, included starting up migration console api.
Also tested locally with FetchMigration container
Check List

[x] New functionality includes testing

[x] All tests pass, including unit test, integration test and doctest

[x] New functionality has been documented
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@AndreKurait Looks like there is an issue with the end to end test cases, can you look into it?

@AndreKurait Looks like there is an issue with the end to end test cases, can you look into it?

Addressed sharing the pipenv environment across the whole container


Example of (or support for) Inference Callable of Triton ensemble definition

Is your feature request related to a problem? Please describe.
I currently have an automated config generator for creating proper ensemble pbtxt configs over a DAG of backends. I currently do not see a natural way of defining and/or referencing Triton ensembles with the PyTriton module.
Describe the solution you'd like
I would like to either:
(a) create the ensemble definition directly with the PyTriton module, or
(b) load all ensemble models and the associated (already generated via above method) ensemble and model configs
Describe alternatives you've considered
Our current solution mentioned in the feature request does the job, but does not utilize PyTriton's friendly interface.
Additional context
N/A

I should add that I'm willing to contribute stronger ensemble support as well, given discussion and RFC.

Hi @markbarna ,
Thank you for providing more details about your scenario. I appreciate your feedback and I will try to update my answer accordingly.
Using a ModelClient object without the context manager has some advantages and disadvantages compared to using the context manager for each inference request, which I will explain below.
Using a ModelClient object without the context manager has the advantage of avoiding the overhead of creating and closing the connection for each request, which may improve the inference latency and throughput. Moreover, this approach may be more efficient because each new instance of ModelClient is downloading the model configuration and runs requests to check if the server is alive and the model is ready.
On the other hand, using the context manager for each inference request has the advantage of ensuring that the connection with the server is always fresh and reliable. This may prevent some potential issues such as connection timeouts, stale connections, or memory leaks.
In summary, both approaches have their own benefits and drawbacks, and you may need to experiment with both to find out which one suits your needs better. However, based on your scenario, it seems that using a ModelClient object without the context manager may be more suitable for you, as long as you ensure that the connection with the server is stable and secure, and that you manage your client resources properly.
We are considering new feature for ModelClient that will allow to serialize and deserialize model configuration. This will allow to create ModelClient object with only connection options and model name, and then load model configuration from file or string. This will allow to avoid downloading model configuration and checking if model is ready for each ModelClient object. This will also allow to create ModelClient object without context manager and without manually calling wait_for_model() method. This feature is not yet implemented, but we are considering it for future releases.
I hope this updated comparison helps you make an informed decision. Please let me know if you have any further questions or suggestions regarding new features.

@michaelhagel  Is there some code about automated config generator in the NVIDIA repo, I am looking for it , Thanks very much if you can give some advices.

@Lzhang-hub if you look for automated config generator you may want to take a look at https://github.com/triton-inference-server/model_navigator. There is a functionality to create the Triton model store with generate config.pbtxt: https://triton-inference-server.github.io/model_navigator/0.7.1/triton/triton_deployment/
However, there is no ensemble support.

@Lzhang-hub sorry, I should have clarified -- that ensemble generator is currently part of some tooling at the company I currently work for, and is (for the moment, hopefully not for long) closed source.
In context of the feature request, I also suppose the config generator part is not truly applicable to my question. To reclarify: there is currently great support for ensembes within Triton Inference Server, where an ensemble is viewed as a single model from the client's point of view. This abstraction is in many cases preferable to writing a single Python backend with multiple steps as each "step" (read: model or processing) can be optimized independently with other Triton primitives. In addition, the ensemble definition is then not in Python code and allows for relatively easy config generation, allowing engineers to abstract away the serving process from the model developers/data scientists.
Here we can see an example of an ensemble of Model Navigator packages are served w/ PyTriton without the Triton ensemble scheduler.
Is this the approach preferred by the PyTriton team? I am still undecided as to whether this approach eschews some of the benefits of the native ensemble scheduling approach or is preferable...

In my opinion PyTriton is not designed to be a model management tool. It does not support creating or loading ensemble models directly from the library. PyTriton is designed to be a Python environment that allows you to run inference requests on Triton Inference Server using a simple and intuitive interface.
Why do you need ensemble in PyTriton?
Why do you need ensembles?
Ensembles are useful for creating complex pipelines of models that can run on different backends and devices. For example, you may want to use an ensemble to combine a PyTorch model that runs on a CPU with an ONNX model that runs on a GPU, and pass the output of one model to the input of another. Ensembles can also help you optimize the performance and latency of your models by using features such as dynamic batching or response cache.
Can you tell me more about your use case and why you need to create and load ensemble models directly from PyTriton? I would like to understand your scenario better.

Which of ensemble features are useful in your scenario?
Do you want to use dynamic batching or response cache?
Do you want to use ensemble for combining models from different backends and devices?

Ensembles-like flow in PyTriton
There are two ways you can implement something very similar to ensembles with PyTriton:

You can run multiple models inside one Inference Callable in PyTriton.
You optimize complex model using the [Triton Model Navigator] tool alpha feature called inplace optimization.

Callable similar to Triton Ensemble Definition
PyTriton can be used to build complex pipeline in Python and run it directly in inference callable. For example, you can use the following code to run two PyTorch models in sequence:
import numpy as np
from pytriton.decorators import batch
import torch

model_1 = torch.nn.Linear(2, 3).eval()
model_2 = torch.nn.Linear(3, 4).eval()

@batch
def infer_fn(**inputs: np.ndarray):
    (input1_batch,) = inputs.vaWe hope this helps you with your AI projects. Please let us know if you have any other questions or feedback. We are always happy to hear from our users.lues()
    input1_batch_tensor = torch.from_numpy(input1_batch)
    # Running ensemble-like flow directly in python
    output1_batch_tensor = model_2(model_1(input1_batch_tensor))
    output1_batch = output1_batch_tensor.detach().numpy()
    return [output1_batch]

Triton Model Navigator inplace optimization
Triton Model Navigator enable zero-copy data passing between heterogeneous model formats, such as PyTorch and ONNX. It contains runners so you can run heterogeneous models in a single inference request.
Navigator developers are also testing an alpha feature for complex models, which can be fully converted into ONNX or similar formats. It is called inplace optimization. It can combine many inference technologies into a single model, which can be deployed on the Triton Inference Server. All that is required is to wrap a module with a single line of code:
import model_navigator as nav

pipeline = Pipeline(...)
pipeline.model = nav.Module(pipeline.model)

pipeline(...)

They prepared an example for this feature for Stable Diffusion. It would be very helpful if you could test this feature and provide feedback for your use case.
Summary
I hope this helps you with your AI projects. Please let me know if you have any other questions or feedback. I'm always happy to hear from users of PyTriton.

@Lzhang-hub if you look for automated config generator you may want to take a look at https://github.com/triton-inference-server/model_navigator. There is a functionality to create the Triton model store with generate config.pbtxt: https://triton-inference-server.github.io/model_navigator/0.7.1/triton/triton_deployment/
However, there is no ensemble support.

Thank you very much, this is what i'm looking for.

Which of ensemble features are useful in your scenario?
Do you want to use dynamic batching or response cache?
Do you want to use ensemble for combining models from different backends and devices?

Hello! Though this issue was closed, I think I am looking for the same ensemble feature. I'd like to answer your question about
Which of ensemble features are useful in your scenario?
I am seeking for methods that enhance GPU utilization (that in the nvidia-smi). In my case, on powerful GPUs, the pre-process and post-process of model which take place on CPUs, often slow down the application, making the GPU utilization about 60%. These pre-process and post-process functions, like cv2.resize(), should be part of the service, rather than part of the client, that is because my application demands so. So my goal is to ensemble these pre-process and post-process functions with the pytorch model. Do you suggest that I should use TIS python backend?

Thank you for your question. I understand that you want to improve the GPU utilization of your application by ensembling the pre-processing and post-processing functions with the PyTorch model. I recommend that you use PyTriton with DALI, which is a library that can perform pre-processing and post-processing on GPU efficiently. You can also use CV2 but it runs on CPU and can slow down the application.
PyTriton is a framework that allows you to define your inference pipeline using Python functions. You can use DALI to create functions that preprocess and postprocess your inputs and outputs on GPU. You can also use PyTorch to create your image processing model.
You can find an example of how to use PyTriton with DALI and PyTorch in the PyTriton repository. The file server.py shows how to define an inference pipeline for video segmentation. The function _infer_fn is the main entry point for the pipeline. It takes the encoded video as input and returns the original and segmented frames as output. The function preprocess uses DALI to decode and resize the video frames on GPU. The function segmentation uses PyTorch to perform the segmentation model on GPU. The function postprocess uses DALI to overlay the segmentation mask on the original frames on GPU. You can see how these functions are decorated with @batch to enable dynamic batching and improve performance.

@batch
def _infer_fn(**inputs):
    encoded_video = inputs["video"]

    image, input = preprocess(encoded_video)
    batch_size, frames_num = image.shape[:2]

    input = input.reshape(-1, *input.shape[-3:])  # NFCHW to NCHW (flattening first two dimensions)
    image = image.reshape(-1, *image.shape[-3:])  # NFHWC to NHWC (flattening first two dimensions)

    prob = segmentation(input)
    out = postprocess(image, prob)

    return {
        "original": image.cpu().numpy().reshape(batch_size, frames_num, *image.shape[-3:]),
        "segmented": out.as_cpu().as_array().reshape(batch_size, frames_num, *image.shape[-3:]),
    }

I hope this example helps you to achieve your goal. Please let me know if you have any questions or feedback.

Thank you for your question. I understand that you want to improve the GPU utilization of your application by ensembling the pre-processing and post-processing functions with the PyTorch model. I recommend that you use PyTriton with DALI, which is a library that can perform pre-processing and post-processing on GPU efficiently. You can also use CV2 but it runs on CPU and can slow down the application.
PyTriton is a framework that allows you to define your inference pipeline using Python functions. You can use DALI to create functions that preprocess and postprocess your inputs and outputs on GPU. You can also use PyTorch to create your image processing model.
You can find an example of how to use PyTriton with DALI and PyTorch in the PyTriton repository. The file server.py shows how to define an inference pipeline for video segmentation. The function _infer_fn is the main entry point for the pipeline. It takes the encoded video as input and returns the original and segmented frames as output. The function preprocess uses DALI to decode and resize the video frames on GPU. The function segmentation uses PyTorch to perform the segmentation model on GPU. The function postprocess uses DALI to overlay the segmentation mask on the original frames on GPU. You can see how these functions are decorated with @batch to enable dynamic batching and improve performance.
@batch
def _infer_fn(**inputs):
    encoded_video = inputs["video"]

    image, input = preprocess(encoded_video)
    batch_size, frames_num = image.shape[:2]

    input = input.reshape(-1, *input.shape[-3:])  # NFCHW to NCHW (flattening first two dimensions)
    image = image.reshape(-1, *image.shape[-3:])  # NFHWC to NHWC (flattening first two dimensions)

    prob = segmentation(input)
    out = postprocess(image, prob)

    return {
        "original": image.cpu().numpy().reshape(batch_size, frames_num, *image.shape[-3:]),
        "segmented": out.as_cpu().as_array().reshape(batch_size, frames_num, *image.shape[-3:]),
    }

I hope this example helps you to achieve your goal. Please let me know if you have any questions or feedback.

Hello~, thanks for your reply. At the moment I won't use DALI to do the work, because there are some other operations that must be on the CPU. And I'm also afraid that the interpolation on CPU and GPU may be different. And I recently find that the preprocess/postprocess may not be the only cause of low GPU utilization. The CPU is launching the kernels slower than the computation. That may also be a reason. In my opinion it seems there may not be a universal solution to guarantee ~100% GPU  utilization.


Allow using fragments as component children without extra braces

Describe the bug
Unable to use twice a custom component.
To Reproduce
Steps to reproduce the behavior:
#[derive(Prop)]
pub struct ContainerProps<'a, G: Html> {
    children: Children<'a, G>,
    fluid: bool,
}

#[allow(non_snake_case)]
#[component]
pub fn Container<'a, G: Html>(cx: Scope<'a>, props: ContainerProps<'a, G>) -> View<G> {
    let children = props.children.call(cx);
    let container_class = match props.fluid {
        true => "container-fluid",
        false => "container",
    };

    view! { cx,
      div(class=container_class) {
        (children)
      }
    }
}

Why can't I add more then 1 child?
#[allow(non_snake_case)]
#[component]
pub fn App<G: Html>(cx: Scope) -> View<G> {
    view! { cx,
      Container {
        fluid: true,
        Col {
          xs: 12,
          xl: 3,
          "This is row 1"
        }
        Col {
          xs: 12,
          xl: 3,
          "This is row 2"
        }
      }
    }
}

The error is:
error: unexpected token
  --> src/app.rs:25:23
   |
25 |                       Col {

Environment

Sycamore: 0.8.0-beta.7

Summary
The current syntax for using view fragments as children to components is as follows:
view! {
    MyComponent {
        props: whatever,
        {
            AnotherComponent {}
            div
            p { "children" }
        }
    }
}

Proposed syntax
The braces around the fragments are unnecessary and are not very ergonomic. The following code should instead be accepted.
view! {
    MyComponent {
        props: whatever,
        AnotherComponent {}
        div
        p { "children" }
    }
}

This also makes the syntax more consistent with html elements.

Superseded by #457


Fix link to hanselman asp.net core article

There still is an error concerning link testing:
checking 605 GitHub repositories... ❌ ERROR {   "0": {     "show": true,     "duplicates": "",     "other_links_error": "",     "github_repos": [       {         "message": "Unexpected end of document"       }     ]   } }
I'll have a look at this this afternoon, open to suggestions on what could cause the issue.

Thanks, I’ll look at why it breaks

still occurs after re-run


Improve coverage over codebase

Codecov Report
Attention: 17 lines in your changes are missing coverage. Please review.

Comparison is base (622bdd6) 89.47% compared to head (cff135f) 92.08%.
Report is 98 commits behind head on beta/v4.0.

Files
Patch %
Lines

weaviate/collections/classes/filters.py
92.18%
5 Missing :warning:

weaviate/collections/batch/rest.py
75.00%
2 Missing :warning:

weaviate/collections/classes/aggregate.py
96.61%
2 Missing :warning:

weaviate/collections/classes/config.py
84.61%
2 Missing :warning:

weaviate/collections/batch/batch_wrapper.py
85.71%
1 Missing :warning:

weaviate/collections/classes/grpc.py
90.00%
1 Missing :warning:

weaviate/collections/classes/orm.py
0.00%
1 Missing :warning:

weaviate/collections/collections.py
87.50%
1 Missing :warning:

weaviate/collections/data.py
92.30%
1 Missing :warning:

weaviate/connect/v4.py
94.44%
1 Missing :warning:

Additional details and impacted files
@@              Coverage Diff              @@
##           beta/v4.0     #803      +/-   ##
=============================================
+ Coverage      89.47%   92.08%   +2.60%
=============================================
  Files            175      163      -12
  Lines          18407    17346    -1061
=============================================
- Hits           16470    15973     -497
+ Misses          1937     1373     -564

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


dont spam recenter on spawn

[ ] test with mods

The spawning spam was mostly for tidally locked spawns so be sure to check on Daylight Savings, Evacuation, Trappist... Can't think of any other tidally locked spawns rn

the reposition stuff still happens. it just doesnt recenter


added cgroup_enable=memory and swapaccount=1 to check for memory enforcement using cgroups

By default, the debian kernels are not started with cgroup_enable=memory and awapaccount=1 parameters. As a result, cgroup does not correctly enforce memory and swap limits on a container. This PR checks for these parameters as part of host configuration (Section 1) in /proc/cmdline. If these parameters are not present, it raises a warning.

Please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "master" git@github.com:salmanbaset/docker-bench-security.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

Ammending updates the existing PR. You DO NOT need to open a new one.

Hi and thanks for the PR.
I'll leave this PR open for the time being, since we'll need a way to seperate CIS and "extra" tests when running the script.
I would also like you to run shellcheck on the code, use $(..) instead of legacy .. for example.

Closing due to inactivity.


Development status

Hello @ihavespoons,
First off, thanks for setting this up.
I wanted to ask if you're actively maintaining this and whether you're open for PRs.
I've got several ideas on how to make this ruleset more feature-complete and more convenient to use.
Looking forward to hearing back from you.
all the best.

Hey @betaboon,
Cheers for reaching out,
I am certainly actively maintaining this although in the past couple of months I haven't actively been adding features due to some role changes.
I'd love to see and hear your ideas and I am happy to assist in anyway I can from review, to writing to whatever you'd like
Thanks

Here's a list of things I'm considering/playing with atm.
Looking forward to hearing your opinions :)
maintenance

update tool versions (done in #3)
update to latest rules-template

it now mirrors what has been here in .github/workflows/ci.yaml
I have this done, will send a PR after #3 is merged

update dependencies

I have this done, will send a PR after #3 is merged

refactor e2e tests

the current bzlmod and generate tests have a very large overlap
I propose merging them into a smoke suite
I have this done, will send a PR after #3 is merged

convenience

have syft.toolchain default to latest

I have this done, will send a PR after #3 is merged

features

introduce grype-toolchain

currently working on this

introduce grype.database, grype_report and grype_test
rename syft_generate_sbom to syft_sbom
rename the type argument of syft_generate_sbom to format

any news? :)

Sorry @betaboon been crook the last few days. Running the workflows on that PR now

no worries.
any opinion on renaming syft_generate_sbom to syft_sbom?
my reasoning for it is, that the majority of rules is named as a noun.

Don't have an issue with rename but lets keep both with a notice in docs that syft_generate_sbom will be deprecated in the future.

sounds good

done in the last commit on my PR

Awesome appreciate it mate, just pushing the last stuff through to BCR now. Then I will do a final review for this grype change.

Out of curiosity what got you onto this ruleset @betaboon? It felt pretty niche when I originally created it hahaha

quite a while ago i evaluated https://github.com/hxtk/rules_anchore, tried to get a PR in, barely any response.
now ~18 month later i had some time on my hands and was about to do what you did here :)

Hahah awesome, you had exactly the same response I did! pretty sure my issues were deleted and all. Well I am glad to have you here. Definitely planned a lot more for this module but life and health got in the way. I want to see if we can get our hands on the rules_anchore module name at some point

i think aside from the grype_updater rule (which imho isn't really compatible with bzlmod) this ruleset has reached feature parity with my PR.
so maybe they're willing to deprecate in favor of this, and donate the name :)

thanks for merging.
could you release on BCR when you find the time?

Hey @betaboon sorry planning to fix up some issues today then get the merge in.

heya. i just sent another PR to update tool-versions.
I'll close this issue, as everything discussed in here is done :)


investigate feasibility of removing rX directories when mutable HEAD is committed

The mutable HEAD uses rX directories to manage changes. These directories are carried forward when the mutable HEAD is committed. It would be ideal if only the contents of these directories was committed, but this complicates the atomicity of installing the mutable HEAD as a proper version.

Decided that it's not possible to remove the rN directories without incurring too much cost.