Compiling openfast with Visual Studio

Bug description
Hello, I am new to open-fast(v8). I have tried to compiler open-fast with Visual Studio 2022 by running the solution file Fast.sln, but it gives me a warning output. And the it seems that Fast, Fastlib and Fast_simulink can't be loaded.
To Reproduce

Steps to reproduce the behavior:

Install Visual studio 2022
Install Intel BaseKit_p_2<IP_ADDRESS>
Install Intel HPCKit_p_2<IP_ADDRESS>
Open /OpenFAST/vs-build/FAST/FAST.sln and build it. Fast, Fastlib and Fast_simulink can't be load and Visual Studio gave output information here:

①D:\Users\research\Openfast_research\OPENFAST\openfast-main\vs-build\FAST\FAST.vfproj : error  : Unable to cast COM object of type 'System.__ComObject' to interface type 'Microsoft.VisualStudio.Shell.Interop.IVsUpdateSolutionEvents'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{A9F86308-5EA7-485D-BAB8-E8989C3CFBDC}' failed due to the following error: 不支持此接口 (Exception from HRESULT: 0x80004002 (E_NOINTERFACE)).
②D:\Users\research\Openfast_research\OPENFAST\openfast-main\vs-build\FASTlib\FASTlib.vfproj : error  : Unable to cast COM object of type 'System.__ComObject' to interface type 'Microsoft.VisualStudio.Shell.Interop.IVsUpdateSolutionEvents'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{A9F86308-5EA7-485D-BAB8-E8989C3CFBDC}' failed due to the following error: 不支持此接口 (Exception from HRESULT: 0x80004002 (E_NOINTERFACE)).
③D:\Users\research\Openfast_research\OPENFAST\openfast-main\vs-build\OpenFAST-Simulink\OpenFAST-Simulink.vfproj : error  : Unable to cast COM object of type 'System.__ComObject' to interface type 'Microsoft.VisualStudio.Shell.Interop.IVsUpdateSolutionEvents'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{A9F86308-5EA7-485D-BAB8-E8989C3CFBDC}' failed due to the following error: 不支持此接口 (Exception from HRESULT: 0x80004002 (E_NOINTERFACE)).
Any advice to how to resolve it? Thanks!
Screenshots

If you right-click on the each of the unloaded Fortran projects, does it give you the option to reload them? They will have to be loaded before you can build.

Tank you very much  for informing me the problem of software version. Now I have re-installed Visual Studio 2019 16.11.15, and they can be loaded now. But when I try to rebuild it with FAST.sln, it shows a error info and many waring info like this.

Could you post the text from the Output window when you build? I'd like to see where in the build order the error shows up.

Here is the Output information you need.  Thank you very much for your help
error_output.txt

I'm not exactly sure why this is giving you a linking error. It seems like the Intel integrations either aren't properly setting the path to the MKL libraries or the MKL libraries didn't get installed with the oneAPI installers.
I am wondering if it is related to this issue. Could you try to build with an x64 configuration instead of Win32?

Thank you very much for your suggestion. I can now complete the Compiling of Fast with an x64 configuration. And I try every module inside Fast.sln. Only the module of Fast should change the compiling configuration, the rest can be complied normally with Win32

It looks like if you want to build for 32-bit Windows, you will have to download and install a separate 32-bit oneAPI base toolkit (which includes the 32-bit MKL) after you install the "normal" oneAPI base toolkit: https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html?operatingsystem=window&distributions=webdownload&options=online. Otherwise, you can just build for x64.


Reworked header.

Function declarations always exist, but when tracing is disabled they are hidden by the dummy macros.

@kjbracey-arm Is this about what you had in mind?

seems that you have to write some more tests because CI fails:
11:36:56 Code coverage enforcement failed for the following metrics:
11:36:56     Lines's stability is 65.41 and set mininum stability is 83.77.
11:36:56     Conditionals's stability is 50.56 and set mininum stability is 64.08

Ah yes, that was actually something I did want comments on: Any other usage macros/aliases to add?

Hm... I don't think the code coverage failure is because of this change. ARMmbed/mbed-trace#20 is failing for that too, no?
Still, regardless of what changes cause the failure, those tests should be added.

Regarding the tr_array macro, I'm not sure it would add much. It isn't really possible to assign any standard values to the arguments of trace_array, like in the case of the other tr_ macros, so it would simply function as another alias for mbed_trace_array, saving 3 characters per line when compared to trace_array.

This is basically what I was thinking of, yes. Like it, just those few comments.

Re-evaluated for each inclusion of the header - can include it more than once in a compilation unit.

Yeah, realized this after pushing, heh.

FEA_TRACE_SUPPORT going to go in via the separate commit, and then resolving the merge conflict?

That's what I was thinking.

@kjbracey-arm. Incorporated your previous comments. Anything else come to mind?

lgtm

test this please

+1

lgtm


ApiListingResource initialized field should not be static

Hello,
I'm on a project in which I have multiple instance of ApiListingResource (for multiple ServletContext).
The fact that the initialized field is declared static cause an issue where the second instance is not initialized (and context swagger attribute is still null) but thinks it is. It only works for the first of course.
Could you remove that "static" modifier ?
It's not coherent with the non-static context field.
And there is nothing else static in that class anyway.
Thank you.
Regards.

No need for static.  Thank you.

Reopened, same issue : https://github.com/swagger-api/swagger-core/issues/1724


JSON-RPC edge-cases, nonce management and eth_getTransactionBySenderAndNonce

During the JSON-RPC meeting in Istanbul, I made a case that eth_getTransactionBySenderAndNonce would be useful for dapp and tool developers. I'm opening this issue to continue the discussion here, and to go into more detail.
Problems with the JSON-RPC
These are the problems that I think could be solved with eth_getTransactionBySenderAndNonce.
Problem 1: Losing track of a transaction
I created this pictures of how a the normal workflow of sending a transaction from a dapp or tool that interacts with a wallet looks like:

As shown above, the flow would be something like this:

The dapp/tool wants to send a transaction, so it calls into a library like ethers/web3/viem.
The library is connected to a wallet, which exposes the same JSON-RPC interface as a client. Note that the wallet doesn't just forward things to a client, it has its own implementation of some methods, like eth_sendTransaction.
To send a transaction, the library sends an eth_sendTransaction to the wallet.
The wallet receives that request and creates and signs a transaction locally.
At this point, the wallet is already aware of the full tx body and hence its hash, but it hasn't communicated it to the the library yet.
The wallet will then do an eth_sendRawTransaction, sending the full tx body to an EL client, which will broadcast it.
The wallet returns a hash as the response of eth_sendTransaction, which the library receives and passes to the dapp.

If after sending the tx to the EL client anything goes wrong with the wallet, library or dapp, a transaction will be broadcasted as requested by the dapp, but the dapp will never be able to access it.
Note that the order of 6 and 7 are unspecified. The wallet could return the hash before sending the raw transaction. As it's unclear, we are forced to assume the worst situation is possible.
If we had eth_getTransactionBySenderAndNonce a dapp could keep track of the nonce and transaction params, and resume from errors if needed, by checking if a transaction with that sender and nonce exists, and if it has the same params as the one that it intended to send.
Problem 2: Replaced and dropped transactions
When a dapp/tool sends a transaction, the tx can be replaced by another one with the same nonce and sender, or dropped from the mempool. What's more, replacement transactions can be equivalent to the one you sent (e.g. just higher fees), or a completely different one (e.g. to a different account).
All of these situations manifest as a eth_getTransactionByHash returning null.
The way to distinguish between some of them (replaced vs dropped) is by using eth_getTransactionCount with different blocktags and analyzing a pretty complex set of cases, including making tons of assumptions about how "pending" works in this case.
If we had eth_getTransactionBySenderAndNonce a dapp could use it to understand what happened if eth_getTransactionByHash returns null.
Proposal
Add a method eth_getTransactionBySenderAndNonce with two parameters:

An address: the sender of the transaction.
A quantity: the nonce of the transaction.

The JSON-RPC server should return the hash of a transaction (or the full tx?) of the transaction sent from the provided address using that nonce. If no such transaction exists, it should return null.
If multiple transactions from a sender and nonce were received, I assume there's one that can be considered the best candidate for inclusion (e.g. pays more fees), and that one should be used.
Things to consider

Does this method have other use cases?
Are there any alternative solution?
How many resources would this method require for transactions that are still in the mempool?
How many resources would this method require for transactions that were included in a block?
How many blocks are really important to solve the problems listed above?
How can RPC/Node providers (e.g. Infura)  support this?

Tagging people present on the in-person discussion, or who I've discussed this in the past: @s1na @lightclient @sambacha @kanej

This is also relevant: https://github.com/manifoldfinance/rpc-eip-drafts/blob/master/eth_getTransactionBySenderAndNonce.md

I just found that it's already available in otterscan: https://github.com/otterscan/otterscan/blob/develop/docs/custom-jsonrpc.md#ots_gettransactionbysenderandnonce

This issue is so commonplace that when ethers.js waits for a transaction to be mined, instead of polling using the tx hash, it will download every single tx on every single block to try and find the tx or a replacement. Needless to say, this would be better served by this RPC call.

FWIW this call is also available in Erigon

hello, Willian from Otterscan here, I wrote the original spec and implementation in Erigon for usage in Otterscan.
Happy to rename it and make adjustments in Erigon if it’s going to be promoted to eth namespace.
I need to add that current implementation only considers txs already included in a block, i.e. no indexing of txpool

I need to add that current implementation only considers txs already included in a block, i.e. no indexing of txpool

That's interesting. Any reason for that?

The other point which @alcuadrado briefly touched on in the questions is the question of pending vs mined transactions. Right off the bat, I think RPC providers simply cannot return pending transactions. Given the heavy dependence of wallets on providers, it begs the question, is it enough to return only mined transactions by their sender and nonce? It seems they can benefit from this method, but it's not enough? the old way of checking things still has to be kept in case the tx hasn't been mined.

Indeed, as a rule of thumb, RPC providers care about the consistency of their responses to users. A node provider will typically load balance requests to different nodes and will therefore never have a consistent state for pending transactions, consequently most disable it.

Here is the issue for our use case, separate of @alcuadrado independent use case
https://github.com/manifoldfinance/mevETH2/issues/175

I need to add that current implementation only considers txs already included in a block, i.e. no indexing of txpool

That's interesting. Any reason for that?

no special reason, it just happened that we didn't need it.

I just found that it's already available in otterscan<EMAIL_ADDRESS>
ots_getTransactionBySenderAndNonce is also available via anvil now too

The other point which @alcuadrado briefly touched on in the questions is the question of pending vs mined transactions. Right off the bat, I think RPC providers simply cannot return pending transactions. Given the heavy dependence of wallets on providers, it begs the question, is it enough to return only mined transactions by their sender and nonce? It seems they can benefit from this method, but it's not enough? the old way of checking things still has to be kept in case the tx hasn't been mined.

This is a good question, and in fact there already exists such providers that sort of do return pending transaction hashes to share (e.g. flashbots mev share). There is a use case for the RPC method to be supported by block builders. If we are also going to make the distinction between pending and mined, we should consider supporting the special strings:

Block Tags
Description

finalized
The most recent crypto-economically secure block  cannot be re-orged outside of manual intervention driven by community coordination

safe
The most recent block that is safe from re-orgs under honest majority and certain synchronicity assumptions

unsafe
The most recent block in the canonical chain observed by the client this block can be re-orged out of the canonical chain

pending
A sample next block built by the client on top of unsafe and containing the set of transactions usually taken from local mempool

Just giving an update, this has been added as a possible idea for the latest Ethereum Protocol Cohort to work on, see: https://github.com/eth-protocol-fellows/cohort-five/blob/main/projects/project-ideas.md#:~:text=⁃ Specification and implementation of eth_getTransactionBySenderAndNonce%2C see ethereum/execution-apis%23494


Add EOS/USD toggle in protocols

[ ] /protocols add EOS/USD toggle (similar to overview)

Would it be possible to add a toggle functionality for the TVL column to switch between currencies?
Displaying USD by default makes the most sense, but I'd also like to be able to toggle it to see the number denominated in EOS.

Implemented


Fix webview doesn't get focus correctly

For #231.
webview.focus() was used to focus on the <webview>, but it's a DOM element function. So only the tag was focused in the host window, and its webContents was not focused. As the result, the shortcuts related to Mattermost UI didn't work in some cases.
Electron is upgraded to 1.2.8 because of webContents.isFocused().
Tested environment
Windows 10 and OS X 10.11.6.
Shortcuts

Ctrl+F (provided by the app)
Ctrl+K (provided by mattermost UI)

Test conditions

After clicking application window.
After clicking a tab.

Working fine on win10 and solves a big big road block for whenever we release 1.4.0

Would be nice to get some testers, builds can be found here:
https://circleci.com/gh/mattermost/desktop/637#artifacts

Works like a beauty, many thanks @yuya-oc!
Would there be anyone from the community who would like to test this on OS X and Linux? You can download the builds here:

OS X
Linux - ia32
Linux - x64

Steps to reproduce:
TEST 1:

Have two windows open side-by-side: one for Desktop and another for text editor
Click on  the text editor
Click on the webview of the Desktop app
Hit "CTRL+K"

Expected: Channel picker UI appears

Hit "CTRL+F"

Expected: Focus set to the search box, with in:[CHANNEL_NAME] added automatically

TEST 2:

Open the desktop app, and add two teams so that you have two active tabs
Switch tabs
Hit "CTRL+K"

Expected: Channel picker UI appears

Hit "CTRL+F"

Expected: Focus set to the search box, with in:[CHANNEL_NAME] added automatically

On OS X, Test 1 passes but both parts of Test 2 fail.

Thanks @yangchen1  for helping to test this!
@yuya-oc, can you help take look at the feedback provided above?

Thanks @yangchen1, but I can see the channel picker and CMD+F focus. Would you check whether you're using the correct artifact?
Then, the behavior which in:[CHANNEL_NAME] disappears is correct. The app can't change the "state" of React in Mattermost. So if you don't edit the input, the text goes back to ''(blank) by React.

@yuya-oc Yup, tried it again, it's working fine now. Probably had the wrong artifact! :)

Great!
@yuya-oc Given it's been tested on Windows and OS X, I think we're ready to merge

Thanks all!


[Minimum] Features Wishlist

Right now we're working on Core Features, which means that functionality is not included unless it is REQUIRED for core functionality. All other functionality has been pushed aside until Core Feature functionality has a table release. This issue acts as a place to list or request minimal features which are intended as quality of life improvements.
Mock ups for each of these will be created for each one of these bullet points, and it will be decided if they will be included in the next release or not. Items not included will be considered as candidates for the release after that.

Ability to add more organization members (buyers, sellers)
Ability to invite a member as a client or supplier
Ability to encrypt a Verifiable Business Card with a passphrase
Ability to see wallet transactions with a Wallet page in Settings
Ability to transfer funds between members of an organization (seller -> admin, admin -> buyer)
Ability to export an Invoice as a JSON file
Ability to create an Invoice as a draft from an Export JSON invoice file
Ability to perform batch actions on a list of selected invoices in the Tray
Ability to Upload and View Company Logo's
Ability to Upload and View User Profile Images

Features that will not be in the Minimum Release, but the one after that

Notifications
New Node Initialization Wizard
Database Level encryption for sensitive data
Ability to add and remove line items for an invoice
Error handling for Nodes that are offline or unresponsive
Error handling for Invoices that have exceeded the expirationDate
Store and view history of which actions we taken on an invoice by which party

Moving from: https://github.com/WebServiceDevelopment/PrivateInvoice/issues/1
Setting a new node wizard which prompts the user to input settings would be a good first step to help adoption. Alternatively we could have an install script where the user puts in what presets they want for a database connection, ipfs connection, web3 provider, currency settings, and host settings.
I think trying to provide a wizard from the start might be difficult as we develop, we will likely find settings that we will need to include or want to include could change until the application reaches a stable state. We also have the issue of do we want to store all settings in config.json, or should we move everything to .env files?
For now we can use this issue as a sketch pad to keep track of how we will implement the setup and get started experience, as well as which settings are required as opposed to which settings would be nice to have. Right now we'll try to only focus on the required fields,.
Required Fields

Instance host (alpha.privateinvoice.io)
Instant protocol (https)
Instance port (443)
Database settings
Session settings
Currency Format settings

We need OpenAPI documentation and tests. I would be tempted to kick these out, but they're likely going to be needed to demonstrate stability for the Core functionality release. Assuming functionality has been completed before Sept 30 2022, we should have some time to go back and write documentation and tests.

Moved from: https://github.com/WebServiceDevelopment/PrivateInvoice/issues/3
Text is copied over as-is. The functionality described has been listed in the list of Minimum features to be considered for next release.

The figure above shows the layout of the contacts. The first step in the flow is to issue your Verifiable Business Card. Potentially we have options for how long the business card is valid for, the option to encrypt the VBC with a passphrase and if the contact being invited is a supplier, client, or partner.

We also have the issue of how to incorporate options within what's allowed by the Virtual Business Card schema. ExprirationDate can be used from the vc-data model for when the contact information can be invalidated. The decision to encrypt the JSON with a passphrase happens outside of the schema, so it can happen independent of the schema content.
Which leaves the ability for the invite type. While this is not in the schema, it seems that the functionality could be provided in the LinkRole, by providing a keyword for what the relationship is, along with the presentation link. While these features will not be added in the current core functionality implementation, we want to make sure that these features could be implemented in a later revision.

The process of discovery for Organizations and Members is not addressed in the current state of the application. The purpose of the application is to allow organizations to control their own data by creating their own PrivateInvoice nodes, without a centralized node. This issue will be addressed as needed with feedback.

Question around making payments with ETH. Could payments be made with all ERC20 tokens? For example with USDC?

Via: https://github.com/eea-oasis/baseline-grants/issues/76#issuecomment-1136173402


Update request: libreoffice-fresh <IP_ADDRESS> → 24.8

Package name: libreoffice-fresh
Latest released version: 24.8.0

Current version on the unstable channel: <IP_ADDRESS>-wrapped
Current version on the stable/release channel: <IP_ADDRESS>-wrapped

[x] Checked the nixpkgs pull requests

Notify maintainers
@7c6f434c

Note for maintainers: Please tag this issue in your PR.

Add a :+1: reaction to issues you find important.

duplicate of https://github.com/NixOS/nixpkgs/issues/336495


Add parameter for timeout

DBUp supports the .WithExecutionTimeout() property.  This should be mapped to an input variable of the task so that scripts which take longer than the default can be supported.

The task already dissables the execution timeout. Not with the method your are referring to, but through:
$configuration.ScriptExecutor.ExecutionTimeoutSeconds = 0

My idea was that one could use the overall build/release timeout. Du you think it is important to be able to specify timeouts on each script? Or is perhaps the timeout disabling as I have implemented it not working?

I am closing this since I do not think there is any issue with timeouts as currently configured.


Faster, faster!

Performance improvements, focussing on putpixel performance.
Plus a few small bonus fixes.

Change ScreenBuffer from u32 to u8 (assume maximum 8bpp).
Moved screen buffer to RGB conversion to renderer.
Only perform a single clipping operation during putpixel by combining screen bounds and clip rect.
Change color_map and transparency_map to use 256 entry arrays rather than HashMaps.
New implementations of hline(), rect(), rectfill().
Added basic performance test: examples/perftest/perftest.py.
Fixed a few bounds checks.
Don't reset camera on (-1,-1), as this could be a valid request.
Fixed clipping to discard pixels outside clipping rect, rather than inside!

Before/after performance results, showing 4x - 20x improvement :-)
[before]
test                iterations          total_time(secs)    time_per_op(us)
cls                  1000000               6.892               6.892
small_rect           1000000               3.588               3.588
small_rectfill       1000000               9.083               9.083
large_rect            100000               2.728              27.284
large_rectfill         10000               8.283             828.340
text                  100000               1.569              15.695
line                  100000               0.795               7.955
circ                  100000               1.398              13.979
circfill               10000               2.533             253.344

[after]
test                iterations          total_time(secs)    time_per_op(us)
cls                  1000000               0.518               0.518
small_rect           1000000               1.470               1.470
small_rectfill       1000000               1.645               1.645
large_rect            100000               0.256               2.561
large_rectfill         10000               0.397              39.729
text                  100000               0.381               3.812
line                  100000               0.179               1.791
circ                  100000               0.225               2.251
circfill               10000               0.139              13.866

I also added some detailed rendering performance measurements, as I added the new screen buffer to RGB translation there.  This showed the RGB conversion step (the first measurement below) to be minimal (about 50us per frame).
I left these measurements in the code - they can be enabled/displayed using a new "blit_perf" feature (disabled by default).
Render performance
gen_rgb:PT0.000063756S update_tex:PT0.000233939S copy_tex:PT0.003474850S present:PT0.002301755S
gen_rgb:PT0.000047360S update_tex:PT0.000262126S copy_tex:PT0.000026018S present:PT0.015566266S
gen_rgb:PT0.000048056S update_tex:PT0.000292806S copy_tex:PT0.000110042S present:PT0.014779848S
gen_rgb:PT0.000046136S update_tex:PT0.000260689S copy_tex:PT0.000025983S present:PT0.015485252S

I've tested with the demos and games, found and fixed a few issues.  Currently have no known issues with the changes.

Awesome !! The improvement is really huge !
Thanks

I did some tests on raspberry pi and it works really correctly

I added some checks that you removed because I got some crashes:
https://github.com/Gigoteur/PX8/commit/7f34f5859c0cd111f2baa52b528aa52d5cbca43d

I think the impact is not really huge


omgd infra project-destroy works fine but ends with error, terraform wants to update state to bucket that doesn't exist

couple options:

try resetting the backend to local -- first terraform init -reconfigure ... as normal, then swap the backends in the terraform file, then run terraform init..., in which -force-copy is already set & ought to copy the state from the GCS bucket to the local area, which should be easier to destroy w/o errors -- ensure a separate user can destroy the infra without setting it up
try scanning the error output internally - as long as the following 4 lines output, the errors ought not to matter (sloppy, would scale oddly, but)

google_storage_bucket.default: Destruction complete after 2s
...
random_id.bucket_postfix: Destruction complete after 0s
...
google_compute_firewall.default: Destruction complete after 12s
...
google_compute_network.vpc_network: Destruction complete after 51s

#1 worked!


ARXIVNG-970 Documentation + examples for HTTP exceptions

See also https://cul-it.github.io/arxiv-zero/architecture.html#exceptions for an entry-point into this topic.

@DavidLFielding Whoops, the "auth" key is actually something that will change. Here's what we're doing now: https://github.com/cul-it/arxiv-accounts/pull/11/files#diff-022e7b83614a44f7ad7396d70c66de9eR113
Basically you can access session information at request.session. The user data will be at request.session.user.


Startup exception when folders exists that are not an app

Issue
When someone creates a folder under C:\Skyline DataMiner\applications that is not an app. The script will try and load that as if it was and then fail to find needed files.

Solution
Check for directories that are not actual low code apps

This is fixed from version <IP_ADDRESS> onwards.


speed up header encryption / decryption

For some reason, the call to cipher.Block.BlockSize() is insanely expensive:
    244         10ms       10ms           func (a *updatableAEAD) EncryptHeader(sample []byte, firstByte *byte, pnBytes []byte) {
    245         30ms       30ms           	if len(sample) != a.hpEncrypter.BlockSize() {
    246            .          .           		panic("invalid sample size")
    247            .          .           	}
    248         10ms       20ms           	a.hpEncrypter.Encrypt(a.hpMask, sample)
    249            .          .           	*firstByte ^= a.hpMask[0] & 0x1f
    250            .          .           	for i := range pnBytes {
    251            .          .           		pnBytes[i] ^= a.hpMask[i+1]
    252            .          .           	}
    253            .          .           }

My suspicion is that for some reason, the compiler doesn't inline the function call.
Replacing this function call by a comparison to the length of the hpMask slice serves the same purpose, since we allocate hpMask to have a length of BlockSize().
BenchmarkHeaderEncryption-12       	50000000	        24.5 ns/op
BenchmarkHeaderEncryptionNew-12    	100000000	        21.9 ns/op

Codecov Report

Merging #2078 into master will not change coverage.
The diff coverage is 0%.

@@           Coverage Diff           @@
##           master    #2078   +/-   ##
=======================================
  Coverage   85.56%   85.56%
=======================================
  Files          99       99
  Lines        7221     7221
=======================================
  Hits         6178     6178
  Misses        762      762
  Partials      281      281

Impacted Files
Coverage Δ

internal/handshake/updatable_aead.go
91.34% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2133d01...a57b9c7. Read the comment docs.


Bug: Event ack'd after calling KafkaMessage.nack with auto_commit=False

Describe the bug
Events are ack'd after calling nack().
How to reproduce
from faststream import FastStream, Logger
from faststream.kafka import KafkaMessage

broker = KafkaBroker()
app = FastStream(broker)

@broker.subscriber("test", group_id="async_subscriber", auto_commit=False)
async def async_subscriber(message: str, logger: Logger, msg: KafkaMessage):
    logger.info(message)
    await msg.nack()

Expected behavior
After publishing an event to the test topic, the message is logged and consumer does not ack the message.
Observed behavior
The message is logged and it appears the consumer acks the message. In kafka-ui I see the consumer group is 0 messages behind.
Environment
Running FastStream 0.2.15 with CPython 3.11.6 on Linux

Interesting... FastStream doen't call consumer.commit() in this case and passes enable_auto_commit=False to the original AiokafkaConsumer correctly. But, the message still acked...

Seems like with aiokafka >= 0.9 all works fine. All tests at 0.3.5 versions are working correctly, so we can close it, I suppose

This is still not fixed for me. I put a breakpoint down in AIOKafkaConsumer.commit and it appears the push_back_watcher.WatcherContext's __aexit__ method is calling KafkaMessage.ack. I don't quite understand what this Watcher thing is doing, but it looks unrelated to auto_commit. The faststream docs seem to imply that just setting auto_commit=False allows one to manually commit messages, but this Watcher thing commits the message reguardless.

Hi, I'm still experiencing this issue with Faststream 0.47.0.
from typing import Any, Dict
from faststream import FastStream
from faststream.kafka import KafkaBroker
from faststream.kafka.annotations import (
    KafkaMessage,
    Logger
)

broker = KafkaBroker("localhost:9093")
app = FastStream(broker)

@broker.subscriber("source", group_id="foo", auto_commit=False)
async def async_subscriber(message: Dict[str,Any], logger: Logger, msg: KafkaMessage):
    logger.info(message)
    await msg.nack() # calling this or NOT calling this doesn't change the behavior

If I send some messages with
for i in $(seq 1 20); do echo "${i}:{\"index\":\"${i}\"}" | kafka-console-producer.sh --bootstrap-server localhost:9093  --topic source --property parse.key=true --property key.separator=:; done

I see
024-04-12 18:08:56,994 INFO     - source | foo | 110-171293 - Received
2024-04-12 18:08:56,995 INFO     - source | foo | 110-171293 - {'index': '1'}
2024-04-12 18:08:56,996 INFO     - source | foo | 110-171293 - Processed
2024-04-12 18:08:58,556 INFO     - source | foo | 111-171293 - Received
2024-04-12 18:08:58,557 INFO     - source | foo | 111-171293 - {'index': '2'}
2024-04-12 18:08:58,557 INFO     - source | foo | 111-171293 - Processed
2024-04-12 18:09:00,131 INFO     - source | foo | 112-171293 - Received
2024-04-12 18:09:00,132 INFO     - source | foo | 112-171293 - {'index': '3'}
2024-04-12 18:09:00,132 INFO     - source | foo | 112-171293 - Processed
2024-04-12 18:09:01,679 INFO     - source | foo | 113-171293 - Received
2024-04-12 18:09:01,680 INFO     - source | foo | 113-171293 - {'index': '4'}
2024-04-12 18:09:01,680 INFO     - source | foo | 113-171293 - Processed
2024-04-12 18:09:03,355 INFO     - source | foo | 114-171293 - Received
2024-04-12 18:09:03,356 INFO     - source | foo | 114-171293 - {'index': '5'}
2024-04-12 18:09:03,356 INFO     - source | foo | 114-171293 - Processed

I was expecting to consume the same message again and again.
Thanks,
lorenzo

@Lancetnik @lorenzobenvenuti
Right now I have no idea why this is happening. Let me try to replicate the same in local.
And then I will work on the fix.

Thanks @Lancetnik @kumaranvpl . You can reproduce the issue with this application.

I had a quick look at the code, as far as I can see aiokafka doesn't commit the offset but it keeps increasing the value  in memory (see FetchResult._update_position) regardless of the commit policy and this means that, even if the n-th message is not ack-ed, the consumer will consume message n+1, n+2 and so on. I don't know if this is the expected behavior: coming from other frameworks (for example Apache Camel) I was expecting that "nack" was seeking the offset back to the last committed one (or just the previous one? I guess in a scenario where you're ack-ing/nack-ing a message at once you want to proceed one message at a time?).
As a workaround, the consumer can explicitly call seek:
@broker.subscriber("source-topic", group_id="foo", auto_commit=False)
async def async_subscriber(body: Dict[str,Any], logger: Logger, msg: KafkaMessage):
    logger.info(body)
    await msg.nack()
    msg.consumer.seek(TopicPartition(msg.raw_message.topic, msg.raw_message.partition), msg.raw_message.offset)

@lorenzobenvenuti
I think we can just use smth like that:
class KafkaMessage:
    async def nack(self) -> None:
         await self.consumer.seek_to_committed()

And it looks correct, but I am not sure about sideeffects

Hello @bradydean,
This bug has been fixed as part of https://github.com/airtai/faststream/releases/tag/0.5.18 release.
Following is a sample code

from typing import Any, Dict
from faststream import FastStream
from faststream.kafka import KafkaBroker
from faststream.kafka.annotations import (
    KafkaMessage,
    Logger
)
import json
import asyncio

broker = KafkaBroker("localhost:9092")
topic = "kafka-nack-test"

app = FastStream(broker)

@broker.subscriber(topic, group_id="foo", auto_commit=False, auto_offset_reset="earliest")
async def async_subscriber(body: Dict[str,Any], logger: Logger, msg: KafkaMessage):
    logger.info(body)
    await msg.nack()

@app.after_startup
async def publish_something() -> None:
    async def _publish_something() -> None:
        i = 10
        print(f"Sleeping for {i} seconds")
        await asyncio.sleep(i)
        message = {"hi": "there"}
        await broker.publish(message, topic=topic)
        print("Published message" + json.dumps(message))

    asyncio.create_task(_publish_something())


How to set a custom mac as a base mac (IDFGH-13184)

Is your feature request related to a problem?
we have burned a customized mac into efuse. I don't understand where the decision is made to use the customer's mac or the ESP's own MAC.
Describe the solution you'd like.
No response
Describe alternatives you've considered.
No response
Additional context.
No response

Did you take a look at this doc: https://docs.espressif.com/projects/esp-idf/en/stable/esp32/api-reference/system/misc_system_api.html#custom-interface-mac
Does it answer your question?

Yes, I have read it.
I am going to follow Custom MAC Address in eFuse . First use esp_efuse_mac_get_custom to read the MAC in eFuse BLK3, and then use esp_base_mac_addr_set to set the base MAC.
But I read esp-idf, and I don't know where to initialize the wifi interface and ble interface.
Please help me guide.

I found the modification, thank you.

Hi @xmlhp!
You can have a look at this example - https://github.com/espressif/esp-idf/blob/master/examples/system/base_mac_address/main/base_mac_address_example_main.c.
It demonstrates how to use a custom MAC stored in efuse or other places.

But I read esp-idf, and I don't know where to initialize the wifi interface and ble interface.

MACs for these interfaces are derived from the BASE MAC address which can be customized:

If CONFIG_ESP_MAC_USE_CUSTOM_MAC_AS_BASE_MAC==y then pre-defined efuse fields are used.
You can manually set it as esp_iface_mac_addr_set(custom_base_mac, ESP_MAC_BASE).

If there are no other questions I close it. Please feel free to reopen.
Thanks.


Rewrite readme

Changes:

Use name Renovate for the update tool, instead of Renovate bot
Update actions/checkout@v2 to @3, first check if this actually works for you
General style improvements

Context:
Here's a free rewrite of your readme.md. 😉

Thanks ❤


database credentials

When I try to access a moor created database in intellij database plugin it asks for database credentials, what might they be?

When I try to access a moor created database in intellij database plugin

If you're using moor_flutter or moor_ffi, that's an unencrypted sqlite file, so there shouldn't be any credentials. The database is only encrypted with the experimental encryption database that's not available on pub.
It's pretty unusual that it would ask you about credentials for a simple sqlite file though, can you share more details on your setup? In particular, how did you obtain a database file to open in IntelliJ?

@simolus3 i try to access the .sqlite file through device file explorer in debug mode. and then in intellij's standart Database plugin, I try to open the file located on my device it goes "Permission denied", I click detatils (see screenshot 1). it spits out this pop up (screenshot 2)

And it won't let me continue with the fields being empty either

Does it work if you copy the sqlite file to your computer and open it from there? sqlite writes some additional lockfiles when opening the database, that might cause problems on a connected device.

piece of software freezes when I try to copy the file

The IntelliJ database plugin is freezing? Maybe you can use the Android File Transfer tool? This is probably not a moor-specific thing (in the end moor creates the database file like any other Android App would)...

the whole IDE freezes, I'll report to JetBrains and check your suggestion about copying the file. Thanks for help!


Add in screenshot crop and share feature.

Do not merge till i get this restart systemUI stuff sorted out
@akhilnarang  @bdogg718k

No worries.


Kibana doesn't work

Hello,
I wanted to try the OVA Image, but I can't access kibana.

I download the OVA
Put in VirtualBox
Start it
And i wrote:
curl https://<IP_ADDRESS>:5601

I got:

This is super old. I would recommend to use a Docker Compose setup with a current version.
For getting this up and running, what's in /var/log/kibana/kibana.log?

Yeah, I just wanted to have a way to test if ELK Stack was good enough for me.
The file is pretty big, so i'm upload it here :
kibana.log

Looks like Kibana is trying to Elasticsearch on http://localhost:9200 but that should be HTTPS?
And to see if it's good enough: This is 3 years old — I don't think that's giving you a great picture. If you mostly want to see it in action, I'd recommend https://demo.elastic.co. If you want to start it locally, then https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#docker-compose-file

I didn't know about demo.elastic.com, thanks I will use that.


Use tf.math.divide_no_nan here

https://github.com/keras-team/keras/blob/f9336cc5114b4a9429a242deb264b707379646b7/keras/layers/pooling/global_average_pooling1d.py#L93

@ehtshamelahiwm,
Could you please provide context/use-case for the issue, why it is needed to change/update the tf.reduce_sum to tf.math.divide_no_nan.
tf.math.divide_no_nan computes a safe divide which returns 0 if y (denominator) is zero whereas tf.reduce_sum tries to Compute the sum of elements across dimensions of a tensor. Both are absolutely different in computation.
Thank you!

I meant that the division which is taking place in that step should be a safe division. If the tf.reduce_sum(mask, axis=steps_axis, keepdims=self.keepdims)  is zero (which can happen), the global average pooling becomes nan

@ehtshamelahiwm,
Could you please provide the test case for the above requested change. So that it will be easy to analyse the modification mentioned. Also using  tf.math.divide_no_nan doesn't have an impact in the code and tf.reduce_sum provides the output which was expected. Thank you!


fix s3 request header

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (2b95c4c) 84.26% compared to head (2cbe8dc) 84.11%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #164      +/-   ##
==========================================
- Coverage   84.26%   84.11%   -0.16%
==========================================
  Files           4        4
  Lines         661      661
==========================================
- Hits          557      556       -1
- Misses        104      105       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


build(ci): use node orb to install deps

This PR resolves []
Description

use node orb rather than our custom stuff for installing node_modules, get caching automatically, seems like lately there have been lots of cache misses so hopefully this will fix it
use small node executor rather than macOS for the nightly beta deploy, we are just pushing to a branch in this job so don't need anything special

PR Checklist

[ ] I have tested my changes on iOS and Android.
[x] I hid my changes behind a feature flag, or they don't need one.
[x] I have included screenshots or videos, or I have not changed the UI.
[x] I have added tests, or my changes don't require any.
[x] I added an app state migration, or my changes do not require one.
[x] I have documented any follow-up work that this PR will require, or it does not require any.
[x] I have added a changelog entry below, or my changes do not require one.

To the reviewers 👀

[ ] I would like at least one of the reviewers to run this PR on the simulator or device.

Changelog updates
Changelog updates

Cross-platform user-facing changes

iOS user-facing changes

Android user-facing changes

Dev changes

use node orb in ci for installing deps, move nightly to smaller machine - brian

Need help with something? Have a look at our docs, or get in touch with us.

Warnings

:warning:

❌ An error occurred while validating your changelog, please make sure you provided a valid changelog.

  Generated by :no_entry_sign: dangerJS against d8bca80b49a4db258f9dac916256fdd1b8c1a2df

Feel free to merge whenever 🎉


feat(curriculum): add English curriculum block 6.3 clarification on code understanding

Checklist:

[x] I have read and followed the contribution guidelines.
[x] I have read and followed the how to open a pull request guide.
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

@naomi-lgbt Thanks for reviewing the PRs. Should I update the status label after applying the review suggestions?


Upgrade zen-observable to 0.9.0

@benjamn When you get a chance, can you bump zen-observable here and in the @apollo/client to use the 0.9.0 release which features the new Observable.all() feature? Additionally, npmjs.org only shows 1.1.0 so there needs a deployment there as well.

Putting together a PR. The new build of zen-observable releases a class based Observable, not a function based one. This means that all the Observable tests largely fail or are useless. This would have implications for any downstream users that for some reason did not invoke Observable with new() or one of the other methods like Observable.of()

Actually @benjamn I could use your input on this. I've been chatting with the author of zen-observable. His newer versions export a class Observable and not a var Observable = function(). This means the test cases in this repo are unnecessary or that they would all fail.
I can simply put up a PR that bumps the version and kills the tests or we have to wait. I have recommended that @zenparsing refrain from making that change until a 1.0.0 release for SemVer reasons but its pretty unrealistic to treat Observable like a function when its intent has always been for it to be a class. What are your thoughts on how to handle this here in zen-observable-ts?


💄 Consumer view

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#66  👈
#65
#63
#62
#61
#60
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @RTLS and the rest of your teammates on  Graphite


fix(output): #4 multiple inputs should roll into one

This addresses #4
Inputs are added to self.argv and when output() is called, instead a subprocess is made.
Note, this isn't as efficient as creating partial results for each globbing and then later running browserify and as a subprocess and telling it to combine things... but I'm unfamiliar with how to engineer that.
Feel free to provide suggestions.
This code has ONLY been tested on python 2.7.10 with flask and flask-assets from pip latest stable.
If you need a test repo, I could furnish one in approximately a week (busy and this was done with private repos atm)

Would you be able to add a test case as well?

Doesn't sound hard. I'd just include two different inputs and make one output, and check that 'require' is only defined once in the resulting output.
If you can wait a day or two, I'll try it out.

Great, take your time!


How to Use Splat.DI.SourceGenerator

Avalonia comes with ReactiveUI and ReactiveUI comes with Splat. I'm trying to use those default tools.
What bugs me off is that Splat by default needs to manually create each object. Splat.DI.SourceGenerator is supposed to solve that problem.
Now in my project I have Avalonia.ReactiveUI and Splat.DI.SourceGenerator
SplatRegistrations is not recognized. I find no way to use this.
SplatRegistrations.Register<IMenuUseCase, MenuUseCase>();
SplatRegistrations.Register<IOtherDependency, OtherDependency>();

I spent several hours trying to get this to work. What am I missing?
Besides the troubles to get this setup; what are the benefits of going the Splat route vs some more established DI containers like Microsoft DI or Simple Injector?

This is not the splat di repo.

https://github.com/reactivemarbles/Splat.DI.SourceGenerator

I would transfer the issue but it's in another organisation

When you do open the issue in the other repo can you include the ide you are using, your csproj file.
Thanks

It's a separate organization -- so this isn't even supported at all by Splat? Isn't the whole point of Dependency Injection to take care of the object initialization tree? Just trying to understand the logic here.

Marbles is the rxui incubation program. It's getting moved sooner rather than later into the main project.


Setting an appropriate Favicon for this site

I would like to point out that this site https://ebookfoundation.github.io/free-programming-books/ currently does not have any Favicon. But having one would really make a difference for users to glance through the tabs and reach the page very easily. Thus contributing to User experience. Even though this does not exactly fall under UI as mentioned in #9626 or #9766 .
But this small change can really make a difference.
Pls, let me know if I can work on this. I have plans of using royalty-free or completely free icons as favicons that relate to the site.

Please assign me this issue

I didn't know you could set favicons on github pages. We would want to use the favicon we use for ebookfoundation.org, https://ebookfoundation.org/favicon.ico

the website config is in https://github.com/EbookFoundation/ebookfoundation.github.io please move discussion there


Timespan request - tomorrow

Is it possible to have a timespan option which would return the forecast for "tomorrow" similar to what is returned via the VRM web interface? Use case; during the day you want to lookup the forecast for the whole of tomorrow, not now+24hr which may cut off some of tomorrows yield depending on when you run the query.
Thanks

It seems to be a bug in determining when that +24 hr ends. I hope to fix that soon (but accept pull requests if someone else feels like fixing it).

i would need also "tomorrow" if theres a change to geht it - i do also often forcastcalculation in relation to desission of chargingtime

@codeworkx did some of these changes before. Perhaps he has some extra time to check this.

I would also be very happy if this was possible.
Please please please.... 😁
Would it be a matter of adding a 'bot' (beginning of tomorrow) in vrm-api.html and modify vrm-api.js to set the start of tomorrow?
If yes, I'm more than happy to make a PR for this

Sure, go ahead. Probably won't be able to merge it in the next two weeks though.

Sure, go ahead. Probably won't be able to merge it in the next two weeks though.

Great, I made the PR, see
https://github.com/dirkjanfaber/victron-vrm-api/pull/23

Merged it. Creating a new release soon.


Class name not matching?

I have the following HTML:

    <input type="text" class="form-control proj-number-currency" id="txtMinimumNetWorthRequired" name="LicenseNetWorthRequirements.MinimumRequiredNetWorth" value="50000">
    Minimum Net Worth Required

And then my top match for a Number fill-in, I've tried setting to all of these values:
.form., formcontrolprojnumbercurrency, projnumbercurrency, proj-number-currency, currency
And various others to try and match the class... Not matching these elements as I expect, though. Just to be sure, I did try matching on .* to make sure the Number data type was working at all, and it certainly is. In fact, matching does seem to work if the ID field has a match.

This is because the attributes are matched in this order: name, id, class and label. At the moment, there isn't a way to changing this through the options but I am open to ideas on how this can be improved.

Sure. I'm sure people have different opinions, but simply being able to select which attributes I want matched in a filter would work for me. From a use case standpoint as a developer, it would be very strange if I was using the same convention - or mix-matching conventions - across name, id, class and label. It's usually going to be one or another, but not multiple.
Have you seen different examples from your experience?

I haven't seen any examples but from what people are asking, it seems that allowing the user to select which attributes the custom field should match will work. So basically, when you create a custom field, you also have a checkbox for name, ID, class and label. The only issue here is what happens when the extension encounters an input field that doesn't have any matching custom field. Thoughts?

The only issue here is what happens when the extension encounters an input field that doesn't have any matching custom fields.

In that case, I either want the other rules to be checked, or I don't want it to match on any of those fields.
Again, I think the extension is great, but in my own experience, I have a lot of control as a developer over how my fields are named. But each project may use a different convention. Some frameworks or libraries may require that I add a data-whatever attribute to every input of a particular type. Some may use classes. Others might have associated labels...
Those are normally very opinionated things enforced by whatever tool I'm using, and I have yet to run into a situation where the same convention is used across both attributes and classes, for example.


Unity installed by unityhub crashes after opening projects

Describe the bug
If I open a project using unityhub it crashes soon after briefly displaying a loading screen.
To Reproduce
Steps to reproduce the behavior:

install any Unity version (I tried multiple 2019.3.* versions, 2018.4 and 2020.1b)
create a project (doesn't matter which template) or open an existing project

Expected behavior
Running Unity
Logs
~/.config/unity3d/Editor.log (after creating/opening a project from the 3d template)
[LicensingClient] ERROR Failed to connect to local IPC
[Licensing::Module] Failed to connect to channel: LicenseClient-philm
[Licensing::Module] Successfully launched the LicensingClient
FailFast:
Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.

   at System.Environment.FailFast(System.String)
   at System.Globalization.GlobalizationMode.GetGlobalizationInvariantMode()
   at System.Globalization.GlobalizationMode..cctor()
   at System.Globalization.CultureData.CreateCultureWithInvariantData()
   at System.Globalization.CultureData.get_Invariant()
   at System.Globalization.CultureInfo..cctor()
   at System.StringComparer..cctor()
   at System.AppDomain.InitializeCompatibilityFlags()
   at System.AppDomain.Setup(System.Object)
[LicensingClient] Timed-out attempting to connect to the Licensing Client on pipe: LicenseClient-philm.
[LicensingClient] ERROR Failed to connect to local IPC
[Licensing::Module] Failed to connect to channel: LicenseClient-philm
IPC channel to LicensingClient doesn't exist; falling back to Legacy licensing
[LicensingClient] ERROR Failed to disconnect local IPC connection
Desktop is 7680 x 2160 @ 60 Hz

LICENSE SYSTEM [2020529 21:58:25] Next license update check is after 2020-05-30T18:33:28

[Package Manager] Server::Start -- Port 38910 was selected

 COMMAND LINE ARGUMENTS:
/home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Unity
-projectpath
/home/philm/dev/work/EDU/test-nixos
-useHub
-hubIPC
-cloudEnvironment
production
-hubSessionId
***
-accessToken
***
Successfully changed project path to: /home/philm/dev/work/EDU/test-nixos
/home/philm/dev/work/EDU/test-nixos
Using Asset Import Pipeline V2.
Adding genesis user token Adding genesis user token [Package Manager] Done resolving packages in 0.73s seconds
[Package Manager] Done checking package constraints in 0.00s seconds

(Unity:22206): Gtk-CRITICAL **: 21:58:28.798: gtk_container_foreach: assertion 'GTK_IS_CONTAINER (container)' failed
[Package Manager]
Registered 51 packages:
  Packages from [https://packages.unity.com]:
<EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS>  Built-in packages:
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.2d.sprite)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.2d.tilemap)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.ugui)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.ai)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.androidjni)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.animation)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.assetbundle)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.audio)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.cloth)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.director)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.imageconversion)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.imgui)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.jsonserialize)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.particlesystem)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.physics)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.physics2d)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.screencapture)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.terrain)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.terrainphysics)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.tilemap)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.ui)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.uielements)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.umbra)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.unityanalytics)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.unitywebrequest)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.unitywebrequestassetbundle)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.unitywebrequestaudio)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.unitywebrequesttexture)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.unitywebrequestwww)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.vehicles)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.video)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.vr)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.wind)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.xr)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.subsystems)

[Subsystems] No new subsystems found in resolved package list.
[Package Manager] Done registering packages in 0.00s seconds
[Unity Package Manager (Upm)]
Parent process [22206] was terminated

~/.config/unity3d/upm.log
[2020-05-29T19:58:26.125Z][INFO] Starting Server
[2020-05-29T19:58:26.128Z][INFO] Server started on port [38910]
[2020-05-29T19:58:26.223Z][INFO] Health Request received
[2020-05-29T19:58:28.169Z][INFO] Resolving dependencies by depth
[2020-05-29T19:58:31.131Z][ERROR] [Unity Package Manager (Upm)]
Parent process [22206] was terminated

~/.config/UnityHub/logs/info-log.json (relevant parts)
{"moduleName":"LocalProject","level":"info","message":"[ 'openProject projectPath: /home/philm/dev/work/EDU/test-nixos, current editor:',\n  undefined ]","timestamp":"2020-05-29T19:58:20.352Z"}
{"moduleName":"RestEvent:Project","level":"info","message":"[ 'onProjectOpen data:',\n  { projectPath: '/home/philm/dev/work/EDU/test-nixos' } ]","timestamp":"2020-05-29T19:58:20.352Z"}
{"moduleName":"UnityIPCServer","level":"info","message":"[ 'client process has connected to /tmp/Unity-hubIPCService.sock!' ]","timestamp":"2020-05-29T19:58:25.618Z"}
{"moduleName":"LaunchProcess","level":"info","message":"[ 'child process exited with code null' ]","timestamp":"2020-05-29T19:58:30.154Z"}
{"moduleName":"HubIPCService","level":"info","message":"[ 'socket has disconnected: [object Object] , false!' ]","timestamp":"2020-05-29T19:58:30.154Z"}

The logs from the 3d template based project also don't say very much interesting.
Packages-Update.log (the only log in /Logs of the project)

=== Fri May 29 22:22:57 2020

Packages were changed.
Update Mode: mergeDefaultDependencies

The following packages were added:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
Additional context
It's kinda weird I had Unity running when I installed unityhub the first time (around 1 1/2 months ago)
But suddenly (a few days after that) it started crashing, soon after opening a project  (I guess it was a nixos-rebuild --upgrade) .
I think I invested now multiple hours to debug this issue, but I'm at the end of my abilities (as I'm also rather new to NixOS) since the error logs are rather sparse regarding this issue.
I deleted all kinds of side-effects of Unity Hub and Unity. At least I think that I deleted all of them: I searched for all kinds of unity variations in the home folder, so I hope I was thorough enough.
Fortunately I got it working with flatpak, but there I have now other issues (they have no git included which is needed for some UPM packages)
But the flatpak Editor.log (in which it's working) might be interesting for context:
Editor.log (I've ommitted most of the log, as it was very long, and in my opinion not relevant for this issue)
[LicensingClient] ERROR Failed to connect to local IPC
[Licensing::Module] Failed to connect to channel: LicenseClient-philm
[Licensing::Module] Successfully launched the LicensingClient
[LicensingClient] Timed-out attempting to connect to the Licensing Client on pipe: LicenseClient-philm.
[LicensingClient] ERROR Failed to connect to local IPC
[Licensing::Module] Failed to connect to channel: LicenseClient-philm
IPC channel to LicensingClient doesn't exist; falling back to Legacy licensing
[LicensingClient] ERROR Failed to disconnect local IPC connection
Desktop is 7680 x 2160 @ 60 Hz

LICENSE SYSTEM [2020529 22:37:54] Next license update check is after 2020-04-28T15:10:12

[Package Manager] Server::Start -- Port 59652 was selected

 COMMAND LINE ARGUMENTS:
/home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Unity
-projectpath
/home/philm/Downloads/test-nixos-lts
-useHub
-hubIPC
-cloudEnvironment
production
-buildTarget
Android
-hubSessionId
***
Successfully changed project path to: /home/philm/Downloads/test-nixos-lts
/home/philm/Downloads/test-nixos-lts
Using Asset Import Pipeline V2.
Rebuilding Library because the asset database could not be found!
Adding genesis user token Packages were changed.
Update Mode: updateDependencies

The following packages were added:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
(Filename: ./Modules/PackageManager/Editor/Requests/PackageManagerUpdateAsyncRequests.cpp Line: 155)

Adding genesis user token [Package Manager] Done resolving packages in 1.44s seconds
[Package Manager] Done checking package constraints in 0.00s seconds

(Unity:102): Gtk-CRITICAL **: 22:38:06.022: gtk_container_foreach: assertion 'GTK_IS_CONTAINER (container)' failed
[Package Manager]
Registered 48 packages:
  Packages from [https://packages.unity.com]:
<EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS><EMAIL_ADDRESS>(location<EMAIL_ADDRESS>  Built-in packages:
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.2d.sprite)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.2d.tilemap)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.ugui)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.ai)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.androidjni)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.animation)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.assetbundle)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.audio)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.cloth)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.director)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.imageconversion)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.imgui)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.jsonserialize)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.particlesystem)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.physics)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.physics2d)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.screencapture)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.terrain)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.terrainphysics)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.tilemap)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.ui)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.uielements)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.umbra)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.unityanalytics)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.unitywebrequest)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.unitywebrequestassetbundle)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.unitywebrequestaudio)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.unitywebrequesttexture)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.unitywebrequestwww)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.vehicles)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.video)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.vr)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.wind)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.xr)
<EMAIL_ADDRESS>(location: /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/PackageManager/BuiltInPackages/com.unity.modules.subsystems)

[Subsystems] No new subsystems found in resolved package list.
[Package Manager] Done registering packages in 0.00s seconds
Targeting platform: Android
Refreshing native plugins compatible for Editor in 0.00 ms, found 0 plugins.
Preloading 0 native plugins for Editor in 0.00 ms.
IsTimeToCheckForNewEditor: Update time 0 current<PHONE_NUMBER>
Initialize engine version: 2019.3.11f1 (ceef2d848e70)
[Subsystems] Discovering subsystems at path /home/philm/Unity/Hub/Editor/2019.3.11f1/Editor/Data/Resources/UnitySubsystems
[Subsystems] Discovering subsystems at path /home/philm/Downloads/test-nixos-lts/Assets
...

Btw. while debugging I tried the recently new released Unity Hub version (2.3.2) with local nixpkgs. I already uploaded it to archive.org with the hash:
07nfyfp9apshqarc6pgshsczila6x4943hiyyizc55kp85aw0imn
I also think that git should be in the extraPkgs as well (as mentioned for flatpak)
Notify maintainers
@tesq0

Metadata
 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.42, NixOS, 20.09pre227294.46f975f81e0 (Nightingale)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.4`
 - channels(philm): `""`
 - channels(root): `"nixos-20.09pre227294.46f975f81e0"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Yes, I experience the same issues. Worth noting, I tried manually running Unity executable installed by Unity Hub and for latest LTS this worked:
steam-run ./Unity/Hub/Editor/2018.4.23f1/Editor/Unity

But it's still pretty unusable because with this approach UI is too buggy (for example dialog windows are closing themselves straight after opening, so there's no way to do many configuration things).


QUIC: Add refcount to protect qvc

Splits free function into 3 parts:

cleanup the sub components like stream manager, packet handler and so on. (ET_NET)
cleanup connection ids from ctable (ET_UDP)
destroy connection (ET_UDP)

It is not the solution for  use-after-free on HTTP1+ UnixNetVC.
It is used for protect qvc  from QUICPollEvent.
Considering this case, we may received udp packet from ET_UDP then create a QUICPollEvent and push to ET_NET. But ET_UDP may destroy this qvc , refcount will give ET_NET another chance to check in_closed_queue member.
For  use-after-free on HTTP1+ UnixNetVC,  there are some suggestions.
we could divide  into 2 types:

Positive close, someone call qvc:do_io_close to close the stream(may be useless).
negative close, timeout receive close frames, stateless reset, idle timeout. It should call sm immediately with specified event(Every opening stream should inc the refcount and stream do_io_close will dec it). If refcount == 0, that means every sm is terminated, we can free this qvc .

TCP connections is different from qvc connections, since TCP uses connection but QUIC uses stream(I'm not sure what we do in h2). So if TCP connections closed, tcp kernel will reject all packet.
if we got the connection error, SM will call client or server session do_io_close(UnixNetVConnection::do_io_close) to close this connection and terminate itself, so we won't get use-after-free anymore.

Sorry, I still don't understand the design.
Both QUICNetVC (for QUIC) and UnixNetVC (for TCP) need to be freed after all users of the NetVC are terminated. Otherwise the users can refer a NetVC that is already freed. And I agree ref counting would solve the issue.
However, classes for TCP (UnixNetVC, UnixNetHandler, etc.) ensure it without ref counting somehow. Why is it possible for TCP connections? Why is it impossible for QUIC connections?
I'm fine with either ref counting or the one used for TCP, however, IMO, both NetVCs should be managed in the same way if possible. Maybe there would be several differences due to TCP vs UDP, but I don't see why the differences make a need for ref counting. This is the reason I suggested ManagedConnection interface class.

For example:
If UnixNetVConnection get an error from tcp kernel, it will signal SM with an error event (VC_EVENT_EOS, VC_EVENT_ERROR or VC_EVENT_XXTIMEOUT). Then SM will call UnixNetVConnection::do_io_close to close the connection (con.close) and never access the UnixNetVConnection again. See https://github.com/apache/trafficserver/blob/master/iocore/net/UnixNetVConnection.cc#L301.
But QVC is a special vc. It could separate  into several streams. IMO QUICStreamIO is more similar to UnixNetVConnection than QVC (It may be wrong), since we use HQClientTransaction to control the QUICStreamIO in QUIC. In TCP, we use HttpClientTransation to control the UnixNetVC.
In TCP Connection : SM = 1: 1, but in QVC : SM = 1: n and QUICStreamIO : SM = 1 : 1. And All streams belong to one QVC. So if QVC got an error, it should signal every QUICStream with an error event, and QUICStream must signal the SM then SM will call do_io_close to close then error Stream. Finally, QVC should be freed after all streams have been closed.
So TCP NetVC doesn't not need refer counting, it just signal the SM with an error event if a connection error occur. And SM will callback to NetVC(by do_io_close) to close the connection.
I'm not sure ManagedConnection  interface class could do this, could you please give more detail ?

Thanks for the clarification. Now it gets clearer.
H2 has streams and they share one NetVC, but it is managed somehow without ref counting. So maybe we can do the same thing for QUIC, I'm not sure which design is better though. If you think using ref counting is better, I'm happy to accept your design.

I'm not sure ManagedConnection interface class could do this, could you please give more detail ?

I just want to destroy all kind of NetVCs in the same way. If ref counting or a magic for H2 work for the both NetVCs, they will have a common code, such as checking current ref count, like shouldDestroy() does. Those code should be in a common parent class. Even if there is no common code, what we need to do is basically the same. Prepare for closing, wait for ready to close, and close. Internal implementation may vary, but the sequence should be able to be generalized. So ManagedConnection itself doesn't solve the issue but I think it makes the sequence consistent and help understanding how NetVCs are freed.
And I guess ref counting work for all kinds of NetVCs because 1:1 is just a special case of 1:N. Using ref counting for UnixNetVC might be overkill, but I prefer consistency. But I'd like you to look into how H2 manage NetVC before proceeding with ref counting.

@maskit @scw00 not looking into the detail, I noticed ++recursion, --recursion in Http2ClientSession::main_event_handler. If this is the magic you guys are talking about, I personally prefer refcountobj.

@zizhong Thanks for the info. Yeah, it should be a part of the magic. I guess it ensure that we are on the bottom of event handling stack because handleEvent is called in the middle of event handling in some cases.
IIRC, we check open stream count somewhere. Maybe we use the stream count instead of the count of reference for NetVC. If it is correct, RefCountObj is definitely a better way.

@shinrich Do you have any concern about using ref count for managing NetVC life time?

[approve ci autest]

updated !

[approve ci debian]

[approve ci debian]

@shinrich Please let me know if you have any concern about making NetVC ref count object. For now, only QUICNetVC will use RefCountObj, however, I think we should destroy and free all NetVCs in a consistent way ideally. Does this affect your server side session design?
@scw00 I'm going to merge this after interop during IETF 101 if no concerns are raised.

@scw00 Please resolve conflicts.

@scw00 Sorry, I made another conflict with my last commit. Please resolve the conflict again, and ping me (or masaori) when it gets ready.

[approve ci autest]

[approve ci]


Access to shared storage from Linux VM in workspace?

Description
In my Azure TRE deployment I am trying to access shared storage from a Linux VM, deployed via guacamole. I can see there's a storage account, and it has a file share in it of the right size, but when I create a Linux VM I don't see anything but the root filesystem. I would expect to see my file share mounted there.
Without the shared storage, I'm stuck with only the local storage on the VM. That makes it hard for me to share my work across multiple machines - e.g. a small one for development and a larger one for heavy computation, which may be switched off much of the time. It also means I lose my work completely if I forget and delete my VM.

we last merged the upstream repo from commit ea63ebf, from Tim on 09/04.

I've tested on main and this is the drive in a linux vm

Hi Tim,
thanks, that helps, and I can see the drive mapped there.
However, if I access the shared drive from a linux VM and from a windows VM, they don't seem to be seeing the same space. I can create files on either machine in the 'shared folder', but they don't appear on the other machine, even after waiting a fair amount of time.
Any suggestions?
Cheers,
Tony

From: Tim Allen @.>
Sent: 03 June 2024 13:18
To: microsoft/AzureTRE @.>
Cc: WILDISH, Tony (BARTS HEALTH NHS TRUST) @.>; Mention @.>
Subject: Re: [microsoft/AzureTRE] Access to shared storage from Linux VM in workspace? (Issue #3959)
This message originated from outside of NHSmail. Please do not click links or open attachments unless you recognise the sender and know the content is safe.
image.png (view on web)https://github.com/microsoft/AzureTRE/assets/139883291/dc3248c2-247f-4b07-8fde-8cdd282bb338
I've tested on main and this is the drive in a linux vm
—
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/AzureTRE/issues/3959#issuecomment-2145050374, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BEQ2NMUNQBZBMRSTLBOKDJTZFRNHVAVCNFSM6AAAAABITDLU6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBVGA2TAMZXGQ.
You are receiving this because you were mentioned.Message ID: @.***>

This message may contain confidential information. If you are not the intended recipient please:
i) inform the sender that you have received the message in error before deleting it; and
ii) do not disclose, copy or distribute information in this e-mail or take any action in relation to its content (to do so is strictly prohibited and may be unlawful).
Thank you for your co-operation.
NHSmail is the secure email, collaboration and directory service available for all NHS staff in England. NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and other accredited email services.
For more information and to find out how you can switch visit Joining NHSmail – NHSmail Supporthttps://support.nhs.net/article-categories/joining-nhsmail/

digging deeper, that doesn't seem to be the shared storage folder after all. It looks like the shared storage is mounted on /fileshares/vm-shared-storage on the Linux boxes, so this thinclient_drives must be something else, looks like an internal Guacamole sharing protocol?


Feature Request: Show column comments in MetaData Viewer and as an option below grid column name

Within the MetaData panel you can see a lot of information about the columns in the table.
however the thing that is missing is the column description information.
The column description information is only available by hovering over the column names, which then show it within a tooltip.
I would much rather have it show up in the meta data panel, which I can move to my other monitor or show up beside the results so I know what I am viewing.
Additionally if possible, it would be awesome if there was a way it could show up underneath the column in the grid panel, which would help those who like to see it directly next to the data they are viewing (better UI experience). Actually both in the grid and the Metadata panel would be ideal, so people can do it anyway they wish.
FYI, this is very useful for legacy databases which have very cryptic column names that are limited to 8-10 characters (depending on database type, such as IBM i DB2 / AS400 databases). Often times the programmers put in better info on what the column meant in the column description fields since they were so limited in column names.

FYI, this is very useful for legacy databases which have very cryptic column names that are limited to 8-10 characters (depending on database type, such as IBM i DB2 / AS400 databases). Often times the programmers put in better info on what the column meant in the column description fields since they were so limited in column names.

Well, I got the point but that's not so easy.
Metadata panel shows metadata of current SQL statement - exactly as it returned by driver (even if you are viewing table contents - it is a result of SELECT * FROM table query).
The problem is - column description is not returned along with other column properties. It is a limitation of JDBC API (and most of other database APIs) - just because it is redundant in most cases.
Column description can be obtained only during table metadata extraction.
Possible solution: I can show column descriptions (if they are known) in grid header on the second line. Like this:

CRAZYNAMEgood name
MADNESSrational name

value
value

value
value

value
value

Of course this should be configurable.
The problem I see - how to handle long descriptions. They can be very long, even multiline.
Another solution (or workaround) - I still can push column description in metadata panel (this will be a kind of hack but I can go this way if there are no other options).

Both options are fine with me. Just as long as I don't have to hover over every column to figure out what it is (which is what I do now). I do see that long descriptions may pose a problem, with our DB2 database I believe the limit is 105 characters in length. I believe in MS SQL you are allowed much longer lengths. Which is why it probably is important to have it available in both places and folks can either turn it on in both the column/grid view, or only show it in the metadata panel view. Metadata panel would would be preferable for very long descriptions as it shows as a row based display which can be moved to another monitor if needed.

Also, this information must already be pulled as they are in the tooltip, so i'm guessing it just needs to be addeed to the grid as a second row.
FYI, this feature is in SQL Workbench/J, which is open source. They have a different way of showing the metadata panel than DBeaver.
Honestly the interface in DBeaver is better, and intellisense works better as well. So if this feature gets added it would definately be my goto SQL Editor as everything about DBeaver is easier to use. And unfortunately IBM's Data Studio and their IBM i specific SQL Editor is light years behind what DBeaver offers.

Optional descriptions in grid header will be added in 3.8.3.
Please provide some feedback.

Just tried this out. AWESOME! Thanks so much for adding it.

Changing metadata panel is quite complicated (panel meta model has to be refactored). Let's consider this task as done with just grid header improvement.
You can create separate issue for just metadata panel. There could be some other improvements besides description column.

This option is quite hard to find, and I think it would be much convenient to add it to right click menu.

anyone knows where that option is in dbeaver 23?


fix length check in encode()

Hi!
This changes encode() to validate the length of the hash function's output, not the input.
Also renames the input parameter from digest to content, to reflect that the function accepts un-digested input data and performs its own hash digest, and updates the doc comment.
Should close #1

Thanks!


Selenium testing can not pick up star rating

I was trying to record a test case on selenium testing and when i try to record star rating selenium does not pick the click

Hello, I've never used Selenium so I'm not sure how to help you with this. You may be able to get more help from StackOverflow. I'm going to close for now, but if you do find there is a problem my end let me know and I'll re-open the issue.


CoffeeLint all warns changes to errors

CodeClimate shows only errors. We should keep track of all style issues.

Coverage remained the same at 32.078% when pulling a32f6a629238ca8fa522e20c15317a63dd696fba on codeclimate_all_errors into fd6c5f911373114609fd463bd6a53bb79b60a901 on master.


Asynchronous JPEG encoding interface

Allow users to use a JPEG-encoding API in PyFunc to submit crops of the GpuMat for hardware-accelerated JPEG-encoding.

nvJPEG on dGPU;
Jetson JPEG API on Nvidia Jetson

Can be implemented with auxiliary streams.


heap and parser errors on import

CrateDB version
5.3
CrateDB setup information
docker
Problem description
When trying to import a 2.6G file in v5.3, with heap set as high as 3g, it would run OutOfMemoryError. Up to 6g, and it failed on a circuit breaker. Yet, I can import the same vile in v 4.5.4 with only 500m heap.
Discovered the issue was that when you have a parse error, it just gobs up memory like crazy in v5. With v4, it correctly ends with a parser error with RETURN SUMMARY, even with only 500g heap. When I fixed the parser error, I was able to import with 1g heap with v5, and successfully with v4 with 500m heap. I did not try lower than 1g with v5.
Steps to Reproduce
Create JSON.  leave out the end quote on an attribute (this resulted from an incorrect SED).  Create a ton of rows until you have a very large file to import.  Alternatively, if you have a large file you can import, just use SED to get rid of a closing quote on a property name.
Actual Result
In description.  Sorry, didn't capture the exact error when I had a parser error.  Just reporting so you can get v5 as good as v4.
Expected Result
no OOM or circuit breaker.  a good summary of the number of rows with a parser error.  My import had 3594282 errors when it failed, and 3594282 successes when it worked.

Thanks for the report. We have an issue about this already https://github.com/crate/crate/issues/14133


[Docs] Add Menu ARIA Example

Hi,
I think SemanticUI is great and while AlloyUI and AttlassionUI are the only frameworks I know that follow these http://www.w3.org/WAI/intro/aria.php Guidelines , it would be awesome if SemanticUI would add this feature, which I think separates a really serious UI Framework from a "prototyping only" Framework.
I believe any help in giving more access to the web to people who have been unluckily disabled is a good thing to do.

I hope no one minds me adding my 2 cents here, but a good example of where Semantic UI falls short on accessibility (and why my company has had to turn down using it despite it being one of the nicest most comprehensive frameworks) is things like setting aria-expanded role on widgets like accordion.
I shouldn't have to add that functionality because in the end to maintain those roles properly is a responsibility of the JS widget, not something I want to patch or modify.
I hope this feedback helps the discussion. :)

+1 for all the accessibility issues here.
I've switched to foundation a couple of years ago: I'm getting disappointed by their lack of support (and I need some more element to play with).
So I was thinking about semantic: I've been following the growth of your project and now that I'm starting a new project of mine I'm really considering accessibility issues.
Something like foundation does would be enough for me to decide and adopt semantic UI, and I guess it would be the same for other developers/UI specialists: (scroll down to "Accessibility") http://foundation.zurb.com/docs/components/accordion.html
Consider this also from a strategy perspective: if people concerned about accessibility could use your framework, they realistically would adopt it easily. More people who use it -> more support and community discussions -> best framework. Isn't it?

I'm all in with helping developers know what WIA-ARIA is and how can help the end user/customer when visiting their content, but I also understand that this has nothing to do with the CSS Framework we choose as I can see @X4 is trying to make it as-one.
We, third-party developers, could easily develop a plugin that would add missing ARIA tags like:
$(".ui.menu").attr("role", "menubar")
$(".ui.menu a, .ui.menu li").attr("role", "menuitem")

but then again, how can we know that we should also add
aria-haspopup="true" aria-labelledby="fileLabel"

that is also important, as well so many other examples that will come with experience.
ARIA should be taken seriously from the developer, not the CSS Framework, though, as said before, we could make something to help the ones that have a rough idea what ARIA is, but have no idea what should be done and what those tags represent and make that plugin to provide really basic stuff like @samueljseay says about the accordion ...
But no mater what road is taken, the developer needs to understand ARIA and add the missing tags...
Other example, could start from the docs, just like Bootstrap does in some examples, and add the ARIA code in the example itself creating aware in developers...

Balexandre thanks for sharing your perspective. I couldn't agree more with everything you're

I just discovered this thread and I'd like to weigh in on the discussion. I'm interested in the topic because my previous job involved a lot of ARIA-related work and in particular interfacing with my company's accessibility team.
My opinion is that it is not reasonable to expect application developers to add ARIA properties to the DOM, because this is a complex task that involves many subtleties, even for simple widgets. While it's possible for the average developer to make this "sort of" work, making an accessible experience that is seamless and polished takes a lot of expertise.
A classic example of this is 'focus trapping' in modal dialogs, which is a recommended practice for the WAI-ARIA standard. This means that when a user is tab-navigating between fields in a modal dialog, when the focus gets to the last field in the dialog it should wrap around to the first field, not go off and focus some random part of the page. (Almost all of the existing JS widget libraries get this wrong, Polymer is one of the few that even tries to get it right.) Doing this correctly is not obvious (especially if you want to support autofocus), and should ideally be a part of the standard model dialog behavior, something that comes for free when you use a widget library, not something the application developer cobbles up themselves.
Other examples, BTW, include things like radio button focus behavior, dynamic states for selection and highlighting, handling disabled items in a way that allows them to be discoverable by a blind user, and just generally making the whole UI able to be keyboard-driven.
To say that "we only supply the visuals, behavior is up to you" is basically to say that your library is incomplete. In general, only 3rd party library developers have the time and expertise to make robust, seamless behaviors that work for many kinds of input methods and devices; to leave that to the application developer, who is generally only focused on short-term feature goals, is a recipe for chaos.

Hi,
it's been a while, but I've been actively following the very interesting and constructive discussion here. I'm greeting everyone involved in this thread :squirrel:
I think the most sensible solution to all parties involved is to delegate the issue in a way that it doesn't interfere with the Semantic-UI Framework itself, but is also not pushed into the hands of the regular developer that is not 100% fond of the standard.
Therefore the best solution is to provide a visual or declarative language that can generate the required or specified WAI-ARIA meta-data accordingly. I think forking a good Bootstrap Editor/Generator or GUI would be a more integrated and direct solution than compromising on generic defaults that may or may not apply depending on the use by the developer.

I just discovered Semantic-UI and I think it may be worth it to invest in accessibility. I mean you have a very fine framework here and for example Bootstrap also wasn’t accessible from the beginning, but they improved very much. It would be a big step forward for Semantic-UI and the effort it would take is manageable. :)

+1 to investing in accessibility.
I agree that for the time being a lot of ARIA stuff can be done manually in the DOM, but what I'm concerned most with are things that (can't be, and) require hacking the JS widgets.
A concrete example would be, say, radio buttons. Whatever you do in the DOM, the actual input element is hidden and ignored by the browser, thus there's no way to interact with it in any way whatsoever, unless you use a mouse.
The lib already looks better than most of the competition, so if you'd also invest some time in under-the-hood functionality, it will definitely be an awesome and complete package.

I'd just like to note that lack of accessibility is a massive legal minefield in the US, thanks to case law regarding the Americans with Disabilities Act. So I don't see how Semantic-UI can honestly be described as "Enterprise Ready" (as per the home page) until its accessibility issues are fixed.

@lpar You're looking at an abandoned repo. Please check Fomantic-UI

OK

A classic example of this is 'focus trapping' in modal dialogs, which is a recommended practice for the WAI-ARIA standard. This means that when a user is tab-navigating between fields in a modal dialog, when the focus gets to the last field in the dialog it should wrap around to the first field, not go off and focus some random part of the page.

We added support for that "only allow tabbing within current modal" in the community fork Fomantic-UI by https://github.com/fomantic/Fomantic-UI/pull/2036


🐞 "pointerenter" and "pointerleave" should also be triggered on groups containing the raycasted object

At the moment, "pointerenter" and "pointerleave" -- which we're simulating by comparing intersections between raycasts -- are not propagated to groups owning the raycasted object.
(And we should also think about if we really want this behavior. If we do it for groups, do we also do it for parent meshes? This may not always be what we need.)

This actually works. Not sure why I opened this issue.

This actually works. Not sure why I opened this issue.


Create version.h from available version information

Signed-off-by: Stefan Weil<EMAIL_ADDRESS>

What about installing this header file like the other public headers?

Sure, that could be a next step. Tesseract 3.05 and Tesseract 4.0.0-beta.1 use different approaches: while 3.05 (and also 4.00.00alpha) has the version information directly in baseapi.h, 4.0.0-beta.1 has that information in version.h which is included by baseapi.h.
I think that the latest API is not easily manageable: the Debian package installs 58 (!) include files to /usr/include/tesseract and adds that directory to the compiler's search path for include files. How will users know which of those include files should be included directly in their code? How can name conflicts be avoided? version.h, but also network.h and other names used are very generic, so chances of name conflicts are very high.