Long term looping all elements

TL; DR -- Full V3 "Looping" capability implemented
This PR implements the full input interface functionality of the NWM routing module simulation elements with

[x] Long-term simulation for large domains (called "looping" or "V3" execution)
[x] Reservoir simulation capability (newly integrated into V3)
[x] Data assimilation capability (newly integrated into V3)
Follow-on PRs will address the following
[ ] Handling non-numeric gage ids (from Canadian and other non-USGS databases)
[ ] Trigger LastObs file write
[ ] Incorporate asynchronous file I/O at the looping level

Note: The V3 designation is only loosely tied to the NWM v3.0 version number for which this capability is targeted; Semantically, this essentially represents a new version of the model, as the input version is now completely different and significant new functionality is introduced.
Major work

Update front-end interface
Split LastObs and TimeSlice preprocessing to recognize relation to initial states vs. continual forcing
Add additional return from compute kernel with LastObs updated through simulated series
Use returned lastobs with other warm states
Adjust logic for TimeSlice read to accommodate repeated calls in consecutive loops
Catch bugfixes in the less-tested looping execution pathway
Prepare a test case utilizing all input capacity (thanks @jmccreight!)
...

Changes to front end
V2: Existing workflow to execute routing calculations and control outputs
python -m nwm_routing -V2 -f CustomInput.yaml
The .yaml file architecture you are currently used to seeing
V3 (“looping”): New workflow permitting sequentially connected simulations
python -m nwm_routing -V3 -f CustomInput_loop.yaml
A new .yaml configuration where user specifies forcing files for each simulation batch in the sequence (next slides)
Modified DA Functions
Functions to handle reading of data assimilation timeslice files and lastobs values were split to permit alternate handling in the different parts of looped execution.
The updated functions are here
https://github.com/jameshalgren/t-route/blob/a4b1d2084c2ffe7e8104ba0222fdd5a317369afe/src/python_framework_v02/troute/nhd_network_utilities_v02.py#L629-L691
With the new calls split here:
https://github.com/NOAA-OWP/t-route/blob/a4b1d2084c2ffe7e8104ba0222fdd5a317369afe/src/nwm_routing/src/nwm_routing/preprocess.py#L328
... and here:
https://github.com/NOAA-OWP/t-route/blob/a4b1d2084c2ffe7e8104ba0222fdd5a317369afe/src/nwm_routing/src/nwm_routing/preprocess.py#L232-L234
... and the unmodified V2 version call here:
https://github.com/jameshalgren/t-route/blob/a4b1d2084c2ffe7e8104ba0222fdd5a317369afe/src/nwm_routing/src/nwm_routing/__main__.py#L576-L578
(notice how the prior version uses a combined call.)
Return LastObs from Compute Kernel
With this simple change, the compute_network_structured compute kernel now returns an additional element in the return array -- a tuple of arrays of the form ([gage ids], [timesincelastobs], [lastobsvalues]):
https://github.com/NOAA-OWP/t-route/blob/a4b1d2084c2ffe7e8104ba0222fdd5a317369afe/src/python_routing_v02/troute/routing/fast_reach/mc_reach.pyx#L1416
This change affects the kernel which will be used for operational execution; an issue has been logged to include the same functionality with the other compute kernels. A placeholder value (just a '0') is used for the return of diagnostic outputs such as courant number, which are returned by other kernels but are not yet possible to return from the operational kernel (see #362 and #380).
LastObs Warmstate
The LastObs arrays returned from the last timestep of the calculation are refomatted into the dataframe expected as input with this new function:
https://github.com/jameshalgren/t-route/blob/a4b1d2084c2ffe7e8104ba0222fdd5a317369afe/src/nwm_routing/src/nwm_routing/__main__.py#L956-L971
The call to this new function is grouped with the other calls to prepare warmstates to 'bootstrap' the execution of the next loop here:
https://github.com/jameshalgren/t-route/blob/a4b1d2084c2ffe7e8104ba0222fdd5a317369afe/src/nwm_routing/src/nwm_routing/__main__.py#L1122-L1126
Bugfixes
See principally commits in
https://github.com/NOAA-OWP/t-route/pull/429/commits/e640e8cdf0b1bcd4d1c15d7b16e50327b21a13bb
https://github.com/NOAA-OWP/t-route/pull/429/commits/69f6f4aa4883c43a333052009f1d4ec6cf634cdb
Test case
A test case with five minute output, gage data assimilation, and reservoir simulation was prepared for testing on the Cheyenne computing infrastructure.
Testing/Screenshots
These screenshots show output from our "parity check", which is a simple comparison of WRF-Hydro simulated output and the same timeseries from the t-route model. A more comprehensive statistical comparison tool is in progress (and will help a lot!!) For now, we see that the V3 update retains perfect fidelity with the earlier V2 capability, and produces no change in the fidelity with respect to the WRF-Hydro output.
V2 vs V3
Gage site 8778363
Tests on the Cheyenne HPC system confirm that V2 and V3 produce identical results:
V3 first five timesteps parity check at gage site 8778363:

Last five (split up because the looping breaks the simulation into several chunks):

V2 first and last five at same site, with summary statistics:

Inflow to Lake Butner
V3 at 8777125, inflow to Lake Butner

V2, same

Falls Lake
Reservoir Outflow, DA OFF
V3 at 166737579, Falls Lake outflow first and last five minutes Data Assimilation OFF. Note the deviation at the end of the time series due to lake of assimilation in the upstream network.

V2, same

Reservoir Outflow, DA ON
V3, same DA ON

V2, same

Falls Lake Inflow, DA ON
V3 at 8782449, Falls Lake Inflow DA ON

V2, same

DA Smoothing
While not the primary purpose of this development, with the five-minute output files to compare to with this test, it is easy to see the slight difference in the treatment of interpolation with the data assimilation scheme. T-route takes advantage of a linear interpolation to smooth the DA values between 15 minute observations vs. a more step-function approach in the WRF-Hydro DA algorithm. The following output shows this fairly clearly -- the difference values oscillate back and forth as the linear interpolation moves across the step-function value:

In the testing, V2 and V3 appear to produce different results at 166737579 with DA OFF.

In the testing, V2 and V3 appear to produce different results at 166737579 with DA OFF.

A closer review of testing confirmed a difference resulting from the interaction of interpolation of data assimilation values through a large gap across the break between looping segments. The linear interpolation respects the long-term trend across the gap, even though it will only fill the values as much as permitted by the max_fill parameter. With the looping tests above, only exactly the number of files needed for the given loop segment were passed to the loop.
With commit https://github.com/NOAA-OWP/t-route/pull/429/commits/9a0b7a885b35b36db1ec1e31c43cf692f6f338f7, the input timeslice lists may now be padded on both sides an arbitrary amount to ensure that interpolation for the looped chunks is the same as it would be if the simulation were performed without breaks.
Please compare these results to the DA_ON results from Below Falls Lake, segment 166737579, in the PR description above. There is still a difference in the values less than the floating point precision of the variables, but otherwise the values are the same.

@awlostowski-noaa


Make grpc compile on win64bit

"long" type is 32bit on Win x64, which brings some trouble.
Please note the new public function gpr_int64toa.

Travis failure unrelated: #4295

Interop test failure seems unrelated, @murgatroid99  is working on fixing the problem.

friendly ping.

h2_proxy_simple_request_test
h2_proxy_simple_request_nosec_test  seem to be consistently failing. Could that be because of this PR?

Looks like h2_proxy_simple_request_test and h2_proxy_simple_request_nosec_test is also failing on master here: https://grpc-testing.appspot.com/job/gRPC_master/13556/#showFailuresLink

I think I've addressed all the comments. PTAL.

Turns out there were no tests for gpr_ltoa, so I added tests for both gpr_ltoa and gpr_int64toa.
In the process of doing that, it turned out that handling for LONG_MIN and INT64_MIN was broken, so I fixed that. PTAL.

Now Linux fails:
src/core/support/string.c: In function 'gpr_ltoa':
src/core/support/string.c:167:12: error: conversion to 'long unsigned int' from 'long int' may change the sign of the result [-Werror=sign-conversion]
     uval = -value;
            ^
src/core/support/string.c:169:12: error: conversion to 'long unsigned int' from 'long int' may change the sign of the result [-Werror=sign-conversion]
     uval = value;
            ^
src/core/support/string.c: In function 'gpr_int64toa':
src/core/support/string.c:193:12: error: conversion to 'gpr_uint64' from 'gpr_int64' may change the sign of the result [-Werror=sign-conversion]
     uval = -value;
            ^
src/core/support/string.c:195:12: error: conversion to 'gpr_uint64' from 'gpr_int64' may change the sign of the result [-Werror=sign-conversion]
     uval = value;

Alright, I did some fix but it looks like this still relies implementation-specific. I will try rewrite the implementation to be 100% compliant with the C spec.

I got rid of the negation from ltoa and int64toa code, hoping that now everything should be compliant with the spec.

Weird error building openSSL, I don't see how this could be related.

Jenkins is sufficiently green.


De-couple oracle_home management

Right now, a ORACLE_HOME is based on the configuration in the oracle_databases structure.
A new structure (oracle_homes) will be created where the oracle_home and its possible patches will be dealt with.
Another list will contain the homes (e.g oracle_homes_installed) a specific configuration should have installed.
oracle_databases will only contain a reference to the oracle_homes structure.

Fixed with https://github.com/oravirt/ansible-oracle/commit/af9e68af4feabdd8c0de2a4ea6130c1675c137b1


How to compare 2 fields, instead of a field and a specific value?

My use case requires 2 fields to be compared, instead of 1 field being compared to a single value.
E.g.
var1 > var2 AND var3 <> var4
Is there a way to do this currently?

There is a way to do this, with a custom ValueEditor and (if you're using formatQuery to generate your SQL) a custom valueProcessor.  See this codesandbox.  After you add a rule, check the box to the right of the value editor to convert it to using fields instead of values.

got it thanks!

@enzoampil in case you're still interested, please see the update to my comment above.


fix sky and warp textures rendering incorrectly on macOS

Metal does not require floats in an array to be 16 byte aligned, so using this strategy on macOS to store lightmap_anim_frames in FrameUniforms means that the uniforms store the wrong data on the GPU, including for time, which affects sky and warp textures. Packing the floats into vec4s and indexing lightmap_anim_frames like a 2D array renders things correctly on macOS, and also saves some space.
Additionally, UniformArrayFloat may be useless now, since to my knowledge, Metal can't have floats aligned to 16 bytes in an array, so using this will always cause discrepancies cross platform. I haven't removed it entirely though, since it's possible there's some way to do this I don't know about, or maybe some other future use case for the struct. It's definitely best used with caution.

Thank you so much! Great find and great fix. Very happy to have this be my first accepted pull request!
After merging this I realised that the scroll speed was set incorrectly, so I fixed that too and added a cvar to control the sky scroll speed.


Feature Request: Allow changing StreamDeck button image based on DCS-BIOS data

A simple example of this would be to switch between two images to simulate the flashing of the master caution button.
One image would be the unlit button and the second would be the lit one. Tying this to the output would be something along the lines of a comparison of if MASTER_CAUTION > 0 then display lit button image
Where this could make DCSFlightPanels way more powerful than any other current solution would be if it supported greater than two image states.
For example take the 5 position TACAN mode dial in the A-10. If 5 image states were available that entire switch could be controlled and simulated on one button by having an image for each state of the switch and using the increment or sequence output data methods.

maki7266 and deep slumber have both requested this feature in chat as well

Check latest alpha.

This can be closed yeah?

yeah i think so, this is implemented and the question of the bit masked output is something distinct in itself


Code not running

Hey, I'm sure this project is super dead to you, but I'd love to get it working for something I'm working on. I haven't been able to get anything more than a black screen and a bunch of errors after dependency update. I'm able to use the model in my three js project, but wanted to see how all your additions (and animations) work out together.
Thanks!

I attempted to do the same about a year ago and ran into the same issues. I opted to start fresh instead of untangling the issue that I think stems from breaking changes in R3F over the years as this project sat dormant.
I'm currently refactoring this entire project based on DNGR (Double Negative Gravitational Renderer), the renderer created for the movie Interstellar and its supermassive black hole. To visualize a blackhole with DNGR required several hundred servers at a time in a render farm 10+ years ago, so I've been attempting to dumb it down for real-time performance while retaining the core principles. More info in the paper here: https://arxiv.org/abs/1502.03808


cv2 error

this is the error i am getting while running the train.py script can you help me where exactly i am going wrong
Traceback (most recent call last):
File "E:\ML\ml_projects\project_folder\icpr\img_to_word\train.py", line 129, in
model.fit(
File "E:\ML\envs\python3\lib\site-packages\keras\utils\traceback_utils.py", line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File "E:\ML\envs\python3\lib\site-packages\mltu\dataProvider.py", line 142, in getitem
data, annotation = preprocessor(data, annotation)
File "E:\ML\envs\python3\lib\site-packages\mltu\preprocessors.py", line 9, in call
return cv2.imread(image_path, self._method), label
cv2.error: OpenCV(4.10.0) :-1: error: (-5:Bad argument) in function 'imread'

Overload resolution failed:

Argument 'flags' is required to be an integer
dst is not a numpy array, neither a scalar
Expected Ptrcv::UMat for argument 'dst'

The error speaks for it self:
Specific Error Message

OpenCV Error: cv2.error: OpenCV(4.10.0) error: (-5:Bad argument) in function 'imread'

Bad Argument: This indicates that the argument passed to cv2.imread is not valid.

Common Causes

Invalid File Path: The image_path passed to cv2.imread might be incorrect or the file does not exist. cv2.imread returns None if the image path is invalid.
Incorrect Data Type: The image_path should be a string representing the path to the image file.
File Access Issues: There might be permission issues preventing the file from being read.
Unsupported File Format: The image format might not be supported by OpenCV.

So, check if what file you try to read with openCV and try to investigate this specific file


Added documentation page for JWK and rotating keys

Added documentation for the JWK environment variables
Added an example about rolling update of the keys

Note: At first I thought about adding it to the JWK page at the Getting Started section but I thought it would be so much details for the "Getting Started" section so I did a separate page and added the link to there. If you would prefer otherwise I can merge the 2 pages.

Sorry for a lot of typos. I will look for a VSCode extension that checks for typos the next time.

Don't worry, the content looks good to me! :)


⚠ Bofrost.de has degraded performance

In 6b18f72, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1572 ms

Resolved: Bofrost.de performance has improved in b1e87e3 after 17 minutes.


Level 3 order of event execution

Summary
For test case 3.1, the order of customer 1 done serving by server 1 and server 1 serving customer 2 is swapped and I'm not sure why this is happening. I was thinking that there could be a problem with EventComparator class or poll() method in PQ class but I'm not very sure.
Description
In EventComparator I just used Double.compare() to compare the event times of both events o1 and o2. Not sure if there's anything wrong with this because it passed all the previous test cases in previous levels.
Screenshots (if any):
new queue in the screenshot below is the updated queue after every poll(), executing the event with the highest priority and adding the next event into the queue if it is not Optional.empty().
first event is the output from queue.poll().first()
I'm not sure why in line 10 the order in the queue is correct (server 1 done serving customer 1 first then server 1 serves customer 2) but after poll() then the order is swapped in line 12

perhaps you could try comparing the customer ID if the event times are the same in the event comparator so the customer ID that is smaller will appear first.

hi rayna, you can refer to #366, basically the customer who arrived first, with a smaller customerID, should be served first. you can try modifying your eventComparator.

Hi! you can try comparing the customer IDs (smaller id number will come first), or you can give different priorities to different types of events ( what i did was i gave a number to each type of event as you can see that eg. Done event is printed first before Arrive > Serve > Wait > leave )

you may want to update your comparator class to take into consideration customer id as well if the event time is the same! I've met with the same error as well

Hiii when the event time is the same, the customer with the smaller ID number will be considered first! This can be done by adding one more if-else statement in event comparator class to check the event time and ID number.

Hi, I had the same problem, realized that I didn't define my EventComparator fully, the smaller ID should come first if the eventTime is the same.

Hello, had the same problem too but realized a quick fix was to just add in a comparison for Customer ID apart from the event time


Saved Paths UI

User Story:
As a user, I want the application to save previously travelled paths so that the car can drive on the same path again instead of giving the same information twice.
Acceptance Criteria:
Given that the user might want to save the current drawn or manually driven path for future use,
When the user draws or manually drives a path,
Then the application will allow the user to save the path to the saved path list.

This is not a vertical User Story as it deals solely with the UI and thus, is marked as invalid and is being closed.


Create Administrator main page - dinamic added and edit

post will be add dinamicly when there new posts and data to see for the user
the user will be able to delete the posts from the page


remove all similar package

To keep the test device clean, we should manage installed package.
Clean all packages like the following.
com.example
com.example.debug
com.example.beta

BTW, this part depends on test environment.
So, i close this issue because I don't care about this point in this application


Change a request from Review to Consult

As a (type of user)
I want (goal)
so that (reason/ why/ value)

Assumptions & Scope
What are the assumptions for this story?
What is in and not in scope?

Dependencies? What is the impact of this dependency? (If so, link dependency in the ticket, make it visible in a team´s backlog)
Validation Rules? (If yes, list here)
Design
@xxx - please link the Design here
Definition of Ready

[ ] Is there a well articulated User Story?
[ ] Is there Acceptance Criteria that covers all scenarios (happy/sad paths)?
[ ] If there is a user interface, is there a design?
[ ] Does the user story need user research/validation?
[ ] Does this User Story needs stakeholder approval?
[ ] Design / Solution accepted by Product Owner
[ ] Is this user story small enough to be completed in a Sprint? Should it be split?
[ ] Are the dependencies known/ understood? (technical, business, regulatory/policy)
[ ] Has the story been estimated?

Definition of Done

[ ] Passes developer unit tests
[ ] Passes peer code review
[ ] If there's a user interface, passes UX assurance
[ ] Passes QA of Acceptance Criteria with verification in Dev and Test
[ ] Confirm Test cases built and succeeding
[ ] No regression test failures
[ ] Test coverage acceptable by Product Owner
[ ] Ticket ready to be merged to master or story branch
[ ] Developer to list Config changes/ Update documents and designs
[ ] Can be demoed in Sprint Review
[ ] Tagged as part of a Release
[ ] Feature flagged if required
[ ] Change Management activities done?

@arielleandrews, question for you - who would change request from Review to Consult? Assume it is the IAO analyst.

@lmullane correct, IAO analysts make the decision to send consultations and take all associated actions

Thanks, @arielleandrews. Is there reasons that need to be selected for a Consult?

Nope, no reasons. Consultations are sent because another public body has authored and/or has interest in the content of the records and we want to get their views on disclosure

Super. Please review above. Following the common pattern we have established.

Thanks @lmullane, a couple questions:

Scenario 3: this reads as incomplete to me. Should this be for the IAO Analyst who is changing the state from Review to Consult? And is able to successfully Save the state change?
Scenario 4: for Ministry Coordinators, would it make sense just to have the Consult state greyed out for them, so they can't select it in the first place? Or should this be written for the IAO Analyst persona?

@arielleandrews, 3 and 4 should be for the IAO analyst. Revised accordingly.

Thanks @lmullane. Scenarios look good to me!

Thanks, @arielleandrews


Sample App: Auto Navigator - clicking on "See All Cars Nearby" display an empty list of cars

Launch app
scroll down to the bottom and tap on "See All Cars Nearby"
this shows up, it's missing the list of cars. Same issue on studio and on Ensemble preview on ios

fixed

its fixed now

@sharjeelyunus please close the issue if fixed


Add conditions to grants

One of the projects that uses this library as a foundation has implemented conditions for the grants. Look here for more information. It supports AND, OR, NOT, EQUALS, NOT_EQUALS, STARTS_WITH, LIST_CONTAINS conditions, which are really powerful to grant permission when a certain state has a certain value. This way permissions can be even more finely grained. Especially for more complex systems this would be a nice addition I think.

Thanks. A similar feature with different API is on the way for v3.

+1

@onury First of all thank you for the amazing library. When do you plan release v3? I do really miss the context feature.


[vcpkg_clean_executables_in_bin] Clean bundles

What does your PR fix?
Ensure that macOS bundles are deleted from /bin, for consistency with handling of plain executables, and for consistent behavior of vcpkg_copy_tools( ... AUTO_CLEAN).

Which triplets are supported/not supported? Have you updated the CI baseline?
unchanged, but affect macOS.

Discovered/tested with fluid app in port fltk.


Inner-join v. cross-join with R-TREE?

I would like to write this query:
Query 1:
            SELECT         p.*
            FROM           portfolio p
            INNER JOIN     buildings b
            ON             ST_Within(p.geometry, b.geometry)

... but the performance is very slow. I do not think it is making use of the R-Tree index I created?
But this query, which is very similar, is very fast. I think it IS taking the R-Tree. Is this a bug? Am I misunderstanding things?
Query 2:
            SELECT       p.*
            FROM         portfolio p, buildings b
            WHERE        ST_Within(p.geometry, b.geometry)

Thanks

Hello!
As written in the docs

The R-tree index will only be used to perform "index scans" when the table is filtered (using a WHERE clause) with one of the following spatial predicate functions (as they all imply intersection): ST_Equals, ST_Intersects, ST_Touches, ST_Crosses, ST_Within, ST_Contains, ST_Overlaps, ST_Covers, ST_CoveredBy, ST_ContainsProperly.

That is, the R-Tree index is not used to accelerate JOINs.

Thanks @Maxxen  . I did read the docs and see that but thought that it still might apply to a join, given in this case it's kind of the same thing. I will re-write my code.

Im not sure why the cross join + is faster, I don't think it will use the R-Tree either given that neither arguments to the where clause are constant. Are you've written the queries properly/they return the equivalent results?

I'll generate some test data and open a few ticket when it's ready. Thanks


Panic in Rust Code When Running tokenise_bio.py on GRCh38.p13 FASTA, Successful on Smaller FASTA File

Issue Description:
While attempting to reproduce the findings in your preprint, I'm encountering an error running the tokenise_bio.py script on large (3GB or larger) FASTA files, including the GCF_000001405.39_GRCh38.p13. It fails with a panic in the Rust code. The code runs successfully on (much) smaller FASTA files (200KB). I have tried using the original .fna file extension as well as .fasta but have not had success in getting past this particular error.
The error log indicates a panic due to a Rust-related error, specifically in the tokenizers-lib library. The panic occurred at the line tokenizers-lib/src/models/unigram rainer.rs:212:53, with the message "called Result::unwrap() on an Err value: Internal." The error thread panicked at called Result::unwrap() on an Err value: Internal, tokenizers-lib/src/models/unigram/trainer.rs:212:53 is related to the Hugging Face Tokenizers library.
To Reproduce
OS: EC2 Instance
Conda venv with python 3.9
Command Used for Error:
RUST_BACKTRACE=full tokenise_bio -i /data/ncbi_dataset/GCF_000001405.39_GRCh38.p13_genomic.fasta -t '/data/generated/ncbi_tokenisers/tokeniser_39_GRCh38.json

Output & error message: https://gist.github.com/stepwise-ai-dev/f23a79faaedd006bf51d486259440dd5
Steps to Reproduce:
Run the tokenise_bio.py script with GCF_000001405.39_GRCh38.p13_genomic.fna as input
Expected output
I expected similar output to previous successfully execution of tokenise_bio.py on a smaller (200KB) FASTA file.
Successful output :https://gist.github.com/stepwise-ai-dev/bbd782f3d09afaca6219b2c3a176bb42
Questions:

Is this a known issue with the code or a known issue specific to certain system specifications?
Are there any additional specific requirements or configurations needed for processing larger genome file sizes?
Are there any additional data pre-processing steps required between downloading the NCBI data and using it as input for tokenise_bio.py?

Any guidance on resolving this issue would be greatly appreciated.
Thank you so much!

Hmm, I confirm that I replicated your issue
Sat Aug 12 21:33:26 AEST 2023
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Internal', tokenizers-lib/src/models/unigram/trainer.rs:212:53
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
COMMAND LINE ARGUMENTS FOR REPRODUCIBILITY:

	 /██████/mambaforge-pypy3/envs/foo/bin/tokenise_bio -i GCF_000001405.39_GRCh38.p13_genomic.fna -t foo.json

Traceback (most recent call last):
  File "/██████/mambaforge-pypy3/envs/foo/bin/tokenise_bio", line 11, in <module>
    sys.exit(main())
  File "/██████/mambaforge-pypy3/envs/foo/lib/python3.9/site-packages/genomenlp/tokenise_bio.py", line 68, in main
    tokeniser.train_from_iterator(
  File "/██████/mambaforge-pypy3/envs/foo/lib/python3.9/site-packages/tokenizers/implementations/sentencepiece_unigram.py", line 142, in train_from_iterator
    self._tokenizer.train_from_iterator(
pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: Internal
Sat Aug 12 21:36:13 AEST 2023

To test if this is an issue with the unigram_trainer, I attempted to replicate this with the original rust implementation of the underlying library, and the error message implies that there is a sentence length limit (where a sentence corresponds to a single fasta read).
conda create -n something sentencepiece
conda activate something

# where the fasta file is (note that this generates files in its dir, so probably make a new dir with this inside only)
awk '/^>/ {OUT=substr($0,2) ".fa"}; OUT {print >OUT}' GCF_000001405.39_GRCh38.p13_genomic.fna
for f in *\ *; do mv "$f" "${f// /_}"; done
for f in *fa; do mv "$f" "${f//,/}"; done
for i in *fa; do sed -i -e 's|>.*||' -e '/^$/d' $i; done
for i in *fa; do cat $i | tr -d '\n' | tr [:lower:] [:upper:] > ${i/fa/fna}; done

...
trainer_interface.cc(408) LOG(INFO) Loaded all 75 sentences
trainer_interface.cc(415) LOG(INFO) Skipped 564 too long sentences.
...
# where each sentence is one fasta read

To test if this is due to sentence length, im segmenting chromosome 1 by itself with 128GB memory. Unknown if this will error, has not completed successfully yet. I'll continue to investigate this sentence/read length limitation further. It looks like others ran into similar memory issues with total corpus size, but considering that ours isn't that big in comparison, I'm not sure if this is also the case.
The only quick workaround I can think of for now is dividing your reads into blocks of eg 2048bp length, which other users also suggested. But I'm not sure how much impact this will have on preserving long-range contextual information. In any case pushed this as v2.7.1 available on conda now, or you can pull the latest tokenise_bio.py and inspect the new -b and -c options.

Thank you so much for that helpful information and for uploading a workaround so promptly! I just tried running v2.7.1 and it gave me a new error in the _gzip_iterator function. That function seems to have an issue In version 2.7.1 where it attempts to manipulate the read object itself, leading to an error. I've tested an approach that extracts the sequence from the read object and then applies the desired transformations (such as case changes or breaking into chunks) directly to the sequence string. This might align better with standard practices for handling strings in Python and leverage screed's native support, resolving the error and ensuring the function's intended behavior.
def _gzip_iterator(infile_paths, break_size: int=None, case: str=None):
    for path in infile_paths:
        with screed.open(path) as infile:
            for read in infile:
                sequence = read.sequence
                if case == "upper":
                    sequence = sequence.upper()
                elif case == "lower":
                    sequence = sequence.lower() # test

                if break_size is None:
                    yield sequence
                else:
                    for i in range(0, len(sequence), break_size):
                        yield sequence[i:i+break_size]

If this looks like a promising fix for the read issue in the _gzip_iterator function, I'd be happy to create a pull request!
I've tested running tokenise_bio.py using this fix and it runs without the read errors, but does still give me the same RUST-related error as before regardless of the given break size (2048, 1024, 512, 256) or reduced vocab size (32k, 16k, 8k, 4k, 2k, 1k). It will still run on much smaller FASTAs (200kb) without producing the RUST-related error.
And one last thing to note is that using "lower" seems to lead to a unique error that does not appear in the other two cases (upper or none) for the case changing transformations, though I haven't yet figured out why.

I just tried running v2.7.1 and it gave me a new error in the _gzip_iterator function. That function seems to have an issue In version 2.7.1 where it attempts to manipulate the read object itself, leading to an error. I've tested an approach that extracts the sequence from the read object and then applies the desired transformations (such as case changes or breaking into chunks) directly to the sequence string. This might align better with standard practices for handling strings in Python and leverage screed's native support, resolving the error and ensuring the function's intended behavior.

Ah yes, I thought i put the correct functions for the read object and str but it looks like I didn't test it thoroughly enough.

If this looks like a promising fix for the read issue in the _gzip_iterator function, I'd be happy to create a pull request!

It does look promising, yes please feel free to create a pull request (i dont have a dev branch so you can create directly on master, then ill retest it again and merge it in).

I've tested running tokenise_bio.py using this fix and it runs without the read errors, but does still give me the same RUST-related error as before regardless of the given break size (2048, 1024, 512, 256) or reduced vocab size (32k, 16k, 8k, 4k, 2k, 1k). It will still run on much smaller FASTAs (200kb) without producing the RUST-related error.

Hmm yes, the root cause of the error is probably the data size (from what we both found so far, and from what others have implied). The short-term workaround would be subdividing the data, but theres probably a better way around it which ill need to brainstorm.

And one last thing to note is that using "lower" seems to lead to a unique error that does not appear in the other two cases (upper or none) for the case changing transformations, though I haven't yet figured out why.

In that case for the short term, we'll make a note that lower case doesnt work until we find and retroactively fix this later. (I can add this in to doc and args helptext after i merge in your pull request).

From previous issue:
To test if this is due to sentence length, im segmenting chromosome 1 by itself with 128GB memory.

Btw, this exceeded 128GB memory on chr1 alone and crashed unfortunately. I'll need to systematically test out a range of sequence lengths to get a better feel for the algorithm's limitations, since interestingly they dont mention that anywhere in their paper or github (or if they did I may have missed it).

_gzip_iterator issue pull request #7  tested with all combinations and OK (or errors correctly) (thanks!).
Continuing investigation into solutions for the long read issue. (Another piece of information supports the "cannot handle long reads").
As the next step, I may truncate the seqs into multiple chunks and pool the token weights at the end and see if that gives a realistic result. Might be an effective workaround.

seq length

Now testing a pooling operation where i break long seqs into shorter ones and compare token intersection + weights variance across the different blocks

seq quantity

Then will repeat for a case where i split multiple seqs into subsets

Added compare_empirical_tokens.py to compare multiple tokenised dataset json files to v2.8.0.
You can use this to examine your own datasets if you are interested - eg compare resulting token set and weights on varying slice lengths or sequence counts.
Results for a sweep on segmented sequence lengths from 2**9..13 compared to full chromosome will be out shortly on Ecoli K12.

Hi @schultzjack
Tested the read splitting in Haemophilus influenzae. I split the 1.8 Mbp whole genome into contigs ranging from 2**9..20 and ran empirical tokenisation.

unsurprisingly, the variance from ground truth (whole genome) was higher the smaller the contig length
however, the variance is actually not that big (for the most highly weighted tokens at least)
similarly, empirical token identity is more similar to ground truth the longer the contigs are (90-95% set identity)
in the process, discovered that tokenisation can handle seqs up to 2Mbp (4Mbp segfaults), so we can take a sentence size of 2Mbp as an upper limit. For normal use I suggest 1Mbp length splits, since the variance from ground truth was already minimal by this length (for this dataset anyway).

Compiling detailed information on the above now and will add a section to documentation under tokenisation when complete.

Documentation now demonstrates a case of handling long sequences

The human genome hg38.p14 was tokenised and hosted in this repository for user convenience.
Corresponding files and README are available here.
Support for other model organisms are under consideration.


replace link in state.yml with correct link

This fixes #1399
I have replaced the link in state.yml. The new link points to the base branch now.

Thanks, there is a typo in the commit message. It says staTe.yml but should say staLe.yml (using upper-case to highlight the problem).

Thanks, there is a typo in the commit message. It says staTe.yml but should say staLe.yml (using upper-case to highlight the problem).

Oh thank you! I didn't notice that. Please, could you adjust it and rebase? @shreyagupta30

Hi, @tyll and @ffmancera I have made the required changes in the commit message. I still figuring out rebasing.

Hi, @ffmancera I have tried rebasing the PR with upstream. Could you please review it?


Does this package log relationship changes?

Hi,
I have recently been working on a more basic version of this package that is designed to log administrator activity. I have been having trouble working out an automatic way of logging changes to relationships (for example deleting a many-to-many relationship),
Have you found a way round this or are you not currently logging this information?
Thanks

It won't if you're logging activity through the model events, since Eloquent doesn't fire events on parent classes when one of it's relationships changes. You can however, build the activity record yourself (including coupling it to the relevant model).
// First do stuff with relations on $myModel...

activity()
    ->performedOn($myModel)
    ->causedBy(auth()->user())
    ->log(':causer.name changed some relationships on :subject.title');

Ok thanks for the info!


New binding implementation scope issue

This is a bit of a bug in the way the new binding works. I noticed you were replacing __bind with _this. However, the following now breaks:
class Store
  @fetchRemote: =>
    do =>
     # context is window!
     console.log @

Store.fetchRemote()

The console logs the window - the context should be the class.

Note that the do => isn't relevant.

Sorry, my diagnosis above was only partly correct—while the .call(this) is anomalous, the problem is really that if you do
class Store
  @fetchRemote: =>
     console.log @

then @ compiles to Store (though there's also a superfluous _this declaration which points to whatever context Store was defined in); whereas if you do
class Store
  @fetchRemote: =>
    do =>
      console.log @

then that _this is used.

It didn't come from an issue -- the idea was to optimize local bound functions where possible. __bind is still available, however, and we can continue to use it for functions of this type.

Well, we shouldn't have to use __bind. We just have to translate this to the class name in nested bound functions on classes. Or, alternatively, have _this in the class body point to the class and use that. It seems like currently, whenever _this is inserted into the class body, it's either unused or (in @maccman's scenario) actively harmful...
Note that the same bug occurs for functions that are scoped in the class body:
class Foo
  func = => @  # points to Foo

class Bar
  func = =>
    do => @  # points to the context in which the class is defined

This seems to cause an addition interesting issue. When I throw a debugger in a fat-arrow function, the function at execution has the right context, but at debug, if I type "this" in the console, the context is again wrong.

Yes, I think I really prefer the proxy approach. What's the main reason behind this new approach? Is there a significant speed difference?

Damn -- that's a nice inconsistency...

Yes, there's a significant speed difference, but perhaps it isn't worth it.

@jashkenas: It's worth it, we just need to be smarter about it.

When I throw a debugger in a fat-arrow function, the function at execution has the right context, but at debug, if I type "this" in the console, the context is again wrong.

Is this really a problem?  So shouldn't you be able to see that the function is using _this instead of this?

Crazy idea I know, but since there's now no overhead, have you considered making every function a 'fat' one - i.e. preserve context everywhere?

@maccman: do you seriously never .call or .apply a function? Never attach a function to a different object? We would never want to do that.

@TrevorBurnham yes, I believe it's definitely a problem. It's an inconsistency in the expected behavior. "this" shouldn't change just because my script execution has paused.

@jscheel: this, as in the context of the function, hasn't changed.  And sometimes, it just isn't. That's not a problem. Do you see what we're saying? this compiles to a reference to a stored reference (_this) to the desired object.

So, occasionally "@" will be compiled down to "_this"?

@jscheel: Sometimes. Try it yourself:
$ coffee -bep 'aBoundFunction = => @woah'
var aBoundFunction,
  _this = this;

aBoundFunction = function() {
  return _this.woah;
};

And sometimes it'll compile to the name of a class, such as the desired behaviour in this issue.

eek, wow, didn't realize this (no pun intended). Has it always been this way? Have I just been really lucky with calling "this" in debugging sessions?

@jscheel: It changed September 25th, and that change wasn't part of a tagged release until 1.1.3, so it's pretty recent. We're still working out the bugs. See #1851, #1844, #1849, this issue.

Just ran into this issue myself, using an example similar to the one @maccman used to start this thread. Can we revert to the previous behavior (i.e. use __bind) until a proper solution is found?

@mjijackson: that's the plan. We just need someone to put in the time required to change it. Pull requests are always welcome.

Any updates on this? Anybody working towards a fix? I'd just want to check before I start working on it over Christmas.

I'd like to tackle it and cut a release when I have a free minute. Perhaps Thursday, or this weekend, if we're lucky. If you get a chance, have at it.

I think the above commit should fix this nicely. Now bound static methods will point their this values to the name of the class object, not _this. No intermediate function required. So, a fancy test case, like this:
class Store

  eq this, Store

  @bound: =>
    do =>
      eq this, Store

  @unbound: ->
    eq this, Store

  instance: =>
    ok this instanceof Store

... will produce this JavaScript output:
Store = (function() {

  function Store() {
    this.instance = __bind(this.instance, this);
  }

  eq(Store, Store);

  Store.bound = function() {
    return (function() {
      return eq(Store, Store);
    })();
  };

  Store.unbound = function() {
    return eq(this, Store);
  };

  Store.prototype.instance = function() {
    return ok(this instanceof Store);
  };

  return Store;

}).call(this);

Nice work! Thanks!


Filter out images to a given filename

Any chance we can have a filtering parameter on file name? So for instance it'll only show up in the image picker whatever we tell it to disclose e.g. (.png) or (amazingPics.)

It uses Apples PhotoKit for fetching, sorting, filtering, etc.
And I don't believe they support filtering on filename, see https://developer.apple.com/documentation/photokit/phfetchoptions


Error when sink is connected to ElasticSearch 8.0.0-SNAPSHOT

A few questions before you begin:

Is this an issue related to the Serilog core project or one of the [sinks]
Sink Elasticsearch

Does this issue relate to a new feature or an existing bug?

[X] Bug
[ ] New Feature

What version of Serilog.Sinks.Elasticsearch is affected?  Please list the related NuGet package.
8.4.1
What is the target framework and operating system? See target frameworks & net standard matrix.

[X] netCore 5.0

Please describe the current behavior?
When connecting to Elastic 7.9.1 logging works fine. Connecting to 8.0 throws this error:
2020-12-11T12:56:03.7531402Z Caught exception while preforming bulk operation to Elasticsearch: Elasticsearch.Net.ElasticsearchClientException: Request failed to execute. Call: Status code 400 from: POST /_bulk. ServerError: Type: illegal_argument_exception Reason: "Action/metadata line [1] contains an unknown parameter [_type]"
11.12.2020, 13:56:03
orderservice-api-745b5975df-2jcbm
b986a1e1f9483f1c4ce20673c607f71eb3608b85ee9b1882f6a0eab169f3a594
at Elasticsearch.Net.Transport`1.HandleElasticsearchClientException(RequestData data, Exception clientException, IElasticsearchResponse response)
020, 13:56:03
orderservice-api-745b5975df-2jcbm
b986a1e1f9483f1c4ce20673c607f71eb3608b85ee9b1882f6a0eab169f3a594
at Elasticsearch.Net.Transport`1.FinalizeResponse[TResponse](RequestData requestData, IRequestPipeline pipeline, List`1 seenExceptions, TResponse response)
11.12.2020, 13:56:03
orderservice-api-745b5975df-2jcbm
b986a1e1f9483f1c4ce20673c607f71eb3608b85ee9b1882f6a0eab169f3a594
at Elasticsearch.Net.Transport`1.RequestAsync[TResponse](HttpMethod method, String path, CancellationToken cancellationToken, PostData data, IRequestParameters requestParameters)
11.12.2020, 13:56:03
orderservice-api-745b5975df-2jcbm
b986a1e1f9483f1c4ce20673c607f71eb3608b85ee9b1882f6a0eab169f3a594
at Serilog.Sinks.Elasticsearch.ElasticsearchSink.EmitBatchAsync(IEnumerable`1 events)
11.12.2020, 13:56:03
orderservice-api-745b5975df-2jcbm
b986a1e1f9483f1c4ce20673c607f71eb3608b85ee9b1882f6a0eab169f3a594
2020-12-11T12:56:03.7533834Z Caught exception while emitting to callback System.Action`1[Serilog.Events.LogEvent]: System.NullReferenceException: Object reference not set to an instance of an object.
11.12.2020, 13:56:03
orderservice-api-745b5975df-2jcbm
b986a1e1f9483f1c4ce20673c607f71eb3608b85ee9b1882f6a0eab169f3a594
at Newton.OrderService.Bootstrap.Program.<>c.<Main>b__0_1(LogEvent e) in /home/vsts/work/1/s/Bootstrap/Program.cs:line 34
11.12.2020, 13:56:03
orderservice-api-745b5975df-2jcbm
b986a1e1f9483f1c4ce20673c607f71eb3608b85ee9b1882f6a0eab169f3a594
at Serilog.Sinks.Elasticsearch.ElasticsearchSink.HandleException(Exception ex, IEnumerable`1 events)
11.12.2020, 13:56:11
orderservice-api-745b5975df-2jcbm
b986a1e1f9483f1c4ce20673c607f71eb3608b85ee9b1882f6a0eab169f3a594
2020-12-11T12:56:11.7557969Z Caught exception while preforming bulk operation to Elasticsearch: Elasticsearch.Net.ElasticsearchClientException: Request failed to execute. Call: Status code 400 from: POST /_bulk. ServerError: Type: illegal_argument_exception Reason: "Action/metadata line [1] contains an unknown parameter [_type]"
11.12.2020, 13:56:11
orderservice-api-745b5975df-2jcbm
b986a1e1f9483f1c4ce20673c607f71eb3608b85ee9b1882f6a0eab169f3a594
at Elasticsearch.Net.Transport`1.HandleElasticsearchClientException(RequestData data, Exception clientException, IElasticsearchResponse response)
11.12.2020, 13:56:11
orderservice-api-745b5975df-2jcbm
b986a1e1f9483f1c4ce20673c607f71eb3608b85ee9b1882f6a0eab169f3a594
at Elasticsearch.Net.Transport`1.FinalizeResponse[TResponse](RequestData requestData, IRequestPipeline pipeline, List`1 seenExceptions, TResponse response)
11.12.2020, 13:56:11
orderservice-api-745b5975df-2jcbm
b986a1e1f9483f1c4ce20673c607f71eb3608b85ee9b1882f6a0eab169f3a594
at Elasticsearch.Net.Transport`1.RequestAsync[TResponse](HttpMethod method, String path, CancellationToken cancellationToken, PostData data, IRequestParameters requestParameters)
11.12.2020, 13:56:11
orderservice-api-745b5975df-2jcbm
b986a1e1f9483f1c4ce20673c607f71eb3608b85ee9b1882f6a0eab169f3a594
at Serilog.Sinks.Elasticsearch.ElasticsearchSink.EmitBatchAsync(IEnumerable`1 events)

Please describe the expected behavior?
No errors posting to the bulk endpoint
If the current behavior is a bug, please provide the steps to reproduce the issue and if possible a minimal demo of the problem
Create a logger and hook it up against Elastic 8.0.0

            Logger = new LoggerConfiguration()
                .Enrich.FromLogContext()
                .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri("http://elasticsearch-master.elk.svc.cluster.local:9200/"))
                {
                    FailureCallback = e => Console.WriteLine("Unable to submit event " + e.MessageTemplate + " Error: " + e.Exception.Message),
                    EmitEventFailure = EmitEventFailureHandling.WriteToSelfLog |
                                       EmitEventFailureHandling.RaiseCallback,
                    AutoRegisterTemplate = true,
                    IndexFormat = $"{Assembly.GetExecutingAssembly().GetName().Name.ToLower()}-{DateTime.UtcNow:yyyy-MM}",
                    MinimumLogEventLevel = Serilog.Events.LogEventLevel.Verbose
                })
                .Enrich.WithProperty("Environment", environment)
                .CreateLogger();

            Serilog.Debugging.SelfLog.Enable(message => Console.WriteLine(message));

There's not clear way to configure this for Elastic v8 at the moment.
I'm not sure that this will work for you but it's working against Elastic v7.10 that probably will be compatible with v8.
If we're looking at the exception it indicates that we're sending _type that's removed in v8 and deprecated in v7.
"Action/metadata line [1] contains an unknown parameter [_type]"

In 8.4.0 we added a couple of features that eventually will work for you.
TypeName = null,
IndexFormat = "logs-my-stream",
BatchAction = ElasticOpType.Create,

If TypeName = null is set null we'll not pass _type when sending data to Elastic.
Note: I think this is only working with configuration as code.
BatchAction = ElasticOpType.Create is needed to be able to write data to data-streams which is a great improvement over write aliases.
We should probably add the possibility to set the targeted Elastic version to v8 that'll suppress the _type automatically.

Elastic v8 is out now, bumping this issue!

TypeName = null,
BatchAction = ElasticOpType.Create,
IndexFormat = "api-local-{ 0:yyyy.MM }"

I tried this config in version 8.4.1 but still getting the error.

Is there active development on this or is the project stalled ? Considering this issue was opened in 2020.

On NuGet you have published a version 8.5.0-alpha0003, but I can't find any corresponding release tag in the GitHub repo.
Anyone knows if there are there any significant changes in the 8.5.0 version?
I'll make a try to see if I can make a pull request and solve this issue.

Did get it working with the following options:
options.AutoRegisterTemplate = true;
options.AutoRegisterTemplateVersion = AutoRegisterTemplateVersion.ESv7;
options.IndexFormat = indexFormat.ToLowerInvariant().Replace(".", "-");
options.BatchAction = ElasticOpType.Create;
options.EmitEventFailure = EmitEventFailureHandling.WriteToSelfLog | EmitEventFailureHandling.RaiseCallback;
options.FailureCallback = @event =>
Console.WriteLine(
$"Unable to log event<EMAIL_ADDRESS>with exception: {@event.Exception?.Message}");
options.TypeName = null;

However it the sink still uses index templates that are deprecated in ElasticSearch 8.

I'm using Serilog.Sinks.Elasticsearch 8.4.1 and i am currently encountering the problem.
Is There any way/workaround tu set TypeName in appsetting directly, i won't setting it in code anymore
Thank you

Problem is that default typeName="_doc", hence you cannot reset it from json/yaml configuration. It should be null for 8.x stream.

How to I set TypeName to null using environment variable?
If I use null it appears as _type : "null"

I had the same problem on a Project based on .Net Framework 4.6.1 and .Net Standard 2.0.
Setting the "TypeName" value to null at startup did fix the issue. Here is the configuration. Hope it helps.
Here are some dependency info:
Serilog.Sinks.Elasticsearch version: 8.4.1  (client project: .Net Framework) <---> ElasticSearch version: 8.3.3 (Backend Server)
...
loggerConfig
.Elasticsearch(new ElasticsearchSinkOptions(new Uri(elasticsearchSinkUrl))
{
AutoRegisterTemplate = true,
BatchAction = ElasticOpType.Create,
AutoRegisterTemplateVersion = AutoRegisterTemplateVersion.ESv7,
CustomFormatter = new ElasticsearchJsonFormatter(),
FailureCallback = e => Console.WriteLine(e.MessageTemplate + e.RenderMessage()),
EmitEventFailure = EmitEventFailureHandling.RaiseCallback,
IndexFormat = eventLogIndexFormat,
MinimumLogEventLevel = Serilog.Events.LogEventLevel.Debug,
// This fix Http 400 POST /_bulk Error.
// (The remote server returned an error: (400) Bad Request.. Call: Status code 400 from: POST /_bulk)
// Consider upgrading to the new version of Serilog.sinks.elasticsearch when possible.
TypeName = null,

});

Experience the same issue (Opensearch running in a local docker)
Resolution was indeed to add the TypeName = null but I also had to remove the .ReadFrom.AppSettings()

Set TypeName to null fix problem for me. Now in my appsettings:
"WriteTo": [   {     "Name": "Elasticsearch",     "Args": {       "nodeUris": "http://elasticuser:elasticuserpass@<IP_ADDRESS>:9200",       "indexFormat": "test_index",       "TypeName":  null     }   } ]

This still is not working for me in the appsettings.json I have tried both "TypeName":  null and "typeName":  null.  I have used the latest Serilog.Sinks.Elasticsearch 9.0.0-beta7


Use my bitmap to find out requisites

Required Information

card.io Android SDK Version: io.card:android-sdk:5.3.2

Issue Description
Can i use my bank card photo to find out requisites?

Hi. I develop an app, which works with some cards (loyalty or bank cards). On app starting user takes a photo of banking card. Then the app check a photo and find a type of card. If there is bank card, i start your scan activity by Intent scanIntent = new Intent(this, CardIOActivity.class); and user forced to scan the card again, although the app got a bank card picture yet!
Question:
Can i put a bank card photo to your library to recognize card number instead of starting a scan activity?


client runs delete nodes and checkboxes missing

overview
previously on client runs there was a delete nodes button and checkboxes on each node row, that seems to be missing now.
before

current

repro steps

load some nodes
navigate to infrastructure > client runs
notice no delete nodes button and no checkboxes

I was in that file not too long ago so I went snooping around and I can confirm that all the html and javascript to delete the nodes is still there.  Its specifically related to the @Input canDeleteNodes. If I manually turn those to true, all the checkboxes and delete button show again.
It looks like its linked to authorizedChecker in client-runs.component.ts

FYI: This bug was introduced in build<PHONE_NUMBER>3636

so.. what is the fix. Do we need to patch our existing chef automate web site?

@powersparks this issue has been resolved; any build after https://automate.chef.io/release-notes/?v=20200529133631 will have the fix


Allow configuration of outbound proxy

GitArena makes a couple of outbound requests to external services (such as Gravatar). We should A
allow configuration of outbound proxies for each request.

reqwest, the http library GitArena uses to make outbound requests, accepts configuring proxies via the HTTP_PROXY and HTTPS_PROXY environment variable. Thereby no code change is required from our side.


issue detection

The earlier tests for List were not correct. We have a new testsuite in ss-store which will run basic tests for all store implementations. Those List tests have been updated to check this behavior properly. So hopefully we wont need these tests now.
If you want to add, add it to the testsuite in proper format. Closing this.


Error in Code Snippet.

Thanks Kevin for #59 and #60 to improve the README. I appreciate your efforts!


Add support for dropping repository

It would be nice if the plugin could also automatically drop the repo after successful promotion.

I thought that Nexus does it for you. Nevertheless I have never verified that.
Btw, do you need to release a different project using the same staging profile just after the first one? Or there is some other issue that you need it?

Huh, so it does.  I guess it just took way longer than I expected.

I'm reopening it as I found dropping repository useful in CD pipelines. In addition there is even PR #13 that I plan to work on.

Do we have an ETA on this enhancement? I have encountered scenarios in which using it would be very helpful and would alleviate some headaches.

I need it in my CDeliveryBoy, so most likely it will be implemented. However, I have also other things, some duplication would need to be removed by the way and I started a migration to OkHttp some time ago, so I cannot promised any dates :(.

Btw, why do you need it so much?

hi @szpak , we also need this enhancement, thanks

@chinkong83 Would enhancement proposed in #29 be sufficient for you? If not, could you write what do you need drop repository for?

Hi, @szpak
I am not sure #29 mentioned for what, but I think PR #13 meet my requirement. I see PR #13 say we need wait for 10min to drop after release, this resolve my problem but not enough, because we adopt travis for our CD pipeline, maybe 2 or above merge occur in very close, so we still need it...

In #13 the requester had a problem with configuration and after my changes the dropRepositories task was not needed anymore. After promotion the artifacts should be dropped automatically with the similar delay that a regular drop operation. Therefore I wonder what is your exact use case.

ok, but how long of delay time? 10min?

It should be done "immediately" after promotion (Nexus itself should call drop internally). Do you have very large artifacts there? What was the shortest time you've got that error?

oh, strange.... Artifact not drop immediately after plugin promote push it, I try twice but always need to drop manually, my artifact file size about 60KB, totally about 30 jar file(include source and javadoc)

Strange. Is it an open source project available on GitHub?

@szpak this is our git repo: https://github.com/yahoo/parsec-libraries

I took a look at your builds.
The following case:

Wrong number of received repositories in state 'open'. Expected 1, received 2

usually occurs when the previous release try failed. It is required to drop trashy repository by hand via UI (and here #29 would be useful). There was a proposal to close all repositories, but it can be dangerous and I will try to implement that rather with #29.
Another group of problems is:

Unexpected state of reveived repository. Received open, expected closed

It usually shouldn't happen, however, I think I have seen it before somewhere and reported #21. It may be related to the fact that you have multiple artifacts and it takes more time on the Nexus side). Could you enable --info and paste the output (a retry mechanism wasn't used in that case for some reason)?

Thanks @szpak
I paste console full log to gist, please take a look:
https://gist.github.com/chinkong83/972eb9b56039851ea103177e6bb4fb07

Oh, I see staging repositories already drop comyahooparsec-1011, but seems it still keep a period of time.

In the log I see that retry mechanism behaved properly in closeRepository, so it should also in promoteRepository. You can increase numberOfRetries to 20 or so (the default 7 seconds can be too short for multiple subprojects and it will be changed in the future version - #12). Anyway I need to improve logging without info to give an user a hint (created #34).
What was the time needed to make it closed? 5 minutes? 10? 1 hour? I propose you to reach the Sonatype guys via the mailing list: ossrh-users AT glists.sonatype.com and ask about that giving them that particular case (with a staging repository name).
I suspect that time can be needed to do release stuff and just after that a drop operation is performed. Therefore, a try to close the repository which is being promoted would fail (at best). However, I may be wrong and if for unknown reason a drop is not called just after a release operation is performed successfully it may be a good to add an extra drop after the release/promote (nevertheless, it would be much better to have it implemented on the Nexus side if in fact not available right now).

ok, thanks @szpak 👍

@chinkong83 Can you share a ticket in Sonatype Jira you created or a thread on the mailing list? I would like to follow that discussion (about delay between releasing and dropping).

Hi @szpak
We have a similar problem when publishing Grails using this plugin. What happens is we have multi-project build where two of the subprojects use the packaging "pom" instead of the default. These two end up in a separate staging repository and then we get the dreaded.
Execution failed for task ':closeRepository'.
> Wrong number of received repositories in state 'open'. Expected 1, received 2

See https://travis-ci.org/grails/grails-core/builds/193426489#L2361
Is there any reason why these two projects end up in a separate staging repo? Or any other workarounds for this issue?

@graemerocher It seems to be justified to have two repositories in your case. In travis-build.sh you:

upload the artifacts
the new repository is implicitly created
close & promote the first repository
upload another artifacts
the 2nd repository is implicitly created (the 1st is already closed or in your case is rather being closed)
close & promote the 2nd repository - there is the issue with the first repository not being closed already

Two ideas to consider:

You seem to upload bigger number of artifacts. Please add --info/-i to your closeAndPromoteRepository tasks execution (in the current configuration) which probably gives a result that timeout is too short. Then increase it with numberOfRetries and delayBetweenRetriesInMillis configuration parameters - it's better than sleep 30 (or more in your case) as it asks proactively e.g. every 5 or 10 seconds and finishes as soon as just one repo is available.
Do you need to separate 2 uploads operations with closeAndPromote? 2 upload commands should  end up in the same staging repository. Then you could just close and promote it (with 2 groups of artifacts).
2b. Btw, I assume that integrationTest does not depends on artifacts available in The Central Repository (aka Maven Central) as it can takes up to several minutes to propagate.

@szpak Thanks for the feedback.

I will try this thanks.

We publish POM-only dependencies that use a different upload setup to the rest of the project, even if those are not published in separate steps they still end up in 2 different staging repositories. I'm not sure of a way around that since Gradle doesn't seem to natively allow publishing just POMs (so we use the Maven Ant tooling via Gradle)

@graemerocher @chinkong83 You can upgrade to 0.8.0. I changed the used REST API methods to have support for "autoDropAfterRelease` (#37). The released repository should be gone when Gradle task is finished.

@szpak great news, thanks!

Closing as the original request:

It would be nice if the plugin could also automatically drop the repo after successful promotion.

is implemented starting with version 0.9.0.

Please create a new issues if you have a good use case to be able to call dropRepository from Gradle on demand.


python3Packages.qiskit*: update qiskit and remove deprecated packages

This pull request is an attempt to fix the broken qiskit package in nixpkgs. It removes deprecated Qiskit-related Python packages from nixpkgs and updates python3Packages.qiskit from version 1.2.2 to 1.2.4. The upstream changes consolidated functionality from qiskit-terra into qiskit, so the update primarily involved migrating the qiskit-terra definition into qiskit and making minor adjustments.
Due to time constraints, I was unable to adjust the extensive list of currently disabled tests, which must be extended to account for new test failures. As a temporary measure, I have removed the pytestCheckHook.
I performed some quick manual testing with qiskit, and everything appears to be working as expected.
Other qiskit-* packages still fail to build and should be addressed in future pull requests.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 357208

x86_64-linux

:fast_forward: 20 packages marked as broken and skipped:

python311Packages.qiskit-aer
python311Packages.qiskit-aer.dist
python311Packages.qiskit-finance
python311Packages.qiskit-finance.dist
python311Packages.qiskit-machine-learning
python311Packages.qiskit-machine-learning.dist
python311Packages.qiskit-nature
python311Packages.qiskit-nature.dist
python311Packages.qiskit-optimization
python311Packages.qiskit-optimization.dist
python312Packages.qiskit-aer
python312Packages.qiskit-aer.dist
python312Packages.qiskit-finance
python312Packages.qiskit-finance.dist
python312Packages.qiskit-machine-learning
python312Packages.qiskit-machine-learning.dist
python312Packages.qiskit-nature
python312Packages.qiskit-nature.dist
python312Packages.qiskit-optimization
python312Packages.qiskit-optimization.dist

:white_check_mark: 4 packages built:

python311Packages.qiskit
python311Packages.qiskit.dist
python312Packages.qiskit
python312Packages.qiskit.dist

Builds and loads into jupyter here, thanks!


Support choices exercise in bibliotheca

What is missing about this? Atheneum import integration, right?

@aguspina but was atheneum integration done? If not, we can't close this issue yet

I'm on it! I'm creating a new issue because this issue vas specific for
bibliotheca
El mié., 9 de nov. de 2016 a la(s) 15:59, Franco Leonardo Bulgarelli <
<EMAIL_ADDRESS>escribió:

@aguspina https://github.com/aguspina but was atheneum integration
done? If not, we can't close this issue yet
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mumuki/mumuki-teacher-tools/issues/201#issuecomment-259495409,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD6mDPKjdMmzsmKOSpmhwlq_r1ZpDSm9ks5q8hf5gaJpZM4KszmH
.

:kissing_heart:


[🐞] Custom 404 is generated, but not used automatically

Which component is affected?
Qwik City (routing)
Describe the bug
I'm trying to use a custom 404 page and followed the example from the docs: https://qwik.dev/docs/advanced/routing/ - as I'm understanding it, I need to create a 404.tsx in routes and go to /sdfdsfdsf to see my custom 404 page instead of the default one. After doing that, I'm seeing the default 404 page and a 404.html file generated which I can visit.
I was under the impression that instead of seeing the default 404 page, it would use my custom 404 page. If only the default 404 is used locally for development purposes, then I guess the docs should reflect that behaviour.
Reproduction
https://stackblitz.com/edit/qwik-starter-uhmynv
Steps to reproduce

Create a qwik city project pnpm create qwik@latest
Add a 404.tsx at the base and fill it with something like "Custom 404"
Go to /avsdgfsdfsdfdsf - notice that the default 404 is used

System Info
System:
    OS: macOS 14.3.1
    CPU: (10) arm64 Apple M1 Pro
    Memory: 87.45 MB / 16.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 18.18.2 - ~/.volta/tools/image/node/18.18.2/bin/node
    Yarn: 1.22.19 - ~/.volta/tools/image/yarn/1.22.19/bin/yarn
    npm: 8.19.4 - ~/.volta/tools/image/npm/8.19.4/bin/npm
    pnpm: 8.15.3 - ~/.volta/tools/image/pnpm/8.15.3/bin/pnpm
  Browsers:
    Chrome: 121.0.6167.184
    Edge: 122.0.2365.52
    Safari: 17.3.1
  npmPackages:
    @builder.io/qwik: ^1.4.5 => 1.4.5
    @builder.io/qwik-city: ^1.4.5 => 1.4.5
    undici: * => 6.6.2
    vite: ^5.0.12 => 5.1.4

Additional Information
In this screenshot you can see that it generated a 404.html file and it's viewable but it's not being used. I tried both dev and preview and neither of them ever worked.

Hi, it's working ad intended
Here is the doc
Extract from the docs

Note: During development and in preview mode, the custom 404 pages will not be rendered, but instead the default Qwik City 404 page will show. However, when building the app for production, the custom 404 page will be statically generated as a static 404.html file.


Be able to easily scroll/navigate larger networks

See https://bjdmeest.github.io/dataflow-visualization/ final example: that's a pretty large network and going from top to bottom takes quite some time. Either being able to click on the minimap to jump to the right location or some kind of scrollbars or whatever would be cool.
Example: https://medium.com/@christian_maehler/implementing-a-scrollbar-for-react-flow-29653c2562fd

2 bugs:

The controls disappear when scrolling down
Cannot zoom when scrollbar is at bottom (if scrollbar is at bottom scrolling has effect on the scrollbar, not on the zooom of the flow)


Add Sevcon by Big Giant Circles

Song Name
Sevcon
Artist Name
Big Giant Circles
Source
Youtube
Youtube Link
yhviMWU_AxM
Direct File Link
No response
Song ID
584131
Start Offset [ms]
200
Extra
Created from Auto Nong.
Created for level: 75715774

reject

spam
you are trying to replace 584131 with an unrelated song. 584131 is used in many rated levels so we can't afford to replace it with a random song


Create devcontainer.json

RIHAN-Bhai-377FCB410B-EX4786== RIHAN-Bhai-6235AA735E-EX4786== RIHAN-Bhai-C3B1DF4251-M4786== RIHAN-Bhai-91FB269771-EX4786== RIHAN-Bhai-C36F70FDE8-EX4786==

This looks like a spam / bot contribution, similar to a number of other contributions from that account. Apologies if I'm mistaken, please feel free to reopen with details about why this change is being proposed.
/close


Hdfs dependency validation while creating an environment

While creating an environment that has either a Hive or HBase service from a service pool we need to add validation satisfying following conditions.

Only one instance of HBase or Hive service is selected
If Hbase and Hive are both selected they have to come from same service pool
Adding either Hbase or Hive to a service pool should automatically bring Hdfs from the same service pool

@sank
Let me know if you have any more questions regarding this

Fixed and merged in commit: https://github.com/hortonworks/streamline/commit/6c06d010e75afe9699e864fad10bd7d0405e98a6


Compiler hangs using v.7.0.x

Bug Report or Feature Request (mark with an x)
- [X] bug report -> please search issues before submitting
- [ ] feature request

Command (mark with an x)
- [ ] new
- [ ] build
- [X] serve
- [ ] test
- [ ] e2e
- [ ] generate
- [ ] add
- [ ] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [ ] config
- [ ] help
- [ ] version
- [ ] doc

Versions
node: v11.0.0
npm: 6.4.1
ng: 7.0.4
OS:
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.1 LTS
Release:	18.04
Codename:	bionic

Repro steps
Cant really reproduce it.
The problem is that after I have saved one or more files 10 - 20 times over a period of time the compiler just hangs.
It could look something like this:
Date: 2018-11-08T20:02:21.788Z - Hash:<PHONE_NUMBER>f614f5d384 - Time: 1256ms
23 unchanged chunks
chunk {app-fund-fund-module} app-fund-fund-module.js, app-fund-fund-module.js.map (app-fund-fund-module) 224 kB  [rendered]
ℹ ｢wdm｣: Compiled successfully.
ℹ ｢wdm｣: Compiling...

So what you see is a previous successful compiled chunk, and the browser reloads fine, then the next time I saved it just said compiling... and nothing more happen. If I press Ctrl+C I still need to find the process and kill it. The process is using 100% of a CPU core, at that point.
It a rather large project and before version 7 of the CLI and Angular this never happened.

Can you try if using Nodejs 8 fixes it? I had a similar issue like that and for me NodeJS 11 was the root cause (although, I am on Windows).

@dhardtke I'll try that! Thank you :-) I'll report back

Seems to be related to the node version. Going back to 10.9.0 fixed the problem.
@dhardtke Thank you very much. It was driving me insane!

Yeah https://github.com/angular/angular-cli/issues/12816 has reports of problem with node 11.. we haven't gotten around to testing it on CI yet :/


defference between cldm_v15_unicontrol_v11.yaml and cldm_v15_unicontrol.yaml

Thanks for your great work! I notice that there are two yamls and two ckpts. For the yamls, it seems that they corresponds to cldm_unicontrol and cldm_unicontrol_v11. The main differences are: (1)whether use zero_conv for self.input_hint_block_zeroconv_0 and self.input_hint_block_zeroconv_1 (2)whether use self.input_hint_block_zeroconv_0[1] and self.input_hint_block_zeroconv_1[1] for (guided_hint) guided_hint (3) whether task_id_layernet and task_id_hypernet. cldm_unicontrol_v11 uses the zero_convs and   self.input_hint_block_zeroconv_x[1], and open the network structures about task_id.
I have two questions: (1) does that mean unicontrol.ckpt correspond to  cldm_v15_unicontrol.yaml and unicontrol_v1.1.ckpt correspond to  cldm_v15_unicontrol_v1.1.yaml ?  (2) I am trying adapting the deblur task to another datastet, which yaml and ckpt shoule I use? (3) what's the special purpose for the above design? (4) Using the released ckpt for  debluring, I got following results which seem to collapse, is it reasonable?
input image:

prompt:
""
output:

My mistake. unicontrol_v1.1.ckpt  is not aligned with cldm_v15_unicontrol_v1.1.yaml. Please use cldm_v15_unicontrol.yaml all the time (even for unicontrol_v1.1.ckpt)!!!

BTW, our HF space is available (https://huggingface.co/spaces/Robert001/UniControl-Demo). Please feel free to play with it!

My mistake. unicontrol_v1.1.ckpt is not aligned with cldm_v15_unicontrol_v1.1.yaml. Please use cldm_v15_unicontrol.yaml all the time (even for unicontrol_v1.1.ckpt)!!!

Thank you!


Directory structure highlighting

I created a new syntax file for highlighting plain-text directory structure diagrams, like this:
|-- app
|   |-- controllers
|   |-- models
|   `-- views

It's extremely simple, just a single small regex to highlight commonly used symbols for this purpose:
Prism.languages.folders = {
    // Just highlight symbols used to denote folder structure
    keyword: /^([-|+`\s]+)/gm
};

Is this something you think others might want? I can give you a pull request very easily if you think so, I just wasn't sure if anyone other than me cares :p

This is awesome @kbjr and @Golmote!
It doesn't look like there are plans to bring this into the core Prism, right? If not, we could probably close this issue,

You're right.

is there any reason why it is not in core? i think it's a very common need to show file structures in code blocks

Hi @muescha,
We generally try to keep the core very small. Is there a reason you need this in the core? With the bundler on the website, there isn't really much difference.

@LeaVerou He meant why is it an external project. But we now have included this feature as a plugin in #2265! 🎉


Type coercion should be stable

That is to say that if x is identified as numerical then the following check should also hold true - else the value should remain of type string:
String( Number( x ) ) === String( x )

For example I have a scenario as follows which leads to an invalid configuration:
foreign_api.verification_token = 000111111       # leading zeros truncated
foreign_api.client_id =<PHONE_NUMBER>.10101010101   # too large for float / data loss

I've gotten around this by prefixing an arbitrary token to the value and using the reviver option to remove it, i.e.
foreign_api.verification_token = --000111111

properties.parse('file', {
    // ...
    reviver: function(key, value){
        if(typeof value === 'string')
            return value.replace(/^--\s*/, '');
        return this.assert();
    },
    // ...
);

This seems clunky though and I think most use-cases of node-properties would benefit from stable type-coercion.

> String (Number("+62"))  === String ("+62")
false

I get the same problem. In our .properties file, there's value that is parsed incorrectly, +62 is parsed as 62


Reproducing WikiSeeAlsoTitles-350K

Hello
When I implement the astec in WikiSeeAlsoTitles-350K, I got the following data error:
File "../runner.py", line 433, in
config=json.load(open(config)))
File "../runner.py", line 376, in run_one
return run_deepxml(work_dir, version, seed, config)
File "../runner.py", line 127, in run_deepxml
_train_time, _ = main(args)
File "/home/zx22/xml/baselines/astec/programs/deepxml/deepxml/main.py", line 425, in main
output = train(model, params)
File "/home/zx22/xml/baselines/astec/programs/deepxml/deepxml/main.py", line 104, in train
surrogate_mapping=params.surrogate_mapping)
File "/home/zx22/xml/baselines/astec/programs/deepxml/deepxml/libs/model_base.py", line 529, in fit
surrogate_mapping=surrogate_mapping)
File "/home/zx22/xml/baselines/astec/programs/deepxml/deepxml/libs/model_base.py", line 157, in _create_dataset
_type=_type)
File "/home/zx22/xml/baselines/astec/programs/deepxml/deepxml/libs/dataset.py", line 28, in construct_dataset
normalize_labels, num_clf_partitions, feature_type, surrogate_mapping)
File "/home/zx22/xml/baselines/astec/programs/deepxml/deepxml/libs/dataset.py", line 102, in init
self._process_labels(model_dir, surrogate_mapping)
File "/home/zx22/xml/baselines/astec/programs/deepxml/deepxml/libs/dataset.py", line 138, in _process_labels
axis=1)
File "/home/zx22/.local/lib/python3.7/site-packages/xclib-0.97-py3.7-linux-x86_64.egg/xclib/utils/sparse.py", line 350, in _map
return _map_cols(X, mapping, shape, oformat)
File "/home/zx22/.local/lib/python3.7/site-packages/xclib-0.97-py3.7-linux-x86_64.egg/xclib/utils/sparse.py", line 323, in _map_cols
col_idx = func(col_idx)
File "/home/zx22/.conda/envs/xml/lib/python3.7/site-packages/numpy/lib/function_base.py", line 2108, in call
return self._vectorize_call(func=func, args=vargs)
File "/home/zx22/.conda/envs/xml/lib/python3.7/site-packages/numpy/lib/function_base.py", line 2192, in _vectorize_call
outputs = ufunc(*inputs)
File "/home/zx22/.local/lib/python3.7/site-packages/xclib-0.97-py3.7-linux-x86_64.egg/xclib/utils/sparse.py", line 322, in
func = np.vectorize(lambda x: mapping[x])
KeyError: 351990
I use the bow feature in the website http://manikvarma.org/downloads/XC/XMLRepository.html and the running command is:
./run_main.sh 0 DeepXML WikiSeeAlsoTitles-350K 0 108
Thanks!

Thanks for raising it. Let me look into this.

Hi
This should be fixed now. Please re-install pyxclib as well.

Thank you very much!


what's the meaning of Plane?

Width 4
Height 1
-5.445015e-02 -9.985028e-01 5.238001e-03 1.661760e+00
the four number stand for what?
thx

It is plane coefficients [a, b, c, d] such that ax+by+cz+d = 0 for any 3D point on the plane. You can refer to some wiki pages for more details.

@bensondou @asharakeh  Hi, could you please tell me which XYZ coordinate does this plane matrix use, the velodyne coordinate or the camera (unrectified) one? Or I just want to know which axis is for depth direction.
Thanks!


Update README.md

The pip install infretis installs a pretty old version. would this be better @andersle? Or should we update the package on pip?


V16 dev

Conversation support
Updated models for v16 (missess assignee still)
Request can access tierConfigs in backwards compatibility model (to be deprecated in 2-3 releases)
Minor changes

Codecov Report

Merging #28 into master will decrease coverage by 12.49%.
The diff coverage is 94.64%.

@@             Coverage Diff              @@
##             master      #28      +/-   ##
============================================
- Coverage     99.47%   86.98%   -12.5%
- Complexity      296      314      +18
============================================
  Files            45       46       +1
  Lines           767      822      +55
============================================
- Hits            763      715      -48
- Misses            4      107     +103

Impacted Files
Coverage Δ
Complexity Δ

src/AutomationEngine.php
100% <ø> (ø)
11 <0> (ø)
:arrow_down:

src/Modules/Core.php
100% <ø> (ø)
10 <0> (ø)
:arrow_down:

src/Modules/TierConfiguration.php
100% <ø> (ø)
10 <0> (ø)
:arrow_down:

src/Modules/Fulfillment.php
88% <100%> (ø)
10 <1> (?)

...c/Runtime/Providers/FulfillmentServiceProvider.php
100% <100%> (ø)
1 <0> (ø)
:arrow_down:

...ime/Providers/TierConfigurationServiceProvider.php
100% <100%> (ø)
1 <0> (ø)
:arrow_down:

src/Runtime/Providers/UsageServiceProvider.php
100% <100%> (ø)
1 <0> (ø)
:arrow_down:

src/Model.php
100% <100%> (ø)
53 <0> (ø)
:arrow_down:

src/Request.php
100% <100%> (ø)
14 <2> (+2)
:arrow_up:

src/Conversation.php
100% <100%> (ø)
3 <3> (?)

... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b88dd69...7aa7e62. Read the comment docs.


Make HyperAdapter accessible

Issue #, if available:
Was trying to use this earlier today to create my own connector and realized the hyper_impls module was private.
Description of changes:
Re-export HyperAdapter as Adapter
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

The module name hyper is conflicting with the hyper crate. Will rename it


Parameters for accuracy

Recommended reading: https://www.otexts.org/fpp/2/5

Using MAPE for final test accuracy reported for the user. If cross-validation is being done for model-selection, choose RMSE as the metric to optimize.


Tech Stack: Learning JSON-RPC

Learning JSON-RPC is a journey into the world of remote procedure calls (RPC) facilitated by JSON, a lightweight data-interchange format. This guide aims to provide a comprehensive understanding of JSON-RPC, from its fundamentals to advanced implementation techniques. We'll explore what JSON-RPC is, how it works, and why it's beneficial, along with step-by-step examples and best practices for using it effectively.

I am waiting to review :)


Allow duplication without a parent

The code for duplicating a layer only works if the initial layer has a parent. Why not allow copying the layer without a parent?

I guess there is no reason haha


Title seem to be included in a column and should be outside

The Title should be put out of the column it seems.
Original on right and franklin on left

After putting the title outside of the columns this was fiyed


Clear terminal output to only show latest message

We are transition this repo to https://github.com/sst/v2 and are closing this issue for now. If it is still relevant please reopen it there - sorry for the trouble


Update README.md with comment on original yolov5's accuracy

Just as the title, update README.md with yolov5 f32 model's accuracy on bdd100k-val.

LGTM


Add types for token

I would like to have types for the token object returned from the access token request.
interface Token {
  accessToken: string;
  tokenType: string;
  expiresIn: number;
}

I can open a PR for it. Thank you

Yeah if you are willing to create a PR, then you can proceed.

Thanks for your contribution!
The changes will be updated soon!

Sorry for a very late response!
The changes you made in #2 has been update in v1.0.5.


add note to credentials plugin cb header comment

attempt to partly address https://github.com/grpc/grpc/issues/8110

LGTM.


msys2: fails to build: FAILED: src/Graphene-1.0.gir

Experienced behavior
With #87 applied, the build fails during introspection scanning:
$ ninja
[...]
"g-ir-scanner" "../src/graphene-box.h" "../src/graphene-euler.h" "../src/graphene-frustum.h" "../src/graphene-macros.h" "../src/graphene-matrix.h" "../src/graphene-plane.h" "../src/graphene-point.h" "../src/graphene-point3d.h" "../src/graphene-quad.h" "../src/graphene-quaternion.h" "../src/graphene-ray.h" "../src/graphene-rect.h" "../src/graphene-size.h" "../src/graphene-sphere.h" "../src/graphene-triangle.h" "../src/graphene-types.h" "../src/graphene-vec2.h" "../src/graphene-vec3.h" "../src/graphene-vec4.h" "../src/graphene-version-macros.h" "../src/graphene-gobject.h" "../src/graphene-alloc.c" "../src/graphene-box.c" "../src/graphene-euler.c" "../src/graphene-frustum.c" "../src/graphene-matrix.c" "../src/graphene-plane.c" "../src/graphene-point.c" "../src/graphene-point3d.c" "../src/graphene-quad.c" "../src/graphene-quaternion.c" "../src/graphene-ray.c" "../src/graphene-rect.c" "../src/graphene-size.c" "../src/graphene-sphere.c" "../src/graphene-triangle.c" "../src/graphene-vectors.c" "../src/graphene-gobject.c" "-pthread" "-mms-bitfields" "-IC:/msys64/mingw64/include/gobject-introspection-1.0" "-IC:/msys64/mingw64/lib/libffi-3.2.1/include" "-IC:/msys64/mingw64/include/glib-2.0" "-IC:/msys64/mingw64/lib/glib-2.0/include" "-IC:/msys64/mingw64/include" "--no-libtool" "--namespace=Graphene" "--nsversion=1.0" "--warn-all" "--output" "src/Graphene-1.0.gir" "--identifier-filter-cmd=C:/msys64/usr/src/graphene/src/identfilter.py" "--c-include=graphene-gobject.h" "--accept-unprefixed" "-DGRAPHENE_COMPILATION" "-IC:/msys64/usr/src/graphene/src" "-IC:/msys64/usr/src/graphene/_build/src" "--include=GObject-2.0" "--symbol-prefix=graphene" "--identifier-prefix=Graphene" "--pkg-export=graphene-gobject-1.0" "-IC:/msys64/mingw64/lib/glib-2.0/include" "-IC:/msys64/mingw64/include" "-mms-bitfields" "-IC:/msys64/mingw64/include/glib-2.0" "-LC:/msys64/usr/src/graphene/_build/src" "-lintl" "-lm" "-LC:/msys64/mingw64/lib" "-lglib-2.0" "-lgobject-2.0" "--library" "graphene-1.0"
CreateProcess failed: The system cannot find the file specified.
ninja: fatal: GetOverlappedResult: The handle is invalid.

$ which g-ir-scanner
/mingw64/bin/g-ir-scanner

$ pacman -Qo /mingw64/bin/g-ir-scanner
/mingw64/bin/g-ir-scanner is owned by mingw-w64-x86_64-gobject-introspection 1.50.0-2

Expected behavior
The build should progress beyond this line.
Steps to reproduce
$ mkdir _build
$ cd _build
$ meson.py ..
$ ninja

Operating system in use
Windows 7 Enterprise SP1.
SIMD implementation in use
Not sure.

Invoking it directly from the command line is a little surprising:

Invoked without the --identifier-filter-cmd option, g-ir-scanner completes, but I guess that isn't helpful.

Presumably the CreateProcess() trying to invoke a .py file in the absence of a Win32 Python.
Invoking it with --identifier-filter-cmd=python {{path to identfilter.py}} seems to sure this oddity, but there's more (when invoked from the shell):
$ "g-ir-scanner" "../src/graphene-box.h" "../src/graphene-euler.h" "../src/graphene-frustum.h" "../src/graphene-macros.h" "../src/graphene-matrix.h" "../src/graphene-plane.h" "../src/graphene-point.h" "../src/graphene-point3d.h" "../src/graphene-quad.h" "../src/graphene-quaternion.h" "../src/graphene-ray.h" "../src/graphene-rect.h" "../src/graphene-size.h" "../src/graphene-sphere.h" "../src/graphene-triangle.h" "../src/graphene-types.h" "../src/graphene-vec2.h" "../src/graphene-vec3.h" "../src/graphene-vec4.h" "../src/graphene-version-macros.h" "../src/graphene-gobject.h" "../src/graphene-alloc.c" "../src/graphene-box.c" "../src/graphene-euler.c" "../src/graphene-frustum.c" "../src/graphene-matrix.c" "../src/graphene-plane.c" "../src/graphene-point.c" "../src/graphene-point3d.c" "../src/graphene-quad.c" "../src/graphene-quaternion.c" "../src/graphene-ray.c" "../src/graphene-rect.c" "../src/graphene-size.c" "../src/graphene-sphere.c" "../src/graphene-triangle.c" "../src/graphene-vectors.c" "../src/graphene-gobject.c" "-pthread" "-mms-bitfields" "-IC:/msys64/mingw64/include/gobject-introspection-1.0" "-IC:/msys64/mingw64/lib/libffi-3.2.1/include" "-IC:/msys64/mingw64/include/glib-2.0" "-IC:/msys64/mingw64/lib/glib-2.0/include" "-IC:/msys64/mingw64/include" "--no-libtool" "--namespace=Graphene" "--nsversion=1.0" "--warn-all" "--output" "src/Graphene-1.0.gir" "--identifier-filter-cmd=/usr/bin/env python C:/msys64/usr/src/graphene/src/identfilter.py" "--c-include=graphene-gobject.h" "--accept-unprefixed" "-DGRAPHENE_COMPILATION" "-IC:/msys64/usr/src/graphene/src" "-IC:/msys64/usr/src/graphene/_build/src" "--include=GObject-2.0" "--symbol-prefix=graphene" "--identifier-prefix=Graphene" "--pkg-export=graphene-gobject-1.0" "-IC:/msys64/mingw64/lib/glib-2.0/include" "-IC:/msys64/mingw64/include" "-mms-bitfields" "-IC:/msys64/mingw64/include/glib-2.0" "-LC:/msys64/usr/src/graphene/_build/src" "-lintl" "-lm" "-LC:/msys64/mingw64/lib" "-lglib-2.0" "-lgobject-2.0" "--library" "graphene-1.0" ; echo "rc=$?"
dlltool.exe: Unable to determine dll name for `C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/lib/../lib/libm.a' (not an import library?)
dlltool.exe: Unable to determine dll name for `C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/lib/libm.a' (not an import library?)
dlltool.exe: Unable to determine dll name for `C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../lib/libgraphene-1.0.a' (not an import library?)
dlltool.exe: Unable to determine dll name for `C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../libgraphene-1.0.a' (not an import library?)
C:\msys64\mingw64\x86_64-w64-mingw32\include\sec_api\string_s.h:42: syntax error, unexpected ';' in '  ;' at ';'
C:\msys64\mingw64\x86_64-w64-mingw32\include\sec_api\string_s.h:69: syntax error, unexpected ';' in '  ;' at ';'
C:\msys64\mingw64\x86_64-w64-mingw32\include\sec_api\string_s.h:71: syntax error, unexpected ';' in '  ;' at ';'
ERROR: can't resolve libraries to shared libraries: m, graphene-1.0
rc=1

The obvious fix,
diff --git a/src/meson.build b/src/meson.build
index bf2fd28..c7dfc30 100644
--- a/src/meson.build
+++ b/src/meson.build
@@ -127,7 +127,7 @@ endforeach

 if build_gir
   gir_extra_args = [
-    '--identifier-filter-cmd=' + join_paths(meson.current_source_dir(), 'identfilter.py'),
+    '--identifier-filter-cmd=python ' + join_paths(meson.current_source_dir(), 'identfilter.py'),
     '--c-include=graphene-gobject.h',
     '--accept-unprefixed',
     '-DGRAPHENE_COMPILATION',

is NOT sufficient to fix this. How do I get gnome.generate_gir() to log sensibly?

The missing m library is mesonbuild/meson#1060 — see also: #82 and #92.
The missing python can be fixed independently.

This is still broken for the reinstated autotools build. I am working on a fix.


⚠ ClamAV has degraded performance

In b82406e, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 78 ms

Resolved: ClamAV performance has improved in b23643e after 5 minutes.


Final App Approval

Final App Approval
Prats Done by @PowerLevel9000

[ ] fetching data from Mission API of Space X
[ ] Displaying them on Table on the missions page
[ ] Conditional Rendering of the status badge
[ ] styling the table
[ ] displaying reversed mission on my profile
[ ] testing every component
[ ] Regarding Functionality

Joining missions and status change to Active mission
Leaving missions and status change to Not A Memeber

Prats Done By @Habtamu210

[ ] fetching data from Rockets API of Space X
[ ] Displaying them on the Rockets (home) page
[ ] Conditional Rendering of the badge
[ ] styling the page
[ ] displaying reversed Rockets on my profile
[ ] testing every component
[ ] Regarding Functionality

Reserving Rocketes and displaying a Reserved Badge
Cancel Reservation and hide the reserved badge

Problem
firefox

firefox can't render our styled module correctly or may be we are missing some specific property
chrome

brave

Edge

happy Coding

Thanks
hi @KayLemba  thanks for your review 🙌🙌🙌🙌

@KayLemba
hi respective CR
I hope you are doing well, I am here to as you about optional suggestions for my project as I am having a week's break I decided to make my projects more attractive
thanks in advance


Add node in overlay routing table before returning overlay response

What was wrong?
We are responding to an overlay request before updating the overlay routing table with the remote NodeId.
This can lead to some unexpected behavior, for example when we receive an OFFER request, we want to check the overlay routing table if the node already exists and start listening for incoming uTP packets, before sending the ACCEPT response.
For reference, the discv5 implementation first adds the remote node to the routing table (after a successful handshake) and then responds with a talk response.
How was it fixed?

Extract a new method from existing code encapsulating the logic for overlay routing table update.
Call the method before building the overlay response.

To-Do

[x] Add entry to the release notes (may forgo for trivial changes)
[ ] Clean up commit history

The rationale for this logic ordering was that we should not delay the response for any processing. In other words, all processing associated with an incoming request should be taken care of after we respond to the request to minimize latency.

We still process the incoming requests after we respond, the only "processing"  before a response is adding/updating the node in the overlay routing table.
In the current implementation, considering the following scenario, there are no guarantees that Node A will be added to Node B's routing table after step 2) and before 3):

Node A sends OFFER request to Node B
Node B responds with ACCEPT message.
Node A sends uTP SYN packet over talk req to node B to initialize uTP stream.

Yes I understand the reason for the change. I would note that, if the node is not already in the routing table, then the function to add the node to the routing table would only be able to add that node if the node exists in the underlying Discv5 routing table. A similar lookup could be done in the handle offer flow, and we could keep things the way they are now. That fallback logic will soon be replaced by a lookup to the ENR cache, which I am working on currently.

@carver the only concern is performance, and those concerns are probably negligible, at least at this time. The PR for the ENR cache and its integration is further away than a day or two, so we can move forward with this change. I think that is more preferable than duplicating the fallback lookup logic in the handle offer flow.


add notice re Gitpod replacing NEAR Studio

this PR addresses a quick triage related to #273 , #274 , #275 which all require more work


AsyncData with a shallow ref

Describe the feature
Right now, useAsyncData uses a ref to store the data. This means that, when returning an object (or an array), the whole object will be made reactive. This can have unwanted performance penalties, depending on the size of the data that is fetched, and depending on the use-case, is often unneeded when returning static data from an API call that is not meant to be changed.
It would be great if you could use a shallowRef instead.
Additional information

[ ] Would you be willing to help implement this feature?
[ ] Could this feature be implemented as a module?

Final checks

[X] Read the contribution guide.
[X] Check existing discussions and issues.

This could really make a significant difference IMO. I can work on this in the upcoming weeks once we decide that it should be implemented @danielroe 🙂