Adding the oc2ls-v1.0-wd14 and oc2slpf-v1.0-wd08 schemas to the project

adding the schema here in hopes that we can utilize your repository to keep track of changes, rather than creating a new one.

I ran the oc2ls-v1.0-wd14.json  file through the JsonSchemaTest suite and found 13 errors (see output below). These are not all errors in your contributed version, but really are indicators of where my interpretation of the spec differed from yours, or where one of us is more restrictive or permissive, or because my version does not allow a Response payload to validate as a Command payload (one of the side effects of using a single .json file as you have done.
In the end, what I can do at least for now until we work through the 13 issues and discuss how to fix it, I'd like to add these to a contrib directory and make it easy to run them through the full automated test suite (with some minor edits). Then, we can talk about how to fix each of the 13 ones that generated a discrepancy.
Pull request accepted (then moving them into contrib in my next commit).
Thanks for your contribution... This is really great!
-Brian

124 ms
junit [JsonSchemaTest]: 13 total, 13 failed

Process finished with exit code 255
124 ms JsonSchemaTest
92 ms contributed - can validate good OpenC2 Commands and Responses(File, String)
72 ms failed results_ext_multiple.json
Exactly one of the following sets of problems must be resolved.
1) [2,15] The object must not have a property whose name is "status". [3,35] The object must not have a property whose name is "status_text". [4,14] The object must not have a property whose name is "results". [19,1] The object must have a property whose name is "action". [19,1] The object must have a property whose name is "target".
2) Exactly one of the following sets of problems must be resolved.
1) [2,15] The value must be constant integer 102.
2) [2,15] The value must be constant integer 200.
3) [2,15] The value must be constant integer 400.
4) [2,15] The value must be constant integer 401.
5) [2,15] The value must be constant integer 403.
6) [2,15] The value must be constant integer 404.
7) [2,15] The value must be constant integer 500.
8) [2,15] The value must be constant integer 501.
9) [2,15] The value must be constant integer 503.
org.opentest4j.AssertionFailedError: expected JSON to be valid, but it did not validate successfully:

5 ms failed results_ext_single.json
Exactly one of the following sets of problems must be resolved.
1) [2,15] The object must not have a property whose name is "status". [3,35] The object must not have a property whose name is "status_text". [4,14] The object must not have a property whose name is "results". [13,1] The object must have a property whose name is "action". [13,1] The object must have a property whose name is "target".
2) Exactly one of the following sets of problems must be resolved.
1) [2,15] The value must be constant integer 102.
2) [2,15] The value must be constant integer 200.
3) [2,15] The value must be constant integer 400.
4) [2,15] The value must be constant integer 401.
5) [2,15] The value must be constant integer 403.
6) [2,15] The value must be constant integer 404.
7) [2,15] The value must be constant integer 500.
8) [2,15] The value must be constant integer 501.
9) [2,15] The value must be constant integer 503.
org.opentest4j.AssertionFailedError: expected JSON to be valid, but it did not validate successfully:

6 ms failed results_empty.json
Exactly one of the following sets of problems must be resolved.
1) [2,15] The object must not have a property whose name is "status". [3,21] The object must not have a property whose name is "status_text". [4,14] The object must not have a property whose name is "results". [5,1] The object must have a property whose name is "action". [5,1] The object must have a property whose name is "target".
2) [4,15] The object must have at least 1 property(ies), but actual number is 0.
org.opentest4j.AssertionFailedError: expected JSON to be valid, but it did not validate successfully:

9 ms failed query_features_all.json
Exactly one of the following sets of problems must be resolved.
1) [2,15] The object must not have a property whose name is "status". [3,21] The object must not have a property whose name is "status_text". [4,14] The object must not have a property whose name is "results". [30,1] The object must have a property whose name is "action". [30,1] The object must have a property whose name is "target".
2) [6,25] The string value must match the pattern "^\d+\.\d+$".
org.opentest4j.AssertionFailedError: expected JSON to be valid, but it did not validate successfully:

32 ms contributed - cannot validate bad OpenC2 Commands and Responses(File, String)
3 ms failed start_container_ext_underscore_first1.json
org.opentest4j.AssertionFailedError: expected JSON to not be valid, but it validated successfully

4 ms failed allow_ipv4net_badip.json
org.opentest4j.AssertionFailedError: expected JSON to not be valid, but it validated successfully

2 ms failed openc2_response_text.json
org.opentest4j.AssertionFailedError: expected JSON to not be valid, but it validated successfully

4 ms failed allow_ipv4net_badcidr.json
org.opentest4j.AssertionFailedError: expected JSON to not be valid, but it validated successfully

6 ms failed query_features_ext_args_nox-.json
org.opentest4j.AssertionFailedError: expected JSON to not be valid, but it validated successfully

3 ms failed openc2_response.json
org.opentest4j.AssertionFailedError: expected JSON to not be valid, but it validated successfully

3 ms failed start_container_ext_underscore_first2.json
org.opentest4j.AssertionFailedError: expected JSON to not be valid, but it validated successfully

3 ms failed results_unknown_profile.json
org.opentest4j.AssertionFailedError: expected JSON to not be valid, but it validated successfully

4 ms failed openc2_command_query_features_all.json
org.opentest4j.AssertionFailedError: expected JSON to not be valid, but it validated successfully

Generated by IntelliJ IDEA on 7/23/19 11:24 AM

Brian,
Than you for accepting my pull request. I ran the test suite myself a few
weeks ago and agree with your assessment. I think the main reason we differ
is because of ambiguity in the language spec. I think it will be important
for us to bring these issues to the language spec and come to a consensus.
I am happy to change my schema once we get clarity as I am sure you are
too. I think the only real difference in the end will be these 2 issues.

keeping the 2 schema as one file or two [command, response]
Your reason for separating the two is entirely logical and makes sense to
me. My original reason for combining them was to try and simplify the
actuator registration process at the orchestrator, additionally I thought
it would allow me to re-use definitions that are shared between both
commands and responses.

The role of the language spec versus the profile specs.
I view the language schema as a way to validate the openc2 language as a
whole and the profile schemas as a way to validate specific profiles. The
majority of devices and actuators will likely validate utilizing profile
schema, but the language schema could have multiple uses. For us it helps
us construct our command generator, it helps us validate at the
orchestrator for structure, it helps us create new custom profile schema,
and one member suggested using it as a reference to the profile schemas so
that those schemas can be very small and just add the changes.

It is my understanding that your schema is different in that it includes
all of the"official" profile custom parts, of course I could be wrong -
these are my assumptions based on what I saw.
My main issue with this is that it will not validate a custom profile, and
secondary issue is that when we have hundreds of profiles it will be very
difficult to deal with adding and subtracting these from the language
schema.
Perhaps we can come to a middle ground in this space.
V/R
On Tue, Jul 23, 2019 at 2:40 PM bberliner<EMAIL_ADDRESS>wrote:

Merged #1 https://github.com/bberliner/openc2-json-schema/pull/1 into
master.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bberliner/openc2-json-schema/pull/1?email_source=notifications&email_token=AGNJFXR7J5J2Y5DPA7PXGM3QA5GBZA5CNFSM4HUD2SA2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOSVALK7A#event-2504045948,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGNJFXVBHMUG4CK6CTLXQ5DQA5GBZANCNFSM4HUD2SAQ
.

--
Danny Martinez
Principal Cyber Security Engineer
HII Mission Driven Innovative Solutions (HII-MDIS)
Technical Solutions Division
302 Sentinel Drive, Suite 300 | Annapolis Junction, MD 20701
*Mobile *(407) 257-0031
Confidentiality Statement: HUNTINGTON INGALLS INDUSTRIES PROPRIETARY - This
e-mail contains information proprietary or private to Huntington Ingalls
Industries, Inc., and is not to be disclosed to, copied by, or used in any
manner by others without the prior express, written permission. If you are
not the intended recipient, please delete without copying and kindly advise
the sender by e-mail of the mistake in delivery.


Eval question

Thanks for amazing job~ One question, how do you use Gpt4o and qwen2-vl for video input testing? Don't they only support single-frame image inputs?

Thank you for asking.
GPT-4o now supports multi-frame input. You can refer to the guides here.

Additionally, check out Qwen-2VL-7B here.


Error with model.predict() when using early stopping

Hi, I trained my model using this  and used early stopping criterian in setfit model. After training saved the model as : model.save_pretrained("setfit_early_stopping") and then I am trying to load the model and predict on my text, using below code:
model = SetFitModel.from_pretrained("setfit_early_stopping")
model.predict([text])
I am getting error of "This LogisticRegression instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator."

Hello!
Can you check whether the "setfit_early_stopping" directory has a "model_head.pkl" file?
Perhaps it wasn't correctly saved, meaning that it just loaded a non-fitted LogisticRegression head instead.

Tom Aarsen

yes, it has model_head.pkl file

Apologies for my late response. That is odd behaviour, let me try and reproduce this.
model.save_pretrained("setfit_early_stopping") is indeed correct, and so is model = SetFitModel.from_pretrained("setfit_early_stopping"). At a glance it seems like you did everything correctly.

Tom Aarsen

I'm struggling to reproduce this using the v1.0.0-pre branch. Perhaps I unknowingly resolved the issue in that branch. If you want, you can try to provide a more complete snippet to reproduce the issue.

Tom Aarsen

I also found out this problem! And even without early-stop, just model.from_pretrained("Path_to_my_local_checkpoint") and then used model.predict. I'm sure that checkpoint is correct and complete.

I'm running into the same issue as @syx1031 ! No early stopping, did everything according to the example but get a NotFittedError: This LogisticRegression instance is not fitted yet error with model.predict

Ran into this today. Storing and loading the model head explicitly fixes this.
To store the model head after training
pickle.dump(model.model_head, open(modelHeadPath, 'wb'))
To load the saved model head
with open(modelHeadPath, 'rb') as mhf:
    inferenceModel.model_head = pickle.load(mhf)

Extract embeddings from the SetFitModel's body
train_embeddings = model.model_body.encode([data['text'] for data in train_dataset])
Fit the LogisticRegression head
model.model_head.fit(train_embeddings, [data['label'] for data in train_dataset])
This code worked for me.

Ran into this as well on SetFit 1.1.0. Even though model_head.pkl is present after running SetFitModel.from_pretrained()


imports: add handling of whitelists

This adds whitelist/blacklist handling. I'm not sure about the approach to std; at the moment it adds std to the whitelist if there is one. The two other options are to have pseudo-packages to specify inclusion or exclusion of std ("std" and "-std" respectively). I think requiring "std" to be added for whitelisting is probably too onerous, so that just leaves a "-std" for cases where std should be removed from the whitelist,... or just leaving it how it is. Thoughts?
Please take a look.
Closes #4.

I am ok with automatically adding "std" to the whitelist.
but couldn't we handle the "-std" as "std" being added to the blacklist?

couldn't we handle the "-std" as "std" being added to the blacklist?

That doesn't allow quite the same set of operations. If you have -w="-std,math,fmt", you can't express that easily by -b=..... The issue here is that the "-std" is preventing addition to the whitelist rather than blacklisting the std.

PTAL


fix(backfill): make tombstone iteration progress across all vnodes per epoch

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Thanks @hzxa21 for sharing his diagnosis and ideas to solve it.
Resolve #17267
This is required because if we only yield the progress of a single vnode, the tombstone iteration of the other vnodes are dropped.
This is not an issue in no shuffle backfill, because the current pos is maintained on a parallelism level for all vnodes in it, rather than on vnode level like arrangement backfill.
We cannot simply yield all the scanned records downstream however, otherwise in each epoch, we now have the invariant of 256 records per epoch for backfill. If the processing latency of 256 records is high, barrier latency will be high.
So what we do is to add an yielded flag. In the case where there's no snapshot read in an epoch, we will set yielded flag to false, and continue to bump the current progress. Since we don't yield the records which were read from the snapshot, in the subsequent epoch, we will scan these same records, but use an inclusive bound with them, so they will be included in the newly scanned snapshot.
For normal snapshot read, yielded will always be true.
Want to add a separate test later.

Checklist

[ ] I have written necessary rustdoc comments
[ ] I have added necessary unit tests and integration tests
[ ] I have added test labels as necessary. See details.
[ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
[ ] My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
[ ] All checks passed in ./risedev check (or alias, ./risedev c)
[ ] My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

[ ] My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

[ ] My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

https://github.com/risingwavelabs/risingwave/blob/d488f652ed9d0dbea75a14245b664e0a392d9547/src/stream/src/executor/backfill/arrangement_backfill.rs#L730-L731
Can we try to provide an option to allow arrangement backfill snapshot read to read in serial?

I tried it before, but performance is worse. We can provide it as a session variable I suppose.
In this case, I think snapshot read in serial may worsen the problem.
Because previously, we can do tombstone iter concurrently, but now we only do tombstone iter in serial across all vnodes.

https://github.com/risingwavelabs/risingwave/blob/d488f652ed9d0dbea75a14245b664e0a392d9547/src/stream/src/executor/backfill/arrangement_backfill.rs#L730-L731
Can we try to provide an option to allow arrangement backfill snapshot read to read in serial?

I tried it before, but performance is worse. We can provide it as a session variable I suppose.
In this case, I think snapshot read in serial may worsen the problem. Because previously, we can do tombstone iter concurrently (just missing a "merge" step), but now we only do tombstone iter in serial across all vnodes.

IIUC, in this tombstone case, the no-shuffle backfill snapshot read is aligned with tombstones of all vnodes so well, hence, it can make progress in parallel which is faster than snapshot read per vnode sequentially.

https://github.com/risingwavelabs/risingwave/blob/d488f652ed9d0dbea75a14245b664e0a392d9547/src/stream/src/executor/backfill/arrangement_backfill.rs#L730-L731
Can we try to provide an option to allow arrangement backfill snapshot read to read in serial?

I tried it before, but performance is worse. We can provide it as a session variable I suppose.
In this case, I think snapshot read in serial may worsen the problem. Because previously, we can do tombstone iter concurrently (just missing a "merge" step), but now we only do tombstone iter in serial across all vnodes.

IIUC, in this tombstone case, the no-shuffle backfill snapshot read is aligned with tombstones of all vnodes so well, hence, it can make progress in parallel which is faster than snapshot read per vnode sequentially.

Yes exactly. So we should keep snapshot read in parallel for arrangement backfill as well. The only difference will be that no shuffle backfill maintains a current pos across all vnodes, so the iteration progress won't be lost. So we adapt that logic to this PR as well. Ensure per epoch, the current pos across all nodes bumps up.

When you mention iter in serial, you mean run each iterator sequentially right? Rather than concurrently. Or do you mean merge sort like no shuffle?

When you mention iter in serial, you mean run each iterator sequentially right? Rather than concurrently. Or do you mean iter concurrently but do a merge sort like we do in no shuffle backfill? And maintain a current pos per partition, rather than per vnode.

I mean run each iterator sequentially, however, in this case, it would be 256 times slower than the no-shuffle backfill concurrent one.

It has passed the tombstone test case now. But it's tombstone iteration is still worse than no shuffle backfill.
8m20s vs 6m9s.

In main-cron the runtime for arrangement backfil tombstone scan test is around 6m as well, so there's a regression: https://buildkite.com/risingwavelabs/main-cron/builds/2644#019021c8-381b-41ba-90df-e7b66cd77834

After optimizing, now it takes 2m35s.


Add translations user feedback

Add translations for the User feedback component
Today we have hardcode the texts/descriptions of the User feedback component. We are moving those texts/descriptions to Strapi Translations
Proposal

Move texts/descriptions to Strapi Translation
Create a context with the feedback translations
Use hook to get the feedback translations without pass them via props

Out of scope
Later we will have this component also in Strapi, so using the context and hook it would be easier to make the component following the Product and UX requirements

I thought we agreed to make it a "classic" strapi page component that contains its own text. did the requirement change?

I thought we agreed to make it a "classic" strapi page component that contains its own text. did the requirement change?

Only the first question of the feedback can be changed via the page component, the other texts it should be translated, because it'll be same text. And it's a ticket requirement


Do a two step checkout to avoid checking out connections before use

I think this is the right fix. I'll merge so you dont have to rebase :D.


feat: add minor argument

I've added a new argument to bump minor version instead of patch.
I wanted to test the feature mocking github requests to have control on the test so I wrapped github dependency with an interface in order to test this feature.
These solution decouples the app code from the github library applying clean architecture practices, allowing the implementation of other repositories as BitBucket or GitLab in the future.

/ok-to-test

The test are failing:
go test -v
# github.com/jenkins-x/**-release-version [github.com/jenkins-x/**-release-version.test]
./main_test.go:94:42: too many arguments in call to adapters.NewGitHubClient
	have (string, string, bool)
	want (bool)
FAIL	github.com/jenkins-x/**-release-version [build failed]
make: *** [test] Error 2

/lgtm

/approve


Allow get_parameter usage on TemplateRef steps/tasks

Is your feature request related to a problem? Please describe.
When using a template ref in a DAG/Steps, you cannot access its outputs via get_parameter, because Hera tries to confirm the parameter exists, which is not currently possible for TemplateRefs.
Describe the solution you'd like
Allow TemplateRefs to use get_parameter syntax without validation (for now).
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Using the Argo templating syntax - {{tasks.prev-step.outputs.parameters.param-name}}
Additional context
Add any other context or screenshots about the feature request here.

This is probably unneeded/stale with the new HEP.


Sho detail option does not work when a song is being played

There is an option of show detail when you click on the menu option on a currently played song.  No details are shown.
https://youtu.be/HQ0ZViru604

Are you downloading the latest version?

Yes

https://github.com/AugustToko/ACG-Player/releases/tag/<IP_ADDRESS>(20190601)


notepad++ plugin doesn't handle split-window session properly

If you've got notepad++ split into two panes, and the a document in the right-hand pane has focus, Change Format (for example) operates on whatever document is in the left-hand pane; alt-tabbing back to notepad++, the right-side document still has the input focus, so it doesn't seem to be a case of losing the input focus by going to the FHIR toolbox screen.

hmm ok, I'll have to figure out how to track the correct pane/focus


nodered/node-red:latest-debian pulls old version 3.1.12

Current Behavior
Tried to upgrade from v4.0.2 to v4.0.3 and now things are in an inconsistent state. Here's my update script:
#!/bin/bash
reset

image="nodered/node-red:latest-debian"

echo -e "Compose down:"
sudo docker-compose down

echo -e "\nRemove $image image:"
sudo docker rmi $image

echo -e "Compose up:"
sudo docker-compose up -d

Expected Behavior
I expected to be upgraded to 4.0.3
Steps To Reproduce
Run the script above or simply: docker pull nodered/node-red:latest-debian
Example flow
No response
Environment

Node-RED version: 4.0.2 (now 3.1.12)
Node.js version: 17.5.0
npm version: 8.10.0
Platform/OS: Ubuntu 22.04.5 LTS
Browser: Chrome Version 128.0.6613.137 (Official Build) (64-bit)

Thanks, I thought we'd caught all of those last time.
I'll respin the 4.0.3 build which will fix that and then look at the build scripts again

closed by https://github.com/node-red/node-red-docker/pull/454

tags should be right now


Cannot add points to Scatter

"Adding_points_to_Scatter" cell in Scatter plot does not work for me. No new points are added by clicking.

@ssunkara1 @SylvainCorlay This seems to be a call to the deprecated enable_add.  Move and delete remain under the old enable framework, but add has been moved to the new way of processing interactions.
@daviddaileo I will open a PR for this. However, if a quick fix is needed, change

scatter2.enable_add = True

to
scatter2.interactions = {'click': 'add'}

And set it to None to remove the add ability.
scatter2.interactions = {'click': None}


Interpreting Genotype Likelihoods of Complex Structural Variants

We have performed WGS at 40x of a number of families and processed structural variants through the speedseq v0.0.3a pipeline.
I was wondering how to interpret genotype-likelihoods (GL) / genotype-quality (GQ) scores, in particular for more complex structural variants than simple deletions, duplications, and inversions?
We identified a few de novo complex structural variants with very poor GL / GQ scores (e.g. 0 GQ), however we went ahead and successfully validated them through PCR.
At the other end of the scale, we have breakpoints that are called in almost everyone and have 0 genotype quality scores, however a few individuals may have GQ scores > 50. These are probably erroneous.
Also, we have complex events with for example two pairs of breakpoints, where one pair has a 0 GQ score, and the other has a reasonable GQ score (>50, n.b. I grouped breakpoints into single events based on their proximity and overlap, e.g. two pairs of breakpoints overlap and one end of each pair are adjacent to each other in opposing orientations).
I suspect it is very challenging to accurately estimate GLs in complex events that have multiple signals at the same locus.
My temptation is to keep anything denovo regardless of GQ score if it involves >1 pair of breakpoints, since the numbers are manageable. I would also keep anything else with >1 pair of breakpoints where at least one pair of breakpoints has a good median GQ score (>50 or >100?) across individuals that have this pair of breakpoints in the cohort. Otherwise filter any events with consistently low or zero GQ scores. Please advise!

Thanks for the feedback on validation of these events with PCR.
First, we have made improvements since the speedseq v0.0.3a pipeline to SVTyper. So even if you're using SpeedSeq for data processing, I'd recommend running it through the more recent SVTyper on this github page.
GQ and GL scores are probably not the best metric for evaluating variant quality. Both of these describe the relative likelihood that the genotype is correct compared to other possible genotypes (0/0, 0/1, 1/1). Thus, a variant may have a poor GQ even though it is almost certainly non-reference because SVTyper is unsure whether it is 0/1 or 1/1.
On a per-sample basis, the "sample quality" (SQ) shows phred quality that the site is non-reference in that particular sample. (The QUAL score, column 6 of the VCF, is the analogous value that the site is non-reference in ANY sample in the VCF.) SQ is the best metric for distinguish reference from non-reference sites.
Even SQ may have some problems at complex events however, as these can violate the assumptions of diploidy in the model. If you're particularly interested in these, then you might want to try the current dev branch version of SVTyper, which outputs the "allele balance" (AB) (alternate/total observations) at each site. This allows more sophisticated filtering on raw counts and is agnostic to ploidy and genotyping model.

Awesome, thanks.
I will use SQ from now on. In the future I will also use a more updated version of speedseq and SVtyper.

Hi Colby,
I reran the genotyping using v0.0.2 of SVtyper. The results look much better, and the complex de novo events genotyped well with high SQ scores.
I am genotyping individuals within families. I’d like to combine the results for each family into one file and compute a quality score for each SV locus. I was wondering how this quality score (column 6 in the VCF) is calculated? Could I generate it myself in the combined data without genotyping everyone in our cohort for every variant with SVtyper?

QUAL is -10 * log(P(locus is reference is all samples)), which is equal to the sum of the SQ scores.
If you have SQ scores for each individual at the locus, you can just add them together to get the QUAL score of the variant.

Hi Colby,
We are using Speedseq 0.1.2 to process the whole genome For ex: NA12877. We are getting many calls with QUAL = 0. Can we just filter them from the vcf file after processing through speedseq workflow? What does the QUAl score or zero or near zero means?
I have seen for speedseq var/ somatic call you have an option to filter the output by QUAL score using -q? Is it applicable in Speedseq SV pipeline?
Thank you,
Sithara

Hi Sithara,
The QUAL score is meaningful for variants that have been genotyped with SVTyper (which can be done as an option through SpeedSeq). We have used varying thresholds for QUAL in publications (≥100 in the SpeedSeq paper and ≥20 in the GTEx paper), and it is generally appropriate to filter low quality variants from the VCF file. In general, low quality SVs reflect poor alignments to the reference genome. These may be interpreted by LUMPY as non-reference events (which is why they are included in the VCF), but more sensitive inspection by SVTyper finds no convincing evidence of a true SV
Hope that helps,
Colby

Hi Colby, Thank you for the detailed response. We decided to go with SVs with QUAl score of >100. Currently I am doing a validation of genome NA12877 sequenced 100X Coverage, with down sampled genome of 30X and 10X and looking at deletions. We are filtering out SVs based on AB (Allele Balance from VCF file), defined heterozygous deletions based on AB of <= 0.3 and <= 0.7 and homozygous deletions with AB > 0.7, considering AB instead of VAF (Variant Allele Frequency). Have you done any validation of Speedseq at low coverage samples? We were expecting that at low coverage we will miss to see small SVs. But considering the combination method for SV call, is this assumption true for Speedseq?
Thank you,
Sithara

Hi Sithara,
That strategy sounds reasonable to me. We have not assessed SVTyper performance by coverage depth. But I agree that lower coverage will decrease call sensitivity, and this may be more pronounced for small deletions. I'm assuming by "combination method" you're referring to the read-depth + SVTyper analysis, and this expectation will remain true under that model as well.
--Colby

Hi Colby,
I did a validation on just deletions at 100X, 30X and 10X, our assumption
of lower coverage will reduce the call sensitivity is not true with my
analysis. By combination method I meant to say Speedseq uses read depth,
split read, read pair and SVtyper breakpoint detection - which could be the
reason that we didn't see much reduction in call sensitivity at reduced
coverage.
Thank you,
Sithara
On Tue, Aug 21, 2018 at 1:32 PM, Colby Chiang<EMAIL_ADDRESS>wrote:

Hi Sithara,
That strategy sounds reasonable to me. We have not assessed SVTyper
performance by coverage depth. But I agree that lower coverage will
decrease call sensitivity, and this may be more pronounced for small
deletions. I'm assuming by "combination method" you're referring to the
read-depth + SVTyper analysis, and this expectation will remain true under
that model as well.
--Colby
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/hall-lab/svtyper/issues/10#issuecomment-414776888,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AfYfyx-SgtApsBRCRbAcnwQ-J2I8VjMQks5uTFIrgaJpZM4E1fwL
.

--
With regards,
Sithara Vivek,
Mobile -<PHONE_NUMBER>


Conflict between dev and prod

Hi!
I'm trying to make sense on the different containers built here and the relation between them. It seems like there is a conflict between the database and web for the two environments? If I build the prod compose first, then that db will be prevalent. So if I then try to bring up the development stack before the production stack, then the database will be "stuck" as the development one. There doesn't seem like these two stacks can co-exist?

No, they aren't mean to co-exist. If you want them to, change the name of the volume for one of them. postgres_data_prod, for example.


Maven central

Any plans on pushing this to Maven central instead of jcenter?

no plans

+1 to the question. It's easy and free to do, but will solve problems for developers.
And one more question: are you planning release any time soon?

I'm not entirely clear on the issue, its quite easy to add jcenter, one of the largest repos as well to any build system.
There are no releases planned anytime soon as there are no outstanding PRs etc.


Implementing logitudinal data

Description
Issue: STEAM-1092
This PR implements longitudinal data to be used in the associated section. Now each file object keeps track of the big six coverage percentages in its recoil state so they can be properly displayed on the line chart in the longitudinal data section.
This PR does not implement proper tracking of trends (the little arrows next to Current and Monthly Average), I think that can be broken off into its own task.
Important Changes
FileUpload.js

Files now compute the main coverage stats on load, so they can be used later in longitudinal data

LineChart.js

Axis, formatting, and tooltip changes to better display longitudinal data

LongitudinalSection.js

Removing dummy data and adding parsing for real longitudinal data

DefaultUploadedFiles.js

Spreading out dates, to better show longitudinal-ness, also adding coverage stats

App.js

Passing necessary data to Longitudinal section

Testing Recommendations

See that the longitudinal data section now displays proper line charts for each section
Add a new file and see that the line charts are properly updated

Checklists
Submitter:

[ ] This PR describes why these changes were made.
[ ] This PR is into the correct branch.
[ ] This PR includes the correct JIRA issue reference.
[ ] Code diff has been reviewed (it does not contain: additional white space, not applicable code changes, debug statements, etc.)

@ACCT1 :

[ ] Code is maintainable and reusable, reuses existing code and infrastructure where appropriate, and accomplishes the task’s purpose
[ ] You have tried to break the code
[ ] Tested all recommendations listed in the "Testing Recommendations" section. The application behaves as expected with this PR.

Nice this looks great! The only thought I have is to eventually modify the bundles to better represent a real world example instead of just dropping to 0% after bundle 1, but we can wrap that in with the metrics task you refrenced in the PR.


Question about fusion pretrain?

Hi, thank you for your amazing work.
As indicated in your project config files, the weights for finetuning fusion-based model are got by "merging the weights of uvtr_lidar and uvtr_cam". So, from that statement, uvtr_lidar and uvtr_cam (pretraining phase) are trained individually before, and you just combine state_dicts of those 2 pretrained models to use for finetuning fusion-based model.
But when I first read your paper, I thought the fusion weights are got by training both lidar and camera branches "simultaneously" (as indicated in your framework), which is quite different from the above implementation.
This makes me confused and I want to ask you if I am understanding your ideas correctly.
Is "camera-based pretrained weights + lidar-based pretrained weights = fusion-based pretrained weights" what you mean in your paper too?
Thank you so much in advance.

Hi, thanks for your interest.
The official training pipeline of uvtr-m is: scratch uvtr-c, scratch uvtr-l -> finetune uvtr-m, where the uvtr-m uses trained weights of scratch uvtr-c and scratch uvtr-l for initialization.
For our pre-training setting, the pipeline is: pretrain uvtr-c, pretrain uvtr-l -> finetune uvtr-c, finetune uvtr-l -> finetune uvtr-m, where the uvtr-m also uses trained weights of finetune uvtr-c and finetune uvtr-l for initialization.
There indeed exist other pretraining pipelines: pretrain uvtr-m -> finetune uvtr-m or pretrain uvtr-m -> finetune uvtr-c, finetune uvtr-l -> finetune uvtr-m, but it would cost more training memory and computational costs, thus we do not carefully verify these pipelines (these pipelines can be easily implemented by directly merging the config of uvtr_lidar_vs0.075_pretrain.py and uvtr_cam_vs0.075_pretrain.py).

Thank you for your response. You are right, pretraining uvtr-m is quite expensive.
May I have one last question? Given your framework, do you think we would finally get similar performance using [pretrain uvtr-m -> finetune uvtr-m] pipeline instead of your current setting [pretrain uvtr-c, pretrain uvtr-l -> finetune uvtr-c, finetune uvtr-l -> finetune uvtr-m]?

For the baseline of uvtr model, scratch uvtr-c, scratch uvtr-l -> finetune uvtr-m performs better than scratch uvtr-m, so I think that
pretrain uvtr-c, pretrain uvtr-l -> finetune uvtr-c, finetune uvtr-l -> finetune uvtr-m would also run better than pretrain uvtr-m -> finetune uvtr-m.

Okay. I got that. Thank you again for your explanation. Have a nice day ~


Make compatible with purescript 0.14.2

The old unary # Type syntax for row kinds is deprecated and should instead be written as Row Type. The kind of RowList has changed from Type to Type -> Type

https://github.com/purescript/documentation/blob/master/migration-guides/0.14-Migration-Guide.md#polykinds--type--type

Thanks for the PR. I was actually going to update it to 0.14.2 somewhere this week, but you beat me to it 😉
LGTM!

Published https://pursuit.purescript.org/packages/purescript-classnames/1.0.0

Cool, thank you


ipam; introduce garbage collection logic

Description
Introduced garbage collection logic in IPAM to clean up IP records not updated for long time and hence considered not in use anymore by any NSM connections.
For this the database model has been extended with UpdatedAt and CreatedAt timestamp fields. UpdatedAt is used by newly introduced garbage collector logic along with a threshold time to determine if a record can be removed. The check is run periodically. Both the threshold and periodicity interval are configurable through environment variables (IPAM_GARBAGE_COLLECTION_THRESHOLD and IPAM_GARBAGE_COLLECTION_INTERVAL).
CreatedAt merely serves as a hint to show when a particular entry was created.
As part of the changes node allocations do update existing database records to keep UpdatedAt values up to date.
The logic takes only records that belong to an NSM connection into consideration. This is achieved by means of applying a filter matching names with "-src" or "-dst" suffixes denoting NSM connection related entries. (For other allocations the IPAM is not consulted periodically.)
A one time cleanup has been also added to allow removal of records created in accordance with the old model, yet have not been updated for a while. This check is triggered after the first periodicity interval elapsed.
The model update re-uses the old database table extending it with UpdatedAt and CreatedAt columns. In case of rollback the excess columns are simply ignored by the "framework" without causing any problem.
When moving from a version using the old model to the new model, then automigrate won't set UpdatedAt and CreatedAt values for existing records. UpdatedAt is going to be set once the record is Updated as part of the node allocation. (CreatedAt is left blank for obvious reasons.)
Notes:
IPAM_GARBAGE_COLLECTION_INTERVAL also determines when the garbage collector will be first invoked. Thus, setting it to a low value must be avoided to allow existing NSM connections to get their IP allocations refreshed as part of the recurring NSM connection refresh which is tied to NSM's MaxTokenLifetime.
IPAM_GARBAGE_COLLECTION_THRESHOLD must be higher than NSM's MaxTokenLifetime.
Issue link
#527
Checklist

Purpose

[ ] Bug fix
[x] New functionality
[ ] Documentation
[ ] Refactoring
[ ] CI

Test

[ ] Unit test
[ ] E2E Test
[x] Tested manually

Introduce a breaking change

[ ] Yes (description required)
[x] No

I don't really like relying on the src and dst prefixes as nothing in the API is saying about the usage of those. I guess if a node or a trench have their name finishing by -src or -dst, problem could happen.

I do agree that the current approach is somewhat ugly and can be problematic.
My goal was to come up with a match criteria that does not require altering any of the existing fields, and could work also for entries inserted by older models. But probably the risk is way too high of removing other records.
Btw in case of trenches names, fortunately there could be no such problem as accidentally matching them, since ipam server already modifies their name based on the IP family type (adding either -0 or -1 to the trench name). Otherwise, for workers and conduits it would be definitely a problem if they would end with either -src or -dst as you pointed out.
some ideas how to move forward:
-Change IP allocation for other "IP resource types" such as trenches/conduits etc. to be recurring and hence updates could be utilized. (Therefore postfix matching could be completely ignored, and GC could be more generic.)
This might introduce a lot of changes and might be tricky to implement (haven't really looked into it). I guess it would be the best, if these updates could be done as part of ipamServer Allocate() (existence of all these are checked anyways here, but an idle system without NSM connections could be a problem).
-Add a new table that would indicate if an entry is "expirable", and GC would only consider those. In which case GC would not be able to clean-up leaked entries inserted according to old model.
Passing the information about an entry if its expirable could involve a lot of changes (inlcluding the API). Unless for example that info would be stored in the context (which normally is not a recommended usage of context yet happens often e.g. in NSM code).
-Trenches have no parents. Thefore use this knowledge to identify threnches, then fetch entries with trench parents thus getting the conduits. Then, find the conduit workers as those have conduits as parents. After that, we should have a list of trenches,conduits,conduit-workers, which could be used by GC to ignore records happaned to match anything in our list.
This obsiously exploits the hierarchy of entries, but it could be used to remove old leaked records created based on the old model.
Combining this with the "expirable" field would mean, that only the one-time GC should do all this "lookups" once.
-Rely on network prefix length. Ugly, because the storage doesn't know anything about such. But it would be an easy check for the GC to only remove entries related to /128 or /32 prefixes.
(Proxy bridges also have /32 and /128 but their name is hardcoded to bridge...)


Is there a way to set up multiple remotes on one ovpn client config file?

How to set up multiple remotes on one client configuration file so that when one server crashes, it can automatically switch to the next one? Anyone with experience, please point me to what I need to learn. Thanks!

there are mutiple ways. One way is to setup a jumping server. You would set your DNS pointing to it and then it watches which vpn server is alive and redirect you to it.
The jumping server can be a monitor system setting the correct DNS records for example for the VPN server that the client connects to.

You need to learn load balancing algorithms


Need of code of conduct.md file

Kindly assign it me .
The repository looks incomplete because of the missing Code of Conduct.md file to specify the rules and regulations to follow while  raising a issue to the repository, I want to add that file to the repo.
Proposal
Add Code of Conduct.md  file to the repository with guidelines.

@Clooos Kindly look into my issue

We don't really need this for now but thanks for the suggestion.


nom ci & cross-env

cross-env version: 5.1.3
node version: v9.6.1
npm version: 5.7.1

What you did:
Installed packages using npm ci command.
What happened:
➜  foo git:(master) ✗ npm ci ; npm run dev
npm WARN init removing existing node_modules/ before installation
added 1664 packages in 18.24s init removing existing node_modules/ before installation

> @ dev /foo
> npm run development

> @ development /foo
> cross-env NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js

sh: cross-env: command not found
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR! @ development: `cross-env NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the @ development script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR!      /Users/vyuldashev/.npm/_logs/2018-02-27T21_52_13_531Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ dev: `npm run development`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @ dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Problem description:
cross-env could not be found.

Hi @vyuldashev,
I'm afraid, that's not going to be a problem with cross-env. I'm not sure how npm ci works, but I'm 99.9% certain there's nothing we can do in cross-env to fix your problem. Sorry. I'm sure folks would appreciate it if you post the solution you find when you work it out. Good luck!


add row level policy in Nereids

Search before asking
I had searched in the issues and found no similar issues.
Description
The following document and code detail row policy
document: ROW POLICY
code: https://github.com/apache/doris/pull/9786
Currently nereids not support row policy.
Subtasks:

[x] 1. add new logical plan before logical relation in LogicalPlanBuilder.
[x] 2. add new rule that will turn new plan to logical filter in NereidsAnalyzer.
[x] 3. find row policies and merge them into a predicate for logical filter.


Return 401 if authentication is enabled and there are no users

The discussion from #2172 brought to my attention that if you have authentication enabled, but no users created, you're always authenticated.
Instead it should only allow the creation of a user, which would be given the root privilege by default.
If the request is to do anything other than create a user, it should return a 401 and specify: no users exist. create a user first or disable authentication.

I think the best spot to check this is here:
https://github.com/influxdb/influxdb/blob/master/server.go#L3719
I'm also wondering if this even makes sense then:
https://github.com/influxdb/influxdb/blob/master/httpd/handler.go#L886
If we are going to delegate all authentication to the database, the http endpoint should probably just stay out of it and only pass in the username/password if present, and not do any validation past that.  In effect, allow the server to determine what is allowable, not the http endpoint.
If we do that, then we have to somehow wrap in auth to this endpoint which assumes someone upstream has already done it:
https://github.com/influxdb/influxdb/blob/master/httpd/handler.go#L455


How can i configure the linter-flake8 on windows10？

I have already installed flake8 with Anaconda,and i also put the Script floder in $PATH,but it just cant't show the erro icon when i code the erro code,so please tell how to configure this plugin on Windows.

@DLI1996 You are hitting https://github.com/PyCQA/pycodestyle/issues/728.
@lockbro I'm going to close this since you haven't responded to the above questions. If you are still having issues feel free to update this, thanks!


Correct states class of several entities

HASS long-term statistics require sensors to have a state_class argument.
Developer docs is here and says (about the TOTAL class)

This state class should not be used for sensors where the absolute value is interesting instead of the accumulated growth or decline, for example remaining battery capacity or CPU load; in such cases state class SensorStateClass.MEASUREMENT should be used instead.

Nearly all the sensors were set as "TOTAL". Some such as heart rates, respiration rates and percentage statistics are better suited to be treated as point-in-time "measurement"

Hello
Is there anything I can do to help progress this and #204 ?
I would love to be able to run your integration rather than maintain a homebrew on the side


Chinese Number Pronunciation

Checks

[X] This template is only for question, not feature requests or bug reports.
[X] I have thoroughly reviewed the project documentation and read the related paper(s).
[X] I have searched for existing issues, including closed ones, no similar questions.
[X] I confirm that I am using English to submit this report in order to facilitate communication.

Question details
In a Chinese sentence, all numbers are pronounced in English.
Can it be arranged so that the entire output is in Chinese instead?

#138 #244 etc.
Turn Chinese number to words.


Update rust_crypto to support AssociatedOid

Would you mind showing some example code that requires this? We should also add a test to show it working properly.

@dobbobalina2 This is a great start! Thank you. Could you try and add a test that uses RSA along with our Sha256 impl? I have a feeling that we're going to need deeper changes but it's a little unclear to me. The current changes fail to compile.

@nategraf @mothran Any ideas here?

See #726 for my attempt to solve this

@dobbobalina2 my recommended solution to this is to use our fork of the sha2 crate. It lives in the risc0/RustCrypto-hashes repo. Adding the following line to your Cargo.toml either as a direct dependency (replacing sha2) or a patch should get full compatibility with RustCrypto.
sha2 = { git = "https://github.com/risc0/RustCrypto-hashes", tag = "sha2/v0.10.6-risc0" }

The sha example also includes this patch.


Variable names for expect pty objects

Change variable names to help code readability
and maintainability.
Signed-off-by: Deb McLemore<EMAIL_ADDRESS>

@stewart-ibm @pridhiviraj - I was doing some cleanup, I think this PR is rebased now, can this be picked up ?


All matching(N) do not use choosing filters

Problem: if you apply any filters for resources and choose All matching(N) in result you see Action window for all resources without filters.
https://github.com/tunezilla/nova-dynamic-action-fields/blob/dff9798e2ed13499f09ccb5d50b083b8089ab0d3/src/DynamicFieldAction.php#L28 ignore filters

Hello,
Can you please submit code to reproduce this issue?
Thanks!


Missing features/tiles problem

Problem: The exact problem is just because when Omar tried to use his new e2e pipeline to run the same targets but observed different results. And he found with his e2e he got more features after tesselation and normalization.
Approach: Going to use the old pipeline to work on a small dataset. And reconstruct the normalized tiles back to WSI. By comparing the results with Omar's e2e pipeline to check why something got rejected
Code snippet related to reconstruct back to WSI:
https://github.com/KatherLab/end2end-WSI-preprocessing/blob/d160a882d905e496b98a52f9cb75595b7c23040f/stainNorm_Macenko.py#L142
Checklist:

[ ] Whether running twice the old pipeline output different/arbitrary results
[ ] Deposit reconstruction func to wanshi-utils
[ ] What part of the code is going wrong, for now everywhere could go wrong

Reason for this issue: in the old preprocessing pipeline more tiles got rejected because of the brightness threshold. Follows up by the canny rejection which only rejects a small part of the remaining tiles.
In e2e preprocessing part only canny rejection(edge < 2) takes action.
Examples of extra rejected tiles from old pipeline:

Canny slide from e2e:

reconstructed canny slide from old:

Result: old preprocessing pipeline rejects 20% more tiles than e2e. Causing e2e to have 20% more feature vectors for each patient. Resulting in better performance in model training and prediction.


[fix] Fix wrong behavior when removing the chunkedMessageCtx

Fixes #104
Motivation
Currently, the consumer ack the last message when the chunked messages exceed maxPendingChunkMessages. This is wrong behavior. This may lead to unexpected data loss.
This PR also fixes serval issues related to maxPendingChunkedMessages:
https://github.com/apache/pulsar-client-cpp/blob/1f7fdb86c409c1486d160528137f10ce07dcf3b2/lib/ConsumerImpl.cc#L387-L407
In the current logic, there are two putIfAbsent operations here, and they are confusing. If a new chunk message is received, it will be added to the chunkedMessageCache. But if the size of the cache reaches the maxPendingChunkedMessages, it will remove at least 1 ctx from the cache due to chunkedMessageCache_.size() - maxPendingChunkedMessage_ + 1. But the message is then put into the cache again. This can lead to unnecessary ctx buffer memory allocations.
Here are some key point of this issue:

Modifications

Fix consumer acked the wrong message when pending chunked messages exceed maxPendingChunkMessages
Fix wrong behavior when remove the ctx from the chunkedMessageCache.

Verifying this change

[ ] Make sure that the change passes the CI checks.

(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Documentation

[ ] doc-required
(Your PR needs to update docs and you will update later)

[x] doc-not-needed
(Please explain why)

[ ] doc
(Your PR contains doc changes)

[ ] doc-complete
(Docs have been already added)

In the current logic, there are two putIfAbsent operations here, and they are confusing

The original logic should have been like the following diff based on your fix:
diff --git a/lib/ConsumerImpl.cc b/lib/ConsumerImpl.cc
index 8658e08..f103144 100644
--- a/lib/ConsumerImpl.cc
+++ b/lib/ConsumerImpl.cc
@@ -392,11 +392,8 @@ Optional<SharedBuffer> ConsumerImpl::processMessageChunk(const SharedBuffer& pay
     auto it = chunkedMessageCache_.find(uuid);

     if (chunkId == 0) {
-        if (it == chunkedMessageCache_.end()) {
-            it = chunkedMessageCache_.putIfAbsent(
-                uuid, ChunkedMessageCtx{metadata.num_chunks_from_msg(), metadata.total_chunk_msg_size()});
-        }
-        if (maxPendingChunkedMessage_ > 0 && chunkedMessageCache_.size() > maxPendingChunkedMessage_) {
+        if (it == chunkedMessageCache_.end() && maxPendingChunkedMessage_ > 0 &&
+            chunkedMessageCache_.size() >= maxPendingChunkedMessage_) {
             chunkedMessageCache_.removeOldestValues(
                 chunkedMessageCache_.size() - maxPendingChunkedMessage_,
                 [this](const std::string& uuid, const ChunkedMessageCtx& ctx) {
@@ -404,7 +401,10 @@ Optional<SharedBuffer> ConsumerImpl::processMessageChunk(const SharedBuffer& pay
                         discardChunkMessages(uuid, msgId, autoAckOldestChunkedMessageOnQueueFull_);
                     }
                 });
-            it = chunkedMessageCache_.find(uuid);  // Need to reset the iterator after changing the cache.
+        }
+        if (it == chunkedMessageCache_.end()) {
+            it = chunkedMessageCache_.putIfAbsent(
+                uuid, ChunkedMessageCtx{metadata.num_chunks_from_msg(), metadata.total_chunk_msg_size()});
         }
     }

I intended to check the size first, then remove the oldest values if necessary. After the oldest values are removed, insert the chunk message context.
But I forgot to remove the first putIfAbsent.

@RobertIndie I think not. The patch calls it == chunkedMessageCache_.end() （before maxPendingChunkedMessage_ > 0) to check if the current chunk exists in the cache, just like putIfAbsent.

@RobertIndie I think not. The patch calls it == chunkedMessageCache_.end()

@BewareMyPower Thanks. I didn't notice that you changed here. This patch gives me some inspiration. I have pushed a commit to refactor this logic to make it more straightforward. PTAL again.


Terrain deformation

Hi, first of all want to thank you for this asset, it is definetly the best one I could find.
The issue I'm having is that I have an uneven terrain (which I made from a mesh with object2terrain script). So whenever i place a road the terrain doesn't rise with the road it seems like the asset considers not the surface of the terrain but the zero height point of it. Am I missing something? Also wherever I place a node it goes under the terrain so I have to raise them manually.
Can you please help me solve it?

Hello,
are you using the standard unity terrain? If yes, this should work. Maybe you need to manually assign a GSDTerrain to your Terrain in the Editor.
Best Regards
Fritzs

Try setting the position of the terrain to 0.0.0 and the road system also.


StorageExplorer.exe - Entry Point Not Found

Storage Explorer Version: v1.17.0
Build Number: v1.17.0
Platform/OS: Windows 10
Architecture:  x64
Regression From:
Bug Description
downloaded from: https://github.com/microsoft/AzureStorageExplorer/releases/download/v1.17.0/Windows_StorageExplorer.exe
Steps to Reproduce
download
install
[x] run after install checked
got error
---------------------------
StorageExplorer.exe - Entry Point Not Found
---------------------------
The procedure entry point av_buffer_create could not be located in the dynamic link library C:\Program Files (x86)\Microsoft Azure Storage Explorer\StorageExplorer.exe.
---------------------------
OK
---------------------------
Expected Experience
expect app to start

additional info, I also tried to install slack and get same problem,
so it is common issue for electron app, but Azure Data Studio is working
Also I did successfully install StorageExplorer on different computer

additional info, I also tried to install slack and get same problem,
so it is common issue for electron app, but Azure Data Studio is working
Also I did successfully install StorageExplorer on different computer

@roboter this isn't an error we haven't really seen before. Likely there is something messed up with your Windows install, related to a .dll that is responsible for this av_buffer_create procedure. We did a quick search for ways to fix it but couldn't find anything. At this point all we can recommend is you reinstall windows, but with additional time spent searching for soluations you might be able to fix this without having to do that.

@roboter this isn't an error we haven't really seen before. Likely there is something messed up with your Windows install, related to a .dll that is responsible for this av_buffer_create procedure. We did a quick search for ways to fix it but couldn't find anything. At this point all we can recommend is you reinstall windows, but with additional time spent searching for soluations you might be able to fix this without having to do that.

Closing due to lack of response. If you are continue to have issues please open a new issue. Thanks.


Cannot redeclare register_extended_post_type()

 Fatal error: Cannot redeclare register_extended_post_type() (previously declared in /srv/www/codelab.com/current/vendor/johnbillion/extended-cpts/functions.php:59) in /srv/www/codelab.com/current/vendor/soberwp/models/dist/johnbillion/extended-cpts/functions.php on line 69

This is a Models issue which was fixed a few days ago


grove: add relay switch support and examples

Added support for the Grove relay switch with corresponding examples
in C++, JavaScript, and Python.
Signed-off-by: Sarah Knepper<EMAIL_ADDRESS>

Combined with other pull request and merged into UPM.


Add Integration tests to be run in a remote cluster

The idea is to have an easy way to run tests in an existing cluster, that might be remote via make test-openshift-remote-cluster or directly via test/run-openshift-remote-cluster.

[test-openshift]


add U9 Benchmark Assembly model

add a model for ICSBEP IEU-MET-FAST-010 (IRPhEP ZPR-FUND-EXP-016)

Thanks @walshjon. The inputs don't work with the latest version of OpenMC but that's true of many others as well -- I'll work on fixing that later.


fix(plugin-legacy): empty base makes import fail (fixes #4212)

Description
fixes #4212
I tested with Chrome 53.
Additional context
Maybe '' could be replaced to './' in resolveConfig instead?

What is the purpose of this pull request?

[x] Bug fix
[ ] New Feature
[ ] Documentation update
[ ] Other

Before submitting the PR, please make sure you do the following

[x] Read the Contributing Guidelines.
[x] Read the Pull Request Guidelines and follow the Commit Convention.
[x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
[ ] Ideally, include relevant tests that fail without this PR but pass with it.

Maybe '' could be replaced to './' in resolveConfig instead?

Yes, this is an option. We'll need to discuss with the team if we want to deprecate both '' and './' in favor of 'auto', or if we decide for ''or'./'and deprecate with a warning or an error in v3 the other one. I would prefer to have a single option, and lately, I'm leaning towards'', false, or null`. I think the relative base not being a string is better to avoid future errors as it will force us to always check the base type.

I'm in favor of having a single option. null could be confusing with omitting the value. So I think false is better if we are going to using a type other than string.

this problem still not fixed in v5.4.3
I set my base option like base: /abc/def/ , and use plugin-legacy, it will lead to errors like:
Uncaught (in promise) Error: assets/index-legacy.d57e0ebc.js
finally i have to change the base to ./, finally solve the problem. But i have to use legacy files in all browser -- morden or old.
Can this problem be solved?
https://github.com/vitejs/vite/issues/4212

@60late please create a new issue with a minimal reproduction against vite latest version. Thanks!

@60late please create a new issue with a minimal reproduction against vite latest version. Thanks!

After some tests on this problem. I found it was solved when i use vite: 5.4.11. I think it may caused by error vite version.
For anyone who may need, upgrade your vite version will solve this problem


Question about code

I have a question about this code in DoGrep().
It masks almost all exceptions and makes it look like the search found nothing.
Was this the intent? It makes it very hard to spot problems that manifest as exceptions.
My guess is that the intent was to say something like if v:exception == 'E480'
    catch
        if v:exception != 'E480'
            call s:WarnNoMatches(a:pattern)
            try
                " go to the last error list on no matches
                if g:EasyGrepWindow == 0
                    silent colder
                else
                    silent lolder
                endif
            catch
            endtry
        else
            call EasyGrep#Error("FIXME: exception not caught ".v:exception)
        endif
        let failed = 1
    endtry

I don't really remember what the intent was there -- I'm guessing I just implemented a WAR (poorly) for an issue I was seeing at the time. Your fix is more robust than the previous code.


Fix flaky testgen

Fix #139

Tested on quite a few CI runs, this should be stable now:
https://github.com/andreaTP/chicory/actions/runs/7451456239


Fixed #652 - Added page sizes docs page

I am not sure which branches should I target exactly so I'll open two requests one for master and the other for gh-pages. Hopefully, this will fix the bug since it is related to website.

Thanks!


Cannot Spawn vehicle with attribute 'role_name' == 'hero'

Hello,
I'd like to generate a log file which contains the information about 100 vehicles. This is so I can repeat data collection from a variety of sensors across different weather conditions in the same environment to measure Localization Algorithm performance across different weather conditions.
In order to find my vehicle to attach sensors to, I'd like to spawn one vehicle with an attribute 'role_name' == 'hero' which can be easily identified in the actorList. Here's the code I'm using:
spawn_points = world.get_map().get_spawn_points()
batch = []
#Spawn 1 ego vehicle (vehicle.audi.tt)
hero_bp = blueprint_library.find('vehicle.audi.tt')
hero_transform = spawn_points[0]
hero_bp.set_attribute('role_name', 'hero')
if hero_bp.has_attribute('driver_id'):
driver_id = random.choice(blueprint.get_attribute('driver_id').recommended_values)
blueprint.set_attribute('driver_id', driver_id)
batch.append(SpawnActor(hero_bp, hero_transform).then(SetAutopilot(FutureActor, True)))
client.apply_batch_sync(batch)
time.sleep(1)
#Check I have created an egoVehicle...
egoFlag = "false"
actorList = world.get_actors()
for actor in actorList:
if(actor.attributes.get('role_name' == 'hero')):
	print("egoVehicle found!\n")
	egoFlag = "true"
	break

if(egoFlag == "false"):
print("Could not find egoVehicle even though I just spawned it!")

This test keeps failing with message "Could not find egoVehicle even though I just spawned it!" - do you have any insight into why?
My full source code is linked below:
https://github.com/Jack-Pinnington/Carla_Project/blob/master/generateFreeDrivingLog.py
Best regards,
Jack

hero_bp.set_attribute('role_name', 'hero')  here could be the problem, because hero_bp is a list (that may contain just one element), you need to use hero_bp[0].set_attribute('role_name', 'hero')  which is your car to set the role_name.

Issue found - sorry for the delay in response, has been fixed for a while.... It's the way I'm finding the egoVehicle that's causing the error - "if(actor.attributes.get('role_name' == 'hero')):" should read "if(actor.attributes["role_name"] == 'hero'):" since the attributes returns a dictionary. Think there was a code change between versions - just moved from 0.9.6 to 0.9.8.


AnalogRead

Does anyone know how to read in an analog value to c# from the Arduino UNO using the ArduinoDriver? I am trying to read in the value from a pressure sensor connected to the Arduino and store it as an int in my c# code. Thanks!

Hi pitt716,
Here is a small code sample that may help you (it should help illustrate the general concept of the library):
const int analogPinToRead = 4;
using (var driver = new ArduinoDriver(AttachedArduino, true))
{
    var analogReadResponse = driver.Send(new AnalogReadRequest(analogPinToRead));
    var readValue = analogReadResponse.PinValue;
}


tests: Bluetooth: add sleep in notify cb testcase

Validates that nothing nefarious happens if the application decides to wait an arbitrary amount of time in notification callbacks.
Also validates that a disconnection during said waiting works as expected.
E.g. in #48505, the user was trying to send data synchronously over UART inside the notification callback. Then the device was disconnected whilst still waiting.
This kind of pattern is not recommended, but the stack should still function correctly, stalling the "BT RX" thread shouldn't result in state corruption or resource leakage.

Could you check that this testcase reproduces the original bug?


[RFC] Apache MXNet 2.0 Roadmap

Overview
Status: https://github.com/apache/incubator-mxnet/projects/18
The status for each project will be updated by the contributor who's driving it. If you have more projects that you intend to drive please first discuss here.
The purpose of this RFC is to organize and present the roadmap towards 2.0. As 2.0 will be a major release, changes that would break backward compatibility are permissible.
The proposed changes in this RFC are either collected from past roadmap discussions such as #9686, or are based on various common issues from the past. This RFC organizes these changes into self-contained projects to facilitate clear definition of project, captures the risks and status quo to the best of our knowledge. To help navigate, the projects are further divided into several high-level areas. Some of the listed projects are already in progress, and are included to provide a clear overview.
The objectives of Apache MXNet 2.0 include:

Improve expressiveness and usability of user-facing API.
Improve expressiveness and usability of the technical stack for lower development cost and maintainability.

In terms of frontend, this roadmap focuses mostly on Python-frontend since MXNet has been taking a Python-first approach. The expectation with respect to other language bindings is that they would evolve along with the backend evolution and make use of the improvements. Given that breaking changes can occur, maintainers of different language bindings are expected to participate in related interface definition discussions.
1. MXNet NP Module
NumPy has long been established as the standard math library in Python, the most prevalent language for the deep learning community. With this library as the cornerstone, there are now the largest ecosystem and community for scientific computing. The popularity of NumPy comes from its flexibility and generality.
In #14253, the MXNet community reached consensus on moving towards a NumPy-compatible programing experience and committed to a major endeavor on providing NumPy compatible operators.
The primary goal of the projects below is to provide the equivalent usability and expressiveness of NumPy in MXNet to facilitate Deep Learning model development, which not only helps existing deep learning practitioners but also provides people in the existing NumPy community with a shortcut for getting started in Deep Learning. The efforts towards this goal would also help a secondary goal, which is to enable the existing NumPy ecosystem to utilize GPUs and accelerators to speed up large scale computation.
1.1. NumPy Operator Testing
Scope:

adopt array_function and numpy existing tests.
extend testing to GPU
investigate numpy testing strategies
decide correctness criteria for acceptance

1.2. NumPy Operator performance profiling
Scope:

Automatically profile the performance of NumPy operators

1.3. NumPy operator coverage
Scope:

improve operator until full NumPy coverage, with prioritization towards operators used in the ecosystem and deep learning in general

Operator coverage as of 07/03/2019
|    module |     NumPy | deepNumPy |       jax |      cupy |
|-----------|-----------|-----------|-----------|-----------|
|        np |       603 |        89 |       445 |       321 |
|   ndarray |        71 |        32 |        71 |        56 |
|    random |        63 |         5 |        15 |        49 |
|    linalg |        31 |         2 |         8 |        15 |

1.4. NumPy Extension Operator Reorganization and Renaming
Scope:

consistent type usage for index input and return values from sort, topk #11031 #11134, #12197
array creation operators with flexible dtype definition #12290. (dtype=None)
moving_mean/moving_var in batchnorm
consistent usage of axis vs dim
promote or deprecate contrib operators

1.5. NumPy ndarray type extension
Scope:

bfloat16 support (not in NumPy yet but useful for deep learning) (low priority — Intel)
boolean type support
complex (for FFT)

1.6. NumPy ndarray boolean indexing
Scope:

allow boolean masks in NumPy ndarray indexing by adding the operator, potentially through extending op.where

1.7. Hybridizable basic (and advanced) indexing
Scope:

Allow operations such as y = x[1:3, 2, ...] to be hybridizable

Note: Preliminary work: https://github.com/apache/incubator-mxnet/pull/15663
2. Graph Enhancement and 3rdparty support
The objective of the following projects is to enable easier development of third-party extensions without requiring changes to be checked in the MXNet project. Examples of such extensions include third-party operator library and accelerators.
2.1. Graph Partitioning for Dynamic Shape Operators
Scope:

partition inside control flow operators (and all cached ops)
partition on operators with dynamic shapes for partial memory planning and caching.

2.2. Improved Third-party Operator Support
Scope:

allow registering custom operators by exposing C API (and frontend API) to register NNVM op at runtime.
verify serialization, deserialization, and graph passes for graphs with these operators are working properly.

2.3. Improved Third-party Backend Support (subgraph property)
Scope:

expose a graph pass for standard graph partitioning with back-end-specific criteria as a C API and frontend API.

2.4. Large tensor support by default
Scope:

enable default support for tensor with int64 dimension sizes
make sure there’s no significant performance regression in operators

Risks:

performance regression may happen in a subset of operators, which can disproportionally affect certain models.
compatibility and silent behavior change.

Notes: in progress (RFC<EMAIL_ADDRESS>3. API Changes
The objective of the following projects is to address the technical debts accumulated during the development of MXNet 0.x and 1.x with respect to the API definition.
3.1. C-API Clean-up
C-API is the foundational API in MXNet that all language bindings depend on.
Scope:

use packed function for flexibility (and potentially efficiency through avoiding string parsing)
do not expose backend accelerator-specific types such as mkldnn::memory in C-API
do not rely on topological ordering for argument passing (#15362).
verification of thread-safety and performance for C API

Risks:

backend integration may require refactoring or even redesign
existing use cases such as other frontend may be broken without substitute
feedback is scattered and we may miss the opportunity to change some APIs in 2.0

3.2. Unify Executor
Scope:

SymbolBlock equivalent in C/C++, unify the executor implementation for symbol/module and the one for gluon blocks
migrate other versions of inference API
Support mirror option in the unified executor

3.3. Gradient of Gradient support
Scope:

higher order gradient support for a subset of operators

Risks:

large number of backward operators could introduce significant technical debt if not properly verified.
ill-informed prioritization may result in usability issue (e.g. common GAN not supported)

3.4. Autograd Extension
Scope:

improve interface to support specifying intermediate output grad nodes
improve interface for better usability. (retain_graph → something not involving graph)
update graph pass for correctness

3.5. NNVM-backend Operator Interface Changes
Scope:

support more than one temporary spaces
split forward shape/type inference and reverse shape/type inference for better error messaging.
deferred initialization removal (or improve error/info message)
accompanying operator implementation changes

Risks:

some changes may make operator implementation less error-prone while less flexible, and thus require some reworking.

4. Gluon 2.0
Since the introduction of the Gluon API, it has superceded other API for model development such as symbolic API and model API. Conceptually, Gluon is the first attempt in the deep learning community to unify the flexibility of imperative programming with the performance benefits of symbolic programming, through trace-based just-in-time compilation.
The objectives of the following projects are:

address usability issue as a result of the divergence in the behavior of NDArray and Symbol.
extend the JIT to improve the coverage of hybridization.
introduce new functionality to facilitate more areas of research such as Baysian methods and AutoML.
improve the usability and performance of the utility in Gluon.

4.1. Unifying symbolic and imperative mode for tensor library
Scope:

unify the operator implementation and behaviors of symbolic and imperative execution modes (#10875)
allow naming for ndarray similar to symbol
address the necessary changes in shape/type inference.

4.2. Unifying Block and HybridBlock
Scope:

move hybridization logic to a JIT decorator
extend parameter management to Block
user-friendly warning for native control flow in JIT code.

4.3. Gluon Block Enhancement
Scope:

inspection of graph internals similar to monitor for Module (PR 15839)
support additional types in argument such as dict, kwargs, None
fused parameters and gradients respectively
register custom parameter

4.4. Enable Symbolic Shape (& Dtype) for Array Creation in NNVM-backend
Scope:

allow flexible creation of array based on shapes of other arrays that are only known at runtime
add constant symbol type as the return value of symbol.shape (?)
support constant symbol as operator arguments (?)
constant folding for constant symbols

4.5. Gluon Distributions Module
Scope:

sampling and pdf definition for distributions. Distribution https://github.com/amzn/MXFusion. https://github.com/apache/incubator-mxnet/pull/14617.
wrap operators into more usable classes.
reproducible global seed

4.6. Gluon Metrics Module
Scope:

address usability and performance issues in mxnet.metric using hybridizable NumPy op

4.7. Gluon Optimizer Module
Scope:

API changes such as consistent weight decay (#9881), change default value to not apply wd on bias terms (#11953)
hybridizable optimizers
new optimizers (#9182)

4.8. Gluon Data API Extension and Fixes
Scope:

address diverging interfaces and remove transform= constructor arg (#11141).
reorganize io/image modules and provide data loader instead.
lowering dataloader to backend for efficiency (#13593)
shared memory propagation?

4.9. Gluon Estimator Extension for Experimenting Utilities
Scope:

logging of configuration (DeepNLU), state, and performance for checkpointing for easier resume
pre-defined estimators for common problems

4.10. Gluon Estimator Refactoring for Examples and Tutorials
Scope:

modularize and refactor unstructured scripts and examples into estimator class utilities

4.11. Gluon Distributed Training Usability Enhancement
Scope:

more flexibility for communication with kvstore UDFs
add distribution strategies to estimator
plugin for communication backends (horovod, byteps, parameter server) for data parallel training
data sharding/sampling/streaming enhancement for distributed training

5. Documentation
Documentation is the most important factor for new adoption of a library. The following projects aim to:

address the usability and discoverability issues in the current MXNet website
improve the quality of documentation to make it correct, clear, and concise.
help adoption of the changes in MXNet 2.0 from existing users.

5.1. MXNet 2.0 Migration Guide
Scope:

document high-level mapping from old functionality to new API for data pipeline, modeling, optimization, training loop, metric, inspection and logging, debugging.

Risks:

parallel development of the doc may result in outdated doc.
auto doc verification is needed.

5.2. MXNet 2.0 Developer Guide
Scope:

carefully document the design and contribution guide for features with low entry bar such as operator, gluon block, doc, optimizer, metric, examples and tutorials.
clear and up-to-date system design overview.
clear roadmap

5.3. Adopt beta.mxnet.io as official website
Scope:

infrastructure change for new doc build
merge into master with NumPy.mxnet.io
improve load time and browsing experience
CDN in popular region such as China, with automated validation and testing.

Note: https://github.com/ThomasDelteil/mxnet.io-v2
6. Profiling and Debugging
Profiling and debugging is a common step in the development of deep learning models, and proper tools can help significantly improve developer's productivity. The objective of these projects is to provide such tools to make it easier to discover issues in correctedness and performance of models.
6.1. Memory Profiler
Scope:

memory profiler logging support in backend
automatic array naming tool based on scope
tree-map visualization tool for inspecting profiler dump

6.2. Enhanced Debugging Tool
Scope:

Enable user-specified error handling
Improve error message
Stacktrace inspection in debug API
Automatic error reporting tool
Runtime API for turning off asynchronous execution

7. Advanced Operators
The objective of these projects are to extend the tensor library and operators for better performance and for advanced use.
7.1. Strided ndarray support
Scope:

support strided array in a subset of operators
support auto-transpose of strided array in graph pass and executor

7.2. Ragged ndarray and operators
Scope:

introduce ragged (variable length) tensor as 1st class tensor. Support zero-copy from RaggedNDArray to NDArray when no dimension is ragged.
Load balancing strategy for operators that take RaggedNDArray as input
cover operators for NLP applications (RNN, transformer)

7.3. Improved Sparse Support
Scope:

sparse format and operator support
scipy coverage
operators for graph neural-networks (e.g. ops in minigun)

Minimum support:

format: csr,
zerocopy to DLPack
integration with minigun kernels

Next-level support:

format: coo and block sparse.

8. Building and Configuration
8.1. CMake improvement and Makefile deprecation
Scope:

reimplement CMakeLists for DMLC dependencies
reimplement CMakeLists for MXNet to support 1) building best performing binary in any platform 2) building portable binary distribution for pip

8.2. MXNet Configurator
Scope:

drop environment variables and centralize them as config.
define functionalities that support runtime-switch (candidates: memory pool, engine, worker thread pools) and expose frontend API
allow saving and loading of mxnet system config

9. Advanced training and deployment
9.1. Automatic Quantization and Quantized Training for NumPy
Scope:

automatic quantization based on heuristic (or learning)
BMXNet

9.2. Mobile and edge-device deployment
Scope:

replace amalgamation with more user-friendly function (TF-lite equivalent).
tutorial and example
metal support

10. Performance
10.1. MXNet Execution Overhead
Scope:

https://github.com/apache/incubator-mxnet/issues/14883

@szha Really great proposal and we may want to add some items in 2.0 too.
Is there a timeline of 2.0?

Is there a plan to create a branch either for the 1.x version and have master reflect 2.0 or to create a branch for the 2.0 version and keep master on 1.x for now?

@pengzhao-intel a tentative target date is by end of Q1 2020.
@zachgk we will create a branch for 2.0. Initially we will keep master to be 1.x and have 2.0 in a new branch. After 1.6 release we will revisit how to make the 2.0 branch the master.

Just a quick cheer up for a new website of MXNet... its way more awesome and beautiful than I expected.
Though minor bugs are still there, for ex- most of the link in the tutorials are broken and not working.
Anyways great work so far.

Any plan to simplify the build of c and c++ api for mxnet2.0?It is hard(or very hard) to build a working version of mxnet with cpp api on different platforms(windows, linux, mac), every new release of the mxnet may or may not break somethings and we need to spend many hours to figure out how to make it work.

@stereomatchingkiss good point. What are you using c/c++ api for?

@stereomatchingkiss good point. What are you using c/c++ api for?

Develop stand alone app on desktop and mobile
Wrapper of another language(ex : php)
Run the inference task on aws lambda, we do not want to prune the libs of python manually if we could build a slim library of mxnet/tensorflow/pytorch.

Maybe you could open a post to ask the users what are they expect for c or c++ api, I guess most of them only need to use the api to perform inference task but not training(python do a great job about this), this should help you shrink the size of the libs and made the codes less complicated.

That's a bit what amalgamation part was for ? a simplified inference interface. The last time I use amalgamation (some years ago) it was often break by update and not really maintain.

The status of MXNet 2.0 project is tracked at: https://github.com/apache/incubator-mxnet/projects/18. The status for each project will be updated by the contributor who's driving it. If you have more projects that you intend to drive please first discuss here.

Once 1.6 release is complete, we will create a branch for MXNet 1.x for future releases and start using master branch for 2.0 development.

Should we create a new branch for 2.0? I think we are also planing for 1.7.0 https://github.com/apache/incubator-mxnet/issues/16864

In the past we always kept development on the master branch, thus how about branching out 1.7.0 release branch and keeping development on master?

+1 for using master branch for 2.0 development. I think we need 3 branches at least:

master branch: for 2.0 development
v1.x:  for 1.x development and maintenance
v1.7.x: for 1.7.x release

That's what I had in mind. The v1.7.x branch doesn't have to be created until code freeze for 1.7.0

3.1. C-API Clean-up
C-API is the foundational API in MXNet that all language bindings depend on.

@szha I'm looking at the item 3.1.2. Could you please explain the scope of C-API? Do you mean those APIs sit in the src/c_api/ folder?

@TaoLv one promising direction that the community is converging to is the interface based on packed function (motivation as described by @tqchen in https://github.com/apache/incubator-mxnet/issues/17097#issuecomment-567604444). What this means to the project is that the existing c API will be updated to follow the packed function interface.

Is there a plan to remove the cudnn_off argument from the neural network operators such as Dropout, Convolution, Pool etc. It creates a few usability issues:
(1) Once a model is exported. It requires users to change this flag in all the layers manually if they want to enable/disable cuDNN. When the cudnn_off is set to true in some layers, the global env variable MXNET_CUDNN_AUTOTUNE_DEFAULT becomes don't care. It's very confusing to users to see an error message like "Please turn off MXNET_CUDNN_AUTOTUNE_DEFAULT" by indeed it does not do anything.
(2) Why did we expose such implementation detail to users at the first place? In the worst case, we should just provide a global variable to turn on/off cuDNN in all layers instead of at operator level.

Thanks for this awesome work, it has benefited me a great deal.
Here are some disadvantages(may be) listed blow:

it seems that c and c++ interface both could work, but can not finish single task only by one;
low bit training or inference is not available via c/c++(ver. 1.6.0 fix fp16 training);
static linking lib is (very) far away from easy to use, cmake configure file(like MxNetConfig.cmake, etc.) will enough for end users to integrate libmxnet.a and other large bunch of static third party libs(it's not easy to maintain gentlemanly demeanor all a day when manually linking these day by day). people could easy to hack loading interface of dynamic library.
smaller size of lib will more friendly to edge device.
more c++ training demo, including how to use kvstore(multiple cards and multiple servers), it's really not easy to understand.

Good day everyone.

@kalcohol please create a new issue about "static linking lib is (very) far away from easy to use", describing your setup in more detail and potentially suggestions how to improve the user experience.

@kalcohol please create a new issue about "static linking lib is (very) far away from easy to use", describing your setup in more detail and potentially suggestions how to improve the user experience.

#17692 add this tiny requist.

We use the c++ interface for inference on a sorting machine. But also we would like to provide the users of our machines an easy and integrated user interface for training new sorting recipes. Now we use python or Mathematica scripts which is far of user friendly for non-programmers. So we want to use the c++ (shielded with a C# wrapper) to provide a custom training environment for non-programmers.
Unfortunately, building the mxnet library with c++ support on Windows machine with MKL / CUDA is an ongoing nightmare. But we really like MxNet

@szha
i checked some docs and projects about distributed training ,
'Horovod' is project from uber team , 'Gloo' is project from facebook team.
The basic idea is to use trick from HPC computing field which is more efficient then traditional param-server:
http://andrew.gibiansky.com/blog/machine-learning/baidu-allreduce/?from=timeline
There is a tool called openmpi on which the 'Horvod' project is based ,but i found openmpi is too difficult to configure and use .
I also check the 'Gloo' which seems to use 'redis' to replace 'openmpi' .
I strongly suggest not to use Horovod directly which is based on openmpi that is too complex and old.
maybe we could figure some way to directly do distributed training over redis ?

@lilongyue the integration of bytePS to mxnet is in this PR https://github.com/apache/incubator-mxnet/pull/17555

@lilongyue the integration of bytePS to mxnet is in this PR #17555
that's great !

A quick comment: DGL contains all sampling implementation and no longer relies on the implementation in MXNet. I think we should deprecate the graph sampling implementation in MXNet.

@szha is there a recent estimate on the timeline for MXNet 2.0? Would you recommend to develop downstream toolkits (e.g. Sockeye) against the master branch now or rather wait a little bit longer?
Is there already documentation on how to transition MXNet 1.x projects to 2.x?

@fhieber we are planning to release the first public beta on this somewhere in August. At the moment we are finalizing some API changes and also validating them in GluonNLP. We will publish a transition doc as part of the public beta.

@szha  We need to add moving AMP package from contrib to core? We will file RFC for this task.

@szha I found an inconvenient thing that there is no concat layer for gluon. Is it possible to add a concat layer for gluon?

Making MXNET_SAFE_ACCUMULATION=1 default when running on float16 would be very convenient!

+1 for turning it on by default.
Get Outlook for iOShttps://aka.ms/o0ukef

From: Davis Liang<EMAIL_ADDRESS>Sent: Tuesday, August 18, 2020 5:32:29 PM
To: apache/incubator-mxnet<EMAIL_ADDRESS>Cc: Sheng Zha<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [apache/incubator-mxnet] [RFC] Apache MXNet 2.0 Roadmap (#16167)
Making MXNET_SAFE_ACCUMULATION=1 default when running on float16 would be very convenient!
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/apache/incubator-mxnet/issues/16167#issuecomment-675784800

I made some good progress with the C# version for v2 changes. I have implemented all the numpy operators and in phase of updating Gluon interface to use numpy ndarray and test it out. Can we include/promote this project from the main website to attact contribitors.
https://github.com/deepakkumar1984/MxNet.Sharp

@deepakkumar1984 awesome work, thanks for contributing to the ecosystem! I think we can definitely highlight it in the ecosystem page as a community project. Feel free to send a pull request to add it there. If you are interested, once it gets close to completion, we could also publish a blog to attract more attention.
How do you envision the codebase to be maintained and hosted going forward?

Thanks @szha, I will start working on the PR to highlight in the ecosystem page. I did started on writing some tutorials eg. https://mxnet.tech-quantum.com/docs-2/getting-started/create-a-neural-network/, but prefer in future to maintain these blogs similar to other bindings like https://mxnet.apache.org/versions/2.0/api/csharp. MxNet Sharp is more than just binding of the api's, I have implemented the Gluon package in version 1.5 itself and now in process of upgrading them. Also the gluon.probability will be implemented after completion of the gluon interface.
I am happy if the core project MxNet.Sharp can be merged with the main branch something like: https://github.com/apache/incubator-mxnet/csharp-package
I have other projects which are making small steps like GluonCV, GluonNLP, GluonTS, AutoGluon and SciKit learn (MxNet version). I can seperate them from my branch and keep them with me for now and probably start linking them in future in the ecosystem page when they are completing one by one.

Cpp-package will be added back in #20131. As this language binding will still rely on symbolic programming, some of the module like APIs removed in #18531 will also be added back. So, we may need to support these module APIs for some languange bindings, especially for cpp-package. @szha @leezu


Crash while looking through emi

Describe the bug: A clear and concise description of what the bug is.
I was looking at the items in the mod using emi and then I crashed after
https://gist.github.com/mxnmnm/0495b09ee413058596637c6c2f2dbae8
To Reproduce: Steps to reproduce the behavior
Idk how to reproduce it :(
Expected behavior: A clear and concise description of what you expected to happen.
no crash
Versions: Delightful, Farmer's Delight, and Forge versions.
neoforge 47.1.84
Delightful-1.20.1-3.4.2
FarmersDelight-1.20.1-1.2.3

Dear,
forge 47.2
Delightful-1.20.1-3.4.2
FarmersDelight-1.20.1-1.2.4
+/- same crash when I open the inventory (Just Enough Items)

I'm getting the same error on version Delightful-1.20.1-3.5
I'm using the Prominence Modpack (Forge/Neoforge) and it crashes for me every now and then with the same error.
Crash report: https://gist.github.com/Terrijoo/5a482cd2d924d2e4fe2c2d4e46073ebe
neoforge 47.1.99
Delightful-1.20.1-3.5
FarmersDelight-1.20.1-1.2.3

Let me know if it persists in v3.5.1

So been playing now for quite a while without a single crash.
Seems like it is fixed, at least for me.
Thanks for the great work!


The file is likely designed for unix and not for Windows so some of the files are causing error and when edit it will be worse. Need to find a way

The file is likely designed for unix and not for Windows so some of the files are causing error and when edit it will be worse. Need to find a way
Originally posted by @DouletMedia in https://github.com/changcheng967/Kata_speed/discussions/10#discussioncomment-10194284

I tried to do it again,  still giving out errors

I will try to compile KataGo and that might work better,  lat time compiling I did something wrong

@changcheng6541 You might need to do the compile

Compile faied

Cheng Chang @.***>于2024年8月7日 周三12:02写道：

Compile faied
—
Reply to this email directly, view it on GitHub
https://github.com/changcheng967/Kata_speed/issues/107#issuecomment-2273814657,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BBBV5SXFFW3WP54LUHORDDDZQJAJZAVCNFSM6AAAAABLXBDLWCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZTHAYTINRVG4
.
You are receiving this because you were mentioned.Message ID:
@.***>

Well that's pretty bad


Raftstore

Checklist

[x] Updated guide with pertinent info (may not always apply).
[x] Squash down commits to one or two logical commits which clearly describe the work you've done.
[x] Unittest is a friend:)

So creating this as a draft for now as it doesn't include any tests regarding persistence (shutting down and starting again). Before going in and adding them, how does the direction look so far?
Closes #438

This change is

sorry for the long quiet was sick the last few days, I'll try and clean this up today :)

Okay cleaned up :)

sorry for the long quiet was sick the last few days, I'll try and clean this up today :)

It's OK. Health first:D

I think the failing tests is unrelated to the changes and rather a flaky test in the main branch, could that be?

Right. Some tests are unstable because an election starts at a random time and sometimes it can not elect successfully due to conflict.

I'm unsure what Mergify wants from me o.O; there is no checkbox on the PR to allow owners to edit the PR.

I'm unsure what Mergify wants from me o.O; there is no checkbox on the PR to allow owners to edit the PR.

Do not know what it wants either. :(

The magic of computers :D I'll try another PR tomorrow and see if I can get the mergify not being angry at me, there is a bit of cleanup that can be done in the rocksdb example I already did locally so that's a good one to test it.


Autoload Encryptor

Spring started complaining about Devise::Encryptor being uninitialized so I figured that it should be autoloaded.

It should not be a problem in a normal workflow because we require it in database authenticatable, but I also don't see any harm in doing the autoload. This probably means we can remove the require, then, would you like to do that as well? Thanks.

@carlosantoniodasilva Ok, perfect. Just removed the require statement.

Thanks.


AWS Predictions (AWSPredictionsService) Index out of Range due to Emojis

Describe the bug
A clear and concise description of what the bug is.
AWS Predictions (text comprehend) crashes when passing in a string of text that contains specific emojis. I am pretty sure this is because some of these emojis count for 1 character but when getting the endIndex swift will return 1 more than the count.
The particular emoji I am referring to is one:👇🏽. This increases the string character count theString.count by 1 but when taking theString.endIndex, it will return 2. I am sure there are many other emojis with the same issue as well. It may have to do with the different colors that are selectable with the emojis, but I am uncertain.
The crash happens around line 140 in the file "AWSPredictionsService+Comprehend.swift" when we try to define the endIndex let endIndex = text.index(text.startIndex, offsetBy: endOffset).
I have implemented a temporary guard bandaid to skip the syntax if the endIndex is out of range, but whenever there is time to fix this, the issue is there. I have a feeling that putting the emoji in the middle of a longer string may also mess up the indexes of the words that are interpreted and return inaccurate strings of the key phrases of the string, but I have not tested this.
To Reproduce
Steps to reproduce the behavior:

Create a string with 👇🏽 at the end of the string.
Pass into Amplify.Predictions.interpret()
Crash occurs.

Expected behavior
A clear and concise description of what you expected to happen.
No crash and accurate interpretation of string returned.
Screenshots
If applicable, add screenshots to help explain your problem.
Unnecessary.
Thank you for the great platform otherwise!

PR was approved, pending push into the main branch, and also pending release.

Released in 1.5.1