HADOOP-17947. Provide alternative to Guava VisibleForTesting

Description of PR
Provide alternative to Guava VisibleForTesting so that we can get rid of Guava dependency to represent testing scope for a class/method/field.

FYI @amahussein @aajisaka @tasanuma @steveloughran. Once this PR gets in, will update PR #3503 to just use this @interface.
Thanks

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
16m 55s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

-1 :x:
mvninstall
5m  5s
/branch-mvninstall-root.txt
root in trunk failed.

+1 :green_heart:
compile
29m 17s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
18m 39s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m  8s

trunk passed

+1 :green_heart:
mvnsite
1m 37s

trunk passed

+1 :green_heart:
javadoc
1m 13s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 42s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 31s

trunk passed

+1 :green_heart:
shadedclient
21m 11s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 56s

the patch passed

+1 :green_heart:
compile
22m 21s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
22m 21s

the patch passed

+1 :green_heart:
compile
19m 25s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 25s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  3s

the patch passed

+1 :green_heart:
mvnsite
1m 31s

the patch passed

+1 :green_heart:
javadoc
1m  8s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 33s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 40s

the patch passed

+1 :green_heart:
shadedclient
21m 54s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 21s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 54s

The patch does not generate ASF License warnings.

190m  4s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3505/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3505

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 7417202ace68 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 8da4df2cd448093bcf4599b14057b842f027fe97

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3505/1/testReport/

Max. process+thread count
3152 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3505/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

If we are going to change the javadocs, maybe make clear this is a guava replacement and what guarantees we offer (none). Here's my draft
  /**
   * Annotates a program element that exists, or is more widely visible than
   * otherwise necessary, for the benefits of testability.
   * More specifically testability within the hadoop-* modules.
   * This gives the implicit scope and stability of:
   *

   * @InterfaceAudience.Private
   * @InterfaceStability.Unstable
   *

   * If external modules need to access/override these methods, then
   * they MUST be rescoped as public/limited private.
   */

Now, should we actually move this to the package org.apache.hadoop.classification in the module hadoop-annotations? It is a classification, after all. I think from a consistency perspective, it makes sense

@steveloughran Javadoc changes, specifically the relation with IA.Private and IS.Unstable makes the doc look better.

Now, should we actually move this to the package org.apache.hadoop.classification in the module hadoop-annotations? It is a classification, after all. I think from a consistency perspective, it makes sense

This is interesting thought. I think it makes sense. In fact, we also have another advantage here - all modules have dependency on hadoop-annotations but not on hadoop-common so all hadoop-* modules can use it. It's worth moving VFT to hadoop-annotations. The only question is, do we need some VOTE or opinions on this PR should be sufficient to move forward?

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
16m 57s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
34m  2s

trunk passed

+1 :green_heart:
compile
22m 56s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
19m 38s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m  0s

trunk passed

+1 :green_heart:
mvnsite
1m 37s

trunk passed

+1 :green_heart:
javadoc
1m  3s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 32s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 25s

trunk passed

+1 :green_heart:
shadedclient
23m 51s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 57s

the patch passed

+1 :green_heart:
compile
22m 21s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
22m 21s

the patch passed

+1 :green_heart:
compile
19m 29s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 29s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 59s

the patch passed

+1 :green_heart:
mvnsite
1m 29s

the patch passed

+1 :green_heart:
javadoc
0m 59s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
1m 34s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
2m 36s

the patch passed

+1 :green_heart:
shadedclient
23m 47s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
17m 13s

hadoop-common in the patch passed.

+1 :green_heart:
asflicense
0m 49s

The patch does not generate ASF License warnings.

217m  7s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3505/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3505

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 2cfa13e3fcc2 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 53929537a26bebf13e36ef0eccb9a1ef037c5e8f

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3505/2/testReport/

Max. process+thread count
2361 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3505/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Adjust to the VFT in Guava, we may want to annotate it with the following annotations.
@Retention(RetentionPolicy.CLASS)
@Target({ElementType.TYPE, ElementType.METHOD})
@Documented

Adjust to the VFT in Guava, we may want to annotate it with the following annotations.
@Retention(RetentionPolicy.CLASS)
@Target({ElementType.TYPE, ElementType.METHOD})
@Documented

Thanks @tasanuma. I tried to adjust it to the one defined here https://github.com/google/guava/blob/master/guava/src/com/google/common/annotations/VisibleForTesting.java
But now I see that GwtCompatible annotation itself has the Retention and Target annotations so they should be applied to VFT as well. Let me make this change.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 54s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
34m 14s

trunk passed

+1 :green_heart:
compile
22m 58s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
19m 30s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
0m 27s

trunk passed

+1 :green_heart:
mvnsite
0m 31s

trunk passed

+1 :green_heart:
javadoc
0m 31s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 30s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
0m 41s

trunk passed

+1 :green_heart:
shadedclient
22m  7s

branch has no errors when building and testing our client artifacts.

-0 :warning:
patch
22m 27s

Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 14s

the patch passed

+1 :green_heart:
compile
22m 16s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
22m 16s

the patch passed

+1 :green_heart:
compile
19m 40s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
19m 40s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 25s
/results-checkstyle-hadoop-common-project_hadoop-annotations.txt
hadoop-common-project/hadoop-annotations: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)

+1 :green_heart:
mvnsite
0m 30s

the patch passed

+1 :green_heart:
javadoc
0m 29s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
0m 28s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
0m 49s

the patch passed

+1 :green_heart:
shadedclient
22m  3s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
0m 28s

hadoop-annotations in the patch passed.

+1 :green_heart:
asflicense
0m 49s

The patch does not generate ASF License warnings.

173m 55s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3505/4/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3505

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 4c87a99d2959 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 06ec961eb5d2af178915fcd3120aa6527dca545c

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3505/4/testReport/

Max. process+thread count
522 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-annotations U: hadoop-common-project/hadoop-annotations

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3505/4/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Merged it. Thanks for your contribution, @virajjasani. Thanks for your review, @steveloughran.


Add log to print testAddUserFailure response

$Subject

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/7943328124

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/7943328124
Status: success


vpc peers: add ability for some peers to be able to get traffic routed back from all subnets

What
https://www.pivotaltracker.com/story/show/184840850
Firstly this switches backing service subnets to use a custom VPC route table rather than the default one, allowing us finer grained control.
Then, based on Ben's #3134, this optionally adds an entry to the ~default VPC~ backing service subnet's routing table to allow packets to be routed to a peer VPC. ~While the rule itself is quite general, this actually affects little more than our backing-services subnets because our infra subnet uses a custom route table as do our cell subnets.~
Security groups remain in place on individual machines, which have the main responsibility for policing access - for backing services, these only allow traffic from cells and brokers . Backing services also have no outbound rules so shouldn't be able to make outgoing connections to the peered VPCs.
I'd still love it if we could get e.g. VPC flow logging in place before we use this in prod.
How to review
You might need me to show you.

Deploy to a dev env with an entry in <env>.vpc_peering.json pointing to a VPC in our experimental account and "backing_service_routing": true.
Accept the peering request from our experimental account through the AWS console.
Set up a routing table entry in the experimental VPC to allow traffic to go to 10.x.x.x through the VPC peering.
Find the login credentials for a brokered database in the dev env.
Clickops-create a security group in the dev env VPC allowing ingress from the peered VPC subnet & add it to the chosen brokered database.
Try to connect to the database from a throwaway EC2 machine in the experimental VPC.

I already have most of the required things set up in the experimental account, so just ask me for all the details if you like.

🚨⚠ Please do not merge this pull request via the GitHub UI ⚠🚨

I tested via the following commit in dev04:
https://github.com/alphagov/paas-cf/commit/5073f848ccf469781d7e50c1337e677ea50b15ea
Manual changes included:

Adding route table to az-healthcheck default routing table
Creation of a test security group
Attaching security group to rdsbroker database in dev04.

It worked.

This will most likely cause a failure in staging and production when deployed because the cf-terraform and the vpc-peers-terraform are run at the same time. We will need to either split this into two or "click again"


Add AprilTag IO classes

Allow to access loggable PhotonCameras which can access info about the best target, along with a pose estimation from that camera via AprilTags

must be tested before merge

TODO: rewrite io classes to only use weird arrays internally


Add requirements.txt for esRollover/Cleaner scripts

Requirement - what kind of business use case are you trying to solve?
Run esRollover.py or esCleaner.py just to see help docs.
Problem - what in Jaeger blocks you from solving the requirement?
Results in the following error if dependencies aren't installed which is not a great user experience:
 ./plugin/storage/es/esRollover.py
Traceback (most recent call last):
  File "./plugin/storage/es/esRollover.py", line 4, in <module>
    import curator
ModuleNotFoundError: No module named 'curator'

Proposal - what do you suggest to solve the problem or improve the existing situation?
Provide a requirements.txt file in plugins/storage/es containing the curator dependency needed to run esRollover/Cleaner.py:
elasticsearch-curator==5.8.3

And instructions in README to run the following to validate dependencies are installed or if not, installs them for the current python environment:
pip install -r requirements.txt

Don't we already have requirements.txt somewhere?
If it's somehow derived from ES distribution then perhaps we shouldn't tempt users to run the .py directly, but use a container image that we publish.

Don't we already have requirements.txt somewhere?

Nope. Not that I can see in this repo at least.

If it's somehow derived from ES distribution then perhaps we shouldn't tempt users to run the .py directly, but use a container image that we publish.

Ah yes, I should have looked around a bit more before creating this issue; I'd forgotten we have a docker image for ES rollover/cleanup.
Turns out the dependencies are installed when building the docker image:
https://github.com/jaegertracing/jaeger/blob/master/plugin/storage/es/Dockerfile.rollover#L6
Thanks, Yuri. Closing this issue.


chance.exp doesn't always generate a future date

Both chance.exp_month and chance.exp_year have future flags (for good reason as outlined in the docs). It seems odd that their parent, chance.exp doesn't have it. Without it, it's not possible to generate random, valid credit card numbers.
Looking at the source, there is code which intent is to ensure the generated date is in the future. However, I'm pretty quickly able to generate an exp date that is earlier this year. Not sure why yet, but if I get a chance I'll look into it and PR a fix.
> chance.exp({raw: true})
< Object {year: "2015", month: "01"}

Hi @pmowrer chance.exp takes care of this for you and always credit card expirations which are valid from the time you create them.
First, only the chance.exp_month actually uses the future flag (the docs I just realized are wrong, will fix!).
chance.exp_year automatically uses a year in the future.
Further, all the future flag on the ex_month does is specify that the month must be beyond the current month. For example, if it is currently June, it makes sure the month picks from the pool July, August, September, November, December.
That is only necessary when the expiration year is the current year. For example, January 2015 is invalid as of today because that is in the past. However, for any other year (2016, 2017, 2018, etc.) January is entirely valid.
Here is the current code:
Chance.prototype.exp = function (options) {
        options = initOptions(options);
        var exp = {};

        exp.year = this.exp_year();

        // If the year is this year, need to ensure month is greater than the
        // current month or this expiration will not be valid
        if (exp.year === (new Date().getFullYear())) {
            exp.month = this.exp_month({future: true});
        } else {
            exp.month = this.exp_month();
        }

        return options.raw ? exp : exp.month + '/' + exp.year;
};

See how it generates a random expiration year (which is any year from now to 10 years from now) then it checks to see if the year it generated is this year. If so it only generates a month from the current month on. Otherwise it uses the full pool of all 12 months.
Anyway, this is all a very wordy way of saying there is no bug here, this method always generates a valid credit card expiration date in the future.


registry: add dump of default logon's password in text format

When using pypykatz registry  in text mode (the default), user login is printed, but not its domain or password, while this information is there if we use --json. This commit adds the user domain and password to the text dump.

thank you!


Use yarn caching for e2e tests

Description
Add Yarn caching to E2E tests.
Other changes
N/A.
Tested
Observed in CI
How others should test
N/A.
Related issues

Fixes #530 #529

Backwards compatibility
N/A.

@noisypigeon unfortunately it appears for e2e tests the cache step is not saving any time as compared to the tests running on main.  Will close this PR if you agree that's the case.

Yeah sure that's fine! If no improvement there's no need.


Adding include_dataset as the opposite of existing exclude_dataset

Hi there, just running this by you before making a PR for it.
We are only interested in a few of the datasets. Right now, the "correct" way is that we should query the catalog to see what is there and remove everything that we are not interested in. However, it would be ideal if we could avoid the call to the catalog api all together.
On top of it: as a cherry on top, it would be nicer if we could add a new env var to change the behavior of indexer as well. This way, we could just import the documents that belong to datasets that we are interested in. It should not make a huge performance boost but it wouldn't hurt either.
I am up for making a PR for the include_dataset parameter (unless you have a better name for it). The indexer stuff: please let me know what you think.

Heya! Appreciate the discussion here a lot. I was also thinking about include_dataset but it does sort of intersect weirdly with the existing scoping of queries in some cases. For context: have you looked at the mechanism for specifying your own manifest YML file (https://www.opensanctions.org/docs/yente/datasets/)? What we've recommended to some customers is to make a custom manifest file like this:
catalogs:
  - url: "https://data.opensanctions.org/datasets/latest/index.json"
    scope: sanctions
    resource_name: entities.ftm.json
datasets:
  - name: usa
     title: USA-related datasets
     datasets:
       - us_ofac_sdn
       - us_ofac_cons
       - us_trade_csl
       - us_bis_denied

What you can see here is that a) the catalog is instructed to import the sanctions dataset, which is smaller than default, and b) you can now run queries like /match/usa on only the listed datasets. (nb: If you'd listed out the datasets individually under scopes in the catalogs section, you'd run into some trouble because the individual datasets use the deduplicated IDs, do you'd have the Putin entity from list A and B both in the index and randomly return one).
I'm a bit loathe to rebuild this via env vars: it saves you one YML file, but then you need some sort of massive nested YML syntax that is less transparent.
All of this, of course, doesn't mean that we cannot have /match/default?include_dataset=us_ofac_sdn&include_dataset=us_ofac_cons to do more ad-hoc scoping.

Hey @pudo. Thanks for the explanation.
Even though we like to run our own instances of yente, we definitely do not want to run a modified version of it. Hence, as long as there is an official way to do this, we are content (with a file or with an env var, it doesn't matter). It sounds like Manifest does what we want so I am closing the issue.
P.S. I can see that with this approach, only the data that we are interested in is going to be stored in opensearch/elasticsearch as well. So this is great.


Sogu web spider making bad requests that raise exception

https://dpul.princeton.edu/eastasian/catalog?f%5Breadonly_creator_ssim%5D%5B%5D=%E8%B2%BB%E5%95%9F%E6%B3%B0%2C+1590-1675&f%5Breadonly_subject_ssim%5D%5B%5D=Smallpox&f%5Breadonly_subject_ssim%5D%5B%5D=Medicine%2C+Chinese%E2%80%94Early+works+to+1800&q=&sort=sort_date

 Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

Blacklight::Exceptions::InvalidRequest: RSolr::Error::Http - 400 Bad Request
Error: {
  'responseHeader'=>{
    'zkConnected'=>true,
    'status'=>400,
    'QTime'=>1,
    'params'=>{
      'f.exhibit_eastasian_readonly_collections_ssim.facet.limit'=>'11',
      'f.exhibit_eastasian_readonly_view-in-catalog_ssim.facet.limit'=>'11',
      'f.exhibit_eastasian_readonly_contributor_ssim.facet.limit'=>'11',
      'f.exhibit_eastasian_tags_ssim.facet.limit'=>'11',
      'f.exhibit_eastasian_readonly_contents_ssim.facet.sort'=>'count',

URI: http://lib-solr.princeton.edu:8983/solr/dpul/select?wt=ruby
Request Headers: {"Content-Type"=>"application/x-www-form-urlencoded; charset=UTF-8"}

Backtrace
line 106 of [PROJECT_ROOT]/app/controllers/catalog_controller.rb: index

View full backtrace and more info at honeybadger.io

Should the application return a 400 as well?
At one point we had put some efforts into handling this sort of request in orangelight. Look into what we did there and ask @christinach if we feel good about how it turned out.

Here's the OL solution: https://github.com/pulibrary/orangelight/blob/a24c7f08c34e0abd55ec65dec6b70ab792aebbb6/lib/orangelight/middleware/invalid_parameter_handler.rb
We look for specific patterns, presumably based on what we've seen in honeybadger.


Fix missing custom headers for searchapi endpoints

As seen in https://github.com/o19s/quepid/issues/890, when setting up an apisearch endpoint, no custom headers are ever sent by the browser.
In this PR I update the searchapi Searcher to get headers the same way the es Searcher does and pass them along to the transport.
In my reading, this was the only Searcher not propagating these headers. I re-used esUrlSvc.getHeaders as this service was already in use in the searchapi Searcher.

This makes sense. It makes me a bit sad that esUrlSvc seems to be taking on a life of it's own... But this looks reasonable change.

Yeah, totally, it seems to be handling some cross cutting concerns. Thanks for merging!


user_groups: Add group level setting for configuring who can manage that group.

This PR adds a group level setting for configuring the permissions for managing the group.
Fixes: #25928 and part of #25929.

Screenshots and screen captures:

Self-review checklist

[ ] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[ ] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

@sahil839 this PR is ready for review now. So, you can have a look. Some important points to note:-

This PR addresses a part of #25929 by adding only a group level setting Who can manage this group and does not make any change for realm level setting.
Currently the API changes allow to accept can_manage_group in form of string while changing the value for existing groups. As it is not required now I think I should make changes and make the endpoint accept only integer values. But just want to confirm before making the change in case we have any reason to keep it.

@Ujjawal3 can you rebase the PR? I am reviewing this now, but this conflicts with the changes merged in #26048.

@sahil839 I have rebased the PR. Also changes suggested in last review have been done. You can review it now.

@sahil839 I have done all the suggested changes except discussed here. Also, I could not figure out the reason for failure in tests while building database. Looking at the errors, I don't think they are related to changes in this PR.

Updated the PR. Changes left -

There is some bug in UI when selecting "You" option in the group creation form, so need to fix that.
I removed the live-update commit since that was incomplete. Will do live-update changes as a follow-up.

There might be a couple of bugs in UI part, so would need to re-check that but the backend part is ready for review.

Fixed the UI bug. Note that for can_mention_group, the "Members of this group" option is not shown in creation form since we would need to add the backend support for it and I think would be good to add it as a separate change since this PR is already large.

I have some fiddly comments about the UI, but I'm not sure it's worth getting into given the alternative proposal in https://chat.zulip.org/#narrow/stream/101-design/topic/group.20permissions.20UI?

Updated the PR -

Default is now set to "Nobody" group.
Removed commits to support the "Creating user" and "Members of this group" for now.

Also, have not fixed the live-updating the UI on changing both old real level setting and new group level setting. Can do that as a follow-up.
Also, #30850 is opened for disabling the group settings in UI when user is not allowed to change them.

Updated the PR -

Default is now set to "Nobody" group.
Removed commits to support the "Creating user" and "Members of this group" for now.

Also, have not fixed the live-updating the UI on changing both old real level setting and new group level setting. Can do that as a follow-up. Also, #30850 is opened for disabling the group settings in UI when user is not allowed to change them.

Updated the PR to fix tests. Have also posted some inline comments.

I gave this a thorough read. A few notes:

When adding new settings to blocks of similar settings, let's try to preserve alphabetical sorting order. Can address this in a follow-up; I'm not sure it's worth rebasing to address it.
How have you tested that the development_environment checks work in production? I worry a bit about it sending some malformed payload in that case, it failing an assertion from trying to read from dropdown widgets that aren't set up yet. We want to be really careful whenever
It looks like there is a functional change in this PR: previously the can_create_group_policy setting had no effect on whether a user was able to edit a group that they'd a member of. With this change, it does.

Looks great, merged, thanks @Ujjawal3 and @sahil839! I'm really excited to have this one finished.


armips will fail the build with gcc-11: error: 'numeric_limits' is not a member of 'std'

Due headers changes gcc-11 fails to build armips:
$ nix-build -E 'with import <nixpkgs> {}; armips.override { stdenv = gcc11Stdenv; }'
...
/build/source/Core/ExpressionFunctions.cpp: In function 'ExpressionValue expFuncMin(const wstring&, const std::vector<ExpressionValue>&)':
/build/source/Core/ExpressionFunctions.cpp:219:36: error: 'numeric_limits' is not a member of 'std'
  219 |         floatCur = floatMin = std::numeric_limits<double>::max();
      |                                    ^~~~~~~~~~~~~~

Notify maintainers
@marius851000
Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute: armips

Closing as INVALID.


"Go to Symbol in Workspace…" is slow with moderately sized solution containing only C# projects

REPRO: I opened a solution in VS Code containing 68 C# projects (according to the response for "/projects" request in OmniSharp output log). Once all projects were loaded (took about a minute), I pressed Ctrl-T and had to wait for at least 15 seconds for the list to be populated. This happened every time Ctrl-T is pressed - not only the first time. Additionally, typing a single character freezes the list again for few seconds which makes the feature only marginally useful in this case.
EXPECTED: IMO OmniSharp could:

Cache results of recently issued "/findsymbols" requests.
Apply a strict timeout when processing "/findsymbols" requests and return cached or partial or simply empty result when the timeout is reached.

IIRC Problem is in vscode rendering thousands of symbols in the ui after server returns them, not on the server side.

Interesting. I was only judging by OmniSharp output log because I see a significant delay after "/findsymbols" request printed and before the response appears. In any case I would argue that "/findsymbols" processing should be limited in both time and the amount of data it is allowed to return. A user IMO is unlikely to just scroll through thousands of entries and s/he is more likely to continue typing the filter to narrow the result to only few entries.

BTW, this issue I opened sounds like a dupe of https://github.com/OmniSharp/omnisharp-vscode/issues/1808

Yup that's it. I remember I just modified "workspaceSymbolProvider.js" manually so I never pass on insanely big symbol lists to vscode, and it solved all the problems. Wanted to make a PR but it just flew over my head.

This is definitely could be a good quick fix, but long term IMO ideally each supported file system (MSBuild, Cake, etc) limits as well how many items it attempts to discover and send back with /findsymbols response.


Add other properties to publish attestation

in the slides
in the attestation description
in the future work for implementation

Done.


[schematics-cli] support run command without relative path

🚀 Feature request
I'm using @angular-devkit/schematics-cli to generate code of Vue, and want to publish as a package so that I can use it at any project. Here is the code provide with hello command.
When I want to generate codes in other project, schematics command schematics <relativePath>:hello requires me  to tell schematics the relative path(such as schematics ../src:hello). My expectation is that if I install the lib  as a dependency，I can generate find where to find the hello .
Description
 `@angular-devkit/schematics-cli`  be able to run command once the project have installed lib with collection, without specifying relative path.
Describe the solution you'd like
 For example,the structure of a project like:
ProjectA
|-- node_modules
|    |-- schematics-lib-with-hello-collection
|-- src
|-- package.json

Once I've installed @angular-devkit/schematics-cli and  running schematics hello at the rootPath of ProjectA, it'll find the hello from the rootPath of ProjectA first.
For the rootPath of ProjectA do not have hello collection, it'll find it at node_modules next. For schematics-lib-with-hello-collection have provided hello collection, it'll generate code.
Be grateful for reading.

Hi @ishowman,
This is already supported. You need to provide the library module name as collection name. The collection name, is resolved using node module resolution.
Example:
schematics schematics-lib-with-hello-collection:hello

Hi @ishowman,
This is already supported. You need to provide the library module name as collection name. The collection name, is resolved using node module resolution.
Example:
schematics schematics-lib-with-hello-collection:hello

Is the library module name the same as name field of package.json

Yes it is.

Yes it is.

Thanks.Have a good day!


Image contains errors

I'm able to get the info.json file for the image, however, when I try to access the image I get a message that the image contains errors:

I'm not getting any errors on the lambda function, I've tried redeploying the application, but the error persists.

Thanks for reporting this issue, @hammo92! The difference between your working and non-working images is that the working ones have 3 color channels, while the non-working ones have 4. It seems the version of libvips packaged with serverless-iiif has a problem with JPEG-compressed pyramidal TIFF images with alpha channels. Please try recreating the two failed source images using the following command and let me know if it resolves the issue (or doesn't):
convert source_image.tif -alpha off -define tiff:tile-geometry=256x256 -compress jpeg 'ptif:output_image.tif'

If it helps, I'll update the docs to include this information and workaround.

Perfect, that's working for me now.
I think it might be good to add it as a note on the docs, it's not an issue which is easy to identify as the images seem fine when viewed locally, fortunately you're a Tiff wizard and spotted it straight away!
Thanks for the help.

I think that's good, it's clear; and even if users don't understand the specifics of the Tiff format and channels the provided commands will get them the right results.


Adding a build command

Is your feature request related to a problem?
To make testing and verifying new fixed issue possible
Describe the solution you'd like
Can we add a make build command like the one used in grid-cli that create a bin on the latest changed happened on the code

and add the make build description in the readme.

My bad command is working but just needed to be added in the docs

Verified,


feature: Add letter-spacing for text styles

Hello.
I noticed the letter-spacing field is missed when the extractor is launching.
I decided to add this field to the text styles file.
Also, Figma can have two variant of this field inside its UI (percent and pixel, moreover pixels turns into em measure inside figma's inspector), but API can get the only pixel variant.
I've added the converter for this field to px or em, I guess it would be convenient.

So, I have changed the logical a bit.
I've removed convert function and this field is enabled by default (without setting from config)


How can i update the server to public-test

Hey the new public-test update is out now. How can i force the update the server to public-test because it's not updating with the normal update check

...
So... my guess would to set STEAMCMD_ARGS to: -beta public-test -betapassword yesimadebackups validate

Ok, so this is what I have put into valheim.env:
STEAMCMD_ARGS=-beta public-test -betapassword yesimadebackups validate

It did not work to jest restart docker images, so I have just deleted everything in data/ and it downlaoded the public-test branch. Downside is, valheim plus seems to not be working 😞


Publishing PWMetrics in GitHub Package Registry

Currently, we are able to publish PWMetrics in the scope of GitHub Package Registry, I'd like to hear from folks if everyone is ok with that.
cc @paulirish @pedro93
P.S. I can do operation work.

Hey @denar90 sorry for the late reply, fine by me.
Since I haven't been able to contribute/maintain this repo feel free to proceed as you see fit, I trust you to good care of pwmetrics :)

sgtm

I found interesting note in docs:

GitHub Package Registry only supports scoped NPM packages

https://help.github.com/en/articles/configuring-npm-for-use-with-github-package-registry#authenticating-to-github-package-registry


dcr/mixing: Mixer stopped automatically.

 I believe this was simply because there came a point where there was only a new utxo (< 2 confirms) in the Unmixed account. I believe the wanted behavior in this instance is to wait for the two confirms and continue mixing. I was able to turn the mixer back on later.
2023-10-10 17:23:47.324 [INF] DCR: [1] unmixed account does not have a mixable output, stopping account mixer
2023-10-10 17:23:47.324 [ERR] DCR: AccountMixer instance errored: context canceled

so i think what happened was the mixer actually finished it's mixing process and then exited hence this log 2023-10-10 17:23:47.324 [INF] DCR: [1] unmixed account does not have a mixable output, stopping account mixer
by design when the mixer no longer has any mixable output, i believes it automatically stops the mixer and potentially locks the wallet(if ticket buyer is running, this would be a problem)
it also seems that ticket buyer instance runs it's own independent mixing, which the regular mixer is oblivious to, and change from the ticket buyer process returns to the unmixed account
would need to go through this process again to confirm, but i think that's what's happening


[DOC] "Mappings" is overloaded

What do you want to do?

[ X] Request a change to existing documentation
[ ] Add new documentation
[ ] Report a technical problem with the documentation
[ ] Other

Tell us about your request. Provide a summary of the request and all versions that are affected.
I recently searched for [mapping] and got two pages at the top, one called Mappings APIs and the other called Mappings and field types. Naturally, I assumed that they were talking about the same thing.
But in fact, the word "mappings" is used in (at least) two radically different ways in OpenSearch. One has to do with the index schema (field types), the other has something to do with Security Analytics (though I don't understand what). Both the page Mappings APIs and the page Mappings and field types are returned by a search on [mappings]. This is very confusing since they apparently have nothing to do with each other. What's more, two different kinds of "mapping" are mentioned in the Security Analytics section: Field mappings and Roles mapping. I understand that mapping is a generic concept -- all the more reason to be clear in each case about what it is being applied to. And "field mappings" sounds the same as the other kind of ...um... field mapping.
It would be nice if (a) we gave them "full" names on first use in a page, e.g., "Security mappings" and "Field mappings" and (b) linked to the definition. The Mappings and field types page does give a definition in the first line, but the Mappings APIs page does not.
More generally, shouldn't all technical terms be linked to their definitions throughout the documentation? These days, we're all used to Wikipedia-style documents where you can start in the middle of a topic and follow links to improve your understanding.
Another protean term is "settings"....
What other resources are available? Provide links to related issues, POCs, steps for testing, etc.

Thanks, Stavros, you make some good points. We can definitely take a look at how we are using the term mappings.
In the meantime, a great way to provide feedback is to use the 'edit this page' link in a page where you are confused, and enter your suggestion and push up a PR. We depend on product managers like you to provide input on context, so your viewpoint is critical. Creating a PR will help to expedite the change.
To your point >>shouldn't all technical terms be linked to their definitions throughout the documentation: we could do this if we published a glossary but we currently do not have one in the docs. It is on our list and we have an issue open for it, which I'll link here.

Thanks for your quick reply, Heather.
I would propose edits except that in this case, I don't actually know anything about Security Analytics mappings.
When I click on Edit this page, I get "You need to fork this repository to propose changes." I assume this is normal.
To submit a PR, I click on "Commit changes...", right?

Thanks for your quick reply, Heather. I would propose edits except that in this case, I don't actually know anything about Security Analytics mappings. When I click on Edit this page, I get "You need to fork this repository to propose changes." I assume this is normal. To submit a PR, I click on "Commit changes...", right?

Yes, fork the repo, make the change, commit the change, then submit the PR. We'll take care of the rest unless we have questions or suggestions. Thanks, @smacrakis !


[11.0.x] ISPN-12375 Add missing license information to wildfly modules

Fix path: missing ${basedir}
Remove hibernate-search-backend-jms
The licenses-plugin is configured to ignore test dependencies and it
was ignoring the dep. Remove it from our pom.xml fix the issues. It
isn't used explicitly anywhere.

https://issues.redhat.com/browse/ISPN-12375

ps. only affects 11.0.x


We should publish builds

It's a bit hard for people who aren't experienced with Rust to try Nana out. It would be nice if we could build executables+installers in GitHub Actions and publish them (probably in GitHub releases).

@rgwood agreed !
Since you are working on this it would be great to get more people on board helping out, giving feedback etc...
So if publishing builds as you suggest would increase marketing exposure of nana and allow more people to use the product because the barrier to entry to run nana is less than sounds like a great idea 😄


Clean CSV Data:  Clear Illness Onset Date Field when Date is Invalid

Problem Statement
As an Experity EMR platform user, the platform does not force me to enter a valid dates so I need the Illness Onset Date value cleared if it is invalid so that this non-essential field does not cause my file to fail.
Acceptance Criteria

Given I

Am logged into ReportStream as a user with Sender permissions
Am a Sender who is using the Experity schema

When I

Log into ReportStream
Upload a CSV file

Then

The System, prior to submitting the file to ReportStream for processing, extracts a properly formatted date from a part of a provided data string. E.g. "10/1/2020", "10/1/20", "a week ago". Dates written in words like "a day go" is invalid. Decode only proper dates.

If a date cannot be decoded then the field is left blank.
If the field is blank then the field is left blank.
Refer to the vbScript for more specific criteria: https://drive.google.com/drive/folders/12qqnBP1JCkLXvJBQBatNxgErn8e74FzG

Design Considerations
Looks like the data is mostly two of four digit year dates with no extra text.  Can the date mapper handle this conversion?  Other data like "3 days ago" will be ignored/set to blank.

Suggested acceptance criteria:

ReportStream is capable of extracting a properly formatted date from a part of a provided data string.  E.g. "test date 10/1/2020".
If a date cannot be decoded then the field is left blank.
If the field is blank then the field is left blank.

A version of this went to production, but upon review with @rhood23699 and @carlosfelix2 there are a few things that need ironed out

A parse error at build time when trying to use the MM/dd/yyyy format (or any format with a / in it). This is a simple fix
When using the "MM/dd/yyyy" in the new nullDateValidator, it would seem that some validation occurs upstream from mapper (CsvSerialize, I believe), replacing the name with the value "^^validationFail". The new mapper will currently transform that value into an empty string, when it should be validating it. A fix for this will have to somehow circumvent that upstream validation in order to achieve this
Closer attention to emulating the OG VBScript is needed. Particularly, some of the examples outlined at the beginning of the DateFormatting() function. The rest of that code may be helpful in manipulating some of those incoming funky dates so that they validate in kotlin.

@sethdarragile6 I'm confused by "a version of this went to production". Was there another related ticket out there?

@loripusey no, no related ticket. what I meant by that was that the code was approved and merged to master (and thus deployable and I believe deployed to production). then, the issues above were discovered while "stage testing" with Rick Hood. Unfortunate, but it doesn't impact anything unless someone actually tries to use this new mapper.

@loripusey I'll help Seth and Aurora on the issues found.  I apologize.  You were nice enough to create this ticket for me back in September, but I didn't review the AC closely enough, so it doesn't reflect exactly what the existing VBScript is currently doing.  I'll do better in the future.

sounds good, I just wanted to clarify and make sure there was not a related ticket


H705C Permanent Outdoor LIghts support

Please explain your feature request in a one or two sentences.
There is a new model H705C or maybe its not new but its not supported with the current plug-in.  right now it only supports H705 A and B.

Is your feature request related to a problem? Please describe.
Govee plugin cannot see the device.  Im  not able to add it in homebridge

Any particular Govee devices that this relates to?
H705C Permanent Outdoor Lights

Anything else?

Hi @peterstylias please try the new version of the plugin


Filter out non-header files from Compile Source phase

Fixes #414.

Looks like you need to update the fixtures: https://github.com/buildbuddy-io/rules_xcodeproj/blob/main/CONTRIBUTING.md#developing

Some changes are needed in Fixtures.swift. You should be able to run the tests locally with bazel test --config=cache //tools/....

Some changes are needed in Fixtures.swift. You should be able to run the tests locally with bazel test --config=cache //tools/....

On it now

Some changes are needed in Fixtures.swift. You should be able to run the tests locally with bazel test --config=cache //tools/....

How do you debug these AssertNoDifferences, I only get a subset of the change and I cannot easily tell which target owns the added PBXObjectReference

Yeah, sometimes the diffs hide context. For these though, you know you are removing files from the Compile Sources, so you'll need to remove things from PBXSourcesBuildPhase initializers (or the phases all together) for targets that had headers there. One example is PBXBuildFile(file: elements["z.h"]!),.


[Feature] Think of a way to automate the execution of validation scripts

Feature description
Think of some way to automate the execution of validation scripts
Use Cases
NA
Test Cases
NA
Additional Information
No response

https://github.com/project-chip/certification-tool/issues/294

https://github.com/project-chip/certification-tool/issues/295


Add configuration to ensure a newline before each comment

This adds a ensure_newline_before_comments configuration that matches the format that black imposes (for better or worse) and allows isort to be used with black without any thrashing/conflicts with regards to blank lines before comments.
Should help address https://github.com/psf/black/issues/251

Codecov Report

Merging #1000 into develop will decrease coverage by 0.09%.
The diff coverage is 69.23%.

@@            Coverage Diff             @@
##           develop    #1000     +/-   ##
==========================================
- Coverage    89.75%   89.65%   -0.1%
==========================================
  Files           12       12
  Lines         2675     2688     +13
  Branches       457      461      +4
==========================================
+ Hits          2401     2410      +9
- Misses         186      188      +2
- Partials        88       90      +2

this implementation would be very helpful.
is there any any estimate to move that forward?

@FuegoFro This is great! Thanks for working on this!

Confirmed current master fixes the problem.
Is there a schedule for the next release? It would help unblock formatting pip with Black+isort.
Thanks!

@hugovk I can get it scheduled comfortably for 2 weeks for now. if it is really urgent I can try to get it the release in good shape within a week. I hate to block the work on pip!
Let me know,
Thanks!
~Timothy

Thanks! I don't know how long it'll take for the PR to be reviewed (it's big!) so let's ask over there: https://github.com/pypa/pip/pull/7084.

I hate to block the work on pip!

Hi hi.
Don't worry about that -- We can easily wait for the next release of isort for doing this, so please feel free to make the release at a time that's comfortable for you. No pressure from our end. :)
AFAIK, we're in no hurry to adopt black (we already use isort!) even though I don't think anyone is opposed to it at this point. 🙃
Thanks for all of your (and other contributors') work on isort!

Hey! Just wondering if there was any news on a release that has this change in it. No rush! Just curious 🙂

Hi @FuegoFro,
I've finished the other project that I wanted to finish up before the release: https://timothycrosley.com/project-7-streamdeck_ui and switched to spending all my OpenSource time working on preparing the isort 5.0.0 release. I hope to have it wrapped up soon.
Thanks!
~Timothy

Hey there! Just checking in again to see if any progress has been made on a new release 🙂

Any news on this? Thanks!

Can we help with the release somehow?

@timothycrosley ANY answer would be really appreciated and much better than just ignoring people 🙄

@timothycrosley would it be possible to push a 4.3.x release with this feature added ahead of the move to 5.0.0? This is currently blocking projects from using isort where we would love to be able to make use of isort and black together.

Looks like 5.0.0 is releasing tomorrow: https://github.com/timothycrosley/isort/commit/2ea917e8dad7116584c0c5927b43f25ca300f819


Android notifications shows profile picture of webhook creator instead of custom webhook picture

Summary
Android notifications shows profile picture of webhook creator instead of custom webhook picture
Environment Information

Device Name: OnePlus 2
OS Version: 10
Mattermost App Version: 1.46.0
Mattermost Server Version: latest

Steps to reproduce

Create an incoming webhook with account A
In the webhook, go to "Profile Picture", and paste a URL of an image there
Send a message with that webhook to a channel user B is in
Look at the Android notifications of user B

Expected behavior
I expect the image in the notification user B gets from the webhook to be the custom image (as it is on desktop or when you open the app).
Observed behavior (that appears unintentional)
It actually is the profile picture of the person who created the webhook (person A)
Possible fixes
This line doesn't check for "override_icon_url". https://github.com/mattermost/mattermost-mobile/blob/c784086595bd1a43f3ab686efa6f1bae88ca92f6/android/app/src/main/java/com/mattermost/rnbeta/CustomPushNotificationHelper.java#L83
Please also check if the username is set correctly when choosing "Override username" (I think the behaviour might be similar)
Since this issue is in the native part of the code, it might also be present on iOS.

Great catch @redfast00 iOS does not have this issue as it does not display the user profile image.
Now that you identified the issue, would you be able to submit a PR to include override_icon? The potential problem that I see here is to follow the server config that is not available native side to check for the EnableIconOverride setting.

No, with the new academic year starting, I unfortunately don't have the time to get worked into the codebase

Ticket: https://mattermost.atlassian.net/browse/MM-38660.

This is fixed in Mattermost Mobile V2 Beta. Please see more details here on how to sign up as a beta tester: https://mattermost.com/blog/mobile-beta-program-v2/.


can't compile with ipp

installed version of Intels Performance Primitives is 2018.1.163
compile error : no rules for target libippvm_l.a
System: Debian Stretch x64

Don't know if anyone has used that feature in quite some time.
Anyhow, a bit more of information could be useful to resolve the issue.

compile error : no rules for target libippvm_l.a

It seems like your  IPP version is too modern :).
The script cmake/FindIPP.cmake dates from 2010 and expects IPP 5.X - 7.X,
see https://software.intel.com/en-us/forums/intel-integrated-performance-primitives/topic/487367


Support multiple time zone

Is your feature request related to a problem? Please describe.
Support multiple time zone
Solution:

Setting of time zone related parameters:
mainly involves two parameters "system_time_zone" and "time_zone".
"system_time_zone" is obtained by calling the system interface and cannot be modified.
"time_zone" parameter can be modified. To be compatible with historical logic, the default setting is CST.
Time zone values can be given in several formats, no case sensitive:

"SYSTEM", indicating that the server time zone is the same as the system time zone
A string representing the UTC offset, such as '+10:00' or '-06:00'
Standard time zone format such as "Asia/Shanghai", "America/Los_Angeles"
Abbreviated time zone formats such as "MET", "CTT" are not supported. Because the abbreviated time zone is ambiguous in different scenarios, it is not recommended.
In order to be compatible with Doris, support CST abbreviated time zone, internally transfer CST to "China/Shanghai" Chinese standard time zone

Time zone in BE: Perceive the time zone by Fe, and then pass two parameters to be. One is the UTC timestamp, and the other is the time zone information.

the time recorded in the fe: such as "show backends", "show load". These time fields should be modified according to the time zone. Fe internally stores timestamps. The timestamp can be converted to a date literal by calling the TimeUtils.longToTimeString function. Inside the function, get the time zone configuration through Session variabled

@HangyuanLiu
I have a question. How can we get system_time_zone? How to keep the values on each node consistent.
And I think this can be split into some small patches.
For example:

support related user function aware of time zone.
Support time zone for our system usage.

@imay
I think system_time_zone is a parameter that represents the time zone of the machine, but the other variable "time_zone" is really takes effect.
So system_time_zone is fetched according to the machine's time zone and cannot be modified. The time_zone function is customizable in doris cluster.
And I will split this feature as three commit


fixed cli help commands, minor jsonrpc api refactor

:clap: :clap: :clap:

wait, this may not be what I thought it was... maybe only one clap was appropriate

yea that's far too many claps for what i did
On Fri, Dec 9, 2016 at 2:51 PM, Jeremy Kauffman<EMAIL_ADDRESS>wrote:

wait, this may not be what I thought it was... maybe only one clap was
appropriate
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/lbryio/lbry/pull/321#issuecomment-266106478, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAWJIERwtsWjZ0a_k9BWrtQMj7XQhjd3ks5rGbFLgaJpZM4LIe5R
.

Is this still in something being worked on? The last commit message "before getting on plane" makes it sound like there is perhaps more to come.

comment from jack:
have you tried lbrynet-cli with and without auth, with functions with and without params?
and did the change to call from __call__ introduce any bugs?

@kauffj want to finish this?

i'm not gonna get to this for at least a week, maybe more. this is not that important atm

totally gonna finish this next week


Slim Application Error when an invalid PNG file is encountered

While I was doing the initial setup, I specified a .png image that is not a valid .png image. This is the result:
[12-Jun-2023 15:38:59 Europe/Berlin] PHP Warning:  imagecreatefrompng(): "/var/www/typemill.test/_uploads/phpeUDwrD" is not a valid PNG file in /var/www/typemill.test/public_html/system/Models/ProcessImage.php on line 503
[12-Jun-2023 15:38:59 Europe/Berlin] Slim Application Error:
Type: TypeError
Message: imagesx(): Argument #1 ($image) must be of type GdImage, bool given
File: /var/www/typemill.test/public_html/system/Models/ProcessImage.php

Ok, thank you for reporting, I will add a check in one of the next releases.


Enh: clusterDecisionResource should also provide labels/annotations, like regular cluster generator

The cluster generator provides labels and annotations from the selected/generated clusters.
These can be used in the template.
The  clusterDecisionResource does only provide data provided in referenced resource from which the decision is made. It would be very beneficial if labels/annotations from the cluster resource from argo cd are provided as well.

I think this could be done with the new merge generator. Use a ClusterDecisionResource generator to as the base generator and then merge with a Cluster generator on the server param.

is there any way to use the metadata clearly than a merge?
we cannot find the proper merge to do what we want to do.
our objetive:
Use a label o annotation (environment and provider) in the clusters imported though ACM to join with a git/directory generator to build as many apps as paths we have.
our folder:
micro1 > INT
PRE
PRO
micro2 > INT
PRE
PRO


Pirate Treasure Video Volume

https://youtu.be/s2UMsHhEMfY?si=LKzmYTzUNFWyMZvL
My computer's sound is working but there's no sound in the video for me.

Saw the latest commit


Archive Blocks: process canonical blocks in chunks

resolves #1738

After the long infinality period, number of canonical blocks is 77614
Process canonical blocks in chunks to avoid OOM
Use db getBinary/batchPutBinary and forkchoice iterateNonAncestors to improve performance

I don't see OOM again with this patch, however maybe we should put onFinalizedCheckpoint in queue. Right now it runs too close to each other and all of them have to go through > 75k block summaries in forkchoice to process, and since the 1st task moves block to archiveBlock db already, the latter ones touch db for nothing.

blocked by #1758 #1759

under testing against an Medalla sync along with #1786

2020-11-23 20:48:23 [CHORES]           info: Archive Blocks epoch #20557  - duration=102053ms
2020-11-23 20:48:23 [CHORES]           info: Archiving of finalized blocks complete. totalArchived=77614, finalizedEpoch=20557
2020-11-23 20:48:23 [CHORES]           info: Started archiving states (finalized epoch #20557)...
2020-11-23 20:48:28 [SYNC]             info: Sync progress currentEpoch=20571, targetEpoch=23714, speed=2.1 slots/s, estimatedTillComplete=13.3 hours
2020-11-23 20:48:28 [CHORES]           info: Archiving of finalized states completed (finalized epoch #20557)
2020-11-23 20:48:28 [CHORES]           info: Archive States epoch #20557  - duration=4388ms
2020-11-23 20:48:30 [CHORES]           info: Finish processing finalized checkpoint for epoch # 20557

ready for review


feat(v2): composition syntax highlighting & live code editors

Motivation
Close #1546
Change how MDX component works, we can replace any component of mdx through mdx provider.
I replaced a with docusaurus/Link to make internal links navigate without a page refresh and i replaced the syntax highlighting to be done via composition (which is recommended by https://mdxjs.com/guides/syntax-highlighting)
Add new live code editor, powered by react-live, enabled with this syntax. This make it still syntax highlightable
```js live

You can try it in netlify first for now, I added simple example at /docs/introduction (need to remove before merge)
Other notes

CodeBlock and playground is swizzle-able

Have you read the Contributing Guidelines on pull requests?
yes
Test Plan
use this https://gist.github.com/endiliey/b5f230c4cb6420641875612e25c3d4e1

Related PRs
(If this PR adds or changes functionality, please take some time to update the docs at https://github.com/facebook/docusaurus, and link to your PR here.)

Gonna merge this out first, yangshun will further customize it and tweak it with new PR

Not sure if this vibes with the swizzle philosophy, but it would be great to be able to specify component blocks from config.js with just require.resolve(./myMDXComponents). The default export of that file then just gets past to MDXProvider. This is really all I wanted to customize


Update index.md

@RehanaMohammed Just doing some cleanup, and it looks like this request refers to an older version of the site. I'm going to close it, but please be in touch if there's anything you need.


Multithreaded UDP socket reading, reuseport for improved performance?

Currently reading from udp socket is slow (golang udp lib not very optimized) and can saturate a CPU core.
Should look into multithreaded udp reading and also reuseport. See https://lwn.net/Articles/542629/

statsd-proxy has REUSE https://github.com/hit9/statsd-proxy i drop less metrics with statsd-proxy + 4x nodejs pids than with statsdaemon latest.

Here is some data on statsd-proxy  vs going straight into statsdaemon. The droprate doubles once switching from statsd-proxy -> 4 classic statsd pids to statsdaemon.

https://github.com/golang/go/issues/23696
https://groups.google.com/forum/#!msg/golang-dev/HAEaU5TakTU/ZfMLVLqEBAAJ
https://go-review.googlesource.com/c/go/+/72810
It looks like go 1.11 will make this possible soon.

Nice !

any plans to implement this feature?


Question: What Python versions are supported?

On PyPI it says you support Python >=3.6 however in unit test workflow I only see 3.7 and 3.8.
What about 3.6? 3.9 (we've been using OOIPy with it for some time, though not in conda)? (Maybe even 3.10 haha?)

I am pretty certain that OOIPy should work will all python version >=3.6 (it might even work with older versions). That we run the unit tests only on 3.7 and 3.8 can probably be attributed to our laziness. Do you know a way for the unit tests to be run on all python versions at and above a specified version? So that we only have to specify the lowest version (lets say 3.6) in the unit_test.yml  and it runs the unit tests automatically for 3.6, 3.7, 3.8, ...

No, I don't think it's possible, you have to specify the version. It's not too much work really, Python releases only once a year!
But covering all versions might be tricky for another reason: you test on conda. You only have conda environment files for 3.7 and 3.8 so other versions simply won't have the corresponding file. Now to the question of why you even have different files for each version, the only difference between them is the Python version.
Testing without conda would be faster but if you want to test that everything works in conda then you'd need to create environment files for each Python version (3.6 and 3.9, 3.10 is not on conda yet).
I would suggest removing Python dependency from conda environment file and keeping only one file instead. Then testing will be possible using just one env file + specifying Python version in workflow (you already do it honestly).

I would suggest removing Python dependency from conda environment file and keeping only one file instead. Then testing will be possible using just one env file + specifying Python version in workflow (you already do it honestly).

We would like to have conda support as the scientific community often times uses conda. I think it's fine to keep the multiple conda files. The general best practice is always better to be explicit rather than implicit.

Alright, got it, so I guess you'd want 2 files added then, to test on Python 3.6 and 3.9?

That would be great! 😄 Thanks!

Closing as the question was answered and PR testing on all supported versions was merged.


Error when trying to configure ComputeSwitch in Windows Admin Center

Following instructions on https://github.com/Azure/AzureStackHCI-EvalGuide/blob/main/nested/steps/4_AzSHCICluster.md I got an error message at ComputeSwitch creation in Windows Admin Center

This is a Nested setup and I followed all instructions (except I change 192.168.0.x/24 IP range with my own 192.168.10.x/24)

Hi - please see steps here: https://github.com/Azure/AzureStackHCI-EvalGuide/issues/8 to resolve. Just run the updated PS to install the PS modules on your nodes and it should allow you to proceed.

Thanks it solves the issue on one VM not the other that seems to have a DNS / WinRM problem. I will break everything and rebuild instead of losing time trying to solve that.

Thanks - if you're seeing a WinRM issue at cluster validation, the fix is easy, and quick - see here: https://github.com/Azure/AzureStackHCI-EvalGuide/blob/main/nested/steps/4_AzSHCICluster.md#troubleshooting-cluster-validation-issues
Thanks for testing!


Feat/lint using super json

Description

This PR improves DX of Lint command
Motivation and Context

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have updated the documentation accordingly. For updating Oclif commands documentation use oclif-dev.
[x] I have read the CONTRIBUTION_GUIDE document.
[x] I haven't repeated the code. (DRY)
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

@lukas-valenta Do you want to give it a quick look? I added lint integration tests where I had to fiddle with stdout mocking and exposing it from child process I also added hidden flag to skip map to profile validation - we use lint in a lot of our tests.


test: fix slow test from previous commit

This test is still too slow for what it is doing but this makes it
acceptable for now.

@lyft/network-team @RomanDzhabarov

Made it comparable to other tests runs: https://github.com/lyft/envoy/pull/217


Conflict with support library version

Hi Mike,
When I update my project with the FastAdapter last version (from 2.1.7 to 2.5.1) in graddle app module, AS tells me that I'm including two different support library versions.
There is no problem with FastAdapter 2.1.7 !
Do you have any idea where the problem may come from?
TIA,
Nicolas

As usual I updated the support libraries with the update of the library.
You can exclude the included support libraries from the FastAdapter
Like shown here: http://stackoverflow.com/a/21765656/325479

I have done that :
 compile('com.mikepenz:fastadapter:2.5.1@aar') {
        transitive = true
        exclude group: 'com.android.support'
    }

but same problem ...
don't worry, app works anyway

you might also just overwrite it for all:
configurations.all {
    resolutionStrategy.force "com.android.support:appcompat-v7:${versions.supportLib}"
    resolutionStrategy.force "com.android.support:cardview-v7:${versions.supportLib}"
    resolutionStrategy.force "com.android.support:recyclerview-v7:${versions.supportLib}"
    resolutionStrategy.force "com.android.support:design:${versions.supportLib}"
    resolutionStrategy.force "com.android.support:percent:${versions.supportLib}"
    resolutionStrategy.force "com.android.support:preference-v14:${versions.supportLib}"
    resolutionStrategy.force "com.android.support:customtabs:${versions.supportLib}"
    resolutionStrategy.force "com.android.support:support-annotations:${versions.supportLib}"
}

It's ok now, thanks a lot :o)


Next steps for wider adoption

This Slack discussion outlines some ideas of what may be required for this library to gain wider adoption. Compiler support may be required.

unfortunately it seems like it'll take some time for something like this to percolate through the library ecosystem though, no?
given that we can't have orphan instances, so everything needs to come pre-equipped with it, or the library needs to be forked

Yeah, that’s the main obstacle

If we adopt Debug, I think it's important to have compiler deriving for it. (Maybe even special rules for allowing orphan instances)

We can derive Debug via Generic with relatively little ceremony already

I'd like to make sure we can reliably omit Debug instances from production bundles... That also means to not have any of the Generic instances that solely exist to implement Debug
I fear relying on DCE after the fact will be brittle
Maybe I'm overthinking it..

Ah I see
I think DCE is smart enough to do that already, but I’m not certain
At least purs bundle ought to be

Basically I'd like to avoid the situation where I need to add 5 instances before I can start debugging something, only to then have to delete those again
having Debug instances for everything ought to be the default, so we need to make sure it's "free"

To make this library practically usable, I very much agree that Debug should be automatically available for all types! No deriving needed, no orphans created. DCE should be powerful enough to automatically remove the generated code for every Debug instance.

I'm hesitant about having it automatically available. There are definitely some types for which the derived instance would be wrong; Map and Set are examples. Also, the library is built to make it as easy as possible to define instances. I would prefer to wait until people start using it more widely so that we can see how it's being used before we start considering compiler support.
I think the main obstacle is this: if this library is to fulfil its purpose, we need to make it a given that every type should have a Debug instance. For that to be possible, the class probably needs to be exported by the Prelude. That's hard because this library depends on a bunch of stuff which isn't in Prelude.

I've been thinking about this and while I think the eventual goal should be to have it be part of Prelude, I also think that needs to happen gradually - it's likely that we'll want to refine the design, and that's hard to do if we move it into Prelude too quickly. I think a good first step would be to move this library into core, and state that we intend to keep maintaining it as part of core, and that it should provide Debug instances for all data types in core. That way, people will be able to have more confidence in depending on it, which will allow it to gain wider adoption and hopefully provide feedback which helps us refine the design more.

Any updates or related issues on this?

I think there are still people interested in making this happen, but a) the focus recently has been on getting 0.15 out, and b) I think it’s still a bit awkward because of the dependency issues I mentioned in my previous comments.


Alpha 9 .Deb libcec4 dependency?

[x] I confirm that this is an issue rather than a question.

Bug report
I installed the alpha 9 x86_64.deb, and it complains about not finding libcec4.
Seem to be similar to https://github.com/hyperion-project/hyperion.ng/issues/1006, but it was mentioned that a new issue should be posted for alpha 9.
Steps to reproduce
sudo dpkg -i Hyperion-2.0.0-alpha.9-Linux-x86_64.deb

What is expected?
I'm assuming it should need libcec6 instead.
What is actually happening?
The following packages have unmet dependencies:
    hyperion : Depends: libcec4 but it is not installable

It still works though, but because of the apt error, I have to keep it held.
EDIT: I'm not getting this error after building from source.
System

Hyperion Server:
- Build:           (HEAD detached at 83338dc3) (Paulchen Panther-ca50487e/83338dc3-1605733041)
- Build time:      Nov 18 2020 21:01:42
- Git Remote:      https://github.com/hyperion-project/hyperion.ng
- Version:         2.0.0-alpha.9
- UI Lang:         auto (BrowserLang: en-GB)
- UI Access:       default
- Avail Capt:      v4l2,framebuffer,x11,xcb,qt
- Database:        read/write

Hyperion Server OS:
- Distribution:   Ubuntu 20.10
- Architecture:   x86_64
- CPU Model:      Intel(R) Pentium(R) CPU G4400 @ 3.30GHz
- CPU Type:       94
- Kernel:         linux (5.8.0-33-generic (WS: 64))
- Qt Version:     5.7.1
- Python Version: 3.5.3
- Browser:        Mozilla/5.0 (Linux; Android 9; SM-T585) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101

Same after docker-compile.sh installation, when u try to run hyperiond :)
./hyperiond: error while loading shared libraries: libcec.so.4: cannot open shared object file: No such file or directory

It is fixed. The following two commands are required to install the Hyperion deb file:
sudo dpkg -i /path/to/hyperion_file.deb
sudo apt-get install -f
Why the second command is necessary is described here:
https://unix.stackexchange.com/a/159103

I did try that, but when I did, it wanted to remove hyperion altogether.

I have experienced the same issue.
Raspberry Pi 4b 8Gb + Ubuntu Desktop 20.10.
libcec6 and libcec-dev was installed.
Running sudo dpkg -i Hyperion-2.0.0-alpha.9-Linux-aarch64.deb errors out due to missing libcec4:
Selecting previously unselected package hyperion.
(Reading database ... 182396 files and directories currently installed.)
Preparing to unpack Hyperion-2.0.0-alpha.9-Linux-aarch64.deb ...
---Hyperion ambient light preinst ---
Unpacking hyperion (2.0.0) ...
dpkg: dependency problems prevent configuration of hyperion:
 hyperion depends on libcec4; however:
  Package libcec4 is not installed.

dpkg: error processing package hyperion (--install):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 hyperion

Following sudo apt-get install -f prompts/ removes Hyperion:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  cryptsetup-bin dctrl-tools dmeventd dmraid dpkg-repack efibootmgr
  gir1.2-timezonemap-1.0 gir1.2-xkl-1.0 grub-common grub-efi-arm64
  grub-efi-arm64-bin grub-efi-arm64-signed grub2-common kpartx kpartx-boot
  libdebian-installer4 libdevmapper-event1.02.1 libdmraid1.0.0.rc16
  liblvm2cmd2.03 libreadline5 libtimezonemap-data libtimezonemap1 lvm2
  os-prober python3-icu python3-pam rdate thin-provisioning-tools
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  hyperion
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 208 MB disk space will be freed.
Do you want to continue? [Y/n]

I ran into the same problem trying to install Hyperion on Raspberry Pi OS 64bit yesterday.
Can we reopen the issue?

I installed with this commands:
sudo apt install libcec6
sudo dpkg -i --force-depends ./Hyperion-2.0.0-alpha.9-Linux-x86_64.deb

I installed with this commands:
sudo apt install libcec6
sudo dpkg -i --force-depends ./Hyperion-2.0.0-alpha.9-Linux-x86_64.deb

I believe this still requires a fix.
Ubuntu 20.10 comes with libcec6, while hyperion.ng-alpha-9 seems to still require libcec4:
$ sudo dpkg -i Hyperion-2.0.0-alpha.9-Linux-x86_64.deb
Selecting previously unselected package hyperion.
(Reading database ... 252659 files and directories currently installed.)
Preparing to unpack Hyperion-2.0.0-alpha.9-Linux-x86_64.deb ...
---Hyperion ambient light preinst ---
Unpacking hyperion (2.0.0) ...
dpkg: dependency problems prevent configuration of hyperion:
 hyperion depends on libcec4; however:
  Package libcec4 is not installed.

dpkg: error processing package hyperion (--install):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 hyperion

Yes, installing with --force-depends forces it to install properly, but then one is not able to use apt anymore:
$ sudo apt upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 hyperion : Depends: libcec4 but it is not installable
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

I haven't found any other way than to uninstall Hyperion in order to use apt again.
@Paulchen-Panther Can we reopen this issue and solve this by e.g. making deb package require either libcec4 or libcec6 (which for Hyperion purpose seem to be compatible, because despite these dependencies problem, software works just fine)?
And by the way, big THANK YOU for this great piece of software. :pray:

The bug fix has been in the master since December 27, 2020.
Proof: https://prnt.sc/zgp74o


Grid validity check

Hello, I want to help add a feature to automatically check the grid validity at the end and graphically flag any cells with problems. The shapely library makes it easy to check that cells are not twisted (polygon.is_valid) and that they touch but don't overlap their neighbors (polygon1.touches(polygon2)). I've added the is_valid check, but the overlap check is more subtle because the poloidal indices jump at target plates and X-points. Since I am not very familiar with the codebase yet, is there a simple way, compatible across the different topologies, to get the indices of the physically adjacent cells?

Hi Sean,
Yes, there is a simple way to access cells. All cells are stored as a 2D list within each patch under attribute cell_grid regardless of topology.
For example, from an INGRID object that has already made a call to CreateSubgrid, you can access cell (ix, iy) from say Patch A2 with:
ig_obj.CurrentTopology.patches['A2'].cell_grid[ix][iy]

Bryan


docs(JavaScript): Add build and test documentation for JavaScript

Add build and test and environment requirements documentation for JavaScript.
Remove the Title Icon in a and keep it consistent with other Title styles (or add all Icon, here I chose to remove the Icon).

Hello everyone, ptal.


Crash while scanning Ferris Ingots

Getting a crash while trying to scan Ferris ingots from Thermal Expansion:
http://pastebin.com/vep8YEKH

What version?

That crash is related to witching gadgets, turn off transmutation in witching gadgets config file

Either way it should already had been fixed in version v1.0.11 which came out in March.


Add a MinMax scaler

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Added MinMax scaler, as is it can't be used in a framework.


move compile/target sdk versions to 29 (#62799)

Unrevert https://github.com/flutter/flutter/pull/62884
Fixed revert by fixing permissions on devicelab. Also add checks in doctor validation.
Fixes https://github.com/flutter/flutter/issues/54382

@fkorotkov do you know what's up with these tests? Do we need to install Java ourselves on these linux dockers?

I think @jmagman provided some insights on Discord about the issue. Please let me know if I can help you somehow.

I think @jmagman provided some insights on Discord about the issue. Please let me know if I can help you somehow.

The test has Android SDK 28.0.3 installed, you bumped the requirement to 29, but there will be a doctor error until there's some automatic download building the Android app.  And the doctor validation fails in setup too.

🤦🏻♂ yup.
Moving the doctor check to another PR since updating the min required SDK for all projects (and modifying Cirrus's docker file) vs updating the template which self-upgrades are 2 different semantic steps anyway.

Is there a reason you didn't choose API 30 which is the latest?

@bernaferrari see https://github.com/flutter/flutter/issues/56597


[ENG-3867] Garden Variety Pickle

Avoid dill
Now that our states are simpler, we can pickle them using the built in c-implementation which is much faster.
Update __module__ for ComponentState
When we create component state substates, cache these generated classes on reflex.state so that the pickler can find them.
Reduce state instantiations
The StateManagerDisk was creating a new state tree for every get_state call, which is a very expensive operation. Instead pull the root state from the self.states cache if it's there.
Simplify loading logic to make it clear that we only populate states on initial load where a client token is not saved in memory.
 Optimize output by only serializing states where _get_was_touched() returns True Moved to a different PR
Bonus
Redis now throws away state when the schema changes, allowing for smoother app updates.
Profile
This profiles 4000 operations against the state. The operation causes the state to grow linearly in size.
"""Measuring round-trip time of different StateManagers."""
import asyncio
import contextlib
import uuid

import async_timeit
from redis.asyncio import Redis

import reflex as rx
from reflex.state import StateManagerMemory, StateManagerDisk, StateManagerRedis, _substate_key

OPS = 4000
TOKEN = str(uuid.uuid4())

class MyState(rx.State):
    d_int: int = 0
    d_str: str = ""
    d_list: list = []
    d_dict: dict = {}
    d_set: set = set()

    def ev(self):
        self.d_int += 1
        self.d_str = str(self.d_int)
        self.d_list.append(self.d_int)
        self.d_dict[self.d_int] = self.d_int
        self.d_set.add(self.d_int)

async def test_state_manager(clz=StateManagerMemory, **kwargs):
    sm = clz(state=rx.State, **kwargs)

    async def modify_state():
        async with sm.modify_state(_substate_key(TOKEN, MyState)) as s:
            (await s.get_state(MyState)).ev()

    print(await async_timeit.timeit(modify_state, number=OPS))

PROFILE = False

@contextlib.contextmanager
def profile(filename):
    if not PROFILE:
        yield
        return
    import cProfile
    profiler = cProfile.Profile()
    profiler.enable()
    yield
    profiler.disable()
    profiler.dump_stats(filename)

if __name__ == "__main__":
    with profile("statemanagermemory.prof"):
        print("StateManagerMemory", end=": ")
        asyncio.run(test_state_manager())

    with profile("statemanagerdisk.prof"):
        print("StateManagerDisk", end=": ")
        asyncio.run(test_state_manager(clz=StateManagerDisk))

    with profile("statemanagerredis.prof"):
        print("StateManagerRedis", end=": ")
        asyncio.run(test_state_manager(clz=StateManagerRedis, redis=Redis.from_url("redis://localhost")))

async_timeit.py
Before (0.6.1)
(VENV-release311) masenf@minicone pickle-perf % python timing.py
StateManagerMemory: 1.048973000026308
StateManagerDisk: 30.536243582959287
StateManagerRedis: 35.89346333395224

After
(VENV-fd) masenf@minicone pickle-perf % python timing.py
StateManagerMemory: 1.0538166250335053
StateManagerDisk: 3.6349450410343707
StateManagerRedis: 11.734799916972406

Removed the StateManagerDisk optimizations, since they are breaking some integration tests.


Support dropping full XP

Currently, the data pack follows the game's logic for dropping XP (levels lost * 7), but the function that actually spawns the XP doesn't care - it just takes a number of points and creates the orbs. I will make a new function that calculates the correct amount of points to get a player at level 0 to that level of XP.

This is another case where return run is useful, so I'm going to look into dropping support for versions older than 1.20.3

Dropping support for versions older than 1.20.5 since Curse of Vanishing support depends on that. This allows me to remove the runtime check.


Score / Color is not initialized correctly on bookmarks panel

On a fresh install, the global preference to choose between score and color is not initialized correctly and so the user sees a broken UI. Need to choose a default if that key is not found in storage.

Adding screenshot:


Actor Artwork Not Updating / Image Browser Not Opening

Ran into this issue, running 5e 3.2.0, Foundry version 12 stable 325.
Up until the last update I've been able to open an NPC (or any other) actor sheet from my sidebar, click their sheet's portrait, and update the token art through the image browser.
I am now seemingly unable to do so (image browser does not open from the actor sheet portrait anymore) and have yet to find a way to update how the actor's art appears in the sidebar and in their sheet.

(In the image above, Commandant Neave and Commandant Nadia both have the same appearance in the sidebar and actor sheets despite having different image sources for any art.)
Upon further testing, I am now unable to update any of the sidebar art that appears (nor the art on each actor's sheet). If I adjust the prototype token art source, it shows the tile correct once placed on the table, but before then it is not visually updated. I also tried creating a new actor but was unable to adjust the art on it either.
I've restarted my hosting computer and Foundry multiple times trying to remedy this issue before I submitted this note.

(In the image above, Commandant Nadia's placed tile appears correctly while her sidebar image and sheet image are still both incorrect.)
Please advise if I'm doing something wrong, if a process updated, or if this is a true bug. I tried finding another issue with this but was unable to locate any others.

Please disable all modules, and then head over to the Foundry discord server for general troubleshooting support.

Thanks @unsoluble, it was definitely one of my modules. Now the fun begins.


Can't set focus before Application.Run - add TopLevel.Running event

One would think that you could call SetFocus when creating controls, before calling Application.Run, but it doesn't work:
			_top.SetFocus (_rightPane);
			Application.Run (_top);

I found a hacky workaround by using Application.Iteration:
...
			// HACK: There is no other way to SetFocus before Application.Run
			Application.Iteration += Application_Iteration;
			Application.Run (_top);
		}

		private static void Application_Iteration (object sender, EventArgs e)
		{
			Application.Iteration -= Application_Iteration;
			_top.SetFocus (_rightPane);
		}

I propose adding an Running event to TopLevel:
...
			_top.Running += (o, e) => _top.SetFocus (_rightPane);
			Application.Run (_top);
		}

I think it would be best called once the MainLoop starts (not before). Like this:
		public static void RunLoop (RunState state, bool wait = true)
		{
			if (state == null)
				throw new ArgumentNullException (nameof (state));
			if (state.Toplevel == null)
				throw new ObjectDisposedException ("state");

			bool firstIteration = true;
			for (state.Toplevel.Running = true; state.Toplevel.Running;) {
				if (MainLoop.EventsPending (wait)) {

					// Notify Toplevel it's running
					if (firstIteration)
						state.Toplevel?.Invoke(null, EventArgs.Empty);
					firstIteration = false;

					MainLoop.MainIteration ();
					Iteration?.Invoke (null, EventArgs.Empty);
				} else if (wait == false)
					return;
				if (!state.Toplevel.NeedDisplay.IsEmpty || state.Toplevel.childNeedsDisplay) {
					state.Toplevel.Redraw (state.Toplevel.Bounds);
					if (DebugDrawBounds)
						DrawBounds (state.Toplevel);
					state.Toplevel.PositionCursor ();
					Driver.Refresh ();
				} else
					Driver.UpdateCursor ();
			}
		}

A simpler implementation would just put it in Begin:
		static public RunState Begin (Toplevel toplevel)
		{
			if (toplevel == null)
				throw new ArgumentNullException (nameof (toplevel));
			var rs = new RunState (toplevel);

			Init ();
			if (toplevel is ISupportInitializeNotification initializableNotification &&
			    !initializableNotification.IsInitialized) {
				initializableNotification.BeginInit();
				initializableNotification.EndInit();
			} else if (toplevel is ISupportInitialize initializable) {
				initializable.BeginInit();
				initializable.EndInit();
			}
			toplevels.Push (toplevel);
			Current = toplevel;
			Driver.PrepareToRun (MainLoop, ProcessKeyEvent, ProcessKeyDownEvent, ProcessKeyUpEvent, ProcessMouseEvent);
			if (toplevel.LayoutStyle == LayoutStyle.Computed)
				toplevel.RelativeLayout (new Rect (0, 0, Driver.Cols, Driver.Rows));
			toplevel.LayoutSubviews ();
			OnLoad?.Invoke ();
			toplevel.WillPresent ();
			Redraw (toplevel);
			toplevel.PositionCursor ();
			Driver.Refresh ();

			// Notify Toplevel it's running
			toplevel?.Invoke(null, EventArgs.Empty);

			return rs;
		}

You can change the OnLoad for that without having multiple actions to invoke. Remember that are static actions, so the last caller supersede the others.

I think the best solution would be to implement ISupportInitializeNotification and ISupportInitialize in Toplevel and its derivatives.
if (toplevel is ISupportInitializeNotification initializableNotification &&
	!initializableNotification.IsInitialized) {
	initializableNotification.BeginInit();
	initializableNotification.EndInit();
} else if (toplevel is ISupportInitialize initializable) {
	initializable.BeginInit();
	initializable.EndInit();
}

Why would implementing an interface be better? This is far more cumbersome.

It's in the Core for some reason. So there could be personalized notification for each top-level regardless of where the call was made, as it would be more object oriented. I agree that it would be more complicated but more rationally functional.

I still don't see the value in adding all the complexity of an interface. This is a notification that simple apps want to receive. Perhaps, under the covers there could be an interface.
Another reason for not doing it with an interface: There are other notifications we're going to want over time. If we do each one via implementing an interface, before long Toplevel will end up looking like this:
 public Toplevel : View, ISupportInitializeNotification, IInterface2, IInterface3, ...

My current proposal is below.
public class Toplevel : View {
		/// <summary>
		/// Gets or sets whether the <see cref="T:Terminal.Gui.MainLoop"/> for this <see cref="T:Terminal.Gui.Toplevel"/> is running or not. Setting
		/// this property to false will cause the MainLoop to exit.
		/// </summary>
		public bool Running { get; set; }

		/// <summary>
		/// Fired once the Toplevel's <see cref="T:Terminal.Gui.MainLoop"/> has started it's first iteration.
		/// Subscribe to this event to perform tasks when the <see cref="T:Terminal.Gui.Toplevel"/> has been laid out and focus has been set.
		/// changes. A Ready event handler is a good place to finalize initialization after calling `<see cref="T:Terminal.Gui.Application.Run"/>(topLevel)`.
		/// </summary>
		public event EventHandler Ready;

		/// <summary>
		/// Called from Application.RunLoop after the Toplevel has entered it's first iteration of the loop.
		/// </summary>
		internal virtual void OnReady ()
		{
			Ready?.Invoke (this, EventArgs.Empty);
		}

...

		public static void RunLoop (RunState state, bool wait = true)
		{
			if (state == null)
				throw new ArgumentNullException (nameof (state));
			if (state.Toplevel == null)
				throw new ObjectDisposedException ("state");

			bool firstIteration = true;
			for (state.Toplevel.Running = true; state.Toplevel.Running;) {
				if (MainLoop.EventsPending (wait)) {
					// Notify Toplevel it's ready
					if (firstIteration) {
						state.Toplevel.OnReady ();
					}
					firstIteration = false;

					MainLoop.MainIteration ();
					Iteration?.Invoke (null, EventArgs.Empty);
				} else if (wait == false)
					return;
				if (!state.Toplevel.NeedDisplay.IsEmpty || state.Toplevel.childNeedsDisplay) {
					state.Toplevel.Redraw (state.Toplevel.Bounds);
					if (DebugDrawBounds)
						DrawBounds (state.Toplevel);
					state.Toplevel.PositionCursor ();
					Driver.Refresh ();
				} else
					Driver.UpdateCursor ();
			}
		}

Example usage:
	_top.Ready += (o, e) => _top.SetFocus (_rightPane);

I've implemented this and tested it for my scenario and it works dandy.

It's looks good since it only will be raised in the first interaction of that top-level only. Great.

Closed with #446


pip install langflow installs v0.0l78 instead of >==0.5.0 and has very few modules.

Describe the bug
pip install langflow installs v0.0.78 instead of >==0.5.0 and has very few modules.
Browser and Version

Safari
Version [e.g. 22]

To Reproduce
Steps to reproduce the behavior:

Open terminal.
pip install langflow
Open http://<IP_ADDRESS>:7860/ in a browser.
Note the version on the lower left.

Screenshots

Additional context
When I enter langflow --help in the terminal, it has only a few options and is missing many of the options that are on the github page.

I tried pip install --no-cache-dir langflow and it still installed v0.0.78

@jeffhaskin I tried installing langflow[local] just now and it started at version 0.2.7 and kept going back looking for a [local] version and never found it. After examining what the python requirements were for older versions of langflow versus current versions it seems earlier versions had requirements that included python<3.12, while current langflow versions were python<3.11. I was trying 3.11, so after installing 3.10 and using the python.exe -m venv for 3.10 it finally installed the correct version of langflow. Not sure if that will resolve your issue, but thought I'd share.

@jeffhaskin I tried installing langflow[local] just now and it started at version 0.2.7 and kept going back even further looking for a [local] version and never found it. After examining what the python requirements were for older versions of langflow versus current versions at pypi.org it seems earlier versions had requirements that included python<3.12, while current langflow versions were python<3.11. I was trying 3.11, so after installing 3.10 and using the python.exe -m venv for 3.10 it finally installed the correct version of langflow. Not sure if that will resolve your issue, but thought I'd share.

@sgjohnson1981 I'll try that. I think I was able to get that to happen with an earlier python version as well (3.9.13 or something like that), but then it didn't have any components in it. I'll try 3.10.


Question: Using Zuul for Internal Testing in Production

As part of our production deployment workflow, we would like to introduce a small cluster of new version of a service and we would like to perform an internal smoke test against that service before increasing the capacity and open it up to public.
My understanding is Zuul can be used for it but I can't figure out the best way to do it.
Do we need to deploy the new version of the service using a different name in Eureka (e.g. ServiceX-V2) and have Zuul route the internal requests to that service?
Or do we deploy a new Zuul and Eureka instance alongside the new service, leave that new Zuul out from the LoadBalancer and perform the internal test against that full stack?

The first option of deploying the new service with a new name (VIP) in eureka is the standard way of achieving what you are trying to do. Zuul can then route specific requests to this new cluster and the remaining requests to the old cluster.

If I have a new version of a micro service down the line (zuul->API->ms_a->ms_b->ms_c) like ms_b and I want initially only traffic from my test client to route to ms_b_v2 (if all goes well I will start allowing live traffic as well). How must I configure my system so that the calls from ms_a are routed to ms_b_v2 only for specific population. Do I have to have another Zuul between ms_a and ms_b?

I have same thing like @ervansetiawan. waiting some detail.


Kanada text printing as ?

I have to print Kanada on paper, Actually, the printer is able to print Kanada but when I try to print using datecs-printer it is printing ?'s . or some unknow letters are printing when i tried to use encoding or utf-8 type

Hello can you get the solution? I try to print Arabic  but get the text just like your picture