Include runtime scoped dependencies in main classloader

Discussed in https://github.com/sormuras/junit-platform-maven-plugin/discussions/75

Originally posted by zregvart July 27, 2021
Seems like dependencies scoped as runtime are not included in the main classloader. For example if I addch.qos.logback:logback-classic dependency in runtime scope, org.slf4j:slf4j-api in compile scope will not detect it and will issue a warning about missing org.slf4j.impl.StaticLoggerBinder class.
The only workaround I've found is to run in isolation of MERGED or NONE. I've created an example to help illustrate this. When running with 1.1.2 I get:
$ mvn test
...
[INFO] --- junit-platform-maven-plugin:1.1.2:launch (injected-launch) @ junit-maven-runtime ---
[INFO] Launching JUnit Platform 1.7.2...
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
...

And if I change the isolation to MERGED or NONE I get:
$ mvn test
...
[INFO] --- junit-platform-maven-plugin:1.1.2:launch (injected-launch) @ junit-maven-runtime ---
[INFO] Launching JUnit Platform 1.7.2...
12:42:31.608 [pool-3-thread-1] INFO  io.github.zregvart.Log - Hello from main
12:42:31.616 [pool-3-thread-1] INFO  i.g.zregvart.RuntimeClasspathTest - Hello from test
...

The real issue I'm having with this is that I would like to declare some dependencies in runtime scope so the main code doesn't depend on them at compile time, and have the implementation discovered via service loader mechanism at runtime.
This might be a conscious decision and I'm wondering about the reason behind this?

Before applying the suggested change:
Isolator Path Layering
main
  classes
  slf4j-api-1.7.25.jar
test
  test-classes
  logback-classic-1.2.3.jar
  logback-core-1.2.3.jar
  junit-jupiter-api-5.7.1.jar
  apiguardian-api-1.1.0.jar
  opentest4j-1.2.0.jar
  junit-platform-commons-1.7.1.jar
launcher
  junit-platform-launcher-1.7.1.jar
  junit-platform-engine-1.7.1.jar
  junit-platform-reporting-1.7.1.jar
  junit-jupiter-engine-5.7.1.jar
isolator
  junit-platform-isolator-worker-1.0.0-M10.jar
  junit-platform-isolator-1.0.0-M10.jar

After applying the suggested change:
Isolator Path Layering
main
  classes
  slf4j-api-1.7.25.jar
  logback-classic-1.2.3.jar
  logback-core-1.2.3.jar
test
  test-classes
  junit-jupiter-api-5.7.1.jar
  apiguardian-api-1.1.0.jar
  opentest4j-1.2.0.jar
  junit-platform-commons-1.7.1.jar
launcher
  junit-platform-launcher-1.7.1.jar
  junit-platform-engine-1.7.1.jar
  junit-platform-reporting-1.7.1.jar
  junit-jupiter-engine-5.7.1.jar
isolator
  junit-platform-isolator-worker-1.0.0-M10.jar
  junit-platform-isolator-1.0.0-M10.jar

Seems to work and does not break existing integration tests.


Can Vue accept http Post Request?

What problem does this feature solve?
I am using Vue JS application which needs Payment Gateway integration. The Payment Gateway, after processing the payment, will send a post response to the return url of the application.
I am getting 405 Method not allowed error when the response from the payment gateway returns the control back to the application.
How should I handle the Post Response  for the url path of Vue application.
Thanks & Regards,
Kuldip Shelke
What does the proposed API look like?
I should be able to access the VueJS application using http Post method.

How did you resolve this? I currently have a similar problem


Feature name poll

Yesterday there was a virtual meeting in which different languages with template capabilities use different terminologies.   C++ has the term 'concept', which is a 'trait' in rust, and  'constraint' in yet other languages.
Ultimately J3/WG5 will have to agree on a term, but the generics subgroup needs a term that we can  agree upon for preliminary discussions and prototyping.   I really dislike the word 'concept' in this context.     I'm strongly in favor of 'constraint' but am also happy with 'trait' and could possibly come around to others.  (In the discussion yesterday, I think that 'trait' was the most popular, but we did not take a poll.)
This issue is to solicit opinions from the community.    Ultimately the generics subgroup of J3 will decide, but we'd likely go with majority opinion.   Please only vote if you have a strong opinion.   (Bonus points if you articulate your reasoning.)

I am in favour of trait and these are the reasons:

The term "concept" is too general and too often used in everyday language
to be able to distinguish and "enclose" what we mean
The term "constraint" is too much focused on excluding things
The term "trait" is much more neutral in that respect than "constraint"
and much more specific than "concept". I know it from a
biological/ecological context and to me (*) it seems the near-perfect term
to indicate a characteristic that is shared by several
individuals/species/...

(*) Caveat: English is not my native tongue, so the word may "taste"
differently for me than for someone born and raised in an Anglophone
environment.
Op vr 23 okt. 2020 om 15:23 schreef Tom Clune<EMAIL_ADDRESS>
Yesterday there was a virtual meeting in which different languages with
template capabilities use different terminologies. C++ has the term
'concept', which is a 'trait' in rust, and 'constraint' in yet other
languages.
Ultimately J3/WG5 will have to agree on a term, but the generics subgroup
needs a term that we can agree upon for preliminary discussions and
prototyping. I really dislike the word 'concept' in this context. I'm
strongly in favor of 'constraint' but am also happy with 'trait' and could
possibly come around to others. (In the discussion yesterday, I think that
'trait' was the most popular, but we did not take a poll.)
This issue is to solicit opinions from the community. Ultimately the
generics subgroup of J3 will decide, but we'd likely go with majority
opinion. Please only vote if you have a strong opinion. (Bonus points if
you articulate your reasoning.)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/j3-fortran/generics/issues/20, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAN6YR3YB2XYLRE4G5FNIN3SMF7U7ANCNFSM4S4RX7XA
.

Hmm.    I see traits (by whatever means) as a mechanism to essentially express the sentiment:  "Allow any type here, so long as the specified interfaces exist for that type."   That is exactly what a constraint is.    Anything that satisfies a constraint is allowed, and anything that does not satisfy a constraint is disallowed.
I agree that the term "trait" suggests that it is shared by multiple entities.  But is that an important property here?      Traits that can be reused are of course useful, but so are constraints that are reusable
I suppose I might feel differently if we were to actually specify traits when the types themselves are defined.   But the existing examples from other languages, and my poor understanding of same, suggest that traits are typically (always?) expressed in templates and the compiler is the one that checks to see if the trait applies to any given type.   I.e., the compiler checks to see if the type has the trait.   Sounds like a constraint to me.    If traits instead were a property declared in the definition of a type, then word would be closer to the conventional meaning.
For fun I googled 'trait' in the context of biology.  The first example that came up was eye color.    Here, trait is not a specific color (which would be the phenotype: blue, brown, hazel, ...), but the generic class of eye colors.    From that perspective, "trait" seems just a bit off to me.    Phenotype would be more precise for our use, but I'm not going to propose that alternative.  To be fair, we may be only interested in "traits" that have a single phenotype and the distinction is unwarranted.
Again - not strongly opposed to using 'trait'.   It at least has the right connotations in my mind.
While English is my native language, I feel that I've still expressed myself poorly here.  I may come back with more refined arguments.  But only if we have a healthy debate going.
Cheers.

@tclune, since you asked "But is that an important property here?" I will suggest the term "Property" as a possible keyword too. :)

The general idea is that of a requirement or prerequisite or expectation or "job qualification" or skill that must be satisfied by an actual type parameter.  I don't think any of the terms that are currently in use by other languages are terribly intuitive for this purpose.  I am a native English speaker, if that matters.
I find the terms "concept" and "trait" to be so vague as to be meaningless until you learn their special definitions in the languages that use them -- one might as well use "idea" or "aspect" instead, and I hope that they seem as bad to you as they do to me.  Please avoid all of these.
I don't mind Haskell's "constraint" as much as I do "concept" & "trait", but I don't think that it conveys its meaning of "must be a member of a particular typeclass" very well either.  Similarly, ML's use of the word "signature" isn't terribly intuitive, and it seems to mean "function prototype" to many (younger) programmers.
So I encourage you to find a new term in the absence of a compelling precedent.  There's lots of great alternatives.

Myself I am partial to "requirement", but don't mind "trait". I think "concept" is terrible.

Not terribly opposed to "property".   It points in the right direction but is a bit of an all purpose word.    Similar to "attribute" in that sense.   Seems safe in the Fortran context though.
@wclodius2 Thanks for the suggestion.   I like "requirement" as well.    I consider it roughly equivalent to "constraint" in my mental map, and has the extra advantage that it is not a meta-term in the standard.   And if we end up needing a verb,  saying that X requires Y reads very clearly.

I like “constraint” or “requirement.”  Both are used in programming by contract and I suspect/hope that what we do will have similarities to that approach.  I think these terms are also used in formal methods, which I hope will inspire some of what we do.
I find “concept” vague and worry that the C++ community is so much bigger than the Fortran community that people will expect our approach to mirror theirs if we borrow their terms.
I’m ok with “trait” or “property.”  I think of them as synonyms for “attribute.”   Whether that is a feature or a bug depends on how similar what we are describing is to what we currently call an “attribute.”  I suspect that what we are looking to do is quite dissimilar from current attributes.
Damian
Sent from my iPhone

On Oct 24, 2020, at 06:00, Tom Clune<EMAIL_ADDRESS>wrote:

Not terribly opposed to "property". It points in the right direction but is a bit of an all purpose word. Similar to "attribute" in that sense. Seems safe in the Fortran context though.
@wclodius2 Thanks for the suggestion. I like "requirement" as well. I consider it roughly equivalent to "constraint" in my mental map, and has the extra advantage that it is not a meta-term in the standard. And if we end up needing a verb, saying that X requires Y reads very clearly.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Taking a look at this again, I wonder if we aren't trying to give one name to what are effectively 3 separate aspects of the generics facility we are trying to implement. Those 3 things being:

Definition of an "interface" or "typeclass"
"Constrain"ing a template parameter to ensure it "satisfies" an "interface" or is a "member" of a "typeclass"
Declaring that a (derived) type "satisfies" an "interface" or is a "member" of a "typeclass"

@FortranFan 's work towards the KART approach has given us a good idea for the requirements of part 1. So far we have assumed that part 2 is sufficiently by specifying the name of a specific 1 for a template parameter, but we could of course expand that and give part 2 it's own name in the syntax. We have discussed allowing part 3, and I believe have mostly agreed it should be optional, but if allowed would also need it's own syntax and/or keyword.
Based on the above partitioning, I would propose we vote on each aspect, with my suggested options for each listed below. I think I would lean towards 1. typeclass, 2. restriction, 3. membership, but I think there are other combinations that would be acceptable too.

property, typeclass, requirement, concept
constraint, restriction
trait, membership, satisfaction

Please don't use the word "typeclass".  That term refers today only to a specific and distinct feature in another language.  And it limits your abstractions to types.  Consider following the classic model of Standard ML's signatures and structures and functors instead; modules correspond nicely to ML's structures, and I think you need to be working at the level of modules anyway if you want type-bound procedures to work.


get_selectors does not work for chromosomes.

The following request:
curl -X GET "http://v3.mutalyzer.nl/api/get_selectors/NC_000001.11" -H  "accept: application/json"

results in an ERETR error. It is unclear why.

The get_selectors endpoint works also for chromosomes, but the ERETR is raised if the source is down or there is any other connection error.

The get_selectors endpoint works also for chromosomes, but the ERETR is raised if the source is down or there is any other connection error.


Data privacy guidelines

Ideas:

Look at how other groups are doing this
Look for best practices published by privacy groups
In the interim proposal: Continue collecting all responses. Have CS add a disclaimer somewhere. I added this to the related issue.


[BUG] Upgrade: rancher-monitoring charts can't be upgraded

Describe the bug

This was observed after https://github.com/harvester/harvester-installer/pull/229.
rancher-monitoring and rancher-monitoring-crd ManagedCharts fail to upgrade.
The message is another operation (install/upgrade/rollback) is in progress.
fleet-local   mcc-rancher-monitoring                        0/1                       ErrApplied(1) [Cluster fleet-local/local: another operation (install/upgrade/rollback) is in progress]; mutatingwebhookconfiguration.admissionregistration.k8s.io rancher-monitoring-admission modified {"webhooks":[{"admissionReviewVersions":["v1","v1beta1"],"clientConfig":{"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJkVENDQVJ1Z0F3SUJBZ0lRUDd5MVA0K2cvRlZ5OGdaSW93bElDVEFLQmdncWhrak9QUVFEQWpBUE1RMHcKQ3dZRFZRUUtFd1J1YVd3eE1DQVhEVEl5TURFd05EQXlNRGcwTUZvWUR6SXhNakV4TWpFeE1ESXdPRFF3V2pBUApNUTB3Q3dZRFZRUUtFd1J1YVd3eE1Ga3dFd1lIS29aSXpqMENBUVlJS29aSXpqMERBUWNEUWdBRXEzQWhrdnh0CmVMUHBpOG45NmVkRmlWbGZiK0xRWHRmSVJFOUJDMmRlVmhxWjNvVUx1R0hucldqc3k0OWJMM0JVZEw0QWg4VEkKQUhORG9sbzNRVElGRHFOWE1GVXdEZ1lEVlIwUEFRSC9CQVFEQWdJRU1CTUdBMVVkSlFRTU1Bb0dDQ3NHQVFVRgpCd01CTUE4R0ExVWRFd0VCL3dRRk1BTUJBZjh3SFFZRFZSME9CQllFRlB3U0xOd2VaeE02VHZMS1dodnNxd2t5Cjc0RzNNQW9HQ0NxR1NNNDlCQU1DQTBnQU1FVUNJUUR3VmlHZEtSdXI3N2dUTElZVkxkNzZXM3N3eGtYd1I5c2gKdDBXN081azdNUUlnYmpRYWVsWEd6dnhQK1dtaHdGYWVndFdFVGJSSkQ2aXpzZVpJOWJ3c3ZQZz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=","service":{"name":"rancher-monitoring-operator","namespace":"cattle-monitoring-system","path":"/admission-prometheusrules/mutate","port":443}},"failurePolicy":"Ignore","matchPolicy":"Equivalent","name":"prometheusrulemutate.monitoring.coreos.com","namespaceSelector":{},"objectSelector":{},"reinvocationPolicy":"Never","rules":[{"apiGroups":["monitoring.coreos.com"],"apiVersions":["*"],"operations":["CREATE","UPDATE"],"resources":["prometheusrules"],"scope":"*"}],"sideEffects":"None","timeoutSeconds":10}]}; validatingwebhookconfiguration.admissionregistration.k8s.io rancher-monitoring-admission modified {"webhooks":[{"admissionReviewVersions":["v1","v1beta1"],"clientConfig":{"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJkVENDQVJ1Z0F3SUJBZ0lRUDd5MVA0K2cvRlZ5OGdaSW93bElDVEFLQmdncWhrak9QUVFEQWpBUE1RMHcKQ3dZRFZRUUtFd1J1YVd3eE1DQVhEVEl5TURFd05EQXlNRGcwTUZvWUR6SXhNakV4TWpFeE1ESXdPRFF3V2pBUApNUTB3Q3dZRFZRUUtFd1J1YVd3eE1Ga3dFd1lIS29aSXpqMENBUVlJS29aSXpqMERBUWNEUWdBRXEzQWhrdnh0CmVMUHBpOG45NmVkRmlWbGZiK0xRWHRmSVJFOUJDMmRlVmhxWjNvVUx1R0hucldqc3k0OWJMM0JVZEw0QWg4VEkKQUhORG9sbzNRVElGRHFOWE1GVXdEZ1lEVlIwUEFRSC9CQVFEQWdJRU1CTUdBMVVkSlFRTU1Bb0dDQ3NHQVFVRgpCd01CTUE4R0ExVWRFd0VCL3dRRk1BTUJBZjh3SFFZRFZSME9CQllFRlB3U0xOd2VaeE02VHZMS1dodnNxd2t5Cjc0RzNNQW9HQ0NxR1NNNDlCQU1DQTBnQU1FVUNJUUR3VmlHZEtSdXI3N2dUTElZVkxkNzZXM3N3eGtYd1I5c2gKdDBXN081azdNUUlnYmpRYWVsWEd6dnhQK1dtaHdGYWVndFdFVGJSSkQ2aXpzZVpJOWJ3c3ZQZz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=","service":{"name":"rancher-monitoring-operator","namespace":"cattle-monitoring-system","path":"/admission-prometheusrules/validate","port":443}},"failurePolicy":"Ignore","matchPolicy":"Equivalent","name":"prometheusrulemutate.monitoring.coreos.com","namespaceSelector":{},"objectSelector":{},"rules":[{"apiGroups":["monitoring.coreos.com"],"apiVersions":["*"],"operations":["CREATE","UPDATE"],"resources":["prometheusrules"],"scope":"*"}],"sideEffects":"None","timeoutSeconds":10}]}
fleet-local   mcc-rancher-monitoring-crd                    0/1                       ErrApplied(1) [Cluster fleet-local/local: another operation (install/upgrade/rollback) is in progress]

To Reproduce
Steps to reproduce the behavior:

Setup a 1.0.0 cluster.
Upgrade with master ISO.

Expected behavior

Support bundle

Environment:

Harvester ISO version:
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630):

Additional context
Add any other context about the problem here.

Note that this might be related to https://github.com/rancher/fleet/issues/637.

I'm still seeing harvester chart goes into this state occasionally.
Even I already pause the managed chart and wait for all Rancehr stuff to settle down.
A trick to get rid of the state is to rollback the chart by using helm command. The managedchart will be applied again.

@johnliu55tw also saw this after an upgrade.

I am debuging #2013, and encountered:
In a single-node harvester with master-head ISO, without upgrade, this error is also observed.
The main operation I am doing is:
delete fleet-agent POD and set the new POD image to debug image.
kubectl set image pod fleet-agent-8649b7d5b6-b5lzx -n cattle-fleet-local-system *=rancher/fleet-agent:dev
Suspect: when previous fleet-agent has some helm operations ongoing, and the POD is deleted, then the new fleet-agent POD may complain Error: another operation (install/upgrade/rollback) is in progress
$ kubectl get bundledeployment -A

NAMESPACE                                NAME                                          DEPLOYED                                                             MONITORED   STATUS
cluster-fleet-local-local-1a3d67d0a899   fleet-agent-local                             True                                                                 True
cluster-fleet-local-local-1a3d67d0a899   local-managed-system-agent                    True                                                                 True
cluster-fleet-local-local-1a3d67d0a899   mcc-harvester                                 True                                                                 True        configmap.v1 longhorn-system/longhorn-storageclass missing
cluster-fleet-local-local-1a3d67d0a899   mcc-harvester-crd                             True                                                                 True
cluster-fleet-local-local-1a3d67d0a899   mcc-local-managed-system-upgrade-controller   True                                                                 True
cluster-fleet-local-local-1a3d67d0a899   mcc-rancher-monitoring                        Error: another operation (install/upgrade/rollback) is in progress   True        mutatingwebhookconfiguration.admissionregistration.k8s.io rancher-monitoring-admission modified {"webhooks":[{"admissionReviewVersions":["v1","v1beta1"],"clientConfig":{"caBundle":"LS..tLQo=","service":{"name":"rancher-monitoring-operator","namespace":"cattle-monitoring-system","path":"/admission-prometheusrules/mutate","port":443}},"failurePolicy":"Ignore","matchPolicy":"Equivalent","name":"prometheusrulemutate.monitoring.coreos.com","namespaceSelector":{},"objectSelector":{},"reinvocationPolicy":"Never","rules":[{"apiGroups":["monitoring.coreos.com"],"apiVersions":["*"],"operations":["CREATE","UPDATE"],"resources":["prometheusrules"],"scope":"*"}],"sideEffects":"None","timeoutSeconds":10}]}
cluster-fleet-local-local-1a3d67d0a899   mcc-rancher-monitoring-crd                    True                                                                 True

The realted debug info:
Helm relases are stored in kube secrets, the last release of rancher-monitoring, is in  "status": "pending-upgrade".
kubectl get secrets -A | grep monit
cattle-monitoring-system                 sh.helm.release.v1.rancher-monitoring.v30                            helm.sh/release.v1                    1      3h49m

$ kubectl get secrets sh.helm.release.v1.rancher-monitoring.v30 -n cattle-monitoring-system -o JSON
..
    },
    "kind": "Secret",
    "metadata": {
        "creationTimestamp": "2022-03-23T11:47:04Z",
        "labels": {
            "modifiedAt": "1648036025",
            "name": "rancher-monitoring",
            "owner": "helm",
            "status": "pending-upgrade",
            "version": "30"
        },
        "name": "sh.helm.release.v1.rancher-monitoring.v30",
        "namespace": "cattle-monitoring-system",
        "resourceVersion": "475385",
        "uid": "723f7644-694c-4371-aed7-24266da0a712"
    },
    "type": "helm.sh/release.v1"
}

	// errPending indicates that another instance of Helm is already applying an operation on a release.
	errPending = errors.New("another operation (install/upgrade/rollback) is in progress")

vendor/helm.sh/helm/v3/pkg/action/upgrade.go

// prepareUpgrade builds an upgraded release for an upgrade operation.
func (u *Upgrade) prepareUpgrade(name string, chart *chart.Chart, vals map[string]interface{}) (*release.Release, *release.Release, error) {
..

	// Concurrent `helm upgrade`s will either fail here with `errPending` or when creating the release with "already exists". This should act as a pessimistic lock.
	if lastRelease.Info.Status.IsPending() {
		return nil, nil, errPending
	}

A very possible reason:
When an fleet upgrade action (it calls helm source code API) executes more than 30 seconds, and the POD is terminated, the     on-going "(install/upgrade/rollback)" may be left there.
fleet-agent default termination time is 30s
$ kubectl get pods fleet-agent-8649b7d5b6-b5lzx -n cattle-fleet-local-system -o JSON
"terminationGracePeriodSeconds": 30,
helm master-head, has difference with the embedded helm used in fleet master-head
Helm will release the  StatusPendingUpgrade release when termination.

When checking the fleet-agent log, it showes, even in a normal running Harvester cluster there are also many "performing update for ... " log.   (Why it does so in under investigation via #2013 .)
There are chances to hit this bug.

@w13915984028 @weihanglo Do you know any workaround to get rid of this state? I did have luck to rollback the revision on a chart before, not sure if there is a better way.

@bk201 I found one possible cause, described in https://github.com/harvester/harvester/issues/1983#issuecomment-1076812969 , but not sure if fleet-agent noticed this and has plan to improve it.

The related fleet-agent issue: https://github.com/rancher/fleet/issues/637 , but looks no related fix.

Use also hit this issue and can't apply workaround:

Getting same error upgrading from 1.0 to 1.0.1. Can't role back helm chart.
`helm history rancher-monitoring -n cattle-monitoring-system
REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION
1 Tue Mar 15 20:18:48 2022 superseded rancher-monitoring-100.0.0+up16.6.0 0.48.0 superseded by new release`

Was able to resolve the issue.

Checked bundels
 kubectl get bundles -A

And also secrets as helm releases are stored in here
kubectl get secrets -A -l  status=pending-upgrade

Means helm deployment rancher-monitoring and the harvester deployment are stuck somehow.
Verified helm status and rolled back rancher monitoring helm status rancher-monitoring -n cattle-monitoring-system
 helm history rancher-monitoring -n cattle-monitoring-system
 helm rollback rancher-monitoring -n cattle-monitoring-system

Verify and Rollback harvesterhelm status harvester -n harvester-system
helm history harvester -n harvester-system
helm history harvester -n harvester-system

The upgrade process will continue shortly after harvester rollback is completed.

update @ 2022.10.06:
Harvester v1.1.0-rc2  has fleet v0.4.0-rc3, which in turn has helm v3.9.0, that includes the context cancel, and should have fixed the issue.
@bk201 @guangbochen We can move this to read-for-test, and keep an eye if this issue is reported again.
harv41:~ # kk get pods -n cattle-fleet-local-system fleet-agent-5b46776c54-v76f6 -o YAML | grep image
    image: rancher/fleet-agent:v0.4.0-rc3

https://github.com/rancher/fleet/commit/eae202f68985ce20ff8f97586cd31ee64e7df020

https://github.com/helm/helm/blob/release-3.9/pkg/action/upgrade.go#L263
func (u *Upgrade) performUpgrade(ctx context.Context, originalRelease, upgradedRelease *release.Release) (*release.Release, error) {

Thanks, @w13915984028 , we can keep an eye on it.
Upgrading Rancher and fleet is the first stop in the upgrade process, so the fix should apply to all managed charts after it.

n1-103upd:~ # kubectl get bundledeployment -A
NAMESPACE                                NAME                                          DEPLOYED   MONITORED   STATUS
cluster-fleet-local-local-1a3d67d0a899   fleet-agent-local                             True       True
cluster-fleet-local-local-1a3d67d0a899   local-managed-system-agent                    True       True
cluster-fleet-local-local-1a3d67d0a899   mcc-harvester                                 True       True
cluster-fleet-local-local-1a3d67d0a899   mcc-harvester-crd                             True       True
cluster-fleet-local-local-1a3d67d0a899   mcc-local-managed-system-upgrade-controller   True       True
cluster-fleet-local-local-1a3d67d0a899   mcc-rancher-logging                           True       True
cluster-fleet-local-local-1a3d67d0a899   mcc-rancher-logging-crd                       True       True
cluster-fleet-local-local-1a3d67d0a899   mcc-rancher-monitoring                        True       True
cluster-fleet-local-local-1a3d67d0a899   mcc-rancher-monitoring-crd                    True       True

rancer


Error with jinja2 3.1.0, due to autoescape attribute

Describe the bug
I've got  the following error when runnig python setup.py extract_messages   AttributeError: module 'jinja2.ext' has no attribute 'autoescape'
Seems that autoescape attribute had been deprecated

Are you running it on User Profiles? Can you copy/paste here the full command and full stracktrace that you see in the output.

[root@pist3-dev invenio-userprofiles]# python setup.py extract_messages
WARNING: The wheel package is not available.
WARNING: The wheel package is not available.
WARNING: The wheel package is not available.
running extract_messages
extracting messages from invenio_userprofiles/init.py (encoding="utf-8")
extracting messages from invenio_userprofiles/admin.py (encoding="utf-8")
extracting messages from invenio_userprofiles/api.py (encoding="utf-8")
extracting messages from invenio_userprofiles/config.py (encoding="utf-8")
extracting messages from invenio_userprofiles/ext.py (encoding="utf-8")
extracting messages from invenio_userprofiles/forms.py (encoding="utf-8")
extracting messages from invenio_userprofiles/models.py (encoding="utf-8")
extracting messages from invenio_userprofiles/validators.py (encoding="utf-8")
extracting messages from invenio_userprofiles/version.py (encoding="utf-8")
extracting messages from invenio_userprofiles/views.py (encoding="utf-8")
extracting messages from invenio_userprofiles/alembic/71634726ec7e_create_userprofiles_branch.py (encoding="utf-8")
extracting messages from invenio_userprofiles/alembic/c25ef2c50ffa_create_userprofiles_tables.py (encoding="utf-8")
extracting messages from invenio_userprofiles/templates/invenio_userprofiles/base.html (encoding="utf-8", extensions="jinja2.ext.autoescape, jinja2.ext.with_")
Traceback (most recent call last):
File "/usr/local/src/invenio-userprofiles/setup.py", line 77, in
setup(
File "/usr/lib/python3.9/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/usr/lib64/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib64/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib64/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/src/invenio-userprofiles/.eggs/Babel-2.9.1-py3.9.egg/babel/messages/frontend.py", line 481, in run
for filename, lineno, message, comments, context in extracted:
File "/usr/local/src/invenio-userprofiles/.eggs/Babel-2.9.1-py3.9.egg/babel/messages/extract.py", line 149, in extract_from_dir
for message_tuple in check_and_call_extract_file(
File "/usr/local/src/invenio-userprofiles/.eggs/Babel-2.9.1-py3.9.egg/babel/messages/extract.py", line 207, in check_and_call_extract_file
for message_tuple in extract_from_file(
File "/usr/local/src/invenio-userprofiles/.eggs/Babel-2.9.1-py3.9.egg/babel/messages/extract.py", line 243, in extract_from_file
return list(extract(method, fileobj, keywords, comment_tags,
File "/usr/local/src/invenio-userprofiles/.eggs/Babel-2.9.1-py3.9.egg/babel/messages/extract.py", line 325, in extract
for lineno, funcname, messages, comments in results:
File "/usr/local/lib/python3.9/site-packages/jinja2/ext.py", line 808, in babel_extract
extensions[import_string(extension_name)] = None
File "/usr/local/lib/python3.9/site-packages/jinja2/utils.py", line 149, in import_string
return getattr(import(module, None, None, [obj]), obj)
AttributeError: module 'jinja2.ext' has no attribute 'autoescape'


build(deps-dev): upgrade jest

Closes #286

:tada: This PR is included in version 3.0.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


etcdctl: add --password flag to the subcommand user add

Fix https://github.com/coreos/etcd/issues/9691
/cc @cristifalcas you can create a user whose name includes : now

@joelegasse thanks for your comment, I'll address it


Problems with extrude when width_function or offset_function is set in any Section

Describe the bug
Due to the recent changes to CrossSection, extrude no longer works properly when a width_function or offset_function is applied to any of the Sections. More specifically, those two parameters are excluded from serialization on the Section class, so if another instance with otherwise identical parameters is cached, that instance is retrieved, rather than recalculating correctly.
To Reproduce
Consider the following example. c1 and c2 are identical, although they should not be. c2 should have a width profile as defined by the provided width function. You can see that just by slightly changing another parameter on the third cross section, this issue is avoided (similarly, you could prove this by clearing the cache, etc.)
import gdsfactory as gf

def linear(t, start: float = 1, end: float = 2):
    return t * (end - start) + start

def multi_layer_cs(width, width2_func=None):
    width1 = width
    s2 = gf.Section(width=width+1, width_function=width2_func, layer=(5, 21))
    cs = gf.cross_section.cross_section(width=width1, layer=(2, 21), sections=[s2])
    return cs

if __name__ == '__main__':
    # with no width2 function
    cs1 = multi_layer_cs(width=1, width2_func=None)
    # with a width2 function -- the cached extrude component is retrieved, ignoring the width_function parameter
    cs2 = multi_layer_cs(width=1, width2_func=linear)
    # all we need to do is just slightly change a parameter to avoid the cache... you see that width_function behaves as expected
    cs3 = multi_layer_cs(width=0.999, width2_func=linear)

    path = gf.path.straight()

    c1 = gf.path.extrude(path, cs1)
    c2 = gf.path.extrude(path, cs2)
    c3 = gf.path.extrude(path, cs3)

    c = gf.Component()

    c << c1
    c2_inst = c << c2
    c3_inst = c << c3
    c2_inst.movey(-5)
    c3_inst.movey(-10)
    c.show()

Output:

Expected behavior
width_function and offset_function must be accounted for when caching to prevent this tricky issue.
Suggested fix
Still thinking about it... For now I am hacking around it by setting the width to a random number in the case that a width_function is provided (since it is ignored anyways, it at least makes all instances unique)

@joamatab, the issue still seems to persist when the width function is a lambda which depends on a closure. the workaround is to not use lambdas (i.e. use partial instead), but I thought I'd flag it here, especially because this is how some gdsfactory functions today are structured. see the example below. note that the third taper should look like the fifth one, but instead it looks like the second taper, which is being erroneously pulled from cache.
import gdsfactory as gf
from functools import partial

def linear(t, start: float = 1, end: float = 2):
    return t * (end - start) + start

def linear_lambda(start: float = 1, end: float = 2):
    return lambda t: t * (end - start) + start

def multi_layer_cs(width, width2_func=None):
    width1 = width
    s2 = gf.Section(width=width+1, width_function=width2_func, layer=(5, 21))
    cs = gf.cross_section.cross_section(width=width1, layer=(2, 21), sections=[s2])
    return cs

if __name__ == '__main__':
    # with no width2 function
    cs1 = multi_layer_cs(width=1, width2_func=None)
    # with a width2 function, generated by a lambda. this works fine but...
    cs2 = multi_layer_cs(width=1, width2_func=linear_lambda(end=2))
    # when you make a new version of the lambda with a different parameterization in the closure, serialization fails to distinguish between the two functions. the first cached version is used instead
    cs3 = multi_layer_cs(width=1, width2_func=linear_lambda(end=5))
    # for contrast, try with a partial instead of a lambda
    cs4 = multi_layer_cs(width=1, width2_func=partial(linear, end=2))
    # this works fine. the issue is with serialization of the lambda
    cs5 = multi_layer_cs(width=1, width2_func=partial(linear, end=5))

    path = gf.path.straight()

    c1 = gf.path.extrude(path, cs1)
    c2 = gf.path.extrude(path, cs2)
    c3 = gf.path.extrude(path, cs3)
    c4 = gf.path.extrude(path, cs4)
    c5 = gf.path.extrude(path, cs5)

    c = gf.Component()

    c << c1
    c2_inst = c << c2
    c3_inst = c << c3
    c4_inst = c << c4
    c5_inst = c << c5
    c2_inst.movey(-5)
    c3_inst.movey(-10)
    c4_inst.movey(-15)
    c5_inst.movey(-20)
    c.show()


[FLOC-3597] Increase OS X timeout up to 60 minutes

We can spend an awful lot of time importing base box for the OS X image. Although we can probably do some things to make this faster, a good first step is to increase the timeout, thus avoiding spurious failures.

thanks dude,
just that comment regarding the timeout change,
address, and merge when green.


Update sbt-mdoc to 2.2.21

Updates org.scalameta:sbt-mdoc from 2.2.18 to 2.2.21.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalameta", artifactId = "sbt-mdoc" } ]

labels: sbt-plugin-update, semver-patch

Superseded by #230.


Issues when changing the Project from the detail window

osx_native_app/7.4.7

Open detail for item for project "redacted"

Change project to "redacted" and see
project at left get updated.
Change back to original project (without closing detail window first)
and see project at left NOT get updated. At this point there is no way to
change the project back to "redacted".

Before that I had been using 7.3.347. The "Check for update" menu item was
greyed out.
Note that this/these bug extends to new projects

Start new tracking item and open detail:
Choose project, main list project is updated:
Click on description, and project drop down in detail tab goes blank:
Closing detail keeps the project in the main list, but step 3 is still
disorienting.

I was able to replicate this issue.
Screenshots can be found in Case #413336

project names redacted on user request

I'm able to reproduce this. Will look for a fix.


Feat: 273 generate tax exemption ID

ZH CARD 273

Well done looks good and I like the bugfixes and code cleanup done along the way!


Why vision requires an open api key

Type: Bug
I am trying to use vison in copilot it is asking for open ai key
Extension version: 0.2.2024111316
VS Code version: Code - Insiders 1.97.0-insider (95386dec787946547812c7ee9d38cba19d6a6398, 2024-12-17T05:04:19.511Z)
OS version: Windows_NT x64 10.0.19045
Modes:

we've mentioned that this is a "bring your own key model". it's intended to be a preview of what will eventually be in copilot chat!


Fixing issues, making specs pass, upgrading to bouncycastle 1.58

These are the changes I needed to get this gem working in my app after I upgraded to JRuby <IP_ADDRESS>

@superchris Can you please make sure to squash any oops commits into their respective parents?


nvidia: fix egl-wayland loading

Motivation for this change
see: @jian-lin's comment: https://github.com/NixOS/nixpkgs/issues/75131#issuecomment-904671763
Nvidia's driver loads EGL external platforms by looking in certain search paths.
This PR:

Moves the configuration file used by egl-wayland to a path structure closer to what is expected
Symlinks the config into a directory that is searched for by the Nvidia driver, since it can't be configured to look in the /run/opengl-driver location (see: https://github.com/NVIDIA/egl-wayland/issues/39)

This casues NixOS Nvidia Wayland users to get working EGL out of the box.
Fixes: #75131
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

I just pushed and removed a comment with a question in it:
    # TODO: should we symlink to the /run/opengl-driver path
    # or just straight symlink into the nvidia package since we know we're going to use it?
    environment.etc."etc/egl/egl_external_platform.d".source =
      # "/run/opengl-driver/share/egl/egl_external_platform.d/10_nvidia_wayland.json";
      "${nvidia_x11}/share/egl/egl_external_platform.d/10_nvidia_wayland.json";

in case someone thinks I made the wrong decision linking it directly into place.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/nvidia-users-testers-requested-sway-on-nvidia-steam-on-wayland/15264/1

What apps are expected to run with this? It seems the only thing I can get to work is Chromium webgl.

@mvnetbiz in particular, I know that this fix allows Steam to run under Gnome-Shell Wayland with NVIDIA.

I've tested this PR on a system with a GTX 970. Steam worked without issues in GNOME on Wayland, and I was also able to start a sway session with the overlay mentioned in the Discourse thread.
Most other GL programs (alacritty, mpv, PCSX2) I tried did not work either in GNOME or sway, but I gather that is to be expected (although it's worth noting that #138180 might fix alacritty). That being said, this still seems like a clear improvement to me.

Just a note, @tcbravo, I think nvidia ships their own embedded egl-wayland, which is what the installed egl_externalplatform json ends up pointing to. If you end up wanting to use that newer egl-wayland, you may want to infact override this /etc/egl... symlink so that it instead causes the updated egl-wayland to be loaded. (As noted in the other Issue, there are some differences between the two already, even without the version bump)

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/nvidia-users-testers-requested-sway-on-nvidia-steam-on-wayland/15264/5

I'm emulating this PR by adding these lines to my configuration:
  environment.etc."egl/egl_external_platform.d/10_nvidia_wayland.json".text = ''
      {
        "file_format_version" : "1.0.0",
        "ICD" : {
            "library_path" : "/run/opengl-driver/lib/libnvidia-egl-wayland.so"
        }
    }
  '';

  environment.etc."egl/egl_external_platform.d/10_nvidia.json".text = ''
    {
      "file_format_version" : "1.0.0",
      "ICD" : {
        "library_path" : "/run/opengl-driver/lib/libEGL_nvidia.so"
      }
    }
  '';

And Steam is working now on Gnome on Wayland. I'm having some problems with some softwares like Meshlab and Firefox can't create WebGL contexts.

@tfmoraes btw, that doesn't actually look exactly right, unless its a copy/paste error. The 10_nvidia.json file should be in an egl_vendor.d dir.

/run/opengl-driver/share
❯ tree
.
├── egl
│   └── egl_external_platform.d
│       └── 10_nvidia_wayland.json
├── glvnd
│   └── egl_vendor.d
│       └── 10_nvidia.json
└── vulkan
    ├── icd.d
    │   └── nvidia_icd.json
    └── implicit_layer.d
        └── nvidia_layers.json

But it probably doesn't affect you since our libglvnd is patched to look in the /run/ path first, where the 10_nvidia.json is already in the right place.

I haven't heard of this making anything worse, so I'm still hoping to get this merged.

@colemickens I'm using this way now:
  environment.etc."egl/egl_external_platform.d/10_nvidia_wayland.json".text = ''
      {
        "file_format_version" : "1.0.0",
        "ICD" : {
            "library_path" : "${pkgs.egl-wayland}/lib/libnvidia-egl-wayland.so"
        }
    }
  '';

  environment.etc."glvnd/egl_vendor.d/10_nvidia.json".text = ''
    {
      "file_format_version" : "1.0.0",
      "ICD" : {
        "library_path" : "${pkgs.linuxPackages_latest.nvidia_x11}/lib/libEGL_nvidia.so"
      }
    }
  '';

Note that libnvidia-egl-wayland.so is from egl-wayland not from nvidia_x11. With libnvidia-egl-wayland.so from egl-wayland firefox is working ok in wayland.

@tfmoraes sounds about right. I think we should consider having another module option somewhere to enable "installing" a different egl-wayland to that dir with a higher priority than the nvidia driver.
That way the nvidia driver can do what it is supposed to do, but NixOS users have an option for overriding that by getting their own egl-wayland loaded first.
But, that can definitely be separate, presumably making your JSON lexi-sort before "10_nvidia_wayland.json" would enable your exact setup to continue, even with this PR.

Is there anything blocking this PR from being merged? I'm using it without any problem.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/nvidia-users-testers-requested-sway-on-nvidia-steam-on-wayland/15264/28

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/kwin-wayland-segfault-with-proprietary-nvidia-drivers/18452/3


Credentials supplied by local environment not working

After updating from 1.11 to 1.13,
Authenticating with
./cloud_sql_proxy -instances=$INSTANCE_NAME -credential_file=/tmp/creds.json
Does work, but
GOOGLE_APPLICATION_CREDENTIALS=/tmp/creds.json ./cloud_sql_proxy -instances=$INSTANCE_NAME
Does not, even thought the documentation states they are equivalent.

due to this https://github.com/GoogleCloudPlatform/cloudsql-proxy/pull/198#discussion_r225649203

Created PR #229 to start a discussion on this.

Submitted #229


Primary and secondary site codes

In a couple of places the /ps and /sc parmaeters are referenced. There's some confusion in the doc because while the doc explicitly suggests "primary site code" must be passed for both of these parameters, I can't help but think /sc should accept just the secondary site code, and not the primary site code.
Also, the descriptions for them don't make logical sense either. They're clear and well written, but for example under "Optional parameters" heading the description for /ps and /sc suggests to use them when cleaning a DP at a primary site. However reading about these parameters under "Required parameters" heading, it suggests the only use these parameters when cleaning a DP from a secondary site.
I would make a PR but I don't have a secondary site to test with unfortunately so can't confirm.
Also, the brief example for how to use these parameters is a little confusing, e.g. /sc <primary site code>, should that be secondary site code? Again, I'm not sure, I could be wrong, I can't test.
So lines of the .md file I propose for review are: 97, 98, 113 and 114.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 15d31569-af98-8c15-a224-f68b83827caa
Version Independent ID: cd2099fe-d6c5-0983-948c-45617a6c0e11
Content: Content library cleanup tool - Configuration Manager
Content Source: sccm/core/plan-design/hierarchy/content-library-cleanup-tool.md
Product: configuration-manager
Technology: configmgr-core
GitHub Login: @aczechowski
Microsoft Alias: aaroncz

Hi @codaamok, it's technically accurate, just subtle. The /ps parameter takes the FQDN of the primary site server. The /sc parameter takes the site code. You can specify these params when running it for a DP at a primary site, but it's not necessary. That's why they show in the optional list.
When you run the tool for a DP at a secondary site, the tool needs to connect to the parent primary site to get necessary details. So in that scenario, these params are required.
I'm just kind of rephrasing/summarizing what's already in the article. So I'm going to close this issue. If you have any suggestions on how to further/better clarify, open a new issue on the current doc in memdocs, or open a PR over there. 😉


Close connections to MySQL dictionary source

Please close dictionary connections to MySQL source (could by configurable). When there are many nodes and dictionaries, connections rapidly ramp up. For example, we have 100+ dictionaries and 60 nodes -- this results in 6000+ open connections. It is certainly possible to add more memory to MySQL or have several MySQL replicas, but there is no rationale to keep connections open all time as well. MySQL connection is very fast to open when needed but consumes server resources.
Please consider.

@alex-zaitsev using proxysql as a workaround should work (wild guess, never tried it together with clickhouse)

There are multiple workarounds possible. At the moment we reduced wait_timeout on MySQL side to terminate inactive connections.


Bump expiration date for WinRM certificate

Current date is 1 Jan 2022 so the script is generating expired certificates and WinRM connection can't be established.

Why to make it static again? My suggestion:
$currentDate = get-date
$expirationDate = $currentDate.AddYears(1)
.\makecert -r -pe -n CN=$hostname -b $currentDate -e $expirationDate -eku <IP_ADDRESS>.<IP_ADDRESS>.1 -ss my -sr localmachine -sky exchange -sp "Microsoft RSA SChannel Cryptographic Provider" -sy 12


Get bazelisk version without launching bazel server

We are looking for a fast way to determine the version of the bazelisk binary. Calling time bazel version currently prints the following:
Bazelisk version: v1.16.0
WARNING: Invoking Bazel in batch mode since it is not invoked from within a workspace (below a directory having a WORKSPACE file).
Build label: 6.1.1
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed Mar 15 15:44:56 2023<PHONE_NUMBER>)
Build timestamp:<PHONE_NUMBER>
Build timestamp as int:<PHONE_NUMBER>

real    0m1.372s
user    0m2.208s
sys     0m0.259s

It takes some time for bazel to start the local server, but in our case we do not need to know the bazel version (because we define it via .bazelversion). We only need the first line: Bazelisk version: v1.16.0

BTW. This old https://github.com/bazelbuild/bazelisk/issues/137 issue/feature request, essentially, wants to achieve the same thing.
I was looking for a way to do exactly that recently. I took a look at the source code and if contributions are accepted I might open a PR since it should be a relatively simple to implement.


menu: no longer accessible with child components in @angular8+

Reproduction
Use StackBlitz to reproduce your issue:
https://stackblitz.com/edit/angular-7-mat-menu (works)
https://stackblitz.com/edit/angular-9-mat-menu (broken)
Steps to reproduce:
Set up a button to be used with a mat-menu
In the mat-menu, use 2 buttons as well as use a custom component that has just a button
Using the keyboard, tab to your button and then open it, try to navigate using arrows
Only the 2 buttons explicitly listed in the template will navigate, the button from the component will not allow keyboard navigation. The manuData is missing for that element.

Expected Behavior
What behavior were you expecting to see?
This was working before we upgraded to version 9. I have created 2 stackblitzes with the same example. @angular 7 works, @angular 8 and 9 do not.
Actual Behavior
What behavior did you actually see?
The items that are listed as direct children are properly navigated to using arrow keys. The ones rendered from custom component are skipped.
Environment
Angular: 9.0.2
CDK/Material: 8.2.3
Browser(s): Chrome
Operating System (e.g. Windows, macOS, Ubuntu): Windows 10

I was able to confirm this regression. cc. @crisbeto if you want to have a look.

Any news on this issue ?

Combining this with https://github.com/angular/components/issues/16457 since it's older and has more activity.

Got confused, it's not the same issue. Reopening.

Anything new?
The _allItems received the items on the ngAfterContentInit, which happens before the child component init.
So I have an ugly work around on stackblitz, but hope to find soon a better solution.
There is a better work-around but it not suitable for my case, because I must use ng-template outside the mat-menu which have the same issue as the child component.

I'm closing this because the behavior before Angular v9 was actually a bug. In Angular's component model, a component is meant to be a black box; parent components aren't supposed to be aware of its contents. The v9 change makes this work the way it's supposed to.


core: NGS adaptation. This will imply dropping MySQL and using some noSQL such as couchdb or mongodb.

Postgres now comes with NoSQL capabilities.

Maybe this is out of scope for now.


Add JWT option

For SANtricity 11.70.4 with enabled JWT, we could add the option of using JWT instead of username/password
But in that case (11.70.4) collector.py would have to bypass Web Services Proxy - is that right?

But in that case (11.70.4) collector.py would have to bypass Web Services Proxy - is that right?

Yes, that's right, because collector.py uses WSP folder API which doesn't exist on SANtricity.
I worked around that by modifying collector.py: I added WSP_BYPASS boolean to collector and I use it to skip the folder stuff and switch API endpoint from a WSP IP to a SANtricity controller IP.
This can't handle a failure of a controller despite multiple controllers in config.json, so a retry to the other controller should be added to make it HA. Or we could use an HA transparent Web proxy (Nginx, etc.) to take care of failover.


Adds Contour Gateway Config Support

Adds support for configuring gateway name/namespace in the Contour ConfigMap.
Requires: https://github.com/projectcontour/contour-operator/pull/246
Fixes: #241

How does someone opt out of GatewayAPI? This always configures Contour with the default gateway of contour/projectcontour.

@stevesloka commit 000e727 resolves this issue.

We should also check that no other instances are configured with the same, otherwise they are going to hijack the same Gateway and serve traffic.

@stevesloka this issue will be resolved when https://github.com/projectcontour/contour-operator/issues/255 is fixed.

xref: https://github.com/projectcontour/contour-operator/issues/244

Commit https://github.com/projectcontour/contour-operator/commit/1410e5952edb1526664319051955a376330657bd introduced a regression to the TestGateway e2e test that is fixed by commit fd216da.

@youngnick commit 33d6b3e resolves your comments above.


Improved Salesforce Reliability

Made use of python TempFile rather than keeping (potentially) large lists of lists of OrderedDicts in memory.
Moved parse_results() and _parse_results() to be global functions. Call these functions on each page of salesforce results within query_all(). Seek to beginning of tempfile and copytoobj() then close tempfile (thus deleting it and cleaning up memory).
Regex now matches case insensitive when extracting field names from soql query.
Recursive get_all_responses() inner function has been replaced by the iterative equivalent (deep recursion was causing a segmentation fault).

Broken.......


Updating/Verifying OS support for python 3.9 and 3.10

We already have the 3.9 and 3.10 versions in the tests matrix. I believe that should be sufficient to mark this as completed?
CC: @santoshkumarradha @jimmylism


Modernize API calls which produce warnings

Need to modernize DNN API calls which produce warnings on DNN 7.1.2

Implemented in 6e3fcf2, with some fixes in 320c4bd


says its commending yet no commends appear

so, i know its an really old report/commend bot, but i was wondering if someone here can help me @dank the problem:

please someone help me!

Уже давно вальвы фиксанули это.


python2nix: old/wrong/missing dependency?

Issue description
python2nix doesn't work on stable-17.09.
> python2nix
Traceback (most recent call last):
  File "/nix/store/z313xd7cpglh486jd08ckxgsjrgifijm-python2nix-20140927/bin/.python2nix-wrapped", line 8, in <module>
    from python2nix import main
  File "/nix/store/z313xd7cpglh486jd08ckxgsjrgifijm-python2nix-20140927/lib/python2.7/site-packages/python2nix/__init__.py", line 5, in <module>
    from python2nix import pip_deps
  File "/nix/store/z313xd7cpglh486jd08ckxgsjrgifijm-python2nix-20140927/lib/python2.7/site-packages/python2nix/pip_deps.py", line 15, in <module>
    from pip.log import logger
ImportError: No module named log

Steps to reproduce
Run
> nix-env -i python2nix
> python2nix

Technical details

System: 17.09.1880.ac2bb5684c (Hummingbird)
Nix version: nix-env (Nix) 1.11.15
Nixpkgs version: 17.09.1880.ac2bb5684c
Sandboxing enabled: yes

I should have read the README first, it shouldn't be run in this way. On the other hand, just failing is not very helpful.

This is an old package that, as far as I know, is no longer developed/maintained, and should therefore be removed.

It has recent commits though in the repo.
https://github.com/proger/python2nix

From the source repo:

Tested only with pip==1.5.6.

This version dependency on a 3.5 year old pip is the cause of the issue, and the breaking change happened just a few months after that release.
This bug and its fix should be tracked in the upstream package. The package will likely remain broken until an upstream fix is made. In the mean time, the nixpkgs manual lists a few alternate options.


Use debugging information to assert that generated layouts/signatures are correct

This would be a huge boost to my confidence in our correctness and ability to identify and catch bugs in our test suite.
Here is a sketch of what I'm imagining:

Compile C/C++ header file into an object file.
Generate and compile Rust bindings for the C/C++ header file.
Read the debugging information for the object file and get the layout of structs/unions and function signatures.
Read the debugging information for the compiled Rust bindings and get the layout of generated structs/unions and function signatures.
Compare (3) and (4) and assert that they match up with each other.

We can use gimli for reading DWARF on unix; we would need some PDB library for Windows. I think it might be practical/pragmatic to start with just unix.
Eventually, we could even drive this process with Csmith.

Note that we generate #[test] functions for struct size and alignment already.

Yes, those tests are valuable and have caught bugs for me in the wild before.
However, that is based on our own understanding of the layout. If our understanding is wrong due to a bug, we can generate incorrect #[test]s which may incorrectly pass. For example, if we generate bindings with different defines from how the C/C++ was compiled, bindgen's runtime view of layout and the generated bindings' #[test] view of layout will agree, but the compiled code's actual layout can differ (I have also hit this bug, and the existing #[test]s did not catch it).
The debugging information for the object file is used by many different consumers (gdb, perf, valgrind, etc) and has existed for a long time, so it is very reliable.

Working on this.

I don't think this is actually worth it.
It's not like we're calculating our own layouts, we're just piping through what libclang gives us, which is the same as what clang is going to put into the debug info.


dapr state&binding interfaces haven't destroy standard api.

if I dynamicly update or add many binding or state component crds, old resources won't be freed, such as redis, mysql and etc  connection pool.
So  how should we free old connection pool ?

Now pub/sub has close interfaces while state not, I am very happy to see it.
WDYT? @pkedy @artursouza

Binding and state components are expected to implement the io.Closer interface too. Daprd does attempt to invoke it where available and you should open issues in dapr/components-contrib for components that need to release resources and are missing a Close() implementation.
Both Redis and MySQL state components mentioned in the issue do have io.Closer implementations. The issue is when daprd invokes Close() which currently is only on shutdown ... it's not my understanding that Dapr supports CRDs being dynamically updated today without taking down Dapr, I thought that was what a lot of the discussion around the config API was around.
Specific to bindings at least, there are examples where the lifetime semantics for Init() and Close() are not independent of the process lifetime today (see #3555), which may create additional complications if we do expect components to be reconfigured without restarting Dapr.

I agree with @CodeMonkeyLeet. We don't support component reloading without restarting sidecar. So, I am curious to know where is the need to handle closing resourced beyond the shutdown case.


Shorthand flags don't work with --build

TypeScript Version:  3.0.3

Search Terms:
project references
watch
build
Code
This doesn't work:
./node_modules/.bin/tsc --build -w
error TS5072: Unknown build option '-w'.

This does:
./node_modules/.bin/tsc --build --watch

Related Issues: https://github.com/Microsoft/TypeScript/issues/25613

We only have a few build flags at the moment (and intend to add more later), so for the time being we're not assigning single-letter variants. Once we're more confident that watch will be the most common w option, we'll make the shorthand equivalents work

@RyanCavanaugh but -w is already the shorthand for --watch when using tsc without --build. Isn't that really weird if -w changes conditionally based on whether or not it's used with --build?
$ tsc  --help
Version 3.0.3
...
 -w, --watch                                        Watch input files.

Just noticed we already have short flags for other stuff under --build... so yeah let's assign w to watch.

@RyanCavanaugh you forgot to reopen, also this might be a good one for help wanted/good first issue?

This is fixed in master?

This appears to be resolved in the nightly build:
$ npm install typescript@next

$ ./node_modules/.bin/tsc --version
Version 3.2.0-dev.20180929

$ ./node_modules/.bin/tsc --build -w
[8:39:12 AM] Starting compilation in watch mode...


A documentation of class "grid-offset" is missing

Maybe you like to explain somewhere what this class is good for.

Yeah, I need to write up an explanation. Basically, it would be for if you want to add a mini "container" around something, to give side spacing.

      Content

      Content


[FEAT] Print btrfs subvolumes like the zpool storage pools

Be sure to read FAQ before submitting a new issue.
Wanted features:
I would like to only print the btrfs sub-volumes similar to the following only printing ZFS zpools.
fastfetch -s zpool -l none
Zpool (zhome): 1.05 MiB / 3.75 GiB (0%, 0% frag) - ONLINE
When I run:
fastfetch --disk-show-subvolumes  -l none
I get all info including the drives which is not what I wanted to see.
fastfetch --disk-show-subvolumes  -l none | grep btrfs
Disk (/opt/devops/automation): 184.15 MiB / 3.00 GiB (6%) - btrfs
Disk (/tmp): 184.15 MiB / 3.00 GiB (6%) - btrfs [Subvolume]
Disk (/usr/share/locale): 184.15 MiB / 3.00 GiB (6%) - btrfs [Subvolume]
Disk (/usr/share/man): 184.15 MiB / 3.00 GiB (6%) - btrfs [Subvolume]
Disk (/var/cache/dnf): 184.15 MiB / 3.00 GiB (6%) - btrfs [Subvolume]
Disk (/var/log): 184.15 MiB / 3.00 GiB (6%) - btrfs [Subvolume]
Disk (/var/log/audit): 184.15 MiB / 3.00 GiB (6%) - btrfs [Subvolume]
Disk (/var/tmp): 184.15 MiB / 3.00 GiB (6%) - btrfs [Subvolume]

Those are not disks but subvolumes in btrfs FS as you stated at the end of the line. There's also a lot of info that one could get for ZFS and btrfs volumes to present it to users in a more consistent manner:
→ btrfs filesystem show /dev/nvme2n1
Label: 'system_vol'  uuid: 65998469-344d-4cd9-a423-e7a9d81ad41c
	Total devices 1 FS bytes used 166.72MiB
	devid    1 size 3.00GiB used 1.14GiB path /dev/nvme2n1

→ btrfs filesystem usage /var/log
Overall:
    Device size:		   3.00GiB
    Device allocated:		   1.14GiB
    Device unallocated:		   1.86GiB
    Device missing:		     0.00B
    Device slack:		     0.00B
    Used:			 181.70MiB
    Free (estimated):		   2.33GiB	(min: 1.40GiB)
    Free (statfs, df):		   2.33GiB
    Data ratio:			      1.00
    Metadata ratio:		      2.00
    Global reserve:		   3.50MiB	(used: 0.00B)
    Multiple profiles:		        no

Data,single: Size:640.00MiB, Used:154.51MiB (24.14%)
   /dev/nvme2n1	 640.00MiB

Metadata,DUP: Size:256.00MiB, Used:13.58MiB (5.30%)
   /dev/nvme2n1	 512.00MiB

System,DUP: Size:8.00MiB, Used:16.00KiB (0.20%)
   /dev/nvme2n1	  16.00MiB

Unallocated:
   /dev/nvme2n1	   1.86GiB

Motivation:
I want to make it easier for my users to see what zpools and btrfs volumes are on the systems when they login. Can the feature to print btrfs volumes and subvolumes be added like zpool was added?

What do you want fastfetch to print?

The same as with zfs (fastfetch -s zpool -l none) I just want to see the btrfs drives on a host. fastfetch -s btrfs -l none

You should test the dev build

@CarterLi Thanks. The implementation works well!


Fix external refresh token error

It fixes external token refresh by making external logout call optional for internal methods, like sdk fetch.

Task linked: SALEOR-5865 Handle external refresh token error


Implementation and documentation of historical-price-full and historical-chart are wrong

As per site doumentation below,

The historical-chartendpoint has the path parameters timeframe and symbol, and the query parameteres from, to and from dates. The implementation does not care for to and from, restricting the usability of the tool.
the same happens with the historical-price-full endpoint where we have the wrong list of parameters and unhelpful documentation on the api...
I will fix and create a pull request if allowed.

@marsson it that fixed?


Default pipelines should be configured.

There should be default pipelines configured in connector, for easer launch out of the box.

There are several problems with prefiguring pipeline:

Documents and Entities must be implemented in project as bundle provides only abstract ones.
Projects will need a way to disable or override these pipelines.
Making abstract services will not make launch any simpler.


GDScript: Improve error messages for invalid indexing

These errors are very common when using an invalid property name or calling on an object of the wrong type, and the previous message was a bit cryptic for users.
Before

After

(was missing quotes in this last one, fixed afterwards)

It's not perfect as is as the same code is used when indexing arrays with [i]:

I might rework it to be a bit more clever though, it's a bit annoying that indexing basically covers property/member access, dictionary keys (which can be accessed like properties) and array indexing.

Marking as draft as I'm not actively working on it now. I'd welcome anyone who wants to take it over and make sure that it gives a meaningful message in all cases.

Superseded by #82639.


environments are listed in random order and have no description

While this may not be the fault of the plugin itself, we still need to work with tox to find a solution that provides a better output:
$ tox -l -v                                                                                using tox.ini: /Users/ssbarnea/c/rdo/ci-config/tox.ini (pid 18738)
using tox-3.15.2 from /Users/ssbarnea/.pyenv/versions/3.8.2/lib/python3.8/site-packages/tox/__init__.py (pid 18738)
default environments:
promoter-promote-images             -> [no description]
py36                                -> [no description]
py27                                -> [no description]
promoter-container-push             -> [no description]
linters                             -> [no description]
jjb                                 -> [no description]
promoter-tripleo-common-integration -> [no description]
lint_all                            -> [no description]
molecule                            -> [no description]

Ideally these environments should be listed one after another, in alphabetical order?

Sort is easy. I'll get to work on description. I'm thinking something like "Auto-generated environment for Ansible role FOO, scenario BAR"?


Add medical evals

https://github.com/epfLLM/meditron/blob/main/evaluation/benchmarks.py
https://github.com/epfLLM/meditron/blob/main/evaluation/README.md

Python code
Medical Competence

MedMCQA
ClosedPubMedQA
PubMedQA
MedQA
MedQA4
MedicationQA
MMLU Medical

Reasoning

ARC
HellaSwag
Winogrande
GSM8K
TruthfulQA
Blurb: Biomedical Language Understanding and Reasoning Benchmark

Final code involves a CLI for testing a given model on a given list of evals. This CLI is called by the runpod.sh program. Which in turn is called by the runpod template after the GPU instance boots up.

Finally in a colab notebook if the following sort of code runs to completion, then we test it with runpod in debug mode.
!export DEBUG=DEBUG
!export GITHUB_API_TOKEN=GITHUB_API_TOKEN
...

!git clone https://github.com/Technoculture/med-llm-autoeval.git
!bash ./med-llm-autoeval/runpod.sh

huggingface-cli login --token $HF_TOKEN


"Inquiry Regarding the Utilization of mAP or cmAP Metrics in Your Research"

Thank you very much for your open-source code and paper. May I ask whether the mAP used in your paper refers to the mAP metric or the cmAP metric?

Hi @xhdr0618, the metric used in the paper is cmAP which uses the same aggregation scheme as in ivtmetrics and the rendezvous baseline.
The mAP metric (without aggregation) is used as a complementary metric.

mail.scut.edu.cn

Thank you very much for your answer.


Fixed new serviceprincipal layout

refers to #2

Hi Bjorn,
Thank you for finding and fixing these bugs!
This are updates that are part of the new Az.Resources module, which has breaking changes.
It's very helpful you have already found a fix!
Could you add one thing?
I want to make sure the module keeps working for people who might not have updated yet. So what I think we could do, is put an if-loop around the bugfixes.
So that would be something like this:
if ((Get-Module Az.Resources).Version.Major -lt 5){
    $PlainTextSecret = $ServicePrincipal.PasswordCredentials.SecretText
}
else {
*the old solution
}

To make this work for the appId, it could be defined in a variable before $Body is created.
Do you have time to add that? If so I will merge the PR and update the module in the Gallery 👍

No problem, glad to help.
Fixed check for installed version and set variables according to your requests.

Awesome!
I tested it on PowerShell 5.1 with the old module and it all worked. So LGTM!
Once it is merged I will update the manifest and upload it to the gallery. Thank you again!


TURN reports failed to handle Refresh-request

Describe the bug
When running in production, the logs would print lines like:
turn ERROR: 2021/10/20 18:55:24 error when handling datagram: failed to handle Allocate-request from <IP_ADDRESS>:21336: relay already allocated for 5-TUPLE
turn ERROR: 2021/10/20 19:21:13 error when handling datagram: failed to handle Refresh-request from <IP_ADDRESS>:12676: no allocation found <IP_ADDRESS>:12676:<IP_ADDRESS>:443

Server

Version: 0.13.5

Client

SDK: Web

To Reproduce
It's not clear how to reproduce this. Related tickets mention FireFox 91 on Android. https://github.com/pion/turn/issues/206

@davidzhao I'm facing this problem when I'm trying to send data message. From console WebRTC: ICE failed, add a STUN server and see about:webrtc for more details. From ICE Stat column I can see all failed. Any work around? This is mostly happening with Firefox.

Having the same issue:
TEST results:
Connecting to signal connection via WebSocket
Connected to server, version 0.15.3.
Establishing WebRTC connection
Can connect via TURN
Can publish audio
published 62 audio packets
Can publish video
ERROR: Could not get RTCStats
Resuming connection after interruption
------ LOGS
livekit_1  | 2022-02-09T05:52:00.675Z   ERROR   livekit.turn    v2@v2.0.6/server.go:184 error when handling datagram: failed to handle Allocate-request from <IP_ADDRESS>:51596: relay already allocated for 5-TUPLE

@heromiyo what browser/OS/device are you using the connection tester with?

Chrome Version 97.0.4692.99 (Official Build) (64-bit)

I have created a pr for<EMAIL_ADDRESS>to fix Allocation already exist
For Refresh-request allocation not found, the only case I can reproduce is packet out of order in client close.
The sequence is :

client allocation relay , connect to sfu
allocation will expire, client send refresh to refresh allocation's lifetime
at same time, client closed(user leave), send refresh with 0 lifetime to delete the allocation

at server side, packet 2, 3 reorder during network transfer, so turn server receive packet 3 first, delete allocation, then packet 2 received, allocation not found.  It only happened in client closing, so will not cause problem.
I think this issue can be closed now @davidzhao
Another information about turn but not this problem, for refresh allocation, the client behavior is different, and some clients will interrupt when refresh-request is lost.

chrome will refresh allocation 1 minute before it expired, and retry until success or allocation expired(will cause rtc connection failed). That is RFC's recommend.
safari will refresh allocation 1 minute before it expired, and retry 30 second then disconnect if failed.
firefox will refresh 10 second before expired, retry until expired. If network have a high lost rate in these 10 seconds, it is easy to fail.


Create a unionized p-value for freq comparison

Test run: b08870cb9260410596f703ce880784d3

LGTM, in the test, 122 variants had p_value<10e-4 compared to 234 before using cmh p-value.


Machine mac-24 lost external connection from iPhone device.

Machine mac-24 lost external connection from iPhone device.

The external connection has been restored by rebooting machine mac-24.


TransformExtensions code review

https://github.com/StansAssets/com.stansassets.foundation/blob/b3332055c35bb65162e22570b14c06d59ff4155e/Runtime/Extensions/UnityEngine/TransformExtensions.cs#L53-L67
using GetComponent is kinda inefficient. you can do this:
for(int i = transform.childCount-1; i >= 0; --i){
    Object.DestroyImmediate(transform.GetChild(i));
}

https://github.com/StansAssets/com.stansassets.foundation/blob/b3332055c35bb65162e22570b14c06d59ff4155e/Runtime/Extensions/UnityEngine/TransformExtensions.cs#L80-L81
if you'd use part.SetParent(transform, false) you could skip the part.Reset() call.

Done in #108


feat: Conglomerate and update dagger dependencies

Summary
As part of the many ongoing features such as parquet, python-udf, etc in Dagger, there will be additions in the dependencies. Hence, the intention is to combine all such planned dependencies and add them to the respective modulebuild.gradle files of Dagger so that the dependencies jar (of the format dagger-core-x.x.x-dependencies.jar) as generated by the gradle task ./gradlew dependenciesJar has all the dependencies packaged together.
Proposed solution
This will reduce migration effort later on when all the feature branches will get merged. Only the main dagger image will need to get changed.
Additional context
N/A

For Parquet Dagger Source:
Diff comparison: https://github.com/odpf/dagger/compare/main...dagger-parquet-file-processing?diff=split
Dependencies to be added in dagger-common
 dependenciesCommonJar ('org.apache.hadoop:hadoop-client:2.8.3') {
        exclude module:"commons-cli"
    }

dependenciesCommonJar 'org.apache.parquet:parquet-column:1.12.2'

Dependencies to be added in dagger-core
def dependenciesVersion = '0.2.2'
. . .
dependenciesJar 'org.apache.parquet:parquet-protobuf:1.12.2'

For Python UDF:
Diff comparison: https://github.com/odpf/dagger/compare/main...feat/add-support-python-udf
Dependencies to be added in dagger-common:
 dependenciesCommonJar ('org.apache.hadoop:hadoop-client:2.8.3') {
        exclude module:"commons-cli"
    }
dependenciesCommonJar 'com.google.cloud.bigdataoss:gcs-connector:1.9.0-hadoop2'

Dependencies to be added in dagger-functions:
 compileOnly 'org.projectlombok:lombok:1.18.8'
 annotationProcessor 'org.projectlombok:lombok:1.18.8'
. . .
dependenciesFunctionsJar 'org.apache.flink:flink-python_2.11:' + flinkVersion

Attached diff images:


Compiling

https://i.imgur.com/Fshlhdm.png

https://i.imgur.com/8J89vw4.png

https://i.imgur.com/6h5Hu24.png

@Shuvi-Moro Retarget your project. and make sure all lib&inc are loaded.

@yaelahrip And where can I download them?

In Visual Studio Installer select and install package for C++ development.


Regarding performance issue w.r.t to pyboolector library

Hi @mballance
I have done the profiling in our generator to find out the bottleneck of the poor performance.
After going through the profiling data it seems like Sat method pyboolector is taking almost 80% of the total internal execution time.
I am attaching the profiling data here for your reference which is for 100 instructions only.

I am curious to know what exactly Sat method of pyboolector does while randomizing the data using vsc module and how can I minimize this as our generator is growing with time and the same is reflected in the performance.
I am not sure if I should be raising this issue here but I would be highly obliged if you can provide me a way to maximize the performance.

Adding @hodjat91 for visibility.

Hi @aneels3,
The Sat method performs solving of the constraint equations. It's used for two purposes -- one to establish correct (but non-random) values for fields and one to provoke random values on the variables such that they still satisfy the constraints.
Do you know how many calls to the randomize API the generator makes in order to generate 100 instructions? I see that there are ~800 calls to Sat. Knowing how many total calls to randomize you make would give a clue on how many solves are done (on average) to produce each solution to a randomize call.
One thing to look out for are constraints on arrays. Since arrays are unrolled, a fairly simple-looking constraint can turn into something quite large and difficult for the solver to process.
Best Regards,
Matthew

I found setting EN_DEBUG = 1 in randomizer.py to identify to large constraints.
Do you know why, specifically, the solver would have a problem with simple constraints on arrays, e.g., each element of an array is between a range:
Pre-Randomize: RandSet
  Field: grid.<unknown>[98] [[256, 2047]]
  Constraint: (grid.<unknown>[98] <= 2047);
  Constraint: (grid.<unknown>[98] >= 256);

Are you aware of a hook in pyboolector to profile solve time?

Hi @mballance
Sorry for the delay on this.
As per our last discussion, I have created two branches in our forked repository profiling_with_constraints and prof_without_constraints.
The command to run the test:
python3 run.py --test=riscv_arithmetic_basic_test --iteration 1 --simulator pyflow --gen_timeout 3000
The riscv_arithmetic_basic_test will run without the directed case for now and with 1000 instr_cnt.
You can decrease or increase the instr_cnt from here
I have also included the logs that I have generated for with_constraints and without_constraints for your reference.
Surprisingly, I see some difference in time for the generation which is about ~100 sec but there is not much effect on the number of ncalls for the Sat method.
Note:
With the latest PyVSC release this https://github.com/fvutils/pyvsc/issues/43 issue is creating problems while running the test but this time with a different error. I am investigating it and will address this in a separate issue or reopen #43.
For now, you need to downgrade the PyVSC version to a particular release(<IP_ADDRESS>10523.1) to make it work.
Regards,
Anil

Hi @aneels3,
Thank you for the instructions. I'll begin investigating when I have a chance (likely at the weekend).
Best Regards,
Matthew

Thanks @mballance
Looking forward to hearing from you.
Best,
Anil

Hi @aneels3,
My first step in investigating this is to establish some baseline numbers: performance for pyflow and performance for a simulator. I am using PyVSC 0.4.6. Using Questa as the simulator, I see the following when running the command above:

pyflow: 23.55s
questa: 11.57s

If I understand the note above, you see ~100s to run the command using pyflow. Am I reading this correctly? Please confirm how long the pyflow and simulator runs take in your environment. If possible, please use Questa so we can do an exact comparison.
Can you confirm the version of PyBoolector that you are using (pip show pyboolector)? I have <IP_ADDRESS>10617.2  installed in my environment.
Thanks and Best Regards,
Matthew

Hi @aneels3,
Okay, I've added in a very rudimentary profiler to collect statistics on what classes are being randomized and their characteristics. It is enabled by setting the VSC_PROFILE environment variable to '1', and produces information like the following at the end of execution:
PyVSC Profile:
RandSites:
    "riscv_instr_gen_config @ /project/fun/pyvsc/perf_investigation/riscv-dv-with_constraints/pygen/pygen_src/test/riscv_instr_base_test.py:73":
      count: 1
      randinfo:
        sets: 5
        cfields: 25
      time: 26
        min: 26
        max: 26
        avg: 26
      sat-calls: 22
        min: 22
        max: 22
        avg: 22
    "<unknown> @ pygen/pygen_src/riscv_asm_program_gen.py:393":
      count: 30
      randinfo:
        sets: 1
        cfields: 1
      time: 44
        min: 1
        max: 2
        avg: 1
      sat-calls: 120
        min: 4
        max: 4
        avg: 4
    "riscv_privileged_common_seq @ pygen/pygen_src/riscv_asm_program_gen.py:519":
      count: 1
      randinfo:
        sets: 0
        cfields: 0
      time: 0
        min: 0
        max: 0
        avg: 0
      sat-calls: 0
        min: 0
        max: 0
        avg: 0
    "riscv_instr @ pygen/pygen_src/riscv_instr_stream.py:260":
      count: 1000
      randinfo:
        sets: 3
        cfields: 15
      time: 9543
        min: 6
        max: 99
        avg: 9
      sat-calls: 7611
        min: 5
        max: 10
        avg: 7

We can see that most of the total time is going into randomization of the 'riscv_instr' class. One opportunity for performance improvement here would be to somehow reduce the number of SAT calls.
By the way, I noticed that the repo you pointed me to had profiling enabled. I disabled profiling to see what overhead it might be introducing. My runtime with profiling disabled is 15.379s.
Can you confirm your total runtime with profiling disabled?

Hi @aneels3,
My first step in investigating this is to establish some baseline numbers: performance for pyflow and performance for a simulator. I am using PyVSC 0.4.6. Using Questa as the simulator, I see the following when running the command above:

pyflow: 23.55s
questa: 11.57s

If I understand the note above, you see ~100s to run the command using pyflow. Am I reading this correctly? Please confirm how long the pyflow and simulator runs take in your environment. If possible, please use Questa so we can do an exact comparison.
Can you confirm the version of PyBoolector that you are using (pip show pyboolector)? I have <IP_ADDRESS>10617.2 installed in my environment.
Thanks and Best Regards,
Matthew

Thanks @mballance
About 100s, I was referring to the logs that I have generated for enabling all constraints and disabling all constraints.
With Constraints Enabled: Total Execution time for 1000 instr using Pyflow is 438 seconds
With Constraints Disabled: Total Execution time for 1000 instr using Pyflow is 342 seconds
The difference is about 96 Seconds (Which I have considered roughly 100 in my previous reply).
The generation time using VCS, QuestaSim, etc is quite good and is around less than 30s.
As I have mentioned earlier about the particular PyVSC version because currently the randomize_gpr() is throwing an error with the latest PyVSC release.
So I check all the PyPI distribution of PyVSC and end up with this version where the randomize_gpr() is working fine.
I have installed the pyvsc to a particular version like this
pip3 install pyvsc==<IP_ADDRESS>10523.1
and the pyboolector version that I am using is <IP_ADDRESS>00615.1 which comes with the above particular pyvsc version.
I think in your case with pyvsc 0.4.6 the generation is not done properly and which ends you up with 23.55 sec execution time.
Can you confirm whether you are getting riscv_arithmetic_basic_tests.S asm file in output_*/asm_test/ dir or not?

Hi @aneels3,
I configured a Python virtual environment to use pyvsc <IP_ADDRESS>10523.1 and pyboolector <IP_ADDRESS>00615.1. With this configuration, I see:

Profiling enabled in riscv_instr_base_test.py: 106.33s
Profiling disabled in riscv_instr_base_test.py: 98.60s

Continuing to use pyvsc <IP_ADDRESS>10523.1, but upgrading to pyboolector 3.2.220210617.2, I see:

Profiling enabled in riscv_instr_base_test.py: 30.389s
Profiling disabled in riscv_instr_base_test.py: 21.601s

Can you confirm you see a similar different when upgrading the PyBoolector version? The 3.2.2 series uses a different SAT solver (CaDiCal) by default, which does have  significantly better performance in some cases.
BTW, how should I confirm that randomize_gpr is failing with newer pyvsc versions? Is there a user-visible failure, or is the error trapped and hidden from the user? I do have an output .S file when using the latest pyvsc version, and it appears to be similar to what is produced with the 0.3.7 version of pyvsc.
Best Regards,
Matthew

Hi @mballance
Thanks for the numbers with different runs.
I have configured a similar environment and ran the tests.
Here are the results:
PyVSC = <IP_ADDRESS>10703.2, Pyboolector = <IP_ADDRESS>00615.1
Profiling enabled in riscv_instr_base_test.py: 167s
Profiling disabled in riscv_instr_base_test.py: 139s
PyVSC = <IP_ADDRESS>10703.2, Pyboolector = <IP_ADDRESS>10617.2 (Latest Release)
Profiling enabled in riscv_instr_base_test.py: 105s
Profiling disabled in riscv_instr_base_test.py: 79s (For now, this is quite good for 1000 instr as compared to ~430s)
PyVSC = <IP_ADDRESS>10523.1, Pyboolector = <IP_ADDRESS>00615.1
Profiling enabled in riscv_instr_base_test.py: 431s
Profiling disabled in riscv_instr_base_test.py: 407s
PyVSC = <IP_ADDRESS>10523.1, Pyboolector = <IP_ADDRESS>10617.2
Profiling enabled in riscv_instr_base_test.py: 131s
Profiling disabled in riscv_instr_base_test.py: 105s
I see some improvement with the latest pyboolector version, so yes you are absolutely right about the CaDiCal SAT solver.
For the above results, I am using profiling_with_constraints branch.
Did you use the same branch for your observation? because in another branch (prof_without_constraints), all the constraints are commented.
And regarding the error, I was addressing.
It was something related to boolector node.  Surprisingly I am not getting any error in any of the pyvsc versions.
for the time being, I think let's forget about it (If I encountered such in the future, I will discuss it in a different issue).

One thing I would like to add here is that,
I was only upgrading or downgrading the pyvsc version for the whole time, believing the latest pyvsc would by default installed the latest pyboolector by checking the pre-installed pyboolector version.
I think I was using the 3.2.0 version of pyboolector for the whole time and I was only upgrading the pyvsc version, whenever there was a new release of it.
Is it possible to install both pyvsc and pyboolector with the latest version (if the user has an older version pyboolector pre-installed) while installing the pyvsc from PyPi?

That's a good point, @aneels3. I've left the version unspecified, since PyVSC functions correctly (to my understanding) with all versions of PyBoolector. However, as you point out, performance is a different story. I'll add a version qualifier to the setup.py to ensure that at least PyBoolector 3.2.2 is installed.
I think ensuring the latest versions of both pyboolector and pyvsc would require you to run pip install --upgrade pyvsc pyboolector. I don't think the 'upgrade' directive extends to package dependencies automatically.

That's a good point, @aneels3. I've left the version unspecified, since PyVSC functions correctly (to my understanding) with all versions of PyBoolector. However, as you point out, performance is a different story. I'll add a version qualifier to the setup.py to ensure that at least PyBoolector 3.2.2 is installed.
I think ensuring the latest versions of both pyboolector and pyvsc would require you to run pip install --upgrade pyvsc pyboolector. I don't think the 'upgrade' directive extends to package dependencies automatically.

Thanks for all the effort.
Is there any other way to minimize the number of SAT calls?
Any WA suggestion, so that I can dig it more on this to boost the speed.

Hi @aneels3,
I experimented with reducing the number of SAT calls (1 is required to establish a valid solution, and the rest are used to improve the random distribution), but this didn't seem to do much to reduce the time. Boolector does operate incrementally, so it seems consistent that the initial solve is the most time-consuming.
In approaching performance, it's probably good to have some target in mind. Do you have a specific performance point in mind?Are you attempting to get the best performance, while maintaining the same constraints used in the SV version? Are you attempting to get the best performance without a need to maintain the same constraints?

Hi @aneels3,
I experimented with reducing the number of SAT calls (1 is required to establish a valid solution, and the rest are used to improve the random distribution), but this didn't seem to do much to reduce the time. Boolector does operate incrementally, so it seems consistent that the initial solve is the most time-consuming.
In approaching performance, it's probably good to have some target in mind. Do you have a specific performance point in mind?Are you attempting to get the best performance, while maintaining the same constraints used in the SV version? Are you attempting to get the best performance without a need to maintain the same constraints?

Hi @mballance
you are right about the SAT calls.
Currently, we are seeing a 20x(slower) difference in performance w.r.t to SV version.
The idea is to minimize as much as possible.
Yes, we want to use the same constraints that are used in the SV version (Most of them are already implemented with the help of pyvsc constraints).

I have been experiencing a similar performance(as it was before) after <IP_ADDRESS>10710.1 release.
with pyboolector version 3.2.2
I am not sure about this behavior.
I think you have added the support for the linting feature in 0.5.1.
Is there something that is missed?
I really don't have any concrete data to prove at this point in time.

Hi @aneels3,
This is puzzling, since I'm not aware of any recent changes that should substantially impact performance. Can you turn on PyVSC profile statistics and share what you collect? To do this, please set the VSC_PROFILE environment variable to 'true' before running the instruction generator (eg export VSC_PROFILE=true). A set of statistics will be reported on exit.
Are you using the 'master' branch of riscv-dv, or some other branch? I want to be sure that I look at the same branch when I look at your profile data.
Thanks and Best Regards,
Matthew

Hi @mballance
I am not able to generate the profile stats even after setting the VSC_PROFILE. I have set it like this setenv VSC_PROFILE true since I am using cshell.
Yeah, you can use the master branch as well. Make sure to decrease the instruction count from here.
Thanks and Regards,
Anil

I have deliberately popped the error to see the stats in the console.

Without any error, I am not getting the stats in the console

Hi @aneels3,
This is truly a strange case. I spent yesterday evening trying to figure out the reason the profile isn't displayed without success. Hopefully you'll have some insight that will help.
PyVSC registers an atexit hook (https://docs.python.org/3/library/atexit.html) that displays the collected profile data. In my test environments, the atexit hook method is called from the same process as the Python code in my test. I'm observing that the Python code in your test runs in a different process (different process ID) than the atexit hook. Are you aware of anything in the pygen code that creates threads, processes, etc?

BTW, after looking at the actual solver profiler and generation runtime, I'm seeing the following:

~6.2s in solving. This includes both time in the SMT solver and time to do data-processing associated with solving
Total runtime of ~26s

It looks to me like something else in the generation process is taking the bulk of the time. Can you run with Python profiling to see where the hotspots are?

With pyvsc latest version

With pyvsc <IP_ADDRESS>10710.1

I have removed the multiprocessing feature in both cases above.
I don't think the atexit hook is a cause here. You have added this in the 0.4.7 release and in 0.4.7 everything is fine.
After 0.5.0 release it is taking some time in the getmodule  and isinstance class for some reason.
I have run the test for 100 instructions(hardcoded). You can see the difference in generation time in the above SS (13s vs 56s).
Is there something in the 0.5.1 release that uses the getmodule and isinstance method a lot of times?

Hmm... I've gone back and performed a diff between 0.5.0 and latest source. It seems the primary difference is that PyVSC captures source information (file, line number) in 0.5.0. I'm surprised this has such an overhead in the RISC-V generator, since it should only happen once when a class instance is created. It shouldn't bring overhead for each randomization unless a new instance of the class is created for each randomization...

Okay.
Since the generator uses lots of instances (For eg. if there are 1k instructions there will be 1k instances with different id). It looks like somehow those functions get called every time a new instance is created or copied.
Because of it, I am seeing the overhead as I am increasing the instr_cnt.
I am not able to think about any WA for this as of now.
Let me know if you can suggest me something to get going with this.

Yes, that's my understanding of the issue. My testing involved creating a single instance with multiple randomizations, which didn't highlight how expensive these operations are. My current thinking is to find a way to turn line number on constraints off by default, and give the user a way to enable them for debug purposes. I'll let you know in the next few days whether that turns out to be a viable solution.

Hi @mballance
I think that's a good idea because I can't change the whole structure as of now.
It would be great if you can provide a Command line switch for this.
Thanks

Hi @aneels3,
Release 0.5.5 has source-information collection and solve-failure diagnosis disabled by default. Please give this a try and report back on performance results.
Thanks and Best Regards,
Matthew

Hi @mballance
This indeed resolved the issue with the performance. Thanks a lot for the changes.
It would be great if you can think about creating a CLI switch for the solve-failure diagnosis so that the users can use this based on their requirements.

Hi @aneels3,
Yes, that's a good idea. Let me give some thought to how this might work best. I'll add a section to the documentation on debug support, then we can close this.

Hi @aneels3,
I've added environment-variable support for enabling solve-fail diagnostics and source-info capture. I've documented the debug controls here: https://pyvsc.readthedocs.io/en/latest/debug.html


Switch to new Google Cloud API.

Among many it allows to specify a path to JSON encoded service account key
or use credentials set through GOOGLE_APPLICATION_CREDENTIALS.

What's the status of this? It was created almost two years ago yet it's still open.


OT: Credentials for Default MySQL on Windows or macOS?

This is a bit offtopic, but someone here might know about this.
What are the credentials for the Default MySQL on Windows and macOS environments? The root:root only seem to work on Linux.

See Software in virtual environments for GitHub Actions.

Windows - It seems like Windows environment have not started MySQL, have not installed MySQL to a system service, and mysqld does not work. It maybe a bug in Actions.
Mac OS - No, it seems like MacOS environment does not contain MySQL.

Thanks, I indeed misread the macOS environment information - on second look I can not see MySQL there :( Shame.


Add libfdk_aac

Would be nice if libfdk_aac was added to ./configure

Please read the readme. libfdk_aac is a non-free codec that needs explicitly to be enabled.


OR condition is not formatted

Sample code:
fn main() {
    /* Common case: The double precision result is fine. */
    if (ui & 0x1fffffff) != 0x10000000  /* not a halfway case */
        || e == 0x7ff /* NaN */
    || (result - xy == z as f64 && result - z as f64 == xy) /* exact */
        || fegetround() != FE_TONEAREST
    /* not round-to-nearest */
    {}
}

What I expect:
fn main() {
    /* Common case: The double precision result is fine. */
    if (ui & 0x1fffffff) != 0x10000000  /* not a halfway case */
        || e == 0x7ff /* NaN */
        || (result - xy == z as f64 && result - z as f64 == xy) /* exact */
        || fegetround() != FE_TONEAREST /* not round-to-nearest */
    {
    }
}

cc #3167.

I believe this has the same underlying issue as #3167. https://github.com/rust-lang/rustfmt/issues/3167#issuecomment-437583821 explains that we don't yet handle comments between binary operators and I believe that's what's causing the issues here.


After I add stage name, take me directly to Add a Step modal

Rancher versions: 2.0 master 4/17
Steps to Reproduce:

Add Pipeline
Add Stage

Results:  After I add the step name, it takes me back to Add Pipeline page, should take me directly to Add A Step modal.

@biblesyme - I meant Stage.


How to use horizontal swing?

I love this integration in combination with the simple thermometer.
I got everything working but I just cannot figure out how to add the horizontal swing to the simple thermometer.
This is my current card config:
    - type: 'custom:simple-thermostat'
      entity: climate.wohnzimmer
      header:
        name: Wohnzimmer
      control:
        hvac: true
        preset:
          _hide_when_off: true
        swing:
          _hide_when_off: true
        fan:
          _hide_when_off: true

What do I need to add to show a control for the horizontal swing?
Thanks for your help!

v1.1.0 has easier horizontal swing support


General purpose abstraction to s3 bucket management

After using architect on a few projects the last month I'd just like to say how great an experience it's been so far. Really well done to everyone involved! 🍻
As I grow one project in particular I'm beginning to feel like there's some potential enhancements that could be made around management of application specific s3 buckets. I acknowledge that the @static configuration option is available, but feel like it's quite limiting.
What I'm imagining is something similar to that of @table, where by you define a buckets and various attributes.
For example:
@app
arc-s3-custom-bucket

@storage
user-uploaded-images # similar to @static this must be unique across all s3, though arc could automatically assign test/staging/production to the bucket?
  create Lambda # similar to @table the ability to assign lambdas to CRUD operations
  public false # a way to control access to objects in the bucket

with the above .arc file in mind, I'd imagine an additional @architect/storage helper be developed to aid in interacting with a preconfigured AWS.S3 service based on environment.

While we agree with some of the use cases here, we haven't heard back on this in a while, and haven't had a lot of community demand. I think we'll likely tackle this in the future, but it doesn't seem to be a priority at the moment. Please feel free to reopen if you'd like to discuss further paths to implementation, submit a PR, or think we need to more aggressively bubble this up. Thanks for this feedback!