Allow multiple owners for components

Currently a kind: Component needs to specify an owner inside spec. But this information may depend of who you ask!
Is the owner the lead engineer, the team, the guy paying , the manager.... Depends!
So I propose that:
spec:
  owner<EMAIL_ADDRESS>
Becomes:
spec:
  owners:
    team<EMAIL_ADDRESS>
Which now allows:
spec:
  owners:
    team<EMAIL_ADDRESS>    manager: xx
    architecture: xx
    documentation: yyy
    quality: zzzz
    sales: yyy
    consumer_service: ddd

The change would not limit the owner keys to the above, it is just an example.

Good feedback! 👍
For now we've just gone with a simple dumb string as the owner, as we haven't really tied it to any kind of identities yet, and we still need to figure out exactly how we want to do that. I do think we're going to have to extend the schema to allow for different types of owners later on.
In the internal version we do have a rule that some team should be the owner of a service, and it might be a model that we would want to encourage through the schema, but not enforce.

We are also interested into having multiple owners. Here is an inspiration from one of the Demo videos:

Now that the relationships and basic ownership is in place, we should have a look at this again. I wonder how such a feature can be implemented, some ideas:
Having multiple distinct owner fields or a complex structure in the entity as Jordi suggested:
spec:
  owners:
    technical<EMAIL_ADDRESS>    product: xx
    ...

or
spec:
  technicalOwner<EMAIL_ADDRESS>  productOwner: xx

In that case we have multiple options for storing it internally:

Having a new technicalOwnerOf relationship and some more relationships following the same pattern. This might get out of hand fast. But would be extensible by following Adding a new relation. One advantage is, that this might be introduced as a non breaking change as the ownership relationship isn't polluted with additional relationships.
Another option is to use the metadata on relationships. In the metadata of an ownerOf relationship could be an additional field like responsibility that marks what kind of an owner the user/group is. Consumers therefore has to filter which responsibility they want to display. This can be an advantage and a disadvantage: Existing code will just display the additional ownerships, but might not make clear which responsibility they have.

Responsibilities as an attribute of the the owning group/users:
Instead of adding the additional responsibilities to the components, we could also add them to groups. So it can be derived from the group which ownerships applies. However this might not be desired if the responsibilities differ between different components owned by the same group.
Here are also multiple implementations possible:

Users have an explicit role attribute. We filter the members of the group by their role and can therefor extract the additional ownerships. This more or less expects that each team has one product owner and so on.
Having different memberOf relationships OR having the metadata as in the solution above.
I think this is quite limiting.

We would be interested to help with the implementation. If we have a goal we can start with an implementation at the beginning of the year.

For what it's worth, internally we have a variant of the second solution above. Groups have a number of extra attributes (in the actual backing systems - not just Backstage) - who's the productManager, designOwner, and techOwner. These map to a user, and are completely detached from the concept of being a member or not of the group. This is the data that we display in the screenshot.

I think that the input model does not necessarily have to map 1:1 to the output model, by the way. Even if we used our source data, we could still emit a hasTechOwner relation on each Component that is owned by the group with that techOwner attribute, if that was what the end result model turned out to be.

Yeah, you are right. The models don't to match, we can rewrite that in a processor. So it's kind of a question about the internal model:

Having separate relations techOwnerOf, productOwnerOf or
Adding a responsibility to the metadata of the existing ownerOf relationship.

Yep. We're more and more starting to think in terms of the relations first. And then, the spec just "follows along" and is one of many ways for people to express things that ultimately end up being relations.

I thought a little bit about this. Having a ownerOf relationship with an additional metadata "responsibility" (technical, product, ...) has the advantage that all code that is working with the ownerOf relationship will continue to work. One example would be the ownership filter of the catalog. If every adopter implements their own relationship it would be hard to still filter by ownership of the user as every instance might have different relations for that. That could be solved if techOwnerOf would be in addition to ownerOf instead of replacing it.

If it is of any use, as we wait for the format here to stabilize we are adding a few sections to metadata and we find it extensible and useful to have role. For instance:
  people:
    - role: "engineering manager"
      email<EMAIL_ADDRESS>    - role: architect
      email<EMAIL_ADDRESS>
or in a different section:
  channels:
    - url: https://xxxx.slack.com/messages/....
      automated_messaging: false
      role: slack
    - role: email
      automated_messaging: true
      url<EMAIL_ADDRESS>
This has been working pretty well, we have 120 services now described by this and no issues. So I'd encourage here also to go for something like that
owner: x
scope: y

Scope or role or responsibility. Not sure. But something which values can be extended in the future.

@JordiPolo If we formalise this, I think we'd probably want to make the owner to be a ref to a User rather than an email. But yea, to be determined.

Totally makes sense, our format out there predates the definition of User.
Just wanted to make a case for a format that allows to use values to extend it for future usage.
We have not adopted User because we have tons of people working on multiple projects and we do not know where to put their User, in one project, in all, in a different repo just forthis? etc

Would be still interested in this, we want two owners, a technical and a product one. We could already archive this by extending the model on our end, but I wonder if this is something that is part of the core Backstage experience.
@Rugvip @benjdlambert @freben Is this something we should reopen?

I came here via a question I put on Discord. We're looking into using Backstage, and having a Product Owner listed per product is important to us, to have a main point of contacts. A lot of project/repositories are ownership of larger groups of people, but it can be unclear who to contact. Of course we could hack it by adding metadata properties like labels or annotations, but making it part of the spec is preferred.
I like the idea of defining a single owner to prevent having teams passing on responsibility: more owners == no owner, so I wouldn't touch that.
So that leads me to think about the idea of @JordiPolo where an addition list of related Groups or Users can be linked to a Component. Just to get the discussion going, something like:
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: artist-web
  description: The place to be, for great artists
spec:
  type: website
  lifecycle: production
  owner: artist-relations-team
  system: artist-engagement-portal
  organization:  # could be a different term
    - contact: group:the-ops-team
      role: operational-support
    - contact: group:artists-solution-architects
      role: architect
    - contact: user:john-doe
      role: product-owner

I'm quite new to the standard, so feel free to point out errors or inconsistencies. And please provide feedback.

Just to gauge things:
How far - how many percent on the way to perfection - does it go, if you were to place contact info in the metadata.description field on human readable form, and not having it in the model as such? It's tempting to put anything and everything in the model, but that also puts even more burden on teams that adopt this, and sometimes less is more. Especially when it then comes to people using roles a little bit differently and the model evolving over time etc...

@jcarres-mdsol Hm let me see if I get this right. You have a signed in user whose auth info has user:default/jcarres, and you have an entity that has spec:owner: somebodyElse but also a people entry saying basically techOwner: jcarres? And you want to see that component as owned when you sign in?
If that's the correct understanding, then a custom processor could indeed translate the techOwner field into an ownedBy relation such that you'll see it in your owned list in the catalog.

Yeah all relations are without restraints; you can have any number of ownedBy edges to users and groups. So I think that should make your situation supported perfectly (but with some custom extra fields in the model, which is fine).

In the yaml definition of the entity itself we can't have multiple owners. We import these yaml via the github entityprovider.
So I imagine that the implementation here would be to write our own processor and add it to backstage/packages/backend/src/plugins/catalog.ts which would read from Backstage's DB each component, read the people, find these users entities in Backstage then add a OWNED_BY relationship there and store that.
Does that sound correct?

Sure, apart from that the processor doesn't control what entities it processes - the processing loop is in control, and occasionally calls your processor methods with an entity, one by one. So your processor would probably hold some internal state, and then for each entity make a lookup in that state to see what decorations it should make / what things it should emit (depending on whether the processor just puts stuff in the entity data, or just skips that step and emits relations directly instead of delegating that to another processor)

I've tried this but can't make it work.
If I add relations were each is something like this:
 {
            type: RELATION_OWNED_BY,
            targetRef: stringifyEntityRef(user),
   }

Then entity page complains that a property is missing.
If I do something like:
 {
           type: RELATION_OWNED_BY,
           targetRef: stringifyEntityRef(user),
           target: {
               kind: "user",
               namespace: "default",
               name: name,
            }
  }

Then I get Error: Malformed envelope, /relations/0 must NOT have additional properties.

Are you emitting the relationship from the processor or directly mutating the entity with the relation? They should be emitted using the emit method.

So you want to look at some bad code eh
    async preProcessEntity(
      entity: Entity,
      _location: LocationSpec,
      _: CatalogProcessorEmit,
      _originLocation: LocationSpec,
    ): Promise<Entity> {

    // Simplified code, stakeholders is the name of our metadata field with people's information
       const stakeholders = getStakeholders(entity);
       const relations = stakeholders?.map(stakeholder => entity_from_stakeholder_data(stakeholder));
      //  Here I've verified that `relations` is correct and played with it as I explained in my previous message

      return merge(
        {
          relations: relations
        },
      entity);
    }
  }

I do not know what I'm doing, simply copied the approach here thinking that if it works there, it would work for me.

@jcarres-mdsol copy the approach here instead 😁

It worked!
In fact we have also multiple teams in some projects and decided to do this first, anyways, my I'm-not-typescript-developer code as reference if someone happens to bump into this ticket
const getTeams =(entity: Entity): string[] => {
  const teams = entity?.metadata.teams;

  if (!Array.isArray(teams)) {
    return [];
  }
  const result = [];

  for (const [_index, entry] of teams.entries()) {
    const team = entry!;
    for (const [key, value] of Object.entries(team)) {
      if (key === "number") {
        result.push(value?.toString() || "");
      }
    }
  }
  return result;
}

class ExtendedOwnershipProcessor implements CatalogProcessor {

  getProcessorName(): string {
    return 'ExtendedOwnershipProcessor';
  }

  async postProcessEntity(
    entity: Entity,
    _location: LocationSpec,
    emit: CatalogProcessorEmit,
  ): Promise<Entity> {
    const selfRef = getCompoundEntityRef(entity);

    // Copy/paste doEmit function from the link in the comment above
    //function doEmit() {}

    const teams = getTeams(entity).map(number => `team-${number}`);

    doEmit(
      teams,
      { defaultKind: 'Group', defaultNamespace: 'default' },
      RELATION_OWNED_BY,
      RELATION_OWNER_OF,
    );

    return entity;
  }
}

I still wish to be able to give a role to ownershp relationships but this right now covers my most pressing use-cases.
Maybe we should close this issue?

Happy it works! Closing issue 👏


[spike] test travis's ability to commit

Spike work completed and an official PR was opened for this work


cleanup: delete catsrc and operatorgroup even if subscription is not found

Description of the change:
Updates the operator-sdk cleanup subcommand to cleanup the catalog source and operator group even when a subscription for the operator does not exist.
Motivation for the change:
In the event that a user runs operator-sdk run and an error occurs after the operator group and catalog source are created, but before a subscription is created, the operator group and catalog source must be manually cleaned up before retrying the operator-sdk run command.
In 1.1.0, the operator-sdk cleanup command will not cleanup the operator group and catalog source in this scenario.
This PR updates the operator-sdk cleanup command so that even when a subscription does not exist, it looks for the expected catalog source and operator group, and deletes them if they exist.
Closes #4085
Checklist
If the pull request includes user-facing changes, extra documentation is required:

[x] Add a new changelog fragment in changelog/fragments (see changelog/fragments/00-template.yaml)
[ ] ~Add or update relevant sections of the docs website in website/content/en/docs~

/cherry-pick v1.1.x


Spelling mistake

There is is a spelling mistake in the documentation of the ICmpEditorUpdatable-Interface on line 13:

Implement this interface in CComponents...

Fixed.


Add Xyce 7.4 (second attempt)

Motivation for this change
Added the analog circuit simulator Xyce previously not present in nixpkgs.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

@bgamari @rongcuid I see that you are the maintainers of ngspice. Maybe you are interested in reviewing this addition of Xyce :-)

I believe that failing of "trilinos, trilinos.passthru.tests, xyce, xyce.passthru.tests on x86_64-linux" was due to an unrelated commit in master that in the mean time has been fixed. I've re-tested my patch on master commit f9a214044be750fb8e186b21e4dbe90ad2e2c0b2 and on my machine (NixOS 21.11 linux-x86_64 )it built successfully.

Hi @talyz, my PR is ready for review. Is somebody going to check it, or should I notify somebody?

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/712

Please don't forget to squash the commit history together that it looks a lot nicer.

Hi @SuperSandro2000, thanks for the review. I merged your changes and squashed the commits.
Regarding your note about fetchFromGitHub: I confirm that using fetchFromGitHub doesn't retrieve the documentation due to the file https://github.com/Xyce/Xyce/blob/master/doc/.gitattributes, see also https://groups.google.com/g/xyce-users/c/MxeViRo8CT4. However, it works for the regression tests and so I changed that.
I was surprised in doing this change to find out that the permission of files are different and this caused some failures. To fix those I slightly modified the postPatch phase to restore necessary permissions.

I am currently pushing little fixes to your commit messages. After that I think we can merge,

xyce fails to build
Testing restricted mode for {,r}epstopdf
!!! Error: Option forbidden in restricted mode: --gscmd
resolvers       | resolving | variable 'SELFAUTOLOC' set to '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/bin'
resolvers       | resolving | variable 'SELFAUTODIR' set to '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021'
resolvers       | resolving | variable 'SELFAUTOPARENT' set to '/nix/store'
resolvers       | resolving | variable 'TEXMFCNF' set to '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf/web2c'
resolvers       | resolving | variable 'TEXMF' set to ''
resolvers       | resolving | variable 'TEXOS' set to 'n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021'
resolvers       | resolving |
resolvers       | resolving | using configuration specification '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf/web2c'
resolvers       | resolving |
resolvers       | resolving | looking for regular 'texmfcnf.lua' on given path '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf/web2c' from specification '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf/web2c'
resolvers       | resolving | found regular configuration file '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf/web2c/texmfcnf.lua'
resolvers       | resolving |
resolvers       | resolving | loading configuration file '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf/web2c/texmfcnf.lua'
resolvers       | resolving |
resolvers       | resolving | locating list of 'home:.texlive2021/texmf-config' (runtime) (tree:///home:.texlive2021/texmf-config)
resolvers       | methods | resolving, method 'locators', how 'uri', handler 'tree', argument 'tree:///home:.texlive2021/texmf-config'
resolvers       | trees | locator 'home:.texlive2021/texmf-config' not found
resolvers       | resolving | locating list of 'home:texmf' (runtime) (tree:///home:texmf)
resolvers       | methods | resolving, method 'locators', how 'uri', handler 'tree', argument 'tree:///home:texmf'
resolvers       | trees | locator 'home:texmf' not found
resolvers       | resolving | locating list of '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-config' (cached)
resolvers       | methods | resolving, method 'locators', how 'uri', handler 'file', argument '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-config'
resolvers       | files | file locator '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-config' found as '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-config'
resolvers       | resolving | hash '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-config' appended
resolvers       | resolving | locating list of '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var' (cached)
resolvers       | methods | resolving, method 'locators', how 'uri', handler 'file', argument '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var'
resolvers       | files | file locator '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var' found as '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var'
resolvers       | resolving | hash '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var' appended
resolvers       | resolving | locating list of '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-project' (cached)
resolvers       | methods | resolving, method 'locators', how 'uri', handler 'file', argument '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-project'
resolvers       | files | file locator '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-project' not found
resolvers       | resolving | locating list of '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-fonts' (cached)
resolvers       | methods | resolving, method 'locators', how 'uri', handler 'file', argument '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-fonts'
resolvers       | files | file locator '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-fonts' not found
resolvers       | resolving | locating list of '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-local' (cached)
resolvers       | methods | resolving, method 'locators', how 'uri', handler 'file', argument '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-local'
resolvers       | files | file locator '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-local' found as '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-local'
resolvers       | resolving | hash '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-local' appended
resolvers       | resolving | locating list of '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf' (cached)
resolvers       | methods | resolving, method 'locators', how 'uri', handler 'file', argument '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf'
resolvers       | files | file locator '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf' found as '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf'
resolvers       | resolving | hash '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf' appended
resolvers       | resolving |
resolvers       | methods | resolving, method 'generators', how 'uri', handler 'file', argument '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-config'
resolvers       | expansions | scanning path '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-config', branch '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-config'
resolvers       | expansions | 2 files found on 1 directories with 1 uppercase remappings
resolvers       | methods | resolving, method 'generators', how 'uri', handler 'file', argument '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var'
resolvers       | expansions | scanning path '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var', branch '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var'
resolvers       | expansions | 76 files found on 17 directories with 1 uppercase remappings
resolvers       | methods | resolving, method 'generators', how 'uri', handler 'file', argument '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-local'
resolvers       | expansions | scanning path '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-local', branch '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-local'
resolvers       | expansions | 1 files found on 0 directories with 1 uppercase remappings
resolvers       | methods | resolving, method 'generators', how 'uri', handler 'file', argument '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf'
resolvers       | expansions | scanning path '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf', branch '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf'
resolvers       | globbing | confusing filename, name: 'custMatTransform.py', lower: 'custmattransform.py', already: 'CustMatTransform.py'
resolvers       | globbing | confusing filename, name: 'setCustomAnchor.py', lower: 'setcustomanchor.py', already: 'SetCustomAnchor.py'
resolvers       | globbing | confusing filename, name: 'widg_addLabel.py', lower: 'widg_addlabel.py', already: 'Widg_addLabel.py'
resolvers       | globbing | confusing filename, name: 'widg_addPolyOpt.py', lower: 'widg_addpolyopt.py', already: 'Widg_addPolyOpt.py'
resolvers       | globbing | confusing filename, name: 'widg_editBezier.py', lower: 'widg_editbezier.py', already: 'Widg_editBezier.py'
resolvers       | expansions | 27938 files found on 2438 directories with 3621 uppercase remappings
resolvers       | resolving |
resolvers       | resolving | saving tree '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-config'
resolvers       | caches | hashing tree '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-config', hash '3b2575f069016ddeaabc9383b17339f2'
resolvers       | caches | hashing tree '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf/web2c/texmfcnf.lua', hash '1dbe71212a391f836718d0dfc155f8d1'
resolvers       | caching | preparing 'files' for '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-config'
resolvers       | caching | category 'files', cachename '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-config' saved in '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/3b2575f069016ddeaabc9383b17339f2.lua'
system          | lua | compiling '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/3b2575f069016ddeaabc9383b17339f2.lua' into '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/3b2575f069016ddeaabc9383b17339f2.luc'
system          | lua | dumping '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/3b2575f069016ddeaabc9383b17339f2.lua' into '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/3b2575f069016ddeaabc9383b17339f2.luc' stripped
resolvers       | caching | 'files' compiled to '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/3b2575f069016ddeaabc9383b17339f2.luc'
resolvers       | resolving | saving tree '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var'
resolvers       | caches | hashing tree '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var', hash 'fa3932c55184ae5e300dece358aaf0e4'
resolvers       | caching | preparing 'files' for '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var'
resolvers       | caching | category 'files', cachename '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var' saved in '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/fa3932c55184ae5e300dece358aaf0e4.lua'
system          | lua | compiling '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/fa3932c55184ae5e300dece358aaf0e4.lua' into '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/fa3932c55184ae5e300dece358aaf0e4.luc'
system          | lua | dumping '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/fa3932c55184ae5e300dece358aaf0e4.lua' into '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/fa3932c55184ae5e300dece358aaf0e4.luc' stripped
resolvers       | caching | 'files' compiled to '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/fa3932c55184ae5e300dece358aaf0e4.luc'
resolvers       | resolving | saving tree '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-local'
resolvers       | caches | hashing tree '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-local', hash '1930a27f3a8ec5f390f25bb2e7b72c7f'
resolvers       | caching | preparing 'files' for '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-local'
resolvers       | caching | category 'files', cachename '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-local' saved in '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/1930a27f3a8ec5f390f25bb2e7b72c7f.lua'
system          | lua | compiling '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/1930a27f3a8ec5f390f25bb2e7b72c7f.lua' into '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/1930a27f3a8ec5f390f25bb2e7b72c7f.luc'
system          | lua | dumping '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/1930a27f3a8ec5f390f25bb2e7b72c7f.lua' into '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/1930a27f3a8ec5f390f25bb2e7b72c7f.luc' stripped
resolvers       | caching | 'files' compiled to '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/1930a27f3a8ec5f390f25bb2e7b72c7f.luc'
resolvers       | resolving | saving tree '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf'
resolvers       | caches | hashing tree '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf', hash '28a03cdd14c63e5c76566ae30b8a381b'
resolvers       | caching | preparing 'files' for '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf'
resolvers       | caching | category 'files', cachename '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf' saved in '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/28a03cdd14c63e5c76566ae30b8a381b.lua'
system          | lua | compiling '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/28a03cdd14c63e5c76566ae30b8a381b.lua' into '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/28a03cdd14c63e5c76566ae30b8a381b.luc'
system          | lua | dumping '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/28a03cdd14c63e5c76566ae30b8a381b.lua' into '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/28a03cdd14c63e5c76566ae30b8a381b.luc' stripped
resolvers       | caching | 'files' compiled to '/nix/store/n7nrzyzsyb1l7qzna5f4pl3hmj3qr603-texlive-combined-2021/share/texmf-var/luatex-cache/context/1dbe71212a391f836718d0dfc155f8d1/trees/28a03cdd14c63e5c76566ae30b8a381b.luc'
mtxrun          |
mtxrun          | elapsed lua time: 0.227 seconds
error: 1 dependencies of derivation '/nix/store/rhd20m54ijhznli6q0skb7pggh0zffw5-xyce-7.4.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/gcz6hsfqa4ij9abhmb5v5afgwy3xjkll-xyce-7.4.0.drv' failed to build
error: build of '/nix/store/gcz6hsfqa4ij9abhmb5v5afgwy3xjkll-xyce-7.4.0.drv', '/nix/store/iww79xv9ibakksq2r4iijdl1c960j73q-trilinos-12.12.1.drv', '/nix/store/rhd20m54ijhznli6q0skb7pggh0zffw5-xyce-7.4.0.drv' failed

Hmm, interesring as it builds on my system
...
shrinking RPATHs of ELF executables and libraries in /nix/store/r33cky2fd5slgwbm1v1va5x8xki5p4vj-xyce-7.4.0-doc
strip is /nix/store/wzmzjrpaxfrsjc37v1km8blkxqwkdgg6-gfortran-wrapper-9.3.0/bin/strip
patching script interpreter paths in /nix/store/r33cky2fd5slgwbm1v1va5x8xki5p4vj-xyce-7.4.0-doc
checking for references to /build/ in /nix/store/r33cky2fd5slgwbm1v1va5x8xki5p4vj-xyce-7.4.0-doc...
/nix/store/l5g501109nb5xpirybcp3a7zsjnyg96a-xyce-7.4.0

However, this is with a checkout a few weeks old (my commits on top of b1ac54ae9be759a5254512d3794ccf198bd81eb6).
Is there a way to see more of the failure log?

Checking the OfBorg logs I see errors related to timeouts
Certification_Tests/BUG_681_SON/bsim4_681.........passed[sh]      (Time:   6s =   2.62cs +   0vs)
building of '/nix/store/fk1pw94cdmhz0z48bjcphvk9xp9g63q7-xyce-7.4.0.drv' timed out after 3600 seconds
error: build of '/nix/store/fk1pw94cdmhz0z48bjcphvk9xp9g63q7-xyce-7.4.0.drv' failed

Is it possible that the error that you reported is also related to a timeout? Compilation as well as running the tests takes a significant amount of time.

Adding enableParallelBuilding = true; to both xyce and trilinos speeds up the build quite a bit on multi-core machines.

Building both xyce and xyce-parallel works fine for me, at least with enableParallelBuilding = true.

Adding enableParallelBuilding = true indeed speeds up compilation. Both variants xyce and xyce-parallel still build and pass all tests for me.  I've added it to my last push.

I've not added enableParallelBuilding = true to trilinos as it uses cmake and it already builds in parallel.

I wasn't familiar with the sed "s|||" syntax (not even described in the info manual) and the escapeRegex function. They are indeed super useful to increase readability. If there are no other suggestions I'll squash commits and push with a clean log.

I cleaned up the log.

I wasn't familiar with the sed "s|||" syntax (not even described in the info manual) and the escapeRegex function. They are indeed super useful to increase readability.

Yep, they're both pretty neat. :) The sed syntax is actually even more general - you can use any character as a delimiter for the s command, so I usually pick one that isn't in the regex.

Thank you for the thorough review and merging my PR :-)

No problem :)


[sil-verifier] Stricter checks for branch arguments

What's in this pull request?

Resolved bug number: (SR-)

Before merging this pull request to apple/swift repository:

[x] Test pull request on Swift continuous integration.

Triggering Swift CI
The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:
Smoke Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please smoke test
    OS X platform               | @swift-ci Please smoke test OS X platform
    Linux platform              | @swift-ci Please smoke test Linux platform

Validation Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please test
    OS X platform               | @swift-ci Please test OS X platform
    Linux platform              | @swift-ci Please test Linux platform

Note: Only members of the Apple organization can trigger swift-ci.

IRGen does not support the following SIL values as arguments of branch instructions:

witness_method instructions
class_method instructions, where the method is an objc-method.

@swift-ci Please test and merge


fix: cte and subquery relation name and columns should be registered on query catalog

Trying to solve https://github.com/sqlc-dev/sqlc/issues/3720
CTE columns cannot be referred on main query
This is because if table name is not found on catalog, sqlc will attempt to find it on query catalog, but once it is found, it is skipped and not registered on QueryCatalog, making it invisible for it to be referred for the next steps. https://github.com/sqlc-dev/sqlc/blob/996f7d1dab74a913afd6739a55ae034a8ed9cac5/internal/compiler/resolve.go#L70-L73
Subquery relation name and columns cannot be referred on WHERE clause
Similar to above case, but this needs to be detected on *ast.RangeSubselect to register alias of subquery and its output column into QueryCatalog

just realized that there is open PR about this issue after I finished create this MR 👀 for now maybe I will wait for PR https://github.com/sqlc-dev/sqlc/pull/3220 to be merged as it has more complete tests


Wrap command text in UI

A fix for #26.

Changes look good to me. I like the left aligned commands better aswell.
Thanks for your contribution @qb60 , I appreciate it !


[glyf] bbox of transformed & anchored component calculated incorrectly

The attached ttx file of a TrueType font (Variable, but this seems irrelevant) has its “O” glyph composed of a pair of “C” glyphs, one rotated by 180 degrees lined up to the other using anchor points (i.e., flag ARGS_ARE_XY_VALUES unset). The XML description of “O” glyph reads:
<TTGlyph name="O" xMin="56" yMin="-12" xMax="612" yMax="672">
  <component glyphName="C" x="0" y="0" flags="0x204"/>
  <component glyphName="C" firstPt="0" secondPt="9" scale="-1.0" flags="0x4"/>
</TTGlyph>

When converted to ttf by ttx then back to ttx, the XML description becomes:
<TTGlyph name="O" xMin="-334" yMin="-1356" xMax="334" yMax="672">
  <component glyphName="C" x="0" y="0" flags="0x204"/>
  <component glyphName="C" firstPt="0" secondPt="9" scale="-1.0" flags="0x4"/>
</TTGlyph>

The glyph bounding box is calculated incorrectly.
It appears as if translation offset for the component is calculated before rotation. These should be done backwards.
SourceSansVariable-Roman.anchor.ttx.zip

this is the code that computes the coordinates for a glyph component:
https://github.com/fonttools/fonttools/blob/c6006a7f8c31cd8e7dddad58894e1ba6c87e1362/Lib/fontTools/ttLib/tables/_g_l_y_f.py#L843-L874
By reading the description of SCALED_COMPONENT_OFFSET and UNSCALED_COMPONENT_OFFSET flags in https://docs.microsoft.com/en-us/typography/opentype/spec/glyf, I am not sure whether these flags apply also to the case when a component  args are not XY values but point numbers.
fonttools defaults to UNSCALED_COMPONENT_OFFSET, i.e. the component is fist scaled/rotated and then moved.

  File "fontTools/ttLib/ttFont.py", line 774, in draw
  File "fontTools/ttLib/tables/_g_l_y_f.py", line 1002, in draw
  File "fontTools/ttLib/tables/_g_l_y_f.py", line 1096, in getComponentInfo
AttributeError: 'GlyphComponent' object has no attribute 'x'

yep
https://github.com/fonttools/fonttools/blob/c6006a7f8c31cd8e7dddad58894e1ba6c87e1362/Lib/fontTools/ttLib/tables/_g_l_y_f.py#L1094-L1100

And that comment is ... 20 years old :)

The number of fonts out there that use this feature must be rather small.

ok so, there are two separate issues here:

getComponentInfo doesn't support components with point numbers (so a glyph containing those can't be drawn using draw/drawPoints api)
the bbox is computed incorrectly because the Glyph.getCoordinates method is computing the offset (firstPoint - secondPoint) before applying the transformation, and then moving the transformed component by the unscaled offset.

The fix would be to compute the offset from the points only after the transform has been applied. Correct?

The fix for issue 2) would be to compute the offset from the points only after the transform has been applied. Correct?

Looking at FreeType's ttgload.c, that seems to be correct.

Although I'm not sure this line is correct:
https://github.com/fonttools/fonttools/blob/c6006a7f8c31cd8e7dddad58894e1ba6c87e1362/Lib/fontTools/ttLib/tables/_g_l_y_f.py#L850
I'm not sure I understand the corresponding FreeType code correctly, but perhaps firstPoint is an index into the point list of the last added glyph, and not the point list as a whole. See the start_point argument to TT_Process_Composite_Component().

I always understood firstPoint as being an index into the whole point list, i.e. the points of all the glyph components added so far

I always understood firstPoint as being an index into the whole point list, i.e. the points of all the glyph components added so far

I checked again using ftview and a test font which I made for this purpose and verified that this is indeed the case. firstPt is an index into the points added so far to the composite glyph, secondPt is the index into the list of points of the other glyph that is being added as a component to the current composite glyph.

@justvanrossum I know it's been 20 years since you wrote this :) but do you remember what you meant by that:

XXX Ignoring self.firstPt & self.lastpt for now: I need to implement
something equivalent in fontTools.objects.glyph (I'd rather not
convert it to an absolute offset, since it is valuable information).

What is fontTools.objects.glyph? How do you suggest we implement pen-drawing support for components with firstPt/secondPt nowadays?
I was thinking that, since pen protocol only accepts components with offsets, we compute that firstPt-secondPt and use that as the offset. But maybe you had something more clever in mind, as the comment suggests.

With fontToold.object.glyph I think I meant something like a proper glyph object, like defcon.Glyph. It sounds like I was pretty much too lazy to figure how to solve this one. I don't think there's a clean way for the pen protocol to handle this. In the meantime: I have never encountered fonts that even used this feature, so I never had any reason to look into it further.
Thinking about it now: I think we should convert it to an offset anyway, and live with the fact that the pen protocol is not capable of round-tripping this rather arcane TT feature.

(I think I had (beginnings of) fontTools.objects.glyph for a brief while, but for decided it was out of scope for fonttools. It didn't turn into something mature outside of fonttools either, though, I think it just died.)


Rewrite in Rust

Rewrite this in the Blazingly :fire: Fast :rocket: :rocket: :fire: language :crab: Rust :crab: :crab:

this


Use alpine as base image

Alpine is not that much bigger than google's distroless image. and
having this OS layer underneath will make it easier to debug.
(Currently: do we have a problem with the network interface name in our
config?)

Test: run ./build-docker.sh, output contains:
2021/11/01 16:34:23 Using base alpine:3.13 for github.com/honeycombio/refinery/cmd/refinery

I am very much in favour of this (it'll also make it easier to run Refinery using ECS on Fargate, related Slack thread here), but any chance something other than Alpine can be used?
Alpine is somewhat well known to have runtime issues due to musl. I had massive DNS issues in Kubernetes due to this, so I am now hyper-aware, even if some bugs were fixed.
Compared to Alpine's 2MB size, alternatives are indeed ten times bigger with Ubuntu images being ~27MB and Debian slim images being ~22MB, but I think the added size has minimal impact.

@Vlaaaaaaad I'm going to merge this PR using alpine as base image. Please raise an issue if you feel you'd still like use to consider us using another base image.


[Bug]: Nodes unable to start/connect after fresh creation

Description
I have a fresh setup of k3s using kube-hetzner and followed the instructions in the README, but the nodes do not start after creation.
module.kube-hetzner.null_resource.control_planes["2-0-control-plane-nbg1-1"] (remote-exec): Waiting for the k3s server to start...
module.kube-hetzner.null_resource.control_planes["0-0-control-plane-fsn1"] (remote-exec): Waiting for the k3s server to start...
module.kube-hetzner.null_resource.control_planes["1-0-control-plane-nbg1"] (remote-exec): Waiting for the k3s server to start...
module.kube-hetzner.null_resource.control_planes["2-0-control-plane-nbg1-1"] (remote-exec): Waiting for the k3s server to start...
module.kube-hetzner.null_resource.control_planes["0-0-control-plane-fsn1"] (remote-exec): Waiting for the k3s server to start...
module.kube-hetzner.null_resource.control_planes["1-0-control-plane-nbg1"] (remote-exec): Waiting for the k3s server to start...
module.kube-hetzner.null_resource.control_planes["2-0-control-plane-nbg1-1"]: Still creating... [12m10s elapsed]
module.kube-hetzner.null_resource.control_planes["1-0-control-plane-nbg1"]: Still creating... [12m10s elapsed]
module.kube-hetzner.null_resource.control_planes["0-0-control-plane-fsn1"]: Still creating... [12m10s elapsed]
module.kube-hetzner.null_resource.agents["2-0-egress"]: Still creating... [12m10s elapsed]
module.kube-hetzner.null_resource.agents["0-0-agent-small"]: Still creating... [12m10s elapsed]
module.kube-hetzner.null_resource.agents["1-0-agent-large"]: Still creating... [12m10s elapsed]

(Logs are from a different run, which is why the nodes are not matching, the output is the same)
After a while I get
╷
│ Error: remote-exec provisioner error
│
│   with module.kube-hetzner.null_resource.control_planes["2-0-control-plane-hel1"],
│   on .terraform/modules/kube-hetzner/control_planes.tf line 174, in resource "null_resource" "control_planes":
│  174:   provisioner "remote-exec" {
│
│ error executing "/tmp/terraform_380665410.sh": Process exited with status
│ 124
╵

The shell script itself looks fine
#!/bin/sh
systemctl start k3s 2> /dev/null
mkdir -p /var/post_install /var/user_kustomize
timeout 360 bash <<EOF
  until systemctl status k3s > /dev/null; do
    systemctl start k3s 2> /dev/null
    echo "Waiting for the k3s server to start..."
    sleep 3
  done
EOF

Checking the nodes' k3s service, shows that all of them have a similar issue; being unable to retrieve the ca certs:
Feb 01 22:14:31 k3s-control-plane-nbg1-lso systemd[1]: k3s.service: Scheduled restart job, restart counter is at 80.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ Automatic restarting of the unit k3s.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Feb 01 22:14:31 k3s-control-plane-nbg1-lso systemd[1]: Starting Lightweight Kubernetes...
░░ Subject: A start job for unit k3s.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit k3s.service has begun execution.
░░
░░ The job identifier is 16815.
Feb 01 22:14:31 k3s-control-plane-nbg1-lso sh[4059]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Feb 01 22:14:32 k3s-control-plane-nbg1-lso k3s[4066]: time="2024-02-01T22:14:32Z" level=info msg="Starting k3s v1.28.6+k3s1 (39a00015)"
Feb 01 22:14:32 k3s-control-plane-nbg1-lso k3s[4066]: time="2024-02-01T22:14:32Z" level=fatal msg="starting kubernetes: preparing server: failed to get CA certs: Get \"https://<IP_ADDRESS>:6443/cacerts\": dial tcp <IP_ADDRESS>:6443: connect: connection refused"
Feb 01 22:14:32 k3s-control-plane-nbg1-lso systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ An ExecStart= process belonging to unit k3s.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 1.
Feb 01 22:14:32 k3s-control-plane-nbg1-lso systemd[1]: k3s.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ The unit k3s.service has entered the 'failed' state with result 'exit-code'.
Feb 01 22:14:32 k3s-control-plane-nbg1-lso systemd[1]: Failed to start Lightweight Kubernetes.

I`m unable to curl the CA certificate as well
k3s-control-plane-hel1-zli:~ # curl -vk https://<IP_ADDRESS>:6443/cacerts
*   Trying <IP_ADDRESS>:6443...
* connect to <IP_ADDRESS> port 6443 from <IP_ADDRESS> port 41572 failed: Connection refused
* Failed to connect to <IP_ADDRESS> port 6443 after 3 ms: Couldn't connect to server
* Closing connection
curl: (7) Failed to connect to <IP_ADDRESS> port 6443 after 3 ms: Couldn't connect to server

Pinging this and also other nodes in the private network works fine. Seems not to be a networking issue.
ping <IP_ADDRESS> -c 4
PING <IP_ADDRESS> (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=64 time=0.025 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=64 time=0.038 ms
64 bytes from <IP_ADDRESS>: icmp_seq=3 ttl=64 time=0.044 ms
64 bytes from <IP_ADDRESS>: icmp_seq=4 ttl=64 time=0.054 ms

--- <IP_ADDRESS> ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3040ms
rtt min/avg/max/mdev = 0.025/0.040/0.054/0.010 ms

I have checked endless other issues, here and also the k3s repository. I'm at a loss, I'm stuck for 6 straight hours now with no clues on what the actual problem is.
Kube.tf file
locals {
  hcloud_token = local.credentials.hetznerCloudToken
}

module "kube-hetzner" {
  providers = {
    hcloud = hcloud
  }

  hcloud_token = var.hcloud_token != "" ? var.hcloud_token : local.hcloud_token

  enable_cert_manager = false

  ssh_public_key = file("~/.ssh/ssh.pub")
  ssh_private_key = file("~/.ssh/ssh")

control_plane_nodepools = [
    {
      name        = "control-plane-fsn1",
      server_type = "cpx11",
      location    = "fsn1",
      labels      = [],
      taints      = [],
      count       = 1
    },
    {
      name        = "control-plane-nbg1",
      server_type = "cpx11",
      location    = "nbg1",
      labels      = [],
      taints      = [],
      count       = 1
    },
    {
      name        = "control-plane-hel1",
      server_type = "cpx11",
      location    = "nbg1",
      labels      = [],
      taints      = [],
      count       = 1
    }
  ]

  agent_nodepools = [
    {
      name        = "agent-small",
      server_type = "cpx11",
      location    = "fsn1",
      labels      = [],
      taints      = [],
      count       = 1
    },
    {
      name        = "agent-large",
      server_type = "cpx21",
      location    = "nbg1",
      labels      = [],
      taints      = [],
      count       = 1
    },
    {
      name        = "egress",
      server_type = "cx21",
      location    = "fsn1",
      labels = [
        "node.kubernetes.io/role=egress"
      ],
      taints = [
        "node.kubernetes.io/role=egress:NoSchedule"
      ],
      floating_ip = true
      count = 1
    }
  ]

  load_balancer_type     = "lb11"
  load_balancer_location = "fsn1"

  dns_servers = [
    "<IP_ADDRESS>",
    "<IP_ADDRESS>",
    "2606:4700:4700::1111",
  ]
}

provider "hcloud" {
  token = local.hcloud_token
}

output "kubeconfig" {
  value     = module.kube-hetzner.kubeconfig
  sensitive = true
}

variable "hcloud_token" {
  sensitive = true
  default   = ""
}

Platform
MacOS 14.1.1 (23B81), M1

@yannicschroeer did you just deleted the .terraform folder or did you complete destroy all resources and then deleting also the .terraform folder?
I'm having the same issue and create a new terraform state and looking how I can "re-import" my cluster resources...

I resolved the issue, it wasn't a really terraform issue. I added another snapshot image with accidently the same labels as used by k3s. So deleting this snapshot and deleting the cloudinit from tf state via terraform state rm module.k8s.data.cloudinit_config.autoscaler_config[0] and then running terraform apply resolved the issue


SF 8.0 CU1

Upgrade packages to SF 8.0 CU1
https://github.com/microsoft/service-fabric/blob/master/release_notes/Service_Fabric_ReleaseNotes_80CU1.md
Description

Describe the proposed change here. Please be sure to communicate why this Pull Request should be accepted. If this fixes a bug or resolves a feature request, be sure to link to that issue.

Classification

Put an x in the box that applies.

Selection
Change Type
Description

[ ]
Bug Fix
Non-breaking change which fixes an bug

[x]
Feature Request
Non-breaking change which new behaviour

[ ]
Breaking Change
Fix or feature that would cause existing behaviour not to work as expected

Checklist

Put an x in the boxes that apply.

Selection
Description

[x]
I have read the contributing document.

[ ]
Style and unit tests pass locally with my changes.

[ ]
I have added tests that prove my fix is effective or that my feature works.

[ ]
I have added necessary documentation (if appropriate).

[ ]
Any dependent changes have been merged and published in downstream modules.

Further Details

If this is a relatively large or complex change, please explain why this solution was chosen and what alternatives were considered.

@darinegan Could you please merge this and publish on chocolatey.org?

Thanks again Jakub for your work ;)

Thank you for publishing! In my company we use Service Fabric a lot and rely on chocolatey.org for updating our developer workstations, so my involvement here is not only for the benefit of the community 😉


Cursor jumps to the end when completing

While i was setting up vim, i noticed that cursor jumped to the end for some characters. You can see in gif what is my configuration.
Note that i tried both with and without auto pairs plugin, but still same.
Here's the gif:

Here's one more that happened in sql:

Can you follow the step in ? Cause from my side I wasn't able to reproduce it.

Yeah it works with min vimrc. I'll try to figure out what's happening and write it here. Thanks.

Found it. It's vim-rsi causing the issue.
Sorry for bothering.


Add design document for the QUIC Stream Receive Buffers module

This is a draft as there are some details to be filled in. But please review and comment.
Checklist

[x] documentation is added or updated

This pull request is ready to merge

Merged to master branch. Thank you for the reviews.

Hi I will help you with everything you need but you have to subscribe to
the youtube channel and I will fulfill all your requests
https://youtu.be/aIaqr4eq18g
Pada Sel, 6 Sep 2022, 11:37 PM Tomáš Mráz @.***>
menulis:

This is a draft as there are some details to be filled in. But please
review and comment.
Checklist

documentation is added or updated

You can view, comment on, or merge this pull request online at:
https://github.com/openssl/openssl/pull/19149
Commit Summary

7ca852a
https://github.com/openssl/openssl/pull/19149/commits/7ca852a3bd8c3b71c8868f7686601c3edd359e62
Add design document for the QUIC Stream Receive Buffers module

File Changes
(1 file https://github.com/openssl/openssl/pull/19149/files)

A doc/designs/quic-design/stream-receive-buffers.md
https://github.com/openssl/openssl/pull/19149/files#diff-3bba8feea02e86eb1cd55f9a4a406ba8a20a770ce77faac8ae6149acbac1a5ff
(123)

Patch Links:

https://github.com/openssl/openssl/pull/19149.patch
https://github.com/openssl/openssl/pull/19149.diff

—
Reply to this email directly, view it on GitHub
https://github.com/openssl/openssl/pull/19149, or unsubscribe
https://github.com/notifications/unsubscribe-auth/APESUCR2XXDRQ6PNEYHQAXDV45QNLANCNFSM6AAAAAAQF6MKDU
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Check for missing arguments in /proc/self/cmdline

Avoids segfault, when argc is zero
Issue #1189

bot, add author to whitelist

@giuseppe PTAL

@rh-atomic-bot r+


owner/(ticdc): fix the disorder problem for ddl events execution

What problem does this PR solve?

Issue Number: close #5406
What is changed and how it works?
In asyncExecDDLJob(), using slice instead of map to retain the correct order for ddl events of a RENAME TABLEs job.
Before this fix: since ddlEventCache is a map and the map iteration order is random, the following case would happen for RENAME TABLE t1 to t2, t2 to t3:
RENAME TABLE t2 to t3;
RENAME TABLE t1 to t2;

As a result, the correct order is broken, which is not expected.
After this fix: using ddlEventCache as a slice, and an extra ddlEventDone slice to indicating whether each DDL event in ddlEventCache is finished. In this way, the correct execution order can be retained.
Check List
Tests

Unit test
Integration test

Release note
None

/run-all-tests

/merge

/run-all-tests

/run-all-tests

/run-all-tests

/run-verify

Codecov Report

Merging #5408 (dd45683) into master (687e248) will decrease coverage by 0.1309%.
The diff coverage is 51.7441%.

Flag
Coverage Δ

cdc
61.0056% <51.7441%> (+0.4069%)
:arrow_up:

dm
52.0938% <ø> (-0.3757%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.
@@               Coverage Diff                @@
##             master      #5408        +/-   ##
================================================
- Coverage   56.1240%   55.9930%   -0.1310%
================================================
  Files           522        529         +7
  Lines         65325      69455      +4130
================================================
+ Hits          36663      38890      +2227
- Misses        25094      26854      +1760
- Partials       3568       3711       +143

/merge


Add Betsy to speed up BC6 compression

Excerpt of #91150.
Implements the Betsy GPU texture compressor to drastically speed up BC6 compression, which is essential for compressed baked lightmaps.
Performance (compression ONLY):
Build - Production 64 bit,
CPU - Ryzen 9 5900x,
GPU - RTX 4060 TI

image
CVTT
Betsy

Symmetrical garden 8k .hdr with mipmaps
92.4s
595ms

Cobblestone Street Night .hdr 4k with mipmaps
26.5s
362ms

Laufenurg Church 8k .hdr with mipmaps
99.3s
522ms

Little Paris 8k .hdr with mipmaps
92.7s
467ms

TODO:

[x] optimize the setup (shader compilation, RGB->RGBA conversion), since that takes up most of the time,
[ ] fix signed compression (when importing a layered texture, some slices are signed and others aren't, so the combined texture breaks).

This is going to help immensely with our lightmapper! Currently the lightmapper doesn't compress the results since it is so slow. But with this, we can finally re-enable compression by default

In terms of performance, a layered 4096x2048x8 image takes ~11 seconds to import on an RTX 4060TI.

Update: After optimizing RGB-RGBA conversion and disabling the forced auto mipmap generation for layered textures (is there any reason why it was enabled in the first place?), the compression time goes down to ~7 seconds, and that's still mostly CPU bound. I'm hoping to bring it down to around 2, 3 seconds.

Signed compression is now supported, though because the sign detection happens on individual layers, most layered images break when combined.

Is there a way to handle:

Is there a way of bc6 using sign detection on individual layers or the entire set at once without too much research time?
This seems like we need to implement a batch system?

This is tricky since each layer is compressed as an independent image, and as such one layer isn't aware of the other ones. IMO the layered/3d/cube texture import pipeline needs a light rework as the current approach also causes other problems due to this issue.

Did some benchmarks, the difference in performance is incredible (up to 155 times faster on some large images).

It's strange how my import speeds are quite a bit slower than @BlueCube3310's though, considering my GPU is significantly faster.

My results only contain the time it takes to execute the compress function, and not the whole import process (decoding, mipmap generation, etc.). On my machine the whole process took ~14s.

I was thinking about fix signed compression (when importing a layered texture, some slices are signed and others aren't, so the combined texture breaks). can we use disable the signed path or somehow resolve or do we have to implement batched signing properly?

I was thinking about fix signed compression (when importing a layered texture, some slices are signed and others aren't, so the combined texture breaks). can we disable the signed path or somehow resolve or do we have to implement batched signing properly?

As far as I'm aware, directional lightmaps depend on signed float data for storing direction information, so forcing the texture to be compressed as unsigned will cause artifacts to appear.
I also found out today that Betsy doesn't really support the signed variant of bc6, so it will need to be properly implemented first.

Adding signed compression to betsy doesn't sound trivial.

Adding signed compression to betsy doesn't sound trivial.

I'll see if I can get it to work by July. If not, signed images will either be uncompressed or converted by CVTT as a fallback.

The GPU compressor can now be toggled from the project settings and the compressor works in rendering modes other than Forward+.

So I tried compressing a directional lightmap with Betsy and noticed some weird artifacts:

I thought this was a compressor-specific issue, so I tried CVTT (the one used by Godot):

I have no idea why this happens. It could be a format limitation, or simply a shared issue between the compressors. Either way, directional lightmaps should probably remain uncompressed.

 Directional Lightmaps are so heavy. Maybe for moment you can use compression on non-directional lightmaps until de issue is solved

Can you compress some thing else that has signed floats for comparison and against some official signed float bc6 encoder?

Can you compress some thing else that has signed floats for comparison and against some stable signed float bc6 encoder?

Left: Microsoft's DirectXTex,
Middle: CVTT,
Right: Betsy

Is there a quality knob we can maximize for Betsy before we try too hard on debugging?

CC @darksylinc, FYI. Finally putting Betsy to good use in Godot!

It looks like ASTC doesn't handle negative values either.

Could we bias directional lightmaps when baking (and reverse this biasing in the shader) so that we don't need the texture to use signed compression? This would make them ASTC-friendly as well.

Could we bias directional lightmaps when baking (and reverse this biasing in the shader) so that we don't need the texture to use signed compression? This would make them ASTC-friendly as well.
Doing so will impact precision somewhat, so I assume we'll need to find a reasonable bias value that is flexible enough for very bright lights that you might use in a real world project (e.g. the equivalent of Color(10, 10, 10)).

Yes. But several biasing strategies would have to be tested to see which one provides overall the best precision.
e.g.
val = val * 0.5f + 0.5f; // Mapping [-1; 1] -> [0; 1]
val = 1.0f - val * 0.5f + 0.5f; // Mapping [-1; 1] -> [1; 0]
val = exp( val ); // Or exp2
val = exp( 1 - val );

Been playing around with a python script and chatgpt4.
import numpy as np

def ulp(x):
    return np.nextafter(x, np.inf) - x

def map_minus_one_to_one(val):
    return val * 0.5 + 0.5 # Mapping [-1; 1] -> [0; 1]

def map_one_to_zero(val):
    return 1.0 - val * 0.5 + 0.5; # Mapping [-1; 1] -> [1; 0]

def map_exp(val):
    return np.exp(val)

def map_exp2(val):
    return np.exp2(val)

def map_exp_inverse(val):
    return np.exp(1 - val)

values = np.logspace(-3, 3, 1000)
mappings = {'map_minus_one_to_one': map_minus_one_to_one,
            'map_one_to_zero': map_one_to_zero,
            'map_exp': map_exp,
            'map_exp2': map_exp,
            'map_exp_inverse': map_exp_inverse}

for name, mapping in mappings.items():
    mapped_values = mapping(values)
    ulps = ulp(mapped_values)
    print(f"Mapping {name}:")
    print(f"Min ULP: {np.min(ulps)}")
    print(f"Max ULP: {np.max(ulps)}")
    print(f"Mean ULP: {np.mean(ulps)}")
    print()

Mapping map_minus_one_to_one:
Min ULP: 1.1102230246251565e-16
Max ULP: 5.684341886080802e-14
Mean ULP: 5.71298564011613e-15
Mapping map_one_to_zero:
Min ULP: 2.168404344971009e-19
Max ULP: 5.684341886080802e-14
Mean ULP: 5.614703363701623e-15
Mapping map_exp:
Min ULP: nan
Max ULP: nan
Mean ULP: nan
Mapping map_exp2:
Min ULP: nan
Max ULP: nan
Mean ULP: nan
Mapping map_exp_inverse:
Min ULP: 5e-324
Max ULP: 4.440892098500626e-16
Mean ULP: 2.0963494167660428e-16

That's not exactly what I meant (although it's not bad, and it's close in spirit).
What I meant is that the average use case is going to use certain ranges more than others; thus the testing would need to generate several lightmaps and compare using real world data.

My understanding is BC6H and ASTC hdr allow negative values.
I don't think we should change the format of dir lightmaps. We should debug the betsy shader code.
We should double check it isn't a shader optimizer issue. For example, in HLSL (some versions of)? fxc / dx11 will assume that reads from Texture2D<float4> return positive values. Something like float4 x = tex2D(...); if (x.x <0) { ... } will be optimized out or float foo = max(0.0, tex2D(...).x) will be optimized to float foo = tex2D(...)
Maybe GLSL optimizers have a similar issue

Once this is added, we should look into compressing ReflectionProbes and Skys when the update mode is set to "update once"

Once this is added, we should look into compressing ReflectionProbes and Skys when the update mode is set to "update once"

VRAM compression is not available in export template builds, so I don't see a way of doing this unless we add ReflectionProbe prerendering.
Besides, the current Once update mode is not a "true" Once update mode as described in https://github.com/godotengine/godot-proposals/issues/2934. This means that if a ReflectionProbe with the Once update mode is moving continuously, VRAM compression would need to be performed every frame, which is too slow.

My understanding is BC6H and ASTC hdr allow negative values. I don't think we should change the format of dir lightmaps. We should debug the betsy shader code.

That is true (for BC6 at least, not sure about ASTC), although we would also have to modify astcenc, since it clamps the values to 0. The required modifications for both astcenc and Betsy will also take significantly more time.

Hi.
BC6H supports negative values, but it is betsy's encoder which does not.
ASTC has two variants: LDR & HDR. The LDR variant does not support negative values. I don't know about the HDR variant. However most GPUs do not support the HDR variant.

Given that the directional lightmap issue also affects the other compressors, and that fixing it will take more time, would it be OK to split that into its own PR? This one could then be merged earlier, and we could get more feedback from the users.

Can you leave the lightmap functionality using the old workflow to split difficulty?

Can you leave the lightmap functionality using the old workflow to split difficulty?

I haven't changed their functionality in this PR, so the lightmaps still work the same way they did before.

Given that the directional lightmap issue also affects the other compressors, and that fixing it will take more time, would it be OK to split that into its own PR? This one could then be merged earlier, and we could get more feedback from the users.

What will this PR affect currently?

What will this PR affect currently?

HDR images' import times (i.e. for panoramic skies, non-directional lightmaps). This will be most significant on low-end hardware, since the compression was a major bottleneck before.

Given that the directional lightmap issue also affects the other compressors, and that fixing it will take more time, would it be OK to split that into its own PR? This one could then be merged earlier, and we could get more feedback from the users.

Ya, its okay to merge Betsy for image importing first and then add support for lightmaps later

Up to 16K images are now supported after #94243.

High-quality encoding seemingly used to be broken, it now functions the same way as in https://github.com/knarkowicz/GPURealTimeBC6H.

High-quality encoding seemingly used to be broken, it now functions the same way as in https://github.com/knarkowicz/GPURealTimeBC6H. This improves quality and makes the compression time ~1.5x faster.

Woah, the "fast" preset looks like it was used for real time compression of ReflectionProbes. We might be able to do something similar to compress ReflectionProbes that are set to "update once". That would save a lot of VRAM and could help performance too

I think I got signed compression to work correctly:

(The image is multiplied by -1 in the shader)
The quality mode has similar artifacts to CVTT, so for now I think the signed mode should be low-quality.

After #95291 and #92291, the import times as a whole have gone down significantly. It takes ~4.5s and ~1.2s respectively to import an 8k and 4k HDRi on a debug build.

Is this ready for merge? What is left?

I'd say it's ready, the only thing I'm worried about is that it will cause crashes on devices that don't fully support compute shaders (like mobile), similarily to the lightmapper. I don't think there's currently a way to verify that, but if there is it should be made to fall back to CVTT.

,similarily to the lightmapper.

For lightmap it kind of ok since android does not support baking lightmap atm.

Apparently some mobile devices support compute shaders, but I'm still worried that we might run into an edge case with some older models that will cause the output texture to be corrupted/empty. Worst case scenario, said device only supports ASTC natively and the user will export the project with broken textures to desktop platforms.
I'm conflicted whether to disable betsy on mobile altogether, or leave it in for now and see the results of wider testing.

I'm conflicted whether to disable betsy on mobile altogether, or leave it in for now and see the results of wider testing.

Prob make it a warning on the mobile device  unable to handle compute shaders which means betsy not being supported. Eitherway most mobile devices can’t handle mobile backend due to vulkan  drivers being broken there, as for compatibility renderer it might  have to be automatically  dissabled and can be enabled back for mobile devices that can handle it .

After some consideration, I've disabled it for mobile since there is currently no easy way (that I know of) to check for whether a specific device has working compute support

After some consideration, I've disabled it for mobile since there is currently no easy way (that I know of) to check for whether a specific device has working compute support. Also, the engine can now be safely built without betsy (previously it would fail the compression if the module was missing, now it will fall back to the default compressor).
That should hopefully be the final bug fix, I think this is ready to merge now.

Any device that supports Vulkan supports compute shaders. So if you are able to create a RenderingDevice, you can assume that compute shaders are supported.

Done as suggested, it should now work on mobile correctly.

Directional lightmaps update (not included in this PR):
After experimenting with several biasing strategies, I settled on one inspired by Frostbite's Flux, which involves storing the result of dividing L1 SH coefficients by L0, then biasing them to the 0.0-1.0 range. So far it seems to not be affected by artifacting:

@BlueCube3310 When ever you feel you have something solid, please open another PR so I can test the directional lightmap compression

Thanks!

Does this PR mean that HDR sky textures will import faster without having to touch the import settings by default?

Does this PR mean that HDR sky textures will import faster without having to touch the import settings by default?

Yes, https://github.com/godotengine/godot/pull/95291 and https://github.com/godotengine/godot/pull/92291 also helps for HDRIs using Lossless compression (or VRAM Uncompressed).


How to authenticate token from AAD in an aspnet core web api passed through signalR?

First of all, I'm a bit new to web dev as a whole but I would like to ask this question since we''re using AAD authentication. I created an aspnet core React application and used the authentication scheme generated by aspnet core webapi on the backend and adal js on the frontend. So I added the code below in Startup.cs and [Authorize] in my controllers as well as my hubs.
 services.AddAuthentication(AzureADDefaults.BearerAuthenticationScheme)
            .AddAzureADBearer(options => Configuration.Bind("AzureAd", options));

From the client side, I attached the token to the headers of my ajax requests and they all work correctly. Below is the function I use to perform the fetch. The adalApiFetch function already adds the authorization inside it
     .......
     let response = yield adalApiFetch(axios, "api/securities", {
        method: "GET",
        headers: {
            "Content-Type": "application/json; charset=UTF-8",
            Accept: "application/json"
        }
    }).catch(function(error) {
      throw error;
    });
    .........

For the hubs, I'm using the code below
   const conn = new SignalR.HubConnectionBuilder().withUrl("/myhub", {
      accessTokenFactory: async () => {
          return await getApiToken();
      }
  }).build();

Like I said above, axios fetch and authentication works, but signalR does not.
Chrome has the following logs
    WebSocketTransport.js:35 WebSocket connection to 'wss://localhost:5001/myhub id=................................&access_token=ey..........................
    failed: HTTP Authentication failed; no valid credentials available
    Error: Failed to start the transport 'WebSockets': undefined

And this is from Edge
    Error: Failed to start the transport 'WebSockets': undefined
    SCRIPT12008: SCRIPT12008: WebSocket Error: Incorrect HTTP response. Status code 401, Unauthorized

Can anyone please guide me on what I am missing?

Hello @exztinct01 ... sorry to hear you ran into a problem. There are only three of us here on the docs repo, so we're not staffed up to handle support requests. We recommend Stack Overflow first. If that fails or you think you've discovered a bug in the framework, we recommend opening an issue in the aspnet/Home issues. I'll ping one of the engineers just in case this is quick fix or there's a resource that you could be pointed to. However, Stack Overflow is your best bet for a quick response. If you come to find a documentation deficiency, please report that back to us here.
cc/ @anurse

This issue looks like it belongs over in SignalR. Could you collect server side logs https://github.com/aspnet/SignalR/wiki/Diagnostics-Guide#server-side-logging and file the issue over there?

I'll close this now. I transferred the issue to the correct repo


fix: infer all Numeric vars as TimeSeries when tsmode=True

Drop the time dependence check for time series type inference

Codecov Report
Patch coverage: 100.00% and project coverage change: -0.02 :warning:

Comparison is base (707d1c8) 89.91% compared to head (3869dd1) 89.89%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1343      +/-   ##
===========================================
- Coverage    89.91%   89.89%   -0.02%
===========================================
  Files          191      191
  Lines         6110     6099      -11
===========================================
- Hits          5494     5483      -11
  Misses         616      616

Flag
Coverage Δ

py3.8-ubuntu-22.04-pandas
89.89% <100.00%> (-0.02%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

tests/unit/test_time_series.py
100.00% <ø> (ø)

src/ydata_profiling/model/typeset.py
95.23% <100.00%> (-0.30%)
:arrow_down:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

@alexbarros this does not solve the request from the PR. Can we please revert this branch?
https://github.com/ydataai/ydata-profiling/issues/1292


uclibc: 1.0.49 -> 1.0.50

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for uclibc is: Embedded C library
meta.homepage for uclibc is: https://uclibc-ng.org
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.0.50 with grep in /nix/store/5qv2skz5pl6l14d8dvdrj19v73jpzs14-uclibc-ng-1.0.50
found 1.0.50 in filename of file in /nix/store/5qv2skz5pl6l14d8dvdrj19v73jpzs14-uclibc-ng-1.0.50

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)

3 package rebuild(s)

First fifty rebuilds by attrpath

uclibc
uclibc-ng

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/5qv2skz5pl6l14d8dvdrj19v73jpzs14-uclibc-ng-1.0.50 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A uclibc https://github.com/r-ryantm/nixpkgs/archive/9d01eec58488cc2ccf71fb6f8c03ea8773783749.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/9d01eec58488cc2ccf71fb6f8c03ea8773783749#uclibc

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/5qv2skz5pl6l14d8dvdrj19v73jpzs14-uclibc-ng-1.0.50
ls -la /nix/store/5qv2skz5pl6l14d8dvdrj19v73jpzs14-uclibc-ng-1.0.50/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

uclibc

Maintainer pings
cc @rasendubi @AndersonTorres for testing.

Add a :+1: reaction to pull requests you find important.

@NixOS/nixpkgs-merge-bot merge


User returned from register endpoint actually is the same as the user passed

Describe the bug
User create endpoint returns user that have been sent, not the user that was created at database.
To Reproduce
Just add custom column to User database model and set it's server_default or default value.
When creating new user don't set that columns' value. Than in response this value will not have default value but sent value.
Simple reproducable example.
Database User model:
class User(Base, SQLAlchemyBaseUserTable):
    """Table for storing users"""

    created_at = Column(DateTime, default=datetime.now, nullable=False)

Pydantic User schemas:
class User(models.BaseUser):
    """User fetch pydantic model"""

    created_at: datetime | None  # None is to create users without Pydantic's validation exception

class UserCreate(models.BaseUserCreate):
    """User's creation pydantic scheme"""

class UserUpdate(models.BaseUserUpdate):
    """User's update pydantic scheme"""

class UserDB(User, models.BaseUserDB):
    """User stored in DB pydantic model"""

Than add registration router and test.
Request and response examples
Request:
{
  "email"<EMAIL_ADDRESS>  "password": "some-password",
  "is_active": true,
  "is_superuser": true,
  "is_verified": true
}

Response:
{
  "id": "ecdd896e-2c76-42ba-b374-d5281e588e87",
  "email"<EMAIL_ADDRESS>  "is_active": true,
  "is_superuser": false,
  "is_verified": false,
  "created_at": null
}

Expected behavior
Successful user creation with automatically set created_at returned from registration endpoint.
For above request expected response is:
{
  "id": "ecdd896e-2c76-42ba-b374-d5281e588e87",
  "email"<EMAIL_ADDRESS>  "is_active": true,
  "is_superuser": false,
  "is_verified": false,
  "created_at": "2022-01-16T00:16:22.626191"
}

Configuration

Python version : 3.10.1
FastAPI version : 0.71.0
FastAPI Users version : 9.2.2

FastAPI Users configuration
Standard configuration from docs

The easiest way to fix is to replace this line:
https://github.com/fastapi-users/fastapi-users-db-sqlalchemy/blob/afecb6b3380d52cdc4b21743f45a64c065acd559/fastapi_users_db_sqlalchemy/__init__.py#L117
with something like this:
    return await self.get(user.id)

to fetch user with already set properties. The same is made by ormar adapter.

Yes, I agree that it will be better like this.

Fixed in e24d824ed77a14cc7e856cccf811aa7789833ad7


[Enhancement] Added Gson from JsonUtil in mock cards

Se utiliza el Gson de JsonUtil para reconcoer el camelCase de las variables.

[ ] Coverage > 90%

[x] Android Lint check

[x] Code Style checked

Codecov Report

Merging #539 into v3 will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##               v3     #539   +/-   ##
=======================================
  Coverage   24.71%   24.71%
=======================================
  Files         197      197
  Lines       15398    15398
  Branches     1376     1376
=======================================
  Hits         3806     3806
  Misses      11157    11157
  Partials      435      435

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 424fe05...05271aa. Read the comment docs.


Creating Suggestion API

Creating semantic suggestion API to be used to suggest topics with given input.
Input might be a simple keyword(search functionality) or be a topic header(suggestion functionality)

Now we can extract entities from topic body, and suggest user these entities to tag topic.
Next : Creating suggestion mechanism at backend.

finished


Swift 5

Hi, thanks, this framework helpes me a lot, couldnt make AX work because i‘m just learning swift, no objC.
Need help with Swift 5 conversion, 2 issues:

deallocate needs no capacity attribute anymore, easily solved.
run.loop needs changement, dont know what to do.

I forked the project and converted it to Swift 5 myself.
I also managed to make Swift Packages work, too.
Huge learning experience!


If function for color property results in a Parse Invalid Error

Hello,
I am currently facing an issue where I cannot dynamically change the color of a text block from the values I get from my JSON payload. I want to use the built-in "if" function but that doesn't seem to work. Neither goes simply having the semantic values expected for the color property to be something I pull directly from the payload work.
Furthermore, I tried using one of the sample templates as an example, the Stock update sample to be specific. There it too has the color property being set using an if function. But when I try editing the sample in the adaptive cards designer, I get the same error.
I am using the 1.2 target version, in case you need that bit of info.
Looking to hear back from you regarding this issue. Thank you.
Dhruv Patel

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 117f7457-f439-e41e-8782-ff627e6b030b
Version Independent ID: 22f86ac6-47f8-447f-82e9-8093b3ddf0fb
Content: Adaptive Cards Template Language - Adaptive Cards
Content Source: AdaptiveCards/templating/language.md
Product: adaptive-cards
GitHub Login: @matthidinger
Microsoft Alias: mahiding

Hi there, can you provide the template payload and data you are using? And it doesn't work in the designer all? If you open the browser dev tools there should be an error in the console window that would help pinpoint what specifically is failing.

StockUpdateSample.txt
StockUpdateSamplePayload.txt
Hey, I've attached the sample codes I was trying to build off of. When I open these in the online Adaptive Cards Designer I get an error regarding the if statement that is used to change the color of the text fields. The specific error that the designer gives me is the following:
[Parse] Invalid value "${if(change >= 0, 'good', 'attention')}" for property "color"
Not sure if it is just an issue with the designer itself, or something to do with the code. Again, would really appreciate the feedback.

Thanks for following up, that is indeed a bug in the Designer. I've filed this bug to track the fix: https://github.com/microsoft/AdaptiveCards/issues/4658
The binding should work properly in actual usage via the SDKs, so please re-open and let us know if that isn't case.


ThemeToggle enabled

#37 I added a toggle button to switch from light theme to dark theme , made sure the transition is smooth and don't feel laggy
please merge this pr as soon as possible

Hey @hardikjumnani
Nice work on adding the toggle button! Could you clarify where exactly you have added the toggle button for the light and dark theme? Let me know, and I'll review it right away!


Implement batch deletion of uploaded songs?

Can you please implement batch deletion for songs?

Hi, I've got something in mind for batch deletions... I was thinking of a tick box for the functionality to 'Delete from YouTube Music When I Delete From My Music Library' option... Is that the kind of think you were thinking, or are you thinking more of a 'pick and choose' what to delete feature?

Basically I mistakenly synced my old Google Play Music library with bad ID3
tags, and some music I had for others. After some time I manually removed
all of the albums only to discover the individual songs are kept.
Checkboxes would be great, with a checkbox for select all at the top? There
are calls on reddit to be able to edit like this for (uploaded and not)
songs, albums, playlists etc, but right now I'm specifically concerned with
my uploaded songs.
On Mon 31 Aug 2020, 18:59 James Brindle<EMAIL_ADDRESS>wrote:

Hi, I've got something in mind for batch deletions... I was thinking of a
tick box for the functionality to 'Delete from YouTube Music When I Delete
From My Music Library' option... Is that the kind of think you were
thinking, or are you thinking more of a 'pick and choose' what to delete
feature?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/jamesbrindle/YTMusicUploader/issues/1#issuecomment-683935783,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC7PTBQLF4UNGCDPHUYCEZ3SDPQIRANCNFSM4QQXZQSQ
.

I see what you mean... Yeah sounds like a plan... The 'delete from YT music when you delete from library' probably won't be sufficient then. It sort of relies on good ID3 tags... What I'd have to do is query all artists from uploaded music, then all songs for each artist. Might take a good few minutes to retrieve all the songs, but I it's definitely doable. I'll add it to my list.

Excellent, thank you.
Could you please post here whenever you get around to it?
On Mon 31 Aug 2020, 19:12 James Brindle<EMAIL_ADDRESS>wrote:

I see what you mean... Yeah sounds like a plan... The 'delete from YT
music when you delete from library' probably won't be sufficient then. It
sort of relies on good ID3 tags... What I'd have to do is query all artists
from uploaded music, then all songs for each artist. Might take a good few
minutes to retrieve all the songs, but I it's definitely doable. I'll add
it to my list.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/jamesbrindle/YTMusicUploader/issues/1#issuecomment-683942512,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC7PTBR3HD7EKO6BDVLVZJ3SDPRX5ANCNFSM4QQXZQSQ
.

Added ability to delete tracks and albums from YT Music via a new form that displays all song uploaded in a checkbox tree view, as well as reset the local DB.

Additional 'Manage YT Music' uploaded songs UI / Cache / Performance improvements and a couple of fixes in version 1.4.1


FR: Static Constructors for UDPSocket

Feature Request
It would be nice to have a shortcut for creating a UDPSocket in common scenarios.
From the documentation:
# Create server
server = UDPSocket.new
server.bind "localhost", 1234

# Create client and connect to server
client = UDPSocket.new
client.connect "localhost", 1234

It'd be nice to have a little cheat to perform both lines in one:
# Create server
server = UDPSocket.bind     "localhost", 1234

# Create client and connect to server
client = UDPSocket.connect  "localhost", 1234

The UDPSocket ctor has only one keyword argument, so it could be added trivially onto the end of the arguments for bind or connect.

Sounds like a good idea for convenience.


http, utility: Fix unicode URL parsing

Commit Message: Using the current version of googleurl and ICU, in combination with how we build it, we fail on initializing GURL with a unicode URL. This patch temporarily uses a mirror of  https://quiche.googlesource.com/googleurl for googleurl and update ICU to 67.1 to enable unicode URL parsing.
This also introduces tools/generate_data.sh to facilitate generating a different set of ICU data, via editing tools/icu/filters.json.
Signed-off-by: Dhi Aurrahman<EMAIL_ADDRESS>Additional Description: This requires a "manual" step to generate ICU data since I'm not sure on how to do proper setup (genrule) for Windows.
Risk Level: Low
Testing: Added. Linux only for now
Docs Changes: N/A
Release Notes: N/A
Fixes: https://github.com/envoyproxy/envoy/issues/12015

This is sort of confusing to me. Why would envoy be handling UI-representations, and not transport URI in these particular cases (punycode, for example)? What's the use case?

@wrowe I'm moving the test from https://quiche.googlesource.com/googleurl/+/refs/heads/master/test/basic_test.cc#31 do you think it is not necessary?

I'm asking, what's the use case? URI at the transport layer aren't presentation elements, they are opaque octets, except as bound by specific RFC's on an element-by-element basis. For example, scheme is very well understood and constrained by familiar ASCII tokens. Hostname must confirm to the DNS spec. We aren't a user agent, so don't care to transform a URI bar, such as in chromium, via punycode, into a valid DNS hostname... Or do we? And when?

@lizan let me try to make it work for windows to generate ICU data. The binary was added since I wasn't sure how to do that in windows.

Seems like we don't really need to embed ICU data for now. I'm closing this and opening https://github.com/envoyproxy/envoy/pull/12376 as a PR for updating ICU (to 67.1) and googleurl versions.


Use preprocessed and cached tfrecord datasets with PyTorch

IceNet version: 0.2.8
Python version: 3.11.7
Operating System: CentOS Linux 7 (BAS HPC)

Description
To accelerate training with PyTorch, it would be useful to make use of the preprocessed and cached tfrecord datasets we've produced for operational Tensorflow training! Loading from these tfrecord files is (should be?) much faster than loading from NetCDF files on the fly using the generate_sample logic in, e.g., the DaskMultiWorkerLoader.generate_sample defined in data/loaders/dask.py which must perform a decent amount of computation.
What I Did
I've implemented an IterableIceNetDatasetPyTorch class which inherits from torch.utils.data.IterableDataset as a first go, and will link it below in a pull request. The following script demonstrates its use and can easily be stepped through in a debugger. The batching logic works properly with torch.utils.data.DataLoader in the script below, but I've seen some weird behaviour (overrunning the number of samples that should be generated per epoch) during training runs when num_workers > 1 in the torch.utils.data.DataLoader so there's room to improve this first implementation.
import os
import torch
from utils import IterableIceNetDataSetPyTorch

dataset_config = "dataset_config.exp23_south.json"

ds = IterableIceNetDataSetPyTorch(dataset_config, "test", batch_size=4, shuffling=False)
dl = torch.utils.data.DataLoader(ds, batch_size=4, shuffle=False)

for i, batch in enumerate(dl):
    x, y, sw = batch
    print(x, y, sw)
    print(i)

CC @bnubald who has done some work in this area already


integrate cosign signature verification feature into enclave-cc

cosign image signature verification feature in image-rs is released in CoCo V0.2.0. This issue request is to enable this feature in enclave-cc and make sure this signature verification method can be configured and used in e2e and operator CI.
dependencies and status:

[x]  dependency 1: rats-tls build into agent (https://github.com/confidential-containers/enclave-cc/pull/48)
[x]  dependency 2: enable image-rs cosign feature for agent (https://github.com/confidential-containers/enclave-cc/pull/87)
[x]  dependency 3: use updated verdictd supporting cosign resources
[x]   dependency 4: agent should fix the bug of agent reading config (https://github.com/confidential-containers/enclave-cc/issues/51#issuecomment-1337310557)
[x]  dependency 5: encrypted and cosigned image for testing purpose: docker.io/eqmcc/helloworld_enc:latest, in CI if kata-cc has same kind of image and resource, we can use that for consistency

ongoing quick start guide review: https://github.com/confidential-containers/documentation/pull/65

ongoing quick start guide review: confidential-containers/documentation#65

This pr may not be merged for some reason. If this doc was placed somewhere else, I'd like to leave the new link.

new doc PR: https://github.com/confidential-containers/image-rs/pull/87

dependencies:

dependency 1: rats-tls build into agent (https://github.com/confidential-containers/enclave-cc/pull/48)

dependency 2: use updated verdictd supporting cosign resources

dependency 3: agent should fix the bug of agent reading config (https://github.com/confidential-containers/enclave-cc/issues/51#issuecomment-1337310557)

[ ]

[ ]

integration POC done in development env: now can load encrypted and cosign signed image docker.io/eqmcc/helloworld_enc:latest after retrieving signature validation and layer decryption resources from newest version of verdictd

@hairongchen

dependencies:

I'd suggest if we want to track these "todo" items in issues, we edit the first comment in the issue. It becomes much easier to find the status

@hairongchen do you feel this is on track to converge to the V0.3.0 release (22nd of January)?

@hairongchen do you feel this is on track to converge to the V0.3.0 release (22nd of January)?

Yes, its on track, thanks!

closed - all dependencies meet and e2e test works as expected


Introducing Azure Security Center module

Some of the commands will assume prepared environment since it is not possible to create some of our resources on demand.
Thanks!

This checklist is used to make sure that common guidelines for a pull request are followed.

[x] The PR has modified HISTORY.rst describing any customer-facing, functional changes. Note that this does not include changes only to help content. (see Modifying change log).

[x] I adhere to the Command Guidelines.

Please reach out to the CLI team to schedule a command review.

@williexu your comments are resolved,
is there any additional change to be done?
I get an error in Travis that I don't get locally about a syntax error, but I can't seem to understand what's wrong.
can you please help me with that?

@williexu fixed