After bingo, free square turns into a button

Get a bingo that includes the center free square. The free space turns into a button instead of staying a free space

I wasn't able to reproduce this—can you take a look at my branch and verify this is working for you now?

Hey Katie,
I'm not at a machine that I can test your branch on right now, but I see
that this repros in master (https://debate-bingo-dev.herokuapp.com) when
doing a top-left to bottom-right diagonal bingo.
--Jonathan
On Fri, Sep 2, 2016 at 11:43 AM, Katie Schick<EMAIL_ADDRESS>wrote:

I wasn't able to reproduce this—can you take a look at my branch and
verify this is working for you now?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/DevProgress/debate-bingo/issues/7#issuecomment-244456548,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AUCcj7dpFHD_JH88kLLh4cY3Gc040yQGks5qmG48gaJpZM4Jx8v0
.

Ah, I see. Yep, this should be fixed now on my branch.

Still repros after commit 23f440f (your last pr). Can you take another look?


do not use Hyrax::BasicMetadata

summary
the purpose of allinson_flex is so that a user can create their own metadata properties. however, Hyrax::BasicMetadata is added to each model generated through the works_generator.rb file. therefore, all properties in BasicMetadata are added to a model, whether or not they are in the metadata profile
suggestion

remove Hyrax::BasicMetadata from models created with allinson_flex

resources

https://github.com/samvera-labs/allinson_flex/blob/3eec46eea7bd8c054990a6809f6d33706e7345d1/lib/generators/allinson_flex/works_generator.rb#L96

Resolved with #104


WIP: Add support for customization of network topologies

This change allows us to define private and public subnets in the top-level of cluster.yaml to be chosen for worker/controller/etcd nodes and a controller loadbalancer.
Thanks to @neoandroid and @sasso for submitting the pull request #169 and #227 respectively which had been the basis of this feature.
Let me also add that several resources including subnets, NAT gateways, route tables can now be reused by specifying id or idFromStackOutput.
Thanks to @icereval for his PR #195 to firstly introducing the idea of type Identifier to add support for existing AWS resources in an universal way.
A minimum config utilizing this feature would look like:
clusterName: mycluster
externalDNSName: mycluster.example.com
hostedZoneId: yourhostedzoneid
keyName: yourkeyname
kmsKeyArn: "arn:aws:kms:<region>:<account>:key/<id>"
region: <region>
createRecordSet: true
experimental:
  waitSignal:
    enabled: true
subnets:
- name: private1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- name: private2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- name: public1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
- name: public2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
controller:
  subnets:
  - name: private1
  - name: private2
  loadBalancer:
    private: false
etcd:
  subnets:
  - name: private1
  - name: private2
worker:
  subnets:
  - name: public1
  - name: public2

This will create 2 private subnets and 2 public subnets. Private ones are used by etcd and controller nodes and the public ones are used by worker nodes and the controller loadbalancer.
It is flexible enough to differentiate private subnets between etcd and controllers:
subnets:
- name: etcdPrivate1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- name: etcdPrivate2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- name: controllerPrivate1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- name: controllerPrivate2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- name: public1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
- name: public2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
controller:
  subnets:
  - name: controllerPrivate1
  - name: controllerPrivate2
  loadBalancer:
    private: false
etcd:
  subnets:
  - name: etcdPrivate1
  - name: etcdPrivate2
worker:
  subnets:
  - name: public1
  - name: public2

It even support using existing subnets by specifying subnet IDs:
subnets:
- name: private1
  id: subnet-12345a
  availabilityZone: ap-northeast-1a
  private: true
- name: private2
  id: subnet-12345b
  availabilityZone: ap-northeast-1c
  private: true
- name: public1
  id: subnet-12345c
  availabilityZone: ap-northeast-1a
- name: public2
  id: subnet-12345d
  availabilityZone: ap-northeast-1c
controller:
  subnets:
  - name: private1
  - name: private2
  loadBalancer:
    private: false
etcd:
  subnets:
  - name: private1
  - name: private2
worker:
  subnets:
  - name: public1
  - name: public2

Or importing subnet IDs from another cfn stack(s):
subnets:
- name: private1
  idFromStackoutput: myinfrastack-privateSubnet1
  availabilityZone: ap-northeast-1a
  private: true
- name: private2
  idFromStackoutput: myinfrastack-privateSubnet2
  availabilityZone: ap-northeast-1c
  private: true
- name: public1
  idFromStackoutput: myinfrastack-publicSubnet2
  availabilityZone: ap-northeast-1a
- name: public2
  idFromStackoutput: myinfrastack-publicSubnet3
  availabilityZone: ap-northeast-1c
controller:
  subnets:
  - name: private1
  - name: private2
  loadBalancer:
    private: false
etcd:
  subnets:
  - name: private1
  - name: private2
worker:
  subnets:
  - name: public1
  - name: public2

Let me also add that several resources including subnets, NAT gateways, route tables can now be reused by specifying id or idFromStackOutput.
Thanks to @icereval for his PR #195 to firstly introduced the idea of type Identifier to add support for existing AWS resources in an universal way.

Current coverage is 56.18% (diff: 40.54%)

Merging #284 into master will decrease coverage by 1.34%

@@             master       #284   diff @@
==========================================
  Files             6          6
  Lines          1288       1342    +54
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits            741        754    +13
- Misses          449        482    +33
- Partials         98        106     +8

Powered by Codecov. Last update be60911...c0221c3

May I propose slightly different structure, which matches underneath CF structure closer.

as subnet names must be unique,then top level subnets: should be turned into a dictionary
any reference to subnets in a stack-template.json should be done by its name { "Ref": "Subnet0" }
Subnets which have no id fields, are created but kube-aws
subnets WITH id fields are passed as a typed default params in CF template. You'll get a free validation as a bonus:

"Parameters": {
  "Subnet1": {
     "Type": "AWS::EC2::SubnetId",
     "Default": "subnet-xxyyzz"
  }
}

Therefore cluster yaml would look something like this:
subnets:
  Subnet0:
    availabilityZone: us-west-2a
    CIDR: <IP_ADDRESS>/24
  Subnet1:
    id: subnet-xxyyzz

To keep backwards compatibility, old 'subnets' key  in cluster.yaml can be converted to new one internally based on it's type: if it is a list then convert it into a map with 'Subnet0', 'Subnet1', 'Subnet2'' keys

@redbaron thanks as always for your comments!

IMHO 1. should be an optional structure for defining subnets because name isn't a required param. I'm considering to extend the feature to have a less verbose way of doing the same setup above with something like:

subnets:
- availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
- availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
controller:
  private: true
  loadBalancer:
    private: false
etcd:
  private: true
worker:
  private: false

i.e. we can omit names if not interested. This extension doesn't seem to match well against the subnets as a hash syntax?

I believe 2. and 3. are already implemented as you've proposed and therefore 4. could be implemented on top of this PR easily

definitly seems to be the way to go for me too. However, not to make this PR too large, would you mind creating an another issue for it?

If believe 1. and 6. should be impemented both/at once if 1. is necessary

should be an optional structure for defining subnets because name isn't a required param.

That is what I am proposing to change. You always refer to a subnet by name therefore it is explicit what is created where.
By listing subnets explicitly you can have controller  and etcd in following setup to pick subnets which might even be different from each other but both private. Or some subnets exist and some - not.
controller:
  private: true
etcd:
  private: true

Thanks again!
Regarding the latter part of your commet, I guest I already know the convenience and efficiency of utilizing cfn parameters after reading through @icerevel's great work. I already introduced a few essence of his work via this PR. Now I'm open for a future PR to achieve what you suggested hence I've suggested to open an another issue dedicated to that :+1:
I'll try to sort out all the issues eventually but it is definitely impossible to manage those alone. Merging everyone's works and desires to make it work together while adding tests and refactoring to make it maintainable at least for me already took several days to happen!
Gradual changes made by not only the maintainer but multiple contributors is only the way to sustainably keep improving kube-aws IMHO.
Raising another issues for the problem which can be addressed independently is the first step toward it.
I know contributors like you are already helping kube-aws and me a lot but I have to request for more.
Fire and motion with me!

By listing subnets explicitly you can have controller and etcd in following setup to pick subnets which might even be different from each other but both private. Or some subnets exist and some - not.

Yes, it is why I made it to allow referencing subnets by names.
However I'd still want to support both cases. The one with subnets whose names are omitted is definitely for demo purpose. The one with explicitly named subnets for production purpose(hence implemented at first in this PR). Thoughts?

The one with explicitly named subnets for production purpose(hence implemented at first in this PR). Thoughts?

my concern about introducing names and not making them keys in a hashmap is that it is naturally can lead to duplicates and you need yet another piece of Go code to validate and report error if there are dupes. If you make all subnet names to be keys in hashmap, then you get uniqueness for free, datastructure itself enforces desired properties. Apart from that it all looks fine by me given the current state of the code.
IN the background I am preparing kube-aws overhaul which takes different approach in the way it translates cluster.yaml into CF template. I'll present it as a separate bracnh for discussion and collaboration once it achieves feature parity.

@redbaron

my concern about introducing names and not making them keys in a hashmap is that it is naturally can lead to duplicates and you need yet another piece of Go code to validate and report error if there are dupes.

Certainly.

If you make all subnet names to be keys in hashmap, then you get uniqueness for free, datastructure itself enforces desired properties

Definitely. Hmm, this is a bit hard decision for me to make at this stage alone. Please let me leave comments to request for confirmations in the related github issues.
Basically if no one needs:
controller:
  private: true
etcd:
  private: true

for demo purpose, we can safely switch to the way utilizing hashmaps as you've suggested.
Personally I'd also prefer it if possible.
Btw,

Apart from that it all looks fine by me given the current state of the code.

Thanks for the kind words!
I'm also looking forward to see your overhaul work!
Will it includes the support for node pools defined inside the top-level cluster.yaml, which is I'm going to tackle next?

I've updated the description of this PR to cover the overview of the all changes and improvements made.
cc @neoandroid @icereval @c-knowles @sasso

This all looks cool and flexible @mumoshu, thanks for your work! For my use cases it is considerably more flexibility than I would need. The one capability here I'd definitely find useful though, and will use if this goes ahead, is the ability to deploy private clusters with only the API and Ingress/Service load balancers public.
controller:
  loadBalancer:
    private: false

The cluster.yaml file is small enough that I remake it for every minor release. I don't want to miss some change or new feature, so I init every time and re-do the customization of cluster.yaml. Therefore I don't mind what syntax in cluster.yaml gets changed between minor, x.y releases.

@whereisaaron you already can achieve that scenario since #169 was merged

Hi @whereisaaron, thanks for the request!
If I understand correctly, I believe your case can be already achieved with:
subnets:
- name: private1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- name: private2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
- name: public1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
- name: public2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
controller:
  subnets:
  - name: private1
  - name: private2
  loadBalancer:
    private: false
etcd:
  subnets:
  - name: private1
  - name: private2
worker:
  subnets:
  - name: private1
  - name: private2

As can be seen in the above example, explicitly choosing private subnets for all the subnets keys for worker and controller, etcd while setting loadBalancer.private to false results to your desired setup.

@whereisaaron Regarding your comment https://github.com/coreos/kube-aws/pull/169#issuecomment-275895730, would it be enough for your use-case to add an option to disable creation of a NAT gateway like:
- name: private1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
  natGateway:
    create: false
- name: private2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
  natGateway:
    create: false

?

@whereisaaron Ah, or you'll need to do something like:
- name: private1
  availabilityZone: ap-northeast-1a
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
  natGateway:
    create: false
  routeTable:
    id: <id of your route table mentioned in https://github.com/coreos/kube-aws/pull/169#issuecomment-275895730>
- name: private2
  availabilityZone: ap-northeast-1c
  instanceCIDR: "<IP_ADDRESS>/24"
  private: true
  natGateway:
    create: false
  routeTable:
    id: routeTable:
    id: <id of your route table mentioned in https://github.com/coreos/kube-aws/pull/169#issuecomment-275895730>

natGateway.create is not yet implemented but it can be added easily if necessary.
routeTable.id for a subnet is already implemented in this PR but would you prefer the top-level routeTableId to be used as the default value for all the subnets (like before)?

Replying to my own comment above.

would you prefer the top-level routeTableId to be used as the default value for all the subnets (like before)?

This should not be allowed because then we don't have clear way to differentiate public/private route tables;

public route table(s) should be provided to/managed by kube-aws to put external kubernetes api loadbalancer
private route table(s) should be provided/managed by kube-aws to put private worker/etcd/controller nodes

as implied by what @whereisaaron mentioned in https://github.com/coreos/kube-aws/pull/284#issuecomment-275814848

@mumoshu as per your comment, I think the above looks fine. My understanding of this change is that it adds more flexibility to subnet customisation. Currently using a slightly older 0.9.2 release it is possible to change the route table associated with subnets generated by kube-aws and therefore possible to make them private/public. I'm not sure if https://github.com/coreos/kube-aws/pull/169 removed that and it's re-added here or just here adds more flexibility, either way is fine. Still a bit concerned about struggling to ensure all these different options are all well tested but I don't know how we can deal with that other than cutting the options and perhaps providing some network CF examples out of the box.

Thanks for looking in this @mumoshu. It is no problem to specify the route table per subnet, so if routeTable.id is already in place for subnets then I can still ensure our region-level or VPC-level NAT instances are used, rather than these new per-deployment ones. That's great 🎉
So I think the only missing piece to retain the previous capabilities is to have some way to not have any NAT instances created, since they serve no purpose if you have specified routeTable.id and workerSecurityGroupIds for VPC-level or region-level NAT instances. If the proposed natGateway.create option would do that, then we should be sorted.
One question about I think the proposed natGateway.create; I think controller, etcd, and worker subnets all use the same NAT instances. So perhaps the natGateway.create is effectively only a global/top-level yes/no choice rather than per-subnet?
@c-knowles creating private subnets with a specified route table is still possible in v0.9.3-rc.5 and it looks like this new way of doing things still retains or re-adds that capability via routeTable.id. The change, as I understand it, is that enabling private subnets now also triggers the creation of NAT instances. That's a great default for people creating a VPC and a first cluster. But if you have a few clusters you can't afford to create VPCs and/or NAT instances for every one, since the AWS limit is only 5 (in both cases).
@c-knowles as soon as there is an alpha/beta/rc release I'll get on with testing private subnet deployments and report back.

@c-knowles @whereisaaron Thanks again!
One of intentions of this PR is to provide enough flexibility to e.g. create worker/etcd/controller nodes in private subnet(s) and create an api load balancer in public subnet(s) but I began to believe that I had unintentionally broken your use-cases not in functionality but in configuration syntax.
It isn't intended at least for now.
You had been using a configuration like the below to put all the nodes and lbs to private subnets i.e. nothing including nods and lbs in public subnets, right?
routeTableId: rtb-for-privatesubnet-withpreconfigurednat
mapPublicIPs: false
controllerLoadBalancerPrivate: true

If so, I'm considering to improve this PR so that the above is translated to something like:
subnets:
- private: true
  natGateway:
    preconfigured: true
  routeTable:
    id: rtb-for-privatesubnet-withpreconfigurednat
controller:
  loadBalancer:
    private: true

// However, to be honest, such translation could be soon deprecated and removed if it turns out to be too hard to maintain. Sorry!
Does it make sense to you two?

@whereisaaron I've implemented natGateway.preconfigured instead of natGateway.create with inverting the bool value. Please see my updated comment at https://github.com/coreos/kube-aws/pull/284#issuecomment-275955785 for more explanation!

So I think the only missing piece to retain the previous capabilities is to have some way to not have any NAT instances created, since they serve no purpose if you have specified routeTable.id and workerSecurityGroupIds for VPC-level or region-level NAT instances. If the proposed natGateway.create option would do that, then we should be sorted.

Yes, I think so too and it is addressed in my last commit 242783d which adds subnets[].natGateway.preconfigured: true which requires subnets[].routeTable.id/idFromStackOutput to be specified.
Did I follow you correctly?

@c-knowles @whereisaaron I've updated my comment https://github.com/coreos/kube-aws/pull/284#issuecomment-275998862 several times. Please re-read if you came from email notifications from github 😃

@whereisaaron

One question about I think the proposed natGateway.create; I think controller, etcd, and worker subnets all use the same NAT instances. So perhaps the natGateway.create is effectively only a global/top-level yes/no choice rather than per-subnet?

My assumption is that: one or more subnet(s) may rely on an NGW created by kube-aws and others may rely on an NGW preconfigured for an existing subnet(s). Providing the flexibility to customize ngw per subnet also support it.
However I believe that it is true in some aspects that a setting like natGateway.preconfigured should optionally be able to be set at the top-level of cluster.yaml so that one doesn't need to write a verbose configuration like https://github.com/coreos/kube-aws/pull/284#issuecomment-275955785 in some cases hence my prev comment https://github.com/coreos/kube-aws/pull/284#issuecomment-275998862

@whereisaaron @mumoshu thanks, it seems my use case is still accommodated. Specifically, the use case is detailed in https://github.com/coreos/kube-aws/issues/44. Summary:

Existing VPC to share NATs and also connect through to some other components inside the same VPC such as RDS deployments. Shared NATs is mainly for cost reasons as not much benefit to have separate managed NATs for different clusters we have.
Private subnets for workers due to best practice (non of the services need direct exposure). Ideally generated by kube-aws as we have a few k8s clusters and adding/removing them from the VPC separately seems to be the wrong place.

So in turn we end up with the solution for kube-aws to create the subnets and link to existing route tables which are easily defined in a shared VPC (some private with NAT and some public with IGW).
YAML setup looks a bit like this:
subnets:
  - availabilityZone: eu-west-1a
    instanceCIDR: "<IP_ADDRESS>/24"
    # private route table to NAT AZ a
    routeTableId: rtb-ID1
  - availabilityZone: eu-west-1b
    instanceCIDR: "<IP_ADDRESS>/24"
    # private route table to NAT AZ b
    routeTableId: rtb-ID2
  - availabilityZone: eu-west-1c
    instanceCIDR: "<IP_ADDRESS>/24"
    # private route table to NAT AZ c
    routeTableId: rtb-ID3
controllerIP: <IP_ADDRESS>
controllerSubnets:
  - availabilityZone: eu-west-1a
    instanceCIDR: "<IP_ADDRESS>/24"
    # public route table
    routeTableId: rtb-ID4
  - availabilityZone: eu-west-1b
    instanceCIDR: "<IP_ADDRESS>/24"
    # public route table
    routeTableId: rtb-ID4
  - availabilityZone: eu-west-1c
    instanceCIDR: "<IP_ADDRESS>/24"
    # public route table
    routeTableId: rtb-ID4

We have a similar setup with node pools as well, the config is essentially the same. @mumoshu I'm happy for you to break config compatibility to keep kube-aws simple, it's trivial to move config around and we could even write a small version migration script.

@c-knowles @whereisaaron To sync up, let me confirm that you've never tried to do something like:
routeTableId: rtb-for-privatesubnet-withpreconfigurednat
mapPublicIPs: false
# !!!!!
controllerLoadBalancerPrivate: false

which is IMHO not recommended because it shouldn't work if you configured your existing AWS resources properly.
My reasoning here is that:

mapPublicIPs: false means you wanted to kube-aws to create private subnets for all the nodes while
controllerLoadBalancerPrivate: false means you also wanted it to create a k8s api ELB in public subnets.
However in this public subnets required by an external ELB aren't created by kube-aws because you've set mapPublicIPs: false. Am I correct?

Anyways, fyi, this use-case is intended to be newly supported via this PR with configuration like explained in https://github.com/coreos/kube-aws/pull/284#issuecomment-275954527.

@c-knowles Thanks as always!
Just a quick response to https://github.com/coreos/kube-aws/pull/284#issuecomment-276008083 but what is and its intention of controllerIP: <IP_ADDRESS>?

@c-knowles @whereisaaron And thanks for the kind words about following breaking changes in configuration syntax! It will definitely accelerate the development of kube-aws.

@mumoshu for your first comment, I'm not using private controller ELBs at all right now although if that's the preferred way once I add in a bastion host then I will use it (happy to go off best practice). For your second comment, I believe there was previously a bug in kube-aws which meant controllerIP had to be specified if we customised the subnets so in turn I preferred to be a bit more verbose in the config to avoid further issues, I think that goes away with HA controllers.

Wow, thanks for the quick work to add natGateway.preconfigured! As soon as there is a test release I'll recreate some test clusters with the new syntax. It sounds like the existing use case will be covered again now 🎉
This is my current successful use case under 0.9.3. Same settings for the cluster and all node-pools.
vpcId: vpc-12345678
routeTableId: rtb-12345678
mapPublicIPs: false
workerSecurityGroupIds:
  - sg-12345678

I deploy everything with private subnets and then get k8s to create public ELBs for Services and Ingress Controllers that should be exposed. Kubernetes specifically supports private subnet clusters with public ELBs with the kubernetes.io/role/elb subnet tag, which you apply to the (possibly empty) public subnets you want to use for public ELBs. You tag the public subnet you want to use for ELBs in each relevant AZ with:

Tag
Value

KubernetesCluster
your-cluster-name

kubernetes.io/role/elb

When Kubernetes is creating a public ELB, it collects all the public subnets associated with the cluster (tagged KubernetesCluster: cluster-name) and picks the first one in each AZ tagged kubernetes.io/role/elb, or if none are tagged kubernetes.io/role/elb it just picks the first public subnet. (It determines 'public' subnets by checking the route table for an 'igw*' entry.) The relevant code is here.
I keep the controller API ELB private, but you can certainly have controllers in private subnets but with a public API ELB. You (or kube-aws) just needs to create one public subnet (or one per AZ) for the ELB.
For mapPublicIPs: false deployments with private subnets, kube-aws could leverage the built-in k8s support for this by creating and tagging an empty public subnet (per AZ) for public LoadBalancer Services.

@whereisaaron @c-knowles To make your migration path extra clear, I've added an additional commit with more validations and backward-compatibility with the older config syntax based on my idea in https://github.com/coreos/kube-aws/pull/284#issuecomment-275998862
Specifically, I guess your (potential) use-case is covered by this test https://github.com/coreos/kube-aws/pull/284/commits/80885cbe58f6d25e03bf4e514749a84decfe31d6#diff-4fd4a4a9a3755708c6909f7f824f5754R207
Would you mind taking a look into it?
Is migration path clear enough for you? Do I understand your use-case correctly?
// Please forgive me if  I'm being too nervous here but I don't really like to break existing use-cases of yours!

Thanks @mumoshu as I read it that does cover the two use cases and seems pretty clean.

If you specify mapPublicIPs: false and NATGateway.preconfigured: true you must also specify routeTableId: rtb-12345678 and it is expected, and the user's responsibility that that route table will provide a NAT gateway. kube-aws will apply your route table to subnets but create no NAT gateways.
If you specify mapPublicIPs: true and NATGateway.preconfigured: true you must also specify routeTableId: rtb-12345678 and it is expected, and the user's responsibility that that route table will provide an Internet gateway. kube-aws will apply your route table to subnets but create no Internet gateways.

Thank you very much for keeping these use cases in play, even though you want to deprecate them 😢 . I would have thought the use case where kube-aws users already use AWS, and already have a working VPC with NAT would be fairly large group, and I'm surprised you plan to drop them.
In the use case where the user has no VPC and kube-aws is creating one, then creating the NAT instances for mapPublicIPs: false is a great new feature for the first install. However, mandating every subsequent install creates yet more NAT gateways seems onerous to me.
So I hope you'll consider keeping a use case where the user has an existing AWS VPC environment to deploy clusters into! 🙏 🤞

Thanks as always @whereisaaron :bow:
Sorry for the confusion if I wasn't very clear about it but I'm intended to deprecate confusing configuration syntax(es) but not your use-case(s)!
For example, I'm wondering if we could deprecate mapPublicIPs: true in favor of the ideal allSubnetsPublic: true and mapPublicIPs: false in favor of the ideal allSubnetsPrivate: true.
Then, when both of the ideal flags are set to false, we can be sure that the user does want to mix private/public subnets.
mapPublicIPs as of today can't be used like that as its default value is true. What I coded in this PR is that mapPublicIPs: true(which is the default) combined with subnets[].private: true results in private subnets! That's just confusing to be explained by not only me but everyone, isn't it?
Once again,

I would have thought the use case where kube-aws users already use AWS, and already have a working VPC with NAT would be fairly large group, and I'm surprised you plan to drop them.

I'm not intended to break that use-case!

Oh yeah, I totally misunderstood! Sorry! 💩
// DEPRECATED AND REMOVED IN THE FUTURE refers to the syntax rather than the use case. I am certainly happy adjust to any new syntax that makes the code or the configuration easier, as often as necessary.
Thanks for you efforts!!

@whereisaaron FYI, for simplicity of cluster.yaml, I've dropped the natGateway.preconfigured while retaining the support for your use-case!
Please read the newly added comments for subnets: key in the updated cluster.yaml to find your use-case and appropriate configuration.

Although important parts are covered by test/integration/maincluster_test.go and tested automatically by passing through real AWS service(=cfn stack validations), it's impossible to test all the use-cases myself.
I'd appreciate it if you could test yours by your own and report back if there's any issue :bow:


Can't set variadic methods

Currentyl it isn't possible to define variadic methods, we have to implement this, just like function does.

Implemented in the commit: https://github.com/clever-lang/clever/commit/4f51767905edc4f778f955ef86bab09c654d4419


Convert GC Forms home link from button to link

Summary | Résumé
Fixes #1333
The page title "GC Forms" was implemented as a button, but should be a Link.

Can we fix the active state?


Concrete rules ambiguity

In both the concrete-rules.txt and kevm.py there is a label named EVM-TYPES.Bytes.range which I can't find in the source files which leads to the following questions:

Kompile should have warnings emitted for each lemma in the concrete rules which is not found. Are any warnings emitted?
Still, there are rules in our codebase that use the [concrete]. Is there a reason for this?
Could we remove the hardcoded list from kevm.py and read the list from concrete-rules.txt?

For 3. is there a reason to have the concrete-rules.txt under tests/specs? Could we use something in python like return open(filename).read().splitlines() to avoid duplication?

Are you able to confirm with a minimal example that the concrete-rule is not actually in the definition, but kmopile does not give a warning? If so, let's open an issue for that one against K repo, with the minimal example.
And actually, I want to move the other direction, remove the concrete-rules.txt instead, because I want to move all the logic into Python. We have it set so that by default you get the pyk kompile: https://github.com/runtimeverification/evm-semantics/pull/1616. So we're really just keeping around the Bash compile "just in case" at this point.

Todo:

Minimal example in the frontend for kompile --backend haskell --concrete-rules MODULE-THAT-EXISTS.rule-that-doesnt-exist, and add as a test and error to th efrontend.
Once that is merged, remove the offending rules from kevm.py in this repo.
Also remove the now unused concrete-rules.txt in this repo.

The following labels do not have corresponding rules (as confirmed by both grep and the error check I added):

EVM-TYPES.#asByteStack
EVM-TYPES.#asByteStackAux.recursive
EVM-TYPES.#asWord.recursive
EVM-TYPES.#padRightToWidth
EVM-TYPES.#padToWidth
EVM-TYPES.#range
EVM-TYPES.Bytes.range
EVM-TYPES.mapWriteBytes.recursive


VSCode is working very slow

Issue Type: Performance Issue
Visual Studio Code is so slow I can't even work. Example: I type in a few characters and wait for Visual Studio Code to make suggestions. I have to wait like 2-3 seconds for it.
VS Code version: Code 1.65.2 (c722ca6c7eed3d7987c0d5c3df5c45f6b15e77d1, 2022-03-10T14:33:55.248Z)
OS version: Windows_NT x64 10.0.19043
Restricted Mode: No

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz (12 x 3192)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onrasterization: enabledskia_renderer: enabled_onvideo_decode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.92GB (8.70GB free)

Process Argv
C:\Users\Mateusz\OneDrive\Pulpit\westbo website --crash-reporter-id 3a8a9774-1fa9-4960-b91e-73d9b97b14c8

Screen Reader
no

VM
0%

Process Info
CPU %	Mem MB	   PID	Process
    0	    89	 20604	code main
    0	   148	  3352	   issue-reporter
    0	   204	  5100	   window (description.css - westbo website - Visual Studio Code)
    0	    24	 17368	   crashpad-handler
    0	   112	 19224	   gpu-process
    0	    93	 20580	   shared-process
    0	    69	 18800	     ptyHost
    0	    71	 20012	     fileWatcher
    0	   317	 21020	   extensionHost
    0	   119	 17596	     "C:\Users\Mateusz\AppData\Local\Programs\Microsoft VS Code\Code.exe" --ms-enable-electron-run-as-node "c:\Users\Mateusz\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\html-language-features\server\dist\node\htmlServerMain" --node-ipc --clientProcessId=21020
    0	   137	 20404	     "C:\Users\Mateusz\AppData\Local\Programs\Microsoft VS Code\Code.exe" --ms-enable-electron-run-as-node c:\Users\Mateusz\.vscode\extensions\dbaeumer.vscode-eslint-2.2.2\server\out\eslintServer.js --node-ipc --clientProcessId=21020
    0	     2	 23840	     D:\Git\cmd\git.exe for-each-ref --sort -committerdate --format "%(refname) %(objectname) %(*objectname)"
    0	   150	 23896	     "C:\Users\Mateusz\AppData\Local\Programs\Microsoft VS Code\Code.exe" --ms-enable-electron-run-as-node "c:\Users\Mateusz\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\css-language-features\server\dist\node\cssServerMain" --node-ipc --clientProcessId=21020
    0	    37	 21140	   utility-network-service

Workspace Info
|  Window (description.css - westbo website - Visual Studio Code)
|    Folder (westbo website): 51 files
|      File types: jpg(28) css(6) html(6) jpeg(4) svg(3) png(3)
|      Conf files:;

Extensions (12)

Extension
Author (truncated)
Version

vscode-eslint
dba
2.2.2

php-intellisense
fel
2.3.14

code-runner
for
0.11.7

beautify
Hoo
1.5.0

csharp
ms-
1.24.1

cpptools
ms-
1.9.7

vsliveshare
ms-
1.0.5449

material-icon-theme
PKi
4.14.1

LiveServer
rit
5.7.5

php-debug
xde
1.25.0

php-pack
xde
1.0.3

php-intellisense
zob
1.0.6

A/B Experiments
vsliv368:30146709
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstes627:30244334
pythonvspyl392:30443607
pythontb:30283811
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
pythondataviewer:30285071
vscod805cf:30301675
pythonvspyt200:30340761
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
vsc1dst:30438360
pythonvs932:30410667
wslgetstarted:30449410
vsclayoutctrt:30451275
dsvsc009:30452663
pythonvsnew555:30457759
vscscmwlcmc:30438804
vscgsvid2:30447481
pynewfile477:30450038

/needsMoreInfo
You should provide more details. Including what language is it and project setup.
If this langiage provided by an extension - you should report issue there.

I also have this problem. The new VS code update is annoying. Auto suggest takes 7 minutes to pop anything up. Stop giving us updates that slow us down, this hurts.

@seunafara please follow steps from https://github.com/microsoft/vscode/wiki/Performance-Issues to report issue with performance.
Unfortunately OP and comment have no hints where problem might be, if it's related to core functionality, extension, or something else.

yea @seunafara my autocomplete is unusable. There's a million threads about this over the last two years.

Hey @hediet, this issue might need further attention.
@Shadoweqq, you can help us out by closing this issue if the problem no longer exists, or adding more information.

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


[Bug]: does not work with gpg

Describe the bug
When my default config has commit.gpgsign set to true and my gpg signing key requires a passphrase obsidian-git does not notify or give me any kind of error. It just leaves the files committed and carries on as if everything is okay.
Relevant errors (if available)
Uncaught (in promise) Error: error: gpg failed to sign the data
fatal: failed to write commit object

    at Object.action (eval at <anonymous> (app.js:1), <anonymous>:2120:29)
    at PluginStore.exec (eval at <anonymous> (app.js:1), <anonymous>:2155:29)
    at eval (eval at <anonymous> (app.js:1), <anonymous>:2594:43)
    at new Promise (<anonymous>)
    at GitExecutorChain.handleTaskData (eval at <anonymous> (app.js:1), <anonymous>:2592:16)
    at GitExecutorChain.eval (eval at <anonymous> (app.js:1), <anonymous>:2576:44)
    at Generator.next (<anonymous>)
    at fulfilled (eval at <anonymous> (app.js:1), <anonymous>:2496:28)

Steps to reproduce
Create a gpg key with a passphrase. Update your ~/.gitconfig to have the following:
[user]
	signingkey = [YOUR KEY]
[commit]
	gpgsign = true

Make a file change to force Obsidian to do a commit
Expected Behavior
You could allow me to specify the gpg signing key in the configuration. However, I think that given the nature of this error you should show some kind of error in the status bar. I think that any other kind of error that occurs within this promise will also be ignored as well. I would recommend a .catch((err) => notifyUserOfError(err)) or something.
Addition context
Quick fix is to tell the local repository to not use gpg signatures with git config commit.gpgsign false or by modifying .git/config directly.
Operating system
Linux

I think this is a duplicate of #133, right?

Indeed it is. My apologies.


Ready state is always false even though a proper link_token is created and passed.

I'm having issues with the ready state never turning true even though a proper link_token is getting passed to the link. I went through the https://github.com/plaid/react-plaid-link/issues/239 post and tried some of their solutions they proposed such as moving the "PlaidLinkButton" into a separate location from the token fetching, but the ready state is still never hitting true for some reason
Here is my linkbutton: https://github.com/SP26GreenTeam/BudgetGreen/blob/NewMain/Screens/PlaidLinkButton.tsx
and here is the "LoginPage": https://github.com/SP26GreenTeam/BudgetGreen/blob/NewMain/Screens/PlaidLoginScreen.tsx
Any suggestions would be appreciated!

Hii, i'm facing this issue a lot. Please can uh tell me what solution you have applied to solve this issue ?


Player json is updating but the Player table in nfldb is not updating

I ran nfldb-update and could see Todd Gurley has LA as Team & RB as Position in the json file.
However, the player table shows UNK as Team & UNK as Position for Todd Gurley.
The player table only has 8 players where Team is LA

Why did you close this? What did you do to fix it?

Apologies, this was not the item I intended to close

What version of nfldb and nflgame do you have? (Run pip list.)

nfldb (0.2.10)
nflgame (1.2.20)

Please upgrade nfldb. Your version is over a year and a half old.

I was able to resolve this by running:
nflgame-update-players
nfldb-update
BTW- nfldb updated as well, nfldb (0.2.17)


Change camera input size for 11_Head_count_topview

About 11_Head_count_topview, we would like to change the MIPI camera input size from 640x480 to 1280x720 because we use our camera.
Is it correct that we need to change the below?
Or Is another sorce code needed to be changed too?
<define.h>
/Camera Capture Image Information/
#define USB_CAM_IMAGE_WIDTH         (640)
#define USB_CAM_IMAGE_HEIGHT        (480)
:
/Camera Capture Image Information/
#define CAM_IMAGE_WIDTH             (640)
#define CAM_IMAGE_HEIGHT            (480)

Thanks for trying this application.
Yes. along with that you need to make the following changes as well.

Changes in prepost_yolov3_headcount_top_cam.yaml
 #######################################
 # Input data
 #######################################
 input_to_pre:
   -
     name: "yuv_data"
     format: "YUY2"
     order: "HWC"
     shape: [720, 1280, 2]
     type: "uint8"

Changes in define.h
/Camera Capture Image Information/
#define USB_CAM_IMAGE_WIDTH         (1280)
#define USB_CAM_IMAGE_HEIGHT        (720)
#define USB_CAM_IMAGE_CHANNEL_YUY2  (2)
#define CAM_IMAGE_WIDTH             (1280)
#define CAM_IMAGE_HEIGHT            (720)
#define CAM_IMAGE_CHANNEL_YUY2      (2)
#define RESIZE_SCALE                (1.0)

thank you for your reply.
I will close this issue

Hello,
the above No1,2 have been changed,
however bounding box is not displayed correctly.
aside from No1,2 ,  which source code  will we have to change?


Support for interfaces?  Or better yet full Dipendency Inection and IoC?

I was curious if there are anything to support DI and IoC?  This is becoming more and more of a common pattern and i could see a lot of value of it being support in this library.
Consider the following
public interface IAnimal
{
     int TagId { get;set}
     int Age {get;set;}
     int Weight {get;set;}
}

public interface IDog : IAnimal
{
    string Breed {get;set;}
}

If you think about it, everything that you need to create tables and insert data can be found in the interfaces at least when it comes to creating the tables and writing the data.  The fun comes with reading and this could be handled via DI or and IoC pattern with a resolver.
Thoughts?

What does this get you?

Those are the interface descriptions -- not the concreate classes.  It was a simplistic use case is all.  For my actually use case, I have a very large scale enterprise application that is 100% based on the Design by Contract pattern to allow for 100% separation of concerns.  This allows for different teams to work on different part of the application without impacting each other during releases (different pieces have different release schedules).   We have a set of "contract resolvers" using Autofac (right now although we may switch), that we use through out the application to get a specific instance of what ever contract we need.
Ex:
var instance = Resolver.Resolve<IAnnualContract>();
this allows someone to work create an annual contract without needing to know much about it (they really shouldn't even need to care if you think about it)
So we are looking for a way to do something like
db.Save<IAnnualContract>(instance.)
or
 db.FetchAll<IAnnualContract>()
the first one is easy (several ORM frameworks handle that).  The second one seems to be the difficult one.  If you look at what json.net does it has a concept of a contract resolver that is used to create a concrete instance of the object based on the type (it uses the resolver pattern above as well) and then hydrates the properties.  This allows for the developers loading the data to again not have to really care or worry about the concrete instance, they just know that it will have the properties defined on the contract loaded correctly from the rdbms.
Does that help?

I see, I do understand what you are trying to do.
What happens if you have an implementation of an IAnnualContract that requires more information to be stored than the other implementations, do you just add the union of all the properties of all the implementations into the interface?

Fair question, however I'd ask that if the developer or architect is deciding to use the interface approach, wouldn't it be safe to say that those other properties are not needed in the persistence layer?  If they were they would be promoted.  The main reason being is - how would manage schema changes in that scenario?   In this pattern is the interfaces that defines the schema not the concrete instance.

I agree managing the schema would be extremely error prone, I was trying to figure out if I was missing something. I suppose if you are confident all of your contract's will always need the same groups of data this would would work.
I'm also curious how something like FetchAll would work. How would it know which class to instantiate?

wouldn't it just return an IList of the Interface?
var single = db.Fetch<IAnnualContract>() --< IAnnualContract
and
var listResult = db.FetchAll<IAnnualContract>()  <-- IList < IAnnualContract >
or am I missing something?

Well it has to instantiate a concrete type which implements the interface, how would it know which to pick? There would also have to be some place in SQLite you register the list of types that implement this interface.

that is what the DI container would handle.  Right now I believe you are using var obj -  Activator.CreateInstance(type), based on that I think the change would be just a matter of converting it to this var obj = container.Resolve(type)  (this is simplistic though).  To be a bit more complete it might make sense to have something like this
Here is an example of a generic DI resolver (Allows for using an DI framework)
public interface IContractResolver
{
    Func<Type, bool> CanResolve {get;set;}
    Func<Type,Type> Resolve { get;set; }
}

public class DefaultContactResolver : IContractResolver
{
   public Func<Type,bool> CanReslove {get;set;}
   public Func<Type, Type> Resolve {get;set;}
}

Used like this
    var container = new Autofac.ContainerBuilder().Build(); // this could just as easily use Ninject or Castle or any other DI framework

    var contractResolver = new DefaultContractResolver() {
         CanResolve = (type) => container.IsRegistered,
         Resolve = (type) => container.Resolve
    }

    var dbConnection = new SQLiteConnection() { Resolver = contractResolver }

then inside the ORM framework you'd have something like this
   var obj = Resolver.CanResolve(type) ?? Activator.CreateInstance(type);

Thank you, I believe this provides enough context for someone to tackle this if they wanted to.

I have taken a shot at the initial implementation and submitted a PR. Interested in taking a look at letting me know what you think?
Side question -- any thought to splitting out the source into smaller files? At 4k+ lines VS is getting a bit on the sluggish side :)


PHOENIX-4008: UPDATE STATISTIC should collect all versions of cells.

@twdsilva @karanmehta93
The test should fail at the assertion showed below, but it didn't. The debugging shows that, even in "UPDATE STATISTICS" command we set the flag to read all versions of cells and pass to the scanner, the scanner still only returns the latest version of the cell. Could you help to check whether the change I made and the test case is correct? I'm correctly debugging the internal.
        // Because I updated the values of the row with primary key 100,
        // I expected now the estimated bytes should include two versions of the cells
        // and the following assertion should fail, but the assertion actually succeed.
        assertEquals((Long) 30L, info.getEstimatedBytes());

Patch looks good overall! I have a few nits, please address them and it should be good to go!

I checked the following two test failures, they even failed in my clean local repository (just cloned from master without any local changes). Any suggestions?
Test Result (2 failures / ±0)
org.apache.phoenix.end2end.ConcurrentMutationsIT.testRowLockDuringPreBatchMutateWhenIndexed
org.apache.phoenix.end2end.ConcurrentMutationsIT.testLockUntilMVCCAdvanced


Use "official" reserved port.

Coverage remained the same at 100.0% when pulling ae254d8fa750c1d84417362b357f5c3bf3ceb82d on stephen-soltesz:master into 93cd5139f8f166feb69931b04e0c93c6c2a73236 on m-lab:master.


How to input geometry into WWT WindowsClient

I want to input some geometry data layers into WWT WindowsClient with pywwt.
e.g. Linestring(192.306622249 83.412894378,212.212193931 77.547512943,216.881430633 75.695992129,222.676357498 74.155503937,230.182150167 71.834016547)
There are hundreds of file like this so I can't do it manually.
When I use "new_layer()" it can not work. I read the documentation find it is not support column name "geometry".
When I use "load()" load a csv file it always pop up "Data Visualization Layer" window , so I have to click "Finish" in the window that I can input the next layer.
My question is How to input geometry data show above without pop up "Data Visualization Layer" window .

@astrofrog @ojustino Do either of you have any insight into this one? Is it possible that there might be a way to wire up the geometry column somewhere such that this could start working?

@qu123xxx I'm personally not familiar with the Windows client, so I'm not sure how to help in that context and haven't yet found someone who can. (That was part of the delay, so thank you for your patience.) Would you be comfortable using the Qt or Jupyter versions of pywwt?
If so -- it seems like you would like to draw several rectangles on the sky, which is possible through the add_polygon method.
You would just need to find a way to convert your list of Linestring positions into the Astropy SkyCoord that the function uses to draw your shapes. As the link shows, there are also a few customizable properties like color and line width that you can edit to your liking.
How does that sound as a start?

@ojustino Thank you for your reply. I can provide some information about this topic.
The WWT EXCEL ADDIN was a way to wire up the  geometry column without pop up "Data Visualization Layer" window. But it is now unavailable because of a not know Windows Security Update for Office.
Maybe you can learn some way from it to make it work in pywwt
I‘m afraid I can't use Qt or Jupyter versions of pywwt because I have to make a tour in Windows Client , and export a WWT Layer File.
I also have time to wait or solve this problem ,because I find some tour in the community of WWT also use  linestring or  TEXT to visualize some data. For example
http://www.worldwidetelescope.org/Community/#/ContentDetail/63
So I think if pywwt can achieve this function, users can use it to visualize more kinds of data.

@qu123xxx I see. It's good that you found a workaround, and we'll consider how pywwt could help with this kind of thing in the future.

Thank you :satisfied:


ls --hyperlink folder link aren't working

Context: https://github.com/microsoft/vscode/issues/163545#issuecomment-1401958467
Using codespaces. Works for files but not for folders
https://user-images.githubusercontent.com/1794099/214308284-708859dc-b777-4000-a6ff-bc883dfd549d.mov

This is working fine on a WSL terminal in a local window and also in a remote WSL window, so many Codespaces/other remote-related?

It also works for me in a Codespace 🤷

Mac local doesn't work as ls doesn't support it, on mac+codespace it works fine for me. Closing for re-verification in Feb


cmake: Look for revision specific overlays also in /boards/

This is a follow-up to commit 77ecd6837d5040734b7d87c9cbfefc67bf736c55.
Add missing check that looks for a revision specific overlay located
in the <app>/boards/ directory (before checking the main application
directory, similarly to how normal overlays are handled).
Signed-off-by: Andrzej Głąbek<EMAIL_ADDRESS>

Superseded by #31864.


Enable alternative to service connection for AzureRmWebAppDeploymentV4

Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Feature
Enter Task Name: AzureRmWebAppDeploymentV4
Environment
Azure Pipelines

Account / Team Project / Build: N/A
Agent - Hosted or Private: N/A

Issue Description
We leverage Hashicorp Vault to manage short-lived secrets for deployments today in our environment.
We would like to be able to pass Subscription/Client ID/Client Secret/Tenant ID as parameters into the AzureRMWebAppDeployment task, so we can source credentials from Vault at deploy time, rather than leveraging a service credential.
We would like to be able to do something like:
      - task: AzureRmWebAppDeployment@4
        inputs:
          ConnectionType: 'AzureRM'
          AzureClientID: <Client_ID>
          AzureSecretKey: <Secret_Key>
          AzureSubscriptionID: <Subscription_ID>
          AzureTenantID: <Tenant_ID>

That way, we can use a previous task to auto-generate the SPN through Vault, and pass that information as inputs.
Task logs
N/A
Troubleshooting
N/A
Error logs
N/A

This requirement is still relevant (Marking to remove stale)


ctk: management section of the component envelope to support dependentAPIs (as well as exposedAPIs)

Description
At the moment, the coreFunction of the component envelope allows you to specify exposedAPIs as well as dependentAPIs, but the management section only has the facility to have a list of exposedAPIs. Should we extend the management section to specify exposedAPIs as well as dependentAPIs to bring it into line with the coreFunction?
FYI: The security section also requires a specific 'partyrole' property that is not in any list.

At the moment, the event notifications are in their own section. Will it be possible to expose and subscribe to management events and security events?

I agree that we should allow exposedAPIs and dependentAPIs across the coreFunction, management and security sections, but they should follow the same format so that there's no structural difference between them. However, the partyrole API is mandatory in the security section, so it might be a good idea to harmonise them by making exposedAPIs and dependentAPIs arrays instead of a list.
Instead of:
exposedAPIs:
    - name: productcatalogmanagement
      specification: https://raw.githubusercontent.com/tmforum-apis/TMF620_ProductCatalog/master/TMF620-ProductCatalog-v4.0.0.swagger.json
      implementation: {{.Release.Name}}-prodcatapi
      apitype: openapi
      path: /{{.Release.Name}}-{{.Values.component.name}}/tmf-api/productCatalogManagement/v4
      developerUI: /{{.Release.Name}}-{{.Values.component.name}}/tmf-api/productCatalogManagement/v4/docs
      port: 8080

...we would have:
exposedAPIs:
      productcatalogmanagement
        specification: https://raw.githubusercontent.com/tmforum-apis/TMF620_ProductCatalog/master/TMF620-ProductCatalog-v4.0.0.swagger.json
        implementation: {{.Release.Name}}-prodcatapi
        apitype: openapi
        path: /{{.Release.Name}}-{{.Values.component.name}}/tmf-api/productCatalogManagement/v4
        developerUI: /{{.Release.Name}}-{{.Values.component.name}}/tmf-api/productCatalogManagement/v4/docs
        port: 8080

Then, in CTK we could test to ensure the security.exposedAPIs section has an element called partyrole.

Also, I think we should have a key in exposedAPIs to specify whether the API is exposed only to the cluster or to the outside world via the gateway. We could call it exposure or scope or visibility with a value of cluster if it's only exposed inside the cluster or gateway if it exposed to the outside world via the gateway. This means we could differentiate between coreFunction APIs that are exposed outside the cluster and security APIs that aren't.

At the moment, the event notifications are in their own section. Will it be possible to expose and subscribe to management events and security events?

@LesterThomas, I think my suggestion could help to address this because we can then expose all event APIs in the same way.

I agree with all of the comments above, and I've also reviewed this with @sharrop
So, in summary the proposal is to create arrays of exposedAPI and dependentAPI in a consistent pattern across coreFunction, management and security and to have an additional exposure property which is an enumeration of [cluster,gateway].
In addition, as part of this change, we should improve the validation of the spec by using the appropriate required fields and also using url, enumeration and other more specific types instead of just string.


Allow commitInfo's operationParameters to have non-string values

For existing tables in the wild we've seen some with "operationParameters":{"version":10,"timestamp":null}. This breaks reads with the following error:
Error: Arrow(JsonError("whilst decoding field 'commitInfo': whilst decoding field 'operationParameters': expected string got 10"))

Despite commitInfo allowing arbitrary JSON (unless constrained by in-commit timestamps), we attempt to prescribe structure when reading. This should almost never fail but due to arrow json reading it errors on attempting to read int 10 as a String.
We likely need to come up with a way to allow deserializing arbitrary JSON in operationParameters? How can we represent that with our existing Schema system?

related: https://github.com/delta-io/delta/issues/2419


noUncheckedIndexedAccess failed when destructuring from a tuple with rest elements

Bug Report

🔎 Search Terms

noUncheckedIndexedAccess destructuring tuple rest elements
🕗 Version & Regression Information

This is the behavior in every version I tried

⏯ Playground Link

Playground link with relevant code
💻 Code

type NonEmptyStringArray = [string, ...Array<string>]

const strings: NonEmptyStringArray = ['one', 'two', 'three']
const [s0, s1, s2, s3] = strings;
//                  ^? const s3: string | undefined
  s3.toUpperCase()
//^? const s3: string

🙂 Expected behavior
s3.toUpperCase() shows type error.
🙁 Actual behavior
s3.toUpperCase() shows no error.
Note that undefined is correctly included inside the assignment statement but then lost.
Related: https://github.com/microsoft/TypeScript/issues/40657

Note that undefined is correctly included inside the assignment statement but then lost.

It's probably cause hovering at that position shows you the "declared type" - but when hovering over the identifier after this assignment you see the "flow type" (the result of the control flow analysis). So my educated guess is that the problem is in the CFA


Replace the binary Wsk with REST interface WskRest

All the non-CLI related tests should sunset the wsk, and re-implement them with wskrest.
import common.wsk
val wsk = new Wsk

need to be changed to
import common.rest.WskRest
val wsk = new WskRest

plus there may be changes to the rest of the tests as well, since the response result may have been changed.

There is one test case:
RSSTests

Closing issue as project is being archived.


Chore: update polkadot/api to 10.9.1

Resolves #670
DRAFT
Changes made:

[x] moved type generation into separate module, using esnext to work around https://github.com/polkadot-js/api/issues/5540
[x] polkadot/api is now hoisted transitively for interbtc-api from the submodule (api-augment)
[x] fix broken imports, and type name change
[x] hacky-but-works™ import of json definitions from @interlay/interbtc-types

TODO:

[ ] Tests pass
[ ] Review and clean up leftover comments, temporary code pieces, remove unused files

Sadly, with this approach I could not make the mocha tests work in their current form, either.
Closing this draft and continuing to migrate mocha/chai/sinon to jest in PR #672


basic autom4te invocation creates mixed ascii/ebcdic m4 file

Running tests/firstbug.sh and then hacking 'm4' to capture the temporary m4 file it generates, as follows:
#!/bin/sh
echo "$" >>/tmp/out
tmparg="$2"
if [ "${tmparg#/tmp}" != "${tmparg}" ]; then
backup=basename ${tmparg}
echo "Copied ${tmparg} to /tmp/${backup}.$$" >>/tmp/out
cp "${tmparg}" "/tmp/${backup}.$$"
fi
exec /usr/local/bin/realm4 "$@"
and also renaming /usr/local/bin/m4 to /usr/local/bin/realm4
then we can capture the temporary m4 file (probably an easier way, but this worked).
The temporary file is marked as IBM-1047 (should probably be ISO8859-1) but the bigger problem is the contents are mixed.
They start in EBCDIC:
divert(-1)
changequote([, ])
_at_MODE(SEPARATOR, ELT1, ELT2...)
----------------------------------
List the elements, separating then with SEPARATOR.
MODE can be:
'at'	      -- the elements are enclosed in brackets.
'star'     -- the elements are listed as are.
'percent'  -- the elements are 'flattened': spaces are singled out,
and no new line remains.
...
but later there is ASCII:
gnu:
line:	<line>
program:    <program>
unix:
builtin:
changecom:
changequote:
debugfile:
debugmode:
decr:
define:
defn:
divert:
divnum:
dnl:
...
This actually. bounces back and forth a few times

this works when using the makeport version of 'make' and the m4port version of 'm4'


add travis-ci integration

This change adds Travis-CI integration, both for meson (which was an incredible struggle to get working on TravisCI since every single version of {python, meson, and ninja} were too old) and make.
You can see a typical build result here:
https://travis-ci.org/travisdowns/libpfc/builds/354614705
To get it running, you just need to sign up for Travis-CI (you can just click the "use my github account") and then enable the switch for the libpfc repo. The build will start on the next commit (or you can trigger one by hand). It will also build PRs which is really handy.

Currently it does the full build including the kernel module, and also install in the meson case. You can't actually insmod the kernel module though: the module reports "no performance counters" which makes sense since this is a virtualized environment.

@travisdowns Astounding. Thank you!


Add perspective

Add .perspective

Oh yeah, this is cool! Thanks @ts1

Hey @ts1, I've been troubleshooting some Saucelabs/CI issues — and I've finally got the development branch back into shape, but all the rebasing has borked your PR.
I can resolve the merge conflicts, but it's important to me that your contribution shows you as the author, so do you mind rebasing #12 and #13 with the latest development ??

Merged into 0.4.0 thanks @ts1 !

Thank you!


Cosmetic issues in formatting of copy/paste instructions

Cosmetic. Step 3 label in Copy/Paste districts/communities should use same font/fontsize/weight as other step labels

fixed


Change spectral rendering range from 400nm-700nm to CIE recommendation, 380nm-780nm

Actually, the recommended range according to ASTM E308–15 - Standard Practice for Computing the Colors of Objects by Using the CIE System, is 360-780nm.
See also https://github.com/colour-science/colour/issues/242
I did some fits with data outside this range, but there is a recommended practice when the data is available on smaller wavelength ranges than 360-780nm. We'll be using the official recommended practice.


feat: navbar sidebar sorting

:tada: This PR is included in version 0.103.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


chore: update gcp-metadata to 0.6.3

This commit also changes the typescript version, so the issue title isn't accurate. Is the typescript change intentional?

The Typescript change is required by the gcp-metadata change.  I'll update the description.


Subsample example data to 50 sequences

Description of proposed changes
Metadata and sequences files are sourced from elsewhere now (see d5bf10829e95552d59c332bc6c2f11dd8aa2c0f5 + a7f3d272b8008729ad215de6fec1dc11f48f423b).
The example_data/ files are still used by CI, but a smaller set of data is sufficient and allows for faster CI runs.
 You can generate your own index ahead of time with `augur index` and pass it with `augur filter --sequence-index`.
364 strains were dropped during filtering
	234 had no metadata
	3 had no sequence data
	127 of these were dropped because of subsampling criteria
50 strains passed all filters

Related issue(s)

Motivation: https://github.com/nextstrain/monkeypox/pull/62#issuecomment-1158047607

Testing
See checks.

Thanks for attending to this @victorlin. However, the metadata downloaded from data.nextstrain.org has had its fields updated recently. Can you regenerate the 50 sequence example data from a fresh download? Also, can you ensure that the 50 genomes include all clades under the field clade and all all lineages under the field lineage?

@trvrb done, 3 notable changes:

Sourced from data.nextstrain.org
Used --group-by clade lineage to ensure all clades and all lineages are accounted for
Used --include-where strain=MK783032 strain=MK783030 to ensure the root (defined in config_hmpxv1.yaml) is included


vmware: Update and expand the documentation

vmware: Update and expand the documentation
Important basic information was missing, like the usage of ovftool
and how static IPs are configured.
The relation of Ignition and coreos-cloudinit is explained because
both use the same data source and Ignition has serveral limitations.
Depending on the use case, some important behavior was documented
such as which guestinfo variables result in what networkd setting
when coreos-cloudinit is used, or how custom templating should be
done with /run/metadata/coreos.
os/provisioning: Mention the ct docker image

Would it make sense to also update the names from guestinfo.coreos.* to guestinfo.ignition.* in the documentation or is further work (e.g. in cloud-init) required before we can do that?

guestinfo.ignition.* support is not released yet on any channel, so I kept the old names.


Downloaded tools is missing the Package Deployer PowerShell module

The PowerShell script on this page to download and install the developer tools is missing the Package Deployer PowerShell module used for importing packages. Can this please be added? I can provide working code if you'd like.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3b85e9ae-06bb-4012-386e-09389bbc1963
Version Independent ID: 35b1020a-0f19-fcc3-3ee6-b7f5559ccd0e
Content: Download tools from NuGet (Developer Guide for Dynamics 365 Customer Engagement)
Content Source: ce/developer/download-tools-nuget.md
Service: crm-online
GitHub Login: @JimDaly
Microsoft Alias: jdaly

Please see pull request #152 for the changes.

Thanks @amervitz . This finally got merged yesterday.
@shmcarth FYI


Transfer a repo - acapycontroller

Please include:

GitHub repo to transfer: https://github.com/OneOf-Inc/acapycontroller
Lab to be added to: https://github.com/hyperledger-labs/hyperledger-labs.github.io/pull/259
Teams with which access: (examples on GitHub team list)

Team name: aries-agent-controller-maintainers
Role:

Write

This is a new team and should have the initial committers added to it

@monar24 : You will need to initiate the transfer of this repo to @ryjones or myself @tkuhrt so that we may move it to the hyperledger-labs organization.

https://github.com/hyperledger-labs/aries-agent-controller

Invites also sent to the maintainers.

please add a MAINTAINERS.md file as well


AYS API raml console JSON error

Installation information

jumpscale version: 9.0.3
operating system: Ubuntu 16.04

Detailed description
The api requests gives json load error if we use the AYS api raml console
Steps to reproduce
1 - ays start
2 - go to: http://<JS9_CONTAINER_IP>:5000/apidocs/index.html?raml=api.raml
3 - use any api endpoint which uses json body (i.e. /ays/template_repo)
4 - fill the body of the request (i.e {'url':'<GIT_URL>'})
Relevant stacktraces
2017-07-02 14:20:48 - (sanic)[ERROR]: Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/sanic/request.py", line 67, in json
self.parsed_json = json_loads(self.body)
ValueError: Expected object or value
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/sanic/app.py", line 471, in handle_request
response = await response
File "/opt/code/github/jumpscale/ays9/JumpScale9AYS/ays/server/ays_if.py", line 178, in post
return await auth(request, ays_api.addTemplateRepo(request))
File "/opt/code/github/jumpscale/ays9/JumpScale9AYS/ays/server/ays_if.py", line 14, in auth
return await func
File "/opt/code/github/jumpscale/ays9/JumpScale9AYS/ays/server/ays_api.py", line 37, in addTemplateRepo
inputs = request.json
File "/usr/local/lib/python3.5/dist-packages/sanic/request.py", line 71, in json
raise InvalidUsage("Failed when parsing body as json")
sanic.exceptions.InvalidUsage: Failed when parsing body as json

The issue was because of using single quotes instead of double quotes in json object


No matlab

@jordandekraker making this draft PR so I can bring it up to date with master, and translate the warps/gifti code too. I've integrated everything else so far and tested only with dry run, will try running for real soon..

This should be ready to merge now:

replaces all matlab/mcr with python
shape injection uses greedy (much faster)
laplace coord
default behaviour will tar up each subject's work folder after completion (--keep_work to prevent)

Some remaining things to fix in a later PR:

midsurf visualization disabled right now, failure could be due to fov of surfaces (?) need to investigate
documentation / install instructions could be updated


Automated cherry pick of #6817: guestnetworks: use math/rand instead of crypto/rand

Cherry pick of #6817 on release/2.13.
#6817: guestnetworks: use math/rand instead of crypto/rand

Codecov Report

Merging #6821 into release/2.13 will decrease coverage by 0.36%.
The diff coverage is 10.10%.

@@               Coverage Diff               @@
##           release/2.13   #6821      +/-   ##
===============================================
- Coverage          6.86%   6.50%   -0.37%
===============================================
  Files               536     551      +15
  Lines             90621   97382    +6761
===============================================
+ Hits               6222    6330     +108
- Misses            83889   90542    +6653
  Partials            510     510

Flag
Coverage Δ

#aFlag
6.50% <10.10%> (-0.37%)
:arrow_down:

Impacted Files
Coverage Δ

pkg/cloudcommon/policy/policy.go
1.58% <ø> (+<0.01%)
:arrow_up:

pkg/cloudcommon/policy/resources.go
0.00% <ø> (ø)

pkg/cloudprovider/securitygroup.go
0.00% <0.00%> (ø)

pkg/compute/models/cloudproviders.go
1.01% <0.00%> (-0.01%)
:arrow_down:

pkg/compute/models/disks.go
0.54% <0.00%> (-0.01%)
:arrow_down:

pkg/compute/models/guestdisks.go
0.57% <ø> (+<0.01%)
:arrow_up:

pkg/compute/models/guestnetworks.go
0.21% <0.00%> (+<0.01%)
:arrow_up:

pkg/compute/models/regiondrivers.go
20.00% <ø> (ø)

pkg/compute/models/secgrouprules.go
4.20% <0.00%> (ø)

pkg/compute/models/secgroups.go
1.88% <0.00%> (-0.01%)
:arrow_down:

... and 38 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7492240...c5812e2. Read the comment docs.


Everyone SBAT know the plan for deploying attestation services to Alfajores/Betanet

Expected Behavior
There should be a plan that everyone can reference that minimizes the impact on the user experience of both mobile users and validators.
Current Behavior
There is no plan anyone can reference

Roadmap for imminent changes to Identity, especially as it relates to the wallet:
Already finished protocol work:

Creation of the attestation service https://github.com/celo-org/celo-monorepo/issues/1039
Helm chart for Clabs to deploy attestation service for clabs validators https://github.com/celo-org/celo-monorepo/issues/1076
Metadata registration + contractkit work to register and access attestation service url https://github.com/celo-org/celo-monorepo/issues/1104

Protocol work necessary before mobile can start work (this sprint):

Make randomness historically available https://github.com/celo-org/celo-monorepo/issues/1181
Addition of the revealIssuers tx to accomodate https://github.com/celo-org/celo-labs/issues/80

Mobile work necessary to move to attestation service (requirement for Betanet?):

Have wallet implement the above changes (remove reveal tx, add revealIssuers tx) https://github.com/celo-org/celo-monorepo/issues/1040
For a smooth transition on Alfajores (assuming we will need it), we can temporarily support revealing both via the old reveal tx (and thus use the central verification pool), and directly to the attestation service to protect against validators not being "compliant" on Alfajores.

Rollout plan (Alfajores):

We update documentation to indicate to validators that at the next Alfajores deploy they are expected to run the attestation service (however we still keep the old reveal functionality in for validators who miss it) as well as register the attestation service URL in their metadata that they have to host.
Once mobile work has finished, we'll do the deploy to alfajoresstaging for testing.
Reset Alfajores (@timmoreton ?)
Have the mobile app run with both modes of revealing and observe if validators are "compliant".

Rollout plan (Betanet):

We update documentation to indicate to validators that they are expected to run the attestation service as well as register the attestation service URL in their metadata that they have to host.

Further work:

Raise the attestation fee to $0.07 (since it can cost more than $0.05 to send) + derivative changes for the wallet
Eventually remove ABE code from celo-blockchain

As talked about in today's sprint meeting, we will no longer have the grace period in which we reveal both ways. Instead the mobile app will detect, similarly to right now, if a validator does not seem to have the right setup for attestation and request additional attestations which hopefully yield validators that will actually run the attestation service

Also this board https://github.com/orgs/celo-org/projects/113

Alfajores re-deploy is tentatively scheduled for 10/23, so we aim to get all the relevant changes in then


python312Packages.pysigma: 0.11.7 -> 0.11.10

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python312Packages.pysigma is: Library to parse and convert Sigma rules into queries
meta.homepage for python312Packages.pysigma is: https://github.com/SigmaHQ/pySigma
meta.changelog for python312Packages.pysigma is: https://github.com/SigmaHQ/pySigma/releases/tag/v0.11.9
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.11.9 with grep in /nix/store/jflhqsbx2nlyc1zfh6kbvwz2k44xggdi-python3.12-pysigma-0.11.9
found 0.11.9 in filename of file in /nix/store/jflhqsbx2nlyc1zfh6kbvwz2k44xggdi-python3.12-pysigma-0.11.9

Rebuild report (if merged into master) (click to expand)

22 total rebuild path(s)

22 package rebuild(s)

First fifty rebuilds by attrpath

python311Packages.pysigma
python311Packages.pysigma-backend-elasticsearch
python311Packages.pysigma-backend-insightidr
python311Packages.pysigma-backend-opensearch
python311Packages.pysigma-backend-qradar
python311Packages.pysigma-backend-splunk
python311Packages.pysigma-backend-sqlite
python311Packages.pysigma-pipeline-crowdstrike
python311Packages.pysigma-pipeline-sysmon
python311Packages.pysigma-pipeline-windows
python312Packages.pysigma
python312Packages.pysigma-backend-elasticsearch
python312Packages.pysigma-backend-insightidr
python312Packages.pysigma-backend-opensearch
python312Packages.pysigma-backend-qradar
python312Packages.pysigma-backend-splunk
python312Packages.pysigma-backend-sqlite
python312Packages.pysigma-pipeline-crowdstrike
python312Packages.pysigma-pipeline-sysmon
python312Packages.pysigma-pipeline-windows
sigma-cli

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/jflhqsbx2nlyc1zfh6kbvwz2k44xggdi-python3.12-pysigma-0.11.9 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python312Packages.pysigma https://github.com/r-ryantm/nixpkgs/archive/9993313ef34dcf5e980a276f894a91a98fb54f34.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/9993313ef34dcf5e980a276f894a91a98fb54f34#python312Packages.pysigma

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/jflhqsbx2nlyc1zfh6kbvwz2k44xggdi-python3.12-pysigma-0.11.9
ls -la /nix/store/jflhqsbx2nlyc1zfh6kbvwz2k44xggdi-python3.12-pysigma-0.11.9/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

10 packages failed to build:

python311Packages.pysigma-backend-elasticsearch
python311Packages.pysigma-backend-elasticsearch.dist
python311Packages.pysigma-backend-opensearch
python311Packages.pysigma-backend-opensearch.dist
python312Packages.pysigma-backend-elasticsearch
python312Packages.pysigma-backend-elasticsearch.dist
python312Packages.pysigma-backend-opensearch
python312Packages.pysigma-backend-opensearch.dist
sigma-cli
sigma-cli.dist

32 packages built:

python311Packages.pysigma
python311Packages.pysigma-backend-insightidr
python311Packages.pysigma-backend-insightidr.dist
python311Packages.pysigma-backend-qradar
python311Packages.pysigma-backend-qradar.dist
python311Packages.pysigma-backend-splunk
python311Packages.pysigma-backend-splunk.dist
python311Packages.pysigma-backend-sqlite
python311Packages.pysigma-backend-sqlite.dist
python311Packages.pysigma-pipeline-crowdstrike
python311Packages.pysigma-pipeline-crowdstrike.dist
python311Packages.pysigma-pipeline-sysmon
python311Packages.pysigma-pipeline-sysmon.dist
python311Packages.pysigma-pipeline-windows
python311Packages.pysigma-pipeline-windows.dist
python311Packages.pysigma.dist
python312Packages.pysigma
python312Packages.pysigma-backend-insightidr
python312Packages.pysigma-backend-insightidr.dist
python312Packages.pysigma-backend-qradar
python312Packages.pysigma-backend-qradar.dist
python312Packages.pysigma-backend-splunk
python312Packages.pysigma-backend-splunk.dist
python312Packages.pysigma-backend-sqlite
python312Packages.pysigma-backend-sqlite.dist
python312Packages.pysigma-pipeline-crowdstrike
python312Packages.pysigma-pipeline-crowdstrike.dist
python312Packages.pysigma-pipeline-sysmon
python312Packages.pysigma-pipeline-sysmon.dist
python312Packages.pysigma-pipeline-windows
python312Packages.pysigma-pipeline-windows.dist
python312Packages.pysigma.dist

Maintainer pings
cc @fabaff for testing.

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python312Packages.pysigma is: Library to parse and convert Sigma rules into queries
meta.homepage for python312Packages.pysigma is: https://github.com/SigmaHQ/pySigma
meta.changelog for python312Packages.pysigma is: https://github.com/SigmaHQ/pySigma/releases/tag/v0.11.10
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.11.10 with grep in /nix/store/y7z1ijkd6cq5asc15n7hwhz5xbxmacji-python3.12-pysigma-0.11.10
found 0.11.10 in filename of file in /nix/store/y7z1ijkd6cq5asc15n7hwhz5xbxmacji-python3.12-pysigma-0.11.10

Rebuild report (if merged into master) (click to expand)

22 total rebuild path(s)

22 package rebuild(s)

First fifty rebuilds by attrpath

python311Packages.pysigma
python311Packages.pysigma-backend-elasticsearch
python311Packages.pysigma-backend-insightidr
python311Packages.pysigma-backend-opensearch
python311Packages.pysigma-backend-qradar
python311Packages.pysigma-backend-splunk
python311Packages.pysigma-backend-sqlite
python311Packages.pysigma-pipeline-crowdstrike
python311Packages.pysigma-pipeline-sysmon
python311Packages.pysigma-pipeline-windows
python312Packages.pysigma
python312Packages.pysigma-backend-elasticsearch
python312Packages.pysigma-backend-insightidr
python312Packages.pysigma-backend-opensearch
python312Packages.pysigma-backend-qradar
python312Packages.pysigma-backend-splunk
python312Packages.pysigma-backend-sqlite
python312Packages.pysigma-pipeline-crowdstrike
python312Packages.pysigma-pipeline-sysmon
python312Packages.pysigma-pipeline-windows
sigma-cli

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/y7z1ijkd6cq5asc15n7hwhz5xbxmacji-python3.12-pysigma-0.11.10 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python312Packages.pysigma https://github.com/r-ryantm/nixpkgs/archive/78252ffd2233608c82db9c0f81f3fd8aca623491.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/78252ffd2233608c82db9c0f81f3fd8aca623491#python312Packages.pysigma

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/y7z1ijkd6cq5asc15n7hwhz5xbxmacji-python3.12-pysigma-0.11.10
ls -la /nix/store/y7z1ijkd6cq5asc15n7hwhz5xbxmacji-python3.12-pysigma-0.11.10/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

10 packages failed to build:

python311Packages.pysigma-backend-elasticsearch
python311Packages.pysigma-backend-elasticsearch.dist
python311Packages.pysigma-backend-opensearch
python311Packages.pysigma-backend-opensearch.dist
python312Packages.pysigma-backend-elasticsearch
python312Packages.pysigma-backend-elasticsearch.dist
python312Packages.pysigma-backend-opensearch
python312Packages.pysigma-backend-opensearch.dist
sigma-cli
sigma-cli.dist

32 packages built:

python311Packages.pysigma
python311Packages.pysigma-backend-insightidr
python311Packages.pysigma-backend-insightidr.dist
python311Packages.pysigma-backend-qradar
python311Packages.pysigma-backend-qradar.dist
python311Packages.pysigma-backend-splunk
python311Packages.pysigma-backend-splunk.dist
python311Packages.pysigma-backend-sqlite
python311Packages.pysigma-backend-sqlite.dist
python311Packages.pysigma-pipeline-crowdstrike
python311Packages.pysigma-pipeline-crowdstrike.dist
python311Packages.pysigma-pipeline-sysmon
python311Packages.pysigma-pipeline-sysmon.dist
python311Packages.pysigma-pipeline-windows
python311Packages.pysigma-pipeline-windows.dist
python311Packages.pysigma.dist
python312Packages.pysigma
python312Packages.pysigma-backend-insightidr
python312Packages.pysigma-backend-insightidr.dist
python312Packages.pysigma-backend-qradar
python312Packages.pysigma-backend-qradar.dist
python312Packages.pysigma-backend-splunk
python312Packages.pysigma-backend-splunk.dist
python312Packages.pysigma-backend-sqlite
python312Packages.pysigma-backend-sqlite.dist
python312Packages.pysigma-pipeline-crowdstrike
python312Packages.pysigma-pipeline-crowdstrike.dist
python312Packages.pysigma-pipeline-sysmon
python312Packages.pysigma-pipeline-sysmon.dist
python312Packages.pysigma-pipeline-windows
python312Packages.pysigma-pipeline-windows.dist
python312Packages.pysigma.dist

Maintainer pings
cc @fabaff for testing.

Add a :+1: reaction to pull requests you find important.

Superseded by https://github.com/NixOS/nixpkgs/pull/339325


remove irrelevant warning for non-httpsTrigger Cloud Functions

TL;DR
Warning "Output 'url' was not set - only httpsTrigger Cloud Functions return this attribute." is fired for non-httpsTrigger functions as well. But nature of the warning says it is relevant to httpsTrigger functions only.
Expected behavior
Warning is not fired for non-httpsTrigger function deploy.
Observed behavior
Warning is fired for non-httpsTrigger function deploy.
Action YAML
name: Deploy-GCF-Identify-Updates-Of-Topics

on:
  push:
    branches : main
    paths:
      - 'src/identify_updates_of_topics/main.py'
      - 'src/identify_updates_of_topics/requirements.txt'
      - '.github/workflows/deploy_identify_updates_of_topics.yml'

jobs:
  deploy_function:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Authenticate in GCP
        id: auth
        uses: google-github-actions/auth@v1
        with:
          credentials_json: ${{ secrets.gcp_credentials }}

      - name: Deploy the CF
        id: deploy
        uses: google-github-actions/deploy-cloud-functions@v1
        with:
            source_dir: src/identify_updates_of_topics
            name: identify_updates_of_topics
            runtime: python37
            entry_point: main
            region: europe-west3
            timeout: 540
            max_instances: 1
            memory_mb: 8192
            event_trigger_type: providers/cloud.pubsub/eventTypes/topic.publish
            event_trigger_resource: topic_name

Log output
2023-01-22T11:56:26.9462961Z Requested labels: ubuntu-latest
2023-01-22T11:56:26.9463016Z Job defined at: ...@refs/heads/main
2023-01-22T11:56:26.9463050Z Waiting for a runner to pick up this job...
2023-01-22T11:56:27.2299797Z Job is waiting for a hosted runner to come online.
2023-01-22T11:56:30.5200052Z Job is about to start running on the hosted runner: Hosted Agent (hosted)
2023-01-22T11:56:33.5259000Z Current runner version: '2.300.2'
2023-01-22T11:56:33.5297317Z ##[group]Operating System
2023-01-22T11:56:33.5298219Z Ubuntu
2023-01-22T11:56:33.5298886Z 22.04.1
2023-01-22T11:56:33.5299391Z LTS
2023-01-22T11:56:33.5300007Z ##[endgroup]
2023-01-22T11:56:33.5300607Z ##[group]Runner Image
2023-01-22T11:56:33.5301219Z Image: ubuntu-22.04
2023-01-22T11:56:33.5301867Z Version: 20230118.2
2023-01-22T11:56:33.5302737Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20230118.2/images/linux/Ubuntu2204-Readme.md
2023-01-22T11:56:33.5303814Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20230118.2
2023-01-22T11:56:33.5304642Z ##[endgroup]
2023-01-22T11:56:33.5305251Z ##[group]Runner Image Provisioner
2023-01-22T11:56:33.5305925Z <IP_ADDRESS>
2023-01-22T11:56:33.5306473Z ##[endgroup]
2023-01-22T11:56:33.5308529Z ##[group]GITHUB_TOKEN Permissions
2023-01-22T11:56:33.5309581Z Actions: write
2023-01-22T11:56:33.5310160Z Checks: write
2023-01-22T11:56:33.5310893Z Contents: write
2023-01-22T11:56:33.5311575Z Deployments: write
2023-01-22T11:56:33.5312321Z Discussions: write
2023-01-22T11:56:33.5312890Z Issues: write
2023-01-22T11:56:33.5313428Z Metadata: read
2023-01-22T11:56:33.5314027Z Packages: write
2023-01-22T11:56:33.5314783Z Pages: write
2023-01-22T11:56:33.5315968Z PullRequests: write
2023-01-22T11:56:33.5316691Z RepositoryProjects: write
2023-01-22T11:56:33.5317321Z SecurityEvents: write
2023-01-22T11:56:33.5317961Z Statuses: write
2023-01-22T11:56:33.5318514Z ##[endgroup]
2023-01-22T11:56:33.5323188Z Secret source: Actions
2023-01-22T11:56:33.5324041Z Prepare workflow directory
2023-01-22T11:56:33.6433094Z Prepare all required actions
2023-01-22T11:56:33.6685482Z Getting action download info
2023-01-22T11:56:33.9562426Z Download action repository 'actions/checkout@v3' (SHA:...)
2023-01-22T11:56:34.6584046Z Download action repository 'google-github-actions/auth@v1' (SHA:...)
2023-01-22T11:56:35.0857505Z Download action repository 'google-github-actions/deploy-cloud-functions@v1' (SHA:...)
2023-01-22T11:56:35.9393366Z ##[group]Run actions/checkout@v3
2023-01-22T11:56:35.9393847Z with:
2023-01-22T11:56:35.9394214Z   repository: ...
2023-01-22T11:56:35.9394835Z   token: ***
2023-01-22T11:56:35.9395362Z   ssh-strict: true
2023-01-22T11:56:35.9395760Z   persist-credentials: true
2023-01-22T11:56:35.9396136Z   clean: true
2023-01-22T11:56:35.9396459Z   fetch-depth: 1
2023-01-22T11:56:35.9396796Z   lfs: false
2023-01-22T11:56:35.9397113Z   submodules: false
2023-01-22T11:56:35.9397463Z   set-safe-directory: true
2023-01-22T11:56:35.9397828Z ##[endgroup]
2023-01-22T11:56:36.3590863Z Syncing repository: ...
2023-01-22T11:56:36.3592905Z ##[group]Getting Git version info
2023-01-22T11:56:36.3593594Z Working directory is '/home/runner/work/...'
2023-01-22T11:56:36.3594372Z [command]/usr/bin/git version
2023-01-22T11:56:36.3793701Z git version 2.39.1
2023-01-22T11:56:36.3829588Z ##[endgroup]
2023-01-22T11:56:36.3858653Z Temporarily overriding HOME='/home/runner/work/_temp/...' before making global git config changes
2023-01-22T11:56:36.3859729Z Adding repository directory to the temporary git global config as a safe directory
2023-01-22T11:56:36.3866320Z [command]/usr/bin/git config --global --add safe.directory /home/runner/work/...
2023-01-22T11:56:36.3927516Z Deleting the contents of '/home/runner/work/...'
2023-01-22T11:56:36.3935502Z ##[group]Initializing the repository
2023-01-22T11:56:36.3939497Z [command]/usr/bin/git init /home/runner/work/...
2023-01-22T11:56:36.4058818Z hint: Using 'master' as the name for the initial branch. This default branch name
2023-01-22T11:56:36.4059925Z hint: is subject to change. To configure the initial branch name to use in all
2023-01-22T11:56:36.4061823Z hint: of your new repositories, which will suppress this warning, call:
2023-01-22T11:56:36.4062677Z hint:
2023-01-22T11:56:36.4063801Z hint: 	git config --global init.defaultBranch <name>
2023-01-22T11:56:36.4064548Z hint:
2023-01-22T11:56:36.4065659Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
2023-01-22T11:56:36.4066690Z hint: 'development'.  The warning is emitted when the GitHub Action does not store the url output. The url output is only available for httpsTrigger functions. Since you are using an event-driven function, the output is not set. As such, the warning is working as intended.


Can't set a custom Icon for WindowCaptionButton

This parameter is never used: https://github.com/leanflutter/window_manager/blob/main/lib/src/widgets/window_caption_button.dart#L30
And if iconName is null, we throw an error: https://github.com/leanflutter/window_manager/blob/main/lib/src/widgets/window_caption_button.dart#L174
I want to add custom button with custom icon, like this:

And if we give a Widget, le color can't be changed on hover, so I propose something like that:
Add a parameter iconData instead icon of type IconData, and do something like that
widget.iconData != null
? Icon(
    color: iconColor,
    widget.iconData,
    size: 16,
  )
: WindowCaptionButtonIcon(
    name: widget.iconName!,
    color: iconColor,
  ),

Edit: and add a verification like this assert(iconData != null || iconName != null)

This widget will be a separate package in a future version. Now, you can copy the code of this widget to your project for customization.


ENH: Add single-sided p-values to t-tests

Reference issue
Part of work being done for gh-12506.
What does this implement/fix?
This enhancement implements the "alternative" parameter in SciPy's t-tests to allow the selection and return of single-sided p-values.
Additional information
•Implementation closely follows the method found in the brunnermunzel test with modifications made to support arrays. •Selection is done by setting the alternative parameter to one of three strings ("two-sided", "less", or "greater") with the default value being "two-sided" for backwards-compatibility.
•This is a draft PR is because tests still need to be written. Before I do so, however, I'd like to verify that everything looks correct.
Edit:
Currently working through some rebase issues, sorry for the duplicate commits.

Let me know if you need some help!


[5.5] Use array_reduce for Arr::collapse

I am not sure why to add an additional overhead here.

@Vendin - have you run a benchmark - is it actually faster?

The current code is already working.


Allow to specify a custom bootstrap action

This PR allows to add a EMR custom bootstrap action.

Hey @danisola! Thanks for this, it looks great. Question: should this be an array rather than a singular one?

Yes, having multiple actions would make it more general. I'll try to get some more time to do it.

Thanks @danisola !

Bringing forwards...

Merged, closing! Thanks so much.


[POC][WIP][distributed kafkachannel] POC for distributed dispatcher with common/consumer

In order to better understand our options for attempting to further consolidate some of the ConsumerGroup logic between the consolidated and distributed Dispatcher implementations, I have created this POC to switch the distributed implementation to use the common/consumer logic.
Proposed Changes

distributed Dispatcher uses the common/consumer KafkaConsumerGroupFactory instead of it's own Sarama ConsumerGroup lifecycle management.
distributed Dispatcher's Handler now implements the common/consumer KafkaConsumerHandler interface instead of the Sarama interface.
Moved / enhanced some consolidated mocks to common/consumer/testing package.
Fixed the distributed unit tests to align with new implementation

TODOs

Minor nit but the common/consumer KafkaConsumerHandler method GetConsumerGroup() function should be renamed GetConsumerGroupId() to better express what it does.  Also, that function exists only for a single log message in the factory so not sure how useful it is.
The common/consumer logic (e.g. SetReady()) expects to be used with the current "Prober" implementation for tracking ConsumerGroup status.  The distributed Dispatcher does not make use of this so there are some loose ends.  Not sure it would be worthwhile to implement support for the "Prober" since it is expected to be replaced with a new control-protocol based implementation in the near future?
Importantly, the error returned from the Handle() function is used in the Prober implementation to set ConsumerGroup status.  While this isn't really an issue in this POC it is something we need to sort out if we move forward with this idea.  The distributed implementation WILL mark the offset of failed messages in order to move on and not block the queue and we wouldn't want an error in sending an event to result in marking the ConsumerGroup status as failed/error.  We could always "swallow" the error inside the Handle() implementation but that doesn't seem correct.  Just a difference to iron out - shouldn't be a huge deal.
I've only performed simple vanilla testing and would need to more closely confirm no message loss on shutdown/restarts etc.

This is currently just a POC for discussion / review...
/hold

/cc @aliok

@travis-minke-sap  Sorry I missed this PR.
I think this is still a valid improvement. I would be happy to see this merged.  Thanks for the work.
I like the direction of the PR, not much to add at the moment.

/test pull-knative-sandbox-eventing-kafka-upgrade-tests

Switching over!
/unhold