probes can put signifcant load on docker and kubelet

from a user:

It's not just the probes (that eat CPU and memory), they are also putting load on kubelet and docker engine as well from what I have observed.

That seems plausible, but we should gather some data.

I tried Kubernetes on Container Linux in Vagrant and then installed Scope from k8s-standalone and microservices-demo.
The result of  top -d 10 is pretty stable:
  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
 6445 root      20   0<PHONE_NUMBER>20  39464 S 14.0  3.6   2:43.76 scope
 1983 root      20   0 1081064  21668   3188 S  7.3  0.5   1:19.81 containerd
 1884 root      20   0 1012252  89556  16876 S  4.3  2.2   1:58.67 kubelet
 1984 root      20   0 1648840  91532  25732 S  3.2  2.3   2:27.37 dockerd
 6797 root      20   0<PHONE_NUMBER>80  20960 S  1.8  3.2   0:32.37 scope
 2553 root      20   0<PHONE_NUMBER>92  54056 S  0.9  4.2   0:20.51 hyperkube
 2616 root      20   0  253184  89808  50008 S  0.6  2.2   0:14.10 hyperkube

strace shows that containerd is constantly reading cgroup files (cpuacct.stat, memory.stat etc). It seems to do that every second for all containers.
After stopping Scope (kubectl destroy -f weavescope.yaml), containerd does not use cpu anymore:
  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
 1884 root      20   0 1012252  89988  16072 S  8.2  2.2   2:42.19 kubelet
 1984 root      20   0 1648840  86944  23492 S  1.3  2.1   2:51.32 dockerd
 2553 root      20   0<PHONE_NUMBER>00  16060 S  1.0  3.3   0:27.18 hyperkube
 2616 root      20   0  253184  57748  15812 S  1.0  1.4   0:18.73 hyperkube
 1575 etcd      20   0  301876  27660   4580 S  0.8  0.7   0:16.44 etcd2
 1059 999       20   0  295924  61456  28384 S  0.6  1.5   0:06.12 mongod
18665 999       20   0  275584  45496  13152 S  0.6  1.1   0:05.92 mongod
25339 999       20   0  299428  64880  28204 S  0.6  1.6   0:06.17 mongod

Docker gives a lot of stats and I don't think Scope uses all of it. But the Docker API does not allow to get stats less often or less detailed stats (see "GET CONTAINER STATS" doc).
We should check if Docker upstream is working on improving its performance, or alternatively see if Scope could get the stats it needs directly from the cgroup filesystem.
I have not seen the kubelet getting more load because of Scope.

The containerd code for stats (cgroups.go) uses crosbymichael/cgroups/prometheus and provides dozen of metrics. There is no granularity to select less details.

CC @fbarl  (we may get rid of the docker stats to show more consistent CPU consumption with processes)

This issue and https://github.com/weaveworks/scope/issues/1544 seem to have different root causes but both in the Docker stats gathering. If we get rid of the docker stats, that could fix both issues.

@crosbymichael mentioned in the referenced issue of containerd that "The 1.0 version of containerd does not use runc events and collects cgroup metrics on its own, in process , which makes this much more light weight."
I see Weaveworks probe using a nearly constant 20% on an 8 core VM with running Rancher / Traefik / Weaveworks ...
CONTAINER ID        NAME                                             CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
f09dadedf215        r-traefik-traefik-6-666e913c                     21.21%              30.39MiB / 7.638GiB   0.39%               0B / 0B             15MB / 47.8MB       39
bde3bdd70554        r-weavescope-weavescope-probe-6-97120dd0         22.89%              55.33MiB / 7.638GiB   0.71%               0B / 0B             506MB / 0B          39
8642aa09bf67        r-nfs-nfs-driver-6-ed02339f                      0.00%               8.277MiB / 7.638GiB   0.11%               0B / 0B             81.5MB / 0B         16
628e76f9cac3        r-traefik-traefik-lb-6-9e5f8701                  1.64%               22MiB / 7.638GiB      0.28%               0B / 0B             26.7MB / 42.9MB     22
9d01044e98db        r-ipsec-ipsec-router-8-a960c7d0                  0.00%               9.738MiB / 7.638GiB   0.12%               0B / 0B             31.9MB / 0B         36
bf5a2edfdaf4        r-ipsec-ipsec-connectivity-check-8-bffae8a1      0.06%               14.2MiB / 7.638GiB    0.18%               0B / 0B             25.1MB / 0B         20
0547a32159a3        r-ipsec-ipsec-8-470ff4cf                         0.00%               88KiB / 7.638GiB      0.00%               0B / 0B             0B / 0B             1
3ae583b933c2        r-healthcheck-healthcheck-7-d0bfa73f             0.33%               13.73MiB / 7.638GiB   0.18%               0B / 0B             80.3MB / 33.7MB     21
eced8a3c7fea        r-network-services-metadata-dns-14-a391430f      0.05%               13.76MiB / 7.638GiB   0.18%               853MB / 40.7GB      44.6MB / 79.7MB     17
459d34296ee5        r-network-services-metadata-14-32e7b65b          1.96%               23.17MiB / 7.638GiB   0.30%               853MB / 40.7GB      66.9MB / 0B         20
dbfc4a2b55fe        r-network-services-network-manager-14-27ade56d   0.37%               13.2MiB / 7.638GiB    0.17%               0B / 0B             69.3MB / 500kB      16
6123ee69119c        r-ipsec-cni-driver-6-fa5e74ba                    0.01%               104KiB / 7.638GiB     0.00%               0B / 0B             0B / 0B             1
edb47ba9dbd5        rancher-agent                                    0.07%               10.04MiB / 7.638GiB   0.13%               0B / 0B             114MB / 613kB       16


Website does not show output.

Hi,
The zzzcode.ai website does not show any output after Submit and waiting for 5 seconds using Firefox or Chrome. Also Login reports a 'redirection error'.
Best regards
Henk van Winkoop
The Netherlands

Hello @ghhvw ,
Could you confirm with me if you are still having the problem? It is currently working fine for me
If you try again a few seconds later, does it work?
I can see in the database that around 1% of requests fail to show correctly (a lot of them due to the application pool recycling, which means the website reset once a day).
I will try to improve some error messages to allow people to understand better what is happening as at this moment, it fail without any error message.
Best Regards,
Jon


Make sure our Okta allowlist policy is correctly configured before Apr. 1

Background
Okta is making some changes to their network infrastructure that require us to make sure our allowlists allow their internal traffic. We have an allowlist in our Okta setup but I don't actually think we're using it to restrict traffic. This ticket is just to make sure we won't be affected by their changes.
Action requested
Make sure that the following Okta update does not affect us:

Notification Overview
Summary: Okta is updating its edge networking, which requires customers to update their allow lists if they use them.
Audience: Any customers using allow lists
Impact Date: April 25th, 2022 new IPs are brought into production rotation.
Product and Service Notification
Okta will be modifying its edge networking through Spring 2022, starting April 25th. These modifications are designed to provide better security and more resilient connectivity for our customers. The changes to our service should occur seamlessly, but allow lists will need to be actively updated to include the new IP addresses being brought into the networking structure.
Information on maintaining allow lists with Okta services is available in our Help Center. If you need any further assistance on implementing the new changes, as always feel free to reach out to our Support Team. If you maintain allow lists and those allow lists are not updated to reflect the new IPs, you will begin experiencing inconsistent connectivity with Okta services beginning on April 25th as we begin transitioning.
If your organization does not use allow lists, no action needs to be taken.
Dates & Impacts
April 4, 2022
New IPs come into rotation for preview cells
April 25, 2022
New IPs come into rotation for production cells
Allow lists using only old IPs may experience intermittent connectivity issues
Required Actions
Any customers that make use of allow lists need to update their lists to include new IPs introduced with the networking change. Information on maintaining allow lists with Okta services is available in our Help Center.
If your organization does not use allow lists, no action needs to be taken.
Additional Support
If you have any questions, please reach out to your Okta representative or open a case with us starting the subject with, “TC: Public IP Updates”

Additional context
The actual updates will go into effect April 25 but the ticket says April 1 to make sure we have plenty of time to prepare if need be.

Verify that we aren't using an allowlist.


copr: fix float64 overflow check in plus/minus real function

Signed-off-by: wshwsh12<EMAIL_ADDRESS>
What problem does this PR solve?
Issue Number: close #xxx
Problem Summary: Port https://github.com/pingcap/tidb/pull/24179 to tikv.
What is changed and how it works?
Proposal: xxx
What's Changed:
Related changes
Check List
Tests

Unit test
Integration test

Side effects

Performance regression

Consumes more CPU
Consumes more MEM

Breaking backward compatibility

Release note
NONE

/cc @skyzh

/cc @lzmhhh123

/merge

/merge

/test

/run-integration-copr-test

/test

/test

/test


Refactor to remove code generation

The current code generation system is less helpful than expected:

The generated files are all very simple, and could simply be copy/pasted instead of generated.
The .graphql files require special decorators, which have a limited syntax.
The versioning strategy (only some command files are versioned, but most are ignored and should be regenerated mid-build) is counterintuitive.
Adding unique features to commands requires ejecting from code generation.

Therefore, we should:

Replace all .graphql files with their generated .ts counterparts.
Remove all code generation scripts & dependencies.

This will remove some warning of backwards-incompatible API changes, so we should also add some automated validation/testing (see #7).


[HoverCard] Open on focus

Continuing this discussion / closes https://github.com/radix-ui/primitives/issues/898
It was a quick change so thought I'd do it while I had a spare minute. Happy to close PR if we don't think this is wise.
My thoughts:

I agree with @joshwcomeau "trigger on focus" is useful
It can be prevented if others feel differently onFocus={event => event.preventDefault()}
I have added escape keypress handling to pass the dismissable success criterion
It seems we should do this to pass the content on hover or focus success criterion which says:

Content which can be triggered via pointer hover should also be able to be triggered by keyboard focus

https://user-images.githubusercontent.com/175330/136620051-da222c47-1c4a-424c-a154-42a715804602.mp4

So do we all agree to add the timer on focus too as discussed here?

I agree with @jjenzz, simply keeping the same delays as for pointers makes the most sense atm. I think that's how this change is currently working anyway.


Undefined property error from proxy when EmbedOne field is not set

BC Break Report

Q
A

BC Break
yes

Version
2.0.0

Summary
When an EmbedOne field is not set in database, it now causes an Undefined property error.
Previous behavior
Previously in 1.2.x the field was simply left null.
Current behavior
An Undefined property is caused by generated proxy __get($name) method.
How to reproduce
For instance, I've got 2 models Lab and Address:
<?php

namespace App\Doctrine;

use Doctrine\ODM\MongoDB\Mapping\Annotations as ODM;

/**
 * @ODM\Document(collection="labs")
 */
class Lab extends BaseModel
{
    /**
     * @ODM\Id
     * @var string
     */
    public $_id;
    /**
     * @ODM\EmbedOne(targetDocument=Address::class)
     * @var Address
     */
    public $address;
}

<?php

namespace App\Doctrine;

use Doctrine\ODM\MongoDB\Mapping\Annotations as ODM;

/** @ODM\EmbeddedDocument */
class Address extends BaseModel
{
    /**
     * @ODM\Field(type="string")
     * @var string
     */
    public $country;
}

Now if I insert a new Lab document without touching the address field, it's also not set in the database as it used to be. However, when I retrieve the document, in the hydrator it's got
    public function hydrate(object $document, array $data, array $hints = array()): array
    {
        /** @EmbedOne */
        if (isset($data['address'])) {
            // omitted
        }
    }

...and in the proxy it's got
    /**
     * @var \Closure|null initializer responsible for generating the wrapped object
     */
    private $initializerc3072 = null;

    /**
     * @var bool tracks initialization status - true while the object is initializing
     */
    private $initializationTrackerce44e = false;

    /**
     * @var bool[] map of public properties of the parent class
     */
    private static $publicPropertiesda3cc = [
        'address' => true,
    ];

    public function __get($name)
    {
        $this->initializerc3072 && ! $this->initializationTrackerce44e && $this->callInitializer728ea('__get', array('name' => $name));

        if (isset(self::$publicPropertiesda3cc[$name])) {
            return $this->$name;  // BREAKS HERE
        }
        // omitted
    }

Since the address field is not set in the hydrator, when the proxy tries to get its value return $this->$name it causes an Undefined property error.
PS: Not sure whether it's relevant but BaseModel is
<?php

namespace App\Doctrine;

use Carbon\Carbon;
use MyCLabs\Enum\Enum;

class BaseModel
{
    protected $excludedFromArray = ['_id', 'lazyPropertiesDefaults'];
    private $preserveOriginalName = false;

    public function __construct(array $data = null)
    {
        if (!empty($data)) {
            foreach ($data as $key => $value) {
                $this->$key = $value;
            }
        }
    }

    public function __get($name)
    {
        $accessor = 'get_' . $this->snakeCase($name);
        if (method_exists($this, $accessor)) {
            return $this->$accessor();
        }
        return $this->$name ?? null;
    }

    public function __set($name, $value)
    {
        $mutator = 'set_' . $this->snakeCase($name);
        if (method_exists($this, $mutator)) {
            // an additional bool parameter ($toRemove = false) is provided
            $this->$mutator($value, false);
        } else {
            $this->$name = $value;
        }
    }

    public function __unset($name)
    {
        $this->__set($name, null);
    }

    public function __isset($name)
    {
        return $this->__get($name) !== null;
    }

    public function __clone()
    {
        foreach (get_object_vars($this) as $key => $value) {
            $this->$key = $this->cloneRecursively($value);
        }
    }

    /**
     * @return array
     */
    public function getExcludedFromArray(): array
    {
        return $this->excludedFromArray;
    }

    /**
     * @param array $excludedFromArray
     */
    public function setExcludedFromArray(array $excludedFromArray): void
    {
        $this->excludedFromArray = $excludedFromArray;
    }

    /**
     * @param mixed $item
     * @return mixed
     */
    private function cloneRecursively($item)
    {
        if (is_object($item)) {
            return clone $item;
        }
        if (is_array($item)) {
            return array_map([self::class, 'cloneRecursively'], $item);
        }
        return $item;
    }

    public function toArray(): array
    {
        $reflect = new \ReflectionClass($this);
        $result = [];
        foreach ($reflect->getProperties(\ReflectionProperty::IS_PUBLIC) as $key) {
            $key = $key->getName();
            if (strpos($key, '__') === 0
                || in_array($key, $this->getExcludedFromArray(), true)
                || method_exists($this, 'get_' . $this->snakeCase($key))) {
                continue;
            }
            $result[$key] = $this->toArrayRecursively($this->$key);
        }
        foreach ($reflect->getMethods() as $method) {
            $method = $method->getName();
            if (preg_match('/^get_(.+)$/', $method, $matches)) {
                $key = $matches[1];
                if (strpos($key, '__') === 0 || in_array($key, $this->getExcludedFromArray(), true)) {
                    continue;
                }
                $result[$key] = $this->toArrayRecursively($this->{$matches[0]}());
            }
        }
        return $result;
    }

    /**
     * @param mixed $item
     * @return mixed
     */
    private function toArrayRecursively($item)
    {
        if ($item instanceof Carbon) {
            return $item->format('c');
        }
        if ($item instanceof \DateTimeInterface) {
            return Carbon::instance($item)->format('c');
        }
        if ($item instanceof Enum) {
            return $item->getValue();
        }
        if ($item instanceof \IteratorAggregate) {
            $item = $item->getIterator();
        }
        if ($item instanceof \Iterator) {
            $item = iterator_to_array($item);
        }
        if (is_object($item) && method_exists($item, 'toArray')) {
            return $item->toArray();
        }
        if (is_array($item)) {
            return array_map([self::class, 'toArrayRecursively'], $item);
        }
        return $item;
    }

    protected function snakeCase($name): string
    {
        if ($this->preserveOriginalName) {
            return $name;
        }
        return strtolower(preg_replace('/[^A-Z0-9]+/i', '_', $name));
    }
}

@Frederick888 could you please report an issue to https://github.com/Ocramius/ProxyManager as it seems to be a shortcoming in the proxying mechanism itself? For the time being you may consider using a getter method that should work just fine as it won't utilize magic __get under the hood.

@Frederick888 out of curiosity, does the error appear when changing property visibility to protected or private? Not saying you should change the code, but I remember there were some special things about public properties in proxies.

@malarzm

report an issue to https://github.com/Ocramius/ProxyManager

No problem. But honestly I've got no idea what OOP proxies are... What would be an appropriate title for the issue there?

you may consider using a getter method

You meant a getter in my own model? It's already got one in BaseModel:
    public function __get($name)
    {
        $accessor = 'get_' . $this->snakeCase($name);
        if (method_exists($this, $accessor)) {
            return $this->$accessor();
        }
        return $this->$name ?? null;
    }

@alcaeus

changing property visibility to protected or private

This time the undefined address error is gone but instead I got undefined excludedFromArray (the toArray filter I've got in BaseModel). I guess it's because the property is unset in the proxy?
    public static function staticProxyConstructor($initializer)
    {
        static $reflection;

        $reflection = $reflection ?? $reflection = new \ReflectionClass(__CLASS__);
        $instance = $reflection->newInstanceWithoutConstructor();

        unset($instance->address, $instance->excludedFromArray);

        \Closure::bind(function (\App\Doctrine\BaseModel $instance) {
            unset($instance->preserveOriginalName);
        }, $instance, 'App\\Doctrine\\BaseModel')->__invoke($instance);

        $instance->initializerc3072 = $initializer;

        return $instance;
    }

@Frederick888 something like "Undefined property error for public property and __get" should do. Also you can link to this issue, but I'm sure @Ocramius will either know what to do or what questions to ask :)

Closing as #2082 was merged


Added dedicated webpack entry point.

I'm using Bluebird 3.4.7 with webpack 1 and noticed it was clobbering the global variable P. I could see the snippet at the end of the script that it puts itself in the global namespace.
                       ;if (typeof window !== 'undefined' && window !== null) {
                             window.P = window.Promise;
                        } else if (typeof self !== 'undefined' && self !== null) {
                             self.P = self.Promise;
                        }

I investigated a bit and found out that webpack will use the "browser" field from package.json if it's available. From the webpack docs:

Check these fields in the package.json for suitable files.
Default: ["webpack", "browser", "web", "browserify", ["jam", "main"], "main"]

This means that webpack picks the "browser" version instead of "main". Running webpack --display-modules confirmed it:
[5] ./~/bluebird/js/browser/bluebird.js 178 kB {0} [built]

Since Blubird's installation docs do not mention it, I think it's expected to include release/bluebird.js from the main field.

I was able to work around this with an alias in webpack.config.js:
alias: {
  'bluebird': 'bluebird/js/release/bluebird'
}

Perhaps you'd like to change the installation docs instead?

For those confused why it doesn't work in webpack 2, default mainFields are different in it
mainFields: ["browser", "module", "main"]

In order to make this work you need to specify them manually
mainFields: ["webpack", "browser", "module", "main"]

An alternative(which I was using prior to this PR) would be to use an alias for bluebird:
resolve: {
  alias: {
    bluebird: path.join(__dirname, './node_modules/bluebird/js/release/bluebird.js'),
  }
}

P.S.: Now you can also decrease bundle size by stripping unneeded modules from it like so:
    new webpack.NormalModuleReplacementPlugin(  /\/node_modules\/bluebird\/js\/release\/(?:assert|each|settle|using|any|race|call_get|filter|generators|nodeify|promisify|map|reduce|settle|some|progress)(?:\.js)?$/,
      path.join(__dirname, './node_modules/lodash/noop.js')
    ),


fail to mark edge-node

What happened:
The edge-node is not marked.
[root@k8smaster bin]# ./yurtctl convert --provider kubeadm --cloud-nodes k8smaster

I1125 10:47:21.752654   17882 convert.go:318] mark k8smaster as the cloud-node
I1125 10:48:01.793163   17882 util.go:542] servant job(yurtctl-disable-node-controller-k8smaster) has succeeded
I1125 10:48:01.793210   17882 convert.go:343] complete disabling node-controller
I1125 10:48:01.794493   17882 convert.go:443] kube-public/cluster-info configmap already exists, skip to prepare it
I1125 10:48:01.805102   17882 convert.go:408] deploying the yurt-hub and resetting the kubelet service on edge nodes...
E1125 10:48:01.808416   17882 util.go:539] fail to run servant job(yurtctl-servant-convert-k8snode2): jobs.batch "yurtctl-servant-convert-k8snode2" already exists
E1125 10:48:01.808802   17882 util.go:539] fail to run servant job(yurtctl-servant-convert-k8snode1): jobs.batch "yurtctl-servant-convert-k8snode1" already exists
I1125 10:48:01.808812   17882 convert.go:414] complete deploying yurt-hub on edge nodes
I1125 10:48:01.808817   17882 convert.go:417] deploying the yurt-hub and resetting the kubelet service on cloud nodes
E1125 10:48:01.811134   17882 util.go:539] fail to run servant job(yurtctl-servant-convert-k8smaster): jobs.batch "yurtctl-servant-convert-k8smaster" already exists
I1125 10:48:01.811145   17882 convert.go:423] complete deploying yurt-hub on cloud nodes

[root@k8smaster bin]# ./yurtctl markautonomous
W1125 10:57:39.404158   20585 markautonomous.go:118] there is no edge nodes, please label the edge node first

In log,it is showed that
11月 25 10:57:51 k8smaster kubelet[1660]: E1125 10:22:51.583366    1660 pod_workers.go:191] Error syncing pod 740f5710-694d-4c13-b251-1d8797403b17 ("yurtctl-servant-revert-k8smaster-4pgh6_kube-system(740f5710-694d-4c13-b251-1d8797403b17)"), skipping: failed to "StartContainer" for "yurtctl-servant" with CrashLoopBackOff: "back-off 2m40s restarting failed container=yurtctl-servant pod=yurtctl-servant-revert-k8smaster-4pgh6_kube-system(740f5710-694d-4c13-b251-1d8797403b17)"

What you expected to happen:
I want to see the result:
'mark k8smaster as the edge-node'
and "markautonomous" is successful.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:

OpenYurt version: v0.5.0
Kubernetes version (use kubectl version): v1.18.0
OS (e.g: cat /etc/os-release): centos7
Kernel (e.g. uname -a):
Install tools:
Others:

others
/kind bug

@GitHubThinking Thank you for raising issue.
/assign @adamzhoul @Peeknut

E1125 10:48:01.808416   17882 util.go:539] fail to run servant job(yurtctl-servant-convert-k8snode2): jobs.batch "yurtctl-servant-convert-k8snode2" already exists
E1125 10:48:01.808802   17882 util.go:539] fail to run servant job(yurtctl-servant-convert-k8snode1): jobs.batch "yurtctl-servant-convert-k8snode1" already exists
E1125 10:48:01.811134   17882 util.go:539] fail to run servant job(yurtctl-servant-convert-k8smaster): jobs.batch "yurtctl-servant-convert-k8smaster" already exists

maybe the old job is stopping yurtctl convert label the node.
please delete the job, and yurtctl reset then yurtctl convert to do all those again.
issue alike #572

E1125 10:48:01.808416   17882 util.go:539] fail to run servant job(yurtctl-servant-convert-k8snode2): jobs.batch "yurtctl-servant-convert-k8snode2" already exists
E1125 10:48:01.808802   17882 util.go:539] fail to run servant job(yurtctl-servant-convert-k8snode1): jobs.batch "yurtctl-servant-convert-k8snode1" already exists
E1125 10:48:01.811134   17882 util.go:539] fail to run servant job(yurtctl-servant-convert-k8smaster): jobs.batch "yurtctl-servant-convert-k8smaster" already exists

maybe the old job is stopping yurtctl convert label the node. please delete the job, and yurtctl reset then yurtctl convert to do all those again.
issue alike #572

Please tell me how to delete the old job, I'm a beginner and hope you can instruct me step by step.

You can get the job in ns: kube-system, and you will see the convert jobs.

kubectl get job -n kube-system -A

then you can delete the specified job by its namespace and name with

kubectl delete job -n < ns> < job-name>

here you can run:

kubectl delete job -n kube-system yurtctl-servant-convert-k8snode2
kubectl delete job -n kube-system yurtctl-servant-convert-k8snode1
kubectl delete job -n kube-system yurtctl-servant-convert-k8smaster


Update pypi version

Version in pypi doesn't work with newest AlphaFold structure

Hi @mooniean Just to let you know that I've added a conda package here on conda-forge
https://github.com/conda-forge/staged-recipes/pull/22855
I added both of us as maintainers of the conda package and apparently as part of this check list to be included in conda-forge we need to add a comment saying we are happy to be the maintainers. If you're happy to be listed, can you add a comment under mine?


Caddy: Token '0' doesn't match specifier '%s'

Seeing this issue when parsing a new log file generated by Caddy v2.7.6 (latest). Older log files seem fine.
$ goaccess /tmp/test.log --log-format=CADDY
Cleaning up resources...
==2779746== GoAccess - version 1.9.1 - Feb  6 2024 01:37:49
==2779746== Config file: /etc/goaccess/goaccess.conf
==2779746== https://goaccess.io -<EMAIL_ADDRESS>==2779746== Released under the MIT License.
==2779746==
==2779746== FILE: /tmp/test.log
==2779746== Parsed 1 lines producing the following errors:
==2779746==
==2779746== Token '0' doesn't match specifier '%s'
==2779746==
==2779746== Format Errors - Verify your log/date/time format

In the example above I used an excerpt from my real log file. The excerpt contained one entry, here it is formatted and redacted:
{
  "level": "info",
  "ts":<PHONE_NUMBER>.3412683,
  "logger": "http.log.access.log0",
  "msg": "handled request",
  "request": {
    "remote_ip": "<IP_ADDRESS>",
    "remote_port": "33910",
    "client_ip": "<IP_ADDRESS>",
    "proto": "HTTP/2.0",
    "method": "GET",
    "host": "example.com",
    "uri": "/",
    "headers": {
      "Accept-Encoding": [
        "gzip, deflate, br"
      ],
      "Sec-Gpc": [
        "1"
      ],
      "Upgrade-Insecure-Requests": [
        "1"
      ],
      "Sec-Fetch-Mode": [
        "navigate"
      ],
      "Sec-Fetch-User": [
        "?1"
      ],
      "Te": [
        "trailers"
      ],
      "User-Agent": [
        "Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0"
      ],
      "Accept": [
        "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"
      ],
      "Accept-Language": [
        "en-CA,en-US;q=0.7,en;q=0.3"
      ],
      "Dnt": [
        "1"
      ],
      "Sec-Fetch-Dest": [
        "document"
      ],
      "Sec-Fetch-Site": [
        "none"
      ]
    },
    "tls": {
      "resumed": false,
      "version": 772,
      "cipher_suite": 4865,
      "proto": "h2",
      "server_name": "example.com"
    }
  },
  "bytes_read": 0,
  "user_id": "",
  "duration": 0.000041019,
  "size": 0,
  "status": 0,
  "resp_headers": {
    "Strict-Transport-Security": [
      "max-age=63072000"
    ],
    "Cache-Control": [
      "max-age=600"
    ],
    "Server": [
      "Caddy"
    ],
    "Alt-Svc": [
      "h3=\":443\"; ma=2592000"
    ]
  }
}

Do you have any insight into why you might be receiving a status of 0? GoAccess does verify HTTP status codes, such as 404 or 500, for validity. If necessary, you can use --no-strict-status to bypass this validation.

I've added extra non-standard HTTP status codes, including 0, for Caddy. Please try building from development and let me know if it resolves the issue on your side. I'll incorporate these changes into the upcoming release. Thank you!


Error adding entities for domain lock with platform kwikset

On Home Assistant 2023.8.0 (Running the OS version with Supervisor 2023.07.1, Operating System 10.3, Frontend 20230802.0 - latest).
Upon HA Core update, the integration stopped working, re-adding the lock (deleting the Kwikset integration entry from the Devices & Services section, re-adding the lock again, entering the credentials) is now broken, the credentials are accepted and the "home" is added and showing but locks are not.
Same error showing three times in the logs, one for each of the three "integrations": Lock, Sensor & Switch.

Logger: homeassistant.components.lock
Source: helpers/entity_platform.py:619
Integration: Lock (documentation, issues)
First occurred: 19:02:40 (2 occurrences)
Last logged: 19:02:40
Error adding entities for domain lock with platform kwikset
Error while setting up kwikset platform for lock
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 510, in async_add_entities
await asyncio.gather(*tasks)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 619, in _async_add_entity
device = dev_reg.async_get(self.hass).async_get_or_create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: DeviceRegistry.async_get_or_create() got an unexpected keyword argument 'serial'

I'm having the same issue.  Got the Refresh Token has been revoked error.  Deleted and tried to re-add, but doesn't load my devices.  Goes through most of the same steps, but doesn't come up with the devices that allows you to add to the area.  Says created configuration, but no devices.
I just updated with the latest Home Assistant Core.
Home Assistant 2023.8.0
Supervisor 2023.07.1
Operating System 10.3
Frontend 20230802.0 - latest
Error Log details
Logger: homeassistant.components.switch
Source: helpers/entity_platform.py:619
Integration: Switch (documentation, issues)
First occurred: 9:26:16 PM (10 occurrences)
Last logged: 9:32:52 PM
Error adding entities for domain switch with platform kwikset
Error while setting up kwikset platform for switch
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 510, in async_add_entities
await asyncio.gather(*tasks)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 619, in _async_add_entity
device = dev_reg.async_get(self.hass).async_get_or_create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: DeviceRegistry.async_get_or_create() got an unexpected keyword argument 'serial'

Update on my above details.  I took a snapshot before the HASS core update.  On a whim I restored the snapshot before the update.  I deleted and re-added the Kwikset integration.  Low and behold it worked as it has in the past.  So, I believe the HASS Core version 2023.8.0 broke something.  Going to hold tight on HASS Core 2023.7.3, which still works.  Hope this helps or will wait for the next HASS Core update and see if HASS fixes whatever it introduced.


Persistent config

A proposed solution for #633.
It provides a config backend that can be used for persisting and reading livebook configuration. It does not break any API (all existing tests pass) and allows to freely swap to other config backends, currently one reading from elixir's config variables and the other storing encrypted information in a file.
This is rather a POC rather than a proper solution so keep that in mind when you see hard coded secret 👀
Any feedback  appreciated.

Hey @Qizot, thanks for kicking this off! Some high-level thoughts below.
I think we should always use the application environment for accessing the configuration at runtime. The values we keep there are generally parsed into Elixir values, for example in case of file systems or the default runtime, we shouldn't dump the structs as-is, because it would break as soon as we change some of the internals.
One direction that we could take this is to make the backend a "primary store" (file, maybe database, or none), so then when setting the initial application config we would read what we have in the store and use a precedence like stored value > cli flags > env vars. Then, whenever there's a config change we just change application config, and if that's something we want to persist, we serialize the value and defer storage to the backend. This way we explicitly control what we load/store. The only tricky bit may be ensuring that store takes precedence over CLI flags (which currently override the config after initialization), but that's just a technical detail.
@josevalim please let me know if this makes sense, or if you have any other take on this :)

Thank you @Qizot for the PR and sorry for the delay when replying. I was thinking about this problem and, because I could not answer all questions, it took me too long. So I decided to write my thoughts down here.
About the persistence, it is likely that we want to store more information in the future besides the configuration. For example, maybe we will allow users to be stored somewhere. Or perhaps configuration to third party nodes, etc.
The other issue is the conflict between flags/env vars and configuration. I think it would be best if we don't mix those altogether: or something is a configuration/env vars or it is something that we fully put in storage, there should be no overlap. In other words, we should decouple the solution from configuration and treat it as a complete separate thing. For example, once we add this feature, there is no environment var for configuring file systems.
I still need to digest a bit more where we want to go, but I thought I would give you some high-level updates.

Ok, I have thought more about this. One of the issues with the format that we choose, is how are we going to migrate the data. For example, we could perhaps use Ecto. We would define the schemas and we could use Etso to persist it in memory/disk and use PostgreSQL if you want to share it across servers. The issue, however, is that, if we add new data to the schema, then we need to migrate PostgreSQL and we need to migrate the structure in memory/disk too.
My conclusion is that, in our case, the data is more important than a particular shape. Therefore, I propose for us to store data using a variant of the Entity-Attribute-Value model. We will have two additional columns: a namespace and the insertion timestamp. For example, to store the user name, we would do:

Namespace
Entity
Attribute
Value
Timestamp

:users
1
:name
"josé"
1643371831822

:users
1
:job
"dev"
1643371831822

Our initial implementation (V1) could store this in ETS like this:
{{namespace, entity}, attribute, value, timestamp}

The timestamp is computed with System.os_time(:millisecond). For now, we can store all data in a single ETS table. For example, to store both the autosave path and filesystem information, we would have:

Namespace
Entity
Attribute
Value
Timestamp

:settings
:global
:autosave_path
"/foo/bar"
1643371831822

:filesystem
rand()
:type
s3
1643371832489

:filesystem
rand()
:bucket_url
"/..."
1643371832489

:filesystem
rand()
:access_key_id
"xyz"
1643371832489

:filesystem
rand()
:secret
"abc"
1643371832489

Where rand() in this case could be :crypto.strong_rand_bytes(6) |> Base.url_encode64() that is generated when a filesystem is added.
For V1, we can focus simply on replacing the filesystem part. We can remove the environment variables and introduce something called Livebook.Storage. Here is the behaviour:
@doc """
Returns a map identified by `entity_id` in `namespace`.

    fetch(:filesystem, "rand-id")
    #=> {:ok, %{id: "rand-id", type: "s3", bucket_url: "/...", secret: "abc", access_key: "xyz"}}

"""
@callback fetch(namespace, entity_id)

@doc """
Returns all values in namespace.

    all(:filesystem)
    [%{id: "rand-id", type: "s3", bucket_url: "/...", secret: "abc", access_key: "xyz"}]

"""
@callback all(namespace)

@doc ...
@callback insert(namespace, entity_id, [{attribute, value}])

@doc ...
@callback delete(namespace, entity_id)

Both insert and delete go though a GenServer. insert performs both inserts and updates. If any particular attribute already exists, it must be replaced. We would only implement the storage for now, without backing it up to filesystem, which would come next.
WDYT? If you agree, do you want to implement this?

@josevalim I really like the idea! I can sit down to it during this weekend or during the following week :)

@josevalim I've implemented the ETS versions of the spec you suggested. Any ideas how to integrate it with the livebook itself (regarding the filesystems configs I can simply remove just the initial configuration and all the other add/remove should work out of the box)?

Any ideas how to integrate it with the livebook itself (regarding the filesystems configs I can simply remove just the initial configuration and all the other add/remove should work out of the box)?

Yes, exactly! Just assume it will be stored in storage. :)

@josevalim any ideas how to cleanly serialize/deserialize different types of filesystem back and forth from storage? My idea was to serialize and serialize the module and make the FileSystem protocol define some functions for serialization/deserialization but it can get tricky once there are things such pids for e.g. local filesystem (would cause trouble if we want further persist that information).

Do we need to deserialize them? The filesystems are created from the configuration, which today is a string, and we will change said configuration to be a map. I don't think we need to go from a filesystem instance to a setting, do we?

I probably have problems understanding how to current flow of managing file systems should look like so correct me if I'm wrong.
Each added file system should have a generated id and it should probably be carried around so that we can easily
remove the file system from storage based on that id.
Moreover, if we want to make the storage a single source of truth we have to keep all the information about the file systems there, meaning that for each Livebook.Config.file_systems() call we need to fetch the information from storage and convert that to the necessary struct such as Livebook.FileSystem.S3 so that the FileSystem protocol works on them.
My problem was that as far as this approach is fine for S3, then for Local file system this is no so great as it is carrying some pid (meaning that the information will be persisted even if we don't want it to).

Livebook.Config should be completely out of the equation. Also I don't think you should treat Livebook.Storage as the source of filesystems. You can assume that the filesystems will be something like this:
@default_fs [Livebook.Filesystem.Local]

def filesystems do
  @default_fs ++
    Enum.map(Livebook.Storage.all(:filesystems), &storage_to_fs/1)
end

And then storage_to_fs is something like:
defp storage_to_fs(%{type: "s3"} = config), do: Livebook.Filesystem.S3.from_config(config)

I am not sure if those are the modules but the important point I want to bring your attention to is that you should not be expecting the data to come ready from the storage. You will use the storage as a mechanism to "hydrate"/create the actual data. This is true in many cases but especially true in our case since we created a very dumb storage layer (which has pros in our case!). :)

I have added a few extra comments. We should fully move away from config but that can be done in another PR. It is your call. :)

I would go for moving all the filesystem configuration away from Config and adjusting it to using ids to be in a separate PR so there are not that many changes.

@Qizot we just need to make the formatter happy and we are good to go!

Whops, some error came in, will fix it in a moment

:green_heart: :blue_heart: :purple_heart: :yellow_heart: :heart:

@Qizot fantastic job! :shipit:


fixed the notification tune issue

https://user-images.githubusercontent.com/99071072/220168575-cdd89a0c-185c-487c-a77d-89e42aec2291.mp4

this is issue #32


No dynamic allocations in Decimal

What have you changed? (mandatory)
This PR removes the dynamic allocation in Decimal. Previously we did that to reduce the struct size of Decimal, however it is rarely useful since:

In non-batch executors, the unit of single operation is Datum, whose size is already large enough due to https://github.com/chronotope/chrono-tz/issues/27 so that reducing Decimal size does not have effect.

In batch executors, the unit of single operation is Vec, whose size does not depend on the size of Decimal.

What are the type of the changes? (mandatory)

Improvement (change which is an improvement to an existing feature)

How has this PR been tested? (mandatory)
No need.
Benchmark result if necessary (optional)
TPC-H Q1 improved by 20%

/run-integration-tests

Could you edit the pr description stating that Decimal is rarely moved, so we are inclined to save the allocatio instead of controlling the struct size?

@sticnarf Done.

@breeswish
can you paste the detailed benchmark result?

@siddontang TPC-H Q1 (Scale Factor = 10) improved from 15s to 12s. Other queries are not tested since this outcome is effective enough.


#208 Upgrade CCM Dependencies, Jenkins, & Jenkins Plugin Versions

Description Of Changes
Go through and update the following things:

Jenkins to V 2.414.1
dotnet-aspnetcoremodule to V 16.0.23196
dotnet-6.0-runtime to V 6.0.21
dotnet-6.0-aspnetruntime to V 6.0.21
All Jenkins plugins to latest as of 08-25-2023

Motivation and Context
Go through and update Jenkins, Jenkins Plugins, and Chocolatey Central Management dependency package versions to latest available.
Testing

Built on Local VM

Operating Systems Testing

Windows Server 2022

Change Types Made

[ ] Bug fix (non-breaking change).
[ ] Feature / Enhancement (non-breaking change).
[X] Breaking change (fix or feature that could cause existing functionality to change).
[ ] Documentation changes.
[X] PowerShell code changes.

Change Checklist

[ ] Requires a change to the documentation.
[ ] Documentation has been updated.
[ ] Tests to cover my changes, have been added.
[X] All new and existing tests passed?
[ ] PowerShell code changes: PowerShell v2 compatibility checked?

Related Issue
Fixes #208

Still need to go through and remove test for powershell Jenkins plugin and add tests for all other plugins still in list.

Passed my internal testing. Ready for review!


Allow argsHash to be nil when testing. Use dummy value for compilerVersion when testing.

Although both are transmitted through the build record, they are different:

argsHash for the current run can be determined, but not easily consed up in a test
compilerVersion cannot be determined (when the frontend path is set to Python for testing), but the tests have no trouble finding one.

So, just as the legacy driver does, allow the "options" field to be omitted from the incoming build record without triggering a mismatch.
And when testing, let the driver set the compilerVersion to a dummy value, which can be recognized when testing.

@swift-ci please test


Tracing event threaded through pipeline

Creates a  new didTrace event with variable-level logging that propagates debugging events back to the client. This approach also resulted in additional advantageous simplifications in class/protocol structure.

 Spokestack iOS enable tracing levels

lgtm


More details about props

Subject
component props
Description
I'm having a difficult time understanding what values to pass in for many of the props.
For instance with Grid I don't see anywhere where there is a list of values for templateColumns. In the example I see repeat(3, 1fr) but no description of what that means.
For IconButton there is color scheme, which I guess the only way to override is to extend the theme or use a complicated box to customize. I don't see anywhere where is tells you 500 is background, 600 is hover, 700, is while clicked.
It's difficult to get started and convert components without more details of how to use chakra. Am I missing docs somewhere?

Hey @j-castellanos! Great to see your interest in using the package.
For each of the component pages, you should find a tab at the top of the page called Props (assuming that you are viewing the site for the most latest version of Chakra, V2.
For the Grid component, you will find the templateColumns component there.
I would keep in mind that even though the docs try to be as explicit as necessary, it's possible that you might not see detail for all props because some used in the example are not specific to a component. For example templateColumns is a common style property for CSS Grid as not a name specific to Chakra.
You should view the Props tab for Grid here: https://chakra-ui.com/docs/components/grid/props
I hope this helps! 😄

@TylerAPfledderer ah got it, I haven't needed to use normal css in a long time so wasn't aware of the grid-template-columns css syntax. It would be nice though it the props that are described as "shorthand for some-prop" link to that prop and that prop links to the documentation for the regular css. So when viewing Grid you see templateColumns which links to gridTemplateColumns which links to https://developer.mozilla.org/en-US/docs/Web/CSS/grid-template-columns

Closing issue, as the documentation provides as much detail as necessary to describe shorthand props in question.
Linking further details about a prop to an outside source is outside the scope of the package, unless the team would favor providing an external source for every single CSS prop, shorthand or otherwise. That is arguably excessive as basic knowledge of CSS is expected when using Chakra UI or any other package that involves CSS.


Date time second precision

*Added Second precision to time-input component
*Changed TimeInput and DateTimePicker to conditionally render seconds
*Updated Documentation
*Updated snapshots, added tests, added translations

Thank you for the contribution but looks like this PR is a duplicate of #722.

@Blackop778 Can you go through this PR and verify the changes made here have also been made in your PR?
I do think I like the name hasSeconds prop better than showSeconds. Not sure what others think

@emilyrohrbough I compared the changes in this PR to my PR and transfered a couple changes in 1290d5c30a07d9d2a19ebe42d417d8b2b282cb7b


Problem training for new language.

I am trying to train my model on a Marathi dataset. It is strange espeak doesnt seem to support it although mentioned in their Supported Languages Page.
In [3]: ph = to_phonemes("प्रदर्शनों के दौरान पुलिस की हिंसा और चुनावों में कथित धोखाधड़ी के ख़िलाफ़ बेलारूस में लोगों का गुस्सा बढ़ता ही जा रहा है")
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-3-944424e45249> in <module>
----> 1 ph = to_phonemes("प्रदर्शनों के दौरान पुलिस की हिंसा और चुनावों में कथित धोखाधड़ी के ख़िलाफ़ बेलारूस में लोगों का गुस्सा बढ़ता ही जा रहा है")

<ipython-input-2-a226f9833051> in to_phonemes(text)
      9                          njobs=1,
     10                          punctuation_marks=';:,.!?¡¿—…"«»“”()',
---> 11                          language_switch='remove-flags')
     12     phonemes = phonemes.replace('—', '-')
     13     return phonemes

~/.virtualenvs/forwardtacoenv/lib/python3.6/site-packages/phonemizer/phonemize.py in phonemize(text, language, backend, separator, strip, preserve_punctuation, punctuation_marks, with_stress, language_switch, njobs, logger)
    158             with_stress=with_stress,
    159             language_switch=language_switch,
--> 160             logger=logger)
    161     elif backend == 'espeak-mbrola':
    162         phonemizer = backends[backend](

~/.virtualenvs/forwardtacoenv/lib/python3.6/site-packages/phonemizer/backend/espeak.py in __init__(self, language, punctuation_marks, preserve_punctuation, language_switch, with_stress, logger)
    145         super().__init__(
    146             language, punctuation_marks=punctuation_marks,
--> 147             preserve_punctuation=preserve_punctuation, logger=logger)
    148         self.logger.debug('espeak is %s', self.espeak_path())
    149

~/.virtualenvs/forwardtacoenv/lib/python3.6/site-packages/phonemizer/backend/base.py in __init__(self, language, punctuation_marks, preserve_punctuation, logger)
     52             raise RuntimeError(
     53                 'language "{}" is not supported by the {} backend'
---> 54                 .format(language, self.name()))
     55         self.language = language
     56

RuntimeError: language "mr" is not supported by the espeak backend

Now my only solution is to run it directly on grapheme. There is very less difference between the grapheme and phoneme for indic script. So I made the following changes.
cleaners.py
def to_phonemes(text):
    # text = text.replace('-', '—')
    # phonemes = phonemize(text,
    #                      language=hp.language,
    #                      backend='espeak',
    #                      strip=True,
    #                      preserve_punctuation=True,
    #                      with_stress=False,
    #                      njobs=1,
    #                      punctuation_marks=';:,.!?¡¿—…"«»“”()',
    #                      language_switch='remove-flags')
    # phonemes = phonemes.replace('—', '-')
    phonemes = text
    return phonemes

symbols.py
""" from https://github.com/keithito/tacotron """

"""
Defines the set of symbols used in text input to the model.

The default is a set of ASCII characters that works well for English or text that has been run through Unidecode. For other data, you can modify _characters. See TRAINING_DATA.md for details. """
from utils.text import cmudict

_pad = "_"
_punctuation = "!'(),.:;? "
_special = "-"

# Prepend "@" to ARPAbet symbols to ensure uniqueness (some are the same as uppercase letters):
_arpabet = ["@" + s for s in cmudict.valid_symbols]

# Phonemes
# _vowels = 'iyɨʉɯuɪʏʊeøɘəɵɤoɛœɜɞʌɔæɐaɶɑɒᵻ'
# _non_pulmonic_consonants = 'ʘɓǀɗǃʄǂɠǁʛ'
# _pulmonic_consonants = 'pbtdʈɖcɟkɡqɢʔɴŋɲɳnɱmʙrʀⱱɾɽɸβfvθðszʃʒʂʐçʝxɣχʁħʕhɦɬɮʋɹɻjɰlɭʎʟ'
# _suprasegmentals = 'ˈˌːˑ'
# _other_symbols = 'ʍwɥʜʢʡɕʑɺɧ'
# _diacrilics = 'ɚ˞ɫ'

_phones = "ँंःअआइईउऊऋऌऍऎएऐऑऒओऔकखगघङचछजझञटठडढणतथदधनऩपफबभमयरऱलळऴवशषसह़ऽािीुूृॄॅॆेैॉॊोौ्ॐक़ख़ग़ज़ड़ढ़फ़य़ॠ॰ॲ"

phonemes = sorted(
    list(
        _pad
        + _punctuation
        + _special
        + _phones
        # + _non_pulmonic_consonants
        # + _pulmonic_consonants
        # + _suprasegmentals
        # + _other_symbols
        # + _diacrilics
    )
)

When I run python preprocess.py --path /home/ubuntu/datasets/Marathi_trim/ i get this error
/home/ubuntu/.virtualenvs/forwardtacoenv/lib/python3.6/site-packages/librosa/util/decorators.py:9: NumbaDeprecationWarning: An import was requested from a module that has moved location.
Import of 'jit' requested from: 'numba.decorators', please update to use 'numba.core.decorators' or pin to Numba version 0.48.0. This alias will not be present in Numba version 0.50.0.
  from numba.decorators import jit as optional_jit

35999 wav files found in "/home/ubuntu/datasets/Marathi_trim/"

+-------------+-----------+--------+------------+-----------+----------------+
| Sample Rate | Bit Depth | Mu Law | Hop Length | CPU Usage | Num Validation |
+-------------+-----------+--------+------------+-----------+----------------+
|    22050    |     9     |  True  |    256     |  103/104  |      200       |
+-------------+-----------+--------+------------+-----------+----------------+

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "preprocess.py", line 56, in process_wav
    m, x = convert_file(path)
  File "preprocess.py", line 42, in convert_file
    peak = np.abs(y).max()
  File "/home/ubuntu/.virtualenvs/forwardtacoenv/lib/python3.6/site-packages/numpy/core/_methods.py", line 39, in _amax
    return umr_maximum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation maximum which has no identity
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "preprocess.py", line 91, in <module>
    for i, (item_id, length, cleaned_text) in enumerate(pool.imap_unordered(process_wav, wav_files), 1):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 735, in next
    raise value
ValueError: zero-size array to reduction operation maximum which has no identity

Am I making a mistake somewhere?
Thanks in advance!

Hi, it seems that its loading an empty wav file and breaks when trying to do perform a np.max() operation, unrelated to the text.

My trim_trailing_leading_silence script was the culprit for this. I will check the durations for the original audio and keep you updated.
Would there be a performance decrease with this workaround?
Also, Does forward tacotron support multi-language tts assuming I use espeak-ng to generate phones? (Marathi + hindi), (English + hindi)

I found that the pronounciation is more stable with phonemes, but it should work pretty ok with chars too. Multi-language would work I guess, but requires you to preprocess the datasets separately with the corresponding languages (or char-wise encodings), then merge them. I will try something similar in the context of multispeaker. Is the same speaker speaking both languages?

Unfortunately the same speaker is not speaking both the languages. But we have the resources to get annotations.
How many hours is required for both languages (assuming Hindi is dominant language)?

Hi, then you would probably need to inject a speaker id. I have a branch called 'multispeaker' thats able to do this, but you would have to mess around with the data prep etc.

@cschaefer26 Is there a reason you are taking only 500 speakers in the multispeaker branch? here
I trained LibriTTS recipe which uses x-vector features with 500 speakers (varying from 15mins to 1.5 hours every speaker) and the results were really bad.
samples.zip
Do you have any benchmarks for Multispeaker? Also, I feel we would need atleast one hour of data per speaker to train this model. Any thoughts?
P.S. I am extremely grateful for your help regarding all of my queries.

No, it just needs to be a number larger than the actual speakers. LibriTTS is pretty low quality - I got pretty decent results with VCTK. You gonna have to massage the dataset quite a bit with cutting silences etc. to make good use of the multispeaker dataset.

I there a phonemizer for Indian english? I trained my indian dataset using en and en-us and it gave me pretty bad results. Lot of words are being swallowed. Are there any alternatives for g2p for indian english?

Is there a phonemizer for Indian english? I trained my indian dataset using en and en-us and it gave me pretty bad results. Lot of words are being swallowed. Are there any alternatives for g2p for indian english?

Indian English is finally working after the FastPitch implementation. Even while using the British English phonemizer. Thanks

Is there a phonemizer for Indian english? I trained my indian dataset using en and en-us and it gave me pretty bad results. Lot of words are being swallowed. Are there any alternatives for g2p for indian english?

Indian English is finally working after the FastPitch implementation. Even while using the British English phonemizer. Thanks

@prajwaljpj @cschaefer26 Im also working on a hindi dataset and I found that in the current master implementation
the phoneme_set is not complete , for example words like 'भालू' phoneme conversion is "bʰaːluː" but since 'ʰ' is not presesnt in phoneme_set its removing 'ʰ' making it to "baːluː" which sounds wrong.
One way could be to update phonemeset with all the IPA symbols.
I wanted to know that if I update the symbols can I finetuned my dataset on the pre-trained model and also is there any better workaround.
Thanks.

Hi, yeah thats unfortunate. The only way currently to deal with this for you would be to fork the repo and update the phoneme symbols. I am thinking of making the phoneme set flexible (to be provided by the config), but that would be a major change. After changing the phoneme set you for sure will have to retrain everything (tacotron and forward-tacotron).

On a side note. If you need a better phonemizer, you could check out my new repo https://github.com/as-ideas/DeepPhonemizer. You will have to train your on phonemizer on an indian phoneme dataset. You could then use ForwarTacotron with 'no_cleaners' and 'use_phonemes=False'.

@cschaefer26 Thank you for your suggestion. I had a look into DeepPhonemizer. Looks really promising for situations where there is a mix of languages because the espeak phonemizer does not have good language switching capability. Is there a benchmark on how much latency DeepPhonemizer will introduce?

Yeah I built the repo mainly because we found that the espeak phonemizer is making too many mistakes (for German). If you use the forward transformer for the DeepPhonemizer, the latency is pretty low, for German I could phonemize around 60 sentences per second (although to be fair the German dictionary is quite large). Its for sure faster than the espeak phonemizer.

@cschaefer26

Yeah I built the repo mainly because we found that the espeak phonemizer is making too many mistakes (for German). If you use the forward transformer for the DeepPhonemizer, the latency is pretty low, for German I could phonemize around 60 sentences per second on my Mac without any additional parallelization (although to be fair the German dictionary is quite large). Its for sure faster than the espeak phonemizer.

Hmm, that's interesting... for German (75 characters sentence) I get
time for i in {1..60}; do ../tacorn/tools/espeak-ng -v de -x --ipa -q -f test-de.txt ; done

real	0m0,659s
user	0m0,364s
sys	0m0,191s

Perhaps the issue is calling it from python via subprocess.run, which forks and causes us some headaches with OOMs.
Or perhaps you run into this issue?
https://github.com/mozilla/TTS/issues/417#issuecomment-650186085

I think there is lots of overhead in re-initializing espeak. If you provide a list to the python phonemizer wrapper, its probably a lot faster. As for normalization, thats another story and I probably won't target it in the phonemizer repo as its quite language dependent.

Thanks for the clarification I'll try to do the same.


Added Catalan translation.

Added Catalan translation.

You need to also export your language in the src/locales/index.ts file

You need to also export your language in the src/locales/index.ts file

Sorry, totally forgot. Done now 👍  Thanks!

I just added two comments with things that a could remember now. Other than that LGTM.
If you are insterested in keeping this language updated, you can join our Telegram.  We ask for help in the Translators channel when there is new text to translate

I just applied all your suggestions. I will join the Telegram channel, thanks!

Before merging I removed strings that are not used anymore.
If you use VS Code, i recommend using the i18n Ally extension, you can see the missing strings for each translation. Helps a lot.


Cant import library into project

Using webpack + es6 + angular.
Expression import ngImgCrop from 'ng-img-crop-full-extended'; shows me an error 'Module not installed'.
Possible solution - create index.js file in root folder, with following code - require("./compile/unminified/ng-img-crop.js");
module.exports = "ngImgCrop";
But in result I still have an error in browser ReferenceError: ColorThief is not defined.
Any ideas?

i am not user of webpage, i can't help.

Hi @DmitryVolkovTulaco
@CrackerakiUA and i haven't worked with webpack.
I personally don't know how to help you with this problem.
Maybe it's an option to ask this on Stackoverflow? And if you have an answer, reopen this issue with a solution.
Sorry but the bets of luck to get it fixed.


Error Cannot find the corresponding private key

When I try keybase login I get

ERROR Sorry, your account is already established with a PGP public key, but this utility cannot find the corresponding private key on this machine.

gpg -K --fingerprint shows a fingerprint that matches the one displayed in the error message above.
I'm using Ubuntu 16.04.
my log id: fe3bd16dd5f6f4e78ad0b41c

@oliver602 I see that you are using Ubuntu 16.04, can you use gpg2 -K --fingerprint?
If there is no output, then that means Keybase is excepting gpg2 instaed of gpg, and because of gpg2 is using version >=2.1.0 as of 16.04 LTS, the secret keyring differs.
To solve it, you'll have to use gpg --export-secret-keys {UID|KeyID|Fingerprint} | gpg2 --import

Thanks @juanminm this did solve my problem. I had been using gpg 1.4.X

Thanks for the help @juanminm. It's too bad version 2.0 and version 2.1 are incompatible while versions 1.4 and 2.0 are compatible.  Makes it very tricky to workaround...


[Request] Aimbot FOV Circle

Please add an FOV circle with modifiable alpha (transparency) and colour values

there is pull requests tab next to the issues tab -_- #2218

Okay so theres bunch of Pulls about this "Not issue" issue.
You can make it yourself too its too easy, if you can't theres bunch of forums you can get help.
Close this issue.

#2218 check this out

I thought github was for people that are coding or learning to code. why would you ask for extra features if there's plenty of information that you can use to implement such a thing yourself???
but I agree with @franiodat and @tirziz when they say to check out #2218 it works fine.


chore: Added travel site project

Related Issue #1820
Description
Added a travel site
Screenshots

Hey @karthiknadar1204 ! 👋
For secure collaboration, remember to check out GitHub's commit signature verification feature. It helps ensure the authenticity of commits and prevents tampering.
Learn how it works: GitHub Docs
Happy coding! 😄

@priyankarpal Please add the label to this pr

@priyankarpal Please add the label to this pr

We are not providing any labels for project additions as they are not associated with solving any issues.( according to gssoc23 rules)


hotfix: exception thrown while inputing address

https://user-images.githubusercontent.com/12393771/151645780-70c940a9-fa89-4a43-b927-5e885aa199b8.mov


Implement the  spanner_dml_batch_update example

The description reads "Code sample showing how to call multiple DML SQL statements.", I cannot find code in golang, C# or Java that shows this. PHP has a way to send an array of SQL statements, so maybe it is language specific?
Created for completeness, we might decide to not implement this after all.

Is this just supposed to use ExecuteBatchDml(): https://github.com/googleapis/google-cloud-cpp-spanner/blob/0f4392f002c59766080a628d532ac6dc09cf566f/google/cloud/spanner/client.h#L423 Or something else?

Judging by the (new?) code sample in golang it just just using ExecuteBatchDml():
https://github.com/GoogleCloudPlatform/golang-samples/blob/5605e9f3002800e64862b445361098776401deed/spanner/spanner_snippets/snippet.go#L1256-L1278


Libpitt/104 column

Change log:

Add source type column
Add tooltip above column name to describe meaning of Last Touch

https://github.com/hubmapconsortium/data-ingest-board/issues/105

@maxsibilla @yuanzhou
If this source_type column is not to be added on HM, then add to NEXT_PUBLIC_EXCLUDE_TABLE_COLUMNS = '["source_type"]'
If it has a different name on HM, then add "source_type": {"field": "donor_type", "name": "Donor Type"} to the object in NEXT_PUBLIC_TABLE_COLUMNS

@libpitt The source type column will only appear for SenNet as HuBMAP only has human donors

Deployment note:

For Data Ingest Board, remember NEXT_PUBLIC_EXCLUDE_TABLE_COLUMNS = '["source_type"]' for the HM config.


Added "No rules defined" to Rules page in Web UI

When you have no alerting rules defined you get a screen sharing this
information in the WebUI. If no rules are defined then you instead see
an empty white screen. This adds a "No rules" defined else clause and
a Rules header to the page.

I think this is @juliusv's area according to the MAINTAINERS list.

@jamtur01 Thanks :+1:


[SPIKE] Azure Memory implementation

Redis Cache reference: https://azure.microsoft.com/en-us/pricing/details/cache/


Fix import paths and a directory name.

Fixed import paths.
Renamed examples/ directory to example_dags/

Hi @pgzmnk, I think this PR is now fairly out-of-date. The structural changes were included by a different PR, and I think we should work with @PubChimps on getting a "clean" version of that dbt ads reporting example DAG into the repository. What do you think?

closing this for now, we can add a dbt example once we get the new dbt provider


Send feedback on Network error sends to 404 page

When trying to save in a slower connection, the app throws a network error informing the user that save failed. In that modal, the send feedback button sends to 404 page as shown in the screenshot:

needs to be retested to determine if this only happens on slow connection

Tested on https://release-v2-1-0--gateway-edit.netlify.app/ with v2.1.0 build 66dd56e QA
Seeing 404 error on the send Feedback button on a fast connection


CSS changes

Minor CSS cleanups, should be barely noticeable but make the website more reactive by removing some accumulated hacks.

Apparently causes breaking changes. Do not merge until that is addressed.


How to handle updates through solution deployments

This documentation outlines the steps to import Synapse Link configurations from a solution. However, from our observations, if you deploy a new version of that solution with changes to the Synapse Link configuration component, regardless of whether the deployment is managed or unmanaged, those changes are not made to the Synapse Link configuration. The only way we've been able to get the configuration to synchronize with the updated component is to unlink and re-import the configuration.
Is this the intended functionality? If so, can it be documented? Our goal with ALM is to establish fully automated deployments, so I am hoping this is a temporary limitation.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 5eaf46c4-e66c-96c6-e780-42ed464a6db3
Version Independent ID: 5eaf46c4-e66c-96c6-e780-42ed464a6db3
Content: Transport Azure Synapse Link for Dataverse configuration - Power Apps
Content Source: powerapps-docs/maker/data-platform/azure-synapse-link-solution.md
Service: powerapps
Sub-service: dataverse-maker
GitHub Login: @sabinn-msft
Microsoft Alias: matp

Thanks for your comment, @melody-universe. This issue has been assigned to the doc author for review.

We are planning to do ALM enhancement around Apr 23 to allow user update Azure side configuration before importing solution. Thanks

Closing issue.

@JasonHQX This is a separate issue from #3503 and is unrelated to configuration in Azure. This is a problem with solution deployments on the Dataverse side. Steps are as follows:

Set up Synapse Link configuration in a development environment.
Add the Synapse Link configuration to a solution in the development environment.
Export the solution from the development environment.
Import the solution (same behavior for both managed and unmanaged) into a downstream environment, such as production.
Import the Synapse Link configuration from the solution into the Synapse section from the maker portal.

Note: The Synapse Link configuration will point to the same Storage Account / Synapse Workspace as the development environment, per #3503, which is a problem for customers, but this is a separate issue.

Go back to the development environment and make a change to the Synapse Link configuration.
Export a new version of the solution with the updated Synapse Link configuration.
Import the new solution into the downstream environment.

The Synapse Link configuration will be updated in the solution component, but not in the active configuration.
In order to activate the changes, the configuration needs to be unlinked and then re-linked.
This will delete the container on the storage account and re-create it, causing all configured tables to need to be re-synced.
This will also force any downstream services to need to re-sync as well.

Not only does this add a manual step to any deployment that includes changes to Synapse Link, but it consumes unnecessary resources, and will force us to bring down our services to our customers as well, as we were using Data Export Service to expose data to external customers through our own API's, but will need to bring down those API's while all data re-syncs.
Will this also be addressed in the Apr 23 update?
@Mattp123 Please re-open this issue.


Minor improvement

Fixed OV_BYTE bug since only value is acceptable for 1 byte size.

However, current codebase isn't using the OV_ENTRY macro.
Current codebase will eventually replace default OV entries to macro usage.

Rename OOVPA_TABLE_COUNT to XBSDB_ARRAY_SIZE

Plus a new macro, OV_MATCH, for ability to insert multiple values in one line instead of multiple lines.

Image proof of above image expands to:

Merging as is.


Just curious how this differs from Miniconda

What would be reasons to choose the one or the other?

Miniconda is a great choice for x86 processors.  The packages in those installer are professionally built and kept up-to-date.  The Miniconda installers for armv6 and armv7 have not been updated for some time and contain older versions of Python, conda, etc.  The Berryconda installers have more recent versions of these packages and automatically include the rpi channel with even more packages for these platforms.

Thanks, didn't take into account the architecture of the builds. Thanks for your efforts 🙏


Split the test-suites depending of the project tested

It's needed to split the unique testing project into several testing (one by project). It will facilitate some nasty issue that we have when developing (unable to run the tests on one project) but will also facilitate the move to NUnit 3.0

[X] Extract Unit tests for NBi.Xml into its own project running with NUnit 3.0
[X] Extract Unit tests for NBi.Core into its own project running with NUnit 3.0
[X] Extract Unit tests for NBi.Framework into its own project running with NUnit 3.0
[X] Extract Unit tests for NBi.GenbiL into its own project running with NUnit 3.0
[ ] Merge NBi.Services with NBi.GenbiL
[ ] Extract Unit tests for NBi.NUnit
[] Use of PrivateBinPath to run NBi.Testing.NUnit with NUnit 3.0 but stay away of NUnit 3.0 for the main project.
[ ] Extract Integration tests
[ ] Extract Acceptance tests

Available in 1.21.0-beta.264 at https://ci.appveyor.com/project/Seddryck/nbi/builds/26124807/artifacts


feat(services/gcs): Add start-after support for list

Closes #2106
This PR add a new feature for GCS:

add list_with_start_after capability

I found a little difference between the gcs start-offset and the s3 start-after. gcs will treat the pivot inclusive but s3 will not.
https://cloud.google.com/storage/docs/json_api/v1/objects/list
https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html

I found a little difference between the gcs start-offset and the s3 start-after. gcs will treat the pivot inclusive but s3 will not.
https://cloud.google.com/storage/docs/json_api/v1/objects/list https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html

Nice catch, we should exclude the start_after itself.

Great, Thanks @suyanhanx
I copied the docs here for references.

GCS

Filter results to objects whose names are lexicographically equal to or after startOffset. -- from docs

S3

StartAfter is where you want Amazon S3 to start listing from. Amazon S3 starts listing after this specified key -- from docs

we should exclude the start_after itself.
So, we follow the S3.

But, Why does the test pass?

test_list_with_start_after

Let me check the test first.

Ok, the test is just ignored. 🤔
---- services_gcs::list_test_list_with_start_after stdout ----
[2023-04-24T16:49:54Z WARN  behavior::services_gcs] service gcs not initiated, ignored

It's getting a bit late, and I haven't yet set up the GCS.
I just pushed a fix to let the CI help me check.
I'll deal with it tomorrow.

It's getting a bit late, and I haven't yet set up the GCS.
I just pushed a fix to let the CI help me check.

I apologize for the negative experience. Unfortunately, Github does not permit the loading of our secrets from a forked repository, which is why we are unable to make our GCS test function properly.

@Xuanwo So quick, Thank you! 👍


Skip email notifications not working as expected

We have configured a batch script to update a set of workitems using a query. We do not want the email notifications to be triggered to users while this script is run. The /skipEmailNotification parameter is not working as expected.

Yes, I found that the flag was not evaluated. I changed that on the branch https://github.com/jazz-community/work-item-command-line/tree/WCL-pre-log4j2

I merged the change into the 6.x branch that works for 7.0.2SR1 and later.

This should be fixed in https://github.com/jazz-community/work-item-command-line/releases/tag/WCL-V5.3.3

This should be fixed in https://github.com/jazz-community/work-item-command-line/releases/tag/WCL-V5.3.3

It is now working as expected. Thanks a lot Ralph.


Cannot export certain graphics as svg animation.

Cant figure out how to translate this error to changes I need to make it work.
Publishing begins for document with GUID: C457B30A-A9EA-4FE6-B468-334F8F57075B
Creating output file : C:\Users\fashaikh\AppData\Local\Temp\SnapSVGAnimator\Untitled41.html
Warnings generated while copying/importing in Untitled-1:

Blend Modes(Layer) is not supported in SnapSVGAnimator document, and has been converted to Normal in an instance of Movieclip .
ColorEffect is not supported in SnapSVGAnimator document, and has been removed in an instance of Movieclip .

The 'SymbolInstance.filters' JS API is not supported for SnapSVGAnimator.
The 'SymbolInstance.filters' JS API is not supported for SnapSVGAnimator.
The 'SymbolInstance.filters' JS API is not supported for SnapSVGAnimator.

It seems like you're trying to use an item that has a bunch of features, specifically filters not currently supported by SnapSVGAnimator, including Blend Modes and Color Effect.  Try removing these.

I imported the .fla file and tried to export that, and it only had
showed this error

The 'SymbolInstance.filters' JS API is not supported for SnapSVGAnimator.

For now, I went ahead with plain Javascript animation, since I could more or less achieve what I wanted by changing the classes of each item in the svg file. I'll close this bug for now.


Core.ExternalSharing / Personal Onedrive for Business / CrossSiteRequestNotSupported

Hey, I've been trying to use the extension methods to share documents in my Personal Onedrive for Business. When I do that I receive the following error message:
    CrossSiteRequestNotSupported
    MountPoint security error: host web isn't a personal site
What I'm doing is loading my site into the context and then sharing a document like this:
    var profileLoader = ProfileLoader.GetProfileLoader(ctx);
    UserProfile profile = profileLoader.GetUserProfile();
    Site personalSite = profile.PersonalSite;
    ctx.Load(personalSite);
    ctx.ExecuteQuery();
    rootWeb = personalSite.RootWeb;
    ctx.Load(rootWeb);
    ctx.ExecuteQuery();
    rootWeb.ShareDocument(id, email, ExternalSharingDocumentOption.View, true);
The SharingResult object returned by that contains the error message I was talking about.

This error was generated by user error. It can be closed now.

This error was generated by user error. It can be closed now.


.prompt method from Inquirer module is not working correctly

I tried the application on Mac OS and it does not generate any LICENSE file.
I investigated and I discovered that the .prompt method is called incorrectly. The method returned a promise, so, based on the documentation of the Inquirer module, it should be used this way:

inquirer.prompt([/* Pass your questions in here */]).then(function (answers) {
    // Use user feedback for... whatever!!
});

I am fixing this

thanks @yurimalheiros I just checked as well and it looks like there were some changes to inquirer in the last update.

@yurimalheiros see #65

Issue resolved by #69


string settings not being passed correctly for parallel execution

Looks like they are not written into the template file

Merged into master


ProteinFoldingResult, XYZ files and plotting of folded Proteins for ProteinFoldingProblem.

Summary
Implementation of the interpret  method in ProteinFoldingProblem. ProteinFoldingResult created by this method is now functional and can return the turns encoding the  shape of the protein, the Cartesian coordinates of each aminoacid, and generate a plot of the structure.
This PR solves issues #235 , #236  and #313.
Details and comments
Three new classes have been created and are arguments of ProteinFoldingResult.

ProteinDecoder: Takes the raw bitstring result and converts it into sequences of turns describing the shape of the protein.
ProteinXYZ: Takes the turn sequences from ProteinDecoder and the peptide from ProteinFoldingResult and generates a xyz file for the protein.
ProteinPlotter: Takes the xyz information from ProteinXYZ and generates a plot of the protein.

Usecase of how the class would work:
from qiskit_nature.problems.sampling.protein_folding.protein_folding_problem import (
    ProteinFoldingProblem,
)
from qiskit_nature.results.protein_folding_result import ProteinFoldingResult
from qiskit_nature.problems.sampling.protein_folding.interactions.miyazawa_jernigan_interaction import (
    MiyazawaJerniganInteraction,
)
from qiskit_nature.problems.sampling.protein_folding.peptide.peptide import Peptide

from qiskit_nature.problems.sampling.protein_folding.penalty_parameters import PenaltyParameters
from qiskit.utils import algorithm_globals

main_chain = "APRLR"
side_chains =  ["", "", "F", "Y", ""]

algorithm_globals.random_seed = 23
peptide = Peptide(main_chain, side_chains)
mj_interaction = MiyazawaJerniganInteraction()

penalty_back = 10
penalty_chiral = 10
penalty_1 = 10

penalty_terms = PenaltyParameters(penalty_chiral, penalty_back, penalty_1)

protein_folding_problem = ProteinFoldingProblem(peptide, mj_interaction, penalty_terms)
protein_folding_problem.qubit_op()

protein_folding_result = ProteinFoldingResult(protein_folding_problem, "0011011")

#Getting the turn sequences of the protein
print("This are the turns describing the main chain:",protein_folding_result.protein_decoder.get_main_turns())
print("This are the turns describing the side chain:",protein_folding_result.protein_decoder.get_side_turns())

#Generating the xyz file
protein_folding_result.protein_xyz.get_xyz_file( name = "NameForMyFile", output_data = True)

#Plotting the structure
protein_folding_result.plotstructure(ticks = True, grid = False)

This is how the plot looks like:

Note: Right now the xyz file is generated in the working directory.

I made some comments in a first look through but I will wait until things are cleaned up to go further.
Some general comments:
Init files need to be updated so that whatever needs to come out in the API reference documentation does so accordingly.
In terms of the xyz file coming out in the current directory, I am not sure if this is a default when the user is explicitly saving such as file - I assume there is value in the user having one of these. Or if its some byproduct during say plotting where I think it ought to be created by default in the users temp folder and arguably deleted once it has been consumed/used.
Perhaps the docs, via one of these classes or init file, should show a small example of an xyz file and some description should a user not have come across this before

I am not a chemist myself but apparently .xyz files are widely used among chemists. They all follow the same formatting. What I could do is add a link into the method's documentation in case someone doesn't know about them.
The file appearing in the working directory however is not a byproduct of anything and I believe should be changed. Any suggestions on how to handle file creation in qiskit?

I am not a chemist myself but apparently .xyz files are widely used among chemists. They all follow the same formatting. What I could do is add a link into the method's documentation in case someone doesn't know about them.

If this is a standard 'xyz' file then I think the format I see here as an xyz example is non-conformant. I believe in this case the first line in the file should be the number of atoms in the file and the 2nd line is an optional comment to name the molecule. So the xyz data would start on the third line. And yes this format is used in chemistry.

The file appearing in the working directory however is not a byproduct of anything and I believe should be changed. Any suggestions on how to handle file creation in qiskit?

If the file creation is always explicitly under the users control then noting its saved by default in the current directory if no path is provided in the method seems fine to me. I think my comment was in part due to seeing a comment in the code. If our code seeks to do this itself, which is what I meant by byproduct and I gave an example should it be needed for plotting, then that would be a different matter. For testing or anything else we would want to ensure these were generated in the temp directory - see Python tempfile - we have code in Nature already using this in places.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

131 of 175 (74.86%)  changed or added relevant lines in 7 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.1%) to 84.614%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

qiskit_nature/problems/sampling/protein_folding/protein_folding_problem.py
6
8
75.0%

qiskit_nature/results/utils/protein_shape_file_gen.py
46
50
92.0%

qiskit_nature/results/protein_folding_result.py
24
29
82.76%

qiskit_nature/results/utils/protein_plotter.py
11
44
25.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.1%

Covered Lines:
9635

Relevant Lines:
11387

💛  - Coveralls

I looked at the resultant html. I would expect to have at least ProteinFoldingResult in there and maybe others too as the release note mentions these. I assume the user gets to see these - if not we do not need to mention in release note since its supposed to be user facing changes. If we do then they need to get added to the docs. Only ProteinFoldingProblem in the release note is an actual link since only it comes out in the API ref.

If we have all the 3 classes listed there in the docs, since the user will see them as return from methods they call or whatever, perhaps we can improve the readability by having a sub-list there for example.

In that case maybe I will remove the plotter from the release note since it can not be accessed by the user. Howerver, I could still talk about the decoder and the file generator since ProteinShapeDecoder can be accessed and used to get the turns of the molecule and same goes for ProteinShapeFileGen since it can return an array containing the xyz file witouth actually saving it.

I have changed the return typehint in ProteinFoldingResult.plot_folded_protein() from ProteinPlotter->"figure" for the return type to match ProteinPlotter.plot().
I believe because of that I should add:
if _optionals.HAS_MATPLOTLIB:
    # pylint: disable=import-error,unused-import
    from matplotlib.pyplot import figure

with all the other inputs. Is that correct?

I have changed the return typehint in ProteinFoldingResult.plot_folded_protein() from ProteinPlotter->"figure" for the return type to match ProteinPlotter.plot(). I believe because of that I should add:
if _optionals.HAS_MATPLOTLIB:
    # pylint: disable=import-error,unused-import
    from matplotlib.pyplot import figure

with all the other inputs. Is that correct?

You should add it in this file at the top before the class.

I have changed the return typehint in ProteinFoldingResult.plot_folded_protein() from ProteinPlotter->"figure" for the return type to match ProteinPlotter.plot(). I believe because of that I should add:
if _optionals.HAS_MATPLOTLIB:
    # pylint: disable=import-error,unused-import
    from matplotlib.pyplot import figure

with all the other inputs. Is that correct?

You should add it in this file at the top before the class.

Yes, plus the typehint should be "figure" because we are not always importing it, right?

When trying to add ProteinFoldingResult to the API I ran into a problem. @woodsp-ibm maybe you can help me with it.
I have added ProteinFoldingResult to the init file in qiskit_nature.results . Now when we are trying to import ProteinFoldingResult we run into a circular import that we didn't have before.

ProteinFoldingResult runs init  file from qiskit_nature.problems.sampling.protein_folding when importing Peptide.
This file is importing ProteinFoldingProblem
ProteinFoldingProblem is importing ProteinFoldingResult.
And this way we get a cyclic import. I believe that by avoiding that ProteinFoldingResult runs the whole init file and just importing Peptide it should work, but even if specify the whole path to Peptide it seems like I have the same problem.
Any idea on how to solve this?

I have checked what happens if you try to use the plotting function in an environement without matplotlib and I get this message:
MissingOptionalLibraryError: "The 'matplotlib' library is required to use 'ProteinFoldingResult.plot_folded_protein'. You can install it with 'pip install matplotlib'."
Is this what we were expecting?

I have checked what happens if you try to use the plotting function in an environement without matplotlib and I get this message: MissingOptionalLibraryError: "The 'matplotlib' library is required to use 'ProteinFoldingResult.plot_folded_protein'. You can install it with 'pip install matplotlib'." Is this what we were expecting?

Yes and if you use it in unit tests then you need to add a skip decorator above the unit test method so that it doesn't run if matplotlib is not installed. I believe there are examples in the code.

I have checked what happens if you try to use the plotting function in an environement without matplotlib and I get this message: MissingOptionalLibraryError: "The 'matplotlib' library is required to use 'ProteinFoldingResult.plot_folded_protein'. You can install it with 'pip install matplotlib'." Is this what we were expecting?

Yes and if you use it in unit tests then you need to add a skip decorator above the unit test method so that it doesn't run if matplotlib is not installed. I believe there are examples in the code.

There aren't any unittests for the plotter.

When trying to add ProteinFoldingResult to the API I ran into a problem. @woodsp-ibm maybe you can help me with it. I have added ProteinFoldingResult to the init file in qiskit_nature.results . Now when we are trying to import ProteinFoldingResult we run into a circular import that we didn't have before.
* ProteinFoldingResult runs init  file from qiskit_nature.problems.sampling.protein_folding when importing Peptide.

* This file is importing ProteinFoldingProblem

* ProteinFoldingProblem is importing ProteinFoldingResult.
  And this way we get a cyclic import. I believe that by avoiding that ProteinFoldingResult runs the whole init file and just importing Peptide it should work, but even if specify the whole path to Peptide it seems like I have the same problem.
  Any idea on how to solve this?

Please see the latest fixes that remove the cycle.

@manoelmarques Thank you very much for solving the circular import!
Let me check if the documentation is referencing the methods properly now and if so I think we are done.

And in case this was overlooked - it was part of the comment about the plot method name

... Are we going to update the protein folding notebook to include this new interpret capability and show this figure there as well etc.

I just found a bug in the plotter. The letters appearing on each aminoacid are not where they should. I am fixing it now.

Is it possible to introduce a unit test that covers this bug? Of course we cannot test an image rendered but maybe the data that the plotter uses?

I just found a bug in the plotter. The letters appearing on each aminoacid are not where they should. I am fixing it now.

Is it possible to introduce a unit test that covers this bug? Of course we cannot test an image rendered but maybe the data that the plotter uses?

This bug came from a typo in the name of a local variable inside of a function.
I basically set:
side_aminoacids = self._protein_shape_gen._main_chain_aminoacid_list

Instead of:
side_aminoacids = self._protein_shape_gen._side_chain_aminoacid_list

There is a test for self._protein_shape_gen._side_chain_aminoacid_list, which is a part of the tests in ProteinShapeFileGen.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.2 out of 3 committers have signed the CLA.:white_check_mark: MarcDrudis:white_check_mark: woodsp-ibm:x: Marc Sanz DrudisMarc Sanz Drudis seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

I think it might make sense to put the notebook into a separate PR that gets merged after this one.

I do not think we will end up with rotatable plots in the documentation anyways so I suggest you simply make these plots static.

We should decide whether we want to include the notebook in this release or not. From my side they are good to go. I don't know if the last part (which is what I added) might be too specific with respect to the rest of the notebook.
On an other topic, the CI is failing on the windows machine. It says it has something to do with the conda environement and pyscf. It doesn't seem to have anything to do with ProteinFoldingResult, and I don't think it has to do with the matplotlib requirement either.

In regards of the depth shading the Axes 3DPlot has a parameter as to whether to do that or not.

depthshade: 	Whether or not to shade the scatter markers to give the appearance of depth. Default is True.

I guess I was wondering if it would be worth on the plotter we have to allow a user to change the value via that logic. We allow grids to be on or off, this could be another parameter that was passed on down. But then I guess we don't expose a way to change colors that its uses  either right, so maybe things are fine as they are and if the user wants/needs more they can plot it directly themselves.

In regards of the depth shading the Axes 3DPlot has a parameter as to whether to do that or not.

depthshade: 	Whether or not to shade the scatter markers to give the appearance of depth. Default is True.

Is this what the query was about "I am not sure I understand. The plot needs to be "3D" even if it doesn't rotate since it represents the shape of the protein. How would we turn that off?"
I guess I was wondering if it would be worth on the plotter we have to allow a user to change the value via that logic. We allow grids to be on or off, this could be another parameter that was passed on down. But then I guess we don't expose a way to change colors that its uses either right, so maybe things are fine as they are and if the user wants/needs more they can plot it directly themselves.

Well, since we are retuning a matplotlip figure the user can change any of those features of the plot. Same way I have manually rotated the plots in the notebook.


Popups in Gedit crash the whole app or the compositor

This happened at commit https://github.com/swaywm/wlroots/pull/885/commits/fa84b267e0fdc3bcdd43dc2d06b55ffad03059a3
Steps to reproduce: Start rootston, start gedit, move cursor over the "Open" menu. A popup should open, but instead gedit crashes, or sometimes rootston. Doesn't happen before this commit. I'm still investigating what exactly is going on.

I think #1842 is a duplicate of this one.

Seems like the same bug, as it happens with other Gtk apps as well.


Enhance chart controls for RBAC, Resource limits, Ingress

This patch enhances chart deployment for production K8S clusters.

Creation of RBAC resources should be optional.
Support for specifying a serviceAccount to run dispatch services
Ability to specify resource limits, requests at a global level
Also, adds the resource values in missing charts.
Ingress annotation prefix should be configurable since it
depends on the deployment.
Ref: https://github.com/kubernetes/ingress-nginx/commit/8f1ff15a6e2c3ec855a3a60a285bd3ddeef9890c

This change is

Closing due to CI upgrade


docs: updated kargo ui snapshot

fix: https://github.com/akuity/kargo/issues/2883
before:

after:

Where are all the little Argo logos coming from? Is there a reason that these should be here?


Mention that keywords should not start with a number.

Keywords starting with a number work in Clojure, but cause an exception in ClojureScript. More information here:
http://dev.clojure.org/jira/browse/CLJS-142

Thanks!  didn't know about this.  I added it to the docs data for keywords.
When we have docs pages built, we'll make the keyword a link from the tooltip, so they can go to the keyword doc page to learn about this gotcha.

Fair warning, I saw mutterings elsewhere on GH today that ClojureScript will shortly support number prefixed keywords for parity with Clojure.

https://github.com/pixie-lang/pixie/issues/294#issuecomment-99553638

thanks @arrdem, sweet intel!  added a note here


openscad: not using qt5 and libsForQt5 directly

Motivation for this change
Additional fix for #65522.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

This would fail to build unless I called it with libsForQt5 and not qt5...
Guess we should switch it to that?


How to search/reference log entry/event by Id

I would like to use Id property the log event to generate a URL to send to a 3rd party system.
But I'm a little confused with the terminology and the data:
If click on "Just find this" I get a search query like @Id = 'event-47cb9d000e7408dd1328485600000000'

Problem is, I can't seem to get this field

Then I copy "Download RAW JSON: I get:
{
  "@t": "2024-11-26T23:44:34.0000000Z",
  "@mt": "Sent Standby status message at 442/FAA58670",
  "@m": "Sent Standby status message at 442/FAA58670",
  "@i": "1c61adeb",
  "@l": "DEBUG",
  "APP_NAME": "XXXXXX",
  "ENVIRONMENT": "development"
}

The @i field looks like what I'm after since this doco describes it as event id; a 32-bit numeric hash of the event's message template:
https://github.com/serilog/serilog-expressions/?tab=readme-ov-file#language-reference
They are obviously not the same, but I can't seem to get a query to work for @i
When I send @i to a webhook, it is no longer HEX, but decimal, i.e<PHONE_NUMBER>
The confusion is around @Id (Event Id) and @i (event id) - both have the same name.

Looks like @Id is available to the Body Template, but just was not documented either here
https://docs.datalust.co/docs/template-syntax or here https://github.com/serilog/serilog-expressions/?tab=readme-ov-file#language-reference

Hi Richard,
The CLEF properties are documented here and the Seq built-in properties are documented here.
Within a message template  you can use either.
I'm not sure why @i is called Event id, when it is actually the event type (@EventType in Seq). Many events may have the same @i value.

Thanks @liammclennan, this is what I was after
https://docs.datalust.co/docs/built-in-properties-and-functions


Mask exemption

For the robust mask policy, there should be an exemption for those who have a medical inability to mask.

Thank you! We've already got a carve out for "appropriate exemptions" from vaccines, so this seems like a fitting extension of that. This one raises a somewhat tricky question though:
How might an event organizer, their volunteers, and participants enforce a mask mandate when it's not necessarily the case that someone not wearing a mask is in violation of the policy?

Perhaps those granted an exemption should wear some sort of "masking exempt" badge? That still has problems, but it keeps them from being pestered by event "security" to mask up.


chore: Add architecture diagram.

Hmm "Error: Unable to build website for locale en."   Since when is that an issue? 🤯

Awesome diagramm, Leo! Love it!! 🎖

Hmm "Error: Unable to build website for locale en." Since when is that an issue? 🤯

let me have a look.