Etcd with tls

Hello,
How to use bootkube and a etcd cluster that is secured by TLS and client auth?
In the kube-apiserver.yaml I need to add
        - --etcd-cafile=ca_cert.pem
        - --etcd-certfile=etcd_cert.pem
        - --etcd-keyfile=etcd_key.pem

But the kube-apiserver.yaml is generated from bootkube render. How to add those extra arguments? And how can the DaemonSet find the certificates?
I tried to patch kube-apiserver.yaml as follow, but no success.
apiVersion: "extensions/v1beta1"
kind: DaemonSet
metadata:
  name: kube-apiserver
  namespace: kube-system
  labels:
    k8s-app: kube-apiserver
spec:
  template:
    metadata:
      labels:
        k8s-app: kube-apiserver
      annotations:
        checkpointer.alpha.coreos.com/checkpoint: "true"
    spec:
      nodeSelector:
        master: "true"
      hostNetwork: true
      containers:
      - name: kube-apiserver
        image: quay.io/coreos/hyperkube:v1.5.4_coreos.0
        command:
        - /usr/bin/flock
        - --exclusive
        - --timeout=30
        - /var/lock/api-server.lock
        - /hyperkube
        - apiserver
        - --bind-address=<IP_ADDRESS>
        - --secure-port=443
        - --insecure-port=8080
        - --advertise-address=$(POD_IP)
        - --etcd-servers=https://xxx:2379
        - --etcd-cafile=/etc/etcd-ssl/ca_cert.pem
        - --etcd-certfile=/etc/etcd-ssl/client_cert.pem
        - --etcd-keyfile=/etc/etcd-ssl/client_key_insecure.pem
        - --storage-backend=etcd3
        - --allow-privileged=true
        - --service-cluster-ip-range=<IP_ADDRESS>/24
        - --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota
        - --runtime-config=api/all=true
        - --tls-cert-file=/etc/kubernetes/secrets/apiserver.crt
        - --tls-private-key-file=/etc/kubernetes/secrets/apiserver.key
        - --kubelet-client-certificate=/etc/kubernetes/secrets/apiserver.crt
        - --kubelet-client-key=/etc/kubernetes/secrets/apiserver.key
        - --service-account-key-file=/etc/kubernetes/secrets/service-account.pub
        - --client-ca-file=/etc/kubernetes/secrets/ca.crt
        - --authorization-mode=RBAC
        - --cloud-provider=
        - --anonymous-auth=false
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        volumeMounts:
        - mountPath: /etc/etcd-ssl
          name: ssl-certs-etcd
          readOnly: true
        - mountPath: /etc/ssl/certs
          name: ssl-certs-host
          readOnly: true
        - mountPath: /etc/kubernetes/secrets
          name: secrets
          readOnly: true
        - mountPath: /var/lock
          name: var-lock
          readOnly: false
      volumes:
      - name: ssl-certs-etcd
        hostPath:
          path: /etc/ssl/certs/etcd
      - name: ssl-certs-host
        hostPath:
          path: /usr/share/ca-certificates
      - name: secrets
        secret:
          secretName: kube-apiserver
      - name: var-lock
        hostPath:
          path: /var/lock

Hello again,
I tried now to add the etcd certificates to the kube-apiserver-secrets.yaml. But the result is the same, please see the bootkube logs. The data of the certificates was created by the following command.
cat etcd/ca_cert.pem | base64 | tr -d '\n'

[ 3964.131983] bootkube[5]: E0320 10:20:02.447691       5 create.go:35] Error creating assets: [error when creating "/core/assets/manifests/kube-apiserver-secret.yaml": secrets "kube-apiserver" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-apiserver.yaml": daemonsets.extensions "kube-apiserver" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-controller-manager-disruption.yaml": poddisruptionbudgets.policy "kube-controller-manager" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-controller-manager-secret.yaml": secrets "kube-controller-manager" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-controller-manager.yaml": deployments.extensions "kube-controller-manager" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-dns-deployment.yaml": deployments.extensions "kube-dns" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-dns-svc.yaml": services "kube-dns" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-flannel-cfg.yaml": configmaps "kube-flannel-cfg" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-flannel.yaml": daemonsets.extensions "kube-flannel" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-proxy.yaml": daemonsets.extensions "kube-proxy" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-scheduler-disruption.yaml": poddisruptionbudgets.policy "kube-scheduler" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-scheduler.yaml": deployments.extensions "kube-scheduler" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-system-rbac-role-binding.yaml": Timeout: request did not complete withi
[ 3964.147177] bootkube[5]: n allowed duration, error when creating "/core/assets/manifests/pod-checkpoint-installer.yaml": daemonsets.extensions "checkpoint-installer" is forbidden: not yet ready to handle request]
[ 3964.152845] bootkube[5]: Error creating assets: [error when creating "/core/assets/manifests/kube-apiserver-secret.yaml": secrets "kube-apiserver" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-apiserver.yaml": daemonsets.extensions "kube-apiserver" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-controller-manager-disruption.yaml": poddisruptionbudgets.policy "kube-controller-manager" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-controller-manager-secret.yaml": secrets "kube-controller-manager" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-controller-manager.yaml": deployments.extensions "kube-controller-manager" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-dns-deployment.yaml": deployments.extensions "kube-dns" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-dns-svc.yaml": services "kube-dns" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-flannel-cfg.yaml": configmaps "kube-flannel-cfg" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-flannel.yaml": daemonsets.extensions "kube-flannel" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-proxy.yaml": daemonsets.extensions "kube-proxy" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-scheduler-disruption.yaml": poddisruptionbudgets.policy "kube-scheduler" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-scheduler.yaml": deployments.extensions "kube-scheduler" is forbidden: not yet ready to handle request, error when creating "/core/assets/manifests/kube-system-rbac-role-binding.yaml": Timeout: request did not complete within allowed duration, error when creating "/co
[ 3964.162104] bootkube[5]: re/assets/manifests/pod-checkpoint-installer.yaml": daemonsets.extensions "checkpoint-installer" is forbidden: not yet ready to handle request]
[ 3964.165903] bootkube[5]: NOTE: Bootkube failed to create some cluster assets. It is important that manifest errors are resolved and resubmitted to the apiserver.
[ 3964.172094] bootkube[5]: For example, after resolving issues: kubectl create -f <failed-manifest>

I will post the kube-apiserver-secrets.yaml and kube-apiserver.yaml.
kube-apiserver-secrets.yaml
core@conf_vm03 ~ $ cat assets/manifests/kube-apiserver-secret.yaml
apiVersion: v1
data:
  etcd-ca.pem: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUdNekNDQkJ1Z0F3S...9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
  apiserver.crt: LS0tLS1CRUd...tRU5EIENFUlRJRklDQVRFLS0tLS0K
  apiserver.key: LS0tLS1CRUdJTi...U5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
  ca.crt: LS0tLS1CRUdJTi...0VSVElGSUNBVEUtLS0tLQo=
  etcd-client.pem: Q2VydGlmaWNhdGU6...bkZjZW9FYXg2akZ3ZVZ2cnNkZz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
  etcd-key.pem: LS0tLS1CR...QRlhtc09iU1NmZmJpcWxGN1l4bFB1ay8rVU1GYXBLSHZnMXpYZz09Ci0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
  service-account.pub: LS0tLS1CRUdJTiBQVU...TElDIEtFWS0tLS0tCg==
kind: Secret
metadata:
  name: kube-apiserver
  namespace: kube-system
type: Opaque

kube-apiserver.yaml
core@conf_vm03 ~ $ cat assets/manifests/kube-apiserver.yaml
apiVersion: "extensions/v1beta1"
kind: DaemonSet
metadata:
  name: kube-apiserver
  namespace: kube-system
  labels:
    k8s-app: kube-apiserver
spec:
  template:
    metadata:
      labels:
        k8s-app: kube-apiserver
      annotations:
        checkpointer.alpha.coreos.com/checkpoint: "true"
    spec:
      nodeSelector:
        master: "true"
      hostNetwork: true
      containers:
      - name: kube-apiserver
        image: quay.io/coreos/hyperkube:v1.5.4_coreos.0
        command:
        - /usr/bin/flock
        - --exclusive
        - --timeout=30
        - /var/lock/api-server.lock
        - /hyperkube
        - apiserver
        - --bind-address=<IP_ADDRESS>
        - --secure-port=443
        - --insecure-port=8080
        - --advertise-address=$(POD_IP)
        - --etcd-servers=https://<IP_ADDRESS>:2379,https://<IP_ADDRESS>:2379
        - --etcd-cafile=/etc/kubernetes/secrets/etcd-ca.pem
        - --etcd-certfile=/etc/kubernetes/secrets/etcd-client.pem
        - --etcd-keyfile=/etc/kubernetes/secrets/etcd-key.pem
        - --storage-backend=etcd3
        - --allow-privileged=true
        - --service-cluster-ip-range=<IP_ADDRESS>/24
        - --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota
        - --runtime-config=api/all=true
        - --tls-cert-file=/etc/kubernetes/secrets/apiserver.crt
        - --tls-private-key-file=/etc/kubernetes/secrets/apiserver.key
        - --kubelet-client-certificate=/etc/kubernetes/secrets/apiserver.crt
        - --kubelet-client-key=/etc/kubernetes/secrets/apiserver.key
        - --service-account-key-file=/etc/kubernetes/secrets/service-account.pub
        - --client-ca-file=/etc/kubernetes/secrets/ca.crt
        - --authorization-mode=RBAC
        - --cloud-provider=
        - --anonymous-auth=false
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        volumeMounts:
        - mountPath: /etc/ssl/certs
          name: ssl-certs-host
          readOnly: true
        - mountPath: /etc/kubernetes/secrets
          name: secrets
          readOnly: true
        - mountPath: /var/lock
          name: var-lock
          readOnly: false
      volumes:
      - name: ssl-certs-host
        hostPath:
          path: /usr/share/ca-certificates
      - name: secrets
        secret:
          secretName: kube-apiserver
      - name: var-lock
        hostPath:
          path: /var/lock

I was hoping to find here some feedback. I tested that setup with both bootkube version v0.3.10 and v0.3.11, and both failed with the same error. The SSL certificates are working fine if I first start the k8s cluster with etcd http://<IP_ADDRESS>:2379 and update the DaemonSet and Secret of the apiserver to switch to https://<IP_ADDRESS>:2379,https://<IP_ADDRESS>:2379.


Tokens should be sorted alphabetically

Currently, tokens seem to be sorted in the reverse order that they were added. This becomes a little unwieldy, the more tokens you have. I think it would make more sense to sort the list in alphabetical order.

Long-tap on a token in the list and drag to re-order it...

The newest token needs to go to the top so that the user can see it immediately. Otherwise, the user might think that token addition silently failed. Afterwards, you can reorder the tokens however you like by using long-press.

FreeOTP could automatically scroll so that the new token is in view. Just a thought.

This is silly. There should be at least a menu option or button to sort the list alphabetically. Having to do so manually is rather painful. Long-press + drag also doesn't scroll the view when you drag to the top/bottom edge of the view.


Tesseract fails for this image

I am trying to recognise the number in this image but doesn't return anything.

Environment

Tesseract Version: <3.04-->
Platform: <ubuntu 14.0->

@earthat  Try to install a newer version from https://launchpad.net/~alex-p/+archive/ubuntu/tesseract-ocr and test.

With tesseract4.0.0-beta.1 and with a modified image after preprocessing, tessdata and tessdata_best give the correct result.

1395-tessdata_fast-eng-1-jpg.txt
1395-tessdata-eng-1-jpg.txt
1395-tessdata_best-eng-1-jpg.txt

what preprocessing steps are considered

@earthat
Increase dpi to 300, double image in size, invert, convert to black and white - see the resulting image.
See https://github.com/tesseract-ocr/tesseract/wiki/ImproveQuality
Please close the issue.


Silu/Swish-1

Silu/Swish-1 - A variant of the Swish activation function, also known as SiLU (Sigmoid Linear Unit), where the Sigmoid function is multiplied by its input. It has been shown to perform well in deep networks.

This Silu class functions identically to the Swish class you've provided, with adjustments made to reflect the naming convention you're aiming for. This is because the mathematical definition of SiLU and Swish is the same, so their implementation will be identical except for the name.


Needed cards for deck sideboard not tracked correctly

Any of my decks for which I do not yet have all the cards generally are supposed to indicate that with something like a -1 or -n in the top right.  However if a card is in both the main deck and the sideboard then the count is incorrect.  This ultimately causes errors downstream in the Need booster counts and in the Booster Advisor calculations.
To Reproduce

Create a new deck.
Choose a card in your collection of which you only have two.
Add three of them to the main deck.
Notice the -1 appear on the name of the card in the deck, like below:

Add one of the card to the sideboard.
Note that there is no indication that you have insufficient cards.
Give the deck a name and save it.
Go look at that deck in My Decks list, and see something like below:

See the problem that the deck only indicates that you are short 1 card.
Look into the deck itself to look at it and see the same problem:

Expected behavior
What I expect is that the deck should be short 2 cards.  In my example case both the main deck and the sideboard should indicate -1.
The problem is worse if I were to put two copies in the main deck and two in the sideboard.  In that case there would be no indication at all that I'm actually missing two cards for the deck.  What should be shown in that case is that I have all the cards in the main deck but in the sideboard I'm at -2 for that card.
If you add the deck to the Craft List, then corrected counts would yield more accurate crafting recommendations.  Currently they would be misleading.
Additional context
The problem right now is that clearly the counts for each of the deck and the sideboard are being determined independently, which is not correct.

Hi! Thank you for your message. We will investigate this problem.

Confirmed.


By pass some purchasableGames that unavailable in user account country

Describe the issue
In this week, there have 3 game available
purchasableGames: [
"Knockout City™ Cross-Play Beta",
"Shadow of the Tomb Raider: Definitive Edition",
"Submerged: Hidden Depths"
]
but in my account country, Knockout City™ Cross-Play Beta is unavailable in my country, but my docker is setup in different country. After enter the captcha, it show the message as shown in below.
Screenshots or Logs

Additional context
Maybe the purchasableGames can be filter based on the user account contry to split the unavailable games or can bypass the unavailable game to continue with second one. Thank you

I literally just fixed these lol

https://claabs.github.io/epicgames-freegames-node/classes/AppConfig.html#countryCode
https://github.com/claabs/epicgames-freegames-node/commit/5bf273e8285b64c4c1f9dfe904baea3abda86640

Nice Work, claabs!!

Is it still working? I put the following in the config and restarted:
"countryCode:": "BY"

but still getting "This product is currently unavailable in your region." message. And my region is definitely BY:

I

Is it still working? I put the following in the config and restarted:
"countryCode:": "BY"

but still getting "This product is currently unavailable in your region." message. And my region is definitely BY:

Any luck? I tried adding it to the docker compose as:
    environment:
      - COUNTRY_CODE=CA

Then, when that didn't work I added it to the config. I don't know .json syntax very well, so I added it as:
{
  "runOnStartup": true,
  "cronSchedule": "0 0,6,12,18 * * *",
  "countryCode": "CA",
  "logLevel": "info",
  "webPortalConfig": {
[the rest redacted]

And I'm still getting a notification to purchase the a demo that is unavailable in my region:

Is it still working? I put the following in the config and restarted:
"countryCode:": "BY"

but still getting "This product is currently unavailable in your region." message. And my region is definitely BY:

@claabs any updates? Should it work or is it broken?

@sviperz See #346 for a temporary solution

@sviperz See #346 for a temporary solution

Did it, but it doesn't work for some reason:


Difference in behavior for deploy and generate for endpoint

Describe the bug
After a few hours of frustrating debugging I found the issue I was stumbling on. There is namely a difference in the prisma deploy and the generated prisma client (prisma generate) which both use the enpoint from the prisma.yml file.
I am running a local prisma server (via docker) and when I tried to run prisma deploy it would always pull up the login page in my browser in stead of connecting to the local running prisma server. I found that when the endpoint is set to an IP adres (from docker) it doesn't work but when setting it to http://localhost:1234 (1234 being the port of your prisma server) it works correctly.
Then the other side of the coin is when you set the endpoint like mentioned above using localhost the client that gets generated also has (at the bottom of the file) the endpoint: http://localhost:1234. When you try to run anything in the playground you will see that you get the request to http://localhost:1234/ failed error. In order to solve it you basically you need to change the localhost to the ip address that was causing the error in the prisma deploy command. (eg. endpoint: http:/<IP_ADDRESS>:1234)

How confusing is that?!

Expected behavior
Should be able to use the endpoint for both the generated client and the prisma deploy command.
Versions (please complete the following information):

Connector: Postgres
Prisma Server: 1.34
prisma CLI: prisma/1.34.1 (windows-x64) node-v10.16.0
OS: Windows 10

This is related to the resolution of MANAGEMENT_API_SECRET, so that's why prisma cli is trying to open a login window.
I think this is a duplicate of https://github.com/prisma/prisma/issues/4483#issuecomment-506482785

I don't think it is something with the MANAGEMENT_API_SECRET because it is currently not set in my case, unless that is what is causing it. The issue #4483 is similar so I will follow that one, thanks!


RTX 3090 failure

Just run the demo code:
import torch
>>> from transformers import AutoModel
>>> from kernl.model_optimization import optimize_model
>>> from transformers import RobertaTokenizer
>>> tokenizer = RobertaTokenizer.from_pretrained("roberta-base")
>>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt").to('cuda')
>>> model = AutoModel.from_pretrained('roberta-base').eval().cuda()
>>> optimized_model = optimize_model(model)
>>> with torch.inference_mode(), torch.cuda.amp.autocast():
...     outputs = optimized_model(**inputs)

and got this error:
----------------------------------------
TORCHDYNAMO: backend compiler failed
Traceback (most recent call last):
  File "<string>", line 21, in _layer_norm_fwd_fused_single_pass
KeyError: ('2-.-0-.-0-05cd73e2a4bdbeb71ab52fd66d1a1ac4-cfed90d463fc30ccffb0eb2fd26372d3-a357695982511d203a134df772c7b4a1-2121719c12e3ab66746f4a57f276d42e-0f76008a374e725ca29ccb33f1ba668f-dc48432b6b79843e2f9c7ad2e7355f59-f40d73592c2578180d3d8e3f64e3957d-0dd03b0bd512a184b3512b278d9dfa59-d7c2e52f8151bec157e9a17a1ec37dd3', (torch.float16, torch.float16, torch.float16, torch.float16, torch.float32, torch.float32, 'i32', 'i32', 'fp32'), (True, False, 1024), (True, True, True, True, True, True, (True, False), (True, False), (False,)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/miniconda3/envs/kernl/lib/python3.9/site-packages/torchdynamo/output_graph.py", line 341, in call_user_compiler
    compiled_fn = self.compiler_fn(gm, self.example_inputs())
  File "/root/kernl/src/kernl/model_optimization.py", line 46, in compiler
    return cuda_graphs_wrapper(gm, example_inputs, pool=pool)
  File "/root/kernl/src/kernl/implementations/cuda_graph.py", line 40, in cuda_graphs_wrapper
    model(*inputs)
  File "/home/miniconda3/envs/kernl/lib/python3.9/site-packages/torch/fx/graph_module.py", line 652, in call_wrapped
    return self._wrapped_call(self, *args, **kwargs)
  File "/home/miniconda3/envs/kernl/lib/python3.9/site-packages/torch/fx/graph_module.py", line 277, in __call__
    raise e
  File "/home/miniconda3/envs/kernl/lib/python3.9/site-packages/torch/fx/graph_module.py", line 267, in __call__
    return super(self.cls, obj).__call__(*args, **kwargs)  # type: ignore[misc]
  File "/home/miniconda3/envs/kernl/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "<eval_with_key>.18", line 19, in forward
    layer_norm_wrapper = kernl_optimizer_layer_norm_layer_norm_wrapper(add_3, layernorm);  add_3 = layernorm = None
  File "/root/kernl/src/kernl/optimizer/layer_norm.py", line 29, in layer_norm_wrapper
    return layer_norm(v, layernorm.weight, layernorm.bias, layernorm.eps)
  File "/root/kernl/src/kernl/implementations/layer_norm.py", line 319, in layer_norm
    return LayerNorm.apply(x, weight, bias, eps, implementation, use_rms_norm)
  File "/home/miniconda3/envs/kernl/lib/python3.9/site-packages/torch/cuda/amp/autocast_mode.py", line 116, in decorate_fwd
    return fwd(*_cast(args, cast_inputs), **_cast(kwargs, cast_inputs))
  File "/root/kernl/src/kernl/implementations/layer_norm.py", line 290, in forward
    implementation[(M,)](
  File "/home/miniconda3/envs/kernl/lib/python3.9/site-packages/triton/runtime/jit.py", line 106, in launcher
    return self.run(*args, grid=grid, **kwargs)
  File "<string>", line 42, in _layer_norm_fwd_fused_single_pass
  File "/home/miniconda3/envs/kernl/lib/python3.9/site-packages/triton/compiler.py", line 1215, in compile
    asm, shared, kernel_name = _compile(fn, signature, device, constants, configs[0], num_warps, num_stages,
  File "/home/miniconda3/envs/kernl/lib/python3.9/site-packages/triton/compiler.py", line 900, in _compile
    name, asm, shared_mem = _triton.code_gen.compile_ttir(backend, module, device, num_warps, num_stages, extern_libs, cc)
RuntimeError: Internal Triton PTX codegen error:
ptxas /tmp/fileuynWNC, line 79; fatal   : Parsing error near ':': syntax error
ptxas fatal   : Ptx assembly aborted due to errors

----------------------------------------
ERROR:root:Error while processing frame
Traceback (most recent call last):
  File "<string>", line 21, in _layer_norm_fwd_fused_single_pass
KeyError: ('2-.-0-.-0-05cd73e2a4bdbeb71ab52fd66d1a1ac4-cfed90d463fc30ccffb0eb2fd26372d3-a357695982511d203a134df772c7b4a1-2121719c12e3ab66746f4a57f276d42e-0f76008a374e725ca29ccb33f1ba668f-dc48432b6b79843e2f9c7ad2e7355f59-f40d73592c2578180d3d8e3f64e3957d-0dd03b0bd512a184b3512b278d9dfa59-d7c2e52f8151bec157e9a17a1ec37dd3', (torch.float16, torch.float16, torch.float16, torch.float16, torch.float32, torch.float32, 'i32', 'i32', 'fp32'), (True, False, 1024), (True, True, True, True, True, True, (True, False), (True, False), (False,)))

During handling of the above exception, another exception occurred:
......

the version is:
pytorch: 1.12.0 cu 11.6
torchdynamo: 1.12.0
triton: 2.0.0.dev20221001
transformers: 4.23.1
Could anyone help? Thanks.

Not sure if it's related but the version of torchdynamo should be torchdynamo-1.13.0.dev0.
I have run the script on my machine and it works on both REPL and as python script.
I am under Python 3.9
If using the right torchdynamo doesn't help, have you tried the Docker image?

Not sure if it's related but the version of torchdynamo should be torchdynamo-1.13.0.dev0. I have run the script on my machine and it works on both REPL and as python script. I am under Python 3.9 If using the right torchdynamo doesn't help, have you tried the Docker image?

Thank you for your help. However a new error appeared after I switched to torchdynamo 1.13.0
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/root/kernl/src/kernl/model_optimization.py", line 49, in run
    with torchdynamo.optimize(compiler):
  File "/root/kernl/torchdynamo-1.13/torchdynamo/eval_frame.py", line 116, in __enter__
    raise RuntimeError(
RuntimeError: torchdynamo.optimize(...) is used with a context manager. Please refer to https://github.com/pytorch/usage-example to use torchdynamo.optimize(...) as an annotation/decorator.

Have you ever seen this?

And one more thing. We should install cuda 11.6 for system, otherwise, if we only install pytorch with cuda 11.6, then the torchdynamo may not execute well. So better to verify by:
nvcc --version

and then verify the installation of torchdynamo

Alright.. after accurately changed torchdynamo to 1.13.0.dev0 by:
git clone --filter=blob:none --quiet https://github.com/pytorch/torchdynamo.git /tmp/pip-install-xbitcrmu/torchdynamo_b51b4ccfd53740999fb89c0b03e83fbb
cd /tmp/pip-install-xbitcrmu/torchdynamo_b51b4ccfd53740999fb89c0b03e83fbb
git checkout 6bd5b8bb8888d5dab0b9574b20f00de975eebc72

Finally worked. But the last step:
with torch.inference_mode(), torch.cuda.amp.autocast():
...     outputs = optimized_model(**inputs)

took around 5-10 seconds to produce a result.
Did I do it wrong? How to correctly use it? Thank you.

:-) wanted to write to you that I have not CUDA on my machine (life is now much better) and it works.
Regarding the timings, it's because you don't warmup.
Have a look to:

https://github.com/ELS-RD/kernl/blob/main/tutorial/bert e2e.ipynb (XNLI + Roberta)
https://github.com/ELS-RD/kernl/blob/main/tutorial/t5 e2e.ipynb (text generation on T5)

Warmup takes time but we are currently working to reduce it by a large margin and making it optional.
Let me know if you have other questions.

:-) wanted to write to you that I have not CUDA on my machine (life is now much better) and it works. Regarding the timings, it's because you don't warmup.
Have a look to:

https://github.com/ELS-RD/kernl/blob/main/tutorial/bert e2e.ipynb (XNLI + Roberta)
https://github.com/ELS-RD/kernl/blob/main/tutorial/t5 e2e.ipynb (text generation on T5)

Warmup takes time but we are currently working to reduce it by a large margin and making it optional.
Let me know if you have other questions.

Thank you and now it worked!

ok tks, closing


Missing latest lwjgl.zip download under Releases -> 3.3.4 -> Assets

Version
3.3.4
Platform
Linux x64, Linux arm64, Linux arm32, macOS x64, macOS arm64, Windows x64, Windows x86, Windows arm64
JDK
NA
Module
NA
Bug description
The latest stable release (3.3.4) is missing the usual zip file for download under Releases. Specifically under the 'Assets' section.
This wouldn't be a big deal, except A) it's linked from the LWJGL download page, and B) as I recall, the folder name/organization is slightly different than the ones output from the website, so swapping breaks configuration. (It's a pain to adjust this, at least not without editing the IntelliJ XMLs by hand.)
The URL you might expect the file to be hosted at is also unavailable.
Stacktrace or crash log output
No response

Hey @philipguin,
I have removed the file until #1005 is resolved. There hasn't been any progress on that though. It still looks like false positives and I'm still not sure what we can do about it.
Also note that there will be a 3.3.5 release soon.

LWJGL 3.3.5 is now available.


Is there storage size limit in etcd V2?

Storage size limit in V2
Storage size limit supports up to 8GB in etcd V3(https://coreos.com/etcd/docs/latest/dev-guide/limit.html), Is there a limit in etcd v2?

yes. no more than 200MB normally: https://github.com/coreos/etcd/blob/master/Documentation/v2/benchmarks/etcd-2-2-0-rc-memory-benchmarks.md#data-size-threshold


cpanel sieve filtering

Hey!
Is it possible to configure sieve filtering to work trough cpanel? My host makes sieve filtering only available in cpanel, and not on an address:port

It would be great ... !


parse rgb string doesn't work

In parse(value) if value is a rgb string, function p2v returns a string, but number is required for absround() later in format functions.
change
	var p2v = function(p){ 		return isPercent.test(p) ? absround(parseInt(p) * 2.55) : p; 	};
to
	var p2v = function(p){ 		return isPercent.test(p) ? absround(parseInt(p) * 2.55) : parseInt(p); 	};
will fix this issue

awesome, thanks, will commit the change when i have some time

fixed in https://github.com/moagrius/Color/pull/6


Revert "Cleanup travis"

#3445 ended up causing problems; tests appeared to be passing due to some issue with TravisCI, even though they shouldn't. Reverting the changes for now.
Reverts pachyderm/pachyderm#3445

LGTM and build is passing. Merging.


Implement sqrt function

Related to https://github.com/rust-num/num-bigint/issues/55
Should it be implemented on top of BigInts or some better approach is possible?

Ratio will be trickier to deal with precision, because you probably don't want to just round √(a/b) by operating on the numerator and denominator separately. That is, ⌊√a⌋/⌊√b⌋ would be quite lossy.

I'm not really sure what's the right answer here.  Approximating irrational numbers as a rational requires some level of control over how much accuracy is desired -- we can't be perfect, but we can get arbitrarily close, especially with BigInt that can grow as much as you like.
I have some code I was playing with for #32, to generate the most accurate approximations for a few interesting constants, given any particular Ratio<T> limit.  For √2 it looks like:
# √2 ≈ 1.4142135624, f64 error 9.667e-17
================================
    type |      error | rational
---------|------------|---------
      i8 |   7.215e-5 | 99/70
      u8 |  -1.238e-5 | 239/169
     i16 |  -1.840e-9 | 27720/19601
     u16 | -3.158e-10 | 47321/33461
     i32 | -2.055e-19 |<PHONE_NUMBER>/1311738121
     u32 | -2.055e-19 |<PHONE_NUMBER>/1311738121
     i64 |  1.493e-38 |<PHONE_NUMBER>338442563/4866752642924153522
     u64 | -2.561e-39 |<PHONE_NUMBER>8186749607/11749380235262596085
    i128 | -7.878e-77 |<PHONE_NUMBER>01103275326877813426364627544/94741125149636933417873079920900017937
    u128 | -1.352e-77 |<PHONE_NUMBER>50740208744750893347264645481/161733217200188571081311986634082331709

These values and their errors were computed based on 100 digits of the constant from Wolfram Alpha.  It's possible to derive that basis mathematically, but I didn't bother because I was doing this for a variety of constants.
Note that even the 32-bit ratios are more accurate than f64 can represent!  Which makes some sense, since a pair of 32-bit values has 64 significant bits together, or 62 w/ signs, vs. a 53-bit mantissa in f64.
And yet, it's probably not so useful to have ratios that are so near their type limits, since a lot of arithmetic requires multiplication that would overflow.  That's not a problem for BigRational, of course.

I actually tried to implement superpi using this crate. However, it wasn't a big success: https://github.com/Pzixel/superpi-rs
It gives 3.1410... for 10 iterations (should give 10^2 correct digits). It's not directly related but it provides some experience.

let new_b = BigRat::new(new_b_square.numer().sqrt(),new_b_square.denom().sqrt());

That's exactly the ⌊√a⌋/⌊√b⌋ which I said would be lossy.
I think even your initial ratio b for 1/√2 might be too inaccurate.  I don't have experience with this algorithm, but it doesn't appear to be good at self-correcting for errors.  If I set the initial b using the u32 ratio I gave above, your code then converges on 3.1415591..., or with my u64 ratio it gives 3.141592653787... (vs the true 3.141592653589...)
Maybe it would fare better with a Ratio::sqrt that doesn't round as much, but I'm not sure.

P.S. It also shows how much clones you need to implement some algorithms.

You can do most operations by reference, e.g. let new_t = t - &p * &a_diff * a_diff;

That's exactly the ⌊√a⌋/⌊√b⌋ which I said would be lossy.

Yep, I know, but I don't know any better approach. Error is too high to make this algorithm work fine, because it should work with any precision, while we have only 11 right digits on u128. It's even worse than f64 which often has 16 significant digits.

You can do most operations by reference, e.g. let new_t = t - &p * &a_diff * a_diff;

Thank you for pointing that out, gonna try that.

We could just use that sqrt-by-parts as an initial guess for the Babylonian method, but the question remains of how many iterations to compute.  Maybe even just one, because the ratio part will quickly get large!

@cuviper maybe sqrt just should take some kind of precision as parameter?..

I don't get it, what is the use case of having a square root function on a rational that does approximations?

Any sqrt that's not represented symbolically will be an approximation, including those for floating point.  A rational sqrt has a chance to improve accuracy.  (Whether that's needed is a separate question.)

But as already noted in the discussion, there are many problems with this:

The problem is ill specified; rationals are not closed under the square root operation

Nor is there the existence of a "nearest" rational, under any possible rounding scheme (at least integers have that!)

If you get a big denominator on a fixed-size integer type, then you can't perform virtually any math on it at all without getting an integer overflow. So what on earth are you going to do with it?

The fact that Rational32 can represent sqrt(2) more accurately than f64 doesn't mean much if any further operation you perform on it causes the world to explode.
This seems like a very niche problem that would be better serviced by a niche solution.  Something that e.g. gives approximate rational roots to any rational polynomial.

N.B. why I care so much:
Just earlier today, working on a diophantine equation solver, I almost wrote the following:
trait SqrtExt {
    /// Compute the (exact) square root, or panic if there isn't one
    fn sqrt(self) -> Self;
}

impl SqrtExt for Rational64 {
    ...
}

but then stopped and decided to make it a free function instead when I considered that it could get overridden by an inherent method.

I have been talking with guys on Math.StackOverflow and they convinced me it doesn't make sense to perform any kind of sqrt for rationals.
So we probably need to implement arbitrary-sized fixed-point arithmetic in new repo like num-fp and implement all such operations there. I with to help if I can.

For fixpoint, consider existing crates like https://crates.io/crates/bigdecimal

Or you could implement continued fractions, whereby all square roots of rationals are known to have repeating expansion. So you only need to evaluate them up to where you detect repetition and the rest of arbitrary precision you get for free.

Quick hack, as yet untested.  YMMV.
/// # Calculates the approximate square root of the value
///
/// Calculates the approximate square root of `value`.  If the returned value is
/// `Ok(_)`, then it is guaranteed to be within `epsilon` of the actual
/// answer.  If `epsilon <= 0.0`, then `Err` is returned (the reason for the
/// bound of `0.0` is because the approximation algorithm is unable to return an
/// exact answer).  If `value < 0.0`, then `Err` is returned (`BigRational` is
/// a real valued object; it cannot represent complex values).  In both `Err`
/// cases, the value will be a `String` explaining what the error actually is.
///
/// # Parameters
///
/// - `value` - The value whose approximate square root you wish to obtain.  If
///     this is less than `0.0`, then `Err` will be returned.
/// - `epsilon` - The maximum acceptable difference between the returned value
///     and the actual value.  The returned value is in the range
///     `[actual - epsilon, actual + epsilon]`.
///
/// # Returns
///
/// If everything went as expected, then `Ok(_)` will be returned, containing
/// a value that is within `± epsilon` of the actual value.  If anything went
/// wrong, then `Err(_)` will be returned, containing a `String` outlining what
/// the problem was.
pub fn approx_square_root(value: BigRational, epsilon: BigRational) -> Result<BigRational, String> {
    if value < BigRational::zero() {
        return Err(format!("approx_square_root() cannot calculate the square \
                            root of negative values.  value = {}", value).to_owned());
    } else if epsilon <= BigRational::zero() {
        return Err(format!("approx_square_root() cannot calculate the square \
                            root with a non-positive epsilon.  \
                            epsilon = {}", epsilon).to_owned());
    }

    // I'm going to use the Babylonian method to find the square root.  This is
    // described at
    // https://en.wikipedia.org/wiki/Methods_of_computing_square_roots#Babylonian_method
    // To do so, I need to have an initial seed value that is the approximate
    // square root.  This will estimate will be refined until it is within
    // epsilon of the real value.

    // Calculates seed values for all values >= 1.0.  This is used below when
    // calculating the seed value.
    #[inline]
    fn calc_seed(value: &BigRational) -> BigRational {
        let bits = value.ceil().to_integer().bits();
        let half_bits = bits / 2;
        let approximate = BigInt::one() << half_bits;
        BigRational::from_integer(approximate)
    };

    let mut x = if value >= BigRational::one() {
        calc_seed(&value)
    } else {
        // Because the value is less than one, I can't use the trick above
        // directly.  Instead, I'm going to find the reciprocal, and then do the
        // trick above, and then use the reciprocal of that as the seed.
        calc_seed(&(value.recip())).recip()
    };

    // We now have an initial seed.  Time to refine it until it is within
    // epsilon of the real value.  Inlined functions could probably be really
    // inlined, but this is slightly easier for me to read.

    #[inline]
    fn calc_next_x(value: BigRational, x: BigRational) -> BigRational {
        let two = BigRational::one() + BigRational::one();
        (x + (value / x)) / two
    };

    #[inline]
    fn calc_approx_error(value: BigRational, x: BigRational) -> BigRational {
        let two = BigRational::one() + BigRational::one();
        ((value - (x * x)) / (x * two)).abs()
    }

    while calc_approx_error(value, x) > epsilon {
        x = calc_next_x(value, x);
    }

    Ok(x)
}

Maybe the sqrt method could return an Option<Ratio> instead, where if the numerator and denominator are perfect squares it returns their square root and None otherwise?

FWIW, I have a highly-optimized implementation of perfect square detection (for integers) that I wrote while working on uutils/coreutils' implementation of integer factorization: factor/src/numeric/sqrt.rs
Given an (exact) integer square root function, getting exact square roots of some x: Ratio<_> would be simply let y = x.reduced(); Ratio::new_raw(exact_sqrt(y.numer())?, exact_sqrt(y.denom())?).
It's not yet upstreamed (mostly because, for that integer factoring application, there were some I/O-related optimizations I'd need to get in first) but I'd be entirely happy to see that code used for your usecase; if there's interest, I could submit a PR to include a (generalized) version in num-integer.

FWIW as well, I'm implementing a crate for irrational numbers. The QuadraticSurd struct can exactly meet the demand for this, as it can represent any irrational numbers in form (a+b√r)/c.
I already implemented a very simple to_rational method for it (basically just rounded down the square root part), and the square root of a rational number can be achieved by QuadraticSurd<T>::from_sqrt(t: Ratio<T>). The missing part might be a method to produce more accurate rational estimation of the surd.
What I have in mind now is to make all irrational types have a method that takes an additional parameter to determine the precision of rational estimation. It might looks like
trait Irrational {
    fn to_accurate_rational(&self, iterations: u32) -> BigRational;
}

With these methods implemented, one can get the rational approximation of the square root by QuadraticSurd::from_sqrt(num).to_accurate_rational(10)
@nbraud num-integer already has a sqrt implementation that has been optimized.

Quick hack, as yet untested. YMMV.
/// # Calculates the approximate square root of the value
///
/// Calculates the approximate square root of `value`.  If the returned value is
/// `Ok(_)`, then it is guaranteed to be within `epsilon` of the actual
/// answer.  If `epsilon <= 0.0`, then `Err` is returned (the reason for the
/// bound of `0.0` is because the approximation algorithm is unable to return an
/// exact answer).  If `value < 0.0`, then `Err` is returned (`BigRational` is
/// a real valued object; it cannot represent complex values).  In both `Err`
/// cases, the value will be a `String` explaining what the error actually is.
///
/// # Parameters
///
/// - `value` - The value whose approximate square root you wish to obtain.  If
///     this is less than `0.0`, then `Err` will be returned.
/// - `epsilon` - The maximum acceptable difference between the returned value
///     and the actual value.  The returned value is in the range
///     `[actual - epsilon, actual + epsilon]`.
///
/// # Returns
///
/// If everything went as expected, then `Ok(_)` will be returned, containing
/// a value that is within `± epsilon` of the actual value.  If anything went
/// wrong, then `Err(_)` will be returned, containing a `String` outlining what
/// the problem was.
pub fn approx_square_root(value: BigRational, epsilon: BigRational) -> Result<BigRational, String> {
    if value < BigRational::zero() {
        return Err(format!("approx_square_root() cannot calculate the square \
                            root of negative values.  value = {}", value).to_owned());
    } else if epsilon <= BigRational::zero() {
        return Err(format!("approx_square_root() cannot calculate the square \
                            root with a non-positive epsilon.  \
                            epsilon = {}", epsilon).to_owned());
    }

    // I'm going to use the Babylonian method to find the square root.  This is
    // described at
    // https://en.wikipedia.org/wiki/Methods_of_computing_square_roots#Babylonian_method
    // To do so, I need to have an initial seed value that is the approximate
    // square root.  This will estimate will be refined until it is within
    // epsilon of the real value.

    // Calculates seed values for all values >= 1.0.  This is used below when
    // calculating the seed value.
    #[inline]
    fn calc_seed(value: &BigRational) -> BigRational {
        let bits = value.ceil().to_integer().bits();
        let half_bits = bits / 2;
        let approximate = BigInt::one() << half_bits;
        BigRational::from_integer(approximate)
    };

    let mut x = if value >= BigRational::one() {
        calc_seed(&value)
    } else {
        // Because the value is less than one, I can't use the trick above
        // directly.  Instead, I'm going to find the reciprocal, and then do the
        // trick above, and then use the reciprocal of that as the seed.
        calc_seed(&(value.recip())).recip()
    };

    // We now have an initial seed.  Time to refine it until it is within
    // epsilon of the real value.  Inlined functions could probably be really
    // inlined, but this is slightly easier for me to read.

    #[inline]
    fn calc_next_x(value: BigRational, x: BigRational) -> BigRational {
        let two = BigRational::one() + BigRational::one();
        (x + (value / x)) / two
    };

    #[inline]
    fn calc_approx_error(value: BigRational, x: BigRational) -> BigRational {
        let two = BigRational::one() + BigRational::one();
        ((value - (x * x)) / (x * two)).abs()
    }

    while calc_approx_error(value, x) > epsilon {
        x = calc_next_x(value, x);
    }

    Ok(x)
}

I wanted to use this code since there doesn't appear to be a good solution available, but I noticed you didn't include a license. Is it safe to assume this is MIT? I don't want to steal your work, could you specify a license? Thanks!

@noahdahlman My apologies!  It is Apache 2.0.  Be warned!  That was really a quick hack without any significant testing whatsoever!  Before relying on it, please spend some time verifying that it actually works as expected!

I am not sure if this was mentioned but it is possible to compute and store square roots of unlimited precision, using continued fractions which will have (guaranteed) recurring group of digits. We only need to store the recurring finite group of them. Such continued fraction can then be evaluated by a lazy function to any required precision on demand.

@liborty, you actually mentioned it earlier in this very same conversation! :wink:
IMHO, @cmpute's crate is the solution that solves @cuviper and @ExpHP's concerns with regards to irrationality.  Perhaps @cmpute and @cuviper could work together to combine forces in some way?  Maybe bringing num-irrational into the num family of crates?

@liborty, you actually mentioned it earlier in this very same conversation! :wink:
IMHO, @cmpute's crate is the solution that solves @cuviper and @ExpHP's concerns with regards to irrationality.  Perhaps @cmpute and @cuviper could work together to combine forces in some way?  Maybe bringing num-irrational into the num family of crates?

I'm glad that you are interested in this. However, I'm actually planning to integrate the num-irrational trait with my dashu crates, because modeling the quadratic surd in a generic way is pretty troublesome.

@cmpute Ah, I see.  I actually thought it was going to be a part of the num family because of it's name; num-irrational really sounds like it belongs there.

There are no formal namespaces, so it's free to be named num-* without being part of this @rust-num org.

@cmpute Ah, I see.  I actually thought it was going to be a part of the num family because of it's name; num-irrational really sounds like it belongs there.

Yes, it's indeed intended to interop with the num crates (hence the name), and it does support conversions pretty well now. I'm open to integrate it into the num ecosystem, but I don't plan to improve the num-irrational crate now.

Hi all - thanks for the great crate.  What's the status of this discussion?  FYI: my use-case is CAD software where I'm hoping to store coordinates in an arbitrary-sized format and I'm considering using this crate.  But, like in many CAD systems, I'll often have to compute the distance between two points which requires a sqrt() function.  Looking over the thread, @ckaran 's solution would work for me as it's easy for me to pick an epsilon that's smaller than physical machines can handle.  Any thoughts about merging that solution into the code?  If this whole thread has gone cold, I can put it up as a pull request...
Please let me know and thanks in advance.

@robs-zeynet I, for one, completely forgot about this until you brought it back up!
I suspect that it won't get rolled into the num-rational codebase because it really violates the spirit of the num-rational crate (the operations are closed under the rationals).  You might be better off just copying my code into your own code.  The advantage there is that while the code I wrote does seem to work, anybody that looks at it for more than 5 seconds can immediately see that there are a lot of opportunities for performance improvements!  In short, take it as a starting point and tune it until it works for you.


multi_gpu_training_r3det

Hi. First of all, thank you for opensourcing your great work!
I am trying to train a model through multi_gpu_training_r3det.py using the configurations given in the repo but I am facing some problem.
My environment is consistent with the README
(1、python == 3.5
2、cuda == 10.0 , gcc = 4.9.4
3、opencv(cv2)
4、tfplot == 0.2.0
5、tensorflow == 1.13.1
However while running multi_gpu_training_r3det.py, my script is stuck at sess.run([train_op, global_step])
with tf.Session(config=tfconfig) as sess:
            sess.run(init_op)

            # sess.run(tf.initialize_all_variables())
            coord = tf.train.Coordinator()
            threads = tf.train.start_queue_runners(coord=coord, sess=sess)

            summary_path = os.path.join(cfgs.SUMMARY_PATH, cfgs.VERSION)
            tools.mkdir(summary_path)
            summary_writer = tf.summary.FileWriter(summary_path, graph=sess.graph)

            if not restorer is None:
                restorer.restore(sess, restore_ckpt)
                print('restore model')

            for step in range(cfgs.MAX_ITERATION // num_gpu):
                training_time = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(time.time()))

                if step % cfgs.SHOW_TRAIN_INFO_INTE != 0 and step % cfgs.SMRY_ITER != 0:
                    _, global_stepnp = sess.run([train_op, global_step])

                else:
                    if step % cfgs.SHOW_TRAIN_INFO_INTE == 0 and step % cfgs.SMRY_ITER != 0:
                        start = time.time()
                        _, global_stepnp, total_loss_dict_ = \
                            sess.run([train_op, global_step, total_loss_dict])

My output right now is as such
(ive commented out the var in graph and ckpt while restoring resnet better display)
Use tf.cast instead.
WARNING:tensorflow:From /mnt/lustre/parkchanho/equivariance-project2/r3det/data/io/read_tfrecord_multi_gpu.py:21: TFRecordReader.__init__ (from tensorflow.python.ops.io_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by `tf.data`. Use `tf.data.TFRecordDataset`.
WARNING:tensorflow:From /mnt/lustre/parkchanho/equivariance-project2/r3det/data/io/read_tfrecord_multi_gpu.py:118: batch (from tensorflow.python.training.input) is deprecated and will be removed in a future version.
Instructions for updating:
Queue-based input pipelines have been replaced by `tf.data`. Use `tf.data.Dataset.batch(batch_size)` (or `padded_batch(...)` if `dynamic_pad=True`).
tfrecord path is --> /mnt/lustre/parkchanho/equivariance-project2/r3det/data/tfrecord/DOTA_train*
WARNING:tensorflow:From ./tools/multi_gpu_train_r3det.py:161: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, use
    tf.py_function, which takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.

Tensor("tower_0/Shape:0", shape=(4,), dtype=int32, device=/device:GPU:0) #############################
we are in Pyramid::-======>>>>
['P3', 'P4', 'P5', 'P6', 'P7']
base_anchor_size are:  [32, 64, 128, 256, 512]
WARNING:tensorflow:From /mnt/lustre/parkchanho/anaconda3/envs/r3det/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Tensor("tower_1/Shape:0", shape=(4,), dtype=int32, device=/device:GPU:1) #############################
we are in Pyramid::-======>>>>
['P3', 'P4', 'P5', 'P6', 'P7']
base_anchor_size are:  [32, 64, 128, 256, 512]
Tensor("tower_2/Shape:0", shape=(4,), dtype=int32, device=/device:GPU:2) #############################
we are in Pyramid::-======>>>>
['P3', 'P4', 'P5', 'P6', 'P7']
base_anchor_size are:  [32, 64, 128, 256, 512]
Tensor("tower_3/Shape:0", shape=(4,), dtype=int32, device=/device:GPU:3) #############################
we are in Pyramid::-======>>>>
['P3', 'P4', 'P5', 'P6', 'P7']

model restore from : /mnt/lustre/parkchanho/equivariance-project2/r3det/output/trained_weights/RetinaNet_DOTA_R3Det_2x_20191108/DOTA_1079996model.ckpt
2020-04-07 16:58:19.716699: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-04-07 16:58:19.723312: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency:<PHONE_NUMBER> Hz
2020-04-07 16:58:19.725536: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x7fdc0a9c33c0 executing computations on platform Host. Devices:
2020-04-07 16:58:19.725561: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>

/mnt/lustre/parkchanho/anaconda3/envs/r3det/lib/python3.5/site-packages/tensorflow/python/ops/gradients_impl.py:110: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
WARNING:tensorflow:From ./tools/multi_gpu_train_r3det.py:315: start_queue_runners (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the `tf.data` module.

WARNING:tensorflow:From /mnt/lustre/parkchanho/anaconda3/envs/r3det/lib/python3.5/site-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
restored model

One of the problems I believe is that when I run the code below as stated in the README, it does not produce anything but an empty file (data/tfrecord/DOTA_train.tfrecord ).
cd $PATH_ROOT/data/io/
python convert_data_to_tfrecord.py --VOC_dir='/PATH/TO/DOTA/'
                                   --xml_dir='labeltxt'
                                   --image_dir='images'
                                   --save_name='train'
                                   --img_format='.png'
                                   --dataset='DOTA'

Could this be the source of the problem? If not can you suggest any directions I should take? Thank you !

Data path problem


NamedValues refactoring

This PR refactors the NamedValues class to mock Python dict and makes NamedValues instances accessible as a property of ASN.1 type.

Codecov Report

Merging #45 into master will decrease coverage by 0.05%.
The diff coverage is 71.55%.

@@            Coverage Diff             @@
##           master      #45      +/-   ##
==========================================
- Coverage   81.62%   81.56%   -0.06%
==========================================
  Files          25       25
  Lines        3738     3781      +43
==========================================
+ Hits         3051     3084      +33
- Misses        687      697      +10

Impacted Files
Coverage Δ

pyasn1/type/univ.py
77.45% <65.71%> (-0.14%)
:arrow_down:

pyasn1/type/namedval.py
77.31% <74.07%> (+2.31%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c159c7e...e4376cb. Read the comment docs.


Move src/* to src/luagraphs/* to fix "module 'data.list' not found"

Thank you for implementing #1!  I installed 1.0-4 and opened a luaprompt to try it out, but received an error that data.list was not found.  This pull request fixes the error on my system by using require('luagraphs.data.list') instead of require('data.list'), and the same for all other luagraphs modules.  I moved src into src/luagraphs to support this.  All tests still pass.  The details follow.
I installed via luarocks:
cxw:~/src$ luarocks install luagraphs
Warning: falling back to curl - install luasec to get native HTTPS support
Installing https://luarocks.org/luagraphs-1.0-4.src.rock
luagraphs 1.0-4 is now installed in /usr/local (license: MIT/X11)

So installation went fine.
cxw:~/src$ luap
Lua 5.2.4  Copyright (C) 1994-2015 Lua.org, PUC-Rio
luap 0.7 Copyright (C) 2012-2015 Dimitris Papavasiliou, Boris Nagaev
>  g = require('luagraphs.data.graph').create(6)

I took the above line from the first example in the README.  But I got an error (marked VVV):
                                                             VVVVVVVVVVV
/usr/local/share/lua/5.2/luagraphs/data/graph.lua:60: module 'data.list' not found:No LuaRocks module found for data.list
        no field package.preload['data.list']
        no file '/home/cxw/.luarocks/share/lua/5.2/data/list.lua'
        no file '/home/cxw/.luarocks/share/lua/5.2/data/list/init.lua'
        no file '/usr/local/share/lua/5.2/data/list.lua'
        no file '/usr/local/share/lua/5.2/data/list/init.lua'
        no file '/usr/share/lua/5.2/data/list.lua'
        no file '/usr/share/lua/5.2/data/list/init.lua'
        no file '/usr/lib/lua/5.2/data/list.lua'
        no file '/usr/lib/lua/5.2/data/list/init.lua'
        no file './data/list.lua'
        no file '/home/cxw/.luarocks/lib/lua/5.2/data/list.so'
        no file '/usr/local/lib/lua/5.2/data/list.so'
        no file '/home/cxw/.luarocks/lib/lua/5.2/data/list.dll'
        no file '/usr/local/lib/lua/5.2/data/list.dll'
        no file '/usr/lib/lua/5.2/data/list.dll'
        no file '/usr/lib/lua/5.2/loadall.dll'
        no file './data/list.dll'
        no file '/home/cxw/.luarocks/lib/lua/5.2/data.so'
        no file '/usr/local/lib/lua/5.2/data.so'
        no file '/home/cxw/.luarocks/lib/lua/5.2/data.dll'
        no file '/usr/local/lib/lua/5.2/data.dll'
        no file '/usr/lib/lua/5.2/data.dll'
        no file '/usr/lib/lua/5.2/loadall.dll'
        no file './data.dll'

Stack trace:
        #0 [C]: in function ?
        #1 [C]: in function 'require'
        #2 /usr/local/share/lua/5.2/luagraphs/data/graph.lua:60: in function 'create'

graph.lua:60 is g.vertexList = require('data.list').create().  The data.list was causing it to look for a package called data with a module called list, not data.list within luagraphs.
        #3 lua:1: in the main chunk
        #4 [C]: in function 'enter'
        #5 /usr/local/lib/luarocks/rocks/luaprompt/0.7-1/bin/luap:210: in the main chunk
        #6 [C]: in function ?

I checked my path:
cxw:~/src$ luarocks path
export LUA_PATH='/home/cxw/.luarocks/share/lua/5.2/?.lua;/home/cxw/.luarocks/share/lua/5.2/?/init.lua;/usr/local/share/lua/5.2/?.lua;/usr/local/share/lua/5.2/?/init.lua;/usr/share/lua/5.2/?.lua;/usr/share/lua/5.2/?/init.lua;/usr/lib/lua/5.2/?.lua;/usr/lib/lua/5.2/?/init.lua;./?.lua'
export LUA_CPATH='/home/cxw/.luarocks/lib/lua/5.2/?.dll;/usr/local/lib/lua/5.2/?.dll;/home/cxw/.luarocks/lib/lua/5.2/?.so;/usr/local/lib/lua/5.2/?.so;/usr/lib/lua/5.2/?.dll;/usr/lib/lua/5.2/loadall.dll;./?.dll'

and I confirmed that my LUA_PATH matched that shown.
I am not an expert on Lua.  I looked at various projects, and as far as I can tell, a common practice is to put the modules in ./src/<project name> and require('<project name>.<package>').  Busted expressly supports this practice by adding ./src/?.lua to the LUA_PATH unless you specify a -m option.  Therefore, this pull request moves the directories in src to src/luagraphs, and updates all the require statements to begin with luagraphs..  On my system, I am now able to run the examples from the README after installing with luarocks make.  I also tried a test of luarocks install from my own repo, and that worked fine, as did luarocks install --local.  busted spec --verbose reports 21 successes / 0 failures / 0 errors / 0 pending.
Separately, I also added assert statements to spec/TopologicalSort_spec.lua, since topo-sort is part of my use case.
Thanks for considering this request!

Thank you!  Works fine with 1.0-5.


fix: Replace deprecated dataset in tutorials and docs

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AWS CodeBuild CI Report

CodeBuild project: GitHubCodeBuild8756EF16-4rfo0GHQ0u9a
Commit ID: e8b52af17aae745a79001455be214adbf034a21c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubDistributedCodeBuild6-jWcl5DLmvupS
Commit ID: e8b52af17aae745a79001455be214adbf034a21c
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


Including an SVG in a template word document causes word doc to become corrupted

Steps:

Load an existing document with an SVG into NPOI. When creating the existing document, either insert SVG from SVG file or go to picture art, stock images, and insert any of the pre existing icons.
Write out NPOI  results to file
Attempt to open word document, word will report that the file has been corrupted

Thank you for your time. Let me know if I can provide more information. This was tested using version 2.6.0.
If you load in an SVG via NPOI API add picture, its able to save just fine. Its when there is a pre existing SVG in a word doc that is loaded into NPOI is where the problem arises.

This might be more nuanced then I though, I need to investigate more since I can't consistently replicate


v1.14: update cilium/certgen to v0.1.11

Backport of https://github.com/cilium/cilium/pull/31863 to the v1.14 branch.

/test-backport-1.14


PackageDir doesn't work

It seems that the PackageDirectory option was only half migrated, changed to PackageDir everywhere except in the actual package manager code.
Also that directory gets inserted in sys.path but vistrails.packages still gets imported. Because of the way Python imports work (and the __path__ feature), this will still not import from that directory.

There seem to be more problems with paths, most of them have defaults that are actually relative to .vistrails, but the code doesn't assume that.

The dataDir and fileDir options are also useless, because core.system doesn't actually use those (only maintains variables __dataDir and __fileDir. The fileDir option gets updated, but never read; dataDir is never touched)
I would change core.system to use the configuration, but that would break at startup. Or should we just remove dataDir and fileDir from configuration?

Where are the underscore variables used? If they are not important, I would vote to eliminate them.
Why would it break if you change core.system?

I am referring to the variables in system/__init__.py
core.system is used before the configuration is set up, so relying on the configuration there is probably not right.
dataDir and fileDir are only keeping track of the last place files where opened from/saved to, so there is not a lot of value in having them persist anyway.

Do they have to be used before the config is set up, or could the ordering be changed?
Otherwise, I'm fine with eliminating them. In any case, they shouldn't be in the GUI preferences (not sure if they are).

It looks that those are probably safe to lookup in the configuration (vistrails_root_directory() and the like are less safe).
The help strings for those were "default vistrail directory"/"default data directory", so it's what #1209 changes them to do.


Frontend: Add nil check for every request

Currently in frontend APIs, we do not check if request arguments are nil or not.
So when someone call frontend directly without request arg, cadence will crash.
Frontend should check and ensure args are not nil before processing.

https://github.com/uber/cadence/pull/642


Init throws "Unable to locate executable file 'az' error" for "Hosted" and "Windows Container" hosted build agents

The following error is thrown for the terraform init command for hosted build agents of types "Hosted" and "Windows Container".
Error: Unable to locate executable file: 'az'. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also verify the file has a valid extension for an executable file.

Steps to Reproduce

Create a pipeline that uses a hosted build agent of types "Hosted" or "Windows Container"
Add terraform init command and check "Create Backend (if not exists)"
Execute the pipeline.

This only occurs for init command if "Create Backend..." is checked.
Suggested Workarounds

Use Hosted 2017 or 2019 build agents
Install azure cli via npm globally before executing terraform init. npm install -g azure-cli.

Closing this as wont fix due to azure devops limitation. The task at this point in time will not assume responsibility for installing azure cli on the agent. All other agents have this installed by default and the ones that dont will at some point in the future.
Leaving conversation open in-case someone encounters this that cannot use the two suggested work arounds.


Feature(command-deploy): support passing a branch when triggering a deploy

Is your feature request related to a problem? Please describe.
Related to https://github.com/netlify/cli/issues/275#issuecomment-730218600
netlify deploy --trigger doesn't support passing an --alias.
Describe the solution you'd like
Ability to trigger a deploy for a specific existing branch/PR.
Describe alternatives you've considered
N/A
Additional context
See https://github.com/netlify/cli/issues/275#issuecomment-638701080
This will require the API to support it.
Can you submit a pull request?
Yes

Any update on this feature? Can we expect it to be implemented?

Hi @matyasf, no timeline for this yet.
The CLI uses https://open-api.netlify.com/#operation/createSiteBuild to endpoint to trigger a deploy which currently doesn't support passing a branch.
We'll first need to add this to our API.

This feature would be really nice to get branch subdomains working out of the box when using Netlify DNS. We are running netlify deploy --alias=$CI_COMMIT_REF_SLUG for branches via Gitlab CI to get review apps/URL's for merge requests.

We'll first need to add this to our API.
Are there any plans to update this API to allow this, or this just in the backlog? I'm wondering in case we want to replace Netlify DNS with our own logic to make this work or not.


Change usedCapturedValues in BailOutInfo to a pointer for consistency

As part of LazyBailOut, we want to perform a shallow copy of a BailOutInfo which includes sharing all field pointers. However, usedCapturedValues is the only field in BailOutInfo that is not a pointer and is initialized together with CapturedValues, which makes it more complicated to do so as we have to perform manual copy of each field inside CapturedValues. With this change, we are more consistent with what we already have and also make it easier for future LazyBailOut work.

@dotnet-bot test please


Add link to cache rules

Caching seems like an integral part for R2, and so, add a link to emphasize that.
Perhaps, the R2 docs need a Caching section all on its own, but I am ill-placed to start one; and hence this PR, in its stead.

Ah, I don't know just how to link to cache rules. But it'd be rad if someone who knew how, added it. Closing this PR.


Loading problems of the worlds

I've now tested your mod does not work with maps mods like voxelmap ect
you should fix that
I've now flown back and forth in the world for a while without voxelmod then ran and loads everything
please fix that one quickly

Please use the issue template and provide sufficient information:
**Description**
Please describe the problem clearly and concisely

**How to reproduce**
- Describe your setup if using anything other than the Mojang launcher or standard forge server
- List any third-party mods (& their version number) that may be relevant to the issue
- Provide a simple step-by-step guide to recreate the problem

**Additional information**
Add screenshots, world-seeds, co-ordinates, log files, links to conflicting mods

mod louder Forge
Mod TerraForged & Voxelmap
launcher_log.txt
I do not know more or can not serve with more have just taken another map mod it did not even run 3m

Ok so you're running with many mods.... One of the ones that adds structures or world-gen features is probably freezing the chunk generator causing the chunks to stop loading. You'd need to go through and test with and without each one.
Again, if you can demonstrate the issue with just TerraForged then let me know.


Add reference to RequireAssertion()

Addresses #6458

@spottedmahn A few touch-ups to conform to our style guidelines. Check to see if the text is still ok.

Thanks @spottedmahn!

  Love the quick turn-around and community involvement!


Devel

update to 0.6.5 patch level 2 (and url download anomaly handling)
molecule/verify: validations for velociraptor server
lint
misc

@juju4 Thanks for working to keep this role up to date.


Re-organise backend files

Issue
The current file organisation in the src/tqec folder does not make a lot of sense.
Tree-view of the tqec backend files
src/tqec
├── detectors
│   ├── measurement_map.py
│   ├── operation.py
│   └── transformer.py
├── display.py
├── enums.py
├── exceptions.py
├── generation
│   ├── circuit.py
│   ├── circuit_test.py
│   └── topology.py
├── noise_models
│   ├── after_clifford_depolarization.py
│   ├── after_reset_flip.py
│   ├── base.py
│   ├── before_measure_flip.py
│   ├── before_round_data_depolarization.py
│   ├── idle_qubits.py
│   ├── __init__.py
│   └── multi_qubit_gates.py
├── non_template_scaling
│   ├── animated_shape_display.py
│   ├── display_data.py
│   ├── __init__.py
│   └── non_template_scaling.py
├── plaquette
│   ├── library
│   │   ├── __init__.py
│   │   ├── xx.py
│   │   ├── xxxx.py
│   │   ├── zz.py
│   │   └── zzzz.py
│   ├── plaquette.py
│   ├── qubit.py
│   └── schedule.py
├── position.py
├── position_test.py
├── templates
│   ├── atomic
│   │   ├── rectangle.py
│   │   ├── rectangle_test.py
│   │   ├── square.py
│   │   └── square_test.py
│   ├── base.py
│   ├── composed.py
│   ├── constructions
│   │   ├── corner.py
│   │   └── qubit.py
│   ├── scale.py
│   └── scale_test.py
└── _version.py

Enhancement
It would be nice to re-organise the file hierarchy above to have a file organisation that makes more sense.
Example of files that, in my opinion, are not in the right place:

plaquette/schedule.py,
all the detectors/ folder,
display.py can likely go in the templates/ folder,

Additionally, the non_template_scaling folder might not be relevant anymore.
Any opinion on that? I would love to have the most feedback possible in order to do these changes only once.

The structure should definitely be refactored, though I don't have strong opinions for now. A few first impressions without deep thought:

Those circuit related modules can go to the same folder, i.e. plaquette/schedule.py, generation/circuit.py. Even the modules in the detectors folder can also be placed in the same place as those.
templates/constructions can be renamed as composed in contrast to atomic. These pre-defined templates can be placed in a folder like templates/library to make their intent more clear.

So for the moment, the changes are basically:

templates/constructions/ -> templates/library/
Create a circuit (idea for a better name? maybe cirq?) folder and move in it

plaquette/schedule.py
generation/circuit.py
the content of tqec/detectors/

display.py -> templates/display.py


Benchmarking CLI should have API to Return Available Benchmarks / Pallets for Automation

Benchmarking CLI should have some commands / API to tell the outside world what available pallets, benchmarks, and maybe even runtimes (strech goal for polkadot) are available to run.
This can be used inside of automated bots or tools to trigger all the benchmarks for a runtime automatically.
For example, when releasing a new version of Substrate, we should be able to do:
/benchmark runtime all

And it would know how to trigger all the benchmarks available in that runtime.
Something like:
cargo run --release --features=runtime-benchmarks -- benchmark --list

Returns:
pallet, benchmark
frame_system, remark
...
pallet_sudo, sudo

@shawntabrizi I can work on this. I think I should start by changing this file?


Highlight return in the same way as function

The word return currently appears in the same color as normal text. It should appear in the same color as function.

return is more or less a regular function in R where function requires parser support.
Seems a bit cosmetical to me.


Thought/Action Item Edit - Add Countdown

Feature Request
The Thought Edit character set is limited.  This request is to add a countdown so a user can understand how many characters they have remaining.
Story
As a participant of a retrospective
I want to understand the number of characters remaining when editing an existing Thought or Action Item
So that I do not get frustrated with the board suddenly stops accepting new characters as I am creating a new Thought or Action Item
Acceptance Criteria
Given the participant is a member of a retrospective
When they click on the functionality that lets them edit an existing Thought or Action Item
Then they are provided with a visual aid that indicates to them the amount of characters they have remaining before the board stops accepting new characters.
Design
This optimal manner in which to provide a visual aid to the user needs to be investigated.

This has been added and migrated to PROD.


update gradle to 2.13

fixed publishing with local signing (jars and pom) and upload to bintray
This is a way to complicated. nebula publishing dropped the signing stuff after
bintray makes it possible to sign the artifacts automatically with their key or
with a private/public keypair linked and stored within the repository.
See https://github.com/nebula-plugins/nebula-publishing-plugin/issues/61 and
https://bintray.com/docs/usermanual/uploads/uploads_gpgsigning.html

add singned pom to publication only if not skipped.

set name and description for pom

fallback to buildscript block to apply plugins only to the subprojects

oh no...the assemble task does not run on the ci server because it has no configured signatory. i'll fix that.

This time it's the test-asciidoctor-upstream script that fails. The tests succeeded.

The failures with openjdk may relate to https://github.com/travis-ci/travis-ci/issues/5227

I think the failing OpenJDK test can be ignored, they are already failing for a couple of months now.
Crazy that this still isn't fixed. Thanks for finding this issue though!!!
Regarding the upstream test I also tested on the current master which worked recently.
Maybe a asciidoctor 1.5.5.dev is really not available atm.

I am wondering about the signing process.
If I read the documentation correctly automatic signing by Bintray has to be activated for the whole Maven repository of the Asciidoctor organization, which already contains 13 different packages.
I don't know if we can simply activate this for all packages in there like asciidoclet, etc.
(I don't seem to have the rights for that)
@mojavelinux Do you have some thoughts on this?
I would definitely love if the signing process could be simplified.

I would definitely love to see an upgrade of Gradle!!!!

Maybe I got the description wrong, when doing a release build I get valid signatures :-)

Hmm...the compatibility stuff drives me crazy. Don't know why it doesn't work for subprojects.
I think it's best to rollback to the workaround and hope the best for gradle 2.14 though.
Very weird....
And yes. The local signing works as expected. The only change is the switch from nebula-publish to the build-in maven-publish, combined with the maven and signing plugin.

I just checked that the classes are now compiled for Java 1.6 again.
@mojavelinux Could you take a look at this? You spent so much time on the publishing plugin that you can probably judge better.

Squashed and cherry-picked into the 1.6.0 branch and made an alpha.2 release: 9ac7714a4fc4d5ee70eae0bbbdcaa25c13ec0105.
Would be great if you could have a look at it.
What I found a bit strange is that the release goes immediately live.
For the past releases I always had to confirm the new artifacts.
Could this be due to the configuration of the publishing plugin or could this be a change at Bintray?

Just seen that in the build that I moved over to the 1.6.0 branch a pom.xml got generated for the distribution which overwrites the original asciidoctorj pom.xml.
Therefore asciidoctorj has deps now on asciidoctorj-diagram, asciidoctorj-pdf and asciidoctorj-epub3.
Will take a look tomorrow what's gone wrong.

@ancho Would be great if you could help out on this and see what I've done wrong on the AsciidoctorJ 1.6.0 branch.
My knowledge about Gradle publishing is too limited.
Major differences to the current master are the support for Snapshot deployments and the documentation module which is not published.
I can only guess the hell @mojavelinux went through when crafting the initial publishing for AsciidoctorJ.

Sorry for answering so late. I'll have a look at it to find a solution. It is strange that the jar for the distribution project is not created, as the jar task is disabled but the pom get generated anyways.
The auto publish is configured in the deploy.gradle file
Maybe it didn't work before. I didn't change that part of configuration.

Thanks! That would be cool.
It is very likely that I've done sth wrong, but I'm not really proficient with this.
You can also see the result when you take a look at the pom here: https://jcenter.bintray.com/org/asciidoctor/asciidoctorj/1.6.0-alpha.2/asciidoctorj-1.6.0-alpha.2.pom

I found the problem for the override of the asciidoctorj pom with the pom form the distrbution.
To prevent it I do not apply the publishing stuff to the distribution package.
There is no need anyway, as the distribution package is not a jar with dependencies on it.
But I couldn't figure out what is wrong with the asciidoctorj-1.6.0 configuration yet ;(
I will continue to spot the differences in the evening.

I tested it the following way:

created a maven repsotory at bintray  called experiment
export BINTRAY_REPO=experiment; ./gradlew -PbintrayPersonal bU

See: https://bintray.com/ancho/experiment/asciidoctorj/1.5.5/view#files/org/asciidoctor/asciidoctorj/1.5.5
It is not synchronized with jcenter or mavencentral. Don't panik :)

Hehe, no panic :-)
Yep, for the master branch the pom looks correct.
Only for the stuff that I probably broke on the 1.6.0 branch the pom is apparently overwritten by a pom created for the distribution.

Keep up the great work!

Thanks for all the hard work on that! 🍻
This is really awesome!
I will also upgrade the 1.6.0 branch to Gradle 2.14 and the Bintray plugin 1.7.
Strange thing though is it seems like artifacts are uploaded twice: https://gist.github.com/robertpanzer/4c020ec0a2ef5e2fed63f596a2ddf59c
(And it also takes the time to upload the files)
But shouldn't be a problem as long as in the end the right files are in the repo.

That's really strange and should be avoided. o.O
I'll check that an see what I can do...
Thanks for the appreciation, I'm just glad to help out and get the chance to give something back ;)


Not working with Platform api

It doesn't work with react-native-router-flux and other packages using Platform api. The error is following:

React: 15.3.2, react-native-macos: ^0.10.5
Using Mac

I suppose that's because they require react-native directly https://github.com/aksonov/react-native-router-flux/blob/b11a84962e4e4d9265770382b8859d21d1a12ea0/src/NavBar.js#L37. Unfortunately, there is no good solution right now for aliasing this, only manually change (fork) the library and abstract react-native. React Native macOS might be updated as a pluggable thing to react-native in the future to support all third-party plugins.

@ptmt, maybe we can write a babel plugin to transform react-native to react-native-macos for import / require? We can add it to transformer by default. Although there may be have a better way.

Have the same issue, maybe someone know good router for using with 'react-native-macos'

@hmelenok This going to be resolved with https://github.com/ptmt/react-native-macos/pull/125 by @jhen0409 , you can try it right now and provide a feedback.

Seems the same now, it not found it in your version of RN

I had use fork from git+https://github.com/jhen0409/react-native-desktop.git#patch-2
.babelrc:
{
  "presets": [
    "react-native"
  ],
  "plugins": [
    [
      "module-resolver",
      {
        "alias": {
          "react-native": "react-native-macos",
        },
      }
    ],
  ],
}

.flowconfig:
...
module.name_mapper='^[./a-zA-Z0-9$_-]+\.\(bmp\|gif\|jpg\|jpeg\|png\|psd\|svg\|webp\|m4v\|mov\|mp4\|mpeg\|mpg\|webm\|aac\|aiff\|caf\|m4a\|mp3\|wav\|html\|pdf\)$' -> 'RelativeImageStub'
module.name_mapper='react-native' -> 'react-native-macos'
...

seems as alias working but without Platform

Added Platform.select support in 0.11.0-rc4. PR for babel transforming is still under testing

0.12.0 now have babel-transform plugin


concurrent map writes error

It occurs when I use multi window in iTerm with broadcast input
related issue:
https://github.com/spf13/viper/issues/268

fixed for single process - multi thread. working on multi process - multi thread. it may requires some lockfile mechanism

resolved at v1.0.0 for multi thread. still have issue with multi process


Add a Network / Host / Dns module

This module could provide network tests like ping / open tcp udp connections / dns resolution.
Example:
assert host.network.resolve('google.com', type='NS', server='<IP_ADDRESS>') == set(['ns{}.google.com'.format(i) for i in range(1, 5)])
assert host.network.resolve('example.com', type='A').ttl = 1982
# ---> this could be implemented by using a smart object that resolve to a host set with additional attributes
assert host.network.reachable('tcp://google.com:443')
assert host.network.reachable('icmp://google.com')
assert host.network.not_reachable('tcp://google.com:22', timeout=3)

I think the implementation could be a mix of dig, nc, ping.
Please feel free to explain what you want to do with such module.

Also we should see what feature has goss, inspec and serverspec, there might be good idea to reuse.

Thank you for your input, I will look into this.

Looking at goss, it seems that they do what i was proposing to do, 1 module for dns tests and another for interface related tests. For now they have one assertion afaik, and that's to test if the record is resolvable, which is more or less what I had implemented. I understand that it was not enough but my hope at the time was that it would gain momentum and lead to more assertions being required and implemented at a later time.
As far as inspec goes I cannot see a dns or even much of network tests apart from what appears to be interface settings, that does seem to be quite elaborate. In both cases they seem to test for things like , speed, state of interface (up or down),  goss is quite simple but inspec seems to implement a few interesting bits.
As far as network goes I think the following assertions should be implemented

ip of interface
Speed
MTU
state (up or down)

In terms of DNS

Assert record is resolvable
Assert record is of expected type
Assert ttl is as expected

In terms of network related assertions

Assert host is pingable
Assert host is recheable on a specific socket (ip address and protocol)

Please share any more ideas you might have .

Speaking of inspec/serverspec: you can take a look at their implementation of similar feature here:
https://github.com/mizzy/specinfra/blob/master/lib/specinfra/command/base/host.rb
https://github.com/inspec/inspec/blob/master/lib/resources/host.rb
Also please note that some systems don't have netcat installed by default, furthermore some consider having netcat installed on server as a potential security breach (mainly due to it's ability to listen on some port and execute arbitrary stuff), it seems it's the reason why Inspec implemented port checks using bash as a fallback:
https://github.com/inspec/inspec/blob/master/lib/resources/host.rb#L155

@philpep

Please feel free to explain what you want to do with such module.

My main use case is testing routing and vpn configurations, making sure that everything works as expected.

@ruimoreira
+1 for adding something like is_up to interface. Speaking of additions to existing modules - it would also be useful to improve Socket module so it can check that socket is not just listening, but is opened by specific process.

@skhoroshavin @philpep  what's your opinion regarding keeping these two features (dns and networking ) separate or should we use the same module for both tasks ?

@ruimoreira I think they should be same module, in my opinion this will lead to more compact and easy to read tests.

Thank you @ruimoreira and @skhoroshavin for digging into what other tools are doing !
I also feel that a single module (let's say "network") would be easier to work with, not a strong opinion though.
This new module could also replace the "interface" module (which is quite poor, only 3 functions).
+1 for having an integration between socket and process module, I opened #367 but I feel this new network module might also need such integration.

@philpep I like idea of creating single "network" module (but probably name like "remote" would be better, because it's focused on checking whether some remote host is resolvable or reachable), but I don't think that merging into it functionality of "interface" module is right. Interface module is about querying state of local network interfaces and while it would be fine to extend its functions I think it doesn't overlap at all with "network/remote"

I was wondering how we can collaborate, I would like to contribute to this module, and more modules in the future ... any ideas ?

@ruimoreira @skhoroshavin feel free to open PRs. Now we have a plan about how the module should be implemented, I'm ok to add features incrementally to it, so a first try implementing "revolve()" or "reachable()" with basic tests is fine to me. For the module name, I still prefer "network" than "remote", but I'm open to other suggestions.
And, sorry, it seems lot of contributors to testinfra feel that submitting PRs and asking questions on issues are like throw a stone in water. But I maintain testinfra on my free time and never get paid for. Actually reviewing a PR takes me a lot of time to me because I'm quite rigorous regarding regressions or broken new features which may generate even more work after being merged. So new features are very "low priority" for me. On another side, I understand contributors expect a fast feedback, I'll try to take at least 5 min per week for that.
But I really appreciate how this new module is designed here, so I'm waiting for a PR :)

@philpep Finally there is a PR #402, hopefully you'll find some time to review

Tests are still failing, however it would be great to have some early feedback if my approach is good enough.

@philpep I've updated PR with documentation and usability improvements


late binding issues

First of all thanks for the great work....
I am using AnglularJS and with the following example ( lots of attributes omitted for brevity)

The chart does not display the values as the chart is drawn before the scope variables are resolved.
Do you know a simple/elegant way to convert your pure jQuery library to a more AngularJS friendly one?
Thank you in advance for the help

Can you try it again with the new plugin that is now based on svg?!


hmac: Add support to return base64 encoded HMAC.

Keeping the now kind of obsolte binary argument and adding a new encoding argument to allow directly encoding the resulting HMAC as base64.
This is needed as some APIs require base64 encoded HMACs, which are currently not possible to create, as starlib's base64 encoding only operates on strings.

Should I check for values other than "hex", "binary" and "base64" and error out?


Make a switcher to disable metric storage

Metric Storage requires persistence storage which is not always possible in test environments. This patch makes a switcher to allow disable metric storage until persistance/ephemeral switcher is inmplemented in telemetry controller per [1]
[1] https://issues.redhat.com/browse/OSPRH-9252
Closes-Bug: OSPRH-9169

What is supposed to happen with configured telemetry if there is no storage provided?

What is supposed to happen with configured telemetry if there is no storage provided?

There is no configured telemetry. Also there is no data migration between 17.1 telemetry and 18.0. It means that telemetry is installed from scratch in 18.0. If we disable Metric Storage this component won't work which is fine until ephemeral storage is implemented.

The idea of entering adoption was that if the user adopts the data plane, we want telemetry to be installed there by default. Can it be enabled as a day-2 post adoption service? Sure, but after the multitude of environments without metrics that we found on 17.1 and previous (telemetry being a day-2 optional step), there has been a strong push from product management to get telemetry enabled by default in any circumstance.
Adopting an existent control plane "should" entail deploying the telemetry control plane and the telemetry agents on the new adopted adataplanes by default, and thats why we have commited this.
This is definitively debatable, but this is the reason.


How do i create a dynamic annotation in line chart with annotation section ?

By default Apex Charts displayed static annotation under line chart section . we want to create a dynamic annotation along x-axis. is there any way to achieve this functionality ?

If you want to create annotations on run time, you can call the addXaxisAnnotation() method.
var chart = new ApexCharts(el, options);
chart.addXaxisAnnotation({
  x: 40,
  label: {
    text: 'Lorem Ipsum'
  },
})


159 anomaly

Anomaly calculation operations have been overhauled and are now properly tested.

Closes #159

Looks like AppVeyor tests are failing because I'm trying to write to disk as part of some tests. Note that some tests absolutely require this (e.g., testing dask integration).

Codecov Report

Merging #165 into master will increase coverage by 0.52%.
The diff coverage is 98.76%.

@@            Coverage Diff             @@
##           master     #165      +/-   ##
==========================================
+ Coverage   72.52%   73.04%   +0.52%
==========================================
  Files          67       67
  Lines        8105     8151      +46
==========================================
+ Hits         5878     5954      +76
+ Misses       2227     2197      -30

Impacted Files
Coverage Δ

cate/ops/anomaly.py
100% <100%> (+52.94%)
:arrow_up:

cate/ops/__init__.py
100% <100%> (ø)
:arrow_up:

cate/ops/subset.py
97.4% <100%> (-0.19%)
:arrow_down:

cate/ops/arithmetics.py
97.95% <96.15%> (+76.53%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f832572...66fea6f. Read the comment docs.

@JanisGailis maybe writing the tests like in xarray may help: https://github.com/pydata/xarray/blob/master/xarray/tests/test_backends.py#L497

Yes, this is exactly what came to my mind as well :) Will see if it works!

Implemented their solution - fancy error ignoring when on Windows :)

If anyone gets round to merging these, it's probably a good idea to start reviewing/merging the older PRs first, to avoid possible conflicts.

Hi Janis, I merged the other 2. Please resolve this one to go ahead!

Resolved!

I'll take the liberty to merge this based on your previous comment.


List all stories on /

I'm working on this. Looks like I don't have permission to assign it to myself.

I have created PR #20 to tidy up the API a bit, but haven't made any real progress on the UI. If anyone wants to work on this during the day tomorrow, please feel free. Otherwise, I'll pick it up again in the evening.

🎉 Big ❤  to @davesteinberg on this one.


use kpt git parsing in rpkg clone

For kpt alpha rpkg clone, make using the --ref and --directory flags optional when using git URLs, instead allow something like
kpt alpha rpkg clone https://github.com/GoogleCloudPlatform/blueprints.git/catalog/bucket@main cloned-bucket \
  --repository=deployments \
  --namespace=default

Throw errors if the ref/directory are both present in the URL and set via flags, and they don't match.

Thanks for the reviews and answering all my questions!