Binance Brokerage implementation

Description

New Binance brokerage implementation (only Spot Trading support for now)

N.B. The following issues need to be addressed for the implementation to work properly:

[ ] The SetAccountCurrency method is not enabled yet for live trading (#110):
https://github.com/QuantConnect/Lean/blob/master/Engine/Setup/BrokerageSetupHandler.cs#L222-L227

[x] Missing pairs in EnsureCurrencyDataFeed for direct conversion to account currency:
we are currently using an incomplete list here:
https://github.com/QuantConnect/Lean/blob/master/Common/Currencies.cs#L42
we are working on a solution reusing the symbol properties database (#4710)

[x] Non-tradable currencies (from airdrops/distributions) need to be handled in EnsureCurrencyDataFeed (setting the conversion rate to zero without terminating the algorithm)
e.g. ADD, ATD, EON, EOP, MEETONE

[ ] Indirect currency conversions:
e.g. for GAS currency we only have the GASBTC pair, for USDT account currency we would need two pairs:
GASBTC and BTCUSDT

[ ] The default benchmark (SPY) requires the USD currency which is not available

Related Issue
Closes #2004
Resubmit of #3511 (thanks @AdalyatNazirov)
Motivation and Context

Largest crypto exchange

Requires Documentation Change
No.
How Has This Been Tested?

Local testing with live algorithm + unit tests

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Refactor (non-breaking change which improves implementation)
[ ] Performance (non-breaking change which improves performance. Please add associated performance test and results)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Non-functional change (xml comments/documentation/etc)

Checklist:

[x] My code follows the code style of this project.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.
[x] My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

any news on margin trading implementation?

any news on margin trading implementation?

Despite i have removed the SetLeverage(1) from my Python Algo .
20210105 22:54:58.565 ERROR:: WorkerThread.<.ctor>b__7_0(): WorkerThread(): exception thrown when running task Python.Runtime.PythonException: ArgumentException : The Binance brokerage does not currently support Margin trading.
  at QuantConnect.Brokerages.BinanceBrokerageModel..ctor (QuantConnect.AccountType accountType) [0x00015] in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Common/Brokerages/BinanceBrokerageModel.cs:42
  at QuantConnect.Brokerages.BrokerageModel.Create (QuantConnect.Securities.IOrderProvider orderProvider, QuantConnect.Brokerages.BrokerageName brokerage, QuantConnect.AccountType accountType) [0x0006e] in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Common/Brokerages/IBrokerageModel.cs:193
  at QuantConnect.Algorithm.QCAlgorithm.SetBrokerageModel (QuantConnect.Brokerages.BrokerageName brokerage, QuantConnect.AccountType accountType) [0x00001] in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Algorithm/QCAlgorithm.cs:985
  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00032] in <b0e1ad7573a24fd5a9f2af9595e677e7>:0
  at Python.Runtime.PyObject.Invoke (Python.Runtime.PyTuple args, Python.Runtime.PyDict kw) [0x00033] in <c56ab175820d412caf052e079c2ab9ef>:0

Despite i have removed the SetLeverage(1) from my Python Algo .
20210105 22:54:58.565 ERROR:: WorkerThread.<.ctor>b__7_0(): WorkerThread(): exception thrown when running task Python.Runtime.PythonException: ArgumentException : The Binance brokerage does not currently support Margin trading.
  at QuantConnect.Brokerages.BinanceBrokerageModel..ctor (QuantConnect.AccountType accountType) [0x00015] in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Common/Brokerages/BinanceBrokerageModel.cs:42
  at QuantConnect.Brokerages.BrokerageModel.Create (QuantConnect.Securities.IOrderProvider orderProvider, QuantConnect.Brokerages.BrokerageName brokerage, QuantConnect.AccountType accountType) [0x0006e] in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Common/Brokerages/IBrokerageModel.cs:193
  at QuantConnect.Algorithm.QCAlgorithm.SetBrokerageModel (QuantConnect.Brokerages.BrokerageName brokerage, QuantConnect.AccountType accountType) [0x00001] in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Algorithm/QCAlgorithm.cs:985
  at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke(System.Reflection.MonoMethod,object,object[],System.Exception&)
  at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00032] in <b0e1ad7573a24fd5a9f2af9595e677e7>:0
  at Python.Runtime.PyObject.Invoke (Python.Runtime.PyTuple args, Python.Runtime.PyDict kw) [0x00033] in <c56ab175820d412caf052e079c2ab9ef>:0

I have set the AccountType.Cash and here what I get
20210105 23:15:09.159 Trace:: DefaultAlphaHandler.Exit(): Exiting...
20210105 23:15:09.163 Trace:: DefaultAlphaHandler.Exit(): Ended
20210105 23:15:09.168 ERROR:: During the algorithm initialization, the following exception has occurred: System.ArgumentException: In order to maintain cash in BTC you are required to add a subscription for Forex pair BTCUSD or USDBTC
  at QuantConnect.Securities.Cash.EnsureCurrencyDataFeed (QuantConnect.Securities.SecurityManager securities, QuantConnect.Data.SubscriptionManager subscriptions, System.Collections.Generic.IReadOnlyDictionary`2[TKey,TValue] marketMap, QuantConnect.Data.UniverseSelection.SecurityChanges changes, QuantConnect.Interfaces.ISecurityService securityService, System.String accountCurrency, QuantConnect.Resolution defaultResolution) [0x0041f] in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Common/Securities/Cash.cs:286
  at QuantConnect.Securities.CashBook.EnsureCurrencyDataFeeds (QuantConnect.Securities.SecurityManager securities, QuantConnect.Data.SubscriptionManager subscriptions, System.Collections.Generic.IReadOnlyDictionary`2[TKey,TValue] marketMap, QuantConnect.Data.UniverseSelection.SecurityChanges changes, QuantConnect.Interfaces.ISecurityService securityService, QuantConnect.Resolution defaultResolution) [0x00023] in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Common/Securities/CashBook.cs:123
  at QuantConnect.Lean.Engine.DataFeeds.CurrencySubscriptionDataConfigManager.EnsureCurrencySubscriptionDataConfigs (QuantConnect.Data.UniverseSelection.SecurityChanges securityChanges, QuantConnect.Brokerages.IBrokerageModel brokerageModel) [0x0002c] in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/CurrencySubscriptionDataConfigManager.cs:140
  at QuantConnect.Lean.Engine.DataFeeds.UniverseSelection.EnsureCurrencyDataFeeds (QuantConnect.Data.UniverseSelection.SecurityChanges securityChanges) [0x00000] in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/UniverseSelection.cs:454
  at QuantConnect.Lean.Engine.Setup.BaseSetupHandler.SetupCurrencyConversions (QuantConnect.Interfaces.IAlgorithm algorithm, QuantConnect.Lean.Engine.DataFeeds.UniverseSelection universeSelection) [0x00006] in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/Setup/BaseSetupHandler.cs:50
  at QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler.Setup (QuantConnect.Lean.Engine.Setup.SetupHandlerParameters parameters) [0x005c7] in /LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/Setup/BrokerageSetupHandler.cs:382 In order to maintain cash in BTC you are required to add a subscription for Forex pair BTCUSD or USDBTC
20210105 23:15:09.168 ERROR:: LiveTradingResultHandler.Update(): Algorithm not yet initialized.
20210105 23:15:09.169 Trace:: BaseDataExchange(CustomDataExchange).ConsumeQueue(): Cancellation requested.  Exiting...
20210105 23:15:09.176 Trace:: StopSafely(): waiting for 'Result Thread' thread to stop...
20210105 23:15:09.269 Trace:: Debug: Binance account base currency: USD
20210105 23:15:09.369 Trace:: LiveTradingResultHandler.Run(): Ending Thread...

Hello. Is there any plans to add Binance Futures? Thank you.


ISPN-12734 Log transaction information on lock timeout

https://issues.redhat.com/browse/ISPN-12734

I think I'd rather separate this into two:

Log an error whenever we discover a leaked lock

Why? except if you have any other action possible when a "lock leaked", I don't see any reason to not automate it.

Add operations in JMX/CLI/REST/web console/etc. to list locks held for "a long time", and to release a lock regardless of who's holding it at the time.

There is already a JIRA for point 2: https://issues.redhat.com/browse/ISPN-12735. I'm not planning to do anything about non-tx caches.

updated

@pruivo this failure looks related:
13:47:40,564 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed: org.infinispan.distribution.rehash.OptimisticPrimaryOwnerCrashDuringPrepareTest.testPrimaryOwnerCrash
java.lang.AssertionError: Wrong exception thrown: expected:<org.infinispan.util.concurrent.TimeoutException>, actual:<org.infinispan.util.concurrent.LockTimeoutException>
	at org.infinispan.commons.test.Exceptions.assertException(Exceptions.java:24) ~[classes/:?]
	at org.infinispan.commons.test.Exceptions.assertException(Exceptions.java:70) ~[classes/:?]
	at org.infinispan.commons.test.Exceptions.expectException(Exceptions.java:100) ~[classes/:?]
	at org.infinispan.distribution.rehash.OptimisticPrimaryOwnerCrashDuringPrepareTest.testPrimaryOwnerCrash(OptimisticPrimaryOwnerCrashDuringPrepareTest.java:67) ~[test-classes/:?]

@danberindei Is this Ok to be merged?

closing this for now. I'm not planning to work on this anytime soon :(


Issues with nvidia device plugin

Environmental Info:
K3s Version:

1.22.3-rc4+k3s1
Node(s) CPU architecture, OS, and Version:

Linux metal01 5.4.0-89-generic #100-Ubuntu SMP Fri Sep 24 14:50:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Cluster Configuration:

1 control-plane node, 3 agents
Describe the bug:

nvidia-device-plugin is crashlooping with the following errors:
failed to try resolving symlinks in path "/var/log/pods/kube-system_nvidia-device-plugin-daemonset-twrj5_07b07c46-45aa-4b4d-b30d-06054a939784/nvidia-device-plugin-ctr/1.log": lstat /var/log/pods/kube-system_nvidia-device-plugin-daemonset-twrj5_07b07c46-45aa-4b4d-b30d-06054a939784/nvidia-device-plugin-ctr/1.log: no such file or directory%

Steps To Reproduce:

Installed K3s: via script

Expected behavior:

Nvidia device plugin is not crashlooping
Actual behavior:

Nvidia plugin is crashlooping and GPU is not usable.
Additional context / logs:

I upgraded cluster from 1.21 where GPU was using runc v1 and everything worked fine with custom containerd config. After upgrade and wiping out whole node I was presented with issues regarding NVML initialization. After following what was described in https://github.com/k3s-io/k3s/issues/4070 I got to a state where container cannot be started due to log message mentioned earlier. Other pods on that node using default runtimeClass are working just fine.
At current state I am not sure if this is some issue on my side, nvidia-plugin side, or k3s so any help would be apreciated.
My deployment manifests are available at https://github.com/thaum-xyz/ankhmorpork/tree/master/base/kube-system/device-plugins
Backporting

[ ] Needs backporting to older releases

It looks like some of the logs got cleaned up and it's confusing containerd? Have you tried running k3s-killall.sh and then rm -rf /var/log/pods, followed by starting K3s again? This will of course terminate all running pods but might fix whatever containerd is struggling with.

There is nothing I can think of that would rotate logs. This is a fresh ubuntu 20.04 instance only with k3s and nvidia drivers installed.
I did remove everything from /var/log/pods as wells as cleared whole /var/lib/rancher/k3s/agent and /var/lib/kubelet (actually done it few times in different order and steps). It did not help.
I also get other error messages from k3s which most likely relate to the issue:
Nov 04 11:21:08 metal01 k3s[930011]: E1104 11:21:08.364474  930011 pod_workers.go:836] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"nvidia-device-plugin-ctr\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=nvidia-device-plugin-ctr pod=nvidia-device-plugin-daemonset-twrj5_kube-system(07b07c46-45aa-4b4d-b30d-06054a939784)\"" pod="kube-system/nvidia-device-plugin-daemonset-twrj5" podUID=07b07c46-45aa-4b4d-b30d-06054a939784
Nov 04 11:21:08 metal01 k3s[930011]: I1104 11:21:08.573763  930011 pod_container_deletor.go:79] "Container not found in pod's containers" containerID="64d8ac27c5d1500c469dc62279c24d1b536382579973328e1fd3e96a64ed2201"
Nov 04 11:21:09 metal01 k3s[930011]: W1104 11:21:09.140176  930011 manager.go:1176] Failed to process watch event {EventType:0 Name:/kubepods/pod07b07c46-45aa-4b4d-b30d-06054a939784/ff954893be5edf196f2ccdddd950da868060bd8de2cf7aa839894d6964835b23 WatchSource:0}: task ff954893be5edf196f2ccdddd950da868060bd8de2cf7aa839894d6964835b23 not found: not found
Nov 04 11:21:09 metal01 k3s[930011]: W1104 11:21:09.140229  930011 watcher.go:95] Error while processing event ("/sys/fs/cgroup/devices/kubepods/pod07b07c46-45aa-4b4d-b30d-06054a939784/64d8ac27c5d1500c469dc62279c24d1b536382579973328e1fd3e96a64ed2201": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/devices/kubepods/pod07b07c46-45aa-4b4d-b30d-06054a939784/64d8ac27c5d1500c469dc62279c24d1b536382579973328e1fd3e96a64ed2201: no such file or directory
Nov 04 11:21:09 metal01 k3s[930011]: W1104 11:21:09.140293  930011 watcher.go:95] Error while processing event ("/sys/fs/cgroup/memory/kubepods/pod07b07c46-45aa-4b4d-b30d-06054a939784/64d8ac27c5d1500c469dc62279c24d1b536382579973328e1fd3e96a64ed2201": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/memory/kubepods/pod07b07c46-45aa-4b4d-b30d-06054a939784/64d8ac27c5d1500c469dc62279c24d1b536382579973328e1fd3e96a64ed2201: no such file or directory
Nov 04 11:21:09 metal01 k3s[930011]: W1104 11:21:09.140318  930011 watcher.go:95] Error while processing event ("/sys/fs/cgroup/cpu,cpuacct/kubepods/pod07b07c46-45aa-4b4d-b30d-06054a939784/64d8ac27c5d1500c469dc62279c24d1b536382579973328e1fd3e96a64ed2201": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/cpu,cpuacct/kubepods/pod07b07c46-45aa-4b4d-b30d-06054a939784/64d8ac27c5d1500c469dc62279c24d1b536382579973328e1fd3e96a64ed2201: no such file or directory
Nov 04 11:21:09 metal01 k3s[930011]: W1104 11:21:09.140389  930011 watcher.go:95] Error while processing event ("/sys/fs/cgroup/pids/kubepods/pod07b07c46-45aa-4b4d-b30d-06054a939784/64d8ac27c5d1500c469dc62279c24d1b536382579973328e1fd3e96a64ed2201": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/pids/kubepods/pod07b07c46-45aa-4b4d-b30d-06054a939784/64d8ac27c5d1500c469dc62279c24d1b536382579973328e1fd3e96a64ed2201: no such file or directory
Nov 04 11:21:09 metal01 k3s[930011]: W1104 11:21:09.140460  930011 watcher.go:95] Error while processing event ("/sys/fs/cgroup/blkio/kubepods/pod07b07c46-45aa-4b4d-b30d-06054a939784/64d8ac27c5d1500c469dc62279c24d1b536382579973328e1fd3e96a64ed2201": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/blkio/kubepods/pod07b07c46-45aa-4b4d-b30d-06054a939784/64d8ac27c5d1500c469dc62279c24d1b536382579973328e1fd3e96a64ed2201: no such file or directory

Stopping k3s does not stop the pods that are running. This is by design. What those logs are telling you is that you have deleted locations out from under running (or at least, created) pods. As suggested at https://github.com/k3s-io/k3s/issues/4391#issuecomment-960513374, to stop everything that can be started by k3s you should run k3s-killall.sh. But since it appears that you have deleted locations that pods expect to exist you would likely be better off invoking k3s-uninstall.sh and starting over.

Stopping k3s does not stop the pods that are running.

I know :) Nodes were drained before stopping k3s.

But since it appears that you have deleted locations that pods expect to exist you would likely be better off invoking k3s-uninstall.sh and starting over.

That's what I did and that's why the instance is fresh as I reinstalled the whole node.

Just to be clear, this is not my first rodeo with kubernetes and I tried multiple options before writing this issue. The solution of "have you tried turning this off and on again" was the first I did :)
The full chain of events on my side:

upgrade from 1.21 to 1.22 resulting in all pods failing to start due to missing containerd-shim binary. This is because I was using nvidia runtime as default with runc v1.
Few tests to figure out what is going on (deleting directories, restarting k3s, etc.). This is when I found https://github.com/k3s-io/k3s/issues/4070
Removing custom containerd config.toml.tmpl and using default configuration shipped with k3s
node drained, k3s restarted. All containers starting up apart from ones using nvidia runtime due to issue described here
Testing few different configurations of nvidia-device-plugin pod, but issue described here seems to be preventing pod from startup
Node teardown to discard issues related to stale confiigurations. Installing only k3s and only nvidia drivers on new node.
Issue still persists.

Right now my gut is telling me that this may be something in the nvidia runtime itself and integration of this runtime with k3s. I tried running the pod with default runtimeClassName and it works just fine (albeit without GPU access). However, setting runtimeClassName: nvidia and recreating pod leads to errors regarding log messages and cgroups.
What is surprising to me is the fact that in 1.21 everything works just fine and 1.22 breaks completely for workloads needing nvidia GPU.

Hmm, I can't see why it would be required, but we did drop the v1 runtime in 1.22 since it's been deprecated for a while: https://github.com/k3s-io/k3s/pull/3903
The Nvidia plugin should work fine without it unless for some reason you had configured the legacy type in your custom containerd toml?
https://github.com/k3s-io/k3s/issues/3105#issuecomment-906672797

experiencing exactly same issue, in 1.21.6 everything works correctly, 1.22.3 adds automatic nvidia-container-runtime detection, but every deployment requesting runtimeclass nvidia is crash looping

@hlacik can you attach the logs from k3s starting up on your node (specifically the runtime detection bit), along with the containerd configuration toml that it is generating?

NAME    STATUS   ROLES                  AGE     VERSION jtsna   Ready    control-plane,master   5h39m   v1.22.3+k3s1 jtsnb   Ready    <none>                 5h38m   v1.22.3+k3s1

@brandond
to me generated config.toml seem fine
this is what i was using on 1.21.5
i was adding this manually via config.tml.tmpl
[plugins.cri.containerd.runtimes.nvidia]
  runtime_type = "io.containerd.runc.v2"
[plugins.cri.containerd.runtimes.nvidia.options]
  BinaryName = "/usr/bin/nvidia-container-runtime"

and it was working. Seems identical except "" which i think is no difference
also i want to note, that this is same OS , i have removed 1.21.5 via k3s-uninstall.sh , and installed fresh 1.22.3 , so i can confirm it has nothing to do with OS configuration/packages

Small log update, I am getting the following events on nvidia device plugin v0.10.0 pod start (using nvidia runtimeclass):
  Normal   Scheduled       95m                   default-scheduler  Successfully assigned kube-system/nvidia-device-plugin-daemonset-h4v5c to metal01
  Normal   Pulling         92m                   kubelet            Pulling image "nvidia/k8s-device-plugin:v0.10.0"
  Normal   Pulled          88m                   kubelet            Successfully pulled image "nvidia/k8s-device-plugin:v0.10.0" in 3m35.07358951s
  Warning  Failed          88m                   kubelet            Error: failed to get sandbox container task: no running task found: container not created: not found
  Warning  Failed          88m                   kubelet            Error: failed to create containerd task: failed to create shim: OCI runtime create failed: container_linux.go:364: creating new parent process caused: container_linux.go:2005: running lstat on namespace path "/proc/4076575/ns/ipc" caused: lstat /proc/4076575/ns/ipc: no such file or directory: unknown
  Normal   Pulled          88m (x2 over 88m)     kubelet            Container image "nvidia/k8s-device-plugin:v0.10.0" already present on machine
  Normal   Created         88m (x2 over 88m)     kubelet            Created container nvidia-device-plugin-ctr
  Warning  Failed          88m                   kubelet            Error: sandbox container "437bb0ac4e63c34e8a678754a1ac4dd71d72cf2ab27a5555aed5b46f193f849b" is not running
  Warning  BackOff         88m (x7 over 88m)     kubelet            Back-off restarting failed container
  Normal   SandboxChanged  88m (x9 over 88m)     kubelet            Pod sandbox changed, it will be killed and re-created.
  Warning  FailedSync      83m                   kubelet            error determining status: rpc error: code = NotFound desc = an error occurred when try to find sandbox: not found
  Normal   Pulled          83m                   kubelet            Container image "nvidia/k8s-device-plugin:v0.10.0" already present on machine
  Normal   Created         83m                   kubelet            Created container nvidia-device-plugin-ctr
  Warning  Failed          83m                   kubelet            Error: sandbox container "ea086ddd278fac26f2d55d82f0f4cafb41e7582796a50d199c1971896b0a886b" is not running
  Normal   SandboxChanged  78m (x278 over 83m)   kubelet            Pod sandbox changed, it will be killed and re-created.
  Warning  BackOff         73m (x523 over 83m)   kubelet            Back-off restarting failed container
  Normal   Created         70m                   kubelet            Created container nvidia-device-plugin-ctr
  Warning  Failed          70m                   kubelet            Error: failed to create containerd task: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:402: getting the final child's pid from pipe caused: EOF: unknown
  Normal   Pulled          70m (x2 over 70m)     kubelet            Container image "nvidia/k8s-device-plugin:v0.10.0" already present on machine
  Warning  BackOff         10m (x2792 over 70m)  kubelet            Back-off restarting failed container
  Normal   SandboxChanged  50s (x3385 over 70m)  kubelet            Pod sandbox changed, it will be killed and re-created.

I can confirm I'm having the same issue here.  Works fine on v1.21.6, but does not work on v1.22.3.

I suspect that perhaps the nvidia device plugin isn't compatible with containerd 1.5?

@kralicky have you tried this out at all?

I have seen the exact errors @paulfantom has and I believe this is related to the clone3/seccomp updates that are in the latest containerd. The workaround for now is to make all pods which use the nvidia container runtime privileged. It is possible that this is an issue on nvidia's end but I am not 100% sure if that is the case.

I can confirm that workaround suggested by @kralicky is working. :+1:

@kralicky we recently saw the clone3/seccomp update issue on updated docker packages on Ubuntu (see nvidia-container-runtime#157). We have published updated packages for the NVIDIA Container Toolkit (including the nvidia-container-runtime) to our experimental package repositories and will be promoting these to stable in the near future.
As an alternative to running the containers as privileged, you could update the nvidia-container-toolkit to at least 1.6.0-rc.2.

I can confirm that with 1.6.0-rc.3 the issue is gone. As such I am closing this bug report. Thank you everyone for your feedback and helping me solving this issue! :100: :+1:

Just in case someone tries with k3s v1.23.8+k3s2
After installing driver and nvidia-container-toolkit, put only this inside the config.toml
sudo mkdir /var/lib/rancher/k3s/agent/etc/containerd
sudo vim /var/lib/rancher/k3s/agent/etc/containerd/config.toml

(Copied from https://github.com/NVIDIA/k8s-device-plugin#configure-containerd)
version = 2
[plugins]
  [plugins."io.containerd.grpc.v1.cri"]
    [plugins."io.containerd.grpc.v1.cri".containerd]
      default_runtime_name = "nvidia"

      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.nvidia]
          privileged_without_host_devices = false
          runtime_engine = ""
          runtime_root = ""
          runtime_type = "io.containerd.runc.v2"
          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.nvidia.options]
            BinaryName = "/usr/bin/nvidia-container-runtime

And of course add the nvidia-device-plugin:
kubectl create -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v0.12.2/nvidia-device-plugin.yml

Add a RuntimeClass:
apiVersion: node.k8s.io/v1 # RuntimeClass is defined in the node.k8s.io API group
kind: RuntimeClass
metadata:
  name: nvidia # The name the RuntimeClass will be referenced by
  # RuntimeClass is a non-namespaced resource
handler: nvidia # The name of the corresponding CRI configuration

Add a gpu pod:
apiVersion: v1
kind: Pod
metadata:
 name: gpu-pod
 namespace: gpus
spec:
 restartPolicy: OnFailure
 runtimeClassName: nvidia
 containers:
   - name: cuda-container
     image: nvidia/cuda:11.0-base
     command: ["nvidia-smi"]
     resources:
       limits:
         nvidia.com/gpu: 1 # requesting 1 GPU

Providing a containerd config template is only necessary if you want to change the default runtime. If you're using runtimeClassName, all you should need to do is install the runtime package for your OS, then restart K3s.

I spent lot of time on this and I finally managed to get it to work. @FischerLGLN got close but the issue is you are not suppose to modify /var/lib/rancher/k3s/agent/etc/containerd/config.toml and you suppose to use the ....config.toml.tmpl according to https://rancher.com/docs/k3s/latest/en/advanced/
This is what I did to get it to work
# Install gpg key from nvidia
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
    && curl -s -L https://nvidia.github.io/libnvidia-container/gpgkey | sudo apt-key add - \
    && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Install Drivers. Use the latest drivers!
apt-get update && apt-get install -y nvidia-driver-515-server nvidia-container-toolkit nvidia-modprobe

# Download template from k3d project
sudo wget https://k3d.io/v5.4.1/usage/advanced/cuda/config.toml.tmpl -O /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl

Try running nvidia plugin and check the logs. If you dont see this then you have to modify the template further.
2022/07/25 20:45:18 Updating config with default resource matching patterns.
2022/07/25 20:45:18
Running with config:
{
"version": "v1",
"flags": {
"migStrategy": "none",
"failOnInitError": false,
"nvidiaDriverRoot": "/",
"plugin": {
"passDeviceSpecs": false,
"deviceListStrategy": "envvar",
"deviceIDStrategy": "uuid"

This used to work previously but with K3S v1.23+ I had issues. You will have to modify /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl add:
[plugins.cri.containerd.runtimes.runc.options]
  BinaryName = "/usr/bin/nvidia-container-runtime"

and modify
[plugins.cri.containerd.runtimes.runc]
  runtime_type = "io.containerd.runtime.v1.linux"

to
[plugins.cri.containerd.runtimes.runc]
  runtime_type = "io.containerd.runc.v2"

This should get everything running. Then either use the nvidia plugin to define resources or if you like me and want to share the GPU accross multiple pods then just add these ENV variables to your pod
  NVIDIA_VISIBLE_DEVICES: all
  NVIDIA_DRIVER_CAPABILITIES: all

Modifying the containerd config template is not necessary. K3s will automatically add runtimes to the containerd config if the nvidia binaries are present on the node when k3s is started. All you need to do is use the RuntimeClass and Pod specs shown in https://github.com/k3s-io/k3s/issues/4391#issuecomment-1181707242

@brandond hmm I'm happy to test again but I can guarantee you that it doesn't work out of the box

Which part of it doesn't work? The runtime binary detection and addition of runtimes to the containerd config, or your runtimeclass/pod spec making use of it?

Adding the runtimes to the containerd config.
I had the RuntimeClass and the pod configured in a fresh installation. Pod started but without the actual GPU exposed. Until I added config.toml.tmpl file.

The current code checks /usr/bin and /usr/local/nvidia/toolkit for nvidia-container-runtime and nvidia-container-runtime-experimental binaries. I can confirm that it finds and adds these if it's present. Can you verify the version of k3s you're using, and that you're using the expected binary paths?
[root@centos01 ~]# ln -s /usr/bin/true /usr/bin/nvidia-container-runtime

[root@centos01 ~]# curl -ksL get.k3s.io | sh -
[INFO]  Finding release for channel stable
[INFO]  Using v1.24.3+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.24.3+k3s1/sha256sum-amd64.txt
[INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.24.3+k3s1/k3s
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Creating /usr/local/bin/ctr symlink to k3s
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink from /etc/systemd/system/multi-user.target.wants/k3s.service to /etc/systemd/system/k3s.service.
[INFO]  systemd: Starting k3s

[root@centos01 ~]# cat /var/lib/rancher/k3s/agent/etc/containerd/config.toml
[plugins.opt]
  path = "/var/lib/rancher/k3s/agent/containerd"

[plugins.cri]
  stream_server_address = "<IP_ADDRESS>"
  stream_server_port = "10010"
  enable_selinux = false
  enable_unprivileged_ports = true
  enable_unprivileged_icmp = true
  sandbox_image = "rancher/mirrored-pause:3.6"

[plugins.cri.containerd]
  snapshotter = "overlayfs"
  disable_snapshot_annotations = true

[plugins.cri.cni]
  bin_dir = "/var/lib/rancher/k3s/data/1d787a9b6122e3e3b24afe621daa97f895d85f2cb9cc66860ea5ff973b5c78f2/bin"
  conf_dir = "/var/lib/rancher/k3s/agent/etc/cni/net.d"

[plugins.cri.containerd.runtimes.runc]
  runtime_type = "io.containerd.runc.v2"

[plugins.cri.containerd.runtimes.runc.options]
	SystemdCgroup = false

[plugins.cri.containerd.runtimes."nvidia"]
  runtime_type = "io.containerd.runc.v2"
[plugins.cri.containerd.runtimes."nvidia".options]
  BinaryName = "/usr/bin/nvidia-container-runtime"

I spent lot of time on this and I finally managed to get it to work. @FischerLGLN got close but the issue is you are not suppose to modify /var/lib/rancher/k3s/agent/etc/containerd/config.toml and you suppose to use the ....config.toml.tmpl according to https://rancher.com/docs/k3s/latest/en/advanced/
This is what I did to get it to work
# Install gpg key from nvidia
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
    && curl -s -L https://nvidia.github.io/libnvidia-container/gpgkey | sudo apt-key add - \
    && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Install Drivers. Use the latest drivers!
apt-get update && apt-get install -y nvidia-driver-515-server nvidia-container-toolkit nvidia-modprobe
reboot

# Check whether GPU recognized. You might have to restart the node after the driver installation to get this working
nvidia-smi

# Download template from k3d project
sudo wget https://k3d.io/v5.4.1/usage/advanced/cuda/config.toml.tmpl -O /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl

# Install nvidia plugin. This is optional. You can simply pass in the env variables to pass in GPU access to pods. However this is a nice way to debug whether you have access to the GPUs on contianerd
kubectl create -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v0.12.2/nvidia-device-plugin.yml

Try running nvidia plugin and check the logs. If you see this on the correct node then you have to modify the .tmpl file further.
2022/07/25 18:14:19 Initializing NVML.
2022/07/25 18:14:19 Failed to initialize NVML: could not load NVML library.
2022/07/25 18:14:19 If this is a GPU node, did you set the docker default runtime to `nvidia`

This used to work previously but with K3S v1.23+ I had issues. You will have to modify /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl add:
[plugins.cri.containerd.runtimes.runc.options]
  BinaryName = "/usr/bin/nvidia-container-runtime"

and modify
[plugins.cri.containerd.runtimes.runc]
  runtime_type = "io.containerd.runtime.v1.linux"

to
[plugins.cri.containerd.runtimes.runc]
  runtime_type = "io.containerd.runc.v2"

This should get everything running. Then either use the nvidia plugin to define resources or if you like me and want to share the GPU accross multiple pods then just add these ENV variables to your pod
  NVIDIA_VISIBLE_DEVICES: all
  NVIDIA_DRIVER_CAPABILITIES: all

It work for me . Thank you.

This used to work previously but with K3S v1.23+ I had issues. You will have to modify >/var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl add:
[plugins.cri.containerd.runtimes.runc.options]
 BinaryName = "/usr/bin/nvidia-container-runtime"

Don't do this; it will change the default runtime to use the nvidia binary instead of runc. As described above, you should be creating a RuntimeClass and setting the RuntimeClassName on pods that you want to use the nvidia container runtime.

@brandond are there any docs on how it should be done and since which k3s version?

Here's what I did to get this working on an Ubuntu node. You should be able to follow similar instructions to get it working on any other distro on any currently supported release of K3s:

Install the nvidia-container repo on the node by following the instructions at:
https://nvidia.github.io/libnvidia-container/
Install the nvidia packages:
apt install -y nvidia-container-runtime cuda-drivers-fabricmanager-515 nvidia-utils-515-server
Install K3s, or restart it if already installed:
curl -ksL get.k3s.io | sh -
Confirm that the nvidia container runtime has been found by k3s:
grep nvidia /var/lib/rancher/k3s/agent/etc/containerd/config.toml
Deploy the  so that pods can request GPU resources:
kubectl apply -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/master/nvidia-device-plugin.yml
Deploy the manifest containing the RuntimeClass, Nvidia Device Plugin, and Pod that uses the to run a benchmark using the GPU device:
kubectl apply -f https://gist.githubusercontent.com/brandond/33e49bf094712f926c95d683d515ac95/raw/nvidia.yaml

Thanks. What I currently still don't understand is why you recommend to change the "upstream" manifests to add runtimeClassName: nvidia instead of changing the default runtime (which seems easier to me down the road).

Changing the default system runtime based on the autodetected presence of the nvidia container runtime binary is more disruptive.
If the container runtime were made the default, but other packages (such as the libraries, kernel module, and so on) are not properly installed, then the node will be unable to run any pods.
Additionally, it is usually only desired to run some pods with the nvidia runtime; for all of the other pods in the system that aren't going to use GPU, the default runtime is fine.

Thanks for your help @brandond !
You are right, not all pods need/use a GPU, but I think if you don't request it, it will not be used... but I clearly need to check on this a bit more.
I'm just searching for the "best-practice" setup and one that also requires the least amount of changes in the manifest/helm charts, regardless of whether I'm deploying it on our on-prem k3s cluster, or our customers cluster or in the cloud....

You're correct, pods that don't request GPU won't get one, but if you change the default runtime to nvidia-container-runtime then everything will run with that, instead of with runc - which may lead to unexpected changes in behavior.

Side question is there anyway to request a nvidia runtime in pod spec and being able to share a single gpu with multiple pods?

3s will automatically add runtimes to the containerd con

Without  pre-installed NVIDIA Container Toolkit and gpu driver， I followed the gpu-operator(v22.9.0) installation guide in k3s(v1.24.3+k3s1) to deploy gpu operator successfully, but I ran the samples from https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/getting-started.html#running-sample-gpu-applications, it failed. I have to add runtimeClassName: nvidia in the pod spec，so I wonder that how these samples ran succeessfully without runtimeClassName: nvidia.

I wonder that how these samples ran succeessfully without runtimeClassName: nvidia.

They will not on K3s. You either need to explicitly specify the nvidia runtime class, or modify the containerd config template to use the nvidia container runtime for all pods.

For those who want to set the default runtime to nvidia, here is what works with k3s v1.24.6+k3s1 using containerd 1.6.8-k3s1: Check that the nvidia runtime was detected as @brandond described above. If yes, get the default config.tompl.tmpl from https://github.com/k3s-io/k3s/blob/master/pkg/agent/templates/templates_linux.go and change it to have
[plugins.cri.containerd]
  default_runtime_name = "nvidia"

similar to what nvidia also describes in the k3s-device-plugin docs. Here is the template I use: config.toml.tmpl

Hello, i tried this on ubuntu 22.04 with k3s with kubernetes 1.25 and this does not work.
Containerd version 1.6.9.
It seems to properly identity proper runtimes though.
[plugins.opt]
  path = "/var/lib/rancher/k3s/agent/containerd"

[plugins.cri]
  stream_server_address = "<IP_ADDRESS>"
  stream_server_port = "10010"
  enable_selinux = false
  enable_unprivileged_ports = true
  enable_unprivileged_icmp = true
  sandbox_image = "rancher/mirrored-pause:3.6"

[plugins.cri.containerd]
  snapshotter = "overlayfs"
  disable_snapshot_annotations = true

[plugins.cri.containerd.runtimes.runc]
  runtime_type = "io.containerd.runc.v2"

[plugins.cri.containerd.runtimes.runc.options]
	SystemdCgroup = true

[plugins.cri.containerd.runtimes."nvidia"]
  runtime_type = "io.containerd.runc.v2"
[plugins.cri.containerd.runtimes."nvidia".options]
  BinaryName = "/usr/local/nvidia/toolkit/nvidia-container-runtime"

[plugins.cri.containerd.runtimes."nvidia-experimental"]
  runtime_type = "io.containerd.runc.v2"
[plugins.cri.containerd.runtimes."nvidia-experimental".options]
  BinaryName = "/usr/local/nvidia/toolkit/nvidia-container-runtime-experimental"

Error thrown by nvidia-device-plugin pod ->
  Warning  Failed     99s (x5 over 3m13s)   kubelet            Error: failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli.real: initialization error: driver error: failed to process request: unknown

Can you try my steps documented at https://github.com/k3s-io/k3s/issues/4391#issuecomment-1233314825 instead?

Can you try my steps documented at #4391 (comment) instead?

So I just did a clean agent install on the GPU node. following your steps.
I get the same config file generated, also instead of using my install of nvidia-device-plugin i used your URL and once the pod starts on the node with the GPU I receive the same error.
I tried running nvidia-smi and within normal containerd
here is the output.
christopher@k8s-gpu:~$ sudo ctr run --rm --gpus 0 -t docker.io/nvidia/cuda:11.0-base cuda-11.0-base nvidia-smi
Thu Nov  3 22:14:02 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
|  0%   37C    P8    12W / 151W |      2MiB /  8192MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

As soon as it's in kubernetes, it doesn't seem to work

the image 11.0-base may have been previously pulled awhile back.
I just tried with 11.0.3 and it worked fine as well.
I tried running the nbody-gpu-benchmark it's unscheduleable because the nvidia-device-plugin pod is unable to complete.
nbody is looking for nvidia.com/gpu
  Warning  FailedScheduling  8s    default-scheduler  0/4 nodes are available: 4 Insufficient nvidia.com/gpu. preemption: 0/4 nodes are available: 4 No preemption victims found for incoming pod.

If it is of interest: I actually abandoned the approach of setting the default runtime to nvidia (although that worked) again and went with @brandond's recommendation of explicitly setting runtimeClassName since longhorn had problems running under the nvidia runtime... Although that is extra work for my GPU workloads and I had to find workarounds to specify the runtimeClassName in flyte, this definitely seems to be the route to go.

@larivierec

the nvidia-device-plugin pod is unable to complete.
Why?

I just tried with 11.0.3 and it worked fine as well.
edit: also tried with 11.8 and also worked fine.

How is this working? Which specific image are you using? Neither docker.io/nvidia/cuda:11.0.3-base nor any of the newer versions I've tried appear to contain the nvidia-smi binary.

@larivierec

the image 11.0-base may have been previously pulled awhile back.

Hmm, that'd have to be from quite a while ago then. What version of K3s are you currently using?

the nvidia-device-plugin pod is unable to complete.

Why not?

I just tried with 11.0.3 and it worked fine as well.

edit: also tried with 11.8 and also worked fine.

How is this working? Which specific image are you using? Neither docker.io/nvidia/cuda:11.0.3-base nor any of the newer versions I've tried appear to contain the nvidia-smi binary.

Hmmmm, with regards to 11.X I don't know maybe perhaps I made modifications to the system containerd?
The k3s version is latest stable version: 1.25-k3s.

@brandond thanks for the help yesterday, turns binary that was being set in the config.toml by k3s was not the one that I installed with apt install.
The fix was simple.

k3s-killall.sh
delete /usr/local/nvidia
restart k3s so it picks up the correct nvidia-container-runtime which was this one /usr/bin/nvidia-container-runtime

Sorry for the hassle,
Cheers :beer:

@larivierec if I remember correctly, the /usr/local/nvidia runtime is installed by the gpu operator, and will be selected over the package manager-installed runtime if it exists. If you're still using the gpu operator it might try to install its own runtime again, so look out for that.

That would make sense, I had installed it previously however, when I removed it awhile back it didn't seem to clean itself up!
Thanks for the heads up

Hello,
Personnaly it works with the following setup (works on Ubuntu 22.04, Nvidia drivers 515 and K3s 1.25) :
As @brandond said, we need to create a runtimeClass resource:
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: nvidia
handler: nvidia

Then, I prefer deploy the helm chart from directly from Nvidia repo
helm repo add nvdp https://nvidia.github.io/k8s-device-plugin
helm upgrade -i nvdp nvdp/nvidia-device-plugin --namespace nvdp --create-namespace --version 0.12.3 --set=runtimeClassName=nvidia

I hope this will help.

The docs have been updated to include coverage of the runtime auto-detection behavior:
https://docs.k3s.io/advanced#nvidia-container-runtime-support
I'm going to lock this issue for now; if anyone has suggestions for docs improvements please open a PR!


Removes a token after use

The pentest recommended token expiry after use.  On review, it seemed more sensible to remove expired tokens.  I haven't made an exhaustive code review.  However, the token_controller creates a new one each time a login is requested and the old ones are never reused for token login.
The remedy for the cryptographic timing attack required a find_batch on all tokens.  Removing tokens as they are used keeps us from having to scope that find_batch to active tokens, setting up a cron job to remove expired tokens later, or just waiting for that find_batch to slow down due to the ever growing number of records.

Change is fine - feel free to merge once you've fixed the conflicts.


Add List.find

This adds List.find, a useful function I've been using in a personal Util.elm file, but which seems like a good candidate for inclusion in core given that it's standard in several languages, for example:

JavaScript
Ruby
Scala
Haskell

This should proxy to Native right? I've been wondering whether there's criteria for Core inclusion.

Would proxying to Native improve this though? With the impending self-TCO this compiles to a while loop anyway, yeah?

Native JS uses C++ cheats.

PS I agree it should be included I just wish we had a criteria for completeness and rejection.

For now the recommended way of extending the standard libraries is in a package, like elm-list-extra or elm-random-extra. There can be lots of experimentation with APIs there that does not block on me or the release cycle of elm-lang/core. At some point, we will go through and try to merge in all the best things about the *-extra world in a coherent way.
I suspect @rehno-lindeque would accept this function in elm-list-extra, try a PR there for now.

Thanks for the clarification @evancz makes sense to me :+1:

Fair enough! Moved the PR to elm-list-extra.


Customize the outgoing network interface used with the client.

Hi,
in the client, how can i connect through my second/additional  ip || interface ..
Thank you

You cannot right now

well, i sugesst it to be an option like this one 'localAddress'
'''
var http = require('http');
var options = {
hostname: 'www.example.com',
localAddress: '<IP_ADDRESS>'
};
var req = http.request(options, function(res) {
res.on('data', function (chunk) {
console.log(chunk.toString());
});
});
'''


Missing documentation for status attributes

The documentation states

status attributes are available under the StatusAttributes property

However there doesn't appear to be a link to any reference documentation regarding these status attributes. In the example, the request charge is being cast as a double:
double requestCharge = (double)results.StatusAttributes["x-ms-request-charge"];

How do we know to cast this specific attribute as double? What about the other attributes in the collection (eg. "x-ms-status-code" or "x-ms-retry-after-ms")? Are any of these attributes documented anywhere? It would be great to have some reference documentation to at least show which attributes could be in this collection along with their underlying types and possible/expected values where appropriate eg. for status codes
The only thing I could find from a google search for "cosmosdb statusattributes" was this document and the Gremlin.NET "getting started" source code on github with the following comments:
// This includes the following:
//  x-ms-status-code            : This is the sub-status code which is specific to Cosmos DB.
//  x-ms-total-request-charge   : The total request units charged for processing a request.
...
//  x-ms-retry-after-ms         : The number of milliseconds to wait to retry the operation after an initial operation was throttled. This will be populated when
//                              : attribute 'x-ms-status-code' returns 429.
//  x-ms-activity-id            : Represents a unique identifier for the operation. Commonly used for troubleshooting purposes.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 49b9cecf-c00d-821f-c5f4-06249c75bf55
Version Independent ID: 12ec16ac-97c7-e351-464c-93a23f67845d
Content: Find the request unit (RU) charge in Azure Cosmos DB
Content Source: articles/cosmos-db/find-request-unit-charge.md
Service: cosmos-db
GitHub Login: @ThomasWeiss
Microsoft Alias: thweiss

@Kirlac  We are checking with author regarding the same and get back to you.
@ThomasWeiss Please check the user question and try to give any insight.

@Kirlac From all the custom response headers exposed through the StatusAttributes property, only the x-ms-request-charge is generally useful to Cosmos DB users as it lets them measure the RU charge of their operations. The other ones are there for debugging purposes when interacting with us and our customer support. In other words, you won't have to look for the other ones unless asked by our support to retrieve an ActivityId for example.

Hi @ThomasWeiss. Thanks for your response. I do however think it's somewhat disingenuous to state that the x-ms-request-charge attribute is the only useful one here. From what I can see, @ashwinisingh01 / @ashwini-ms worked hard to expose details like Request Charge, Rate limiting/Retry policy details etc. in the Apache Tinkerpop SDK (PR #915) to be more easily accessed from the Gremlin client.
This must have been done for a reason, I assume most likely the same reason why I'm here now - because enterprise-grade production systems require a high level of fault tolerance, auditing, etc. so need a way to access this information. I opened this issue because these attributes seem to be undocumented, meaning users will need to spend a lot of time reverse engineering them and digging through github issues etc. in order to implement this. In issue #30218 a Microsoft representative points towards a stack overflow question for handling the 429 rate limit errors in Gremlin.Net - the answer being that retry policies would need to be implemented manually as only the Azure SDKs handle this natively. As Microsoft.Azure.Graphs has been deprecated in favor of Gremlin.Net users now need some documentation around this to be able to handle it effectively. As such, saying

you won't have to look for the other ones unless asked by our support

is blatantly false as we already do have to look for these to implement appropriate fault tolerance in our applications. Initially for catching the 429 rate limited response and honoring the retry-after attribute, but also to help identify other errors such as timeouts, server errors, etc. and take the appropriate actions in these cases.
Unfortunately, even though it is stated that "StatusAttributes is a typed Json" the collection exposed appears to be a Dictionary<string, object> (at least in the .net driver) and I wasn't able to see anything in that repository documenting what attributes this could contain, their types, and expected values (eg. for response status/substatus codes). I don't expect to find any documentation there as these are obviously Cosmos Db specific attributes and would not be relevant to the gremlin sdk when using a different server, but this documentation also appears to be missing from the Microsoft docs.
Is it possible to get some better documentation for these? At this point even a link to a JSON schema or class definition file would be a huge improvement over what's currently available.
Thanks in advance.

Hi @Kirlac,
I apologize for having incomplete documentation for Cosmos DB Gremlin service. We have covered some aspects of what you're interested in Self Help Article about Response Headers. Documentation doesn't explicitly prescribe each type, but implies that types are primitive. It should be sufficient to run a traversal and peek inside the response payload to confirm exact types they are converted into on the client side. If you are having trouble with this exercise please let us know and we'll post the type of each header here while we work on getting official documentation online.

@olignat The Link provided does not work and throws a 404 error. Please check. Please let us know if any other information is required in this regard.

This is correct - I'm also getting a 404 from that link.
I apologize, I thought I had responded saying that but I guess my comment didn't post. Maybe the link is for a private repository which I do not have access to or something?

I am wrapping up a public articles on this topic and it should be available shortly. I'll see if I can get it published in the next few days.

Please check this out - https://docs.microsoft.com/en-us/azure/cosmos-db/gremlin-headers. Let me know if this article helps or you'd like me to add anything else.

This is exactly what I was looking for. Thank you for your help and the quick turnaround getting this updated.

@Kirlac I just wanted to apologize for the answer I gave originally, I overlooked the fact that you do need access to these headers in order to implement your throttling management and retry logic. I'm glad that Oleg provided you the complete information.

Hi @ThomasWeiss. Thanks for that but there's no need to apologize. I understand this is a relatively immature technology by industry standards, particularly in regards to the alternative API's - and our usage probably falls a little outside of the general use-case for this if the documentation is anything to go by. Gremlin/graph api usage on cosmos seems to be a lot less common especially in an enterprise environment - for example I know it's still lacking bytecode support which has likely blocked adoption for many people. From what I understand the official cosmos db sdks handle throttling/retry logic/etc a lot better in a native fashion so I can certainly see where the confusion comes from.
Thanks again everyone for your help with this.


Display Problem on Chinese AssetManagement Website

https://www.webpackjs.com/guides/asset-management/
The change from main.js to bundle.js(line 35-36) is not truly highlighted on the webpage which may lead to certain misunderstandings or omissions.
This change is supposed to be highlighted as it is crucial for all the changes we will make in latter sections.

Not a problem can be fixed in this repository. You might want to create an issue on the Chinese translation site https://github.com/docschina/webpack.js.org/issues


Feature - allow ability to switch Bar's hover behavior

Added a new feature that allows the user to be able to set the hover behavior over bar chart.
Description
Added a new function to Bar's API called hasSingleHover which accepts a boolean value.

If hasSingleHover is not called (the default value is true), the behavior of the hover mode behaves as default: if one hovers over a bar, the hovered bar is darkened while the rest stay the same.
If hasSingleHover is set to false, on hovering the bar all the bars but the hovered one will darken instead (the opposite).

Motivation and Context
This feature will allow the user to choose between different bar hover behavior based on the one that makes sense. I saw this issue https://github.com/eventbrite/britecharts/issues/485 which gave me an idea of making it configurable.
How Has This Been Tested?
Added 3 new tests to bar.spec.js:

One that checks setter and getter of hasSingleHover
Checks if the default behavior (color switch on hover) stays the same as before.
When hasSingleHover(false) check that the color of the hovered bar stays the same (since others are darkened).

Screenshots (if appropriate):

hasSingleHover(false) - only the hovered bar kept its color while others are darkened

hasSingleHover() - by default returns true, in this case the behavior is the same as before.

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

The implementation is great, and it completely solves #485, but I am not sure if this is what we want for this chart.
Specifically, I am worried about restricting our options here. The current implementation is restrictive as it doesn't allow the user to choose what to do with the hovered bar, we just darken it and that's it. If the user wanted to lighten it, fade it or something else, he won't be able to do it.
Thinking out loud here, but if we just get rid of the darkening action, wouldn't the user be able to do whatever he wants just with d3 selections? Maybe is a bit complicated, but given that we provide the hovered bar in 'e', probably we can select all the bars, remove the one passed here and darken them, basically doing what you did in the lines 649-654.
We could argue that this is right, but too complex to do for our users, given that is such a usual thing to do.
Maybe another solution would be to keep what you are proposing in this PR, but creating a function that deals with the styling of the selected bar. By default it would be darkening it, but it could be something different. This solution won't break the current functionality.
Let me state this solution better, it would mean:

Change hasSingleHover into 'hasSingleBarHighlight' that is true as default
Create a 'highlightFunction' that, for a passed in bar, it applies a darkening by default. We would use this as well when 'hasSingleBarHighlight' is false

What do you think?

I agree that this function limits the user to only do the hover effect without passing the data point. I think that this function can be used for highlighting fast and I like the second solution that you proposed which is to change hasSingleHover to hasSingleBarHighlight that is set to true by default and create new function that will allow user to perform more complex operations on a bar within passed callback.
As I understand, hasSingleBarHighlight will be a function for quick solution and for those who might be less "tech-savy" while highlightFunction will give an option to customize it. That sounds good to me.

Yeah, awesome!
Thanks a lot for your late push on Britecharts, you are on fire!! 🔥


Support  yielding Task types (e.g. data.task)

I find myself working with folktale, ramda, and the other related libraries.  It would be cool if co could support yielding on Task types.  The change seems simple (but I am a bit simplistic, so perhaps I missed something):
A couple of helpers
function taskToPromise(a) {
  return new Promise(function(resolve, reject) {
    a.fork(reject, resolve)
  })
}

function isTask(obj) {
  return 'function' == typeof obj.fork;
}

and then a modification of toPromise:
function toPromise(obj) {
  if (!obj) return obj;
  if (isPromise(obj)) return obj;
  if (isGeneratorFunction(obj) || isGenerator(obj)) return co.call(this, obj);
  // this seems like a good place to stick this
  if (isTask(obj)) return taskToPromise.call(this, obj);
  if ('function' == typeof obj) return thunkToPromise.call(this, obj);
  if (Array.isArray(obj)) return arrayToPromise.call(this, obj);
  if (isObject(obj)) return objectToPromise.call(this, obj);
  return obj;
}

Putting it into action, to test above.  First a couple of helpers random num generators, and then task factory that retunrs a Task that resolves some computation within a random 3 seconds:
const random = () => (Math.floor(Math.random() * 10000) + 999)
const timeout = () => (Math.floor(Math.random() * 3000) + 100)

// makeTask :: (boolean, (a->b)) -> Task
const makeTask = (bool, computation) => new Task((reject, resolve) => {
    const x = computation()
    const t = timeout()
    setTimeout(() => {
        if (bool) {
            resolve({"success": x});
        } else {
            reject({"failed": x});
        }
    }, t)
})

And then create the test and kick it off
const test = function*() {
    const x = yield makeTask(success, random)
    return x
}

co(test).then(console.log, console.error)

I understand every library can't support everything, but this would make it is easier to integrate the Task type with the legacy code I have in place.  Always happy to learn, so if this is a bad idea I would love insight why.
Thanks

Tasks are flawed in the same way that Promises are in the sense that they don't share state among multiple asynchronous calls so I've seen pretty, um, interesting approaches to handling that (pumping arrays up full of various params).
Generator-based control flow like co() solves this rather elegantly.
if anything, co() + Maybe should give you the same flow you seek but folktale's Maybe has kind of an ugly interface.

co already does too much stuff sorry! people should be preparing for async/await, which does not support the same things.
cheers

I did it thought :weight_lifting_man:. I think of it more as a coding exercise really.


trained data not found in data dir when ./configure --prefix is used with a path containing double slashes

Environment

Tesseract Version:  4.1.1
Commit Number: N/A
Platform:
Linux rapidspace-hosting-002 4.9.0-14-amd64 #1 SMP Debian 4.9.240-2 (2020-10-30) x86_64 GNU/Linux

The problem only happens when using a custom --prefix (or --datadir) to ./configure which contains double slashes, such as for example:
./configure --prefix=/tmp//tesseract/
make
make install
curl https://raw.githubusercontent.com/tesseract-ocr/tessdata/4.1.0/eng.traineddata > /tmp//tesseract/share/tessdata/eng.traineddata

Current Behavior:
tesseract can be installed, but at runtime the trained data are looked up in /tmp/tessdata:
$ /tmp//tesseract/bin/tesseract --list-langs
Error opening data file /tmp/tessdata/eng.traineddata
Please make sure the TESSDATA_PREFIX environment variable is set to your "tessdata" directory.
Failed loading language 'eng'
Tesseract couldn't load any languages!
List of available languages (0):

strace -e open /tmp//tesseract/bin/tesseract --list-langs
...
open("/tmp/tessdata/eng.traineddata", O_RDONLY) = -1 ENOENT (No such file or directory)
...

The traineddata is looked up in /tmp/tessdata and not /tmp//tesseract/tessdata.
My understanding is that something like this is happening:
The prefix is set in Makefile, this is the case for 4.1.1. On master it uses cmake but the pattern seems the same (I did not verify the behaviour on master though):
https://github.com/tesseract-ocr/tesseract/blob/75103040c94ffd7fe5e4e3dfce0a7e67a8420849/src/ccutil/Makefile.am#L4-L6
and propagated in the source with a sequence of define:
https://github.com/tesseract-ocr/tesseract/blob/a843f3b2589afe6c5d1edec87cb9dacbb1efcf8f/src/ccutil/mainblk.cpp#L71-L77
The problem here is that when the path contain // it is a comment for the preprocessor, so the _XSTR(TESSDATA_PREFIX) uses /tmp instead of /tmp//prefix, because everything after // is a  comment.
Expected Behavior:
The tessdata files should be loaded from the datadir, even if at configure time it was specified with a double slash.
Suggested Fix:
Frankly, I'm not an expert at all with autotools, but it seems it would be possible to use macros such as AC_DEFINE_UNQUOTED . I noticed this same pattern in ghostscript's  tesseract integration and I filled this in https://bugs.ghostscript.com/show_bug.cgi?id=703905 as well, probably the pattern in ghostscript is inspired from here
For ghostscript it seemed possible with some autotools magic, maybe now that tesseract uses cmake there is a better solution with cmake, but I don't know about cmake either, so I can not really suggest a good fix.
Another solution that seems perfectly acceptable to me is to close this issue and decide that this will not be fixed, because it does not really make sense to install in a prefix with double slashes.
I guess  using double slashes in paths is really exceptional. I wanted to file this bug to describe the problem, I'm not  really convinced it's worth fixing. In any case, thanks for your time at least reading this

We can keep this issue open but it is at low priority.

The problem is not specific to autotools, but also happens with cmake. Build instructions:
mkdir build
cd build
cmake .. -DTESSDATA_PREFIX=/usr//local
make

Result:
bin/tesseract --list-langs
Error opening data file /usr/tessdata/eng.traineddata
[...]

It is caused by a "feature" of the preprocessor: datadir = _XSTR(TESSDATA_PREFIX) "/tessdata"; handles // as start of a comment, so TESSDATA_PREFIX=/usr remains.

autotools passes -DTESSDATA_PREFIX=/usr//local to the compiler.
cmake adds #define TESSDATA_PREFIX /usr//local to config_auto.h.
The issue can be fixed by using a C string for TESSDATA_PREFIX, for example by writing #define TESSDATA_PREFIX "/usr//local" to config_auto.h. Then there is no need for the stringification which causes the trouble.

The issue should be fixed in latest git master.

Hi
After this commit.
There is a build error if you set TESSDATA_PREFIX through variable CXXFLAGS
tesseract_5.0.0~git5670-386dd8a0-1_amd64.log
./autogen.sh
...
./configure --host=x86_64-linux-gnu \
                  --build=x86_64-linux-gnu \
                  --disable-tessdata-prefix \
                  --prefix=/usr --libdir=\${prefix}/lib/x86_64-linux-gnu \
                  --mandir=\${prefix}/share/man --infodir=\${prefix}/share/info \
                  CXXFLAGS="-g -O2 -fdebug-prefix-map=/build/tesseract-5.0.0~git5670-386dd8a0=. -fstack-protector-strong -Wformat -Werror=format-security -Wall -g -fPIC -DTESSDATA_PREFIX=/usr/share/tesseract-ocr/5" \
                  LDFLAGS="-llept -Wl,-z,defs -Wl,-z,relro -Wl,-z,now"
...
make
...
./src/ccutil/errcode.h:56:4: note: in definition of macro 'ASSERT_HOST'
   (x) ? DO_NOTHING : ASSERT_FAILED.error(#x, ABORT, "in file %s, line %d", __FILE__, __LINE__)
    ^
  CXX      src/ccutil/libtesseract_ccutil_la-clst.lo
  CXX      src/ccutil/libtesseract_ccutil_la-elst2.lo
  CXX      src/ccutil/libtesseract_ccutil_la-elst.lo
  CXX      src/ccutil/libtesseract_ccutil_la-errcode.lo
  CXX      src/ccutil/libtesseract_ccutil_la-mainblk.lo
  CXX      src/ccutil/libtesseract_ccutil_la-serialis.lo
  CXX      src/ccutil/libtesseract_ccutil_la-scanutils.lo
  CXX      src/ccutil/libtesseract_ccutil_la-tessdatamanager.lo
src/ccutil/serialis.cpp: In function 'bool tesseract::SaveDataToFile(const std::vector<char>&, const char*)':
src/ccutil/serialis.cpp:58:72: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<char>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
   bool result = static_cast<int>(fwrite(&data[0], 1, data.size(), fp)) == data.size();
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
src/ccutil/serialis.cpp: In member function 'char* tesseract::TFile::FGets(char*, int)':
src/ccutil/serialis.cpp:198:44: warning: comparison of integer expressions of different signedness: 'int' and 'std::vector<char>::size_type' {aka 'long unsigned int'} [-Wsign-compare]
   while (size + 1 < buffer_size && offset_ < data_->size()) {
                                    ~~~~~~~~^~~~~~~~~~~~~~~
src/ccutil/serialis.cpp: In member function 'int tesseract::TFile::FRead(void*, size_t, int)':
src/ccutil/serialis.cpp:226:23: warning: comparison of integer expressions of different signedness: 'long unsigned int' and 'int' [-Wsign-compare]
   if (SIZE_MAX / size <= count) {
                       ^
In file included from src/ccutil/serialis.cpp:21:
src/ccutil/serialis.cpp: In member function 'int tesseract::TFile::FWrite(const void*, size_t, int)':
src/ccutil/serialis.cpp:277:31: warning: comparison of integer expressions of different signedness: 'long unsigned int' and 'int' [-Wsign-compare]
   ASSERT_HOST(SIZE_MAX / size > count);
                               ^
src/ccutil/errcode.h:56:4: note: in definition of macro 'ASSERT_HOST'
   (x) ? DO_NOTHING : ASSERT_FAILED.error(#x, ABORT, "in file %s, line %d", __FILE__, __LINE__)
    ^
src/ccutil/mainblk.cpp: In member function 'void tesseract::CCUtil::main_setup(const string&, const string&)':
<command-line>: error: expected primary-expression before '/' token
src/ccutil/mainblk.cpp:72:15: note: in expansion of macro 'TESSDATA_PREFIX'
     datadir = TESSDATA_PREFIX "/tessdata";
               ^~~~~~~~~~~~~~~
<command-line>: error: 'usr' was not declared in this scope
src/ccutil/mainblk.cpp:72:15: note: in expansion of macro 'TESSDATA_PREFIX'
     datadir = TESSDATA_PREFIX "/tessdata";
               ^~~~~~~~~~~~~~~
<command-line>: error: 'share' was not declared in this scope
src/ccutil/mainblk.cpp:72:15: note: in expansion of macro 'TESSDATA_PREFIX'
     datadir = TESSDATA_PREFIX "/tessdata";
               ^~~~~~~~~~~~~~~
<command-line>: note: suggested alternative: 'unshare'
src/ccutil/mainblk.cpp:72:15: note: in expansion of macro 'TESSDATA_PREFIX'
     datadir = TESSDATA_PREFIX "/tessdata";
               ^~~~~~~~~~~~~~~
<command-line>: error: expected primary-expression before '-' token
src/ccutil/mainblk.cpp:72:15: note: in expansion of macro 'TESSDATA_PREFIX'
     datadir = TESSDATA_PREFIX "/tessdata";
               ^~~~~~~~~~~~~~~
<command-line>: error: 'ocr' was not declared in this scope
src/ccutil/mainblk.cpp:72:15: note: in expansion of macro 'TESSDATA_PREFIX'
     datadir = TESSDATA_PREFIX "/tessdata";
               ^~~~~~~~~~~~~~~
make[3]: *** [Makefile:6324: src/ccutil/libtesseract_ccutil_la-mainblk.lo] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: Leaving directory '/build/tesseract-5.0.0~git5670-386dd8a0'
make[2]: *** [Makefile:8148: all-recursive] Error 1
make[2]: Leaving directory '/build/tesseract-5.0.0~git5670-386dd8a0'
make[1]: *** [debian/rules:22: override_dh_auto_build] Error 2
make[1]: Leaving directory '/build/tesseract-5.0.0~git5670-386dd8a0'
make: *** [debian/rules:19: build] Error 2

@AlexanderP, please try -DTESSDATA_PREFIX='\"/usr/share/tesseract-ocr/5\"' as a temporary solution. A dedicated option for configure would be a better solution, for example configure --datadir=/usr/share/tesseract-ocr/5, but that must be implemented first.

@stweil Thanks, it helped

@stweil , this is a bit late, but thank you for taking care of this issue


Adhearsion dies on invalid Asterisk response

It would be nice if this totally invalid Asterisk response didn't take down all the other calls in the system.


Error "no notification handler registered" on diff

Describe the bug
Using vim-fugitive, the command ":Git difftool -y master" spams me several (4) times with the error message.
Comment https://github.com/rcarriga/vim-ultest/issues/44#issuecomment-859566430 looks similar
no notification handler registered for "/Users/myuser/.local/share/nvim/site/pack/packer/start/vim-ultest/rplugin/python3/ultest:function:_ultest_update_positions"
Enabling debug logs shows nothing, no matter if the command is started manually or via commandline (ULTEST_LOG_LEVEL=DEBUG ULTEST_LOG_FILE=vim-ultest.log nvim -c ":Git difftool -y master")
To Reproduce
:Git difftool -y
Expected behavior
No error
Screenshots
n/a
Additional context
n/a

Thanks for the report! Please try with the latest commit :grin:


ESHIP

Updated ESHIP Token details in json file and logo including some corrections. Please check. Thank you very much!
Token Name: EliteShipperToken
Token Symbol: ESHIP
Smart Contract Address: 0x06bead2ead661b51307b646f7419d5284330c135

是的，请。这样我们的徽标和令牌细节将在ImToken中可见。谢谢。
Yes please. So that our logo and token details will be visible in ImToken.
Thank you.
Token Name: EliteShipperToken
Token Symbol: ESHIP
Smart Contract Address:
0x06bead2ead661b51307b646f7419d5284330c135
On Wed, Jul 18, 2018 at 8:28 PM, ForBTM<EMAIL_ADDRESS>wrote:

是不是state 需要修改下
微信LALALA199814490
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/consenlabs/token-profile/pull/1136#issuecomment-405913545,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AmfJhwQrf8spvh1rXJJ1mZgzED9PVX0_ks5uHynwgaJpZM4VUP4_
.

你好，请移除多余的文件，谢谢。


DirectURL setting in configuration.js won't apply after modified

As mentioned by author, DircetURL's setting should be set in the file configuration.js then webUI will read it
But after many times I tried, its not worked as expected.
Finally I found a temporary solution, in js/services/rpc/rpc.js :
...
  var subscriptions = []
    , configurations = [authconf]
    , currentConf = {}
    , currentToken
    , timeout = null
    , forceNextUpdate = false;

  var cookieConf = utils.getCookie('aria2conf');
  // try at the start, so that it is presistant even when default authconf works
  if(cookieConf) configurations.unshift(cookieConf);
  if (['http', 'https'].indexOf(uri.protocol()) != -1 && uri.host() != 'localhost') {
    configurations.push({
      host: uri.host(),
      path: '/jsonrpc',
      port: 6800,
      encrypt: false,
      directURL: 'http://core:81/data_drive/'     // To add this line, as your DirectURL
    },
    {
      host: uri.host(),
      port: uri.port(),
      path: '/jsonrpc',
      encrypt: ( uri.protocol() == 'https' )
    },
    {
      host: uri.host(),
      port: uri.port(),
      path: authconf.path,
      encrypt: ( uri.protocol() == 'https' )
    });
  }
...

After changing, you may need to clear the cookies. Then every time you visit aria2-webUI (even refresh the page, restart aria2-rpc server), the default DirectURL can be read.
I have no idea why configuration.js does not work (zero knowledge in AngularJS).

Reproduction
On Linux
when accessing through file:/// then works well
using http://localhost/some/path/ then not working

okay, found the problem.
you should clean the cookies to make it working well.
in order to clean cookies, you MUST leave the index.html page first, since it will constantly set cookie/set cookie when leaving page.
easy way is to leave the page and remove all cookies
better way is to remove the cookies using developer tools.

@no1xsyzy You are correct. I should make sure the cookies of aria2-WebUI is completely clear first. Now configuaration.js works like a charm. Thank you for your help!

I cleared cookies in my browser but it doesn't work. I tried firefox and tried editing js/services/rpc/rpc.js and tried to change client. None of them works.


Moved inotify declarations up so that it works on android too.

Following my previous pull request to include inotify bindings, I would like to move the code so that it benefits android too.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @gnzlbg (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

:umbrella: The latest upstream changes (presumably #1231) made this pull request unmergeable. Please resolve the merge conflicts.

@bors: r+

:pushpin: Commit d25a054afe553f33c8225d7a3c175bd04ba31a8f has been approved by gnzlbg

:hourglass: Testing commit d25a054afe553f33c8225d7a3c175bd04ba31a8f with merge 501ca548b2e001d82ba88b716080c25759d11ca7...

:sunny: Test successful - checks-cirrus, checks-travis, status-appveyor
Approved by: gnzlbg
Pushing 501ca548b2e001d82ba88b716080c25759d11ca7 to master...


AssurancyOfAEP is throwing an exception

I am getting the following exception when trying to create the results for the scenario:

This is where i call that line:

This was using a rating curve and a stage damage curve that did not have overlapping stages. I switched my data to have overlapping stages and i don't get this exception anymore. Cody's study is called: New Model Updates

I am glad to see you found the bug, and identified the cause. I'll work on getting it fixed.


[WIP] Apply ocp global secrets to project created during tests

What type of PR is this?
/kind failing-test
/kind feature
What does this PR do / why we need it:
I have added dockerhub free account secrets to the psi clusters. This pr should enable the projects to use the account and import docker.io images. It should customise ocp secret configuration thing in proper way.
Which issue(s) this PR fixes:
Fixes #4739
Fixes part - #4606
PR acceptance criteria:

[ ] Unit test

[x] Integration test

[ ] Documentation

[ ] Update changelog

[x] I have read the test guidelines

How to test changes / Special notes to the reviewer:
All the tests need secret should pass successfully on prow CI and psi

@prietyc123 any update on this PR?

@prietyc123 any update on this PR?

Not working actively on this pr. I will close it for now.


fix(jwt-token): provide customclaims as multivalued attributes

Checklist

[x] the individual contributor license agreement is signed
[x] commit message follows commit guidelines
[ ] tests are included
[ ] documentation is changed or added
[ ] message properties have been updated with new phrases
[ ] view conforms with WCAG 2.0 AA

Description of change

Provide all user attributes defined in org.apereo.portal.security.oauth.IdTokenFactory.customClaims as multivalued.

I think it's the better way to obtains user attributes instead of using api /api/v5-0/people/me

This is a great change -- it's something we need.
I hope to take a look at this one (and the dialects one too) shortly.

/cc @mrapczynski

Sample JWT with this  change applied:
{
  "iss": "http://localhost:8080/uPortal",
  "sub": "jdoe",
  "aud": "http://localhost:8080/uPortal",
  "exp":<PHONE_NUMBER>,
  "iat":<PHONE_NUMBER>,
  "name": "Jane Doe",
  "given_name": "Jane",
  "family_name": "Doe",
  "groups": [
    "Students"
  ],
  "Student": [
    "A_572",
    "F_571",
    "A_571",
    "C_571",
    "A_576"
  ]
}

Looks perfect!  :+1:
Time to merge.


Tag 7.0.0-beta.0 as "beta" on npm

<EMAIL_ADDRESS>isn't tagged as "beta" on npm and is therefore automatically marked as "latest".
This causes automated checking of dependencies to give false positives.

I did this intentionally to get more eyeballs on the new version, but I didn't realize it would false-alarm automated tools. Added a dist-tag.

Here's what I get when checking directly with npm:
$ npm show eslint-config-standard dist-tags
{ latest: '6.2.1', beta: '7.0.0-beta.0' }

Sounds like either a caching issue, a bug in npm-check-updates, or maybe you're accidentally passing one of the following flags:
-n, --newest             find the newest published versions available instead
                         of the latest stable versions
-t, --greatest           find the highest versions available instead of the
                         latest stable versions


Password Field: Error in Validation

Hi
I think I've found a (very small) bug. When setting the password of a user (both in edit and create) the model validates when you exactly type in one character in the 'Confirm Password' textbox, and you don't change that character. When submitting, of course the value from the 'Password' textbox will be used, but you shouldn't be able to submit when the password entered in the 'Confirm Password' textbox isn't the same.
This is probably something Angular.js specific, and since I just know very little of it, I didn't dig in the code to find the issue. Just wanted to mention it, though.

Ok, thanks. I'll look into it.

Thank you too. :)

Fixed. Pushed to nuget as 1.0.0-beta5-4.


Public Domain Collection

Hi,
My name is Marcela, and I head up one of the Partnership Programs at Pond5.
As part of our library, we host a large public domain collection of 80.000+ HD and 4K files available for research and free consumption: http://goo.gl/vr7cM8.
I'd appreciate if you can add our library here.
:)
All the best.

Thanks for the contribution, great resource. I'll add soon.

Thanks a lot!
:)
Marcela Casarin
Youtube Sponsorship Project | Brand Building Team
E<EMAIL_ADDRESS><EMAIL_ADDRESS>https://www.facebook.com/pond5/ https://twitter.com/pond5
https://www.instagram.com/pond5official/
Pond5 memberships -  Learn more > http://pond5.com/memberships
On Sun, Apr 10, 2016 at 8:41 AM, Tony Phipps<EMAIL_ADDRESS>wrote:

Thanks for the contribution, great resource. I'll add soon.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/neutraltone/awesome-stock-resources/issues/106#issuecomment-207967801


cargo_generate_version is not respected

I noticed that cargo_generate_version doesn't work. Docs here.
I created a branch were I specified ther requirement:
[template]
cargo_generate_version = ">0.99.0"

But cargo generate still works if I run:
cargo generate --git https://github.com/rust-github/template --branch  wrong-version-test

Am I missing something?

Did you use the master branch version for testing?

Oooops, from master it works, sorry!
With the following commands it works:
$ git clone https://github.com/cargo-generate/cargo-generate
$ cd cargo-generate
$ cargo run -- generate --git https://github.com/rust-github/template --branch  wrong-version-test


Timeout with MachineGun.get!, but not with HTTPoison.get!

I’ve elided some secrets for a query that fails with MachineGun but succeeds with HTTPoison. I can share privately if you wish to work with this. I tried to reproduce just with gun, but don’t have enough familiarity with the Gun API to see what’s going on.
iex> endpoint = "https://b.aldodatahub.io/r/look_book_pages/c:-LJFx3L41up18hV4ngZz"
iex> headers = [
...>  {"authorization", "Bearer USERTOKEN"},
...>  {"accept", "application/json"},
...>  {"content-type", "application/json"},
...>  {"client-token", "APPTOKEN"},
...>  {"datahub-catalog", "aldo-ca"},
...>  {"datahub-client", "mce_ios"}
...>]
iex> HTTPoison.get!(endpoint, headers)
%HTTPoison.Response{
  body: "{…}",
  headers: […],
  request_url: "https://b.aldodatahub.io/r/look_book_pages/c:-LJFx3L41up18hV4ngZz",
  status_code: 200
}
iex> MachineGun.get!(endpoint, headers)
** (MachineGun.Error) :request_timeout
    (machine_gun) lib/machine_gun.ex:65: MachineGun.request!/5

This isn’t an HTTP/1 vs HTTP/2 issue, as far as I can tell (the server isn’t explicitly set up for HTTP/2, but it appears that the version of nginx we’re running as a proxy is taking care of that for us), so here’s the equivalent cURL with an appropriate log:
% curl -i -v "https://b.aldodatahub.io/r/look_book_pages/c:-LJFx3L41up18hV4ngZz" \
     -H 'Accept: application/json' \
     -H 'Authorization: Bearer USERTOKEN' \
     -H 'Client-Token: APPTOKEN' \
     -H 'Content-Type: application/json' \
     -H 'Datahub-Catalog: aldo-ca' \
     -H 'Datahub-Locale: en-CA' \
     -H 'Datahub-Client: mce_ios'

HTTP/2 200
date: Mon, 08 Oct 2018 19:54:20 GMT
content-type: application/json; charset=utf-8
cache-control: max-age=0, private, must-revalidate
status: 200 OK
x-request-id: 67477dc7-e4b2-4f95-82a4-8455519dc86e
x-xss-protection: 1; mode=block
x-runtime: 0.065549
datahub-catalog: aldo-ca
etag: W/"41aff44003038ae2860de69aadd3c1a2"
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-powered-by: Phusion Passenger 5.3.5
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 466b28a66bcfab66-YYZ

{…}

Looks like your backend server has issues with lowercase headers, which are required by HTTP/2:
https://tools.ietf.org/html/rfc7540#section-8.1.2
You can change headers to use capitalized headers and then force MachineGun to use HTTP/1 by adding this to your config:
config :machine_gun,
  default: %{
    conn_opts: %{
      protocols: [:http]
    }
  }

To see actual error you can increase request timeout:
MachineGun.get!(endpoint, headers, %{request_timeout: 30_000})

Here’s the error when the request timeout is increased.
iex(3)> MachineGun.get!(endpoint, headers, %{request_timeout: 60_000})
** (MachineGun.Error) {:stream_error, :protocol_error, :"Stream reset by server."}
    (machine_gun) lib/machine_gun.ex:65: MachineGun.request!/5

At this point, I’m fairly certain that the header case is not the cause. I was just reading the HTTPoison and Hackney code and I saw nothing that transformed the headers in any way, and I literally passed the same headers structure to both HTTPoison.get! and MachineGun.get! (so both had lowercase headers). I also modified my cURL request:
curl -i --http2 \
  "https://b.aldodatahub.io/r/look_book_pages/c:-LJFx3L41up18hV4ngZz" \
  -H 'accept: application/json' \
  -H 'authorization: Bearer USERTOKEN' \
  -H 'client-token: APPTOKEN' \
  -H 'content-type: application/json' \
  -H 'datahub-catalog: aldo-ca' \
  -H 'datahub-locale: en-CA' \
  -H 'datahub-client: mce_ios'

[SSL handshake elided]
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fd673805800)
> GET /r/look_book_pages/c:-LJFx3L41up18hV4ngZz HTTP/2
> Host: b.aldodatahub.io
> User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36
> Referer:
> accept: application/json
> authorization: Bearer USERTOKEN
> client-token: APPTOKEN
> content-type: application/json
> datahub-catalog: aldo-ca
> datahub-locale: en-CA
> datahub-client: mce_ios
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!

  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0< HTTP/2 200
< date: Tue, 09 Oct 2018 15:02:29 GMT
< content-type: application/json; charset=utf-8
< set-cookie: __cfduid=dfcb770b92160beb4bbaac1bccf79274b1539097349; expires=Wed, 09-Oct-19 15:02:29 GMT; path=/; domain=.aldodatahub.io; HttpOnly
< cache-control: max-age=0, private, must-revalidate
< datahub-model-image-size: 1480
< status: 200 OK
< x-request-id: 46774351-b8f9-47c5-a3d9-2979c9b38822
< x-xss-protection: 1; mode=block
< x-runtime: 0.060170
< datahub-catalog: aldo-ca
< etag: W/"a7a468d10dbf226e98df0ccfea4f4b03"
< x-content-type-options: nosniff
< x-frame-options: SAMEORIGIN
< x-powered-by: Phusion Passenger 5.3.5
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< server: cloudflare
< cf-ray: 4671ba7918b427e0-YYZ
<

HTTP/2 200
date: Tue, 09 Oct 2018 15:02:29 GMT
content-type: application/json; charset=utf-8
set-cookie: __cfduid=dfcb770b92160beb4bbaac1bccf79274b1539097349; expires=Wed, 09-Oct-19 15:02:29 GMT; path=/; domain=.aldodatahub.io; HttpOnly
cache-control: max-age=0, private, must-revalidate
datahub-model-image-size: 1480
status: 200 OK
x-request-id: 46774351-b8f9-47c5-a3d9-2979c9b38822
x-xss-protection: 1; mode=block
x-runtime: 0.060170
datahub-catalog: aldo-ca
etag: W/"a7a468d10dbf226e98df0ccfea4f4b03"
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-powered-by: Phusion Passenger 5.3.5
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 4671ba7918b427e0-YYZ

{"valid": "json lives here"}

Have you tried capitalizing headers and forcing HTTP/1?

Yes, you are right. "Stream reset by server." does not happen with curl.

Please try without content-type header. We have issue when content-type is specified for GET requests.

That was the problem.


feat: open weaviate functionalities to docarray

There are certain features in Weaviate that are not included with DocArray which are helpful to me and may be helpful to others.
Goals:

add the ability to pass additional properties to Weaviate
add the ability to pass sorting options to Weaviate
add the ability to limit return based on Weaviates Certainty score
update the docs

Ok, Ill try again.


Add colour config option to paperbox.

The core rulebooks have paperboxes in various colours. This patch makes the colour configurable per-box.

I agree that paperbox should have a configurable color. However the way this is implemented is a breaking change. If the paperbox environment is made with \DeclareTColorBox instead of \newtcolorbox, the functionality can be added without breaking existing documents. Can you make the following changes?

Remove the change to example.tex.
Swap where arguments #2 and #3 are used in lib/paperbox.
Change the declaration of the paperbox environment from \newtcolorbox to the following:

# Usage \begin{paperbox}[tcolorbox options]{title}[color]
\DeclareTColorBox{paperbox}{O{} m O{monstertandark}}{

I am sure we are going to run into some breaking changes that we cannot get around, but I think this is a case where we can support both the old and the new.

Would you please take a look at #48? I think that would be a better solution for changing the default color of a paperbox. Your solution here would then fit in for the few times someone wanted individual paperbox environments to have a different color. What do you think?

The solution in #48 is way more elegant. I'm in favour of it for both convenience and backward comparability reasons.

I am glad you like #48, but I would also like your solution with the modifications above for the times a user wants a one-off color change. Would you be willing to make that update?

@tsegers, thanks for getting this enhancement going!


The scraping no longer works.

@Proteusiq The scraping of reviews is no longer working.
Calling t.get_reviews() gives out an empty defaultdict like defaultdict(list, {}).

I will fix it tomorrow. Possible the change their API

I test a non-api route and it worked. So I think the API has changed
`import json
from requests import Session
from bs4 import BeautifulSoup
URL = 'https://dk.trustpilot.com/review/www.if.dk'
session = Session()
r = session.get(URL)
soup = BeautifulSoup(r.text,'html5lib')
data = soup.find('script',{'type':'application/ld+json'})
print(json.loads(data.getText(strip=True)))
`

@Proteusiq Even I thought so and tried scraping with requests and BeautifulSoup. But when I am looping over to get all the reviews and then checking the website manually through the browser, I found that it detects suspicious behavior on my IP and then I have to verify that "I am a human".

I see that I can restore it using BeautifulSoup with sleep function. I will wait in fixing it to find out the legality. As BeautifulSoup will overload their servers if this project is misused.

@Proteusiq Did you try anything further on this?