Not able to read vector<ULong64_t>

You would probably want to allow uproot to read  vector<ULong64_t> branches as a an array of integers
In [6]: a = events.array("TOT")
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-6dcabaf4504c> in <module>
----> 1 a = events.array("TOT")

/usr/local/miniconda3/envs/hepML/lib/python3.6/site-packages/uproot/tree.py in array(self, branch, interpretation, entrystart, entrystop, flatten, awkwardlib, cache, basketcache, keycache, executor, blocking)
    503     def array(self, branch, interpretation=None, entrystart=None, entrystop=None, flatten=False, awkwardlib=None, cache=None, basketcache=None, keycache=None, executor=None, blocking=True):
    504         awkward = _normalize_awkwardlib(awkwardlib)
--> 505         branches = list(self._normalize_branches(branch, awkward))
    506         if len(branches) == 1:
    507             if interpretation is None:

/usr/local/miniconda3/envs/hepML/lib/python3.6/site-packages/uproot/tree.py in _normalize_branches(self, arg, awkward, allownone, allowcallable, allowdict, allowstring, aliases)
    814
    815         elif allowstring and isinstance(arg, string_types):
--> 816             for x in self._normalize_branches([arg], awkward):
    817                 yield x
    818

/usr/local/miniconda3/envs/hepML/lib/python3.6/site-packages/uproot/tree.py in _normalize_branches(self, arg, awkward, allownone, allowcallable, allowdict, allowstring, aliases)
    852                         interpretation = interpret(branch, awkward)
    853                         if interpretation is None:
--> 854                             raise ValueError("cannot interpret branch {0} as a Python type\n   in file: {1}".format(repr(branch.name), self._context.sourcepath))
    855                         else:
    856                             yield branch, interpretation

ValueError: cannot interpret branch b'TOT' as a Python type
   in file: Root_Files/Run_5001_withSatHit.root

For clarity this is what happens when you read the same branch with RDataFrame:
In [2]: df = ROOT.RDataFrame("data", "Root_Files/Run_5001_withSatHit.root")
In [4]: npy1 = df.AsNumpy()
In [7]: print(npy1["TOT"])
[<ROOT.ROOT::VecOps::RVec<ULong64_t> object at 0x10aeb4000>
 <ROOT.ROOT::VecOps::RVec<ULong64_t> object at 0x10aeb4028>
 <ROOT.ROOT::VecOps::RVec<ULong64_t> object at 0x10aeb4050> ...
 <ROOT.ROOT::VecOps::RVec<ULong64_t> object at 0x10af15a08>
 <ROOT.ROOT::VecOps::RVec<ULong64_t> object at 0x10af15a30>
 <ROOT.ROOT::VecOps::RVec<ULong64_t> object at 0x10af15a58>]
In [8]: print(npy1["TOT"][1])
{ 1, 1, 3, 7, 19, 2, 2, 3, 13, 3, 3, 1, 5, 9, 12, 3, 2, 3, 6, 7, 4, 3, 3, 1, 1, 2, 1, 1, 2, 2, 8, 3, 16, 9, 11, 7, 3, 6, 3, 10, 6, 5, 1, 2, 4, 3, 1, 1, 1, 1, 4, 3, 1, 9, 12, 8, 19, 4, 8, 3, 5, 2, 2, 3, 1, 1, 2, 1, 1, 1, 1, 4, 1, 4, 4, 5, 3, 4, 3, 5, 8, 3, 4, 4, 8, 6, 5, 3, 2, 1, 2, 2, 1, 1, 1, 4, 1, 2, 2, 2, 1, 2, 5, 4, 5, 3, 3, 5, 3, 3, 6, 7, 3, 3, 1, 1, 1, 2, 1, 4, 3, 1, 4, 1, 1, 2, 2, 1, 2, 2, 2, 1, 2, 1, 1, 2, 1, 1 }

@chrisburr

This type is supported, but it seems that it's not being recognized in this case. The most likely reason is that your ROOT file doesn't have a streamer for it and the fallback that checks fClassName (a string) is looking for "vector<unsigned long>" and not vector<ULong64_t>. I wonder how many of these aliases there are...
I'll start a pull request so that you can confirm that this is, in fact, your issue, and then I'll need to be on the lookout for what other strings I might be missing.

PR #366. Let me know if that works. If so, I should also put in checks for "Long64_t", "UInt32_t", and "Int32_t", but I don't know what that whole set is.

Regardless of whether that works, I've added all the numerical types I found here: https://root.cern.ch/root/html528/ListOfTypes.html, which should be a good start at least.

PR #366. Let me know if that works. If so, I should also put in checks for "Long64_t", "UInt32_t", and "Int32_t", but I don't know what that whole set is.

I can confirm it now works:

With

(hepML) ➜  ~ pip install -e ~/Downloads/uproot-issue-365
Obtaining file:///Users/fredi/Downloads/uproot-issue-365
Requirement already satisfied: numpy>=1.13.1 in /usr/local/miniconda3/envs/hepML/lib/python3.6/site-packages (from uproot==3.10.3) (1.17.2)
Collecting awkward<1.0,>=0.12.0 (from uproot==3.10.3)
  Using cached https://files.pythonhosted.org/packages/71/e2/1b43063252f38ff7092428962e2eaa61970e736d9fe866369e32461493bc/awkward-0.12.12-py2.py3-none-any.whl
Collecting uproot-methods>=0.7.0 (from uproot==3.10.3)
  Using cached https://files.pythonhosted.org/packages/dc/c7/687a2764f61176fed7172c3ea51199b92490e08f37dac75d2f65c4c65d33/uproot_methods-0.7.1-py2.py3-none-any.whl
Collecting cachetools (from uproot==3.10.3)
  Downloading https://files.pythonhosted.org/packages/2f/a6/30b0a0bef12283e83e58c1d6e7b5aabc7acfc4110df81a4471655d33e704/cachetools-3.1.1-py2.py3-none-any.whl
Installing collected packages: awkward, uproot-methods, cachetools, uproot
  Running setup.py develop for uproot
Successfully installed awkward-0.12.12 cachetools-3.1.1 uproot uproot-methods-0.7.1

And therefore

(hepML) ➜  mc-maria-elena pip list
Package              Version   Location
-------------------- --------- ---------------------------------------
[...]
uproot               3.10.3    /Users/fredi/Downloads/uproot-issue-365
uproot-methods       0.7.1
[...]

I get:

In [2]: import uproot

In [3]: events = uproot.open("Root_Files/Run_5001_withSatHit.root")["data"]

In [4]: a = events.array("TOT")

In [5]: print(a)
[[2 1 5 ... 3 3 3] [1 1 3 ... 2 1 1] [1 3 4 ... 1 6 2] ... [3 1 3 ... 2 3 5] [3 3 3 ... 1 3 2] [9 3 8 ... 1 2 2]]

Thanks!

Great! Then I'll make it a new uproot version (probably today). Thanks for the report!


Update HOW_TO_RUN_THE_GAMES.md

Remove the dead links, and add a command-line option for compiling and running kotlin

thank you!


Sys-info route is removed in MS v0.13.0

According to this change in MS (https://github.com/meilisearch/MeiliSearch/issues/808), all the SDK should remove the GET sys-info route access.
⚠ This change is planned for the v0.13.0, not the v0.12.0
TODO:

[x] meilisearch-go
[x] meilisearch-js
[x] meilisearch-swift
[x] meilisearch-php
[x] meilisearch-python
[x] meilisearch-ruby

Finally closing this issue!


repeat_interleave or alternative needed to unpack quantized weights

I'm working on a Triton kernel to compute matmuls on quantized linear layers.   In particular where there are more than one parameters packed into a single value of an int32 Tensor.
The issue is that I could not find a way to "unpack" such Tensors in Triton.  For example, imagine I have an int32 Tensor of size [1, N//8], where each int32 represents eight 4-bit parameters.  Inside a Triton kernel how do I expand this into a [1, N] Tensor?
Something like PyTorch's repeat_interleave would work, as it would allow one to unroll the packed tensor.  From there one can apply shifting and masking to get the correct values unpacked at each index.
My current hack is the following:
b_ptrs = b_ptr + ((offs_k[:, None] // 8) * stride_bk + offs_bn[None, :] * stride_bn)   # (BLOCK_SIZE_K, BLOCK_SIZE_N)
shifter = (offs_k % 8) * 4

...
# Inside the inner loop:
	b = tl.load(b_ptrs)   # (BLOCK_SIZE_K, BLOCK_SIZE_N), but repeated

	# Now we need to unpack b (which is 4-bit values) into 32-bit values
	b = (b >> shifter[:, None]) & 0xF  # Extract the 4-bit values
	b = b * scales[None, :] - zeros[None, :]  # Scale and shift

This is based on the matmul tutorial code.  The major difference is that I divide the b_ptrs indexes by // 8.  This causes them to repeat along the K axis.  So I'm basically making tl.load act like repeat_interleave for me.  Then I can finish unpacking the values like normal.
The downside is that, as far as I'm aware, this results in 8x as many loads as compared to fetching the packed Tensor directly which is 8x smaller.
Having a built-in similar to repeat_interleave would allow me to unpack those values in SRAM and save the bandwidth.  Or maybe a way to index a Tensor?  Then I could build an interleaved index and do b[indexes].  But I didn't see any examples of indexing Tensors like that, so I assumed it wasn't possible in the language.
Does this functionality already exist?  Is there a better implementation?  Or should this be a feature request?
Thank you!

#974:

Yeah, on-chip indexing through shared memory isn't supported yet. It's on the roadmap though, but it's a pretty advanced feature so we haven't come up with a specific timeline yet.

Looks like we might see this in the future

#974:

Yeah, on-chip indexing through shared memory isn't supported yet. It's on the roadmap though, but it's a pretty advanced feature so we haven't come up with a specific timeline yet.

Looks like we might see indexing support in the future

Hello, I would like to confirm when this feature is expected to be supported?

bumping this

Advanced tensor indexing feature wanted!

When you divide the indices offs_k[:, None] // 8 you actually end-up with interleaved indices. Loading is pretty fast with this approach on some devices like ADA gpus like the 4090 / A6000 Ada, but I noticed loading is pretty slow on the A100 / H100.
Reading the small chunk and interleaving aka "repeat_interleave" with something like this is actually even worse:
b = tl.load(b_ptrs).trans()
b = tl.interleave(b, b)
b = tl.interleave(b, b)
b = tl.interleave(b, b).trans()

I reported a similar issue here: https://github.com/triton-lang/triton/issues/4906


I made follow guide, but there is issue. I don't know how to solve, please help me

External Native Build Issues
Build command failed.
Error while executing process C:\Users\DavidSonNguyen\AppData\Local\Android\Sdk\cmake\3.6.4111459\bin\cmake.exe with arguments {-HC:\Users\DavidSonNguyen\Desktop\TestCipher\app\build\cipher.so\cmake -BC:\Users\DavidSonNguyen\Desktop\TestCipher\app\outputs\cmake\cmake\debug\armeabi-v7a -DANDROID_ABI=armeabi-v7a -DANDROID_PLATFORM=android-15 -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=C:\Users\DavidSonNguyen\Desktop\TestCipher\app\build\intermediates\cmake\debug\obj\armeabi-v7a -DCMAKE_BUILD_TYPE=Debug -DANDROID_NDK=C:\Users\DavidSonNguyen\AppData\Local\Android\Sdk\ndk-bundle -DCMAKE_CXX_FLAGS= -DCMAKE_TOOLCHAIN_FILE=C:\Users\DavidSonNguyen\AppData\Local\Android\Sdk\ndk-bundle\build\cmake\android.toolchain.cmake -DCMAKE_MAKE_PROGRAM=C:\Users\DavidSonNguyen\AppData\Local\Android\Sdk\cmake\3.6.4111459\bin\ninja.exe -GAndroid Gradle - Ninja}
-- Check for working C compiler: C:/Users/DavidSonNguyen/AppData/Local/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe
-- Check for working C compiler: C:/Users/DavidSonNguyen/AppData/Local/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Users/DavidSonNguyen/AppData/Local/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/windows-x86_64/bin/clang++.exe
-- Check for working CXX compiler: C:/Users/DavidSonNguyen/AppData/Local/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/windows-x86_64/bin/clang++.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:5 (add_subdirectory):
C:/Users/DavidSonNguyen/Desktop/TestCipher/app/build/cipher.so/cmake/CMakeLists.txt:5
when parsing string
C:\Users\DavidSonNguyen\Desktop\TestCipher\app\build\cipher.so
Invalid character escape '\U'.
-- Configuring incomplete, errors occurred!
See also "C:/Users/DavidSonNguyen/Desktop/TestCipher/app/outputs/cmake/cmake/debug/armeabi-v7a/CMakeFiles/CMakeOutput.log".
Build command failed.
Error while executing process C:\Users\DavidSonNguyen\AppData\Local\Android\Sdk\cmake\3.6.4111459\bin\cmake.exe with arguments {-HC:\Users\DavidSonNguyen\Desktop\TestCipher\app\build\cipher.so\cmake -BC:\Users\DavidSonNguyen\Desktop\TestCipher\app\outputs\cmake\cmake\release\armeabi-v7a -DANDROID_ABI=armeabi-v7a -DANDROID_PLATFORM=android-15 -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=C:\Users\DavidSonNguyen\Desktop\TestCipher\app\build\intermediates\cmake\release\obj\armeabi-v7a -DCMAKE_BUILD_TYPE=Release -DANDROID_NDK=C:\Users\DavidSonNguyen\AppData\Local\Android\Sdk\ndk-bundle -DCMAKE_CXX_FLAGS= -DCMAKE_TOOLCHAIN_FILE=C:\Users\DavidSonNguyen\AppData\Local\Android\Sdk\ndk-bundle\build\cmake\android.toolchain.cmake -DCMAKE_MAKE_PROGRAM=C:\Users\DavidSonNguyen\AppData\Local\Android\Sdk\cmake\3.6.4111459\bin\ninja.exe -GAndroid Gradle - Ninja}
-- Check for working C compiler: C:/Users/DavidSonNguyen/AppData/Local/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe
-- Check for working C compiler: C:/Users/DavidSonNguyen/AppData/Local/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Users/DavidSonNguyen/AppData/Local/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/windows-x86_64/bin/clang++.exe
-- Check for working CXX compiler: C:/Users/DavidSonNguyen/AppData/Local/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/windows-x86_64/bin/clang++.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:5 (add_subdirectory):
C:/Users/DavidSonNguyen/Desktop/TestCipher/app/build/cipher.so/cmake/CMakeLists.txt:5
when parsing string
C:\Users\DavidSonNguyen\Desktop\TestCipher\app\build\cipher.so
Invalid character escape '\U'.
-- Configuring incomplete, errors occurred!
See also "C:/Users/DavidSonNguyen/Desktop/TestCipher/app/outputs/cmake/cmake/release/armeabi-v7a/CMakeFiles/CMakeOutput.log".
C:\Users\DavidSonNguyen\AppData\Local\Android\sdk\ndk-bundle\build\cmake\android.toolchain.cmake
Warning:(63, 98)  (include)   CMakeLists.txt
Warning:(63, 98)  (include)   CMakeLists.txt

This issue is cased by file separator under the windows os.

How can i fix it?

Fixed by v0.0.3, thanks for feedback~


Start developing semaphores

It would be great if you could check this first implementation of semaphores.
The "blocking" functions are not implemented yet, since they are not necessary. I'll be doing that today, but for the small tests i've made, this is starting to work.
There are many commented lines of code that are related with the blocking functions

I like the solid progress! The sem interface is looking 👌

At this point I'm just throwing warnings or pointing out corrections "for correctness". You can decide if you want to apply any of them and merge 👍

At this point I'm just throwing warnings or pointing out corrections "for correctness". You can decide if you want to apply any of them and merge +1

Im constantly learning about your suggestions, so feel free to comment anything that can be done better


Add field has_custom_avatar to user profile or similar table.

Hello, we fixed this issue in askbot - each user has a denormalized record on whether they have a real avatar or not.
Actually - the record stores type of avatar: auto-gravatar, uploaded gravatar, locally uploaded avatar.
Would you like to include this into django-avatar? Probably most of the code can be borrowed from askbot, we could help too.
This record helps to avoid displaying auto-generated gravatars, which look quite ugly especially if you show a few side by side.

@evgenyfadeev Sure - just make a PR.


introducing the SqlManagement in Microsoft.SqlVirtualMachine

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[x] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[x] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

Automation for azure-sdk-for-ruby

Automation for azure-sdk-for-python

Automation for azure-sdk-for-java

Automation for azure-sdk-for-go

Can one of the admins verify this patch?

@ praries880 could you help to take a look at this change? we need to this out sooner than later to unblock an announcement this week.

If you want to use "LUNs" instead of "LUN", you might need to add the word to https://github.com/Azure/azure-rest-api-specs/blob/master/custom-words.txt

@SeliuMSFT Kindly fix the spelling mistakes pointed out in :
https://dev.azure.com/azure-sdk/public/_build/results?buildId=60485&view=logs&j=eca5a1cb-f12a-57d8-9254-390bd587c201&t=be275096-42fe-5449-ec4b-01c6f327d2b3&l=18
To whitelist "LUNs" you will need to update : https://github.com/Azure/azure-rest-api-specs/blob/master/custom-words.txt


Failed to communicate with heketi service. kubectl version 1.15

On the master branch the master branch the gk-deploy commands fails to complete with kubectl v 1.15
output  kubectl version:
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:54Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:05:50Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
The command and output
_root@KUBEM01:~/glusterfs/gluster-kubernetes/deploy# ./gk-deploy -g topology.json --admin-key "a7UQ9hMJb1WkwAOcExmn" --user-key "a1AJ2coFUAZBZZLjHj4L"
Welcome to the deployment tool for GlusterFS on Kubernetes and OpenShift.
Before getting started, this script has some requirements of the execution
environment and of the container platform that you should verify.
The client machine that will run this script must have:

Administrative access to an existing Kubernetes or OpenShift cluster
Access to a python interpreter 'python'

Each of the nodes that will host GlusterFS must also have appropriate firewall
rules for the required GlusterFS ports:

2222  - sshd (if running GlusterFS in a pod)
24007 - GlusterFS Management
24008 - GlusterFS RDMA
49152 to 49251 - Each brick for every volume on the host requires its own
port. For every new brick, one new port will be used starting at 49152. We
recommend a default range of 49152-49251 on each host, though you can adjust
this to fit your needs.

The following kernel modules must be loaded:

dm_snapshot
dm_mirror
dm_thin_pool

For systems with SELinux, the following settings need to be considered:

virt_sandbox_use_fusefs should be enabled on each node to allow writing to
remote GlusterFS volumes

In addition, for an OpenShift deployment you must:

Have 'cluster_admin' role on the administrative account doing the deployment
Add the 'default' and 'router' Service Accounts to the 'privileged' SCC
Have a router deployed that is configured to allow apps to access services
running in the cluster

Do you wish to proceed with deployment?
[Y]es, [N]o? [Default: Y]: y
Using Kubernetes CLI.
Using namespace "default".
Checking for pre-existing resources...
GlusterFS pods ... found.
deploy-heketi pod ... not found.
heketi pod ... found.
gluster-s3 pod ... not found.
Error: unknown flag: --show-all
Examples:
List all pods in ps output format.
kubectl get pods
List all pods in ps output format with more information (such as node name).
kubectl get pods -o wide
List a single replication controller with specified NAME in ps output format.
kubectl get replicationcontroller web
List deployments in JSON output format, in the "v1" version of the "apps" API group:
kubectl get deployments.v1.apps -o json
List a single pod in JSON output format.
kubectl get -o json pod web-pod-13je7
List a pod identified by type and name specified in "pod.yaml" in JSON output format.
kubectl get -f pod.yaml -o json
List resources from a directory with kustomization.yaml - e.g. dir/kustomization.yaml.
kubectl get -k dir/
Return only the phase value of the specified pod.
kubectl get -o template pod/web-pod-13je7 --template={{.status.phase}}
List resource information in custom columns.
kubectl get pod test-pod -o custom-columns=CONTAINER:.spec.containers[0].name,IMAGE:.spec.containers[0].image
List all replication controllers and services together in ps output format.
kubectl get rc,services
List one or more resources by their type and names.
kubectl get rc/web service/frontend pods/web-pod-13je7
Options:
-A, --all-namespaces=false: If present, list the requested object(s) across all namespaces. Namespace in current context is ignored even if specified with --namespace.
--allow-missing-template-keys=true: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats.
--chunk-size=500: Return large lists in chunks rather than all at once. Pass 0 to disable. This flag is beta and may change in the future.
--field-selector='': Selector (field query) to filter on, supports '=', '==', and '!='.(e.g. --field-selector key1=value1,key2=value2). The server only supports a limited number of field queries per type.
-f, --filename=[]: Filename, directory, or URL to files identifying the resource to get from a server.
--ignore-not-found=false: If the requested object does not exist the command will return exit code 0.
-k, --kustomize='': Process the kustomization directory. This flag can't be used together with -f or -R.
-L, --label-columns=[]: Accepts a comma separated list of labels that are going to be presented as columns. Names are case-sensitive. You can also use multiple flag options like -L label1 -L label2...
--no-headers=false: When using the default or custom-column output format, don't print headers (default print headers).
-o, --output='': Output format. One of: json|yaml|wide|name|custom-columns=...|custom-columns-file=...|go-template=...|go-template-file=...|jsonpath=...|jsonpath-file=... See custom columns [http://kubernetes.io/docs/user-guide/kubectl-overview/#custom-columns], golang template [http://golang.org/pkg/text/template/#pkg-overview] and jsonpath template [http://kubernetes.io/docs/user-guide/jsonpath].
--raw='': Raw URI to request from the server.  Uses the transport specified by the kubeconfig file.
-R, --recursive=false: Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.
-l, --selector='': Selector (label query) to filter on, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)      --server-print=true: If true, have the server return the appropriate table output. Supports extension APIs and CRDs.
--show-kind=false: If present, list the resource type for the requested object(s).
--show-labels=false: When printing, show all labels as the last column (default hide labels column)
--sort-by='': If non-empty, sort list types using this field specification.  The field specification is expressed as a JSONPath expression (e.g. '{.metadata.name}'). The field in the API resource specified by this JSONPath expression must be an integer or a string.
--template='': Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview].
-w, --watch=false: After listing/getting the requested object, watch for changes. Uninitialized objects are excluded if no object name is provided.
--watch-only=false: Watch for changes to the requested object(s), without listing/getting first.
Usage:
kubectl get [(-o|--output=)json|yaml|wide|custom-columns=...|custom-columns-file=...|go-template=...|go-template-file=...|jsonpath=...|jsonpath-file=...] (TYPE[.VERSION][.GROUP] [NAME | -l label] | TYPE[.VERSION][.GROUP]/NAME ...) [flags] [options]
Use "kubectl options" for a list of global command-line options (applies to all commands).
unknown flag: --show-all
Failed to communicate with heketi service._
The fix
This can be fixed by removing the '--show-all' in line:
https://github.com/gluster/gluster-kubernetes/blob/7246eb4053c8c5336e4da68d86b76124d435eb3e/deploy/gk-deploy#L924

#589 & #582


Some googleclouddisk e2e test improvements

IAM roles and service account are cleaned up when test is done
debug flag is set for each turbiniactl command
e2e return value parsing improvements
include GCS stored output of each task in local archive (excluding StringAsciiTask on whole disk due to size)

Travis fails with "ImportError: cannot import name 'collections_abc'" <- this seems to be something else and nothing to do with the changes in this PR.

Yeah, I filed https://github.com/google/turbinia/issues/604 for the unrelated test error, and I'm still looking into it, but it seems like it's a version conflict in upstream cloud libraries.


Improving segment generation code in hadoop

Adding indexing parameters that add startree, inverted indices, hllconfig, partition config, and no dictionary columns on the segment generation side based on parameters pulled from table config.

Codecov Report

Merging #2966 into master will decrease coverage by 0.13%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #2966      +/-   ##
==========================================
- Coverage   69.18%   69.04%   -0.14%
==========================================
  Files         914      914
  Lines       43513    43538      +25
  Branches     5934     5938       +4
==========================================
- Hits        30103    30061      -42
- Misses      11391    11463      +72
+ Partials     2019     2014       -5

Impacted Files
Coverage Δ

...indexsegment/generator/SegmentGeneratorConfig.java
56.18% <0%> (-5.45%)
:arrow_down:

...lix/EmptyBrokerOnlineOfflineStateModelFactory.java
86.66% <0%> (-13.34%)
:arrow_down:

.../impl/dictionary/LongOffHeapMutableDictionary.java
81.81% <0%> (-10.91%)
:arrow_down:

...pinot/controller/validation/ValidationManager.java
77.19% <0%> (-7.61%)
:arrow_down:

...egation/function/customobject/MinMaxRangePair.java
68.96% <0%> (-6.9%)
:arrow_down:

...inkedin/pinot/common/config/RealtimeTagConfig.java
93.33% <0%> (-6.67%)
:arrow_down:

...impl/dictionary/DoubleOnHeapMutableDictionary.java
68.88% <0%> (-6.67%)
:arrow_down:

.../impl/dictionary/FloatOnHeapMutableDictionary.java
86.66% <0%> (-6.67%)
:arrow_down:

...edin/pinot/common/metadata/ZKMetadataProvider.java
70.06% <0%> (-6.13%)
:arrow_down:

...elix/core/relocation/RealtimeSegmentRelocator.java
73.33% <0%> (-5%)
:arrow_down:

... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d87add7...dc82b68. Read the comment docs.


Fixing import of lalsimulation.gwsignal to allow for debuggers

For some reason, when stepping through import lalsimulation.gwsignal, the debugger hangs and is unable to step over the import. This is despite when disconnecting the debugger (or alternatively just running the file with regular python) the import works fine.

This fix moves the location of the import so that lalsimulation.gwsignal is only imported when the NewInterfaceWaveformGenerator is instantiated. The functions from lalsimulation.gwsignal are then sent to the global namespace.

This PR also removes pyseobnr as a required dependency for installation. You can install dingo with pyseobnr by running pip install dingo-gw[pyseobnr].

With conda my suggestion would be to not ship pyseobnr with dingo by default and just tell the user they need to install it on their own if they want to use v5.

@nihargupte-ph I think this is going to cause some tests involving SEOBNRv5PHM to fail if pyseobnr is not installed:
https://github.com/dingo-gw/dingo/blob/b93eb0367fac2cc1ff948625fe4666fa55d204f0/tests/gw/waveform_generator/test_wfg_m.py#L40

@nihargupte-ph I think this is going to cause some tests involving SEOBNRv5PHM to fail if pyseobnr is not installed:
https://github.com/dingo-gw/dingo/blob/b93eb0367fac2cc1ff948625fe4666fa55d204f0/tests/gw/waveform_generator/test_wfg_m.py#L40

Hmm yes you are right. What should be the expected behavior here, I think we should run the pipeline tests with pyseob installed. But for users that don't have pyseob installed maybe we can add a line that checks if pyseob is installed and if it is not it just passes the test?

I think if pyseobnr is not installed, then skip the test for SEOBNRv5.


Rancher/server reports wrong info about host

rancher/server:v0.21.3
In host info view it's displayed a wrong docker version and no storage info.

docker version
Client version: 1.6.1
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): 97cd073
OS/Arch (client): linux/amd64
Server version: 1.6.1
Server API version: 1.18
Go version (server): go1.4.2
Git commit (server): 97cd073
OS/Arch (server): linux/amd64

docker info
Containers: 5
Images: 85
Storage Driver: devicemapper
 Pool Name: docker-253:0-2995115-pool
 Pool Blocksize: 65.54 kB
 Backing Filesystem: xfs
 Data file: /dev/gene/docker-data
 Metadata file: /dev/gene/docker-metadata
 Data Space Used: 2.094 GB
 Data Space Total: 107.4 GB
 Data Space Available: 105.3 GB
 Metadata Space Used: 4.841 MB
 Metadata Space Total: 2.487 GB
 Metadata Space Available: 2.482 GB
 Udev Sync Supported: true
 Library Version: 1.02.88 (2014-08-05)
Execution Driver: native-0.2
Kernel Version: 3.18.9-hardened-052391
Operating System: Gentoo/Linux
CPUs: 8
Total Memory: 15.55 GiB
Name: gene
ID: 2Z7Y:GRV3:FBUZ:3BYV:QQWY:6WCD:LL2F:BZ3V:F36H:5I5S:G7P3:5SMN

@cloudnautique we need to make sure we get the docker version of the server, not the client as we package the client in the agent image.

build from master - June 10th (afternoon)
Verified fixed


Add properties schema for .nodePools and .nodeClasses

This PR

adds properties schema for .nodePools and .nodeClasses
Moves some property definitions into the common $defs section, to enable re-use
normalizes indentation

Towards https://github.com/giantswarm/roadmap/issues/1738 https://github.com/giantswarm/roadmap/issues/1181
Checklist

[x] Update changelog in CHANGELOG.md.
[x] Make sure values.yaml and values.schema.json are valid.
[ ] Update Lastpass values if required.

@marians Thanks a lot for improving the schema. Did you prepare this PR manually? Do we have a tool to generate this automatically now?

Did you prepare this PR manually?

Yes

Do we have a tool to generate this automatically now?

No, we don't. And I wonder whether this is actually a problem.
If you run make schema-gen, you cannot apply all changes when committing, as this would overwrite (delete) additions made manually. (This should come up in PR reviews and should mean that reviewers ask questions!) You could selectively commit the desired changes instead.
As we want our schemas to be more complete, the value of make schema-gen is questionable anyway. While it's useful to create an initial scaffold, it does not help filling in title, description, examples etc. Also it does not set any restrictions like minimum, maximum or minLength, maxLength. Bottom line: I think we must drop schema-gen for maintenance.
As for alternatives, we have a bit of discussion already in https://github.com/giantswarm/roadmap/issues/1733 and I'd be happy to follow up there.


[TIR][Schedule] Opaque block does not satisfy stage-pipeline

Stage pipeline property requires all stmts in scope are schedulable stmts, which means opaque block (block without block vars) does not satisfy the property.
This PR fixes the bug.
cc @junrushao1994 @spectrometerHBH

If users write an opaque block and try to do transformations on it, we will not raise any schedule errors but only provide error result IR.

Any updates?

Any updates?

Let's close this PR since it might need a generic solution.


Create an empty ~/.dockercfg if one isn't present.

I originally wanted to change bin/bootstrap to be more like this and just use ~/.dockercfg if provided, but the AWS CLI doesn't really like JSON. Maybe I'm doing something blatantly wrong.
Closes https://github.com/remind101/empire/issues/537

:+1:


Decide on "object" or "component", and use throughout

Let's decide which terminlogy to use, and use it consistently.

I think we can include this in 3.3?

We could have "Component Id" for RSMP core/messages (we have "cId" JSON tag to adress the component), actually used in majority cases.
And we could perhaps keep "object" (and "object type") for abstract part (with link for signals) if wanted (SCADA, objects, type of objects, ...)
In first terms, we have the definition to understand the link :
"An object is identified using component id. Please note that an object is not necessarily the same thing as an NTS object."
Enough?

My suggestion was to use one or the the other throughout.


chore: pin op-alloy-consensus

Description
Small PR to pin the op-alloy-consensus dep version.
Prepares for workspace release.

#304  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @refcell and the rest of your teammates on  Graphite

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 79.7%. Comparing base (28721fa) to head (9191ca0).

Additional details and impacted files

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


hook to open the chat from a button or event

Is your feature request related to a problem? Please describe.
It is an enactment, I have a case where would be good to have a method to open the chat in case the user clicks on contact Me
Describe the solution you'd like
a hook that opens the chat so the user can see the popup and type the message there.

I second this.


Spark-operator issue with istio

Currently I have istio enabled on the aks cluster.
However, even though the istio-proxy & spark-operator container are running, there are still some issue.

The webhook fails to communicate and kubectl logs {operator-pod} -c {operator-container} give the following for action in webhook.go.
It is quite hard to debug, even by checking istiod logs and istio proxy log.

yyyy/MM/dd 03:40:00 http: TLS handshake error from <IP_ADDRESS>:port: EOF
yyyy/MM/dd  03:40:00 http: TLS handshake error from <IP_ADDRESS>:port: EOF

After installing spark-operator without istio, we find that corresponding log should be
I0xxx 03:30:05.xxxx       9 webhook.go:246] Serving admission request
I0xxx 03:30:05.xxxx       9 webhook.go:540] Pod  in namespace xxx is not subject to mutation

After submitting a sparkapplication, it is found that sparkapplication is successfully created/deleted by checking "kubectl get sparkapplication", but the corresponding actions on driver podwould be performed only after an nondeterministic period (maybe few hours).  Operator logs will also be flooded after this period. Within this nondeterministic period, no logs except the webhook error log mentioned above are shown.

After the driver pod is generated, it is found that within also an nondeterministic period since the pod is generated (or more accurately is that the spark-operator starts to work), new sparkapplications and driver pods can be generated immediately. Same apply to delete action.

For 2 & 3, I suspect is the spark-operator (even for controller) are somehow getting blocked? Because the controller fails to watch the CRD events immediately.

All the above issue disappears when we do not enable istio injection.
However avoiding istio-injection may not be a valid workaround for my case... (even though it seems istio is currently not well-supported by spark-operator).
Has anyone also encountered similar issue? Any method to debug those issue? Appreciate if anyone in the community could help.

These are a few of the things we have done in our cluster to get it working with Istio:

We exclude the webhook port from istio to allow the API to connect to the port properly
We run a PeerAuthentication config that makes port 4040, 7078 and 7079 to permissive for matchlabels for spark-role: driver
We run an internally built operator docker image that runs the entrypoint script with envoy-preflight as well as pulls in a cherry picked gencert.sh script to work around an issue that has been resolved in this repo but hadnt been released yet

May ask if you have encounter the long delay issue?


NextPage does 2 steps at once

Calling WizardControl.NextPage(wizardPage, true) results in opening wizardPage,
but also in an inmediate continuation to the page following that in the WizardControl.Pages,
which may even cause the entire wizard to finish.
Strangely, this second next doesn't fire the Commit event.
But is does fire a second SelectedPageChanged.
This breaks my navigation.
I currently have no way of preventing it.
This is a try-out in which I have already put considerable effort.
This may be the breaking problem.
Please react to this issue.

Update.
The way to interrupt this behaviour is to set WizardPageConfirmEventArgs.Cancel = true
in the Commit handler.


Unauthenticated Users - Products

As a user, I want to view the full list of products (the product catalog), so that I can see everything that's available
Implementation: write an Express route to serve all products on the backend (GET /api/products) should serve up the name, price, quantity, and imageUrl of all the products from our Postgres database. No special access control is needed, since any visitor should be able to receive this information.

As I user, I want to refine product listings by category, so that I can narrow down my choices to see only the types of items I'm interested in.
As I search product listings, so that I can find specific products I want by name.
Implementation:


IdNames rule for classes?

I just enabled the IdNames rule and discovered that it doesn't apply to classes, and there doesn't appear to be an equivalent for classes. Any chance of adding a rule for this?

More than happy to accept a pull request!


Exception system usability

In the process of incorporating the middleware from this package into a Slim-based service I'm building, I wrote a separate middleware to handle errors from this one. I've enclosed what I've come up with so far below.
My middleware is admittedly more complex than I would like, but I think this is reflective of this package's current exception system. I developed this middleware iteratively by throwing requests at the service for basic use cases: an unrecognized method, an unsupported method, a body that didn't conform to the endpoint's schema, etc.
Here are a few specific points of note.

There is no central base class from which all other exceptions extend, making it more difficult to implement a comprehensive handler for exceptions from this package.
While it's good that exception classes make use of the $previous parameter, exception chains can make it tedious to get to the crux of a problem.
While there are some areas that could be subject to debate (e.g. using a 422 response code for an invalid request payload, as opposed to say 400), there are also areas with an intuitive established behavior (e.g. 404 for an unknown path, 405 for an unsupported method, etc.). It would be nice if this package could handle commonly agreed upon cases, perhaps with an additional middleware.

I don't have a concrete solution to this, so I'm opening this issue mainly to spark discussion around this subject, whether in the form of BC-breaking changes to the exception system in this package's next major version or an additional middleware to encapsulate the complexity of handling exceptions for common use cases.
use League\OpenAPIValidation\PSR15\Exception\InvalidServerRequestMessage;
use League\OpenAPIValidation\PSR7\Exception\NoOperation;
use League\OpenAPIValidation\PSR7\Exception\ValidationFailed;
use League\OpenAPIValidation\PSR7\Exception\Validation\InvalidBody;
use League\OpenAPIValidation\Schema\Exception\SchemaMismatch;
use Fig\Http\Message\StatusCodeInterface as Status;
use Psr\Http\Message\ResponseFactoryInterface as ResponseFactory;
use Psr\Http\Message\ResponseInterface as Response;
use Psr\Http\Message\ServerRequestInterface as Request;
use Psr\Http\Server\MiddlewareInterface;
use Psr\Http\Server\RequestHandlerInterface;
use Slim\Exception\HttpException;
use Throwable;

class ErrorMiddleware implements MiddlewareInterface
{
    private ResponseFactory $responseFactory;

    public function __construct(ResponseFactory $responseFactory)
    {
        $this->responseFactory = $responseFactory;
    }

    public function process(Request $request, RequestHandlerInterface $handler): Response
    {
        try {
            return $handler->handle($request);
        } catch (Throwable $e) {
            return $this->handleException($request, $e);
        }
    }

    private function handleException(Request $request, Throwable $exception): Response
    {
        while ($exception instanceof InvalidServerRequestMessage
            || $exception instanceof InvalidBody) {
            $exception = $exception->getPrevious();
        }

        if ($exception instanceof NoOperation) {
            $route = $request->getAttribute('route');
            if (empty($route)) {
                $code = Status::STATUS_NOT_FOUND;
                $message = 'Path not found: ' . $request->getUri()->getPath();
            } else {
                $code = Status::STATUS_METHOD_NOT_ALLOWED;
                $message = 'Path does not support method: ' . $request->getMethod();
            }
        } elseif ($exception instanceof ValidationFailed || $exception instanceof SchemaMismatch) {
            $code = Status::STATUS_UNPROCESSABLE_ENTITY;
            $message = $exception->getMessage();
        } else {
            $code = Status::STATUS_INTERNAL_SERVER_ERROR;
            $message = get_class($exception) . ': ' . $exception->getMessage();
        }

        $response = $this->responseFactory->createResponse($code)
            ->withHeader('Content-Type', 'application/json');
        $response->getBody()->write(json_encode(['message' => $message]));
        return $response;
    }
}

There is no central base class from which all other exceptions extend, making it more difficult to implement a comprehensive handler for exceptions from this package.

What about

There is no central base class from which all other exceptions extend, making it more difficult to implement a comprehensive handler for exceptions from this package.

What about

What about ValidationFailed ? If SchemaMismatch is leaking out in PSR7 validator - this must be fixed. This code clearly states that MessageValidator throws only ValidationFailed

Fair enough, I stand corrected on this point.

I didn't get this point at all, sorry. Chaining exceptions is best practice. What is your suggestion?

Best practice for allowing debugging to be conducted through an application layer and multiple levels of supporting libraries, sure.
However, chaining within the same library can make it more difficult to get to the exception that initially caused a given issue. It requires a loop similar to this one to get to the root exception.
 while ($exception instanceof InvalidServerRequestMessage
    || $exception instanceof InvalidBody) {
    $exception = $exception->getPrevious();
}

The last exception on the chain (i.e. the one that ends up bubbling up) should have a message that makes the root problem and how to fix it obvious.
For example, why are exceptions chained in these places, rather than just letting the originally thrown exceptions bubble up?  Slim middleware is historical legacy here, so we're not removing it for now.
But we will be happy if one is built around this package as a separate package.

Was this discussed on an issue that I could review? I'm trying to understand the reasoning behind this.
As it stands now, any new user of this library has to go through the same process I did of iteratively writing code to handle common exceptions (e.g. 404, 405) with corresponding responses. A lot of reasonable behaviors could be implemented in a separate middleware in this library (to make the behavior opt-in and not break BC) that would make it more user-friendly.
Nothing says this has to be Slim-specific; my Slim example just happened to be the one I had most readily available to illustrate my points. There may be a PSR-7/15-compatible way to handle this.

What about ValidationFailed ? If SchemaMismatch is leaking out in PSR7 validator - this must be fixed. This code clearly states that MessageValidator throws only ValidationFailed

Fair enough, I stand corrected on this point.

I didn't get this point at all, sorry. Chaining exceptions is best practice. What is your suggestion?

Best practice for allowing debugging to be conducted through an application layer and multiple levels of supporting libraries, sure.
However, chaining within the same library can make it more difficult to get to the exception that initially caused a given issue. It requires a loop similar to this one to get to the root exception.
 while ($exception instanceof InvalidServerRequestMessage
    || $exception instanceof InvalidBody) {
    $exception = $exception->getPrevious();
}

The last exception on the chain (i.e. the one that ends up bubbling up) should have a message that makes the root problem and how to fix it obvious.
For example, why are exceptions chained in these places, rather than just letting the originally thrown exceptions bubble up?  Slim middleware is historical legacy here, so we're not removing it for now.
But we will be happy if one is built around this package as a separate package.

Was this discussed on an issue that I could review? I'm trying to understand the reasoning behind this.
As it stands now, any new user of this library has to go through the same process I did of iteratively writing code to handle common exceptions (e.g. 404, 405) with corresponding responses. A lot of reasonable behaviors could be implemented in a separate middleware in this library (to make the behavior opt-in and not break BC) that would make it more user-friendly.
Nothing says this has to be Slim-specific; my Slim example just happened to be the one I had most readily available to illustrate my points. There may be a PSR-7/15-compatible way to handle this.

A lot of reasonable behaviors could be implemented in a separate middleware in this library (to make the behavior opt-in and not break BC) that would make it more user-friendly.

There is no real difference between writing separate backage depending on this one and implementing as a part of this one for end-users. The real difference is for maintainers. Each middleware can represent it's own use-case. I.e we can have psr-7, slim, symfony middlewares. For each middleware we can be asked to do any kind of improvements like logging (with customizable levels). You can be asked to tune your response (i.e add custom message, or other content to the response body, headers, and so on). This potentinally will lead to the maintenance pit to make code suitable for each usecase not breaking others.
So I'm not saying that implementing any kind of logic around validator is bad, no. I'm just saying custom use-cases shouldn't be the part of this package.
For example @osteel implemented this package for testing purposes around validator and sf/http-foundation
https://github.com/osteel/openapi-httpfoundation-testing
Back to the story

For example, why are exceptions chained in these places, rather than just letting the originally thrown exceptions bubble up? What value does that add?

The same reasoning I've mentioned in first message. SchemaMismatch is thrown for validating custom data against object spec. You cannot use it in PSR-7 validation, since there you're validating psr-7 message against OAS3 Spec. Because you can get invalid message in any part of http message - body, headers (cookies as special case), path, query, etc.
There are also some unique cases, for example when your message matches multiple paths, but doesn't match any of them by schema. You cannot clearly say which schema validation failed.
Also there is cases when data from the message was not validated at all. It happens when your http message doesn't match any given route in spec.
So in general there is the following logic:
ValidationFailed is the top level exception for validating HTTP messages against OAS3 spec. In particulart there are following subcases for now:

Routing failed - which leads to NoPath (404), NoOperation (405) (if we speaking of request) and also NoResponseCode (for responses)
Ambiguous routingbut doesn't match any of matched operation schema - MultipleOperationsMismatchForRequest
Routing ok, matches single operation - AddressValidationFailed

AddressValidationFailed in turn has several cases depending on what part of schema failed: Headers, Path, Cookies, Body, etc. There is one outlier -  RequiredParameterMissing. This one should be rewritten to match one of the parts validator.
AddressValidationFailed in general has two use cases

when there is no required value
when value (data) doesn't match object spec

For second we come to the SchemaMismatch. There are a lot of subcases, but in general on this level you only need message and breadcrumb from the exception.
So in general with PSR-7 validator you always have chain of
ValidationFailed (i.e InvalidBody ) -> SchemaMismatch (if applicable, i.e KeywordMismatch)
Using PSR-15 middleware gives you two more cases - RequestValidationFailed and ResponseValidationFailed, since middleware process both of them at once, and you need to know which body - request or response - has failed. So for middleware you always get a chain of
ResponseValidationFailed|InvalidServerRequestMessage (to know which one failed) -> ValidationFailed (i.e InvalidBody ) -> SchemaMismatch (if applicable, i.e KeywordMismatch)
So If you not misusing use-cases you should always have the same parsing path for the exception.

A lot of reasonable behaviors could be implemented in a separate middleware in this library (to make the behavior opt-in and not break BC) that would make it more user-friendly.

There is no real difference between writing separate backage depending on this one and implementing as a part of this one for end-users. The real difference is for maintainers. Each middleware can represent it's own use-case. I.e we can have psr-7, slim, symfony middlewares. For each middleware we can be asked to do any kind of improvements like logging (with customizable levels). You can be asked to tune your response (i.e add custom message, or other content to the response body, headers, and so on). This potentinally will lead to the maintenance pit to make code suitable for each usecase not breaking others.
So I'm not saying that implementing any kind of logic around validator is bad, no. I'm just saying custom use-cases shouldn't be the part of this package.
For example @osteel implemented this package for testing purposes around validator and sf/http-foundation
https://github.com/osteel/openapi-httpfoundation-testing
Back to the story

For example, why are exceptions chained in these places, rather than just letting the originally thrown exceptions bubble up? What value does that add?

The same reasoning I've mentioned in first message. SchemaMismatch is thrown for validating custom data against object spec. You cannot use it in PSR-7 validation, since there you're validating psr-7 message against OAS3 Spec. Because you can get invalid message in any part of http message - body, headers (cookies as special case), path, query, etc.
There are also some unique cases, for example when your message matches multiple paths, but doesn't match any of them by schema. You cannot clearly say which schema validation failed.
Also there is cases when data from the message was not validated at all. It happens when your http message doesn't match any given route in spec.
So in general there is the following logic:
ValidationFailed is the top level exception for validating HTTP messages against OAS3 spec. In particulart there are following subcases for now:

Routing failed - which leads to NoPath (404), NoOperation (405) (if we speaking of request) and also NoResponseCode (for responses)
Ambiguous routingbut doesn't match any of matched operation schema - MultipleOperationsMismatchForRequest
Routing ok, matches single operation - AddressValidationFailed

AddressValidationFailed in turn has several cases depending on what part of schema failed: Headers, Path, Cookies, Body, etc. There is one outlier -  RequiredParameterMissing. This one should be rewritten to match one of the parts validator.
AddressValidationFailed in general has two use cases

when there is no required value
when value (data) doesn't match object spec

For second we come to the SchemaMismatch. There are a lot of subcases, but in general on this level you only need message and breadcrumb from the exception.
So in general with PSR-7 validator you always have chain of
ValidationFailed (i.e InvalidBody ) -> SchemaMismatch (if applicable, i.e KeywordMismatch)
Using PSR-15 middleware gives you two more cases - RequestValidationFailed and ResponseValidationFailed, since middleware process both of them at once, and you need to know which body - request or response - has failed. So for middleware you always get a chain of
ResponseValidationFailed|InvalidServerRequestMessage (to know which one failed) -> ValidationFailed (i.e InvalidBody ) -> SchemaMismatch (if applicable, i.e KeywordMismatch)
So If you not misusing use-cases you should always have the same parsing path for the exception.

I'm actually considering adding my own layer of exception instead of just passing on this package's like I do at the moment, because it's true that the root cause of the validation issues is quite buried at present

I'm actually considering adding my own layer of exception instead of just passing on this package's like I do at the moment, because it's true that the root cause of the validation issues is quite buried at present


Related project

Here is source  and a video demo for a related project - Flowify by Equinor.
Perhaps there are useful modules or concepts which could be shared / merged with this project.

I see. Perhaps there is not much potential for shared modules today. At least now you have a link to the Flowify project, so you can be aware of any future developments there.


Some websites fail to load - getColorScheme error

In android devices certain urls give
[INFO:CONSOLE(5)] "Uncaught TypeError: Cannot read properties of null (reading 'getItem')", source: https://medial.app/privacy-policy (5) [INFO:CONSOLE(1)] "TypeError: Cannot read properties of undefined (reading 'getColorScheme')", source: https://medial.app/_nuxt/entry.4795caa5.js (1) [INFO:CONSOLE(1)] "[nuxt] error caught during app initialization Error: Cannot read properties of undefined (reading 'getColorScheme')", source: https://medial.app/_nuxt/entry.4795caa5.js (1)
webview version: 1.8.8
tested urls: medial.app/privacy, medial.app/terms-of-service
Tested in web chrome browser and they work fine.
I have enabled dom storage, but still getting this error

@LaatonWalaBhoot Thanks for your feedback! I also reproduce the error. After conducting some tests, I have identified that the problem is caused by a delay in applying the web settings. We are currently using LaunchedEffect to initialize the web settings, but it seems to be delayed until the WebView is initialized. As a result, our custom settings are not being applied to the WebView at all.
To solve it, we can simply use DisposableEffect to init the WebSetting like that:
DisposableEffect(Unit) {
        state.webSettings.apply {
            androidWebSettings.domStorageEnabled = true
            desktopWebSettings.disablePopupWindows = true
            allowFileAccessFromFileURLs = true
            logSeverity = KLogSeverity.Debug
            customUserAgentString =
                "Mozilla/5.0 (Macintosh; Intel Mac OS X 11_1) AppleWebKit/625.20 (KHTML, like Gecko) Version/14.3.43 Safari/625.20"
        }

        onDispose {  }
    }

I apologize for any inconvenience caused by this issue. If you have any further questions, please let me know.

@KevinnZou Thanks. Now it makes sense. My loading bar is immediately displayed as well.
Maybe add this to the documentation?
You can go ahead and close this issue. It is resolved for me.
PS: No need to apologize. Kudos to you for this amazing library.

@LaatonWalaBhoot Sure, I will update the documentation and example later. Thanks for your feedback again!


Export as "Github Gist" instead of "Github"

Testing #167161
In the export flow, there's the step to "Export as...". The options are "File" and "Github"

I can make the educated guess that I'm not exporting the whole of Github from my VS Code window, but it'd be better to say "Github Gist" instead, or "Export to" instead of "Export as". I prefer the former since it's clear where it's going--otherwise people might assume it creates a repo on their account or something.

It was GitHub Gist before, after feedback I changed to GitHub. I will bring this to UX and see what other think too.

Good feedback. We will also do a user study next week so we will see how users react here.
Alternative rendering: "GitHub" and then "gist" as the description a bit to the right - QuickPick should support this.

There was some confusion in the user study when exporting as GitHub. They were expecting that it will be part of a repository.
So calling it GitHub Gist might be the way to go.

I think adding Gist as description would be good

Lovely, tried it out. Verified.


startup option for channel_reserve_satoshis (LND)

Background
Today,  when a channel is opened by A to B, A is requesting B to hold 1% of the capacity in reserve if and when B has a balance greater or equal of the reserve amount (channel_reserve_satoshis).  This can lead to strange situations like the case when A opens a 1,000,000 sat channel to B, A sends 10,000 sat to B but B can't do anything with these 10,000 sat as they should be kept as reserve.

Bolt 2: channel_reserve_satoshis:  1% of the channel total is suggested. Each side of a channel maintains this reserve so it always has something to lose if it were to try to broadcast an old, revoked commitment transaction.

Clearly, if B had in the past most of the capacity  (say, 50%-99%) and is now having only 1%, this 1% will not keep the "wrong" person from trying to broadcast an old, revoked commitment transaction.
I would like to suggest a new startup Option --chanreserve which is the percentage of the capacity to be  used as channel_reserve_satoshis. Default will be 1%.
Please let me know if you are happy with this and I will try to add support.
Thanks
Offer

This can lead to strange situations like the case when A opens a 1,000,000 sat channel to B, A sends 10,000 sat to B but B can't do anything with these 10,000 sat as they should be kept as reserve.

This will always be the case. B can't start to send outbound HTLC's until they're safely above the reserve.

this 1% will not keep the "wrong" person from trying to broadcast an old, revoked commitment transaction.

It's that that it will "keep" them from broadcasting. The goal is to avoid a costless attack to ensure that the risk of failure to attempt a contract is tangible at all times.

Thanks.
I get your view point. Still I would claim:

For an attacker, Risking 1% of the channel capacity of a possible win of 30%-99% provides a good risk reward. If she knows that her peer is offline or not monitoring the chain, she may take the risk. So 1% (or even 5%) does really provides protection. Clearly, 1% is better then 0%.

In situation where there is trust between the peers, setting the reserve to 0 may make sense. I have at least two usage scenarios  which are problematic to achieve without setting the fundee reserve to be zero.

Taking your comments into account I suggest to implement this feature on the "openchannel" function and not as a startup/config option. Are you happy with this?

Take a look at how the remote_csv_delay option is handled. It is both an option to openchannel and a default can be set at startup. A chan_reserve option can be added in a similar fashion.

This overlaps with #800 btw.

Sorry, I  have missed #800.
Thank you


Official Docker image and documentation

For #849 :)
Adds a basic Dockerfile that starts from python:3.6 and a includes custom startup script that allows you to start in any mode.
Includes a first attempt at documentation.
I added a Travis stage to build the image but for releasing I think the better approach is for someone to create a DockerHub account for locustio and set up an automated build for the repository. I believe they do security scanning on the resulting image as well.
Let me know if there's any docs changes, extra features, etc you'd like to see or if I can help with the DockerHub setup. I'm hanging around the Slack server as @JDiPierro as well :D

thanks for this!
I'd like a few maintainers to give this a review and try it out, so we are able to maintain it and answer questions that come up.
@aldenpeterson-wf @mbeacom can you guys give this a look and try it out when you have a chance?

Hey! I think having an official Docker image for Locust would be great.
Do we really need the docker_start.sh shell-script? It introduces a new API and interface that you have to "learn" to use Locust with Docker. Couldn't we just set the Docker entrypoint to the locust binary?

I have tried the docker image as is in this PR branch and it works great!  There was a Locust Helm chart in the stable repository that hasn't been updated in about a year.  I made all of the changes necessary to do the update to v0.9.0 and also added a bunch of other Kubernetes items in there to make it work a little nicer like nodeSelectors/Taints and an Ingress.
I have this PR for the Helm Chart update but I should probably wait for an official image instead of using the image in my Docker Hub repo.  Here is the updated Chart: https://github.com/helm/charts/compare/master...sekka1:with-v0.9.0?expand=1#diff-c9789cc33518b31f9230b6284b2b6116R4
The last response to the updates was about a month ago.  I'm happy to continue on if @JDiPierro is too busy now.
Thoughs @mbeacom

I'm happy to continue on if @JDiPierro is too busy now.

@sekka1 this PR was superseded by #882 , which has already been merged... so Docker stuff is in master branch now.   Could you give it a try?
I am going to close this one now.

@cgoldberg Is there any plans for an official Locust Docker hub image?

Is this docker image built and pushed to a public registry yet?

Is this docker image built and pushed to a public registry yet?

Nevermind. There is an ongoing discussion around this topic - https://github.com/locustio/locust/issues/849


Property hosts being required in PlayModel and import_playbook not allowed

I'm using the Ansible Language extension in vscode that apparently uses the ansible-playbook.json schema.
There are two rules in PlayModel which I think doesn't apply:

Property "hosts" being required (I believe it's in line 17026)
Import_playbook not allowed

As an example, I believe the below is a valid playbook:
---
- name: Import play A
  import_playbook: play-a.yml

- name: Import play B
  import_playbook: play-b.yml

Thank you to take the time to address this issue @ssbarnea.
Could you clarify to me if the property hosts is required as in line 17029 or is it not, like the example I gave earlier?

Hosts is not allowed at include_playbook level but is required for plays. That line is correct and if you also upgrade vscode-ansible to 0.4.4 you should see it reporting correctly. Do not confuse PlayModel with Playbook file. A playbook file contains a list of plays and import_tasks.


Load in-game textures

Some visual effects were introduced in Cataclysm. Such textures could be used directly from game data instead of being embedded in the addon folder.

Implemented in #245.


{Service Connector} disable too-many-locals for pylint

Description
Fix https://github.com/Azure/azure-cli/pull/21766#discussion_r836293266
#21766 together with #21763 causes CI failure:
https://dev.azure.com/azure-sdk/public/_build/results?buildId=1464664&view=logs&j=36dd4138-4d53-5e46-00d9-e5cd9744be05&t=1cf3879c-0a42-5ccd-7693-7a4781d739d8&l=94
src/azure-cli/azure/cli/command_modules/serviceconnector/custom.py:244:0: R0914: Too many local variables (26/25) (too-many-locals)

These PRs' CIs are run separately, but the final merged PR exceeds local variable limitation 25.

Service Connector


they changed manifest.json cdn domain format

1.8.0's mainfest.json
Global now is https://snowbreak-dl.amazingseasuncdn.com/118c343979b2407f4a6b3ad2b84d6d79/PC/updates/manifest.json
China Mainland is https://cbjq.xoyocdn.com/118c343979b2407f4a6b3ad2b84d6d79/PC/updates/manifest.json

looks like they use this now

They don't just change the URL.
The patching method is also changed. They use binary_diff tool to patch now.
Probably to save download size by downloading only changes and then patching existing files on local disk.
I will fix it when I somehow grasp their patching tool.

They don't just change the URL. The patching method is also changed. They use binary_diff tool to patch now.
Probably to save download size by downloading only changes and then patching existing files on local disk. I will fix it when I somehow grasp their patching tool. Or someone could help me with that.
Until then, the launcher is considered broken and can't be used.

In China Mainland, they start rolling out Seasun game launcher instead of old launcher for new players who download game from offical site. the old one still works.
And looks like Seasun game launcher use different download method?  and files download URL is changed from "cbjq.xoyocdn.com" to "cbjq-client.xoyocdn.com".
Seasun game launcher will auto detect where the game file is and break old launcher detection(it will say need update and you will need to download entire game if you open old launcher after used new launcher).
I am not sure will they release that new launcher for global.

I'm not sure, it's actually depending on the publisher that they have different distribution system.
That said, the gist of the global's patching system is:

There are 4 additional data fields in the manifest.json: base, diff, size diff and isPrimary. I kinda figured out the first 3, but still don't know primary of exactly what.....
If global see the file match the base's hash, it will download the diff and do binary-patch on the existing file. Otherwise, it will redownload the base file again to do the binary-patch.
I still need to know what does primary field means exactly before rolling out the design that match their system...

I was wrong.
The old patching system can still usable, but it's somehow break the game if trying to update the existing game, redownload full game may fix that.
The new patching works like:

If the file doesn't exist, pull latest file from server.
If the file exist but is multiple versions behind or is damaged/corrupted, also pull latest from server.
If the file exist and is one version behind, download binary-diff file and perform binary-patch on existing file.

primary was simply just a meant to determine whether the file must exact match the hash, or just the existence alone (hash mismatch allowed) is enough.

@LemonPAKA I've updated the launcher codebase. Please checkout the sample file of the new manifest. The new launcher will use the json data in the v2 directory.


Another crazy format

This surprising formats comes from Excel 💩

9/22/88 12:00
6/2/08 12:00

Thanks for the report!   Always blows my mind how many  keep popping up.


Bunch of updates and fixes.

Thanks :+1: , maybe some day @tel will merge some of these back upstream.

Oh! I wasn't even aware this was existed. I'd be more than happy to accept a PR from someone who's actively using the project. I'd also be more than happy to grant commit bits.

Heh well I think there have been a handful of pull requests pending for a while. It would be great if this could get released on Hackage too. I've gotten a bunch of emails from non-Haskellers that want to use the Ceph/Riemann tool I hacked on a while back and it would make installation much easier if cabal install worked :-).

Huh. I think I somehow unsubscribed myself from it. Sorry for the bad maintainership!
Let me examine these PRs and try to get some motion going.

I merged the protobuf availability PRs in just now. My GHC is in a bad state while I fix some Nix stuff, but once I get over those—or once I can get some saner version dependencies and the like—I'd be more than happy to release a 0.x.y to Hackage.

At the very least.

And given there's some Nix expertise on this thread, I think I'm locked up with this issue atm: https://github.com/NixOS/nixpkgs/issues/6148

I can't really say anything helpful about NixOS/nixpkgs#6148 ... I'm running on a Mac, but I found nix on OS X to be pretty painful and eventually moved to running NixOS in a VM ... and haven't looked back.

I'm in about half a mind to do that as well...

It actually looks like there's a whole bunch of nix-on-osx work going on right now ("Please test NixOS/nixpkgs.git on darwin" thread on nix-dev mailing list) also maybe this is of use: https://nixos.org/wiki/Nix_on_OS_X ?    ... so maybe the situation is about to get significantly better.

Yeah, that's why my GHC is in a funk today—I nuked my /nix/store to get back on the mainline developing happing there.

It's clearly vastly improving... but it's also still a giant pain.


Upload a github action artifact to LATEST release

I want to upload a artifact to latest release... without creating a new release.
This action rquires a upload_url, which is generally taken from creating a release in previous step with create-release action.
I tried to print output url from create_release-
https://uploads.github.com/repos/atiqg/test/releases/28579698/assets{?name,label}
Then I changed it to direct to latest release-
https://uploads.github.com/repos/atiqg/test/releases/latest/assests
Which oblivously did not work out and thrown this error-
##[error]Multipart form data required
Is there any way I can do this? I don't want to create a new release from actions.
I want to create release normally then action should upload artifact to latest release...

You can do it like this:
    - name: Get upload URL
      id: geturl
      run:   |
         upload_url=$(curl -sL https://api.github.com/repos/:owner/:repo/releases/latest?access_token=${{ secrets.GITHUB_TOKEN }} | jq -r '.upload_url')
         echo ::set-output name=upload_url::$upload_url
    - name: Upload Release Asset
      id: upload-release-asset
      uses: actions/upload-release-asset@v1
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      with:
        upload_url: ${{ steps.geturl.outputs.upload_url }}
        asset_path: ./my-artifact.zip
        asset_name: my-artifact.zip
        asset_content_type: application/zip

Look here: StackOverflow

Yes that's me on stackoverflow...
But it's not the perfect answer as it will fail/throw error when there is asset with identical name. There is no option for overwrite...

You could theoretically run a job to delete the asset before the upload. I found an action for this.
You can do it like this:
    - name: Get Release ID
      id: getid
      run:   |
         rel_id=$(curl -sL https://api.github.com/repos/${{github.repository}}/releases/latest?access_token=${{ secrets.GITHUB_TOKEN }} | jq -r '.id')
         echo ::set-output name=rel_id::$rel_id
    - name: Remove Release Asset
      uses<EMAIL_ADDRESS>      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      with:
        release_id: ${{ steps.getid.outputs.rel_id }}
        asset_name: 'asset_name.zip'

Hmm this will work I think So, But It would much cleaner if this actions does it with an optional argument

@AtiqGauri, see #22.


Useability and toggling for IE contact/grasping and dynamically spawn…

…ing InteractionBehaviours.
Toggling: Just added appropriate setters for ContactEnabled and GraspingEnabled.
Spawning InteractionBehaviours: Spawning InteractionBehaviours onto objects from a script is not fun because you get lots of errors about missing an attached InteractionManager and InteractionMaterial even if you set them right after creating the behaviour. This solves that problem by adding a default InteractionMaterial setting to the InteractionManager and also has InteractionBehaviours try to find a manager on-load if it doesn't have one.

alright everything looks great!


Change override entrypoint to /usr/bin/env

@grahamdumpleton ptal.
fixes https://github.com/openshift/source-to-image/issues/559
[test]

@php-coder ptal.

LGTM.
BTW, maybe it would be good to mention this in the docs (for example, docs/debugging-s2i.md). As far I understand this case is rare, but as a user that will find it, I would like to have a bit info about this. For example, we could mention that S2I prefers images with /usr/bin/env and if user doesn't have such image, he could create a symlink.

[merge]

@php-coder it remains to be seen if this is the long term solution, there's still some discussion going on, so i'm not going to bother doc'ing it just yet.

@php-coder (discussion is here: https://github.com/openshift/source-to-image/issues/475#issuecomment-238088669)


Dask groupby count & standard deviation gives incorrect result after datetime to date conversion

I've found a bug that shows up if you convert a timestamp column to a date, then groupby the date, and then calculate standard deviation and count aggregations over a column.  The dask count column is twice what the count should be.
Minimal Complete Verifiable Example
import pandas as pd
import dask.dataframe as dd
from io import StringIO

# Create df
raw = '''groupby_col1,agg_col,utc_timestamp,mydate\n'''\
      '''a,3.0,1578104105000,2020-01-04\n'''\

pdf = pd.read_csv(StringIO(raw))
df = dd.from_pandas(pdf, npartitions=1)

# comment out these next 2 lines and bug is no longer apparent
pdf['mydate'] = pd.to_datetime(pdf.utc_timestamp, unit='ms').dt.date
df['mydate'] = dd.to_datetime(df.utc_timestamp, unit='ms').dt.date

pgrp = pdf.groupby(['groupby_col1', 'mydate'])
grp = df.groupby(['groupby_col1', 'mydate'])

# if you don't include std in the aggregation, the bug is no longer apparent
presult = pgrp.agg({'agg_col': ['count', 'std']})
dresult = grp.agg({'agg_col': ['count', 'std']}).compute()

assert dresult.equals(presult)

The dresult count column is 2, when it should be 1.

Thanks for the issue report, I'm able to reproduce. Here's a simplified version:
import pandas as pd
import dask.dataframe as dd

# Create df
df = pd.DataFrame(
    {
        "groupby_col1": ["a", "b"],
        "agg_col": [3.0, 4.0],
        "utc_timestamp": [1578104105000,<PHONE_NUMBER>000],
        "mydate": ["2020-01-04", "2020-01-04"],
    }
)

# Comment out and the test passes
# After this, `df.mydate.dtype` is still `object`, but the contents are `datetime` objects instead of strings
df['mydate'] = pd.to_datetime(df.utc_timestamp, unit='ms').dt.date

ddf = dd.from_pandas(df, npartitions=1)

sol = df.groupby(['groupby_col1', 'mydate']).agg({'agg_col': ['count', 'std']})
res = ddf.groupby(['groupby_col1', 'mydate']).agg({'agg_col': ['count', 'std']})

assert sol.equals(res.compute())

Playing around a bit more, it looks like this only fails if you're grouping on multiple columns, and one of them happens to be an object dtype where the elements are datetime objects. It also only fails if you use .agg(...) - if you call .agg_col.count() instead after grouping you get the correct answer. Because of this, I suspect the bug is in the agg implementation itself (located in dask/dataframe/groupby.py (note that the implementation is admittedly a bit complicated). Would you be interested in taking a look to see if you can figure out what the issue is?

cc also @TomAugspurger, who I think was looking at another groupby issue this week.

Thanks for taking a look @jcrist.  I tried tracking this down for a few hours before posting, but unfortunately I think this is beyond my ability right now.

Maybe this is coming down to MultiIndex handling inside agg?  There is an odd doubling of data during the _apply_func_to_column call when data is a timestamp:
ipdb> df_like.get_group(('a', Timestamp('2020-01-04 00:00:00')))
                         count-agg_col-481c9d6b32d1be653208c632fb461c4c  sum-agg_col-48b9d136fe0670cb4c57a004c7a62612  sum2-agg_col-b052421682445bba4117dd200015910a
groupby_col1 mydate
a            2020-01-04                                               1                                           3.0                                            9.0
             2020-01-04                                               1                                           3.0                                            9.0

When The data is an object we no longer see this doubled data within the group:
ipdb> df_like.get_group(('a', '2020-01-04'))
                         count-agg_col-481c9d6b32d1be653208c632fb461c4c  sum-agg_col-48b9d136fe0670cb4c57a004c7a62612  sum2-agg_col-b052421682445bba4117dd200015910a
groupby_col1 mydate
a            2020-01-04                                               1                                           3.0                                            9.0

I think the issue is coming down to how count/sum are operating on the index.  For example below we see how an Timestamp index is converted to a datetime.date.  Then when dask combines count/sum/sum2 Pandas struggles.
ipdb> df_like.groups
{('a', Timestamp('2020-01-04 00:00:00')): Int64Index([0], dtype='int64'), ('b', Timestamp('2020-01-04 00:00:00')): Int64Index([1], dtype='int64')}
ipdb> func(df_like).index.values
array([('a', datetime.date(2020, 1, 4)), ('b', datetime.date(2020, 1, 4))],
      dtype=object)

Alternatively, the issue could be that _compute_sum_of_squares leaves the Timestamp dtype inplace.  @TomAugspurger do you have advice here ?

What output are people seeing? Seems like it may be fixed on master, perhaps by #6186?

Unfortunately, it is note resolved by #6186. The output is NaNs vs 0.0:
In [5]: sol
Out[5]:
                        agg_col
                          count std
groupby_col1 mydate
a            2020-01-04       1 NaN
b            2020-01-04       1 NaN

In [6]: res.compute()
Out[6]:
                        agg_col
                          count  std
groupby_col1 mydate
a            2020-01-04       2  0.0
b            2020-01-04       2  0.0

I think there is some strangeness around  mixing Timestamp and datetime.date

Huh, locally I see
In [3]: res.compute()
Out[3]:
                        agg_col
                          count std
groupby_col1 mydate
a            2020-01-04       1 NaN
b            2020-01-04       1 NaN

In [4]: sol
Out[4]:
                        agg_col
                          count std
groupby_col1 mydate
a            2020-01-04       1 NaN
b            2020-01-04       1 NaN

Ah, you're right @TomAugspurger .  I hadn't fully merged in master into my local clone.  @balast can you confirm latest master resolves the issue ?

Yes, after merging with master, the issue described here was resolved. :)

Thanks for reporting @balast!


How to characterize crosstalk errors with PyGsti?

In the paper "Experimental Characterization of Crosstalk Errors with Simultaneous Gate Set Tomography", the authors used PyGsti to fit the crosstalk model to the experimental data. Is this operation built into PyGsti? Is it possible to accomplish this with a few lines of code?

Closing this as part of issue cleanup for the next minor release as we answered the question. Please reach out again if this example was not a sufficient starting point!


Swift 3

Will this be supported to Swift 3?

Can't until SwiftMoment does an official Swift 3 release. https://github.com/akosma/SwiftMoment/issues/72


fix: Resolve main.tsx URL on nested route refresh

How to reproduce

run pnpm examples:dev:07_router
navigate to http://localhost:3000/nested/baz
refresh the page
open the network tab in the browser's dev tools
observe that the request URL for main.tsx is displayed as http://localhost:3000/nested/main.tsx

Ohhh, I forgot about it. That was the reason we had /. Can you fix all examples + website for consistency?

Ohhh, I forgot about it. That was the reason we had /. Can you fix all examples + website for consistency?↳
It was my mistake in #92.↳

Done


Support worker nunmber for reconcile runtime controller

Ⅰ. Describe what this PR does
Ⅱ. Does this pull request fix one issue?

Ⅲ. List the added test cases (unit test/integration test) if any, please explain if no tests are needed.
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

Codecov Report

Merging #789 (bd8a9f9) into master (b202db3) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #789   +/-   ##
=======================================
  Coverage   12.05%   12.05%
=======================================
  Files         106      106
  Lines        4745     4745
=======================================
  Hits          572      572
  Misses       4113     4113
  Partials       60       60

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b202db3...bd8a9f9. Read the comment docs.


Feature request: [expose initial_guess for nonconvex_objective]

please expose initial_guess for nonconvex_objective
different initial weights will get different result

@wukan1986
Thanks for the suggestion – I'll include it in the next update, which will be pushed by the end of the week (hopefully)


Add missing parameters to webhook and set nullable fields in contact object

Hey @winkelco
Thanks so much for your improvements.
Can you modify the PR title to be more specific?
Finally, can you add some unit and (if possible) integration tests?
Thanks!

Done

Hi @aalbarca can u check the tests please


Add support for webtasks and extensions

So, for example, the clear recipe can delete all extensions.

Resolved by 3626cb13491ccbae373cb34aa7f2b68eced53c91


SpaceX is not using casbin

Hi! Sorry for being a party pooper, but SpaceX Inc is not really using casbin. The commit linked on your site points to github.com/r-spacex, which is described as:

A central location for information and collaboration on all of the awesome projects coming out of r/SpaceX

which is a subreddit. Furthermore upon reading their repo it states that they're not affiliated with SpaceX.

Thereby I think your claim of SpaceX using Casbin is incorrect.

Hi, it has been fixed in: https://github.com/casbin/casbin-website/commit/7ac05647b1047785e66c2e8630d8d0c174a84e88 . Thanks for the info!


Refactor Clique Consensus to handle initialization internally

What was wrong?
The current clique implementation requires attaching a DB to the chain class (not chain instance). This is problematic (see discussion in Trinity)
It is also flawed as it is leaking a stateful consensus engine instance on different chain instances that should be independent.
For instance:

chain = GoerliChain()
second_chain = GoerliChain()

# this may add a signer which is now leaking to the second_chain instance
chain.validate_seal(...)

How was it fixed?

This introduces a VirtualMachineModifierAPI interface defined as

class VirtualMachineModifierAPI(ABC):

    @abstractmethod
    def __init__(self, base_db: AtomicDatabaseAPI) -> None:
        ...

    @classmethod
    @abstractmethod
    def amend_vm_configuration_for_chain_class(
            cls,
            config: VMConfiguration) -> Tuple[Tuple[BlockNumber, Type[VirtualMachineAPI]], ...]:
        """
        Make amendments to the ``VMConfiguration`` such as redefining its methods for different
        consensus schemes.
        """
        ...

    @abstractmethod
    def amend_vm_class_for_chain_instance(
            self,
            vm_class: Type[VirtualMachineAPI]) -> Type[VirtualMachineAPI]:
        """
        Make amendments to a VM class that are not supposed to be shared across different chain
        instances.
        """
        ...

The amend_vm_configuration_for_chain_class defines a classmethod which can perform all kind of class relevant actions on the VMConfiguration. This includes things like setting extra_data_max_bytes to 65535 which is safe to share across all GoerliChain instances.
But it also defines an instance method amend_vm_class_for_chain_instance which performs modifications that are not safe to share across different chain instances. Such as replacing validate_seal with an implementation that is stateful.

It also means that validate_seal on the VM had to become an instance method which also means that  validate_chain on the VM and  validate_chain on the Chain had to become instance methods. A fact that @carver had also pointed out in the linked Trintiy thread.

This also adds  a PowConsensus and a NoProofConsensus engine and refactors some test helpers.

There's a test that proves that stateful consensus engines do in fact not leak across different chain instances.

To-Do

[x] Add entry to the release notes

Cute Animal Picture

Direction looks solid!

@carver Alright, this has taken more time than I anticipated. I think this is ready for review. There's also a draft PR of the Görli integration that uses this branch: https://github.com/ethereum/trinity/pull/1196

Replaced by #1899


Stop sequence for Claude model

I had the Claude model give out Human: in the end of it as response. This was due to no stop sequence configured. I think it might be a good practice to have it built-in within the adaptor for Claude

Do you have the same issue with v3?


Upgrade PDO

This PR upgrades PDO, whose last version contains a fix for the attestation verification procedure.
This PR was tested in SGX HW-mode.
Signed-off-by: Bruno Vavala<EMAIL_ADDRESS>

@bvavala can you please rebase? Update branch does not work, complaining about DCO check.


Fabric wide dash

This PR is my attempt to rebase all of my previous fabric-wide-dash work with my more recent work bringing Go dashboard support up to parity with the JVM.
A significant achievement is that the dashboard can run either with or without a Fabric Server.
When index.html looks like this, the dashboard works as before
<meta property="fabricServer" content="__FABRIC_SERVER__">

However, when the endpoint for a Fabric Server is provided, all other index.html meta tags are ignored and the dashboard pulls services and metrics information from the Fabric Server
<meta property="fabricServer" content="http://localhost:9990">

This should allow the dashboard to be deployed per-instance as before and also in the new manner where a single dashboard monitors an entire fabric.
To test running against a Fabric server:
npm run mock-sds and then configure the fabric server meta tag
To test running directly against a Go microservice without a Fabric server:
docker pull drfogout/metricssimple
docker run -it -p 1337:10001 -p 20001:20001 -p 30001:30001 drfogout/metricssimple

and then configure the runtime meta tag in index.html. Make sure the fabric server meta tag is not set.
Known Issues:

Fabric Server mode does not yet load threads metrics from the Fabric Server when the threads component mounts
The mock-sds mock service does not yet mock go metrics
Groups and other aspects of the Fabric Server have not been implemented
Per agreement with the backend engineers, instances is not included in the response to the GET /services route. The mock-sds endpoint will have to be modified in anticipation of these changes.

I now see what you mean about modularizing the service instance components.  We certainly have an opportunity to reduce some of the redundant code in instances/golang and instances/jvm.  In RoutesGrid components, the only difference is the import of getRoutesTable.  An HOC that conditionally requires the module could be a quick solution.  SummaryGrid components are quite different from each other but still have some similarities that can factored out.

I quickly tested the dashboard this morning using the mock JVM service and all seems to be working.  Yesterday I tested against the mock SDS and Doug's Go microservice.  You did a great job of maintaining stability despite the massive size of this PR.
Excellent work!  LGTM 👍


Utilize user-supplied object data to populate request/assertion values

So we are already doing this for security properties...should we consider adding this for path parameters? This might be a good next step for the issue, as they are simple data types, no potential for complex objects (i.e. in the body or formData)

What if path parameter is generated dynamically?

POST /pet to make a pet, grab petId
GET /pet/{petId} to make sure it's created
DELETE /pet/{petId} to remove it

Are you saying it's a reason not to implement loading from an environment variable or just saying we should consider it as a possible implementation? That sounds like it would be part of the "request series" feature we were considering. I'm thinking about all the individual tests for api calls that use the same path parameter...

I'm not saying it's a bad idea. It's actually very useful but it's only one case of many different things that people want to do.
What we're discussing here is a higher level of abstraction on top of what this module does. You can take output of this module and fiddle with the generated code (changing order of operations and filling in placeholders) to generate tests that need much less work to get it running.
We can put that higher level of abstraction in a module that uses this module or provide a new more verbose API that does it in this module.

Right, I agree, path parameters is just a drop in the bucket of data injection, but it might be the easiest place to start?
If we built another module for test data injection, we would need to standardize the data placeholders so they are easy to manipulate post-generation. Also, the attribute that the placeholder is standing in for has to be identifiable so that it's easy to inject the data corresponding to that attribute.
This makes me wish we could generate the placeholders as Handlebars variables, i.e. {{userId}}, so that it could be run through another module with an API to handle the desired data. However, such a placeholder wouldn't be valid JavaScript unless wrapped in quotes or commented out.

Could we leverage Mocha's built in dynamic test generation to our advantage here? This feature is pretty cool, but might be more useful for edge-case test generation of a single path and/or redoing how we generate tests for an operation's different response codes (its the same API call, just a different input per expected response)

If I recall correctly, originally we wanted to have no test generation and do everything programmatically. With that all you needed to do was giving the tester module your swagger and some meta-data to test your API. The reason we went to code generation path was we didn't want to do stuff under the hood and only think about happy path.
Now with test generation we can still use some of that idea, putting all request payload and other metadata like that into a config file is a good idea. But putting it calls in a loop puts stuff somehow under  the hood. We want users generate code and then modify it based on their specific requirements.
But still, putting things in a config file can get hairy as I commented above about path parameters.
To be honest, I'm not sure if we should do this at all.

OK i'm glad we talked this through in some more detail, I'm convinced that this feature is outside the desired scope of this module. I am going to close the issue. Thanks @mohsen1!


How to return  clustered data matrix after using ClusterMapPlotter function

cm =ClusterMapPlotter(data=df_heatmap,  z_score=0)
cm.data2d will not return clustered matirx but return a matrix with original row/colunm order. So how to get a clustered matirx like using sns.clustermap.
Thanks.

Please refer to issue 2.
Briefly, you can use cm.row_order and cm.col_order to get the clustered rows and columns.
I hope my answer is helpful to you.
Thanks for your feedback.


Defualt values of component inputs are overridden with MockRender

https://stackblitz.com/edit/ng-mocks-examples-mkp6fd?file=app/app.component.spec.ts
In the provided stackblitz example, I expected the testcase to pass.
When we use MockRender(AnyComponent), the template is generated as
<app-any [input1]=input1 [input2]=input2></app-any>
This overrides the default values of input1 and input2 defined in the component class.
This problem can be solved by using MockRender('<app-any></app-any>').
I feel it will be better if there is a configurable option for controlling template generation.

when the second argument to MockRender() is null/undefined the input/outputs can be ignored in the template generation.
eg: MockRender(AnyComponent) should generate the template as <app-any></app-any>

consider only the params provided in second argument for the template generation.
eg: MockRender(AnyComponent, { input1: 'abc'}) should generate as <app-any [input1]="input1"></app-any>

Hi @vinayk406,
thanks for the report, I'll take a look deeper during next days.
As far as I remember, the problem is that default values aren't always the case if optional chaining is used.
<app-any [input1]="my?.data?.obj?.input1"></app-any>

Although input1 can be typed as input1: string = 'defaultValue', it will receive null if my is empty. The right declaration is this case is input1: string | null = 'defaultValue'.
In order to show this behavior <app-any [input1]=input1 [input2]=input2></app-any> is generated.
Anyway, I'll take a look for a possible solution besides MockRender('<app-any></app-any>').

Hi @vinayk406,
could you verify that the new version works as you expect?
ng-mocks.zip
Thanks in advance!

Thanks @satanTime
LGTM.
Is there any way to memoize the response of Mockrender between tests to improve performance?

Great, thanks for checking.
Could you create a separate issue about memoization?
It should be possible with ngMocks.faster and MockRender in beforeAll instead of beforeEach. At least it was the intention. But this feature hasn't been much tested and may not work as expected.

There are some ongoing checks of other changes. A release with the fix will be on the next weekend.

Can we eliminate the creation of <mock-render> intermediary component in the below case?
MockRender(XComponent, {})
In this case, can we bypass the creation of intermediary component and call TestBed.createComponent(XComponent) directly to speed up the processing?

Hi @vinayk406,
unfortunately, it is not only about passing / not passing params.
If XComponent is an onPush component, then, without <mock-render>, it cannot be tested properly. Because nothing is rerendered in default TestBed.createComponent despite input / output changes.
MockRender doesn't forbid usage of TestBed.createComponent(XComponent), it is a function which creates proper environment like a real angular app. If you don't need the environment, it is fine to use TestBed.createComponent(XComponent) instead.

You can try to use TestBed.createComponent here to reproduce the issue: https://github.com/ike18t/ng-mocks/blob/6b07067d751f29b4eba6d5db2773d605a35b3add/tests/issue-537/test.spec.ts

Sounds good. Thanks for the

Hi @vinayk406,
unfortunately, it is not only about passing / not passing params.
If XComponent is an onPush component, then, without <mock-render>, it cannot be tested properly. Because nothing is rerendered in default TestBed.createComponent despite input / output changes.
MockRender doesn't forbid usage of TestBed.createComponent(XComponent), it is a function which creates proper environment like a real angular app. If you don't need the environment, it is fine to use TestBed.createComponent(XComponent) instead.

Sounds good. Thanks for the explanation.