Improve EIP/VIP failover when the node running CPEM fails

Right now in a cluster with an HA control plane using CPEM for EIP management you can have the following situation.
Situation
3-controlplane-node Kubernetes cluster, created with CAPP. Has CPEM, with the control plane EIP pointing at one of the nodes (not kube-vip or metallb, but rather CPEM’s direct control), CPEM running on the same node that EIP is pointing to.
CP nodes
A    <--- EIP ; CPEM here
B
C (edited)
Node A goes away.
CPEM should move the EIP to node B or C. But CPEM is gone too, so it cannot do so until k8s controllers move it to node B.
Problem: It never happens. Controller and scheduler never seem to learn that it has something to do, CPEM stays dead, EIP never moves, we are stuck.
What does work: Each CP node’s controller-manager and scheduler do point at their local IP kube-apiserver, which in turn points at the local etcd.
###Root cause
kubelet. /etc/kubernetes/kubelet.conf points to the EIP:
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data:  // etc. etc.
    server: https://<IP_ADDRESS>:6443/      // <===== this is the EIP, not the local
  name: default-cluster

So all of the components are working, but the kubelets cannot connect to the apiserver, so controllers do not actually know what the state of anything is.

I did investigation into what options cluster-api lets you pass down to kubeadm configuration for changing what IPs the kubelet is configured to use. Not only is there nothing that lets you override the address kubelet talks to with cluster-api OR kubeadm, I eventually found this:
Note: kubeadm applies the same KubeletConfiguration to all nodes in the cluster. To apply node specific settings you can use kubelet flags as overrides by passing them in the nodeRegistration.kubeletExtraArgs (i’ve checked, there are no flags that help us) field supported by both InitConfiguration and JoinConfiguration. Some kubelet flags are deprecated, so check their status in the kubelet reference documentation before using them.
So, the long and short is that my research is indicating that overriding the IP that kubelet is set to talk to just for the controplanes isn’t really a normal pattern (at least not for kubeadm).
I’m concerned we need to look more at changing how CPEM runs or just go with a hack that plumbs the main EIP onto each control planes localhost interface.

I took a look at a bunch of the k8s vendors to see what they all did for their cloud providers.
Daemonset
AWS
OpenStack
Linode
Azure (has a node manager daemonset as well)
Vultr
OpenStack
vSphere
alibaba
Oracle
Deployment
Equinix Metal
DigitalOcean
Azure (deployment with multiple replicas or static pods on each CP)
IBM (3 replicas)

@deitch has noted:
We once looked at having it in 2 modes - DaemonSet and Deployment - simultaneously; Deployment for typical CCM functions, DaemonSet for IP control. But we really didn’t want to get into the maintenance and support headaches.
So going the daemonset or static pod route could be biting off more than we can chew.

Alternately, it could improve end user reliability to move people away from CPEM managed EIPs and to use kube-vip instead.

Some extensive updates. @cprivitere rightly pointed out that many other CCMs run either as a DaemonSet, or as a Deployment/StatefulSet with replicas > 1. We always were hesitant about doing this due to worries about race conditions.
It turns out the k8s.io/cloud-provider already provides built-in leader election, for precisely this reason. This means it should be safe to run multiple copies of CPEM; e.g. as a DaemonSet restricted to control plane nodes, as @cprivitere suggested. This is subject to testing of course. I will tackle it once we get further along with the design.
In order for this to work, there are several parts and considerations. I am putting them all together in a single list.

For core services (the ones supported by k8s.io/cloud-provider), nothing more needs to be done, other than to test it.
For additional services, the ones for which we run informers - notably our EIP management - we would need to get the informers from the right configuration passed to the initialization callback from k8s.io/cloud-provider. I do not believe we do this now, we would need to change it. We should first change it "as is" and make sure it works, and only then move to testing in parallel.
This all depends on leader-election, which in turn relies on access to the kubernetes service. Given that the entire purpose of this exercise is to handle failure of the control plane (kube-apiserver) node to which the EIP points, this may very well fail. On the other hand, it may very well work, since access to the service/kubernetes is via iptables rules. So it may be ok.

All of this leaves several layers of changes and testing:

Change our informers-driven "additional" components to use the informers from the provided configuration; test
Run multiple CPEM, test the core components for race conditions and logs to show proper behaviour
Run multiple CPEM, test the additional components for race conditions and logs to show proper behaviour
Create cluster with multiple control plane nodes and EIP pointing at one node; stop CPEM; bring down the node; test if able to access service/kubernetes from other nodes
Create cluster with multiple control plane nodes and EIP pointing at one node with CPEM on it; bring down the node; test if CPEM restarts on other nodes

I suspect that even with service/kubernetes working, this will fail, as it depends on kubelet connections to kube-apiserver, but let's see.

Run multiple CPEM, test the core components for race conditions and logs to show proper behaviour
Run multiple CPEM, test the additional components for race conditions and logs to show proper behaviour

OK, confirmed that this works. It works even without changing our informers, but we should do so anyways, as without it, it might fail when leader election switches things around.

Create cluster with multiple control plane nodes and EIP pointing at one node; stop CPEM; bring down the node; test if able to access service/kubernetes from other nodes

Confirmed this does not work. It tries to access the service represented by  the Service default/kubernetes in the cluster, but it fails, as that routes to the primary node. It works fine if the CPEM is on a node other than the node that the EIP points at (which we knew already), but if on the same node, CPEM will not bring the EIP around, even if multiple running, because it does not know that the node is gone.
It does not even work correctly - unless you catch the timing just right - if CPEM is on a different node, as it depends on the node event to determine that it is time to move the EIP over.
Basically, CCM is architected around an assumption that the default/kubernetes service always will be available via routing somehow, at a level beyond CPEM.
We can fix this partially by:

Having multiple CPEM run at once (run it as a DaemonSet, as @cprivitere suggested)
Adding a separate control loop, completely independent of the usual loops or informer events, that checks the EIP status.

We had something like that a while ago, and replaced it with this generally more robust solution, but now have a case where it doesn't completely work.
This won't completely work, though. Because the entire startup depends on leader election, if the node with both CPEM and the EIP fails, then we end up with leader election stuck, and the separate loop will not run. The above partial solution resolves the CPEM-on-different-node-than-EIP issue, not CPEM-on-same-node-as-EIP issue.
Maybe it is time to say goodbye to CPEM-managed EIP? Or to have it as a separate DaemonSet container that specializes?

we should do so anyways, as without it, it might fail when leader election switches things around

Deeper investigation confirms that we actually are doing this correctly. Our pattern is a little different than the one used by Azure and AWS, but from the same basic clientbuilder. So we are fine there.
The cases to be addressed (each more complex than the previous):

Node dies, contains neither CPEM nor primary EIP: no problem at all.
Node dies, contains CPEM but not primary EIP: with apiserver still working, pod controller should start a new CPEM on a different node. However, it will take time, as without CPEM, k8s doesn't know if it is a temporary or permanent loss, and waits. This can be addressed by switching to a DaemonSet that runs on all control plane nodes. This has 2 benefits:

leader election will cause a different CPEM instance to take over immediately
the new instance will immediately see the node failure is permanent and inform k8s

Node dies, contains CPEM and primary EIP: nothing we have done will help. Leader election will fail (no k8s API available), so no new CPEM, so no switching EIP, so no access to apiserver for moving pods over.

I will open a PR for DaemonSet to help with issue 2; for issue 3, we need some better ideas.

Let's open a new issue for 2 and close this issue if the solution for case 3 is working (it seems this was the initial issue ask).


Service Discovery for K8s

As per the requirement Runtime API should be capable of listing services available in the k8s cluster that the Runtime API is running and make them available for the Runtime API via exposing an endpoint.
As per the OAS defined in https://github.com/wso2/apk/issues/42 it has following endpoints for service discovery related operations.

/services - List/search services
/services/{serviceId} - Get details of a particular service
/services/{serviceId}/usage - Get usage details for a particular service

We did the initial implementation for above using the client-go library by directly calling k8s API server and accessing services across namespaces.

Closing this issue as this is tracked in https://github.com/wso2/apk/issues/78


chore: 🤖 clean up build scripts to run in parallel

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

I am unsure why random integ tests are failing (pass on local)


100% CPU usage on Google Chrome

Every site that has holder.js loaded uses 100% CPU for me as long as I have it open. The Chrome Development Tools profiler says that (program) is the culprit, so there's not much help from that.
I also tried resetting Google Chrome to see if it was a problem with some extension, but it didn't help.
System info:
Browser: Google Chrome Version 43.0.2357.132 m
OS: Windows 10 Pro Insider Preview

This is sufficient to replicate the bug:

Hello

What OS? Have you looked at the Activity Monitor in OS X or Task Manager in Windows or Ubuntu? CPU usage goes to 100% while profiling - are you sure it's Holder-related and that it can't be replicated with any other library?

A git bisect using the html-file above and running gulp between each iteration tells me that the issue was introduced in commit 41c9258.

That seems unlikely. Can you replicate the issue in a different browser and answer my previous questions?

I said Windows 10 in the bottom of my post.
No, I haven't found any other library that causes the issue. I discovered it on the getbootstrap.com website and pinpointed it to holder by removing js code until the issue disappeared. And since just including the library on a webpage is sufficient I concluded that this was causing it.

No I can't replicate it in another browser.

I agree that it seems unlikely that the commit I mentioned is the cause, but it became visible after that commit.
I guess it's some kind of bug in Google Chrome, but it is being triggered by your library.

(And no, I only started the profiling after I discovered that the webpage was using 100% cpu, so that is not the cause)

@imsky More specifically, I believe that the line: global.requestAnimationFrame(visibilityCheck); is the culprit. It seems likely since the 100% cpu usage is only an issue when the page is visible. If it is in a background tab, it is using normal amounts of cpu.

http://stackoverflow.com/questions/28390358/high-cpu-usage-with-canvas-and-requestanimationframe

OK can you replicate it with just requestAnimationFrame? If it's a problem with rAF it's a Chrome bug and should be filed there.

Yes, it seems so. The example (http://www.duckware.com/test/chrome/requestanimationframe.html) on https://code.google.com/p/chromium/issues/detail?id=321632 gives the same result. Thanks for the help.

Although I'm not entirely sure that the way you are using the function is correct. You are essentially polling for visibility on every frame instead of using some callback.
What you are using the function for isn't actually animation as far as I can tell, which the function is meant for.

More details:
This code determines which framerate rAF uses:
(function(){var a = 0;var af = function(){window.requestAnimationFrame(af);a +=1;};af();window.setTimeout(function(){af = function(){console.log(a)};},1000)})()

On Firefox it reports 60FPS as expected. But on Google Chrome on Win10 it reports 2000FPS, so it doesn't seem to limit the framerate at all.

Good catch - I'll look into using that snippet and throttling rAF calls if necessary.

This should be addressed on the latest build, let me know if you still see 100% CPU usage.

Yes, it seems to be fixed. Thanks!

Great to hear, thank you for reporting this.

For me it still drains battery too fast. I have two tabs opened with holder.js and they consume about 10% of CPU. Notebook is heating. Chromium profiler shows most times goes to setTimeout and requestAnimationFrame.
The point I can't understand where animation is done? Placeholders are static.

if there are any fluid or auto-sized placeholders, then there will be a loop running for the whole page.
do you have an example page?


SLIM-753: Refactor Helpers

Splits up Helpers classes into classes with a certain purpose.
Same is done for the RepositoryHelper class.

 All committers of the pull request should sign our Contributor License Agreement in order to get your pull request merged.1 out of 2 committers have signed the CLA.:white_check_mark: robindenadel:x: web-flow


Web App displays a white screen

Let me start by saying that I love your app. I have been using it for hours every day for like the past year or so. But it suddenly stopped working today and I only get a white blank screen on the website. I tried it on 3 different browsers and my phone too, without success.
There are also some Type Errors in the dev console.

Hi @clarkissac , thanks for reaching out and glad to hear you are loving Lofi FM 🩷
The app was down due to some of the unexpected changes from youtube's APIs, i have pushed some changes and right now the app should be working for you
Feel free to comment here if you are facing issues

cool thanks for the quick fix :D


GraalVM Support

Describe the bug
I was playing around with getting a GraalVM native image up & running on Google Cloud Run.
During startup following error was thrown
org.springframework.beans.factory.BeanDefinitionStoreException: @Configuration classes need to be marked as proxyBeanMethods=false. Found: [org.springframework.cloud.gcp.autoconfigure.core.GcpContextAutoConfiguration, org.springframework.cloud.gcp.logging.LoggingWebMvcConfigurer, org.springframework.cloud.gcp.autoconfigure.logging.StackdriverLoggingAutoConfiguration]

Checking other Spring Cloud Configuration classes I detected following issue: https://github.com/spring-cloud/spring-cloud-commons/issues/623 and corresponding commit: https://github.com/spring-cloud/spring-cloud-commons/commit/18a4227c4ef1bdf86c3103e64bce75847a53073e#diff-878bfcb22b53612b49e3b34e803d12d2
Would it make sense to move all
@ Configuration

to
@Configuration(proxyBeanMethods = false)

If so, I'm happy to do this.
Sample
Google Cloud Run hello-world java sample with graalvm adaptions described here. I can create a sample repo if needed
https://repo.spring.io/milestone/org/springframework/experimental/spring-graalvm-native-docs/0.8.0/spring-graalvm-native-docs-0.8.0.zip!/reference/index.html#getting-started-buildpacks

Sounds reasonable, I saw @elefeint made an issue a while back for this too: https://github.com/spring-cloud/spring-cloud-gcp/issues/2500
Not too familiar with what proxyBeanMethods mean though; wonder if we used any of these in our integrations. Let's hear from others on the team their thoughts.

It should be safe to just copy/paste the parameter in all our autoconfig. proxyBeanMethods=true is only needed when bean methods call each other directly.


D3D11 support

The Hazel engine I've been following is very good and very interesting, especially being able to create components and scripts in C#.
Are there any future plans to support DirectX 11? It would be really interesting if Hazel could support DX11, as not all GPUs support Vulkan directly.
From what I could see in the code, vulkan was directly inserted in the rendering layer without any extra abstraction, which makes it difficult to modify it, having to change several parts hoping the code doesn't break.

Are there any future plans to support DirectX 11? It would be really interesting if Hazel could support DX11, as not all GPUs support Vulkan directly.

There's no future plans to support any DirectX version currently, as Cherno has said that he wants to support as many OSes as possible, and by having DirectX 11 / 12, that would kill that possibility! And Cherno has also already said, that he won't support multiple Graphics API's, so when we get to implement Vulkan, that will mean that we kill OpenGL support! Lastly, most GPU's these days support some kind of version of Vulkan, so it shouldn't be a problem anyway.

Yeah i'm with old GPU (GeForce 710M), and sadly it does not support any Vulkan version 😞

Yes, my GPU is really old (GeForce 710M) unfortunately it doesn't support Vulkan 😕
But anyway, thanks for the info!

Yes, my GPU is really old (GeForce 710M) unfortunately it doesn't support Vulkan 😕
But anyway, thanks for the info!


More affirmative content on the storage account RBAC and identity restrictions.

[Enter feedback here]
In the  Note section, it would be good to tweak it to make it look more affirmative like:
Adding an Azure Cache for Redis instance as a storage blob data contributor through system-assigned identity and check “Allow Azure services on the trusted services list to access this storage account” is needed for export to work with storage account with firewall exceptions. If you're not using managed identity and instead authorizing a storage account with a key, then having firewall exceptions on the storage account will break the persistence process and the import-export processes.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 35c80a11-1c37-ce38-8681-c20253539b1b
Version Independent ID: 8fdaaac0-ec05-0fd8-261d-6876ead226d0
Content: Managed identity for storage accounts - Azure Cache for Redis
Content Source: articles/azure-cache-for-redis/cache-managed-identity.md
Service: cache
GitHub Login: @flang-msft
Microsoft Alias: franlanglois

@VigneshB-AzurePaaS
Thanks for your feedback! We will investigate and update as appropriate.

Hi @flang-msft,
Could you please look into this document request, thanks.

@Naveenommi-MSFT @shaktisingh-msft @flang-msft , created a PR for this, https://github.com/MicrosoftDocs/azure-docs/pull/105752

@Vignesh-AzurePaaS - we closed the PR you reference here. If this does not reslove the issue, please feel free to re-open.
#please-close


Moving expval calculation into backends and adding torch differentiation

Why does Qiboml need a further function in the backend?
This is coupling once more the frontends with the backends, and defeating the original purpose of making them independent.
So, to have a circuit on hardware (QibolabBackend) differentiable by Pytorch, we will need to maintain a whole PytorchBackend, same for each possible other library, that in principle could be extended even beyond TensorFlow + Pytorch + JAX at the price of implementing a single function, not a whole backend.
Execution and derivative presentation are completely separate domain, thus we should keep them separate to preserve modularity.

Why does Qiboml need a further function in the backend?
This is coupling once more the frontends with the backends, and defeating the original purpose of making them independent. So, to have a circuit on hardware (QibolabBackend) differentiable by Pytorch, we will need to maintain a whole PytorchBackend, same for each possible other library, that in principle could be extended even beyond TensorFlow + Pytorch + JAX at the price of implementing a single function, not a whole backend.
Execution and derivative presentation are completely separate domain, thus we should keep them separate to preserve modularity.

This is extremely true. Thanks for catching this before further modifications. While experimenting, yesterday, I lost this fundamental point.
I'll move all the torch features into the sym backend (that part is working anyway) and close this.


Remove overhead fan in master bedroom

Too close to our heads. No one likes it anyway.

Has it been done Duane? You're leaving us hanging here.

I'm not a fan.

418 - I'm a teapot.

Alas, it has not been done :) Your inquiry, however, does help me
prioritize house repairs.
On Fri, May 22, 2015 at 11:00 AM, Maxime Rouiller<EMAIL_ADDRESS>wrote:

418 - I'm a teapot.
—
Reply to this email directly or view it on GitHub
https://github.com/canadaduane/house/issues/5#issuecomment-104713852.

Have you tried assigning the bug to a local Github-er?
In true open source fashion, you should tag some of them for "up-for-grabs" :smile:


Using tap-hold, no output with multiple modifiers and 1 key tapped

I have 3 tap-hold keys in my setup, space and the two keys on either side. All the modifiers work with all the other keys, and a single tap-hold modifier works with any of the other tap-hold key. However, holding two tap-hold modifiers and tapping the 3rd tap-hold key doesn't do anything.
using evtest with the "My Kmonad Output" I get:

Hold
Tap
Expected
Actual

@met_ret
@met_spc
met+spc
met+spc
:white_check_mark:

@met_spc
@met_ret
met+ret
met+ret
:white_check_mark:

@met_ret+@ctl_esc
a
met+ctl+a
met+ctl+a
:white_check_mark:

@met_ret+@ctl_esc
@met_spc
met+ctl+spc
met+ctl
:x:

(defcfg
  output (uinput-sink "My KMonad output")
  cmp-seq lalt
  cmp-seq-delay 5
  fallthrough true
  allow-cmd true
)

(defsrc lalt spc ralt)

(defalias
  ctl_esc (tap-hold-next-release 200 esc lctl)
  met_spc (tap-hold-next-release 200 spc rmet)
  met_ret (tap-hold-next-release 200 ret lmet)
)

(deflayer qwerty @met_ret @ctl_esc @met_spc)

Are you sure, that the issue is not with the maximum key-rollover of your keyboard?
It is common, that certain keys can not be pressed together.
You can test that online for example:
https://www.mechanical-keyboard.org/key-rollover-test/
However this doesn't work if your the keys are captured by your browser.
You can also try whether the keys work together while kmonad is not running to check whether the issue is with your keyboard or kmonad.

Without kmonad it works, with kmonad it doesn't. That doesn't sound like a key-rollover issue to me, but I know very little about the internals of keyboards.
Happy to provide additional info if you're interested in investigating or if it comes up with someone else.


Add dream-html example

Fix #318

Thank you!
I'll integrate this into the rest of the docs and the main README in follow-on commits.


Evaluate options for big file uploads and implement

There are several options for multi-POST, big file uploads:

https://github.com/pgaertig/nginx-big-upload
https://github.com/vkholodkov/nginx-upload-module
https://github.com/openresty/lua-resty-upload

Figure out what works best, install it, and figure out the client scripting needed to make it useful.

nginx-upload-module is not working after 1.3.9.

If a special build is required, then I should probably step back and take another look at having separate containers for nginx and django, because I don't want to add all the steps for the nginx build to the existing Dockerfile.

Gave up on trying to do it through HTTP: CLI works.


Error with transfroms.Scale when feeding iterable size

I'm having a problem with transforms.Scale function, which complains
TypeError: unsupported operand type(s) for /: 'tuple' and 'int'
when feeding an iterable size.
It persists even after installed latest packages.
I don't see any problem in a source code, but seems like the if statement gets True for isinstance(self.size, int)
Here's how it fails.
>>> import torch
>>> import collections
>>> from torchvision.transforms import Scale
>>> from PIL import Image
>>> im = Image.open('pos19.jpg')
>>> Scale(size=(512,256))(im)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/ntomita/pytorch-env/lib/python2.7/site-packages/torchvision-0.1.8-py2.7.egg/torchvision/transforms.py", line 139, in __call__
TypeError: unsupported operand type(s) for /: 'tuple' and 'int'
Now checking each portion in a source.
>>> transform = Scale(size=(512, 224))
>>> isinstance(transform.size, int)
False
>>> (isinstance(transform.size, collections.Iterable) and len(transform.size) == 2)
True
Condition terms look fine, but when calling,
>>> transform(im)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/home/ntomita/pytorch-env/lib/python2.7/site-packages/torchvision-0.1.8-py2.7.egg/torchvision/transforms.py", line 139, in __call__
TypeError: unsupported operand type(s) for /: 'tuple' and 'int'
It fails. If I call resize directly,
>>> im.resize(transform.size, transform.interpolation)
<PIL.Image.Image image mode=L size=512x224 at 0x7F7F8C6D4F50>
works.

By the way here's the version of torchvision I use
    torchvision: 0.1.8-py27_2 soumith
Is this the latest one?

I have the same issue. I also use 0.1.8, which seems to be the latest version.
The problem is easy to solve, apparently they forgot to add the option of size being a tuple (even if it specified this way). In I believe something like this would solve it:
    def __call__(self, img):
        w, h = img.size
        if (w <= h and w == self.size) or (h <= w and h == self.size):
            return img
        if w < h:
            ow = self.size
            oh = int(self.size * h / w)
            return img.resize((ow, oh), self.interpolation)
        else:
            if isinstance(self.size, tuple): #<--- check if tuple, in which case just use it
                 return img.resize(self.size, self.interpolation)
            oh = self.size
            ow = int(self.size * w / h)
            return img.resize((ow, oh), self.interpolation)

I see, I just avoid using the function and directly call resize() for now.
def _scale(size):
    def resize(im):
        return im.resize(size, Image.BILINEAR)
    return resize

Hi @antspy this has been fixed in this PR. However, a new version of torchvision has not been released. If you install from source you will have this fix.
cc @fmassa I think we can close this


Spatial shader triplanar mapping binormals are flipped on the y axis.

Godot version: 3.2.4 RC3
Windows 10.0.18363
GTX 650 Ti 2GB, Driver version 461.40, GLES3
Issue description:
After activating the triplanar checkbox in a spatial material, the normals seem to be flipped in some directions.
After converting to a shader material and looking at the vertex function, it seems the binormals are flipped.
In the shader they are as such:
BINORMAL = vec3(0.0,-1.0,0.0) * abs(NORMAL.x);
BINORMAL+= vec3(0.0,0.0,1.0) * abs(NORMAL.y);
BINORMAL+= vec3(0.0,-1.0,0.0) * abs(NORMAL.z);

While changing them to:
BINORMAL = vec3(0.0,1.0,0.0) * abs(NORMAL.x);
BINORMAL+= vec3(0.0,0.0,-1.0) * abs(NORMAL.y);
BINORMAL+= vec3(0.0,1.0,0.0) * abs(NORMAL.z);

Seemed to fix the problem.
Steps to reproduce:

Create a new project.
Add a mesh instance and assign any mesh.
Create a spatial material and assign it to the mesh.
Assign a normal map to the spatial material.
Click the triplanar checkbox in the UV 1 dropdown.
Add a light to the scene and observe the behavior of the material when the light is moved.
Triplanarbug.zip

Minimal reproduction project:
Triplanarbug.zip

This doesn't look like a proper normal map. Tangent space normal maps are supposed to be blue-ish in flat regions.

This normal map is generated in Material Maker and I assumed Godot doesn't use the blue channel. It works well on UV mapped surfaces, I added a blue channel to it and the problem still persists. If I flip the green channel if fixes the triplanar material, but breaks the UV mapped material. The UV mapped materials seem to use X+ Y+ Z+ normal maps and the triplanar seems to use X+ Y- Z+ normal maps.
With this one, it looks fine if I use UVs, but broken if I use triplanar mapping.

This one looks fine with triplanar mapping, but broken with UVs.

All I did was switch the green channel.

@Zylann Godot performs RGTC compression of normal maps.

@Calinou That's strange, because the first normal map from my previous comment is working as intended, while the second is not. I even tested in 3.2.3 and the same thing happens. But the issue is that UV mapping and triplanar mapping expect different Y directions.

@Zylann Godot performs RGTC compression of normal maps. The blue channel is discarded as it can be reconstructed in the shader from the two other channels.

This is why I precised in parenthesis that this is the PNG as is, not the imported result. I know Godot does this. You mean it can also accept pre-compressed normalmaps?

You mean it can also accept pre-compressed normalmaps?

It should accept normal maps already in RGTC format.

Just wanted to confirm as well. I've just been running into this issue
https://user-images.githubusercontent.com/4955051/123541507-e2528300-d73c-11eb-926e-304e2eed068a.mp4
Left: Standard UV. Middle: Standard triplanar UV. Right: Custom triplanar shader with binormals inverted.
I had found the the same solution as victfv of inverting the three binormals.

I can confirm this issue on master Git 659178e85:


Try ccache speed up compilations

See https://www.virag.si/2015/07/use-ccache-with-cmake-for-faster-compilation/

conda recipe available here https://github.com/omnia-md/conda-recipes/tree/master/ccache

Complicated interaction with conda gcc and CMake, not necessary since cfc831e0cb3b, closing.


[6.5.x][JBPM-5282,RHBPMS-4107] recognize GetTaskByWorkItemIdCommand when pr…

…ocessing JAXB responses
@mrietveld could you please review this?

+1 Thanks!


[Design] navbar

use svg icons: #286
always refer to figma for latest design reference

According to @gr8Mu1 the navbar is almost finished except for the shading of the qr-code scanner.


How to make the page view shrink?

Trying to run this on responsive mobile and can't figure how to make the actual page shrink. I can get the viewer to shrink but the page is still like 1000px wide.
is there a way to make the page shrink so it is like 95% of the container width?

The only way of doing this would be to have a listener onRisize and change the scale prop dynamically.
Unfortunatelly there is no way of changing width and height of the pdf itself, only the scale prop


Add two issues and solutions to Troubleshooting.md

Thanks for submitting a pull request! Please provide enough information so that others can review your pull request:

Unless you are a React Native release maintainer and cherry-picking an existing commit into a current release, ensure your pull request is targeting the master React Native branch.

(You can skip this if you're fixing a typo or adding an app to the Showcase.)
Explain the motivation for making this change. What existing problem does the pull request solve?
Prefer small pull requests. These are much easier to review and more likely to get merged. Make sure the PR does only one thing, otherwise please split it.
Test plan (required)
Demonstrate the code is solid. Example: The exact commands you ran and their output, screenshots / videos if the pull request changes UI.
Make sure tests pass on both Travis and Circle CI.
Code formatting
Look around. Match the style of the rest of the codebase. See also the simple style guide.
For more info, see the "Pull Requests" section of our "Contributing" guidelines.

By analyzing the blame information on this pull request, we identified @hramos and @knowbody to be potential reviewers.

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign up at https://code.facebook.com/cla - and if you have received this in error or have any questions, please drop us a line at<EMAIL_ADDRESS>Thanks!

Hey @greggb, typically we only cherry-pick commits into releases once they are in master. Can you open a new PR against master and not the 0.29 branch? Thanks!

Sure thing, thanks for looking.


Turbo is unable to compile source with pdfjs

Verify canary release

[X] I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue
https://github.com/Matts/pdfjs-and-turbo-incompat
What package manager are you using / does the bug impact?
npm
What operating system are you using?
Mac
Which canary version will you have in your reproduction?

Describe the Bug
The code that crashes is never run, but the fact that it "could" run causes turbo to freak out on both client and server side
Expected Behavior
This should work
To Reproduce
See https://github.com/mozilla/pdf.js/discussions/18804

Create a nextjs + turbo project
Follow the example installation at https://mozilla.github.io/pdf.js/examples/index.html#interactive-examples
You will receive a compilation error from turbopack (see image below)

it seems that turbo pre-interprets import statements and tries to resolve them. I see that there is a webpack ignore statement, possibly there has to be a similar statement for turbo

Additional context
I am not technically included enough to reproduce this within turborepo but seeing as this does not happen with old dev server it must be turbo.

Hello, Turbopack is no longer part of this repository. Could you file an issue in it's new repository vercel/next.js?


Fix cross-platform random Int64 generation

In the DataBunch definition, this replaces a call to random()  (which works on Linux but not on macOS), with the cross-platform expression Int64.random(in:Int64.min..<Int64.max).
This also removes an Glibc import which had no effect on Linux and broke macOS.
This PR only updates the notebooks, and not the pre-generated Swift packages. (Should I update those too?)
I tested by running the 04 and the 05 notebooks. 04 works fine. 05 fails in the same way before and after the change, at cell 46, due to what I believe are unrelated issues with unresolved or identifiers.

Thanks!


Added BMI calculator

This is a BMI calculator which generates BMI when entered height and weight. It also tells whether you are underweight, overweight or normal.
https://github.com/Priyanshi662/FunFusion/assets/140010647/43a5004c-2c5a-4bd3-84da-09ed1157e165

@Aashi19234 has conflicts please resolve and make PR

@Priyanshi662 Resolved the conflicts.


Can't fix the test.. need more info about the test purpose and why the schema is different?

When debugging the tests i found it is checking this schema which has three @attributes.. so why is this? and can we get more info about the purpose of this test?

Issue:
1- Running this test throw this error
kosmos 'j.data.schema.test(name="load_from_dir")'
test:
https://github.com/threefoldtech/jumpscaleX_core/blob/development/JumpscaleCore/data/schema/tests/8_load_from_dir.py
Schema used by the test:
https://github.com/threefoldtech/jumpscaleX_core/blob/development/JumpscaleCore/data/schema/tests/schemas_toml/threefoldtoken_wallet.toml
@url = threefoldtoken.wallet
@importance = true
@systemprop = 1
jwt = "" (S)                # JWT Token
addr** = ""                   # Address
ipaddr = (ipaddr)           # IP Address
email = "" (S)              # Email address
username = "" (S)           # User name

@john-kheir
it works Good at development_fixes branch but we need to understand
what this schema mean ?
@importance = true
@systemprop = 1

we will close it for now


Roborock app support for Roborock S6 MaxV

Hello,
I just got the Roborock S6 MaxV today and tested both available apps.
With the Xiaomi Mi Home App you can name the rooms and I got a newer firmware version as with the Roborock app.
During the connection establishing phase I was also able to sniff the required token with Charles Proxy (iOS).
The Roborock app seems to be far more optimized for these vacuum cleaners even if you can't name your rooms.
It has some features I don't want to miss, like the camera view.
But what's far more concerning for me: I just found out that on every new connection the device token changes itself.
On an issue this was confirmed. Is there any way to gather the device token while using the Roborock app?
The sniffing method didn't work. The only result I was getting was the following response to a GET request to https://api-eu.roborock.com/user/homes/
{
    "api": "è·åå®¶åºè¯¦ç»ä¿¡æ¯",
    "result": {
        "id": "<id>",
        "name": "Mein Zuhause",
        "lon": null,
        "lat": null,
        "geoName": null,
        "devDisplayOrder": null,
        "products": [{
            "id": "<id>",
            "name": "Roborock S6 MaxV",
            "code": "a10",
            "model": "roborock.vacuum.a10",
            "iconUrl": null,
            "attribute": null,
            "capability": 2,
            "category": "s6",
            "schema": [{
                "id": "101",
                "name": "rpc_request",
                "code": "rpc_request",
                "mode": "rw",
                "type": "RAW",
                "property": null,
                "desc": null
            }, {
                "id": "102",
                "name": "rpc_response",
                "code": "rpc_response",
                "mode": "rw",
                "type": "RAW",
                "property": null,
                "desc": null
            }]
        }],
        "devices": [{
            "duid": "<duid here>",
            "name": "<name here>",
            "attribute": null,
            "activeTime": "<activetime here>",
            "localKey": "<localkey here>",
            "runtimeEnv": null,
            "timeZoneId": "Europe/Berlin",
            "iconUrl": "",
            "productId": "<product id here>",
            "lon": null,
            "lat": null,
            "share": false,
            "shareTime": null,
            "online": false,
            "fv": "01.46.00",
            "pv": "1.0",
            "roomId": null,
            "tuyaUuid": "<uuid here>",
            "tuyaMigrated": false,
            "extra": "{\"xxxx\": \"xxxx\", \"RRPhotoPrivacyVersion\": \"1\", \"RRMonitorPrivacyVersion\": \"1\"}"
        }],
        "receivedDevices": [],
        "rooms": []
    },
    "status": "ok",
    "success": true
}

As you can see there are multiple fields like duid, localKey and tuyaUuid which could(?) be responsible for the secure connection.
localKey changes on each connect so unfortunately this can't be used as a replacement for token (didn't look into miio to validate if token can be replaced by anything so this is full speculation). Also localKey is too short in comparison with token.
Did anyone else take any effort to make this plugin usable with the Roborock app or better did anyone manage to extract the used token while using the Roborock app?
In every case thank you for developing and sharing such a great plugin!
//Update 1:
I looked into the app by doing a backup of it and couldn't spot any obvious places where such a token could be stored.
One place which is very similiar to the sniff at the top and stores some more things which look suspicious is located at Roborock.app\Container\Library\Preferences\com.roborock.smart.plist
Obviously this file contains some details I won't publish so if someone has interest to look into it he can write me private.
//Update 2:
I also tried to use an old version of the Roborock app on an android emulator and gather the files of the application.
There are also some interesting strings but nothing clearly showing "token". I've already tried different strings but no luck...

I looked into the app by doing a backup of it and couldn't spot any obvious places where such a token could be stored.
One place which is very similiar to the sniff at the top and stores some more things which look suspicious is located at Roborock.app\Container\Library\Preferences\com.roborock.smart.plist
Obviously this file contains some details I won't publish so if someone has interest to look into it he can write me private.

Hi, there is a discussion to try to find out if we can get Roborock app to work with this plugin: https://github.com/homebridge-xiaomi-roborock-vacuum/homebridge-xiaomi-roborock-vacuum/issues/104
But you are correct, apparently, the issue with the Roborock app is that it reissues the token, voiding the token assigned to this plugin in the config.
I personally don't use that app (and knowing it breaks the integration, I feel like I won't even try 😅). But I'd love it if anyone can find a way to extract the token from that app (everyone praises it, so I bet it's way better than the Xiaomi Mi app).

I gave up to gather the local token (with the assumption that they aren't using any local control, instead an online-only control) but tried instead to replicate the login behaviour in the app in order to write a homebridge plugin which utilizez their API but that is also very painful.
They are using HMAC for authentication and at some point my decompiler fails to decompile the APK properly. Basically they are using different sources for their secret like some data gathered from a request to a tuya server, md5(url) and at some point they use the Tuya SDK to do the HMAC_SHA_256 encryption. I bet they are adding some salt or something there because their function don't look very standardized.

Wow! Thank you for looking into that! It does seem complex indeed.

@Zer0x00 I stumbled on this page googling for info and though I would let you know some things.
I use home assistant and had fine integration using the mi app.. when i switched to the roborock app, it no longer worked. (as you well know) however some folks in the home assistant forums found when you switch apps, it also switches protocols the device uses, when using the roborock app it now uses the Tuya platform (also the ports change on the device) several people were able to get a S5 working using the roborock app, but i have an s6 maxV and have not been able to get mine to work. hence all my google searching for info. but the thing with the Key is changes when you reset the wifi, not every time you use it. they are using the local key  as the token. that info is found in (on android anyways) in the files/rr_cache foler extract those files, they are gzipped (just add .gz and then unzip) the info is in json format. anyways. just though i'd through out what i know for now. if you want to dig in some more search the home assistant forums and you should find all the info there.

@ghvader Thanks for your info. The S6 series is afaik currently the only one which changes the token when changing from Tuya to Roborock so no luck there :/
It seems to me that they are using MQTT with SSL authentication to provide their functions.

I posted this on another thread, but wanted to relay it here. I found the token in the roborock app / backup files ripped from an iOS device (iphone). Using a program called iBackup Viewer.
Its located in the following file:
Library/Preferences/com.roborock.smart.plist
Then located under:
UserDefaultKeyUserModel
Sub:
token
With this information, Im thinking two things can happen.

figure out a work around with the miio plugin
write a new plugin
or someone comes up with another idea (whos smarter then me) lol

All input is welcomed! :)

Just curios... wonder if there is a work around using either IFTTT, Shortcuts, or both...
Example: I have a wall mounted Ipad. lets say its logged into the RoboRock App. wonder if you could create an relay, from Homekit>homebridge>IFTTT>Roborock OR something similar...


Conflict warning about swift-syntax

I'm getting this warning during package resolution:
'hexcolors' dependency on 'https://github.com/apple/swift-syntax.git' conflicts with dependency on 'https://github.com/swiftlang/swift-syntax' which has the same identity 'swift-syntax'. this will be escalated to an error in future versions of SwiftPM.

I didn't dig a lot further but I suspect they only changed the repo from apple to the swiftlang org. So it's probably only a matter of importing the package by using the new URL.

PR #3


IndexError: bytes to write exceed buffer size

I installed Carla by Quick Installation <IP_ADDRESS>. And install carla-ros-bridge as developers. When I try to test ros-bridge through roslaunch carla_ros_bridge carla_ros_bridge_with_example_ego_vehicle.launch. Some errors occur as follows

How can I fix it?  Thank you! Thank you!

@joel-mb could you please ask for further information and take a look at this issue?

Hi @shizhuocaoa ,
Can you provide more information about your setup? Is the manual control (without using ROS) working? Try running the manual_control.py script provided inside /opt/carla-simulator/PythonAPI/examples.
Please, take a look to the solution of this similar issue also (https://github.com/carla-simulator/carla/issues/991).

I seemed have met the same problem,u can try change to install new pygame version using "pip install " rather than "apt".

Installing pygame using "install" solved my issue too! I used the following command: python2 -m pip install -U pygame --user
Then running manual_control.py using python2.7 rather than python3.6.
With python3.6, I am getting ImportError even when running spawn_npc.py. However, with python2.7 it works fine.

@yasser-h-khalil  the command is useful :  python2 -m pip install -U pygame --user
Thank you!


Negative Binomial Regression Implementation

This is a tutorial for Negative Binomial Regression using Turing. All code is commented and theory is also explained.
I tried to stick to the Tutorials Roadmap described in #1 .

@cpfiffer I am getting NaN values while sampling. Could you please explain to me how I can correct it. Regards.

If you are talking about this:
┌ Info: [Turing] looking for good initial eps...
└ @ Turing.Inference /home/riteshk/.julia/packages/Turing/yF1Mq/src/inference/support/hmc_core.jl:245
┌ Warning: Numerical error in gradients. Rejecting current proposal...
└ @ Turing.Core /home/riteshk/.julia/packages/Turing/yF1Mq/src/core/ad.jl:169
┌ Warning: grad = Real[NaN, NaN, NaN, NaN]
└ @ Turing.Core /home/riteshk/.julia/packages/Turing/yF1Mq/src/core/ad.jl:170
┌ Info: [Turing] found initial ϵ: 0.013036584854125979
└ @ Turing.Inference /home/riteshk/.julia/packages/Turing/yF1Mq/src/inference/support/hmc_core.jl:237
┌ Warning: Numerical error in gradients. Rejecting current proposal...
└ @ Turing.Core /home/riteshk/.julia/packages/Turing/yF1Mq/src/core/ad.jl:169
┌ Warning: grad = Real[NaN, NaN, NaN, NaN]
└ @ Turing.Core /home/riteshk/.julia/packages/Turing/yF1Mq/src/core/ad.jl:170
┌ Info:  Adapted ϵ = 0.010067677574384128, std = [1.0, 1.0, 1.0, 1.0]; 200 iterations is used for adaption.

Then that is normal. You don't need to do anything there.
I believe I mentioned this in Slack, but this PR is remarkably similar to an existing PR (#25) --- what is this tutorial adding that's not found there?

@cpfiffer This notebook demos negative binomial regression using Turing. It closely follows #25  Poisson regression example except for the data here is negative binomially distributed instead of Poisson distributed.
Negative binomial regression is used to model count data for which the variance is higher than the mean. The negative binomial distribution can be thought of as a Poisson distribution whose rate parameter is gamma distributed, so that rate parameter can be adjusted to account for the increased variance.
I have tried to show the same principle, in the notebook, where I have shown data derived from Poisson and NB Regression and compared the two. I attempted to implement one example from PyMC3.
Please let me know your views on it. And if there is anything I can do to improve it.

Could you:

Repackage this in a negativebinomial.jl file with minimal comments
Keep all the setup code and the model, and remove anything that has to do with plotting, diagnostics, or otherwise describing the chain
Submit a PR to this repo instead: https://github.com/TuringLang/TuringExamples

We're going to try to get smaller tutorials like this into our model zoo, so that people can browse through them and figure out how to make models a little quicker. I'

@cpfiffer I have added the negativebinomial.jl file here. Please review it. If it looks good then I will open a PR on  https://github.com/TuringLang/TuringExamples

That is perfect, thank you!


Version not updated in footer for last two updates

Hi team,
Love the work - have noticed that the version hasn't been updated in footer.php for the last two updates (still reflecting 2.2.1). Didn't think it worth an individual pull request, but figured it was worth highlighting.
Dave

I know


Use cudatoolkit=11 in both tensorflow and pytorch images

Currently tensorflow (ml-notebook) uses cudatoolkit=10, and the new pytorch image uses cudatoolkit=11. It would be good to keep those versions in sync. Full discussion here:
https://github.com/pangeo-data/pangeo-docker-images/pull/315#pullrequestreview-955549154

@scottyhq although I have access to pangeo hub, I haven't really been able to get going with usage (I do have significant allocated compute resources elsewhere and so I don't want to squeeze shared resources). However, if I remember correctly, the GPUs on pangeo hub were quite old, like M6 or Q 4000 or something like that --- is that still the case?
As you rightly pointed out in the other discussion, the cuda-related software is pretty huge and so unless the hardware is equipped to handle cudatoolkit=11+, I wouldn't necessarily update.

Another related issue (perhaps for another discussion):
It may be useful to think about splitting the containers directions in the future. Maintaining the GPU-optimized builds has been really challenging on conda-forge (for example, tensorflow 2.9 is about to be released and we haven't been able to build 2.8 yet...). In my current research work, I tend to only do the preliminary testing/exploratory in a conda/pip/etc environment, but once I get going, I very quickly move to working with NGC containers. This usually yields significant performance improvement in my experience.
The point is: it may be useful to build the ML-GPU images on top of NGC containers instead of the other containers here. The NGC containers come equipped with optimizations for GPUs and we could then add the rest of the software stack on top of that. In the case of tensorflow, likely it would be done via pip; but I believe the NGC container for pytorch uses conda by default --- I haven't used it in a few months, so not sure if they changed. Either way, I would strictly follow whatever installation technique they used (this is pip-esque in the tensorflow containers)
Anyway, just a thought. While per-instance performance improvement might be small, if we are talking about large-scale deployments with a lot of researchers and a lot of academics using these containers, that can be quite significant (e.g. saving power, resources, time, etc.)

But also a considerable amount of work.

I actually think it may be slightly less work because the NGC people update their images every month with exact versions, etc. --- I will start looking into this to see how far I could go by installing the core deps from the containers into the NGC containers.

The challenge is that the rest of the pangeo stack (particularly packages with hdf5 and gdal binary dependencies) does not install well via pip. So if we want an image with both optimized ML libraries AND all of those funky geo packages, it's going to be tricky.

YES, this is the really hard part. However, we might be able to pull it off! I will start some preliminary work and report back.

There are many different Pangeo hubs, operated by different communities. The LEAP Pangeo Hub run by 2i2c has K80s.

I think we should be careful how far we go optimizing the software without paying attention to the hardware... The NGC containers support Ampere, Turing, Volta, and Pascal only (at least according to their website). The base images are pretty substantial fwiw, around 15GB.

The base images are pretty substantial fwiw, around 15GB.

🤯 that's huge!
Depending on the NGC containers would be a major refactor for pangeo docker images. But I'm excited by your enthusiasm and would welcome this improvement. 🚀  Let's definitely give it a try and see how far we can get.

There are many different Pangeo hubs

Ooops, sorry, I meant pangeo cloud: https://pangeo.io/cloud.html

Let's definitely give it a try and see how far we can get

I started a new repo to investigate building on top of NGC containers: ngam/ngc-ext-pangeo (second PR there: https://github.com/ngam/ngc-ext-pangeo/pull/2). I think I've isolated a few of the problematic packages (e.g. cartopy) and will likely need to figure out a way around that (i.e. build proj, geos, etc. libs from scratch). Baby steps at first, but we will see how far we get. I will add tests and other customizations later (or even better, maintainers and contributors here can do a much better job than me!)

The base images are pretty substantial fwiw, around 15GB.

🤯 that's huge!

I know... I never really work with the Docker images themselves. I convert them into Singularity SIF images because of the HPC policies, and for some reason, the size of the tensorflow image ends up being closer 6 GB only as opposed to the 14--15 GB Docker size. I've just submitted a Singularity image build; only adding netcdf4 on top of the NGC container results in this https://cloud.sylabs.io/library/ngam/nvtf/2204 which is just shy of 6 GB

A smaller update and a correction:

I've got almost everything in place with only a few deps remaining to be resolved, now time to fine-tune this and fix corner cases, but also make it more like a legit workflow with checks, and optimizations, etc.
Above, I took the 15GB figure from the ngc website, they are actually only 6GB compressed (same as the sif files) which is in the same region as the current pytorch container here.

Fantastic! Thanks for the update @ngam! Let us know how we can help or whenever you might be ready for someone else to test drive your images.

My only other concern is that the NGC containers are a little more proprietary/less transparent that the current stack

Someone with more knowledge / experience would need to address this, not me 😅 But as far as I could tell, it's the same license. When one installs cudatoolkit from conda-forge, they print out that message, right? Also, you may think all of conda-forge is open-source --- nope! A lot of software from nvidia and intel is actually simply repackaged, i.e. binary repackaging, not even rebuilt from source. This is a frustrating issue for people who actually care about these things, but if I understand correctly, it makes no difference at all for academic/research usage, which is my area, and so I never pay close attention. At any rate, even with my skills of sleuthing around the internet, I couldn't find how exactly they build these containers, but I think I have a decent idea how based on playing around. The way they build the more basic cuda containers is a little more open (available on gitlab).

Optimizing for different NVIDIA GPU generations (e.g. K80s, P100s, V100s, A100s, T4s, H100s, etc) is a tricky business, and I'm wondering what those NGC containers are currently designed for (my guess is the newer harware, i.e. A100 and up).

According to their website: "Ampere, Turing, Volta, and Pascal". But I agree with your guess, I would say most of the more modern optimizations won't make a difference for anything lower than sm 7.0, so I would say these NGC containers won't make much of a difference for anything besides Volta and Ampere, maybe T4 as well...

I will try to upload some images later this week. We can at least document the process for interested community members if they have access to V100 or A100 GPUs and want some more performance!

xref: https://github.com/ngam/ngc-ext-pangeo/issues/24, docker pull ngam00/ngc-pt-pangeo to pull an image, and there are still missing packages: https://github.com/ngam/ngc-ext-pangeo/issues/21

Another note: last night, I've started an effort in conda-forge to build packages using singularity (as an alternative to docker). This way people, like me, who have access to vast computational resources but with no docker can help build complex packages that are not possible under the 6-hour limit for open-source repos. For now, we rely on the goodwill and generosity of maintainers to build locally, meaning spending literally days to build packages like tensorflow and pytorch, upload them separately, review the logs manually, and then move to the conda-forge channel. On an HPC, we can cut that by 10x or so.
I also am trying to add the optimizations from these ngc containers in the build process to the conda-forge's recipes, so the gap between the two should significantly narrow going forward. And with the singularity build process, I am hoping people can help lessen the strain on the main maintainers by offering to build on their HPCs.

In #345 I push tf to cuda112 and added pytorch to the ML notebook (not proposing removing the pytorch for now, so not disrupt people's workflows).
Quite soon, we should be able to add jaxlib==*=*cuda* to the mix. The issue with that is the icu and absl migrations --- these lead to wide and confusing conflicts, especially with the very complex geo packages. However, I am closing on getting them all in order.
Feel free to drop the pytorch part in #345 to keep it simple, but it is absolutely doable to combine all of them into one notebook. Note the size won't change much, the added pytorch will be just a few 100s MB


Remove bower and use npm to install dependencies

DRAFT: implements the changes from the PR #6213 on jupyter/notebook by @minrk and is based on PR #96 by @echarles

[X] remove bower

[ ] address 403 response for favicons

@RRosio Thx for this PR. Can you please add a topic to today agenda https://hackmd.io/gl6x_SL0S_aYVn0Y0lSBpQ?both to discuss this.

Closing this PR to open another based off of current main branch.


chore(deps): [release-1.3] bump go-toolset container image

Description

Which issue(s) does this PR fix or relate to

Fixes #issue_number

PR acceptance criteria

[ ] Tests
[ ] Documentation
[ ] If the bundle manifests have been updated, make sure to review the rhdh-operator.clusterserviceversion.yaml file accordingly

How to test changes / Special notes to the reviewer

/assign @kim-tsao @nickboldt

/refresh

/hold cancel

/close


FIELDS.F62_Reserved_Private - array out of bounds

With the FIELDS.F62_Reserved_Private, the app always crashes with array out of bounds index issue.
Please help. If you want, I can provide the DE62 data.

@imohsenb


HELP NEEDED: AI write convert.py, intent to convert pth to onnx, now i have no clue where's the problem.

import torch
import os.path as osp

# Load required libraries and modules
from config import get_args

args, unparsed = get_args()
import utils
from loss import Loss

if "cain_encdec" in args.model.lower():
    from model.cain_encdec import CAIN_EncDec as ModelClass
elif "cain_noca" in args.model.lower():
    from model.cain_noca import CAIN_NoCA as ModelClass
else:
    from model.cain import CAIN as ModelClass

def load_model(args):
    args, unparsed = get_args()

    device = torch.device("cuda" if args.cuda else "cpu")

    # Define your chosen architecture based on command line arguments or default values here.
    print(f"Building {args.model} model")
    if "encdec" not in args.model.lower() and "noca" not in args.model.lower():
        depth_param = 3
        model = ModelClass(depth=depth_param)

    elif "noca" in args.model.lower():
        model = ModelClass(depth=depth_param)

    checkpoint_path = f"./model/cain.pth"
    assert osp.exists(checkpoint_path), f"No such file: '{checkpoint_path}'"

    utils.load_checkpoint(args, model=model, optimizer=None)

    return model.to(device)

@torch.no_grad()
def export_to_onnx(model):

    # Define dummy inputs with same shape as expected by your model.
    dummy_input1 = torch.randn((1, 3, 256, 448)).to(next(model.parameters()).device)
    dummy_input2 = torch.randn((1, 3, 256, 448)).to(next(model.parameters()).device)

    # Specify which axes should be treated dynamically (e.g., batch size).
    dynamic_axes = {"input_0": {0: "batch_size"}, "input_1": {0: "batch_size"}}

    # The name you want for outputs in the exported onnx file.
    output_names = ["output"]

    try:
        print(f"Exporting model '{args.model}' to ONNX...")

        # Export the PyTorch model using torch.onnx.export method.
        torch.onnx._optimize_trace = (
            False  # Disabling optimization temporarily due to known issues.
        )
        torch.onnx.export(
            model,
            args=(dummy_input1, dummy_input2),
            f=f"./{args.exp_name}_model_best.onnx",
            input_names=["im1", "im2"],
            output_names=output_names,
            export_params=True,  # Store trained parameters into .onnx
            opset_version=13,  # Choose a proper version of ONNX operator set.
            do_constant_folding=False,  # Whether to fold constants during conversion or not.
            verbose=True,  # Print more information about operations being processed.
            training=torch.onnx.TrainingMode.EVAL,  # We're exporting an inference-only graph here so we choose EVAL mode.
            keep_initializers_as_inputs=True,  # Keep initializers as inputs
            dynamic_axes={"input_0": {0: "batch_size"}, "input_1": {0: "batch_size"}},
        )
        print(f"Model '{args.model}' successfully exported.")
    except Exception as e:
        print("An error occurred while trying to convert your PyTorch model:")
        print(e)

def main():
    load_model(args)
    export_to_onnx(load_model)

I figure out it's a dumb quest anyway.


Any design details to be exposed about CaffeOnSpark?

I found this project really useful and want to do some interesting things with it as well as some possible pull requests. While it seems that the related materials is so limited that we have to bite the bullet to read the source code for a systematic understanding. That's really not an easy effort and worse we may produce some misunderstanding during this process.
So my question is that is there any design details to be exposed about this project in the near future? Or may you open source some detailed framework references just like caffe's.
Really appreciate for any useful comment!

This would be useful for new players.

@anfeng @mriduljain

It is unclear what you are looking for. Can you ask specific questions?
BTW, we are giving talks about CaffeOnSpark at both Spark Summit and Hadoop Summit this month. We are happy to meet user community and answer questions in person.
Andy Feng
Sent from my iPhone

On Jun 2, 2016, at 7:29 PM, Shiqing Fan<EMAIL_ADDRESS>wrote:
@anfeng @mriduljain
―
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.


Counting current number of blocks

Just came across abit of an issue which requires a check on the number of certain blocks that have been added.
Where abouts would it be I could see where the counter is for the blocks so I can check against the number in a block? (I know it's already there as there's the restriction on block numbers available in the config)
If it's not to hand I'll go diggin'

https://github.com/madebymany/sir-trevor-js/blob/master/src/block-manager.js#L23
var editor = new SirTrevor.Editor();
editor.block_manager.blockCounts ?

Maybe there's what you are looking for.
In older versions I think it was exposed on the editor object too, but shouldn't rely on that.

I've had a play around but no dice.
Couldn't get it to return a figure, so I've gone the hackey route of adding a counter in for the custom block I needed it on which seems to be doing the trick.

@SgtOddball You can also call the _getBlockTypeCount() method on the block manager. It accepts a blockType as an argument and will return an integer for you.

@cjbell  Thanks for that, I've given it a try and it's still playing up on me, I'll stick with my hack for now until I can spend some more time understanding what it's doing wrong (or more likely where I'm being cackhanded with it) as I'll be the first to admit I'm not the strongest javascript coder in the world (read. making it up as I'm going along).


How to fixe F-Keys when use Bluetooth

After I follow the tutorial to configure the F-Keys on Ubuntu everything starts work like a charm when I'm using the KeyBoard by cable.
If I use the KeyBoard with Bluetooth the F-Keys stops to work.
Is there anyone that could help?

I am sorry but I do not know a solution if anyone does please let us know and I will update the documentation (or send a PR and I will merge it).

Hey there, I love the keyboard, for me too, it does not work under Ubuntu 18.04, the num lock issue arises when using it over bluetooth. Pressing F6 twice does nothing (I don't know which version of the firmware I am running, was this a feature added to some firmware update?). Also, while using the cable, the num pad lock is still on, but the keys are recognized correctly. Any help?

If I knew a solution in would be in the k2.md! If you manage to find a solution let me know and I will update the repo - I should say this is not affiliated with the Keychron company, I am just a fellow user who wanted to put all my notes in a handy place :)

@nicolastakashi & @lucaruzzola I have been informed that this might fix it, let me know if it works for you and I will add it to the readme? https://unix.stackexchange.com/questions/121395/on-an-apple-keyboard-under-linux-how-do-i-make-the-function-keys-work-without-t

I have updated the k2.md to reflect the above article, this fix appears to be working for multiple users in the Keychron Linux Facebook group.


Using locked cells breaks filter functionality for that column

Describe the bug
Cell with lo:1 causes filter to not show up for the entire column.
To Reproduce
Steps to reproduce the behavior:

Make any cell as lo: 1
Try applying filter to its column

Expected behavior
Filter option should be visible.
@baian1

@sanchit3008 @gazedreamily i see #304 pr, filter.ts#L132, not allow readonly line be filter


#881 Create TCP/IP listener

PR is added to the queue for testing as 2. task. (0a57107)

Hi @dersim-davaod, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

Test (Success) 83452617 build is completed (0a57107)
See https://github.com/ThaliTester/TestResults/tree/834526170a57107__881_Create_TCP/IP_listener_dersim-davaod/ for the logs

Test 834526170a57107(0a57107) has successfully finished without an error
See https://github.com/ThaliTester/TestResults/tree/834526170a57107__881_Create_TCP/IP_listener_dersim-davaod/ for the logs


Layout: Priorities Page

Hero images no longer abut the sidenav.
The "Show more" lists aren't working.
https://intranet.fws.gov/region4/demo/priorities/

5a304e83027bafbb39b4236d39f55b1e3d4c7577


When site_id_lookup returns sth else than string messages are not dispatched to long polling clients

Noticed that if site_id_lookup returns an Integer instead of the string you will think everything works but messages will be retrieved only on the start of the request (after the client establishes the connection). Publishing new message will not transfer them in long polling chunked mode.
To explain better, once a client connects client.backlog is called to get all existing messages, that calls @bus.backlog which encodes site_id into a string and there's no issue there. Works as expected.
But later MessageBus::ConnectionManager is used to push out new messages, it uses @subscriptions[msg.site_id] to look up any clients (which is string even if the message was published with site_id set to an integer) but add_client uses client.site_id established by site_id_lookup which if it returns anything else than a string will break lookup in connection_manager.rb:25
Leaving for anyone scratching their heads.
Generally would be great to either assert returned type from site_id_lookup or convert it to a string in connection_manager.rb
That was my fault but would be great if API helped diagnose problems like that.

Yeah I totally support something like
site_id = site_id.to_s if site_id != nil

care to make a PR and maybe add a test for it?


⚠ ClamAV has degraded performance

In 109da4b, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 246 ms

Resolved: ClamAV performance has improved in 2452f64.


TypeError: cannot read property 'viewBoxWidth' of undefined related to Icons

I keep getting the following error when trying to use anything with an Icon property:
TypeError: Cannot read property 'viewBoxWidth' of undefined

I've installed both solid and regular fontawesome libraries as described in the documentation. I'm new to react so I could be doing something really dumb but I've tried just copy and pasting the code in the docs into the playroom provided by yourselves and it throws up the same error.
Any help is greatly appreciated.

Hi, can you please send me your usage of Icon & also your theme config? Thanks!

Oh, sorry, you experienced that in playroom. That’s probably happening as icons aren’t imported into playroom :(

Hi Jxom, apologies in advance if I'm just being thick, but this is what I've got. I'm using Gatsbyjs and trying to implement 'fannypack' (which has a slightly different meaning in english but hey ho 👯 ).
import { ThemeProvider } from 'fannypack';
import { faPen, faShare, faFileSignature, faIgloo, faSearch, faClipboard, faThumbsUp } from '@fortawesome/free-solid-svg-icons';
import { faAddressBook } from '@fortawesome/free-regular-svg-icons';
const theme = {
  Icon: {
    iconSets: [
      {
        icons: [faPen, faShare, faFileSignature, faIgloo, faSearch, faClipboard, faThumbsUp],
        prefix: 'solid-',
        type: 'font-awesome'
      },
      {
        icons: [faAddressBook],
        prefix: 'regular-',
        type: 'font-awesome'
      }
    ]
  }
};

const IndexPage = () => (
  <ThemeProvider theme={theme}>
    <Tabs.Container>
      {tabs => (
        <Tabs>
          <Tabs.Tab tab="documents" {...tabs}>
            <Icon icon="solid-file" marginRight="major-1" /> Documents
          </Tabs.Tab>
          <Tabs.Tab tab="messages" {...tabs}>
            <Icon icon="solid-comments" marginRight="major-1" /> Messages
          </Tabs.Tab>
          <Tabs.Tab tab="profile" {...tabs}>
            <Icon icon="solid-user" marginRight="major-1" /> Profile
          </Tabs.Tab>
        </Tabs>
      )}
    </Tabs.Container>
  </ThemeProvider>
);

Hi, it seems like you aren't importing the specific icons you are trying to use. Try importing faUser, faFile and faComments as well from @fortawesome/free-solid-svg-icons, and adding it to the icon set.
e.g.
import {faPen, faShare, faFileSignature, faIgloo, faSearch, faClipboard, faThumbsUp, faUser, faFile, faComments} from '@fortawesome/free-solid-svg-icons';

and then to add to the icon set:
const theme = {
  Icon: {
    iconSets: [
      {
        icons: [faPen, faShare, faFileSignature, faIgloo, faSearch, faClipboard, faThumbsUp, faUser, faFile, faComments],
        prefix: 'solid-',
        type: 'font-awesome'
      },
      {
        icons: [faAddressBook],
        prefix: 'regular-',
        type: 'font-awesome'
      }
    ]
  }
};

Hey @Cigoler, @thepenskefile's solution should work.
Also, "Fannypack" used to be called "Bumbag", however, we renamed to Fannypack as it was a more well-known term used globally, and assumed people who are used to "Bumbag" would know that a "Fannypack" is the same thing. In Australia, we call it a "Bumbag".

That did the trick guys, thanks. Sorry for bothering you with a silly error on my part, I jumped to conclusions with it not working in the playground.
@jxom Although I am aware of the relation between the two, being British and hearing 'fanny' is still amusing to me. Please forgive my childish sense of humour 💃


Nina 4.7.3 - Open views not working

Hi,
I get the error attached with Open Views

Thank you.

Hey Outis, the command is working again, to install the tool again, first unistall the one you have using "Add or Remove programs" from windows. It should delete the folder Nina.bundle located in C:\ProgramData\Autodesk\ApplicationPlugins.
Then, you can run the installer again. You can find the latest version here.
https://www.ninabim.com/
Have a great day.


Update mutation not erroring or actually saving

I'm having some trouble, most likely due to incorrect use (again). I'm neither receiving an error or seeing the effect take place. I have a struct that looks like so:
// Model for updating a user
#[derive(Serialize, Deserialize, Default, Debug)]
pub struct UserUpdate {
    pub uid: String,
    pub username: Option<String>,
    pub password: Option<String>,
    pub email: Option<String>,
    pub updated_at: Option<DateTime<Utc>>
}

I then have an endpoint that receives a request an updates the user conditional on if values exist in the payload:
let req = request.into_inner();
let mut user = UserUpdate {
    uid: usr.unwrap().uid.to_string(),
    username: None,
    password: None,
    email: None,
    updated_at: Some(Utc::now())
};
// Update values if supplied to the endpoint
if req.username.to_string() != "" {
    user.username = Some(req.username.to_string());
}
if req.email.to_string() != "" {
    user.email = Some(req.email.to_string());
}
if req.password.to_string() != "" {
    user.password = Some(req.password.to_string());
}
// Submit the record to upgrade
let mut txn = self.graph.new_txn();
let mut mututation = dgraph::Mutation::new();
mututation.set_set_json(serde_json::to_vec(&user).expect("Invalid JSON"));
// Finalize the commit
let res = txn.commit();
if res.is_err() {
    return Err(Status::internal("INTERNAL_ERROR"));
}

There might be a better way to structure this but it's the only way I could think of structuring it. When I run a request there's no error, the result comes back successful but my record isn't modified at all. I'm passing in the correct uid and everything.
Wondering where I went wrong :-/ if this doesn't work I was going to just run an Operation with the syntax for the mutation written out.

You forgot to call txn.mutate(mutation).expect("...") before commiting it. The current code commited an empty transaction.
A sidenote: You don't need to convert &str to String if you are just comparing strings.

....wow. Sorry for posting this. You're absolutely right. I missed it. Apologies!


support HTTP endpoint

Support HTTP endpoint, TODOs:

[ ] ~HTTP reader unit test~
[x] pull request to add http reader to parquet-go-source
[x] USAGE.md

I'm going to leave out unit test as I hope parquet-go-source will take the http reader.


How can I write a rule to rise an alert when conditions met on 2 indice's?

Hi,
Could someone give me a sample for multi indeices alert in one rule.
Thank you!


flake8 flags "W503 line break before binary operator" rule violations

Describe the problem
My clone sees flake8 pre-commit hook fail on these files:

mdformat/src/mdformat/renderer/_util.py
mdformat/src/mdformat/_conf.py
mdformat/src/mdformat/_cli.py
mdformat/tests/test_cli.py

black doesn't seem to care about this, and neither does pylint (which flags other violations - see context, below).
I'm not quite sure what's going on, is this reproducible on someone else's system?
Link to your repository or website
No response
Steps to reproduce
$ git clone git@github.com:executablebooks/mdformat.git
Cloning into 'mdformat'...
remote: Enumerating objects: 2159, done.
remote: Counting objects: 100% (846/846), done.
remote: Compressing objects: 100% (374/374), done.
remote: Total 2159 (delta 563), reused 641 (delta 439), pack-reused 1313
Receiving objects: 100% (2159/2159), 586.12 KiB | 369.00 KiB/s, done.
Resolving deltas: 100% (1476/1476), done.

$ pre-commit run --all-files
Check Yaml...............................................................Passed
Check Toml...............................................................Passed
type annotations not comments............................................Passed
check blanket noqa.......................................................Passed
check blanket type ignore................................................Passed
Strip unnecessary `# noqa`s..............................................Passed
isort....................................................................Passed
black....................................................................Passed
docformatter.............................................................Passed
flake8...................................................................Failed
- hook id: flake8
- exit code: 1

src/mdformat/_cli.py:87:17: W503 line break before binary operator
src/mdformat/_cli.py:88:17: W503 line break before binary operator
src/mdformat/_cli.py:90:21: W503 line break before binary operator
src/mdformat/renderer/_util.py:20:5: W503 line break before binary operator
src/mdformat/renderer/_util.py:21:5: W503 line break before binary operator
src/mdformat/renderer/_util.py:22:5: W503 line break before binary operator
src/mdformat/renderer/_util.py:23:5: W503 line break before binary operator
src/mdformat/renderer/_util.py:24:5: W503 line break before binary operator
src/mdformat/renderer/_util.py:25:5: W503 line break before binary operator
src/mdformat/renderer/_util.py:70:9: W503 line break before binary operator
src/mdformat/renderer/_util.py:71:9: W503 line break before binary operator
src/mdformat/renderer/_util.py:130:13: W503 line break before binary operator
src/mdformat/renderer/_util.py:153:9: W503 line break before binary operator
src/mdformat/renderer/_util.py:154:9: W503 line break before binary operator
src/mdformat/renderer/_util.py:167:13: W503 line break before binary operator
src/mdformat/_conf.py:52:13: W503 line break before binary operator
tests/test_plugins.py:206:9: W503 line break before binary operator
tests/test_cli.py:164:9: W503 line break before binary operator
tests/test_cli.py:189:9: W503 line break before binary operator

The version of Python you're using
Python 3.7.3
Your operating system
macOS
Versions of your packages
No response
Additional context
pylint seems happier with black's formatting, only raising C0114 and C0116 docstring rule violations:
$ pylint src/mdformat/renderer/_util.py
************* Module mdformat.renderer._util
src/mdformat/renderer/_util.py:1:0: C0114: Missing module docstring (missing-module-docstring)
src/mdformat/renderer/_util.py:29:0: C0116: Missing function or method docstring (missing-function-docstring)
src/mdformat/renderer/_util.py:44:0: C0116: Missing function or method docstring (missing-function-docstring)
src/mdformat/renderer/_util.py:56:8: C0103: Variable name "c" doesn't conform to snake_case naming style (invalid-name)
src/mdformat/renderer/_util.py:66:0: C0116: Missing function or method docstring (missing-function-docstring)
src/mdformat/renderer/_util.py:84:0: C0116: Missing function or method docstring (missing-function-docstring)

-----------------------------------
Your code has been rated at 9.44/10

The W503 check is disabled by default in flake8, so it seems that you're reading a flake8 configuration from somewhere outside this repository that enables that check.
Regarding the steps to reproduce: you're missing
cd mdformat/

after the git clone. Maybe you're running pre-commit in the wrong directory?
Regarding pyling: we don't use that tool in mdformat.

I think there's some problem with Python on my system which is interfering with the default configuration 😦. Yes, I missed one step; not worth fixing now; pre-commit was run from the repository root directory. This problem appears to be restricted to my systems, so closing.

FWIW, this was caused by some particularly capricious behaviour on the part of flake8 which was hard to track down: exclude and ignore lines in ~/.config/flake8 have the pernicious side-effect of silently including and enabling rules. and the flake8 documentation doesn't warn about this. Avoiding silliness like this was one of the reasons I switched, more than 2 years ago, from flake8 to pylint https://pylint.org/. We should consider switching in mdformat.

I much prefer flake8 to pylint. You should use extend-exclude and extend-ignore to avoid overriding the defaults.

I much prefer tools that are not prone to unaccountable changes in behaviour. Yes, I know about extend-exclude and extend-ignore now, but it cost me more than a few hours of frustration trying to find where the rule had been explicitly enabled (even you expected this) — it hadn't.


Use lodash v4

Sending a PR on behalf of @jdalton :-)
Is there anything else that needs to be done here?

[x] Use Lodash 4 (@jdalton)
[ ] Actually wait for Lodash 4 to be released
[ ] Update “no dependencies” notice in README with something sensible
[ ] When React Native 0.10 is out, drop __DEV__ completely and depend on process.env.NODE_ENV being polyfilled again

We are actually waiting for Lodash 4 to be released, making second task done! :stuck_out_tongue_winking_eye:

Thanks for getting this rolling @gaearon!
I'll add some comments inline to talk through some of the changes.

@gaearon

[ ] Update “no dependencies” notice in README with something sensible

There's no need to update the notice since lodash methods will be built-in so still no external deps required.

Fair enough! I guess technically it looks like an (inner) dependency, but I'll just point people to this thread. :-)

Instructions to build lodash v4.

Build lodash dep for Redux

cd redux
mkdir node_modules && cd node_modules
node ../../lodash-cli/bin/lodash modularize exports=node -o ./lodash && node ../../lodash-cli/bin/lodash -d -o ./lodash/index.js
cd lodash && npm init

<go through wizard and ensure version is 4.0.0>

cd ../../ && npm i

Just to confirm.. we're still waiting for Lodash 4 to land right?

Correct.

Got it, thanks!

Lodash v4 is out :-)
https://github.com/lodash/lodash/commit/9a2748c1ead0308557a35ac47ab0b0f2f62db74e

Lodash v4 is out :-)

Not yet, but close. It'll be published and tagged on Tues (Jan 12th)

Lodash 4: https://github.com/lodash/lodash/releases/tag/4.0.0

Would somebody like to revive this?

I'll take it up this weekend for sure.

Oh man, thank you. No rush.

Sorry, haven' gotten to this. Though I think it should wait to 4.0.1 for the release bugs to be shaken out.

4.0.1 is out. Are we good to continue, do you think?

Yep, nows a good time with v4.1 out the private packages are flat too (not hidden away in a directory).
So like 'lodash/_toFunction'.

FYI I think we might not need anything else except isPlainObject: https://github.com/rackt/redux/pull/1329

Updated.

With this in place it'll be trivial to use another lodash method when needed.
If needed later the webpack.NormalModuleReplacementPlugin was something like:
  plugins: [
    ['lodash/_baseIteratee.js', './_toFunction.js'],
    ['lodash/keys.js',           './_baseKeys.js'],
    ['lodash/keysIn.js',         './_baseKeysIn.js']
  ].map(function(pair) {
    var resourceRegExp = RegExp('\\b' + _.escapeRegExp(pair[0]) + '$');
    return new webpack.NormalModuleReplacementPlugin(resourceRegExp, pair[1]);
  }).concat(
    new webpack.optimize.OccurenceOrderPlugin()
  )

Just so it's known (no judgement)
master
redux.min.js  5.73 kB

lodash
redux.min.js  5.85 kB

I'd rather spend 130 bytes to not have to maintain a standard function not critical to Redux. And if someone is already bundling in Lodash, then the net result is actually going to be negative for them.
I don't see any big issues with this PR. LGTM.

What is the post gzip difference?
(Just curious. I agree it's not significant.)

2062 vs 2153 bytes, so 91 bytes.

dist git(master):
2.05 kB
dist git(lodash): gzip-size redux.min.js| pretty-bytes
2.14 kB

:+1:

Interestingly, I have trouble trying to run tests on the UMD builds. If I change
--- a/test/createStore.spec.js
+++ b/test/createStore.spec.js
@@ -1,5 +1,5 @@
 import expect from 'expect'
-import { createStore, combineReducers } from '../src/index'
+import { createStore, combineReducers } from '../dist/redux'

I get
/Users/dan/p/redux/dist/redux.js:411
				var objectProto = global.Object.prototype;
				                        ^

TypeError: Cannot read property 'Object' of undefined
    at Object.<anonymous> (/Users/dan/p/redux/dist/redux.js:405:26)
    at Object.exports.__esModule (/Users/dan/p/redux/dist/redux.js:467:31)

This is new with this PR. However I'm also able to run require('./redux') from node in dist folder so I'm confused what actually broke.

In tests, Mocha transpiles everything that isn't in node_modules. Previously Redux worked even when transpiled twice so I could manually test dist builds with Mocha. However after this PR it doesn't work when transpiled twice. In particular, 'use strict'; inserted by Babel breaks function () {return this;} check that Lodash does to find the global.
In general we don't want to encourage people to transpile Redux UMD build, but I wouldn't be surprised if some people do that. Feels like breaking them isn't a very nice thing to do.

@gaearon do you think this belongs over in the lodash issue tracker?  Something like "lodash breaks when transpiled twice"?  If so, would you mind opening it over there for tracking?  I know JD is keeping an eye on this thread but I figured it would be helpful.  Thanks.

Filed as https://github.com/lodash/lodash/issues/1916

If we only need a few methods, and if those methods are working fine within Redux, why do we need depend on another library which may cause troubles?

Because if we can avoid maintaining them, we want to.

I just force-pushed to master to not include this PR. Let's figure out a solution that doesn't break double transpiled builds. If we can't, I'm happy to bump major and ship, but that would be a breaking change.

@cht8687

if those methods are working fine within Redux, why do we need depend on another library which may cause troubles?

It turns out DIY solutions were a burden on the project and caused support issues. Problems that were solved by lodash since it does utilities very well. Since lodash is a widely used utility libs it has more eyes on it and resolves more issues making it a solid choice when choosing whether to DIY or use lodash. Redux using lodash further feeds lodash's support chain as any issues it has in the future will pipe directly to lodash improving it for all other users.
@gaearon

I just force-pushed to master to not include this PR. Let's figure out a solution that doesn't break double transpiled builds. If we can't, I'm happy to bump major and ship, but that would be a breaking change.

Can you describe the issue a bit more. The unit tests passed in this PR. If you liked it, then you shoulda put a unit test on it – Beyonce (probably)

Can you describe the issue a bit more. The unit tests passed in this PR. If you liked it, then you shoulda put a unit test on it – Beyonce (probably)

Yeah, so this is something I tend to test manually because I don't have a good solution for doing that automatically. (I agree I should create an issue for that.)
I just changed imports inside tests to point to ../dist/redux instead of ../src to make sure that the UMD build didn't regress. The UMD builds themselves are fine, but I found another issue I described in https://github.com/lodash/lodash/issues/1916.
People are supposed to use UMD builds “as they are” but I know some people have bad transpiler configs that transpile third party modules. So we end up in a situation where the UMD build of Redux might be transpiled with Babel by somebody else, and it used to work until now. I'm not saying I want to support this (I'm fine to break it in a major and declare we don't want people to transpile our already transpiled UMD builds) but it used to work. However now, due to Babel inserting use strict into modules it transpiles, Lodash breaks because it depends on non-strict behavior. At least this is my understanding of the issue.

 UMD build still works despite being transpiled “again” (behavior in master)
Actual:
3. UMD build is broken due to crash in Lodash (behavior in this PR)

Let's continue discussion in https://github.com/lodash/lodash/issues/1916

@jdalton
Thank you very much for helping me understand this issue and how global detection works in Lodash!
I wonder if there is any way you can publish an official version of


Update Windows Local VM version

Description
This PR is going to update the current Windows Local VM version downloadable from Microsoft Developer portal.
Downloadable Windows version for Azure VM is still set to 10.
Why
Windows 10 for Local VM is no more downloadable from that source (check here)
Microsoft Reviewers: Open in CodeFlow

thanks @mattia-sanfilippo !


v1.3.0

Coverage remained the same at 81.893% when pulling ee88b5bf77d31671d88e8e43c4e29f4b5c417d90 on develop into c1bf9cd4fa7cdbf927cc5789fa51d5ba73fb01d7 on master.

Coverage remained the same at 81.893% when pulling 6d35cc0717210ebf6b8ef122351007a9644cdf93 on develop into c1bf9cd4fa7cdbf927cc5789fa51d5ba73fb01d7 on master.


Recommended local dev environment

We briefly discussed options around local dev stacks, just gathering a new notes/thoughts here to expand on.
None of these will be required, but, recommendations/guidelines might help folk get up to speed.
There are a few use cases for local environments, which might have different solutions.

Local dev environment for ortelius service development
This is not necessarily requiring a full environment, and could comprise of simple containers or if contracts between services are well defined, isolated service being worked on with contracts to validate against.

Local dev environment for validating deployment/integration
Depending on the recommended deployment approach, this will likely need some form of k8s + deployment tooling. Potentially something along the lines of kind/k3s + skaffold. Cross platform is a must.

Local dev environment for integrations with external tooling
This will depend on the external tooling ( saas vs self hosted with monolithic app vs self hosted with helm deployment ).

Local dev is likely to be a mix of the previous two considerations depending on the tool being integrated with.

A couple of other notes, it's worth including information around how to debug against the recommended local dev stack. This can be a stumbling block for folk particularly when it comes to debugging services against k8s. Could make for some good blog posts off the back of it as well.

Need to make sure we handle a dev env for a least Windows and OS/X.

Closing. See Ortelius-In-A-Box by Sacha


grok_exporter exits when monitored file is moved away

We run grok_exporter with the following path config (sensitive samples removed):
input:
  type: file
  fail_on_missing_logfile: false
  paths:
  - "/path/to/logs/*.log"

Regularly, after certain events, a subdir is created in the logs directory, and all log files in the monitored directory are moved to that subdir. Sometimes at this point grok_exporter would exit, with RC=-1 reporting this problem to syslog:
Aug 20 08:28:09 hostname grok_exporter[10133]: error reading log lines: watch list inconsistent: received a file system event for an unknown directory
Aug 20 08:28:09 hostname systemd[1]: grok-exporter.service: main process exited, code=exited, status=255/n/a

Looks like grok_exporter is receiving an inotify with unexpected path, which triggers immediate exit.
Such behavior is very undesired, because exporter drops all the collected state and jeopardizes stats collected and reported to monitoring.
Is there a workaround to it?
Could you please fix grok_exporter to silently ignore and/or drop the file from watchlist, when receiving inotify from unknown dir, instead of jumping outside of window?

This should not happen.
Before I add a fix ignoring the event, I'd like to try to understand what causes this. Do you see any way how I could reproduce this?
I assume you are running on Linux, as you mention inotify.
I tried creating a subdirectory and moving the logs to the subdirectory a couple of times, but the error didn't occur. Is there any way I can reproduce this on my machine?

Could not reproduce it reliably in lab conditions, but it happens consistently under certain circumstances:

This is Oracle Cloud (OCI) mounted volume, not physical HW. Don't know the exact details, the FS is ext4.
The amount of log files watched is approx 500. On lesser loads, this never happened.
Logs are moved to a subdir by use of the following command:
 find ..path../logs/ -maxdepth 1 -type f -exec mv -n -t ..same..path../logs/subdir  {} +"
And each . and the subdir have exact same owner and group and exact same 755 perms.

Funny thing is, I saw the exact same error before, executing similar scenario on NFS volume. Then I simply attributed the problem to NFS and pushed to have logs stored "locally". Now, I see the same problem on ext4.
I am willing to help with chasing down the problem, please ask for details, I'll try to provide answers.

Set-up for reproducing is up and running, but so far the error didn't reproduce. I'll keep it running for a few days.
Meanwhile I'll look into not terminating grok_exporter when the error occurs.

Thank you for your efforts. I'd not be so optimistic about reproducing the issue, though.
Anyways, looking forward for the fix. Re-syncing the dir contents upon receiving such error would be ideal, but simply ignoring it would do.

Is there any chance you can build grok_exporter from source and run this in your environment? I added an improved error message on the master branch to figure out which file system event causes the issue. I'm hoping that if we can figure out which event causes the issue, we can handle that specific event.

The reproducer should have fired by now. Something in the scenario must be missing.
We are running lots of ephemeral environments, so there is no telling which one reproduces this problem, if at all. I could analyze the logs once I have spare time, and introduce the master version to the most probable location.
Don't know how to put it, but. If the scenario and the problem fall into "it should not happen"category, then it would probably be safe to ignore it. Could you just do the logging and the ignore parts? I hate being responsible for one heisenbug that I can't even explain, to both sides :)

Hi!
I have the same issue. In the grok_exporter log I see
error reading log lines: watch list inconsistent: cannot find directory for file system event "<nil>" with watch descriptor -1 in the list of watched directories [Wd=1 ("/logs/")]

Hello, we have same problem too, but in 3 nodes with same config(all the same), only one reproduce this error:
Jul 29 13:44:42 example-srv grok_exporter[3985517]: error reading log lines: watch list inconsistent: cannot find directory for file system event "<nil>: " with watch descriptor -1 in the list of watched directories [Wd=1 ("/opt/myapp/var/log")]
Jul 29 13:44:42 example-srv systemd[1]: grok_exporter.service: Main process exited, code=exited, status=255/EXCEPTION
Jul 29 13:44:42 example-srv systemd[1]: grok_exporter.service: Failed with result 'exit-code'.
Jul 29 13:44:42 example-srv systemd[1]: grok_exporter.service: Consumed 3min 20.565s CPU time.
Jul 29 13:44:42 example-srv systemd[1]: grok_exporter.service: Scheduled restart job, restart counter is at 2.
Jul 29 13:44:42 example-srv systemd[1]: Stopped Prometheus Grok Exporter.
Jul 29 13:44:42 example-srv systemd[1]: grok_exporter.service: Consumed 3min 20.565s CPU time.
Jul 29 13:44:42 example-srv systemd[1]: Started Prometheus Grok Exporter.
Jul 29 13:44:42 example-srv grok_exporter[3993402]: Starting server on http://example-srv:9144/metrics
Jul 29 13:44:43 example-srv grok_exporter[3993402]: error reading log lines: watch list inconsistent: cannot find directory for file system event "<nil>: " with watch descriptor -1 in the list of watched directories [Wd=1 ("/opt/myapp/var/log")]
Jul 29 13:44:43 example-srv systemd[1]: grok_exporter.service: Main process exited, code=exited, status=255/EXCEPTION
Jul 29 13:44:43 example-srv systemd[1]: grok_exporter.service: Failed with result 'exit-code'.
Jul 29 13:44:43 example-srv systemd[1]: grok_exporter.service: Consumed 3.436s CPU time.
Jul 29 13:44:43 example-srv systemd[1]: grok_exporter.service: Scheduled restart job, restart counter is at 3.
Jul 29 13:44:43 example-srv systemd[1]: Stopped Prometheus Grok Exporter.
Jul 29 13:44:43 example-srv systemd[1]: grok_exporter.service: Consumed 3.436s CPU time.
Jul 29 13:44:43 example-srv systemd[1]: Started Prometheus Grok Exporter.
Jul 29 13:44:43 example-srv grok_exporter[3993425]: Starting server on http://example-srv:9144/metrics
Jul 29 13:44:44 example-srv grok_exporter[3993425]: error reading log lines: watch list inconsistent: cannot find directory for file system event "<nil>: " with watch descriptor -1 in the list of watched directories [Wd=1 ("/opt/myapp/var/log")]
Jul 29 13:44:44 example-srv systemd[1]: grok_exporter.service: Main process exited, code=exited, status=255/EXCEPTION
Jul 29 13:44:44 example-srv systemd[1]: grok_exporter.service: Failed with result 'exit-code'.
Jul 29 13:44:44 example-srv systemd[1]: grok_exporter.service: Consumed 1.884s CPU time.
Jul 29 13:44:44 example-srv systemd[1]: grok_exporter.service: Scheduled restart job, restart counter is at 4.
Jul 29 13:44:44 example-srv systemd[1]: Stopped Prometheus Grok Exporter.
Jul 29 13:44:44 example-srv systemd[1]: grok_exporter.service: Consumed 1.884s CPU time.
Jul 29 13:44:44 example-srv systemd[1]: Started Prometheus Grok Exporter.
Jul 29 13:44:45 example-srv systemd[1]: grok_exporter.service: Consumed 1.695s CPU time.
Jul 29 13:44:45 example-srv systemd[1]: grok_exporter.service: Scheduled restart job, restart counter is at 5.
Jul 29 13:44:45 example-srv systemd[1]: Stopped Prometheus Grok Exporter.
Jul 29 13:44:45 example-srv systemd[1]: grok_exporter.service: Consumed 1.695s CPU time.