Kong Ingress Controller cannot scale beyond a limit

Current Behavior
Scenario:
Create 1500 Secrets in a namespace with approximately 1MB data in each of the Secret.
This was done to reproduce an issue that we faced in production where we had thousands of Secrets, with a cumulative data size of 1.8 GB. This test we do simulates this real-world scenario with ~1.5GB in the cluster.
Expected Behavior
Kong restarts should work fine, both the proxy and the ingress controller should come up and stay in Running state.
Steps To Reproduce

Create Secrets totalling ~1.5 GB of data, preferably in the same namespace where Kong is running. (I have a go module that does this, let me know if that helps and I will try to make it available)
Restart Kong.
Observe that Kong goes into an infinite CrashLoopBackoff

Kong Ingress Controller version
2.1.1, but should exist in main too. Going through the code makes me think that this is embedded deep inside the sig.k8s.io module which doesn't handle pagination effectively. Kong is running in DB-less mode.
Kubernetes version
$ kubectl --context test version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.15", GitCommit:"58178e7f7aab455bc8de88d3bdd314b64141e7ee", GitTreeState:"clean", BuildDate:"2021-09-15T19:23:02Z", GoVersion:"go1.15.15", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"19+", GitVersion:"v1.19.15-eks-9c63c4", GitCommit:"9c63c4037a56f9cad887ee76d55142abd4155179", GitTreeState:"clean", BuildDate:"2021-10-20T00:21:03Z", GoVersion:"go1.15.15", Compiler:"gc", Platform:"linux/amd64"}

Anything else?
Discussion in https://discuss.konghq.com/t/kong-ingress-crontroller-in-dbless-mode-error-failed-to-wait-for-secret-caches-to-sync-timed-out-waiting-for-cache-to-be-synced/9923.
Might be related to https://github.com/kubernetes/kubernetes/issues/108003.

Thank you for your report.
Could you provide the events and logs from the ingress controller deployment and pods for the crash loop to show the cause? Is it oomkiller?
Also in version v2.0.0+ we added the --watch-namespaces option which allows you to select specific namespaces to watch for [Ingress and other relevant] resources rather than watching all namespaces. It sounded like most of these secrets were not Kong relevant secrets (e.g. Credentials for Consumers) so I wanted to check whether using this to filter out namespaces with bulk Secrets might have a mitigating effect for you while we otherwise look for a long term fix?

Logs are as in the discussion link:
time="2022-03-08T17:23:57Z" level=debug msg="no configuration change, skipping sync to kong" subsystem=proxy-cache-resolver
I0308 17:23:57.491252       1 trace.go:205] Trace[1752425436]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.23.0-alpha.1/tools/cache/reflector.go:167 (08-Mar-2022 17:22:57.489) (total time: 60001ms):
Trace[1752425436]: [1m0.001703338s] [1m0.001703338s] END
E0308 17:23:57.491272       1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.23.0-alpha.1/tools/cache/reflector.go:167: Failed to watch *v1.Secret: failed to list *v1.Secret: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 197; INTERNAL_ERROR; received from peer
time="2022-03-08T17:24:00Z" level=debug msg="no configuration change, skipping sync to kong" subsystem=proxy-cache-resolver
...
...
...
time="2022-03-08T17:24:57Z" level=debug msg="no configuration change, skipping sync to kong" subsystem=proxy-cache-resolver
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for secret caches to sync: timed out waiting for cache to be synced" logger=controller.secret reconciler group= reconciler kind=Secret
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for kongplugin caches to sync: timed out waiting for cache to be synced" logger=controller.kongplugin reconciler group=configuration.konghq.com reconciler kind=KongPlugin
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for kongconsumer caches to sync: timed out waiting for cache to be synced" logger=controller.kongconsumer reconciler group=configuration.konghq.com reconciler kind=KongConsumer
time="2022-03-08T17:24:57Z" level=info msg="context done: shutting down the proxy update server" subsystem=proxy-cache-resolver
time="2022-03-08T17:24:57Z" level=error msg="context completed with error" error="context canceled" subsystem=proxy-cache-resolver
I0308 17:24:57.489210       1 trace.go:205] Trace[1371614229]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.23.0-alpha.1/tools/cache/reflector.go:167 (08-Mar-2022 17:23:58.818) (total time: 58670ms):
Trace[1371614229]: [58.670670788s] [58.670670788s] END
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for ingress caches to sync: timed out waiting for cache to be synced" logger=controller.ingress reconciler group=networking.k8s.io reconciler kind=Ingress
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for service caches to sync: timed out waiting for cache to be synced" logger=controller.service reconciler group= reconciler kind=Service
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for tcpingress caches to sync: timed out waiting for cache to be synced" logger=controller.tcpingress reconciler group=configuration.konghq.com reconciler kind=TCPIngress
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for endpoints caches to sync: timed out waiting for cache to be synced" logger=controller.endpoints reconciler group= reconciler kind=Endpoints
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for kongclusterplugin caches to sync: timed out waiting for cache to be synced" logger=controller.kongclusterplugin reconciler group=configuration.konghq.com reconciler kind=KongClusterPlugin
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for kongconsumer caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for udpingress caches to sync: timed out waiting for cache to be synced" logger=controller.udpingress reconciler group=configuration.konghq.com reconciler kind=UDPIngress
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for ingress caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="Could not wait for Cache to sync" error="failed to wait for kongingress caches to sync: timed out waiting for cache to be synced" logger=controller.kongingress reconciler group=configuration.konghq.com reconciler kind=KongIngress
time="2022-03-08T17:24:57Z" level=info msg="stop status update channel."
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for service caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for tcpingress caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for endpoints caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for kongclusterplugin caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for udpingress caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for kongingress caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="failed to wait for kongplugin caches to sync: timed out waiting for cache to be synced"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="context canceled"
time="2022-03-08T17:24:57Z" level=error msg="error received after stop sequence was engaged" error="context canceled"
Error: failed to wait for secret caches to sync: timed out waiting for cache to be synced
Error: failed to wait for secret caches to sync: timed out waiting for cache to be synced

It is not an OOM, although if I keep adding Secrets, it sometimes goes to OOM.
I have already evaluated --watch-namespaces option and it is unfortunately not an option for us as we need to watch all namespaces in the cluster.

@shaneutt Can we get a quick triage from the Kong team? I am already working on a fix, but have a hard time understanding the code flow, esp in client-go wrt SharedIndexInformer.

We may not be able to provide fully synchronous help at this moment, but if you have specific questions about the code please feel free to ask them here, OR we can continue in the Kubernetes Slack thread where we originally started talking about this: https://kubernetes.slack.com/archives/CDCA87FRD/p1648732270879879

tl;dr I think you may need to add a new feature to client-go to control the page size of the controllers created by SharedInformers. What was the change mentioned here?

The default pagination in client-go to use the pager module is set to 500, I have reduced it to 100 and recompiled and tested. The results are the same and there is no improvement.

I'm not sure if that's the same thing I found or if it'd override it further upstream and have the same effect. You should be able to hard-code WatchListPageSize: <whatever> into the SharedInformer Run() to see if it works instead.
As far as the code, we don't manage the client-go instance that interacts with the Kubernetes API directly. We configure a controller-runtime Controller and let it handle the client operations/feed us the result. That's created in controller setup (from a code generator).
That Watch() adds a Kind source, which runs its Start() method when the controller manager starts. It waits for its cache to sync by calling the related client-go function. This just loops over a bunch of provided sync functions to confirm they're returning true. These functions are SharedInformer HasSynced() functions of each of the Kind sources.
That top-level HasSynced(), as I follow it, calls down through a few other levels of other HasSynced() functions before ultimately reaching the controller FIFO queue HasSynced(), though I didn't review that enough to explain the condition accurately.
Anyway, ignoring what that actually does, I noticed there's a WatchListPageSize option for cache.Controllers which controls the ListAndWatch() pager page size, but controller-runtime never calls cache.New(). It does call the informers Run() method though, and that creates a new controller. Sadly, the controller config is built from informer parameters in Run(), and informers don't provide a way to populate WatchListPageSize.
Afterword:
Increasing the Controller CacheSyncTimeout may help--not quite sure, it may just allow it to be broken longer.
Are the large Secrets content KIC is indeed supposed to ingest? I'd expect not, since individual credentials shouldn't be 1MB each, and I'm unsure how many realistically-sized credentials you'd need to hit that threshold. Our credentials implementation has a design flaw, where we use a kongCredType key to identify Secrets we should ingest, but should have instead used labels, as we can filter watches on those. However, changing to a label-based system is a breaking change, and we haven't planned a migration to that--this is more indication that we probably should.

As @shaneutt had requested in the Slack chat, here is a stack trace at the time the error happens:
E0401 08:43:07.341118       1 runtime.go:78] Observed a panic: &fmt.wrapError{msg:"stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 191; INTERNAL_ERROR; received from peer", err:http2.StreamError{StreamID:0xbf, Code:0x2, Cause:(*errors.errorString)(0xc00011fa60)}} (stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 191; INTERNAL_ERROR; received from peer)
goroutine 563 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic({0x1659e00, 0xc0004d6080})
	/workspace/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:74 +0x85
k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0x2540be400})
	/workspace/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:48 +0x75
panic({0x1659e00, 0xc0004d6080})
	/usr/local/go/src/runtime/panic.go:1038 +0x215
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch.func1(0xc000a0e000, 0xc000c99200, 0xc00031af60, 0xc000c82b00)
	/workspace/vendor/k8s.io/client-go/tools/cache/reflector.go:331 +0x94e
k8s.io/client-go/tools/cache.(*Reflector).ListAndWatch(0xc000a0e000, 0xc00031af60)
	/workspace/vendor/k8s.io/client-go/tools/cache/reflector.go:374 +0x345
k8s.io/client-go/tools/cache.(*Reflector).Run.func1()
	/workspace/vendor/k8s.io/client-go/tools/cache/reflector.go:223 +0x26
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x7f188d06a3b0)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x67
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0006d4190, {0x1a4aa40, 0xc000350ff0}, 0x1, 0xc00031af60)
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0xb6
k8s.io/client-go/tools/cache.(*Reflector).Run(0xc000a0e000, 0xc00031af60)
	/workspace/vendor/k8s.io/client-go/tools/cache/reflector.go:222 +0x237
k8s.io/apimachinery/pkg/util/wait.(*Group).StartWithChannel.func1()
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:56 +0x22
k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1()
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:73 +0x5a
created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start
	/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:71 +0x88
panic: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 191; INTERNAL_ERROR; received from peer [recovered]
	panic: stream error when reading response body, may be caused by closed connection. Please retry. Original error: stream error: stream ID 191; INTERNAL_ERROR; received from peer

Unfortunately, this is affecting our production and I cannot afford to spend more time on this right now. I will try to mitigate the risks by trying to refactor our code to avoid large number of Secrets (pull from S3 or soemthing as such). Anyway, it would be a long time to get an MR out, and then to get it merged in either way.
I will get back to this later if time permits.

severity/high is most commonly used for "stop the world and get this fixed" type things. You could also think of "severity" in this context as "impact". Given that you are the only one to have reported this problem in the last year and you're using a workaround/mitigation for it and the additional context is that --watch-namespace restriction and KubeAPI configuration tuning are potentially valid mitigations as well: the impact is not the same as say, a bug that would break production for every user in every configuration and there's no known mitigations. I'm sorry of the "medium" language seems off-putting, we would be open to re-assessing the language we use for our work organization if it's not friendly or clear enough.
As for assignment nobody is working on this right at this moment but we will be pulling it into a future sprint to be worked on by maintainers and will endeavor to do so quickly. If a non-maintainer contributor wants to take it instead in the meantime we welcome that. If you're a Kong customer we would recommend you report this issue to our support team for their awareness and aid.

Acceptance criteria for a fix

[ ] Ensure that pagination happens in the initial List() call that controller-runtime performs on startup of KIC
[ ] Load-test KIC with 1.5GB of secrets to validate the fix

@pmalek the updates from maintainers in https://github.com/kubernetes-sigs/controller-runtime/issues/532 so far seem to suggest that we've stumbled upon a more fundamental issue with controller-runtime that wont be fixed by adding pagination in the way we thought initially.
I'm starting to think we might want to take a slightly different approach with this one: The fact that the initial listing can be so large that it trips the default api server limits does not appear at all specific to Kong, and in fact seems like it should be a problem for all controller-runtime based implementations as a whole.
Perhaps we should try and create a reproduction environment: an example controller from scratch with kubebuilder or operator-sdk which would also perform this initial listing, create a kind cluster, deploy a huge number of objects to the cluster, and then deploy our example operator to the cluster to reproduce this issue outside of Kong context.
If we can provide a clear list of reproduction steps that doesn't require anything Kong we should be able to report this as a new and separate issue upstream and bring it up for consideration at the next community sync to get people's heads thinking on it. Let me know your thoughts?

Perhaps we should try and create a reproduction environment: an example controller from scratch with kubebuilder or operator-sdk which would also perform this initial listing, create a kind cluster, deploy a huge number of objects to the cluster, and then deploy our example operator to the cluster to reproduce this issue outside of Kong context.

I am already stealing time at work and on my way to:

Create a KinD cluster with custom configs: set apiserver request timeout to 10s
Deploy our custom services + Kong on them
Have a tool that can create multiple Secrets in parallel to trigger this issue

As mentioned before, I can open source the tool to create Secrets in parallel, but (as of now) cannot open source the KinD setup scripts as they are too detailed and try to replicate our cloud deployments to a large extent. But I can strip this down later if needed, and open source a subset of work. Wrt not testing with Kong, I can create a new operator, but would try to avoid the work and just try to repro things with Kong and try to see if we can do something.

@krish7919 PTAL at my comment https://github.com/kubernetes-sigs/controller-runtime/issues/532#issuecomment-1141132416
I believe there might be a bigger issue related to how a big number of secrets (Only? Or is this related to ConfigMaps as well?) with relatively big payloads (1MB in my experiments) are causing high CPU and memory usage on apiserver which (in turn) slows down the communication with the cluster.

I do not think it is a CPU or a memory issue, it is purely a timeout issue and the inefficient listing of Secrets. As I mentioned above, I can reproduce this issue at will now with a KinD cluster running locally with a request timeout set to 5s and by creating ~25-40 Secrets each with 1MB of data.
I am still trying to understand what the other issues are saying and why a direct etcd listing would be worse or why pagination wouldn't work. I am still thinking that it should be solved using pagination (as I solved a similar issue in Helm as mentioned in https://github.com/Kong/kubernetes-ingress-controller/issues/2382#issuecomment-1084899198 above)
Maybe my knowledge is limited here as to what Kong Ingress Controller does special here, and why it (or client-go) cannot use pagination.

Let us know if you're still planning on digging in here @krish7919 and any findings you uncover.

I am blocked on this as I cannot figure out the flow on the controller-runtime side of things as the comment above suggested. Pagination does not seem to work either.

Here's a summary of my analysis so far. As mentioned before, I had a hunch that this is a pagination issue and adding pagination on the client side will fix the issue. This doesn't seem to be the case.
The workflow that Kubernetes, from my understanding, is as follows:

The client sends a request for the secrets like so GET /api/v1/secrets?limit=500&resourceVersion=0.
However, for pagination to work, the apiserver reads all the secrets from etcd and then stores them in memory.
This full dump of Secret data is then paginated to the client.

What is actually happening here:

The client (here the Kong Ingress Controller) sends a request for the secrets like so GET /api/v1/secrets?limit=500&resourceVersion=0.
The apiserver cannot read the Secrets from etcd due to the scale issue. Cluster has ~1.5GB of Secret data. And KIC needs to read this at startup to keep its local cache store in sync with data in etcd.
Even though the client can specify pagination (with say a limit of 20), it is the apiserver which times out as it cannot read the data from etcd in the default timeout period; 60s by default in our production Kubernetes cluster (using AWS EKS), and 5s in my local tests using Kind.
This causes the KIC to go in a crash loop and never recover.

The stack trace from kube-apiserver is as follows:
2022-03-08 17:37:27.000,"E0308 17:37:27.074039      10 runtime.go:78] Observed a panic: &errors.errorString{s:""killing connection/stream because serving request timed out and response had been started""} (killing connection/stream because serving request timed out and response had been started)
goroutine 872445776 [running]:
k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime.logPanic(0x3f96260, 0xc000698a10)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:74 +0xa6
k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0xc012ef9c98, 0x1, 0x1)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:48 +0x89
panic(0x3f96260, 0xc000698a10)
	/usr/local/go/src/runtime/panic.go:969 +0x1b9
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).timeout(0xc0b8155120, 0xc01fc2fe00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:257 +0x1bc
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP(0xc01c7910a0, 0x5171da0, 0xc167a1e000, 0xc09bd44300)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:141 +0x2f3
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters.WithWaitGroup.func1(0x5171da0, 0xc167a1e000, 0xc09bd44200)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters/waitgroup.go:59 +0x137
net/http.HandlerFunc.ServeHTTP(0xc01c7955f0, 0x5171da0, 0xc167a1e000, 0xc09bd44200)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithRequestInfo.func1(0x5171da0, 0xc167a1e000, 0xc09bd44100)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters/requestinfo.go:39 +0x269
net/http.HandlerFunc.ServeHTTP(0xc01c795620, 0x5171da0, 0xc167a1e000, 0xc09bd44100)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuditAnnotations.func1(0x5171da0, 0xc167a1e000, 0xc09bd44000)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters/audit_annotations.go:37 +0x142
net/http.HandlerFunc.ServeHTTP(0xc01c7910c0, 0x5171da0, 0xc167a1e000, 0xc09bd44000)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithWarningRecorder.func1(0x5171da0, 0xc167a1e000, 0xc03bb7df00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters/warning.go:35 +0x1a7
net/http.HandlerFunc.ServeHTTP(0xc01c7910e0, 0x5171da0, 0xc167a1e000, 0xc03bb7df00)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithCacheControl.func1(0x5171da0, 0xc167a1e000, 0xc03bb7df00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters/cachecontrol.go:31 +0xa8
net/http.HandlerFunc.ServeHTTP(0xc01c791100, 0x5171da0, 0xc167a1e000, 0xc03bb7df00)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/httplog.WithLogging.func1(0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:91 +0x2f2
net/http.HandlerFunc.ServeHTTP(0xc01c791120, 0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters.withPanicRecovery.func1(0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters/wrap.go:51 +0xe6
net/http.HandlerFunc.ServeHTTP(0xc01c791140, 0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.(*APIServerHandler).ServeHTTP(0xc01c795650, 0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/handler.go:189 +0x51
net/http.serverHandler.ServeHTTP(0xc0087190a0, 0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.initALPNRequest.ServeHTTP(0x5177620, 0xc05cc33500, 0xc0f0096380, 0xc0087190a0, 0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/usr/local/go/src/net/http/server.go:3415 +0x8d
k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*serverConn).runHandler(0xc0a0e46c00, 0xc027936fd0, 0xc03bb7de00, 0xc0b8154fa0)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/golang.org/x/net/http2/server.go:2152 +0x8b
created by k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*serverConn).processHeaders
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/golang.org/x/net/http2/server.go:1882 +0x505"
2022-03-08 17:37:27.000,"E0308 17:37:27.074113      10 wrap.go:39] apiserver panic'd on GET /api/v1/secrets?limit=500&resourceVersion=0
http2: panic serving <IP_ADDRESS>:47184: killing connection/stream because serving request timed out and response had been started
goroutine 872445776 [running]:
k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*serverConn).runHandler.func1(0xc027936fd0, 0xc012ef9f8e, 0xc0a0e46c00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/golang.org/x/net/http2/server.go:2145 +0x16f
panic(0x3f96260, 0xc000698a10)
	/usr/local/go/src/runtime/panic.go:969 +0x1b9
k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0xc012ef9c98, 0x1, 0x1)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:55 +0x10c
panic(0x3f96260, 0xc000698a10)
	/usr/local/go/src/runtime/panic.go:969 +0x1b9
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).timeout(0xc0b8155120, 0xc01fc2fe00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:257 +0x1bc
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP(0xc01c7910a0, 0x5171da0, 0xc167a1e000, 0xc09bd44300)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:141 +0x2f3
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters.WithWaitGroup.func1(0x5171da0, 0xc167a1e000, 0xc09bd44200)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters/waitgroup.go:59 +0x137
net/http.HandlerFunc.ServeHTTP(0xc01c7955f0, 0x5171da0, 0xc167a1e000, 0xc09bd44200)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithRequestInfo.func1(0x5171da0, 0xc167a1e000, 0xc09bd44100)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters/requestinfo.go:39 +0x269
net/http.HandlerFunc.ServeHTTP(0xc01c795620, 0x5171da0, 0xc167a1e000, 0xc09bd44100)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuditAnnotations.func1(0x5171da0, 0xc167a1e000, 0xc09bd44000)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters/audit_annotations.go:37 +0x142
net/http.HandlerFunc.ServeHTTP(0xc01c7910c0, 0x5171da0, 0xc167a1e000, 0xc09bd44000)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithWarningRecorder.func1(0x5171da0, 0xc167a1e000, 0xc03bb7df00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters/warning.go:35 +0x1a7
net/http.HandlerFunc.ServeHTTP(0xc01c7910e0, 0x5171da0, 0xc167a1e000, 0xc03bb7df00)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithCacheControl.func1(0x5171da0, 0xc167a1e000, 0xc03bb7df00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/endpoints/filters/cachecontrol.go:31 +0xa8
net/http.HandlerFunc.ServeHTTP(0xc01c791100, 0x5171da0, 0xc167a1e000, 0xc03bb7df00)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/httplog.WithLogging.func1(0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:91 +0x2f2
net/http.HandlerFunc.ServeHTTP(0xc01c791120, 0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters.withPanicRecovery.func1(0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/filters/wrap.go:51 +0xe6
net/http.HandlerFunc.ServeHTTP(0xc01c791140, 0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/usr/local/go/src/net/http/server.go:2042 +0x44
k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server.(*APIServerHandler).ServeHTTP(0xc01c795650, 0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apiserver/pkg/server/handler.go:189 +0x51
net/http.serverHandler.ServeHTTP(0xc0087190a0, 0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/usr/local/go/src/net/http/server.go:2843 +0xa3
net/http.initALPNRequest.ServeHTTP(0x5177620, 0xc05cc33500, 0xc0f0096380, 0xc0087190a0, 0x5164460, 0xc027936fd0, 0xc03bb7de00)
	/usr/local/go/src/net/http/server.go:3415 +0x8d
k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*serverConn).runHandler(0xc0a0e46c00, 0xc027936fd0, 0xc03bb7de00, 0xc0b8154fa0)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/golang.org/x/net/http2/server.go:2152 +0x8b
created by k8s.io/kubernetes/vendor/golang.org/x/net/http2.(*serverConn).processHeaders
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/golang.org/x/net/http2/server.go:1882 +0x505"
2022-03-08 17:37:27.000,E0308 17:37:27.074450      10 writers.go:107] apiserver was unable to write a JSON response: http: Handler timeout
2022-03-08 17:37:27.000,"E0308 17:37:27.074469      10 status.go:71] apiserver received an error that is not an metav1.Status: &errors.errorString{s:""http: Handler timeout""}"
2022-03-08 17:37:27.000,E0308 17:37:27.075645      10 writers.go:120] apiserver was unable to write a fallback JSON response: http: Handler timeout
2022-03-08 17:37:27.000,"I0308 17:37:27.076749      10 trace.go:205] Trace[1088740983]: ""List"" url:/api/v1/secrets,user-agent:manager/v0.0.0 (linux/amd64) kubernetes/$Format,client:<IP_ADDRESS> (08-Mar-2022 17:36:27.074) (total time: 60002ms):
Trace[1088740983]: ---""Writing http response done"" count:9384 59993ms (17:37:00.076)
Trace[1088740983]: [1m0.002636616s] [1m0.002636616s] END"

I also looked at the pagination proposal for Kubernetes clients and apiserver here and the documentation here.
It specifically states that:
Some clients such as controllers, receiving a 410 error, may instead wish to perform a full LIST without chunking.

    Controllers with full caches
        Any controller with a full in-memory cache of one or more resources almost certainly depends on having a consistent view of resources, and so will either need to perform a full list or a paged list, without dropping results

But no matter whether we page or not at the client side, this will cause issues reading from etcd.
The KEP also provide an Alternatives section at the end stating:
Alternatives

Compression from the apiserver and between the apiserver and etcd can reduce total network bandwidth, but cannot reduce the peak CPU and memory used inside the client, apiserver, or etcd processes.
...

So is this a kube-apiserver CPU and/or memory issue we are hitting here?
I am going to try to raise the limits in our various EKS clusters and perform a scale test again to see if it improves the situation in any way.

@krish7919 Thanks for your analysis.

it is the apiserver which times out as it cannot read the data from etcd in the default timeout period; 60s by default in our production Kubernetes cluster (using AWS EKS), and 5s in my local tests using Kind

Just for completeness, do you mind mentioning how are you setting this in your Kind cluster? Is this via runtimeConfig?
Commenting on your findings so far: it seems that for now we're in a position in which we can't really do anything in the controller code which would improve the experience in use cases like yours, unless there's a proposal upstream in k8s that would allow server side listing with pagination (between apiserver and etcd).
At this point, we'll be looking forward for your further results and we'll leave this open in case there are any developments that would allow us to improve the experience.

Yes, this is my Kind config for reference:
            kind_config =
# Refer https://github.com/kind-ci/examples/blob/master/gitlab/kind-config.yaml
apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
networking:
  apiServerAddress: "<IP_ADDRESS>"

# Add an extra `kube-apiserver` argument to set a custom request timeout
kubeadmConfigPatches:
  - |
    kind: ClusterConfiguration
    apiServer:
      extraArgs:
        request-timeout: {}s

# Add to the apiServer certSANs the name of the docker (dind) service in order
# to be able to reach the cluster through it
kubeadmConfigPatchesJSON6902:
  - group: kubeadm.k8s.io
    version: v1beta2
    kind: ClusterConfiguration
    patch: |
      - op: add
        path: /apiServer/certSANs/-
        value: host.docker.internal

nodes:
  - role: control-plane
    extraMounts:
      - containerPath: /var/lib/kubelet/config.json
        hostPath: {}
    extraPortMappings:
      # For postgres
      - containerPort: 32000
        hostPort: 5432
        listenAddress: <IP_ADDRESS>
        protocol: TCP
  - role: worker
    kubeadmConfigPatches:
      - |
        kind: JoinConfiguration
        nodeRegistration:
          kubeletExtraArgs:
            node-labels: "{}-{}/node-type=general-worker"
    extraMounts:
      - containerPath: /var/lib/kubelet/config.json
        hostPath: {}
  - role: worker
    kubeadmConfigPatches:
      - |
        kind: JoinConfiguration
        nodeRegistration:
          kubeletExtraArgs:
            node-labels: "{}-{}/node-type=general-worker"
    extraMounts:
      - containerPath: /var/lib/kubelet/config.json
        hostPath: {}
""".format(
                apiserver_request_timeout_secs,
                ecr_auth_file.name,
                platform_name,
                env_name,
                ecr_auth_file.name,
                platform_name,
                env_name,
                ecr_auth_file.name,
            )

Commenting on your findings so far: it seems that for now we're in a position in which we can't really do anything in the controller code which would improve the experience in use cases like yours, unless there's a proposal upstream in k8s that would allow server side listing with pagination (between apiserver and etcd).
At this point, we'll be looking forward for your further results and we'll leave this open in case there are any developments that would allow us to improve the experience.

I would not say that you shouldn't do anything; being a company that ships a controller on Kubernetes, you can have the means/clout to drive this with the Kubernetes community OR put in resources to fix this from your end. :)

That's pretty good news @krish7919 🎉
Given your findings I'd like to propose the following:

We'd like to follow this up with an implementation of a runtime/feature flag (yet to decide which one) which will either:

disable compression on the whole client (as you've proposed in your diff)
disable it only for the initial list and the recreate the client with the already established defaults (compression enabled)

This will require us to perform some tests w.r.t the impact this setting might have on KIC's runtime performance as a whole (not only looking at the initial list which we were discussing here)

If possible we'd like to propose that you continue doing your testing and let us know if you encounter any other issues

@pmalek could you please create a feature request issue for the runtime flag, reference this issue in that FR issue, and close this? thank you :bow:

@mflendrich Added an issue for implementation of the flag: #2675

This is now going to be tracked in #2675 in order to implement a feature flag for disabling compression.


Optional branches with CUT_VALUE_DEBUG

Created dummy variables for CUT_VALUE_DEBUG flags.
The flag can be enabled during sdl_make_tracklooper via the option -d.
Once enabled the LSTNtuple.root tree will contain some additional branches e.g. t5_dummy, tc_dummy, ...
This is just a template, so that each analyzers can add variables for these.
NOTE that the function to write these optional branches is separated from the main performance evaluating main output branches.
This is so that we don't interfere with the main function bloating it like last time with gazillion CUT_VALUE_DEBUG flags written all over the places.

This is fine with me, I'll work on the efficiency part to add the lower object efficiencies and set up a PR for that


Mongoid 4

I would like a proper PR that makes mongoid_fulltext compatible with Mongoid 4. We haven't been using it for a while, so it hasn't been touched.


Codeunit 90 Change Parameter in OnBeforePostCommitPurchaseDoc event

Is it possible to change the event OnBeforePostCommitPurchaseDoc changing the parameter CommitIsSuppressed passing it by var?
Actual Event:
[IntegrationEvent(false, false)]
local procedure OnBeforePostCommitPurchaseDoc(var PurchaseHeader: Record "Purchase Header"; var GenJnlPostLine: Codeunit "Gen. Jnl.-Post Line"; PreviewMode: Boolean; ModifyHeader: Boolean; CommitIsSupressed: Boolean; var TempPurchLineGlobal: Record "Purchase Line" temporary)
begin
end;
Modification Request
[IntegrationEvent(false, false)]
local procedure OnBeforePostCommitPurchaseDoc(var PurchaseHeader: Record "Purchase Header"; var GenJnlPostLine: Codeunit "Gen. Jnl.-Post Line"; PreviewMode: Boolean; ModifyHeader: Boolean; var CommitIsSupressed: Boolean; var TempPurchLineGlobal: Record "Purchase Line" temporary)
begin
end;
Similar event is managed in Codeunit 80

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


libuvc_camera_nodelet.xml missing

The CMakelists wants to install libuvc_camera_nodelet.xml which I can't find in the repository. The install command was added in https://github.com/orbbec/ros_astra_camera/commit/efd14c789e2dcfe2ff5c3e4c7fbad19db7c3db85.
install(FILES libuvc_camera_nodelet.xml
DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
)

It shouldn't exist. I've already removed those two lines. Did you encounter any issues while compiling?

No, the rest worked nicely :)


Don't run in xcode Version 6.0.1

To tun project in xcode 6.0.1 in os x 10.9.4 don't start and show this log:
2014-09-22 09:55:46.168 Xcode[17235:303] [MT] DVTAssertions: UNCAUGHT EXCEPTION (NSInternalInconsistencyException): Extension Xcode.IDEKit.CmdHandler.ShowViewFrames class 'DBGMenuController' not found for required key 'handlerClass'
UserInfo: {
DVTExtensionClassNameErrorKey = DBGMenuController;
DVTExtensionIdentifierErrorKey = "Xcode.IDEKit.CmdHandler.ShowViewFrames";
DVTPlugInExecutablePathErrorKey = "/Applications/Xcode.app/Contents/PlugIns/DebuggerUI.ideplugin";
DVTPlugInIdentifierErrorKey = "com.apple.dt.dbg.DebuggerUI";
}
Hints: None
2014-09-22 09:55:46.667 Xcode[17235:303] [MT] DVTAssertions:
Backtrace:
0  0x00007fff89c00244 __exceptionPreprocess (in CoreFoundation)
1  0x00000001001e6184 DVTFailureHintExceptionPreprocessor (in DVTFoundation)
2  0x00007fff8f647e75 objc_exception_throw (in libobjc.A.dylib)
3  0x000000010004ebd7 -[DVTExtension _adjustClassAttribute:forKey:inParameterData:] (in DVTFoundation)
4  0x000000010004e633 -[DVTExtension _adjustClassReferencesInParameterData:usingSchema:] (in DVTFoundation)
5  0x00000001000d210c -[DVTExtension _fireExtensionFault:] (in DVTFoundation)
6  0x00000001000d1a75 __63-[DVTExtension _valueForKey:inParameterData:usingSchema:error:]_block_invoke (in DVTFoundation)
7  0x0000000101fa8925 _dispatch_client_callout (in libdispatch.dylib)
8  0x0000000101fa98df _dispatch_barrier_sync_f_invoke (in libdispatch.dylib)
9  0x000000010021cdfd DVTDispatchBarrierSync (in DVTFoundation)
10  0x000000010001fbc4 -[DVTDispatchLock performLockedBlock:] (in DVTFoundation)
11  0x00000001000d1851 -[DVTExtension _valueForKey:inParameterData:usingSchema:error:] (in DVTFoundation)
12  0x00000001000d1c44 -[DVTExtension valueForKey:error:] (in DVTFoundation)
13  0x00000001014173de __55+[IDECommandManager cacheCommandDefinitionsAndHandlers]_block_invoke (in IDEKit)
14  0x0000000101fa8925 _dispatch_client_callout (in libdispatch.dylib)
15  0x0000000101fa891b dispatch_once_f (in libdispatch.dylib)
16  0x00000001011d0a60 +[IDECommandManager cacheCommandDefinitionsAndHandlers] (in IDEKit)
17  0x00000001011d0054 -[IDEApplicationController applicationWillFinishLaunching:] (in IDEKit)
18  0x00007fff89bcee0c CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER (in CoreFoundation)
19  0x00007fff89ac282d _CFXNotificationPost (in CoreFoundation)
20  0x00007fff8c1397ba -[NSNotificationCenter postNotificationName:object:userInfo:] (in Foundation)
21  0x00000001001c6e6e -[NSNotificationCenter(DVTNSNotificationCenterAdditions) _dvt_postNotificationName:object:userInfo:] (in DVTFoundation)
22  0x00007fff8881ecde -[NSApplication finishLaunching] (in AppKit)
23  0x000000010074a665 -[DVTApplication finishLaunching] (in DVTKit)
24  0x00007fff8881e7f3 -[NSApplication run] (in AppKit)
25  0x00007fff88809783 NSApplicationMain (in AppKit)
26  0x00007fff8deac5fd start (in libdyld.dylib)

Want to back this issue? Place a bounty on it! We accept bounties via Bountysource.

@JugglerShu this fixed this crash for me. It looks like this change should be checked in?

For others reading this, this has been fixed on the current develop branch.

Thanks!! for fix this problem.


canGetProperty() and canSetProperty() does not take in account ActiveRecord::attributes()

Methods Object::canGetProperty() and Object::canSetProperty() does not take in account ActiveRecord::attributes(), always returning false for them:
/**
 * Properties, which are attributes from source table:
 *
 * @property integer $id
 * @property string $username
 */
class User extends \yii\db\ActiveRecord
{
    ...
}

$user = new User();
$this->assertTrue($user->canGetProperty('id')); // FAILS !!!
$this->assertTrue($user->canSetProperty('id')); // FAILS !!!

Inside yii\db\BaseActiveRecord we override __set() and __get() methods, but there is no redeclaration of canGetProperty() or canSetProperty().

BaseActiveRcord extends Model, Model extends Component, Component extends Object but functions canGetProperty() and canSetProperty are redeclared in Component (behaviors added). Don't you think it is better to redeclare the function in BaseActiveRecord within adding block of searching key in attributes, do you?
Another question why in Component you don't use parent::canGetProperty, but copy the code from Object?

if a attribute is seen as a 'settable' property, which I do think is logical and in fact rather straightforward, I would assume that both would include a check on BaseActiveRecord::hasAttribute().

Resolved by commit 75439d367d2d1270bb61642f236df6a552acf29e


How can I configure the live running server, My JSON Server

Hi @typicode ,
I could access JSONPlaceholder from my Angular program on stackblitz.
But I wanted to use my own data so I tried to use My JSON Server free service.
Then I got the following error:

Failed to load https://my-json-server.typicode.com/typicode/demo/posts/1: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://angular-http-test-cors.stackblitz.io' is therefore not allowed access. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

So I want to know if I can configure the live running server, My JSON Server.
My sample is here.

I also have the same issue. The client is on http://localhost and the response header is missing 'Access-Control-Allow-Origin' header.

Same issue here after a year! Any fix?

same issue here after another year... I guess thats a no fix


Add maintainer & contributors details in README

@KIET7UKE  Assign me this issue

Please assign this to me

@abhishekpatil4  @error707-persona  Assigning both of you, Made a great contribution :)


Firebase missing deploy script

We should probably add firebase-cli as a dev dependency, and also add deploy script

The official documentation recommends to install firebase-cli globally - I suppose it's because of authentication.


CX Command_Injection @ src/main/java/com/scalesec/vulnado/CowController.java [master]

Command_Injection issue exists @ src/main/java/com/scalesec/vulnado/CowController.java in branch master
The application's run method calls an OS (shell) command with cmd, at line 7 of src\main\java\com\scalesec\vulnado\Cowsay.java, using an untrusted string with the command to execute.
This could allow an attacker to inject an arbitrary command, and enable a Command Injection attack.
The attacker may be able to inject the executed command via user input, input, which is retrieved by the application in the cowsay method, at line 12 of src\main\java\com\scalesec\vulnado\CowController.java.
Severity: High
CWE:77
Checkmarx
Lines: 12

Code (Line #12):
    String cowsay(@RequestParam(defaultValue = "I love Linux!") String input) {

Issue still exists.
SUMMARY
Issue has total 1 vulnerabilities left to be fix (Please scroll to the top for more information)

Issue still exists.
SUMMARY
Issue has total 1 vulnerabilities left to be fix (Please scroll to the top for more information)


HDDS-7922. [FSO] S3G folder support fso layout filestatus s3A compatibility

What changes were proposed in this pull request?
s3g currently says:

/volume/bucket/dir/  exists.
/volume/bucket/dir  exists.

s3a is expecting:

/volume/bucket/dir/  exists.
/volume/bucket/dir  does not exist.

Proposed is an implementation to resolve the issue and to cover corner cases to support fso folders over s3g by modifying the s3g ObjectEndpoint to check whether the requested path is a directory, if so, to only return ok if path has trailing '/'.  This would support s3 Head requests and Get requests.  The type of object, where file or directory is determined in the head-object and get-object request through checking the keyInfo (modifying the OzoneKey and OzoneKeyDetails) isFile().
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7922
How was this patch tested?

manual
unit
smoke

cc @SaketaChalamchala @hemantk-12 @tanvipenumudy

@duongkame @hemantk-12 thank you for the review!

@tanvipenumudy thanks for the review! Adding one more test for above-mentioned case makes perfect sense.

@kerneltime any chance we can get a motion on this PR? It's been a while and all the comments have been addressed.

@mladjan-gadzic I saw an integration test (filesystem) failure, could you help take a look? Thanks.

@mladjan-gadzic I saw an integration test (filesystem) failure, could you help take a look? Thanks.

@DaveTeng0 thanks for having a look! Failing tests are part of already known issue which has its own Jira https://issues.apache.org/jira/browse/HDDS-8526.

Thank you @smengcl for the review. I merged upstream/master and resolved conflicts. CI on my fork passed. Unfortunatelly I do not have a permission to run CI on upstream. If you could do so, that would be great!

Thanks @duongkame @hemantk-12 @tanvipenumudy @whbing @DaveTeng0 for reviewing this.


AsRef, AsMut, Borrow, Deref

Investigate if AsRef, AsMut, Borrow, Deref traits should be implemented to convert from &T1 to &T2 if T1 is a C++ subclass or T2.

Deref and cpp_utils::StaticUpcast are now implemented for derived classes.


dynamic element validation

I have a problem with validating inputs that are generated dynamically.
Here is the fiddle that shows the problem: http://jsfiddle.net/flsilva/8JzDb/17/
this is taken from: https://groups.google.com/forum/?fromgroups=#!topic/angular/qLeUWZVuQMI
the name is generated fine but it's not bound to the form scope. Because I can name it correctly but the form is not bound, it seems that this behavior is inconsistent.
Thoughts?

@caitp I wish I found this thread a year ago when I first had this issue. I used the ng-form hack. Unfortunately, this hack does not work when your dynamic forms that are being repeated several times have radio buttons... because even though the form validation "works", now all of your radio buttons are part of the same radio group and so you can only select 1 on the entire page of forms. But I digress...
We are stuck on 1.2.x for the moment and so my question is, did @Thinkscape 's decorators (found above) prove to be as effective as the eventual solution that is implemented in 1.3.x?

@spencerb02 you mean the decorators from my plunker? :p
So, the stuff that was merged into 1.3.x has some extra meat to it. In particular, it will remove controls from the form controller and re-add them if the interpolated value changes, and the goal is to remove them when they are destroyed, too.
The decorators just interpolate one time, so it can cause problems during ngRepeat if an $index changes, for instance.

@caitp excellent. Thank you. Forgive my naivety, but when would indexes change that wouldn't also be a redraw? Also, if I have another unique key to tie it to rather than $index, this won't be an issue, right?

@spencerb02 it's not exactly that simple, if the form is outside of ngRepeat, and the length changes, you can still have stale stuff in the form --- during a redraw, if an element is moved, it is not re-transcluded. http://plnkr.co/edit/rRYgWgdo4M9sUnGiNwqv?p=preview << so, the controller wouldn't be re-created, and it wouldn't create a new scope, just the scope values would change.

+1

+1

+1

+1

+1

+1

+1

+1

Thank you to the AngularJS devs for the changes in 1.3 because I had no trouble getting validation to work for dynamically-added form elements using ngRepeat in AngularJS 1.3.
To summarize a slightly ambiguous discussion from earlier this year, AngularJS 1.3 now allows you to dynamically assign the name inside ngRepeat using the double curly braces.
name="formControl_{{uniqueId}}"

To reference the form element name inside an object, use this pattern:
ng-messages="myForm[ 'FormControl_' + uniqueId].$error"

There is no longer a need to use subforms (embedded ngForm directives) to make validation work for dynamically added form elements in version 1.3

+1

+1
as @jeremyml said about complicated paths, i ended up using the next path to access a dynamic field inside of an ng-form form, inside of an ng-repeat
mainForm['extraInfoForm_' + $index]['name_' + $index].$error

What a workaround!
Angular version 1.4.5

Thanks @rafa-gda for the ng-name directive! Works like a charm on 1.4.5

I'm glad to hear it @matfiz but you do not need that workaround in v1.4.5 since the form directive interpolates the provided name by default. If you use name attribute instead of ng-name in your form it should work just fine.

@rafa-gda, thanks! But I am using it on input fields, so I can handle more complex validations on nested, dynamically added form elements (conditions on multiple fields). The name attribute in v1.4.5 on input did not interpolate, I had to use your hacky ng-name :)

@matfiz, input fields support interpolated name attributes since v1.3.0-beta.11 (so all stable 1.3.x and 1.4.x versions). Maybe the problem lies somewhere else.

@gkalpak, thanks for the useful remark! You were right, thanks to your information I have found an issue elsewhere

@jodinathan, yes this has been fixed. If you are having problems using dynamic element names, I would suggest creating a live demo of the issue and posting it to one of the appropriate support channels.
Or, if you think you found a bug/regression, please open a new issue with a detailed description (and preferrably a live demo - e.g. using CodePen, Pnlkr etc).

is this issue fixed ?  I am struggling same issue . Could you please work around other than nested ng-form

I am getting below error when i use {{}} symbol in ng-messages directive
Error: [$parse:syntax] Syntax Error: Token '{' is an unexpected token at column

@diwanoli, if you think this is a bug, please open a new issue providing all necessary details (ideally a runnable demo of the problem). If you are not sure if this is a bug in Angular, try asking a question on one of the support channels.

+1

+1


Sanitize invalid file names

This adds the option to automatically sanitize invalid file names that may be created from movie titles such as "The Lord of the Rings: The Return of the King"

Thanks for catching this! A better option is for user to specify a replacement character and default to "-" if not set, since invalid characters will always break file creation

@aaachen I've updated per your recommendation and changed the name to make a little more sense now that it will always replace invalid characters.

Lgtm, thanks for making the change! I'd release it after I figure out the process


Feature Request: GUI for command history

It would be nice to view a list of my command history (and easily scroll then click a command to reuse it).

Just use the up and down keys to cycle through the history. 😊

@leo Yup, I know that's there.
At the minimum, Hyperterm should expose a command for getting a specific index in the history. I despise hitting the up key like a maniac whenever looking for a command further back in the history.
What do you think?

@aleclarson I highly reccommend checking out https://github.com/junegunn/fzf. It's a command line fuzzy finder.
With it, you can hit Ctrl-R from your terminal and get a fuzzy search window of all your previously issued commands :ok_hand:

You can also set up things like a vimf command that will launch the fuzzy finder for opening a file in vim.

@flybayer Thanks for the help! 👍  Do you know if it would work in something like the NodeJS repl?

At the minimum, Hyperterm should expose a GUI for copying a command
(from the history) using a specific index.

😄

I would recommend   peco

@aleclarson I doubt it works with the node REPL out of the box, but it could probably be configured to work with it.


503 Service Unavailable No package matching 'php7.4' is available"

Issue Type

Bug Report / Support Request

Your Environment

Vagrant 2.2.14
VirtualBox 6.1.22r144080
ansible 2.10.9
  config file = None
  configured module search path = ['/Users/deborahfuzetto/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/Cellar/ansible/3.4.0/libexec/lib/python3.9/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.9.5 (default, May  4 2021, 03:36:27) [Clang 12.0.0 (clang-1<IP_ADDRESS>)]

Your OS

macOS (version)

Full console output

TASK [geerlingguy.firewall : Check if ufw package is installed (on Ubuntu).] ***
fatal: [nidcr]: FAILED! => {"changed": false, "cmd": ["service", "ufw", "status"], "delta": "0:00:00.010523", "end": "2021-06-07 23:55:34.423076", "msg": "non-zero return code", "rc": 3, "start": "2021-06-07 23:55:34.412553", "stderr": "", "stderr_lines": [], "stdout": "* ufw.service - Uncomplicated firewall\n   Loaded: loaded (/lib/systemd/system/ufw.service; disabled; vendor preset: enabled)\n   Active: inactive (dead)", "stdout_lines": ["* ufw.service - Uncomplicated firewall", "   Loaded: loaded (/lib/systemd/system/ufw.service; disabled; vendor preset: enabled)", "   Active: inactive (dead)"]}
...ignoring
TASK [geerlingguy.php : Ensure PHP packages are installed.] ********************
fatal: [nidcr]: FAILED! => {"changed": false, "msg": "No package matching 'php7.4' is available"}
Summary

BLT <IP_ADDRESS>
Vagrant 2.2.14
Virtual Box 6.1.22
drupal-vm 6.0.4
composer 2.1.2
I'm getting 503 Service Unavailable most likely since it cannot install php.

Hi there,
I was getting this for the past 2 days and i found that updating my ubuntu to 18.04 (from 16.04) fixed this, I am not sure about a fix for 16.04 though unfortunately
vagrant_box: geerlingguy/ubuntu1804 in my config.yml

That got me a litter farther.  After destroying and rebuilding, I got past the php error but now getting this error:
TASK [geerlingguy.adminer : Download Adminer to configured directory.] *********
fatal: [nidcr]: FAILED! => {"changed": false, "dest": "/opt/adminer/index.php", "elapsed": 0, "msg": "Request failed", "response": "HTTP Error 404: Not Found", "status_code": 404, "url": "https://www.adminer.org/latest.php"}

Seems https://www.adminer.org/latest.php is down or removed. https://github.com/geerlingguy/ansible-role-adminer/issues/15
There's no variable to change the download url but personally for now I just edited line 18 in drupal-vm/provisioning/roles/geerlingguy.adminer/tasks/main.yml:
url: https://github.com/vrana/adminer/releases/download/v4.7.7/adminer-4.7.7-mysql-en.php

I just removed adminer from installed_extras for now.  It appeared to be successful except when I visit from the browser I get the Apache2 Ubuntu Default Page so it still seems like something didn't complete.
Also, not sure if this is related but I'm still getting the Uncomplicated firewall error:
TASK [geerlingguy.firewall : Check if ufw package is installed (on Ubuntu).] ***
fatal: [nidcr]: FAILED! => {"changed": false, "cmd": ["service", "ufw", "status"], "delta": "0:00:00.013970", "end": "2021-06-08 13:25:53.379676", "msg": "non-zero return code", "rc": 3, "start": "2021-06-08 13:25:53.365706", "stderr": "", "stderr_lines": [], "stdout": "● ufw.service - Uncomplicated firewall\n   Loaded: loaded (/lib/systemd/system/ufw.service; disabled; vendor preset: enabled)\n   Active: inactive (dead)\n     Docs: man:ufw(8)\n\nJun 08 12:34:31 local systemd[1]: Stopping Uncomplicated firewall...\nJun 08 12:34:31 local ufw-init[2593]: Skip stopping firewall: ufw (not enabled)\nJun 08 12:34:31 local systemd[1]: Stopped Uncomplicated firewall.", "stdout_lines": ["● ufw.service - Uncomplicated firewall", "   Loaded: loaded (/lib/systemd/system/ufw.service; disabled; vendor preset: enabled)", "   Active: inactive (dead)", "     Docs: man:ufw(8)", "", "Jun 08 12:34:31 local systemd[1]: Stopping Uncomplicated firewall...", "Jun 08 12:34:31 local ufw-init[2593]: Skip stopping firewall: ufw (not enabled)", "Jun 08 12:34:31 local systemd[1]: Stopped Uncomplicated firewall."]}
...ignoring

I'm running into the same issue as @oxyc regarding adminer this morning
TASK [geerlingguy.adminer : Download Adminer to configured directory.] *********
fatal: [xyz]: FAILED! => {"changed": false, "dest": "/opt/adminer/index.php", "elapsed": 1, "msg": "Request failed", "response": "HTTP Error 404: Not Found", "status_code": 404, "url": "https://www.adminer.org/latest.php"}

@cainaru - I just updated Drupal VM to use the latest version of the adminer role which uses the new variable adminer_download_url — if you update to the latest version of Drupal VM (master branch, not the latest tag), then you can set that variable to the URL @oxyc mentioned above and override the URL.

@geerlingguy Great, looks like that worked! Thank you!!

I'm going to close this issue and merge follow-up discussion into new issue #2169.

Fixed most of my issues: vagrant destroy && vagrant up
Still having trouble with Adminer.
Thank you for all of the help!


Release Chef Server 12.11.1

Signed-off-by: Steven Danna<EMAIL_ADDRESS>

👍 (My comment is a micronitpick, ignore at will)


Terraform Login

Feature description 💡
https://developer.hashicorp.com/terraform/cli/commands/login
Wanted to see we can restrict more auth like terraform login for preventing run from local machine
Anything else?
No response

Can you explain a little bit more about this?

Even if we are doing it on local machine then appropriate permission are required otherwise mistakenly if someone run destroy then it will destroy whole stuff

For now there is no way to prevent unwanted terraform actions if you are using terraform remote state.
You can give access to certain groups to a organization, but once you have "manage workspace" you can do anything unless you do some kind of customization inside the terrakube templates

I am using the template to restrict the deployment to only selected branches, now the new version supports a similar feature for the webhook, I am wondering for VCS workflow, should deployment and destroy be restricted to be kicked off only from the Terrakube UI? aka, only plan can be run from local. I remember this is the action TFC takes.


How to proper show translation/transcript along with the original digitisation?

I am looking for some library (like this one) to show digitised ancient manuscripts, but I would love to show the translation and/or the transcript of the text next to the original image at the same time. What alternatives do you guys think I have? I am talking about thousand of documents for several cultural institutions. I really appreciate any idea.

Do you want the translation text to appear in the same viewer, panning and zooming with the scan? Or do you want it in a scrolling panel next to the viewer? If it's the former, you could use OpenSeadragon along with one of the overlay plugins to position the text where you want it. If it's the latter, you'd just do the text in regular HTML / JavaScript, and you can use OpenSeadragon events to synchronize with navigation. You'd have to build some of the functionality either way, as I'm not aware of an off the shelf solution for exactly what you're looking for.
Have you looked into http://projectmirador.org/? It uses OpenSeadragon, and I believe it has some sort of annotation functionality, though I don't know if it's exactly what you need.


how to use it?

I want to extract HFS files from a "Vindictus" Korean server.
When you compile and use the source, only "unknown.bin" is generated continuously.
I want to extract each information file and organize the game strategy and information in the Korean community.
If you help, I will use it in a useful place. Thank you.

Using the release version:

Open cmd
Navigate to the exe folder

Type the following:
HFSExtract.CLI.exe hfs_directory extract_directory [summary.txt]
Where hfs_directory is the location of the hfs files, extract_directory is where you want the extracted files to go and summary.txt is an optional log file.


listroute() df should indicate which order waypoints appear in on route

The listroute() function returns a data frame with waypoint names as row indexes. In PyDodo, these are currently sorted in alphabetical order (rather than the order in which they are meant to be passed on the route).  Fix order to reflect route order.
Investigate whether this is also an issue in rdodo.

Ffixed and tested on the new_endpoints branch  in PyDodo.


Update sbt-mima-plugin to 1.1.1

Updates com.typesafe:sbt-mima-plugin from 0.9.2 to 1.1.1.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.typesafe", artifactId = "sbt-mima-plugin" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "@monthly" },
  dependency = { groupId = "com.typesafe", artifactId = "sbt-mima-plugin" }
}]

labels: sbt-plugin-update, early-semver-major, semver-spec-major, commit-count:1

Coverage decreased (-1.1%) to 93.671% when pulling ce582368a9cf266baf0a1b336105e950effa1bb0 on scala-steward:update/sbt-mima-plugin-1.1.1 into 6d0e09cb4eba1784601b6b114c1126fde31dd7e8 on iheartradio:master.

Superseded by #253.


Implement another data retriever that requests information by PID

The current patient information retriever sends a HL7v2 message that requests information by case number.
For some systems it could make sense to request this information by Patient IDs.

Merged with #100 into #88 and then with #102 into v3.2


pidfile is the same when running with -T

Pid file is the same regardless of interface name , when running with -T :
 /run/.pid
Therefore it is impossible to run multiple instances of dhcpcd on different interfaces in test mode.
The bug doesn't exist in Ubuntu 22.04 ( dhcpcd 7.1.0) and appeared in Ubuntu 24.04 ( dhcpcd 10.0.6)

That's an odd one. Running separate instances of dhcpcd works fine if it's not in test mode. I wonder what test mode does differently to end up with the same PID file. That's clearly a mistake.

Why do you want to use test mode on many interfaces? Sounds like what you really want is the --noconfigure option?

Also, I can't replicate this at all. When running in test mode a pidfile is never created by dhcpcd.

Here is how to reproduce. Get a system with 2 network cards, echo them to the stdin of the script.
echo enp0s3 enp0s8  | xargs -n1| xargs  -II -P10  -- bash -c 'bash -c "dhcpcd -4 -1 I -T | grep new_ip_ad" 2>&1 | sed "s@^@I==@" '|sort
Output :
enp0s3==DUID 00:01:00:01:2e:b6:24:a4:02:47:5b:32:b6:f2
enp0s3==dhcpcd exited
enp0s3==dhcpcd-10.0.6 starting
enp0s3==enp0s3: IAID 5b:32:b6:f2
enp0s3==enp0s3: leased <IP_ADDRESS> for 86400 seconds
enp0s3==enp0s3: rebinding lease of <IP_ADDRESS>
enp0s3==new_ip_address='<IP_ADDRESS>'
enp0s8==dhcpcd exited
enp0s8==dhcpcd-10.0.6 starting
enp0s8==main: pidfile_lock 1076662: File exists

You see, they started to fight for the same pidfile. And its content is :
root@px24 [~] # cat /run/.pid
1076662

I need it to simply query DHCP server status on multiple interfaces without configuring them. Unfortunately --noconfigure  doesn't output lots of useful info (full dump of the DHCP response).
So the solution would be to include interface name to the  pidfile  filename when running with -T.

fixed on my side with unshare -m bash -c "mount -t tmpfs -o size=10M tmpfs /run/; dhcpcd -1 $DEV -T"

--configure has all the information in the environment prefixed with _new.
You could write an exit hook to dump this to a file as well if you wanted.
I'm unsure how to replicate this though as in no instance does dhcpcd create a pidfile with -T for me.


Ground truth appears to be occasionally incorrect at intersections

Hi Dian,
It sometimes looks like the ground truth computed from by the action labeler gives incorrect supervision at red lights. See the below example:

I don't think I've changed any of the action labeling logic, but it looks like the ground truth assigns a very low probability to braking even though that would be the correct action in this situation. Have you seen anything like this/have any suggestions for fixing the issue? In general, it looks like red light infractions make up the vast majority of infractions, and they often lead to other infractions like vehicle collisions or route deviations.

Do you see this all the time for traffic lights?
If so you definitely changed the traffic light related logic.

diffing bellman.py between your code and mine shows no differences as far as I can tell. How large are the no speed reward zones for traffic lights when they are yellow/red? Do they extend into the intersections?

Thanks for the clarification. There are several things going on here.
First, when you mention traffic light infraction makes up most of the error, do you mean the image agent you trained, or the image agent with the downloaded weights (provided in this repo), or the Q agent?
Second, do you see examples posted above everywhere, in every town? Or does it mostly appear in Town10HD?
One thing I am sure: traffic light infraction by default does not make up most of the infractions, and this is also shown in our public leaderboard entry's metrics.

I've benchmarked both the pretrained weights and my own trained weights (initialized from pretrained). They both appear to have this red light infraction distribution. Just to be absolutely certain, I'm rerunning the pretrained weights on routes_testing right now in case I messed something up.
I don't want to put extra work on your shoulders, but do you happen to have results from the routes_testing split available? Sometimes I suspect that the training/testing/held-out routes have weird distributions through towns and stuff.

Sometimes I suspect that the training/testing/held-out routes have weird distributions through towns and stuff.

Could be true, but I currently do not have spare compute to verify this.
Generally, I do not think there is a problem for traffic lights infraction (see table 6 of paper).
However, in the later US towns (Town10HD, for example), the traffic light trigger box definition is different for certain intersections, and if the vehicle is deep into the intersection, it is out of the trigger box even if you see a red light in the RGB image. And I don't think you want the to stop in that situation either: the crossing traffic will hit the ego vehicle if it blocks in the intersection.


Forgot password ?

If i forgot my password, is there any way to recover it ? or can i just copy the .md file elsewhere and open it again. ?

If you forget your password, please leave your commonly used email and I will send you the solution.

Also, what is the version number of the password protection plugin you are using?

What system are you using? Windows or iOS or Android?

If i forgot my password, is there any way to recover it ? or can i just copy the .md file elsewhere and open it again. ?

Delete the extension, reinstall. Doesn't matter


fine level correspondance

Great work!
Could you explain a little bit why the self-supervised loss (symmetric epipolar distance) will encourage the model to find the most informative point in patch A?

Thank you for your question @Moreland-cas,
We define the ground truth pairs of matches and then train the model. However, finding explicitly the most highly matchable point in a patch to extract ground truth is challenging. Instead, using a self-supervised loss to implicitly detect the highly matchable points is more reasonable.
We used a detector network to estimate a score for each point in patch A. It is observed that the points in the flat regions (with similar intensity values) will yield the same score while the peak point will give the highest score. Therefore, the peak point will be chosen as the keypoint of interest.

Thank you for your question @Moreland-cas,
We define the ground truth pairs of matches and then train the model. However, finding explicitly the most highly matchable point in a patch to extract ground truth is challenging. Instead, using a self-supervised loss to implicitly detect the highly matchable points is more reasonable. We used a detector network to estimate a score for each point in patch A. It is observed that the points in the flat regions (with similar intensity values) will yield the same score while the peak point will give the highest score. Therefore, the peak point will be chosen as the keypoint of interest.

Thanks, it make sense that points within flat regions will produce similar intensity values.
Also, have you investige how many pixels of error still exist in the fine level stage, is there any room of improvement?

@Moreland-cas , we didn't. But as you might know, the fine-level correspondences are estimated at a half-resolution of input images. So if we can refine the matches at the full resolution of images, we can improve the performance. But this comes with higher computation costs

@Moreland-cas , we didn't. But as you might know, the fine-level correspondences are estimated at a half-resolution of input images. So if we can refine the matches at the full resolution of images, we can improve the performance. But this comes with higher computation costs

I see, Thanks again for your work and quick reply :)

@Moreland-cas , we didn't. But as you might know, the fine-level correspondences are estimated at a half-resolution of input images. So if we can refine the matches at the full resolution of images, we can improve the performance. But this comes with higher computation costs

I see, Thanks again for your work and quick reply :)


Rubric For the Final Project

I know this was briefly touched on in class but what does the final project entail?
-Do we explain what the algorithm does? And do we make our code to show an example?
-Do we demonstrate real life examples or are we only focused on the algorithm?

Is this a purely coded project or do we make a powerpoint-like presentation for it (or both)?
Is there a rubric we can follow to make the project?

The format on finals day will be:

Each team presents 3-4 minute "lightning talk". You'll probably want to make a couple slides for that. Focus on what you find interesting about your chosen topic and try to relate it to themes in our class.
You can focus on applications, algorithms, performance, or any combination. You should show whatever you did, such as implementing an algorithm, testing a library implementation on an application, etc.
After the lightning talks, we'll split into breakout groups of about 5 with no more than one member from each team. Your peers will ask questions that go into more depth -- each person will have about five minutes to answer questions from the group. Be prepared to answer questions like those appearing in your README.md.


[libusb] build failure on Windows - <assert.h> not found

Operating system
Windows
Compiler
MSVC
Steps to reproduce the behavior
vcpkg install --triplet=x64-windows-static-md --x-install-root=3rdparty_vcpkg

Failure logs
vcpkg.log
build-x64-windows-static-md-rel-out.log
Additional context
I have the following vcpkg.json:
{
  "dependencies": [
    { "name": "libusb", "platform": "windows" }
  ]
}

And I build with this command:
vcpkg install --triplet=x64-windows-static-md --x-install-root=3rdparty_vcpkg
CMake configure runs fine, the build starts, and cl.exe complains that <assert.h> is not found. All the relevant logs attached. There is no out-err.log file (there is but it's 0 size).
The same works on 2 other computers, only fails on this one. At first I was sure it's a problem with my Visual Studio installation and I spent hours reinstalling everything. Didn't help. hen I started checking, and realized that other C and C++ libraries compile just fine, and they also include <assert.h>. For example, I checked sqlite3. Also checked raylib (a large C library), exiv2 (a rather large C++ library). I couldn't find any other port that fails in this way, so now reporting this against the libusb port seems justified. But you will have a hard time reproducing the failure as I have no idea what triggers it.

Any chance someone could look into this? I understand vcpkg is an open-source community-supported project, but I can't work on my project without libusb. And I don't have a slightest idea how I could investigate it myself, as I'm barely familiar with CMake, let alone with how it searches for installed compilers and CRT/SDK.


Teams - Upload file failed

It is confusing that based on the licensing side by default there is a 2gb cloud storage for file attachment. But every time we try to attach a file we receive an error.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f897ae9b-852a-b617-7de0-d76dbd9f1820
Version Independent ID: 161b1845-53fd-8912-a5f9-737d3bcdaadc
Content: Manage the Microsoft Teams Exploratory experience - Microsoft Teams
Content Source: Teams/teams-exploratory.md
Service: msteams
GitHub Login: @LolaJacobsen
Microsoft Alias: lolaj

@Yoeeee thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.


[Docgen] Hide internal, Q# → qsharp, and fix ms.topic.

/azp run

/azp run

/azp run

/azp run

By the way, I think the pipelines should automatically run on every push without needing /azp run - is that not working for you?

By the way, I think the pipelines should automatically run on every push without needing /azp run - is that not working for you?

@samarsha: I've had enough cases where  that hasn't worked that I've gotten a bit paranoid, I guess...

@samarsha: I've had enough cases where  that hasn't worked that I've gotten a bit paranoid, I guess...


Add riak-admin repair leveldb command [JIRA: RIAK-2483]

This command can seed the repair with the correct data dir with tiers loaded as well. Would also need to work in a multibackend setup.

Must also check to make sure node is down.


⚠ ClamAV has degraded performance

In 4f04759, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 68 ms

Resolved: ClamAV performance has improved in 163be1c after 18 minutes.


Car Modes

User story 2.2 : Car modes
As a mail man I want to be able to choose an automatic mode so that the car delivers mails automatically.**
Acceptance criteria:
The application should provide choices that allows the mailman to choose to control the car via auto-drive or manual drive.

I updated the issue to follow the backlog that we have created.

This issue is out of scope, we are only using manual mode because of lack of time


[staging-next] pdal + friends update

Description of changes
Fix build.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Looks like this supersedes #273948?

Kind of. qgis is probably still broken.

PR for PDAL is already created (https://github.com/NixOS/nixpkgs/pull/273948) and is not merged because QGIS-LTR can't be built with this version (see: https://github.com/NixOS/nixpkgs/pull/273948#issuecomment-1854800559)

Yeah, the problem is that PDAL itself also no longer builds on staging-next with the libxml update. I guess we can try backporting the changes instead. Anyway, I'll just let you take it from here - this was something I did on a lark without really understanding the ecosystem.

Thanks @K900


[js] syntax highlighting broken after bracket in the same line as 'if' or 'for'

VSCode Version: 0.10.11
OS Version: Win 7

Steps to Reproduce:

Write some code with an 'if' statement, and a variable compared with an smaller-as '<' operator (without separating space).
Add a code block bracket on the same line.
Write more code in the block

Minimal example:
var j;
if(j<2){
    var i=123;
}
Result in the dark+ standard scheme:

btw. what should the color of the jbe? It is different in the second example.

Also occurs with for loop, e.g.:

duplicate of https://github.com/Microsoft/TypeScript-TmLanguage/issues/11


IPSec auth not working

I have similar issue to #1625, but in my case, auth script is in right place, but it still fails with following, OPNsense 17.7.7_1:
Nov 29 13:30:30 | charon: 07[IKE] Could not authenticate with XAuth secrets for 'x.x.y.y' - 'test'
Nov 29 13:30:30 | charon: 07[IKE] XAuth-SCRIPT failed for user 'test' with return status: -1.
Nov 29 13:30:30 | charon: user 'test' could not authenticate.
Nov 29 13:30:30 | charon: 07[ENC] parsed TRANSACTION response 608599183 [ HASH CPRP(X_USER X_PWD) ]
users xauth password is defined, he is assigned xauth role, but it seems the hash is incorrect. any ideas?

I have been literally pulling my hair out because of this. The password is correct, it is the same string as the username. I have enabled Mutual PSK + XAUTH. Is there perhaps a way to debug xauth script?

Please post your VPN: IPsec: Mobile Clients settings, without them this won't make any sense at all :)

Which file will I find it in?

Your GUI settings under IPsec: mobile clients

On 23. Dec 2017, at 09:35, zhouska<EMAIL_ADDRESS>wrote:
Which file will I find it in?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

Ok, here goes (not sure if there is a way to export it as a text):
IKE Extensions
Enable > Yes
Extended Authentication (Xauth)
User Authentication > Local Database
Group Authentication > system
Client Configuration (mode-cfg)
Virtual Address Pool > Network:  <IP_ADDRESS>/24
Everything else if left untouched (not other box is ticked).

That looks ok, and can you confirm the password for the user with system: access: tester?


[Feature]: New connector for MFT gateway

Connector Name
connectors/openapi (Open API Connector)
Suggested feature
MFT Gateway is a Cloud-hosted Managed File Transfer Gateway offered as a Service.  The MFT Gateway is a hosted AS2 communications solution, offered as Software as a Service (SaaS), and requires no hardware, software, installation, or maintenance.

[x] Background check
[x] Generate the connector with existing swagger
[x] Resolve issues
[x] Optmize operations in the connector
[x] Review
[x] Release

Related issues
No response

https://github.com/ballerina-platform/openapi-connectors/pull/653

Released the connector https://central.ballerina.io/ballerinax/aayu.mftg.as2


Create View+ConditionalModifier.swift

This PR adds a conditional modifier to a SwiftUI View.

@alemannosara it would be great to add some tests if possible :)

@alemannosara we should as well upgrade the xcode version and testing device for the CI https://github.com/qoqa/Qit/blob/master/.github/workflows/ci.yml


Roadmap question on Purview (not real issue)

I was wondering if Purview could evolve to a tool to measure the quality of the data contained in the underlying databases?
Just like you have thresholds for automatic classification and put glossary terms on database, will purview evolve to a system where you could check or have a report on % of data that have the correct format eg is the data  in the e-mail field technical/format wise a e-mail and for which % of the the records is the format of the field OK?
Is that something one can or will be able to do with Purview

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 09ff3183-0560-3b4c-5a58-465bf7fc36b2
Version Independent ID: 244bee19-459f-1d6d-8cc3-96ee95971807
Content: Introduction to Azure Purview (preview) - Azure Purview
Content Source: articles/purview/overview.md
Service: purview
Sub-service: purview-data-catalog
GitHub Login: @hophanms
Microsoft Alias: hophan

Thank for the suggestion. Please find the feature enhancement request or post new one here: https://feedback.azure.com/forums/932437-azure-purview
#please-close


feat(WMGV2): Improved method of adding genes to heatmap

Reason for Change

#4766

Changes

add

Improved method of adding genes to the heatmap

remove

Old QuickAdd code

Notes for Reviewer
Note we now have three duplications of the same component, one in cell cards, one for cell type filter, and one for gene add

lint errors are in unused (WMGV1) files, will address in the morning, but not required for review

@seve SDS 15.4.2 has the Autocomplete fix now, so maybe a followup PR to use it 🙏 ? Thank you!


Dev issue #532

This commit solves issue #532 to allow modelling the power consumption of a number of parallel identical pumps (specified by the integer variable GroupCount in the pump object SPump operating at different relative speeds s following the approach given by @LRossman. The code was tested on a simple network with the topology shown below.

The pump characteristics were measured by recording pressure heads in nodes R1 and J1 for different values of demands set in node J1.
The network was simulated with one pump and two pumps in parallel operating with s=1.0 and s=0.75 respectively. The results form OWA EPANET (marked as points) were compared against calculations "by hand" in MATLAB/OCTAVE (marked with lines).
First, the scaling of pump curves was tested for the pump type: EN_CUSTOM and EN_POWER_FUNC respectively (please see the results below)

Second, we tested the efficiency curve scaling and power consumption where the efficiency curve of a single pump operating at s=1 is described with an interpolated curve (multipoint) and analytical third order approximation function with coefficients calculated with Eq.8 in B. Ulanicki, J. Kahler, and B. Coulbeck, “Modeling the Efficiency and Power Characteristics of a Pump Group,” Journal of Water Resources Planning and Management, vol. 134,no. 1, pp. 88–93, February 2008.
The results are shown in the figures below.

Note: We can sometimes see slight mismatches for some points in plots where the efficiency curve in OWA EPANET was interpolated. The reason for this is that for cross-testing interpolation was not modelled in OCTAVE and instead, the curves in OCTAVE were calculated using an analytical function. The mismatches arises in points where the interpolated efficiency is different to efficiency calculated from the analytical function.
Finally, scaling was also tested for a global constant efficiency (for a single pump) eta = 0.75

@lepton1979 good job. I made some changes as follows:

A group with fixed power pumps should have them operate in parallel like other pump groups do, not in series.
The calculation of a pump group's head loss and gradient is simpler than what you coded. Just compute these for a single pump's flow using the existing code and then adjust the gradient by the group count.
I reformatted most of the getenergy function in hydraul.c to replace the tabs with spaces and make some of the comments clearer.

Some remaining things to do are:

Change line 1134 in report.c to be:

if (hyd->LinkFlow[j] > hyd->LinkSetting[j] * pump->Qmax * pump->GroupCount)

Allow a pump's GroupCount to be changed by a control (which is the real value of using pump groups).


Add validator to DropdownMenu

Use case
There is no validator property on DropdownMenu.
It is tricky to show error messages on DropdownMenu because of not knowing when the validation is being ran on the parent form.
Proposal
When using DropdownMenu, it'd be great to have a validator property, as we already do for FormField:
https://api.flutter.dev/flutter/widgets/FormField/validator.html

Isn't there DropdownButtonFormField already made for this?

Isn't there DropdownButtonFormField already made for this?

I guess you're right and I just had not found that component when opening this issue.

Actually I looked into the issue as I couldn't remember what it was, and the thing is DropdownButtonFormField wraps DropdownButton, while I am looking for DropdownMenu. I think the main difference is that DropdownButton doesn't allow for keyboard input.

The drop down  button is more like deplicated version,  I think we need the validator and also the search enabled on android and ios for dropdown menu, currecntly you have to reset focus after filter/search on mobile

Here looking for this feature also. DropdownMenu is the updated Material 3 version of the DropdownButton component. Not having validation support on the DropdownMenu component Is making it difficult to implement Material 3 compliant dropdowns.


Add user search

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Current dependencies on/for this PR:

main

PR #2

PR #3

PR #4   👈

This stack of pull requests is managed by Graphite.


Transfer form. Add dropdown with child account list to recipient field

Add the ability to select a seed account in the "To" field

@OlegMakarenko maybe we can use address book here somehow?

@cryptoBeliever yes since we have a address book dropdown here now, it would be good to add the wallet's account names to the list

@OlegMakarenko closing this one because you created https://github.com/nemgrouplimited/symbol-desktop-wallet/issues/1098


Feat/UI setup

Change to the project structure to support multiple scenes.
Implement homescreen & input name screen

Styling is quite basic at the moment, will replace placeholders with nice ones!

conflict, but looks good to me 👍


[Feature Request] Add OSRSWiki support

Can get a player stats from: http://services.runescape.com/m=hiscore_oldschool/index_lite.ws?player=X
For quests i don't seem to find an endpoint.

"totalRank,totalLevel,totalXp" at the start
Then for each skill: "rank,level,xp" (space delimited) in the following order: Attack Defence Strength Hitpoints Ranged Prayer Magic Cooking Woodcutting Fletching Fishing Firemaking Crafting Smithing Mining Herblore Agility Thieving Slayer Farming Runecraft Hunter Construction
From here

Maybe check the skills automatically and then let user check from a list if they have started or finished a quest.


RU translation

This needs to wait until the completion of moving strings to resources and forming official translation teams.


[Explicit Modules] Provide an API for querying whether a job is a main module job.

This will be useful for SwiftPM (and other build systems) to distinguish intermediate dependency artifacts versus the main module build, in order to correctly pass in cross-target dependencies.

@swift-ci please test


ng update --next does not offer to update to the latest version of CLI tool

Is this correct?

Angular CLI: 8.0.0
Node: 11.14.0
OS: win32 x64
Angular: 8.0.0
... animations, cli, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.800.0
@angular-devkit/build-angular     0.800.0
@angular-devkit/build-optimizer   0.800.0
@angular-devkit/build-webpack     0.800.0
@angular-devkit/core              8.0.0
@angular-devkit/schematics        8.0.0
@ngtools/webpack                  8.0.0
@schematics/angular               8.0.0
@schematics/update                0.800.0
rxjs                              6.5.2
typescript                        3.4.5
webpack                           4.30.0

Hi, yeah that is correct cause at the moment the next version of the cli is not yet published.


Fixes removeItem not passing row data to callback.

Call to row.data('data') was happening after the .remove() call, which deletes the retained data. This was passing undefined back to the callback for removeItem.
I've changed the data retrieval to a temporary variable before the delete call.

Coverage decreased (-0.1%) to 88.186% when pulling 3864a7d40e3ec1fbd8fdece51188d789eab8bc3f on jthomas:master into e6b5552cbac0b041f8a1785a585a620d9dcc0b20 on node-red:master.


RadioBox should be compatable with form actions

Describe the feature in detail (code, mocks, or screenshots encouraged)
As of now, I am not able to use RadioGroup directly in form actions (use:enhance can be used through) but it would be awesome if I could pass in the name prop and recieve the value in formData.
Now I might be missing something, so please let me know if there is a way to do so. Here is my code:
<RadioGroup.Root value="dev">

          <RadioGroup.Item value="dev" id="dev-radio" name="type" />
          looking for gigs

          <RadioGroup.Item value="org" id="org-radio" name="type" />
          looking to hire

</RadioGroup.Root>

What type of pull request would this be?
None
Provide relevant links or additional information.
No response

You can. Here's how to do so with your implementation.
<RadioGroup.Root value="dev">

          <RadioGroup.Item value="dev" id="dev-radio" />
          looking for gigs

          <RadioGroup.Item value="org" id="org-radio" />
          looking to hire

		<RadioGroup.Input name="type" />
</RadioGroup.Root>

Feel free to open up a discussion or ask on Discord if you have any further questions!