[text] Add option to load all glyph of a font instead of just the range between 32...126

Please, before submitting a new issue verify and check:

[x] I tested it on latest raylib version from master branch
[x] I checked there is no similar issue already reported
[x] My code has no errors or misuse of raylib

Issue description
Maybe, when a TTF file is loaded with default parameters, instead of specifically asking which glyphs the font should load, it should load the entire glyph collection by default.
IF someone actually loads a giant font collection I know it may crash, but that's another problem.
This come, from the neccesity of giving my user, for example, a name input, and I would like to see names like "Noël" widhtout me to knowing how many glyphs I should load beforehand.
Maybe giving -1 to the size of the LoadFontEx could do that.
Environment
The platform it's not really a bug, as it is the current expected behaviour.
However, I'm using 2.5.0 and tested it in both windows and mac osx mojave.
Code Example
Instead of:
LoadFontEx(name, size, NULL, SOME_ARBITRARY_BIG_NUMBER);

it could be swapped by:
LoadFontEx(name, size, NULL, -1); // This checks how many glyphs on the fonts are there and automatically adds all glyphs

If not a new function like LoadFontPro or LoadCompleteFont(name, size), could be cool too:
LoadFontPro(name, size);

What happens if you try the maximum size an int could possibly have to load as the charsCount? Does that make the function crash?

I'm afraid that's not the best approach to font loading. LoadFontEx() already allows a custom set of codepoints (3rd parameter of the function) defined by user:
int codepoints[5] = { 32, 45, (int)'A', 4567, 96 };   // Define required codepoints
Font font = LoadFontEx("resources/myfont.ttf", 32, codepoints, 5);  // Load required codepoints glyphs from font

User is responsible of providing the right font and loading the required glyphs. Also, DrawText() will draw a ? if the required glyph is not available in the loaded font.


Sync test/schema.sql file with the hub

Originally posted by @ChaituVR in https://github.com/snapshot-labs/snapshot-sequencer/pull/181#discussion_r1342321343

This github action seems an appropriate solution: https://github.com/marketplace/actions/repo-file-sync-action
Need to create a new github personal access token, the GITHUB_TOKEN does not work with any github actions with sync feature


Limited editor features for @bind:after

I'm using @bind:after to define a custom event on binding. My code looks like this:

This code is valid but it's missing syntax highlightling, autocomplete, renaming  and go to definition features for everything inside @bind:after
Since this issue has also something todo with renaming it might be a good idea to add it to #8541
@maryamariyan

@MarvinKlein1508 thanks for the report. Would you be able to share a more descriptive repro steps? some minimal steps that would easily reproduce the issue/s by tweaking the new blazor server app we create from the templates in VS perhaps?

Pretty sure this is caussd by the lambda, and we have an existing issue tracking it, but I can't find it. Definitely shouldn't be duped to the rename issue, or it would disappear. There is something more fundamentally wrong here, possibly a compiler issue.

@davidwengier same behavior even without lambda.

Would still be good to get a minimal repro for this.

Would still be good to get a minimal repro for this.

I will provide you one by tomorrow

@maryamariyan here you go: https://github.com/MarvinKlein1508/BindAfter

Hi again, it seems like the same thing applies to @bind-Value:set and @bind-Value:after

This is due to a source mapping not being added by the compiler. We do not generate source mappings for either bind:after.

@adamint this seems to be the case for all @ which have somelogic after :

ComponentBindLoweringPass is stripping out the intermediate nodes that would otherwise be used to add source mappings in DefaultDocumentWriter.

I wonder if my PR I have open might help here, as it adds intermediate nodes for every attribute node. Though that begs the question: does the attribute value end up in the generated C# at all at the moment?

I had a look, because this was nagging at me, and the attribute values do make it in to the generated C#, so there must be some kind of node in the tree for them, though perhaps one that has thrown away any knowledge of the source locations.

@davidwengier Anything left to do here?

Yes, but I can't remember what it is 😛
I'll close for neatness, and we can log a new issue if something else comes up. I think it was something I saw internally, and maybe didn't have user impact anyway.


[Elasticache] - Allow option to autogenerate password for auth

What problem are you facing?
We are looking at migrating from the community AWS provider to the upbound ones. One significant difference we see is that this provider strictly requires a secret being passed in for auth, whilst we can see the use for this it would also be good to have the ability to autogenerate the password as we currently do and send the result out to writeConnectionSecretToRef in a similar way to what happens when creating dynamodb tables using this provider.
The reason we have this requirement is our infrastructure is entirely gitops based including the manifests used in crossplane so any situation where we have to provide passwords means entering them into git which isn't ideal
How could Official AWS Provider help solve your problem?
Without this migrating to the upbound provider is quite difficult if not potentially impossible (investigations still ongoing)

Hi @dhumphries-sainsburys,
Thank you for raising this issue, if you would like to contribute we would appreciate it, here is a sample PR: https://github.com/upbound/provider-aws/pull/628

Having the same issue, really tricky to migrate the provider for elastic cache. @dhumphries-sainsburys even with this feature I am not sure how it is possible to recreate the secret with the password in there, I guess it is not possible.


[SMALLFIX] Deflake FileSystemIntegrationTest.

This test fails with a high chance on hadoop profiles without this fix.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/15847/
Test PASSed.

LGTM. Thanks Calvin


fixes #168 #280 added unit test for Garbage collection iterator, fixed b...

...ugs

fixed bug where garbage collection iter was using wrong timestamp for DEL_LOCK invalidation
fixed bug where ACKs where not being GCed

+1


adding log scale to lineplot and changing scatter plot for its behavior

This will address https://github.com/VEuPathDB/web-eda/issues/1157
Also, this depends on https://github.com/VEuPathDB/web-components/pull/363

This commit addressed two issues among others: a) adding log scale to lineplot Viz; b) "For scatter plot, we should not allow trendlines and log scale at the same time. The log scale can always be available, but if i'm in smoothed mean mode and click log scale, the plot mode goes back to raw"


Forward auto differentiation not implemented for torch softplus nonlinearity

Found using functorch.hessian and in a model with a torch.nn.Softplus() nonlinearity. Error message:
NotImplementedError: Trying to use forward AD with softplus_backward that does not support it.

Writing the nonlinearity by hand fixed the issue. Reporting as requested

Thanks for reporting, @lrast. I assume that you're using PyTorch 1.11 and functorch 0.1.*.
This problem should be fixed in the pytorch nightly + functorch main branch. If you're interested in trying that out, please see the instructions over at https://github.com/pytorch/functorch#installing-functorch-from-source .


Loop waits until all members are scheduled before starting loop

Submitting a large scale 5000 member loop workload. It seems that argo waits before the entire loop is expanded before processing the loop. simple example with stress:
resources:
        limits:
          cpu: "1"
      command: ["stress"]
      args: ["--cpu","1","--timeout","40s"]

Expected container runtime is 40s but is in processing for well over 1 hour until all 5000 members are submitted and then processed and completed.
  argo: v2.6.3
  BuildDate: 2020-03-16T17:56:00Z
  GitCommit: 2e8ac609cba1ad3d69c765dea19bc58ea4b8a8c3
  GitTreeState: clean
  GitTag: v2.6.3
  GoVersion: go1.13.4
  Compiler: gc
  Platform: darwin/amd64

Is this expected behavior? Should the loop members start processing during loop expansion or after loop expansion?

Labeling with scalability.

Here is a simple reproducible example
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: loops-param-result-
spec:
  entrypoint: loop-param-result-example
  templates:
  - name: loop-param-result-example
    steps:
    - - name: generate
        template: gen-number-list
    # Iterate over the list of numbers generated by the generate step above
    - - name: sleep
        template: sleep-n-sec
        arguments:
          parameters:
          - name: stress
            value: "{{item}}"
        withParam: "{{steps.generate.outputs.result}}"

  # Generate a list of numbers in JSON format
  - name: gen-number-list
    script:
      image: python:alpine3.6
      command: [python]
      source: |
        import json
        import sys
        json.dump([i for i in range(1, 1000)], sys.stdout)

  - name: sleep-n-sec
    retryStrategy:
      limit: 10
      retryPolicy: "Always"
    inputs:
      parameters:
      - name: stress
    container:
      image: polinux/stress
      resources:
        limits:
          cpu: "1"
      command: ["stress"]
      args: ["--cpu","1","--timeout","40s"]

The main delay is caused in 'createWorkflowPodwhich is running synchronously (blocking call).  I tried with span the goroutine to executecreateWorkflowPod` workflow is creating nodes fast and start updating the finished pods.
Changes in operator.go line 1839
	go woc.createWorkflowPod(nodeName, *tmpl.Container, tmpl, &createWorkflowPodOpts{
		includeScriptOutput: includeScriptOutput,
		onExitPod:           opts.onExitTemplate,
	})

@jessesuen and @alexec  can you provide the your feedbacks?

maybe use a wait group with this, might be good to limit the number of goroutine with a work queue

if the user wants to limit their goroutines. they can use parallelism on their workflow.

Another suggestion is the Queue and worker design. All pod creation will be pushed into Queue and workers will create pods.

@amrragab8080 can you test your scenario with my dev Build ?
Here my docker image tag: sarabala1979/workflow-controller:issue-2739


Question: Should I stay with this extension or move to bitlang.gnucobol?

This isn't a bug report but a question, if this isn't the right place to ask then please point me to the correct one.
Context: A customer wants to drop his costs and asked for a gnucobol port of the application. I've found there was a "3.2 preview" release two weeks ago with claimed improved support for MF so I'll try to compile everything with "cobc -std=mf" and see where I land. I'm lucky that there is no object oriented cobol in the code, gnucobol obviously lacks this completely, this is an Oracle only application so no files need to be converted either.
The code will stay "as-is" where possible and hopefully don't need too much adjustments - in every place it does conditional compilation should be used.
While I'm not sure yet how this works out, I'd like to keep the IDE support.
Questions:
Do I need to wait for a newer version of bitlang.gnucobol or should I use an older version of gnucobol (3.1.2 vs. 3.2)?
Would it be reasonable to move to bitlang.gnucobol for this project or stay with bitlang.cobol?
Maybe I should install both, but I guess only one can be active.
Additional question: Do you have a suggestion how to use conditional compilation so that "cob" will use the code as-is and only "cobc" will use the new one?

Re: context
I would recommend you & your customer consider speaking to Micro Focus about the reasons for moving away from them.
The benefits of using the Micro Focus COBOL platform are huge and worthy of further consideration.
Next, it is worth understanding that Visual Studio Code is a code editor, not an Integrated Development Environment.

Do I need to wait for a newer version of bitlang.gnucobol or should I use an older version of gnucobol (3.1.2 vs. 3.2
When 3.2 is available is GA and rolled out, I expect the bitlang.gnucobol will catch up with the support.

It is worth noting, I don't know of any gnucobol debugger that works the bitlang.gnucobol extension.

Maybe I should install both, but I guess only one can be active.
You can enable/disable extensions on a per-workspace basis.

Additional question: Do you have a suggestion how to use conditional compilation so that "cob" will use the code as-is and only "cobc" will use the new one?]

Stick with Micro Focus and the problem goes away :-)

@Smitty2018  - Happy with the answer?

I'm not in the position to decide about sticking with Micro Focus, but passed the suggestion. I was told that there are some big $$$ paid for annual licenses and as Micro Focus bills only increased over the years no one believed that the account manager even has the option to cut that bill without decreasing the number of licenses...
So I'm kind of back in the original place.
Rechecking showed that the Micro Focus extension actually does provide quite a nice debugging experience. It is much easier to get started with it than with Animator; I guess there's likely some features missing of that on the other hand... not sure but as I like it I'll surely do some serious debugging with the Micro Focus extension :-)
Some testing with a debugger for GnuCOBOL showed very nice results with the gnucobol debug extension from the market place - but that extension seems to only work with simple programs / setup as it always does a recompile when clicking on debug... note: that extension suggests to use bitlang.cobol :-)
Checking more led to another debugging extension found at https://gitlab.cobolworx.com/COBOLworx/cbl-gdb-vsextension. It needs a one time recompile with the cobc wrapper cobcd; but as one can always use it (there's even no .idy as with Micro Focus and I did not found a speed difference either) that's not that big deal. This extension provides "less good but working results".
You may want to test those two if you want to find out about a debugger that works with bitlang.gnucobol.
It just was... quite strange that the bitlang.cobol extension totally freaked out as long as those debugging extensions were enabled (even when I was not using them); I guess that's part of the answer to my original question and says "there is a conflict between bitlang.cobol and bitlang.gnucobol".
bitlang.gnucobol also warned about duplicate functionality in bitlang.cobol; but this was just a warning instead of the "I won't work correct, so I disable myself" seen there. I have to admit that I fail to see why it doesn't work correct when there's an extension enabled that isn't used and think a warning like in the other extension would be enough.
As I now have the Micro Focus extension active I've thought that bitlang.cobol isn't needed anymore in the Micro Focus environment, but from the documentation "go to" and friends need VC8 - and there is only VC7 installed. Using bitlang.cobol for view/edit and the Micro Focus extension for debugging does work quite well, so that's no problem.
As noted above I'm still not sure what the answer to my question is. Maybe I need to create a second vscode installation using "portable mode"  https://code.visualstudio.com/docs/editor/portable for the gnucobol experiment and only install the bitlang.gnucobol and cbl-gdb in there?

One of the disadvantages of vscode is the way it handles languages, if you have multiple extensions that work on the same language, you either get missing, duplicate or recursive events or it simply uses the wrong extension (due to the extensions registering different languages id for the same file extension)... then comes the problem of ensuring the visualisation of code matches the debugger and breakpoints can set etc..
Given these numerous problems, I have found the best thing was to try and guide/find mismatches between various extensions and warn or disable this extension.
With this extension, I try to ensure it will work as well as it can with the "real" Micro Focus COBOL extension, especially the debugging support, so I am glad you find it useful.
Have you or your customer considered speaking to someone other than pre-sales/sales?   If, so I would contact: Scot Nielsen, feel free to mention me, "Stephen Gennard", as Scot knows me :-)

Have you or your customer considered speaking to someone other than pre-sales/sales? If, so I would contact: Scot Nielsen, feel free to mention me, "Stephen Gennard", as Scot knows me :-)

I'm not the one speaking to Micro Focus, but I've only heard of sales contacts so far. I don't think that anyone has considered speaking to someone else but sales about the license topic. I'll drop that note and see where this is going to and think this suggestion will help others, too.
Until my project is changed (I'm still tasked with the evaluation, did not found much time for that in the last week), I'm going with a single vscode installation (just updated to 1.73) using two code-workspace approach. One workspace is used for Micro Focus using their extension and bitlang.cobol [especially for debugging] and the other one is used with COBOLworx.cbl-gdb and bitlang.gnucobol as using "deactivate in this workspace" for the other extensions works well.
Note: the gnucobol extension seems to miss some parts, for example the IntelliSense example given in bitlang.cobol does not work ("add 1 to" shows the alphanumeric variables first) and the "peek definition" command is missing as well as the copybook extension setting. Should I detail these as issues in https://github.com/spgennard/vscode_cobol4gnucobol or better request a "don't check for incompatible addons" here? (bitlang.cobol seems to work much better than bitlang.gnucobol but deactivates itself when the other debugging extension is enabled)

I should also said that Micro Focus also provide the COBOL-IT product, which includes an eclipse environment and a debugger.

I should also said that Micro Focus also provide the COBOL-IT product, which includes an eclipse environment and a debugger.

Thanks for pointing out more possible alternatives!

So you say I should ask to possibly switch to COBOL-IT instead of GnuCOBOL if this experiment doesn't work out?
For using COBOL-IT along with vscode, I'd then stay with bitlang.cobol, but there isn't a debugging frontend for vscode available yet, correct?

The website https://www.cobol-it.com is currently down - Do you know if ít has current updates and plans to support vscode?
And most important: Should it be possible to switch from Micro Focus COBOL to COBOL-IT without changes / with less changes than GnuCOBOL?
I have not found anything but different command line options and the need to set some runtime options in GnuCOBOL so far, but I've not compiled the complete source either, much less ran more than some smaller programs.

Just an update on the side issue - the COBOL-IT website is back, now forwarding to https://www.microfocus.com/en-us/cobol-it. That seems like an option if the "leave" is already fixed. As noted I'll keep that in mind (and have now found that this is Eclipse centered without any note on possible updates for vscode support; with portlist from 2016!). So it is more likely to either stay with MF or move to gnu.
Concerning the original "question issue" - this is solved for me so I'm closing it now. Thank you for all the pointers you've shared!
I'll create a "real issue" for the other point next.


App Crash with iOS 14

Bug Report
Problem
The app crashes after the permissions window appears and you click "allow." If you re-open the app, the permission is granted and you can access the contacts correctly. Has anyone else seen this or know any fixes/forks that address this?
What is expected to happen?
The app will not crash after allowing contact permissions.
What does actually happen?
It crashes.
Information

I have set the NSContactsUsageDescription in Info.plist and see that when it prompts so I know that isn't the problem. From debugging the crash log there is an error that says:
Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Call must be made on main thread'
I suspect this is a problem with the way the plugin is implemented not using the main thread
Command or Code

The pickContact method being called for the first time in an app, causing the permissions window to appear. After accepting you should see the crash.
Environment, Platform, Device

All iOS devices on version 14 (could have been happening earlier but I didn't notice it).
Version information

Cordova 10, cordova-plugin-contacts 3.0.1 (latest)
Checklist

[x] I searched for existing GitHub issues
[x] I updated all Cordova tooling to most recent version
[x] I included all the necessary information above

Can you check if you have this with my Contacts Plugin aswell? https://github.com/EinfachHans/cordova-plugin-contacts-x

I have the same problem but it's a simple fix. See the PR.

We are archiving this repository following Apache Cordova's Deprecation Policy. We will not continue to work on this repository. Therefore all issues and pull requests are being closed. Thanks for your contribution.


Only fetch projects when tab is active

Uses "visibilityjs" to create an interval that only calls cb when tab is active
resolves #17

It should use the old behavior when sound alerts are enabled. I'd propose to make this a config option (backgroundRefresh: false defaulting to true).

This is now an option

Thanks!


[Question] Conversational vs Retrieval Conversational  Agents

I have done a lot of testing using these two agents. Conversational Agent was much better in respecting the prompts and gave less hallucinations. overall, the response quality was very good. The only issue with Conversational Agent is that it is does not support streaming and from a user experience perspective a delay in response is observed.
Switching to Retrieval Conversational Agent, response is much better, and streaming is supported BUT the agent is not respecting the full system prompt as Conversational Agent do specially when the conversation goes on with more back and forth calls (First questions the agent respects the system prompt but after like 5 or 6 question and answers if will start loosing it).
Any advice here, Also is thier a way to auto clear the context while a conversation is running when the agent feels that the response quality is getting lower.

Please note all testing was done using GPT 3.5-turbo Model

You can try OpenAI Function Agent and see.
Reason that Retrieval Conversational Agent might not respect the full prompt after more conversations is because this agent is using intermediate steps (reasoning, thoughts) to build the response, so the more context, the weaker it will stick to the original prompt

I guess for remembering intermediate steps as promoted in langchains blog post the Conversational Retrieval Agent requires the OpenAIAgentTokenBufferMemory, which is available in the Conversational Retrieval Agent Toolkit (another langchain abstraction).
It seems there are currently two ways to instantiate the CRA:

createConversationalRetrievalAgent with the advanced memory builtin
initializeAgentExecutorWithOptions with a configurable memory or simple buffer memory

I have added a ConversationalRetrievalAgentToolkit to our flowise fork. It works just fine. By the way, the original ConversationalRetrievalAgent in flowise sets the default system prompt to "You are a helpful AI assistant.", which is different to the langchain default of "Do your best to answer the questions. Feel free to use any tools available to look up relevant information, only if necessary.".

@zeitderforschung, Thanks. hope it will get committed to the main branch as well.

@HenryHengZJ I guess for remembering intermediate steps as promoted in langchains blog post the Conversational Retrieval Agent requires the OpenAIAgentTokenBufferMemory, which is available in the Conversational Retrieval Agent Toolkit (another langchain abstraction).
It seems there are currently two ways to instantiate the CRA:

createConversationalRetrievalAgent with the advanced memory builtin
initializeAgentExecutorWithOptions with a configurable memory or simple buffer memory

I have added a ConversationalRetrievalAgentToolkit to our flowise fork. It works just fine. By the way, the original ConversationalRetrievalAgent in flowise sets the default system prompt to "You are a helpful AI assistant.", which is different to the langchain default of "Do your best to answer the questions. Feel free to use any tools available to look up relevant information, only if necessary.".
Additionally we added source urls to the output of the RetrievalTool and told the agent to output the answer together with links to the sources when available. This was an huge improvement.

thank you so much for the improvement!! will take a look and definitely put it to Flowise!

@zeitderforschung when you have the retrievertool to return answer all together with metadata source and title, how does that improve the agent response? as in like what difference have you noticed, with and without sources?

@HenryHengZJ Thanks for asking. Maybe I should talk about our use case first.
We have a large JSON lines file with a MediaWiki Export that includes the content, source url and title of the hundreds of wiki pages. This JSON Lines file is imported into a vector store. For this we have created a modified version of the JSON Lines Reader that also stores the source url and the page title for every json lines item (our documents) in the vector store.
What we want is a chatbot with the Conversational Retrieval Agent that we can ask about the wiki content and that not only shows us the answer from the wiki pages, but also includes nicely formatted markdown links in the answer, so that the user can jump to the source of the answer. For this we have created a version of the retriever tool that puts the source link in front of each text snippet. finally we had to change the system prompt of the agent to output the source links when available. The prompt looks like that: "Do your best to answer the questions. Feel free to use any tools available to look up relevant information, only if necessary. Always list markdown links contained in the corresponding answer of the tool."
So the improvement for us is, that there is an answer that contains cited sources with clickable markdown links. I think thats a common use case. As an alternative you can use a Conversational Retrieval Chain with source chips, but thats not as powerful as the agent version with embedded links and a more natural kind of conversation.

@zeitderforschung thanks for the details explanation!


TCP connectivity tests pass and fail randomly

What happened:
I created two GCP Kubernetes clusters with kubeadm and Calico, having overlapping CIDRs. I installed Submariner with globalnet enabled. When using the subctl verify command, all the service-discovery tests passed, but some TCP connectivity tests failed with a timeout. And the weird thing is those tests sometimes pass, but most of the time fail.
Summarizing 7 Failures:

[Fail] [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod connects via TCP to the globalIP of a remote service when the pod is not on a gateway and the remote service is not on a gateway [It] should have sent the expected data from the pod to the other pod
github.com/submariner-io/shipyard@v0.13.2/test/e2e/framework/network_pods.go:188

[Fail] [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod connects via TCP to the globalIP of a remote service when the pod is on a gateway and the remote service is not on a gateway [It] should have sent the expected data from the pod to the other pod
github.com/submariner-io/shipyard@v0.13.2/test/e2e/framework/network_pods.go:188

[Fail] [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod matching an egress IP namespace selector connects via TCP to the globalIP of a remote service when the pod is not on a gateway and the remote service is not on a gateway [It] should have sent the expected data from the pod to the other pod
github.com/submariner-io/shipyard@v0.13.2/test/e2e/framework/network_pods.go:188

[Fail] [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod matching an egress IP pod selector connects via TCP to the globalIP of a remote service when the pod is not on a gateway and the remote service is not on a gateway [It] should have sent the expected data from the pod to the other pod
github.com/submariner-io/shipyard@v0.13.2/test/e2e/framework/network_pods.go:188

[Fail] [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod with HostNetworking connects via TCP to the globalIP of a remote service when the pod is not on a gateway and the remote service is not on a gateway [It] should have sent the expected data from the pod to the other pod
github.com/submariner-io/shipyard@v0.13.2/test/e2e/framework/network_pods.go:188

[Fail] [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod with HostNetworking connects via TCP to the globalIP of a remote service when the pod is on a gateway and the remote service is not on a gateway [It] should have sent the expected data from the pod to the other pod
github.com/submariner-io/shipyard@v0.13.2/test/e2e/framework/network_pods.go:188

[Fail] [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod connects via TCP to the globalIP of a remote headless service when the pod is not on a gateway and the remote service is not on a gateway [It] should have sent the expected data from the pod to the other pod
github.com/submariner-io/shipyard@v0.13.2/test/e2e/framework/network_pods.go:188

Errors were like this:
STEP: Waiting for the listener pod "tcp-check-listenerk89rk" on node "cluster-b-kadm-node-2" to exit, returning what listener sent
Dec 19 17:27:01.038: INFO: Pod "tcp-check-listenerk89rk" on node "cluster-b-kadm-node-2" output:
listening on <IP_ADDRESS>:1234 ...
nc: timeout

What you expected to happen:
All the verification tests should pass.
How to reproduce it (as minimally and precisely as possible):

create two GCP clusters using kubeadm
install Calico on the clusters, enable VXLAN and disable BGP
install Submariner with globalnet enabled
configure Calico ippools with globalCIDR for each cluster

Anything else we need to know?:
Everything is correct when runng subctl show all and subctl diagnose all.
Environment:

GCP clusters
Calico version: v3.24
Kubernetes version: v1.24.9
subctl version: v0.13.2

Thanks for reporting the issue @huiyizzz. When the problem is reproduced, please attach the output of the following.

subctl gather --dir  # On both the clusters, after the completion of this command, tar the folders and attach it - it contains the logs from various pods and some additional info which will help troubleshoot the problem.
subctl diagnose all # on both the clusters.

I have attached the subctl gather command output.
submariner-gather.tar.gz
Here's the subctl diagnose all ouput:
Cluster "cluster-a"
 ✓ Checking Submariner support for the Kubernetes version
 ✓ Kubernetes version "v1.24.9" is supported

 ✓ Checking Submariner support for the CNI network plugin
 ✓ The detected CNI network plugin ("calico") is supported
 ✓ Trying to detect the Calico ConfigMap
 ✓ Calico CNI detected, checking if the Submariner IPPool pre-requisites are configured

 ✓ Checking gateway connections
 ✓ All connections are established

 ✓ Globalnet deployment detected - checking if globalnet CIDRs overlap
 ✓ Clusters do not have overlapping globalnet CIDRs
 ✓ Checking Submariner pods
 ✓ All Submariner pods are up and running

 ✓ Checking Submariner support for the kube-proxy mode
 ✓ The kube-proxy mode is supported

 ✓ Checking the firewall configuration to determine if the metrics port (8080) is allowed
 ✓ The firewall configuration allows metrics to be retrieved from Gateway nodes

 ✓ Checking the firewall configuration to determine if intra-cluster VXLAN traffic is allowed
 ✓ The firewall configuration allows intra-cluster VXLAN traffic

 ✓ Checking Globalnet configuration
 ✓ Globalnet is properly configured and functioning

Skipping inter-cluster firewall check as it requires two kubeconfigs. Please run "subctl diagnose firewall inter-cluster" command manually.

Cluster "cluster-b"
 ✓ Checking Submariner support for the Kubernetes version
 ✓ Kubernetes version "v1.24.9" is supported

 ✓ Checking Submariner support for the CNI network plugin
 ✓ The detected CNI network plugin ("calico") is supported
 ✓ Trying to detect the Calico ConfigMap
 ✓ Calico CNI detected, checking if the Submariner IPPool pre-requisites are configured

 ✓ Checking gateway connections
 ✓ All connections are established

 ✓ Globalnet deployment detected - checking if globalnet CIDRs overlap
 ✓ Clusters do not have overlapping globalnet CIDRs
 ✓ Checking Submariner pods
 ✓ All Submariner pods are up and running

 ✓ Checking Submariner support for the kube-proxy mode
 ✓ The kube-proxy mode is supported

 ✓ Checking the firewall configuration to determine if the metrics port (8080) is allowed
 ✓ The firewall configuration allows metrics to be retrieved from Gateway nodes

 ✓ Checking the firewall configuration to determine if intra-cluster VXLAN traffic is allowed
 ✓ The firewall configuration allows intra-cluster VXLAN traffic

 ✓ Checking Globalnet configuration
 ✓ Globalnet is properly configured and functioning

Skipping inter-cluster firewall check as it requires two kubeconfigs. Please run "subctl diagnose firewall inter-cluster" command manually.

I had a close look at the logs of various Submariner pods like submariner-globalnet, submariner-gateway (the errors related to public-ip resolution can be ignored, its not fatal) and route-agent, but I don't see any errors which can affect the datapath.
Also, I see that IPsec connections are successfully established on the GW nodes.
000 Total IPsec connections: loaded 1, active 1
000
000 State Information: DDoS cookies not required, Accepting new IKE connections
000 IKE SAs: total(1), half-open(0), open(0), authenticated(1), anonymous(0)
000 IPsec SAs: total(1), authenticated(1), anonymous(0)

Also, when the subctl e2e verify tests were executed, I dont see any errors in the pods. For example,
In Cluster1, Globalnet Pod:

I1221 19:18:51.042310       1 global_egressip_controller.go:138] Processing created GlobalEgressIP "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-e2e-egressip-e2e-tests-dataplane-gn-conn-nd-9jn2n", NumberOfIPs: 1, PodSelector: (*v1.LabelSelector)(nil), Status: v1.GlobalEgressIPStatus{Conditions:[]v1.Condition(nil), AllocatedIPs:[]string(nil)}
I1221 19:18:51.042723       1 global_egressip_controller.go:200] Allocating 1 global IP(s) for "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-e2e-egressip-e2e-tests-dataplane-gn-conn-nd-9jn2n"
I1221 19:18:51.042904       1 ipset.go:349] Running ipset [create SM-GN-CNFI27FCMHZB4CSG5Y4SYEUXW hash:ip -exist]
I1221 19:18:51.046187       1 iface.go:273] Installing iptable egress rules for Namespace "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-e2e-egressip-e2e-tests-dataplane-gn-conn-nd-9jn2n": -p all -m set --match-set SM-GN-CNFI27FCMHZB4CSG5Y4SYEUXW src -m mark --mark 0xC0000/0xC0000 -j SNAT --to <IP_ADDRESS>
I1221 19:18:51.053281       1 global_egressip_controller.go:261] Allocated [<IP_ADDRESS>] global IP(s) for "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-e2e-egressip-e2e-tests-dataplane-gn-conn-nd-9jn2n"
I1221 19:18:51.153672       1 global_egressip_controller.go:355] Started pod watcher for "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-e2e-egressip-e2e-tests-dataplane-gn-conn-nd-9jn2n"
I1221 19:18:51.154112       1 base_controllers.go:208] Updated: &v1.GlobalEgressIPStatus{Conditions:[]v1.Condition{v1.Condition{Type:"Allocated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2022, time.December, 21, 19, 18, 51, 53262791, time.Local), Reason:"Success", Message:"Allocated 1 global IP(s)"}}, AllocatedIPs:[]string{"<IP_ADDRESS>"}}
I1221 19:18:58.275244       1 egress_pod_watcher.go:97] Pod "e2e-tests-dataplane-gn-conn-nd-9jn2n/customdbgqv" with IP <IP_ADDRESS> created/updated
I1221 19:18:58.279717       1 ipset.go:349] Running ipset [add SM-GN-CNFI27FCMHZB4CSG5Y4SYEUXW <IP_ADDRESS> -exist]
I1221 19:22:07.523356       1 global_egressip_controller.go:138] Processing deleted GlobalEgressIP "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-e2e-egressip-e2e-tests-dataplane-gn-conn-nd-9jn2n", NumberOfIPs: 1, PodSelector: (*v1.LabelSelector)(nil), Status: v1.GlobalEgressIPStatus{Conditions:[]v1.Condition{v1.Condition{Type:"Allocated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2022, time.December, 21, 19, 18, 51, 0, time.Local), Reason:"Success", Message:"Allocated 1 global IP(s)"}}, AllocatedIPs:[]string{"<IP_ADDRESS>"}}
I1221 19:22:07.523742       1 base_controllers.go:167] Releasing previously allocated IPs [<IP_ADDRESS>] for "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-e2e-egressip-e2e-tests-dataplane-gn-conn-nd-9jn2n"
I1221 19:22:07.523890       1 iface.go:287] Deleting iptable egress rules for Namespace "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-e2e-egressip-e2e-tests-dataplane-gn-conn-nd-9jn2n": -p all -m set --match-set SM-GN-CNFI27FCMHZB4CSG5Y4SYEUXW src -m mark --mark 0xC0000/0xC0000 -j SNAT --to <IP_ADDRESS>
I1221 19:22:07.528469       1 ipset.go:349] Running ipset [destroy SM-GN-CNFI27FCMHZB4CSG5Y4SYEUXW]
I1221 19:22:07.552381       1 global_egressip_controller.go:308] Successfully deleted all the iptables/ipset rules for "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-e2e-egressip-e2e-tests-dataplane-gn-conn-nd-9jn2n"

In Cluster2, Globalnet pod logs

I1221 19:18:50.894226       1 service_export_controller.go:152] Processing ServiceExport "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-svc-tcp-check-listener"
I1221 19:18:50.896231       1 service_export_controller.go:183] Creating GlobalIngressIP object e2e-tests-dataplane-gn-conn-nd-9jn2n/test-svc-tcp-check-listener, TargetRef: "ClusterIPService", "test-svc-tcp-check-listener"
I1221 19:18:50.912800       1 global_ingressip_controller.go:138] Processing created e2e-tests-dataplane-gn-conn-nd-9jn2n/test-svc-tcp-check-listener, TargetRef: "ClusterIPService", "test-svc-tcp-check-listener", Status: v1.GlobalIngressIPStatus{Conditions:[]v1.Condition(nil), AllocatedIP:""}
I1221 19:18:50.912848       1 global_ingressip_controller.go:177] Allocated global IP ["<IP_ADDRESS>"] for "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-svc-tcp-check-listener"
I1221 19:18:50.939479       1 base_controllers.go:208] Updated: &v1.GlobalIngressIPStatus{Conditions:[]v1.Condition{v1.Condition{Type:"Allocated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2022, time.December, 21, 19, 18, 50, 939462618, time.Local), Reason:"Success", Message:"Allocated global IP"}}, AllocatedIP:"<IP_ADDRESS>"}
I1221 19:22:07.718134       1 global_ingressip_controller.go:138] Processing deleted e2e-tests-dataplane-gn-conn-nd-9jn2n/test-svc-tcp-check-listener, TargetRef: "ClusterIPService", "test-svc-tcp-check-listener", Status: v1.GlobalIngressIPStatus{Conditions:[]v1.Condition{v1.Condition{Type:"Allocated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2022, time.December, 21, 19, 18, 50, 0, time.Local), Reason:"Success", Message:"Allocated global IP"}}, AllocatedIP:"<IP_ADDRESS>"}
I1221 19:22:07.718207       1 global_ingressip_controller.go:297] Deleting the service "e2e-tests-dataplane-gn-conn-nd-9jn2n"/"submariner-qapolibs46wg6okghgvgf6gljwggtyl7" created by Globalnet controller
I1221 19:22:07.761239       1 service_controller.go:115] Service "e2e-tests-dataplane-gn-conn-nd-9jn2n/submariner-qapolibs46wg6okghgvgf6gljwggtyl7" deleted
I1221 19:22:07.886304       1 service_controller.go:115] Service "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-svc-tcp-check-listener" deleted
I1221 19:22:08.101235       1 service_export_controller.go:192] ServiceExport "e2e-tests-dataplane-gn-conn-nd-9jn2n/test-svc-tcp-check-listener" deleted

Can you run the subctl verify --only connectivity .... command with --verbose option, it will display additional logs. Please attach the full output of subctl verify --verbose ... and the subctl gather ... command when you notice the failures.

I could not reproduce this issue. I created new clusters and submariner worked fine for the new cluters. After I stopped the instances and started them again, there are runtime errors but only have three failures this time:
Summarizing 3 Failures:

[Fail] [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod connects via TCP to the globalIP of a remote service when the pod is not on a gateway and the remote service is not on a gateway [It] should have sent the expected data from the pod to the other pod
github.com/submariner-io/shipyard@v0.13.2/test/e2e/framework/network_pods.go:188

[Fail] [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod matching an egress IP namespace selector connects via TCP to the globalIP of a remote service when the pod is not on a gateway and the remote service is not on a gateway [It] should have sent the expected data from the pod to the other pod
github.com/submariner-io/shipyard@v0.13.2/test/e2e/framework/network_pods.go:188

[Fail] [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod matching an egress IP pod selector connects via TCP to the globalIP of a remote service when the pod is not on a gateway and the remote service is not on a gateway [It] should have sent the expected data from the pod to the other pod
github.com/submariner-io/shipyard@v0.13.2/test/e2e/framework/network_pods.go:188

The output of subctl diagnose all is the same as above
I have attached the full output of subctl verify and subctl gather for the new failures.
submariner-verify.tar.gz
submariner-gather.tar.gz

From the logs, I see that all the three tests failed when the listener (or server) pod was scheduled on the cluster-b-kadm-node-2 of cluster-b.
STEP: Waiting for the listener pod "tcp-check-listenerf6w67" on node "cluster-b-kadm-node-2" to exit, returning what listener sent
Dec 23 12:26:39.519: INFO: Pod "tcp-check-listenerf6w67" on node "cluster-b-kadm-node-2" output:
listening on <IP_ADDRESS>:1234 ...
nc: timeout

When I had a look at the submariner-routeagent pod running on that node, I see that it was not able to discover the CNI Interface on the node which is necessary for some use-cases.
I1223 19:51:42.190846       1 cni_iface.go:72] Interface "lo" has "<IP_ADDRESS>" address
I1223 19:51:42.191128       1 cni_iface.go:72] Interface "ens4" has "<IP_ADDRESS>" address
E1223 19:51:42.191443       1 main.go:115] Error while annotating the node: error annotating node with CNI interface IP: Error retrieving the CNI interface for <IP_ADDRESS>/16: unable to find CNI Interface on the host which has IP from "<IP_ADDRESS>/16"

This error is seen only on the cluster-b-kadm-node-2 node. It looks like the Calico CNI agent on that node took a while to create the CNI Interface, so Submariner route-agent tried to discover the CNIIface it failed (the side effect of which will be some missing rules on the nodes which can cause failures).
Can you try restarting the submariner-routeagent pod on cluster-b-kadm-node-2 node and check if the new route-agent pod shows any similar errors. Once you confirm that there are no errors, you can re-run the subctl verify ... command to re-verify the use-case.
PS: While debugging this issue, noticed that we were not capturing the iptables from the nodes for the Calico CNI. The following PR fixes it https://github.com/submariner-io/subctl/pull/469

@huiyizzz Any more feedback?

@huiyizzz Closing for lack of feedback for now, but if you have more to share we're happy to reopen the issue and discussion.


fea-request: flash sha1 to text file

I would like to see afterward what binary was flashed to the device. Build sha1 could be good unique value for this. This feature would be very useful for CI/automation.
e.g. Use case:

user#1 flash binary to the device
-> daplink write binary sha1 to some text file (e.g. DETAILS.TXT)
user#1 unplug device and give device to user#2
user#2 plug device
user#2 want to verify what binary was flashed
-> read text file and check sha1

@PrzemekWirkus And I have also discussed this, would be really useful for determining when the device is fully flashed as well! This coupled with a "flash count" could be really helpful to be sure all file operations are completed and the target is ready to be reset. The "flash count" wouldn't have to be persistent across power cycles, but it'd be a great indicator in addition to the drive remounting that the flashing process is complete.

There were certain features similiar to this discussed between me and Russ.
For example volatile and non volatile counters available via DETAILS.TXT:

(volatile) Resets since last power cycle.
(nonvolatile) Total reset/power on cycles.

Hard to say if SHA can be provided in bare-metal system (maybe too expensive to have this algorithm inside DAPlink f/w).
I think we just should know DAPlinjk's roadmap, it would be easier to know what is there and maybe give ideas what automation wants.

Sounds like a good feature but need to make sure we don't add too much magic to the filesystem that other DAPLink like builds wont offer. We can add all sorts of features but need to really understand what is a requirement and how best to implement it such that all mbed-like boards can achieve the same status.

Can I suggest SHA2 instead of SHA1? Even if security isn't necessarily critical to this application, SHA1 is in it's retirement and we shouldn't be using it for anything new. cc @sbutcher-arm


use gear router pattern as root trace operate name

Coverage decreased (-1.7%) to 81.081% when pulling f1fe8891fec1930671cb23689da72fc944d4c03c on refactor/router-as-opname into 9f5a8185d4272ff260c7767ffa2e899880309767 on master.


High-level wire implementation, complete with unit tests

See #181
Tracks addresses read from/written to on a high level using maps and vectors, as well as the content at that address, of course.
Let me know if you'd like more tests, though I think I covered everything that we deemed useful at the moment.

I'm guessing the test failures are related to me overwriting the "map" from AvrMath with the class template from C++ (lines 25-27). I'm assuming there isn't as shorthand a way to fix it like the "max" function in lines 7-22.

Rather than removing functionality that is part of the core Arduino system (map), we should use the C++ namespace feature to avoid the name conflict. So, remove line 36 and replace all your references to map with std::map. Then remove lines 24-29.

Rather than removing functionality that is part of the core Arduino system (map), we should use the C++ namespace feature to avoid the name conflict. So, remove line 36 and replace all your references to map with std::map. Then remove lines 24-29.

I tried this and my terminal gets flooded with errors related to the Arduino core map being used incorrectly. I deemed that ifdef patch job necessary at first, but I'll try and play around with namespaces to confirm when I get a chance.

Ah, yes, I see. The existing macro will indiscriminately changes to all your uses of map and that's bad. And, you are right, the best solution would be something like what was done with max(). Absent that, we may need to come up with something else. This requires more thought.

Let's try to rewrite this to use something other than a map.

@ianfixes, With a couple minor edits, this is an implementation of Wire that I'd like to have you consider. Please let me know your thoughts. Thanks!

Looks good. Thanks!

I appreciate the work that has gone into this, especially the unit tests that cover the added functionality.  @jgfoster let me know when your comments are addressed

@ianfixes, if you are satisfied with the comments in the test file, then I'm happy to have it merged. Thanks!

Thanks for taking the time to add clarity to those test cases!


Perfect mask, but worms not identified.

I am trying to optimize parameters for recording a full 10 cm plate for 1 hr with images taken every 10 s. So the worms are quite small and the distance between a given worm in frame N and frame N + 1 is quite large.  The parameters I have now work quite well for generating the mask, but after analysis the worms aren't recognized and identified as worms and thus no tracks are created. I am only analyzing 5-8 worms per plate, so I don't think crowding is an issue. I suspect I will have to edit some of the hidden parameters (perhaps traj_max_allowed_dist?). Do you have any suggestions for where to start?
I have a 76 frame (235 MB) AVI that I can send for testing, if you'd like.

You are right it is probably that parameter. We normally used quite high frame rate and I have a hard threshold to reject large jumps between frames. It is probably easier if you can share the video.

I realize that with such a low frame rate and resolution many of the measured behavioral features will be unreliable, but I would still like to get tracks and midpoint velocity metrics. Video attached.
test.zip

I was able to get it to recognize worms using the attached parameters, but when i go to get features all of the worms have frac_valid_skels == 0, so the features aren't calculated. It also errors out if I have "Use manually edited trajectories?" set to True (error is printed below). I'm not sure what the issue could be...
Extracting features summary. File 0 of 1 done. Total time: 0:00:00   Traceback (most recent call last):     File "/Users/njwheeler/software/miniconda3/envs/tierpsy/lib/python3.6/site-packages/tierpsy/gui/AnalysisProgress.py", line 62, in run     self.worker_fun(**self.worker_args)   File "/Users/njwheeler/software/miniconda3/envs/tierpsy/lib/python3.6/site-packages/tierpsy/summary/collect.py", line 83, in calculate_summaries     df = summary_func(fname) ValueError: No objects to concatenate
parameters.zip

I was able to get it to recognize worms using the attached parameters, but when i go to get features all of the worms have frac_valid_skels == 0, so the features aren't calculated. It also errors out if I have "Use manually edited trajectories?" set to True (error is printed below). I'm not sure what the issue could be...
Extracting features summary. File 0 of 1 done. Total time: 0:00:00 Traceback (most recent call last): File "/Users/njwheeler/software/miniconda3/envs/tierpsy/lib/python3.6/site-packages/tierpsy/gui/AnalysisProgress.py", line 62, in run self.worker_fun(**self.worker_args) File "/Users/njwheeler/software/miniconda3/envs/tierpsy/lib/python3.6/site-packages/tierpsy/summary/collect.py", line 83, in calculate_summaries df = summary_func(fname) ValueError: No objects to concatenate
parameters.zip

Hello,
The issue seems to be really that the worms move too much between frames traj_max_allowed_dist to 500 and it seems to be working but i am not sure how reliable is the trajectory linkage at that low frame rate.
The real problem is that with that resolution the skeletonization algorithm will simply fail. We investigate that in the Fig S3 on the Nat Meth paper. Since must of the features rely on the presence of an skeleton, the only once that will be calculate are the blob features (blob_*). This include area, perimeter, length, width, solidity, compactness and the hu moments, and its derivatives. However, it does not include speed or orientation, neither information about the path.
It would not be too far fetch to modify the code (at least tierpsy features) to calculate more sophisticated features without the skeletonization (@aexbrown suggest it at some moment), however it is definitely not in the current pipeline. @luigiferiani do you know if somebody would be interested?
Cheers
Avelino

We're having a Tierpsy hackathon in mid December and improving the support of low res videos is on the list of things to work on.  So we might have a significant update in December or January that will help but no guarantees since the list of things for the hackathon is already pretty long.

I understand, thanks for looking into it.
In the end, for this assay I'm only interested in the paths, and I can probably get what I need from the HDF5 results now that the worms are correctly identified. I assume that the trajectory/coordinate data is there even if it doesn't pass tests for skeletonization.

In that case you can do something like this:
import pandas as pd
fname = 'PATH_TO_featuresN.hdf5'
fid = pd.HDFStore(fname, 'r')
trajectories_data = fid['trajectories_data']

delta_time = 1

velocities = []
for w_index, w_data in trajectories_data.groupby('worm_index_joined'):
    x = w_data['coord_x']
    y = w_data['coord_y']
    worm_velocity = np.sqrt((x[:-delta_time] - x[delta_time:])**2 +
y[:-delta_time] - y[delta_time:])**2)
    velocities.append(worm_velocity)

velocities_stats = np.percentiles(np.concatenate(velocities), [10, 50, 90])

Cheers
AEJ
On Fri, Nov 22, 2019 at 3:01 PM Nic Wheeler<EMAIL_ADDRESS>wrote:

I understand, thanks for looking into it.
In the end, for this assay I'm only interested in the paths, and I can
probably get what I need from the HDF5 results now that the worms are
correctly identified. I assume that the trajectory/coordinate data is there
even if it doesn't pass tests for skeletonization.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ver228/tierpsy-tracker/issues/65?email_source=notifications&email_token=AB72CUGEYE7LDTFMW7PNT3TQU7X4DA5CNFSM4JQFXL42YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE54DRY#issuecomment-557564359,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB72CUFRPXH3ISGLTZCQJXTQU7X4DANCNFSM4JQFXL4Q
.

In that case you can do something like this:
import pandas as pd
fname = 'PATH_TO_featuresN.hdf5'
fid = pd.HDFStore(fname, 'r')
trajectories_data = fid['trajectories_data']

delta_time = 1

velocities = []
for w_index, w_data in trajectories_data.groupby('worm_index_joined'):
    x = w_data['coord_x']
    y = w_data['coord_y']
    worm_velocity = np.sqrt((x[:-delta_time] - x[delta_time:])**2 +
y[:-delta_time] - y[delta_time:])**2)
    velocities.append(worm_velocity)

velocities_stats = np.percentiles(np.concatenate(velocities), [10, 50, 90])


Extremely slow after every ~51 minutes

ATS 7.1.2 (ATS 7.1.1 has same issue).
After each ~51 minutes, ATS processess suddenly drops memory usage and that has strong effect on response times.

ATS is used in reverse proxy mode and we run regular tests (never-cache in cache.config) against test URL on proxied server(s). This test GET is ran as HAproxy health check after ~15 seconds for two local HAproxy backends which both pass requests to single local ATS. Quite complex setup, but point here being that tests run frequently and give information about ATS response times over long period.
Test response size is only 609 bytes. Usually response time fluctuates around 200-300ms. Here is the output of tests log that contains all the extremely slow responses:
2018-01-31T02:04:05.753130+00:00 7827
2018-01-31T02:04:05.753133+00:00 32288
2018-01-31T02:04:05.753135+00:00 29907
2018-01-31T02:04:05.753140+00:00 27643
2018-01-31T02:04:05.753144+00:00 25320
2018-01-31T02:55:37.653872+00:00 24587
2018-01-31T02:55:37.653875+00:00 25135
2018-01-31T02:55:37.653877+00:00 19670
2018-01-31T02:55:37.653879+00:00 20322
2018-01-31T03:46:59.615018+00:00 37404
2018-01-31T03:46:59.615021+00:00 36758
2018-01-31T03:46:59.615023+00:00 12769
2018-01-31T03:46:59.615045+00:00 12090
2018-01-31T03:46:59.615047+00:00 7905
2018-01-31T03:46:59.615049+00:00 32639
2018-01-31T03:46:59.615051+00:00 7231
2018-01-31T03:46:59.615053+00:00 32167
2018-01-31T03:54:19.921082+00:00 30047
2018-01-31T03:54:19.921085+00:00 29497
2018-01-31T03:54:19.921092+00:00 24995
2018-01-31T03:54:19.921094+00:00 25833
2018-01-31T04:45:42.015852+00:00 23138
2018-01-31T04:45:42.015855+00:00 23654
2018-01-31T04:45:42.015857+00:00 18541
2018-01-31T04:45:42.015860+00:00 18928
2018-01-31T05:37:04.012258+00:00 24042
2018-01-31T05:37:04.012260+00:00 8563
2018-01-31T05:37:04.012263+00:00 33162
2018-01-31T05:37:04.012265+00:00 28331
2018-01-31T05:37:04.012269+00:00 19311
2018-01-31T06:28:25.914533+00:00 30438
2018-01-31T06:28:25.914535+00:00 24839
2018-01-31T06:28:25.914540+00:00 25882
2018-01-31T06:28:25.914542+00:00 20251
2018-01-31T07:19:47.829894+00:00 7438
2018-01-31T07:19:47.829905+00:00 33133
2018-01-31T07:19:47.829910+00:00 31439
2018-01-31T07:19:47.829931+00:00 28387
2018-01-31T07:19:47.829933+00:00 26775
2018-01-31T08:10:59.574120+00:00 35405
2018-01-31T08:10:59.574122+00:00 10067
2018-01-31T08:10:59.574124+00:00 28619
2018-01-31T08:10:59.574128+00:00 30758
2018-01-31T08:11:09.579351+00:00 24121
The question is: why does ATS regularily slow down? Are there some internal management jobs done, that use same single queue as requests coming in and then requests are blocked in queue until internal processes finish?
Output from X-Debug plugin:
X-Milestones: PLUGIN-TOTAL=0.000022228, PLUGIN-ACTIVE=0.000022228, DNS-LOOKUP-END=6.195555960, DNS-LOOKUP-BEGIN=6.195555960, CACHE-OPEN-WRITE-END=6.255239371, CACHE-OPEN-WRITE-BEGIN=6.195555960, CACHE-OPEN-READ-END=6.195555960, CACHE-OPEN-READ-BEGIN=0.000046346, SERVER-READ-HEADER-DONE=6.488958343, SERVER-FIRST-READ=6.488958343, SERVER-BEGIN-WRITE=6.255239371, SERVER-CONNECT-END=6.255239371, SERVER-CONNECT=6.255239371, SERVER-FIRST-CONNECT=6.255239371, UA-BEGIN-WRITE=6.489539157, UA-READ-HEADER-DONE=0.000000000, UA-FIRST-READ=0.000000000, UA-BEGIN=0.000000000

Do you know if it is going to disk or RAM for cache?  What type of disks are you using, ssd or rotational?  If this is still an issue please reopen.

This definitely looks like a disk issue of some sort, are you running the cache on the filesystem or RAW devices ?


Throw error when SP URL is null

While testing a command I used a variable to store my site URL so it didn't clutter my command args too much. However, I was getting a strange error as shown below:

This error was due to the fact that I made a typo and --webUrl was null (because the variable I specified didn't exist).
This is because the validator we are using is as follows:
https://github.com/pnp/cli-microsoft365/blob/3152fc710914e28624817732724d906de2630df0/src/utils/validation.ts#L349-L360
In the command (and other commands) we validate it like this:
https://github.com/pnp/cli-microsoft365/blob/3152fc710914e28624817732724d906de2630df0/src/m365/spo/commands/file/file-add.ts#L139-L142
Shouldn't we output an error message when the value is null? The current error Error: false doesn't help at all.

I even got something worse 😉

I even got something worse 😉

Yes, that's the importance of #initTypes right now. We should type all options as much as possible. If we don't, Minimist will decide what the type of variable is and when you pass --webUrl null, it translates to --webUrl which is a flag. In that case, it will get the value true, and .indexOf is not a function of a boolean value. We have not paid enough attention to this in the past.

awww right.  Bummer, what if we create an issue to fix this problem globally and add the missing #initTypes?

Bummer, what if we create an issue to fix this problem globally and add the missing #initTypes?

It's a lot of work since this applies to every single option in every single command. The same thing will happen when you run entra m365group get --id $iDoNotExist.
I know this is a known issue, and I try to fix this in every command I work on.
The good news is, with the implementation of ZOD, this will all be fixed because every option will be typed automatically. So I suggest that we just deal with it right now. Otherwise, we have to fix this in 90% of all commands right now, and we still have to implement ZOD in every command, which will fix it anyway. So that's kind of (a lot of) double work.

Can I work on it?


Bug, please fixe issue asap for angular 8: An accessor cannot be declared in an ambient context.

ERROR in node_modules/@antv/g2/lib/chart/controller/annotation.d.ts:14:9 - error TS1086: An accessor cannot be declared in an ambient context.
14 get name(): string;
node_modules/@antv/g2/lib/chart/controller/base.d.ts:20:18 - error TS1086: An accessor cannot be declared in an ambient context.

20 abstract get name(): string;

node_modules/@antv/g2/lib/chart/controller/gesture.d.ts:18:9 - error TS1086: An accessor cannot be declared in an ambient context.
18 get name(): string;
node_modules/@antv/g2/lib/chart/event.d.ts:19:9 - error TS1086: An accessor cannot be declared in an ambient context.

19 get target(): IShape;

node_modules/@antv/g2/lib/chart/event.d.ts:21:9 - error TS1086: An accessor cannot be declared in an ambient context.
21 get event(): any;
node_modules/@antv/g2/lib/chart/event.d.ts:23:9 - error TS1086: An accessor cannot be declared in an ambient context.

23 get x(): number;
~
node_modules/@antv/g2/lib/chart/event.d.ts:25:9 - error TS1086: An accessor cannot be declared in an ambient context.

25 get y(): number;
~
node_modules/@antv/g2/lib/chart/event.d.ts:27:9 - error TS1086: An accessor cannot be declared in an ambient context.

27 get clientX(): number;

node_modules/@antv/g2/lib/chart/event.d.ts:29:93m9 - error TS1086: An accessor cannot be declared in an ambient context.
29 get clientY(): number;
node_modules/@antv/g2/lib/util/bbox.d.ts:17:9 - error TS1086: An accessor cannot be declared in an ambient context.

17 get minX(): number;
~~~~
node_modules/@antv/g2/lib/util/bbox.d.ts:18:9 - error TS1086: An accessor cannot be declared in an ambient context.

18 get maxX(): number;
~~~~
node_modules/@antv/g2/lib/util/bbox.d.ts:19:9 - error TS1086: An accessor cannot be declared in an ambient context.

19 get minY(): number;
~~~~
node_modules/@antv/g2/lib/util/bbox.d.ts:20:9 - error TS1086: An accessor cannot be declared in an ambient context.

20 get maxY(): number;
~~~~
node_modules/@antv/g2/lib/util/bbox.d.ts:21:9 - error TS1086: An accessor cannot be declared in an ambient context.

21 get tl(): Point;
~~
node_modules/@antv/g2/lib/util/bbox.d.ts:22:9 - error TS1086: An accessor cannot be declared in an ambient context.

22 get tr(): Point;
~~
node_modules/@antv/g2/lib/util/bbox.d.ts:23:9 - error TS1086: An accessor cannot be declared in an ambient context.

23 get bl(): Point;
~~
node_modules/@antv/g2/lib/util/bbox.d.ts:24:9 - error TS1086: An accessor cannot be declared in an ambient context.

24 get br(): Point;
~~
node_modules/@antv/g2/lib/util/bbox.d.ts:25:9 - error TS1086: An accessor cannot be declared in an ambient context.

25 get top(): Point;
~~~
node_modules/@antv/g2/lib/util/bbox.d.ts:26:9 - error TS1086: An accessor cannot be declared in an ambient context.

26 get right(): Point;
~~~~~
node_modules/@antv/g2/lib/util/bbox.d.ts:27:9 - error TS1086: An accessor cannot be declared in an ambient context.

27 get bottom(): Point;
~~~~~~
node_modules/@antv/g2/lib/util/bbox.d.ts:28:9 - error TS1086: An accessor cannot be declared in an ambient context.

28 get left(): Point;
~~~~
node_modules/@antv/g2plot/lib/dependents.d.ts:14:8 - error TS1192: Module '"D:/Guru/GURU_UI/node_modules/@antv/component/lib/tooltip/css-const"' has no default export.

14 import TooltipCssConst from '@antv/component/lib/tooltip/css-const';

node_modules/@antv/g2plot/lib/util/bbox.d.ts:14:9 - error TS1086: An accessor cannot be declared in an ambient context.
14 get minX(): number;
node_modules/@antv/g2plot/lib/util/bbox.d.ts:15:9 - error TS1086: An accessor cannot be declared in an ambient context.

15 get maxX(): number;

node_modules/@antv/g2plot/lib/util/bbox.d.ts:16:9 - error TS1086: An accessor cannot be declared in an ambient context.
16 get minY(): number;
node_modules/@antv/g2plot/lib/util/bbox.d.ts:17:9 - error TS1086: An accessor cannot be declared in an ambient context.

17 get maxY(): number;

node_modules/@antv/g2plot/lib/util/bbox.d.ts:18:9 - error TS1086: An accessor cannot be declared in an ambient context.
18 get tl(): {
~~
node_modules/@antv/g2plot/lib/util/bbox.d.ts:22:9 - error TS1086: An accessor cannot be declared in an ambient context.
22 get tr(): {
~~
node_modules/@antv/g2plot/lib/util/bbox.d.ts:26:9 - error TS1086: An accessor cannot be declared in an ambient context.
26 get bl(): {
~~
node_modules/@antv/g2plot/lib/util/bbox.d.ts:30:9 - error TS1086: An accessor cannot be declared in an ambient context.
30 get br(): {

Reference: http://www.webyuweb.com/a/renwu/19638.html
Our project using angular8, and can not upgraded to angular 9 as some dependence.
Please just spend 2 minus, and  update several G2 coding to fxied it.   Thanks!

please fixe issue asap for angular 8: An accessor cannot be declared in an ambient context

我也是这个问题  楼主你解决了嘛 ？

see here: https://stackoverflow.com/questions/60092642/ts1086-an-accessor-cannot-be-declared-in-ambient-context

see here: https://stackoverflow.com/questions/60092642/ts1086-an-accessor-cannot-be-declared-in-ambient-context


[Connector-V2] [ElasticSearch] Add ElasticSearch Source/Sink Factory

Purpose of this pull request

Add ElasticSearch Source/Sink Factory
Add Common Schema Option

Check list

[ ] Code changed are covered with tests, or it does not need tests for reason:
[ ] If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
[ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
[ ] If you are contributing the connector code, please check that the following files are updated:

Update change log that in connector document. For more details you can refer to connector-v2
Update plugin-mapping.properties and add new connector information in it
Update the pom file of seatunnel-dist

@ashulin @TyrantLucifer @hailin0 PTAL. Thanks!

merge https://github.com/apache/incubator-seatunnel/pull/3328/files

Should can close this pr?


Fix missing file

How did this work beforehand?

We didn't use the extra script beforehand, I simply forgot to include that in the latest PR: https://github.com/nobleo/balena-chrony/pull/3


UWP Support

Will this SDK ever support UWP, and if not, why?
On a similar track, why was a UWP/WinRT driver not created that supports this variant of the Kinect in a similar manner to the previous generation?  We would have been able to use our existing code as is, no?

Thanks for the request @Noemata , we do not have plans to support UWP today. Last //build conference we started IWANTKINECT survey based on those results we prioritized cross-platform SDK that works with Win32 and Linux.
We are working on getting feedback system available that can inform us better on overall community interests/needs so we can continue prioritizing development. I will post a link here as soon as that is available.

The waffling on platform support initiatives have been painful to both watch and endure over the years.  All of a sudden, UWP gets the bottom rung.  WPF suffered a similar demotion when UWP first game out.  And now Microsoft is somehow intent on resurrecting Win32 anew??  The Silverlight saga should have been a pretty good object lesson.    Which is it now, WPF, Winforms, UWP, DirectX + Win32 (sarcasm)?  My point being, UWP support cannot slip.  There will be no third chance since this time around we'll be at strike three.

Hi @Noemata. Thanks for your feedback.
We're are collecting community suggestions on the future of this project at our new user feedback forum. I just added your request there, and you can upvote it to voice your support.
The Azure Kinect DK is still a developer kit and we are looking for specific ways that you want to use the device. Please comment on the user feedback forum since this will help us make decisions about what platforms and features will be most useful going forward.

Curiously, UWP support is one of the more requested items at the user feedback forum.  Has anyone noticed?  Or is there a vote threshold required before things get attention.  I can arrange for an entire college student body that needs this to put in a vote.


Validation of identity objects in /upload and PUT /document/identity

Using image docker.sunet.se/dc4eu/apigw:apiv28
When uploading a document or using the PUT /document/identity call for adding identities, it is possible to submit invalid identity objects like this:
"identities": [
        {
            "name": 2
        }
    ]

We propose that all identity objects in the array should be validated to make sure that all required attributes are set and the types are matching.

should be fixed with, https://github.com/dc4eu/vc/commit/e1ea46f3d3bd5e1393e1686af6d9c465efd1d4ca

I tried with the same invalid identity object as i posted before, but still get no validation errors. The objects with the submitted identities array don't seem to be validated

I am sorry, i guess i didn't outline the problem precisely enough.
What i meant is that the identity objects in the array are not validated. So if i submit a request like
{
    "authentic_source": "data_existent_in_db",
    "document_type": "data_existent_in_db",
    "document_id": "data_existent_in_db",
    "identities": [
        {
            "name": 2
        }
    ]
}

This just works fine even though the submitted identity object is obviously not valid. Of course i get a validation error as well if i leave out any of the top-level attributes like document_id, etc.
Same behaviour with the upload call. So our request is that that every submitted identity object should be validated


DWDWeather is overriding deprecated methods on an instance of WeatherEntity, this is not valid and will be unsupported from Home Assistant 2023.1

Version of home_assistant

HA 2022.07.0 Running on the latest HA OS
Version of the custom_component

1.2.22
Describe the bug
A clear and concise description of what the bug or error is and when it happens.
Log warning about deprecating feature. This „bug“ is low priority and more a hint from my side from the logs, coming with the newest HA release.
Debug log

Logger: homeassistant.components.weather
Source: components/weather/__init__.py:308
Integration: Weather (documentation, issues)
First occurred: 22:04:44 (1 occurrences)
Last logged: 22:04:44

custom_components.dwd_weather.weather::DWDWeather is overriding deprecated methods on an instance of WeatherEntity, this is not valid and will be unsupported from Home Assistant 2023.1. Please report it to the custom component author.

Is anyone still having this message? I am unable to reproduce the error message.

Yes, I am having this message each time I restart Home Assistant.
It seems that it does get logged only once when HA reloads the component.
I did right now a restart, here is a copy of the log entry:
Logger: homeassistant.components.weather
Source: components/weather/__init__.py:198
Integration: Weather (documentation, issues)
First occurred: 13:25:14 (1 occurrences)
Last logged: 13:25:14

custom_components.dwd_weather.weather::DWDWeather is overriding deprecated methods on an instance of WeatherEntity, this is not valid and will be unsupported from Home Assistant 2023.1. Please report it to the custom integration author.

This is the actual setup I use:
DWD V1.2.22
Home Assistant 2022.8.7
Supervisor 2022.08.5
Operating System 8.5
Frontend 20220802.0 - latest

Thanks for your quick answer. I'll fix this today

Update is out, this should resolve the message.


ISSUE:  Lost Notifications after Core update 2024.5.5

After yesterday's core update (core_2024.5.5) I no longer get notifications when my door lock codes are entered.  Also lost an auto alarmo disarm automation based on an event.  I noticed that listening to events under keymaster_lock_state_changed
no longer show any events up entering coed into my lock

As another person posted, I also have the pin status stuck on adding, and one of them stuck on deleting.

I am running HAOS
zwave-js-ui: 9.13.3
zwave-js: 12.9.1
Core 2024.6.1
Supervisor 2024.06.0
Operating System 12.3
Frontend 20240605.0
Lock type. Yale assure

Dupe #369


[CodeCompletion] '#file', '#line', et al. after '#'

Implement completion for #file, #line, et al. after # in expression context
Polymorphic type annotation for #file, #line, et al. depending on context types
Suggest #selector and #keyPath only if the context type matches
Add CompletionLookup::CurrModule instance field to avoid recurring computation for CurrDeclContext->getParentModule()

rdar://problem/47169238

@swift-ci Please smoke test

@swift-ci Please smoke test

this was always what it was supposed to do, right?

After # we used to unconditionally emit them.


[SSL: WRONG_VERSION_NUMBER]

Thank you for the package, it was helpful!
I ran into an issue where when I run this setup with an outlook account I received the error:
[SSL: WRONG_VERSION_NUMBER]
I was able to correct the issue by changing this:
with smtplib.SMTP_SSL(
smtp_server, smtp_port, context=ssl.create_default_context()
) as email:
email.login(sender_email, email_password)
email.sendmail(sender_email, receiver_email, email_message)
TO THIS:
with smtplib.SMTP(
smtp_server, smtp_port
) as email:
email.starttls(context=ssl.SSLContext(ssl.PROTOCOL_TLS))
email.login(sender_email, email_password)
email.sendmail(sender_email, receiver_email, email_message)
I would like to provide the solution in the form of a commit, I will work on doing this :)
Also, I noticed you using port 465 as the default which is fine but just curious if you knew about port 587. Without typing it all out here is a great resource about it vs port 465:
https://sendgrid.com/blog/whats-the-difference-between-ports-465-and-587/
for this, I implemented starttls as well as mentioned in the article above :)

I'm glad it was helpful. It might be worth it to create a lookup table for different email providers or simply let the user pass in their own SSL instance to make this a more compatible library.


Disable agenda popup if there are no cards

Fixes THRONETEKI-112

This actually should fix itself once you deploy the corresponding server side changes (comes from bannerCards being missing since server side serialization hasn't been deployed) but it may be a good safety check regardless.


Improve bounded count check

An ISeq is always an IPersistentCollection but the converse is not true.
Vectors and maps are not ISeq and for instances of such, the current protocol implementation doesn't check the limit on the whole collection, but only on its items.

LGTM


tmux send-keys command cant press enter in MultiAdmin

Start the game server via MA in tmux session
Wait for the game server to load
We send the command with pressing enter via the tmux command: tmux send-keys -t SCPSL:0 "/pm reload" Enter
Result: the command cannot be sent without forcibly pressing Enter via ssh

Tmux version: tmux 3.0a
MA Version: <IP_ADDRESS>-alpha
use_new_input_system: false cant fix problem

Did this happen in the earlier versions?

Did this happen in the earlier versions?

On the release version of MultiAdmin, this was not the case

On the release version of MultiAdmin, this was not the case

On version <IP_ADDRESS>-alpha is not playable

On version <IP_ADDRESS>-alpha is not playable

Alright, I'll look into this later today, thanks for reporting 👍

Alright, I'll look into this later today, thanks for reporting 👍

Unhandled Exception:
System.ArgumentOutOfRangeException: Value must be positive and below the buffer height.
Parameter name: top
  at System.TermInfoDriver.SetCursorPosition (System.Int32 left, System.Int32 top) [0x0003e] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.TermInfoDriver.WriteSpecialKey (System.ConsoleKeyInfo key) [0x00062] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.TermInfoDriver.Echo (System.ConsoleKeyInfo key) [0x0001d] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.TermInfoDriver.ReadKey (System.Boolean intercept) [0x00011] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.ConsoleDriver.ReadKey (System.Boolean intercept) [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Console.ReadKey (System.Boolean intercept) [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at MultiAdmin.ServerIO.InputHandler.GetInputLineOld (MultiAdmin.Server server, System.Threading.CancellationToken cancellationToken) [0x0008a] in <3f42baba8ce0450ea0673582420ebb59>:0
  at MultiAdmin.ServerIO.InputHandler.Write (MultiAdmin.Server server, System.Threading.CancellationToken cancellationToken) [0x0014b] in <3f42baba8ce0450ea0673582420ebb59>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__7_1 (System.Object state) [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context (System.Object state) [0x00007] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserve
SyncCtx) [0x00071] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx)
 [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00021] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.ArgumentOutOfRangeException: Value must be positive and below the buffer height.
Parameter name: top
  at System.TermInfoDriver.SetCursorPosition (System.Int32 left, System.Int32 top) [0x0003e] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.TermInfoDriver.WriteSpecialKey (System.ConsoleKeyInfo key) [0x00062] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.TermInfoDriver.Echo (System.ConsoleKeyInfo key) [0x0001d] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.TermInfoDriver.ReadKey (System.Boolean intercept) [0x00011] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.ConsoleDriver.ReadKey (System.Boolean intercept) [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Console.ReadKey (System.Boolean intercept) [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at MultiAdmin.ServerIO.InputHandler.GetInputLineOld (MultiAdmin.Server server, System.Threading.CancellationToken cancellationToken) [0x0008a] in <3f42baba8ce0450ea0673582420ebb59>:0
  at MultiAdmin.ServerIO.InputHandler.Write (MultiAdmin.Server server, System.Threading.CancellationToken cancellationToken) [0x0014b] in <3f42baba8ce0450ea0673582420ebb59>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__7_1 (System.Object state) [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context (System.Object state) [0x00007] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserve
SyncCtx) [0x00071] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx)
 [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00021] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
st

Unhandled Exception:
System.ArgumentOutOfRangeException: Value must be positive and below the buffer height.
Parameter name: top
  at System.TermInfoDriver.SetCursorPosition (System.Int32 left, System.Int32 top) [0x0003e] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.TermInfoDriver.WriteSpecialKey (System.ConsoleKeyInfo key) [0x00062] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.TermInfoDriver.Echo (System.ConsoleKeyInfo key) [0x0001d] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.TermInfoDriver.ReadKey (System.Boolean intercept) [0x00011] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.ConsoleDriver.ReadKey (System.Boolean intercept) [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Console.ReadKey (System.Boolean intercept) [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at MultiAdmin.ServerIO.InputHandler.GetInputLineOld (MultiAdmin.Server server, System.Threading.CancellationToken cancellationToken) [0x0008a] in <3f42baba8ce0450ea0673582420ebb59>:0
  at MultiAdmin.ServerIO.InputHandler.Write (MultiAdmin.Server server, System.Threading.CancellationToken cancellationToken) [0x0014b] in <3f42baba8ce0450ea0673582420ebb59>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__7_1 (System.Object state) [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context (System.Object state) [0x00007] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserve
SyncCtx) [0x00071] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx)
 [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00021] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.ArgumentOutOfRangeException: Value must be positive and below the buffer height.
Parameter name: top
  at System.TermInfoDriver.SetCursorPosition (System.Int32 left, System.Int32 top) [0x0003e] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.TermInfoDriver.WriteSpecialKey (System.ConsoleKeyInfo key) [0x00062] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.TermInfoDriver.Echo (System.ConsoleKeyInfo key) [0x0001d] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.TermInfoDriver.ReadKey (System.Boolean intercept) [0x00011] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.ConsoleDriver.ReadKey (System.Boolean intercept) [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Console.ReadKey (System.Boolean intercept) [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at MultiAdmin.ServerIO.InputHandler.GetInputLineOld (MultiAdmin.Server server, System.Threading.CancellationToken cancellationToken) [0x0008a] in <3f42baba8ce0450ea0673582420ebb59>:0
  at MultiAdmin.ServerIO.InputHandler.Write (MultiAdmin.Server server, System.Threading.CancellationToken cancellationToken) [0x0014b] in <3f42baba8ce0450ea0673582420ebb59>:0
  at System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__7_1 (System.Object state) [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context (System.Object state) [0x00007] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserve
SyncCtx) [0x00071] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx)
 [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00021] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00074] in <d13c8b563008422a8c5aaec0a74089cc>:0
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <d13c8b563008422a8c5aaec0a74089cc>:0
st

Okay, so a temporary workaround to this issue should be setting hide_input to true, I'll continue to look into the cause of this issue and see if I can implement a more proper workaround into MultiAdmin. Thank you for the stacktrace.

Okay, so a temporary workaround to this issue should be setting hide_input to true, I'll continue to look into the cause of this issue and see if I can implement a more proper workaround into MultiAdmin. Thank you for the stacktrace.

Actually, you mentioned that use_new_input_system: false doesn't work, does use_new_input_system: true work at all?

Actually, you mentioned that use_new_input_system: false doesn't work, does use_new_input_system: true work at all?

Yes, it works, but the problem persists

Yes, it works, but the problem persists

Hmmm... I'm wondering if this might actually be an issue with Mono itself? I'll look into it further and let you know.

Hmmm... I'm wondering if this might actually be an issue with Mono itself? I'll look into it further and let you know.

use_new_input_system: true
hide_input: true
cant send command via tmux

use_new_input_system: true
hide_input: true
cant send command via tmux

use_new_input_system: false
hide_input: true
I also can't send a command via tmux

use_new_input_system: false
hide_input: true
I also can't send a command via tmux

This issue should be resolved at this point by usingconsole_input_system: Original on MultiAdmin v<IP_ADDRESS> and up. Please re-open this if the issue persists.


Improve the install_deps.sh, or make the documentc around it better.

Is your feature request related to a problem? Please describe.
It is quite frustrated to install GraphScope using the install_deps script. Either we make the script more robust or make the document more complete around it.
Describe the solution you'd like
Here is how we made it work:

Requirement: Ubuntu 20.x

sudo apt update
sudo apt-get install -y llvm-11-dev lld-11 clang-11 libre2-dev libc-ares-dev
export LLVM11_HOME=/usr/lib/llvm-11
sudo ln -fs /usr/lib/llvm-11/bin/clang /usr/bin/
sudo ln -fs /usr/lib/llvm-11/bin/clang++ /usr/bin/
sudo ln -fs /usr/lib/llvm-11/bin/clang-cpp /usr/bin/

git clone https://github.com/alibaba/GraphScope.git
cd GraphScope/
./scripts/install_deps.sh --dev
rustup component add rustfmt

source $HOME/.graphscope_env
make graphscope

Describe alternatives you've considered
None.
Additional context
None.

Thanks for you suggestion.  Could you tell us what problem you encounter when using install_deps script, that would give us a direction to revise the script and enhance the document.

You could reproduce it by starting a standard EC2 instance on AWS with Ubuntu 20.x. Just to give you more context, here are the extra things we need to do besides install_deps.sh:

re2, c-ares are not auto installed and CMake folder settings are missing.
clang/llvm is not auto installed.
LLVM11_HOME is expected by one of the components build.xml. I would expect that to use latest llvm available or at least use clang,clang++ relative to LLVM11_HOME.
Had to run rustup component add rustfmt in the middle

Thanks! @parkerzf


Classes for LSC measurements

Instead of working with these large dicts, we should have classes for LSC measurements. In the future this will help us automate the data aquisition from the LSC
Proposed interface:
background_sample = LSCSample(activity=1 * ureg.Bq, name="background)
my_LSC_sample = LSCSample(activity=20 * ureg.Bq, name="a name")

my_LSC_sample.substract_background(background_sample)

For a whole bubbler:
baby_sample_1 = BubblerSample(
    vial_1=LSCSample(activity=20 * ureg.Bq),
    vial_2=LSCSample(activity=0 * ureg.Bq),
    vial_3=LSCSample(activity=12 * ureg.Bq),
    vial_4=LSCSample(activity=1 * ureg.Bq),
    time="SOMEDATEANDTIME",
)

baby_sample_1.substract_background(background_sample)

For a whole run (several bubblers samples):
baby_run_12 = Run(baby_sample_1, baby_sample_2, ...)

baby_run_12.plot(...)
baby_run_12.do_something(...)

Completed in #17


Show SSL library info in About dialog and client / server info

SSL library is a core component to SoftEther VPN and its version is also useful to users.
This PR makes the lib version visible in:

Client Manager / Server Manager: About dialog
Server Manager: About this VPN Server
vpncmd - tools: About
vpncmd - server: ServerInfoGet
vpncmd - client: VersionGet

If other TLS libraries are supported in the future, this info can be more useful.
For privacy reasons, this info will not be transferred between a client and a server, only on either side.

I accidentally include Mayaqua/CMakeLists.txt where I added two standard libs for win32 builds. These libs are automatically added by vcpkg but need to be manually added when building with OpenSSL from other sources.
Should I leave or remove them?

The original idea should be credited to some comment in the issues. But I can't find it.

I accidentally include Mayaqua/CMakeLists.txt where I added two standard libs for win32 builds. These libs are automatically added by vcpkg but need to be manually added when building with OpenSSL from other sources. Should I leave or remove them?

Moving the change into a separate commit would be ideal.

Thank you. I shouldn't use the OPENSSL_VERSION_TEXT macro because it's fixed at build time.
Now with the version function it shows the correct version at runtime.
CMake change is in a separate commit.

macOS build fails.... Why is that?
[97/102] Linking C shared library libcedar.dylib
FAILED: libcedar.dylib
: && /Applications/Xcode_13.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -fsigned-char -O2 -g -DNDEBUG -isysroot /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -mmacosx-version-min=11.7 -dynamiclib -Wl,-headerpad_max_install_names  -o libcedar.dylib -install_name @rpath/libcedar.dylib src/Cedar/CMakeFiles/cedar.dir/Account.c.o src/Cedar/CMakeFiles/cedar.dir/Admin.c.o src/Cedar/CMakeFiles/cedar.dir/AzureClient.c.o src/Cedar/CMakeFiles/cedar.dir/Bridge.c.o src/Cedar/CMakeFiles/cedar.dir/BridgeUnix.c.o src/Cedar/CMakeFiles/cedar.dir/BridgeWin32.c.o src/Cedar/CMakeFiles/cedar.dir/CM.c.o src/Cedar/CMakeFiles/cedar.dir/Cedar.c.o src/Cedar/CMakeFiles/cedar.dir/Client.c.o src/Cedar/CMakeFiles/cedar.dir/Command.c.o src/Cedar/CMakeFiles/cedar.dir/Connection.c.o src/Cedar/CMakeFiles/cedar.dir/Console.c.o src/Cedar/CMakeFiles/cedar.dir/DDNS.c.o src/Cedar/CMakeFiles/cedar.dir/Database.c.o src/Cedar/CMakeFiles/cedar.dir/EM.c.o src/Cedar/CMakeFiles/cedar.dir/EtherLog.c.o src/Cedar/CMakeFiles/cedar.dir/Hub.c.o src/Cedar/CMakeFiles/cedar.dir/IPC.c.o src/Cedar/CMakeFiles/cedar.dir/Layer3.c.o src/Cedar/CMakeFiles/cedar.dir/Link.c.o src/Cedar/CMakeFiles/cedar.dir/Listener.c.o src/Cedar/CMakeFiles/cedar.dir/Logging.c.o src/Cedar/CMakeFiles/cedar.dir/NM.c.o src/Cedar/CMakeFiles/cedar.dir/Nat.c.o src/Cedar/CMakeFiles/cedar.dir/NativeStack.c.o src/Cedar/CMakeFiles/cedar.dir/NullLan.c.o src/Cedar/CMakeFiles/cedar.dir/Proto.c.o src/Cedar/CMakeFiles/cedar.dir/Proto_EtherIP.c.o src/Cedar/CMakeFiles/cedar.dir/Proto_IKE.c.o src/Cedar/CMakeFiles/cedar.dir/Proto_IPsec.c.o src/Cedar/CMakeFiles/cedar.dir/Proto_IkePacket.c.o src/Cedar/CMakeFiles/cedar.dir/Proto_L2TP.c.o src/Cedar/CMakeFiles/cedar.dir/Proto_OpenVPN.c.o src/Cedar/CMakeFiles/cedar.dir/Proto_PPP.c.o src/Cedar/CMakeFiles/cedar.dir/Proto_SSTP.c.o src/Cedar/CMakeFiles/cedar.dir/Proto_Win7.c.o src/Cedar/CMakeFiles/cedar.dir/Proto_WireGuard.c.o src/Cedar/CMakeFiles/cedar.dir/Protocol.c.o src/Cedar/CMakeFiles/cedar.dir/Radius.c.o src/Cedar/CMakeFiles/cedar.dir/Remote.c.o src/Cedar/CMakeFiles/cedar.dir/SM.c.o src/Cedar/CMakeFiles/cedar.dir/SW.c.o src/Cedar/CMakeFiles/cedar.dir/Sam.c.o src/Cedar/CMakeFiles/cedar.dir/SeLowUser.c.o src/Cedar/CMakeFiles/cedar.dir/SecureNAT.c.o src/Cedar/CMakeFiles/cedar.dir/Server.c.o src/Cedar/CMakeFiles/cedar.dir/Session.c.o src/Cedar/CMakeFiles/cedar.dir/UT.c.o src/Cedar/CMakeFiles/cedar.dir/UdpAccel.c.o src/Cedar/CMakeFiles/cedar.dir/VLanUnix.c.o src/Cedar/CMakeFiles/cedar.dir/VLanWin32.c.o src/Cedar/CMakeFiles/cedar.dir/Virtual.c.o src/Cedar/CMakeFiles/cedar.dir/WaterMark.c.o src/Cedar/CMakeFiles/cedar.dir/WebUI.c.o src/Cedar/CMakeFiles/cedar.dir/WinUi.c.o src/Cedar/CMakeFiles/cedar.dir/Wpc.c.o src/Cedar/CMakeFiles/cedar.dir/__/__/3rdparty/BLAKE2/sse/blake2s.c.o  -Wl,-rpath,/Users/runner/work/1/b  /usr/local/Cellar/libsodium/1.0.18_1/lib/libsodium.dylib  /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/lib/libcurses.tbd  /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/lib/libform.tbd  /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/lib/libreadline.tbd  libmayaqua.dylib  -lpcap && :
Undefined symbols for architecture x86_64:
  "_OpenSSL_version", referenced from:
      _NewCedar in Cedar.c.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
##[error]Bash exited with code '1'.

Is it because I shouldn't import openssl in cedar?

Moved openssl stuff to mayaqua.

It's actually weird that it only failed on macOS, as we don't link Cedar to OpenSSL.

Since we use cmake for all platforms, I am wondering if we can add more useful stuff (platform, compiler version, ssl version...) simply by accessing to cmake variables.
https://cmake.org/cmake/help/latest/manual/cmake-variables.7.html

Windows:
vpncmd command - SoftEther VPN Command Line Management Utility Developer Edition
SoftEther VPN Command Line Management Utility (vpncmd command)
Developer Edition
Version 5.02 Build 0   (English)
Compiled 2022/11/11 03:42:05 by nobody at nowhere
(Windows-10.0.22621;CMake 3.24.202208181-MSVC_2;Ninja;Clang 15.0.1;OpenSSL 3.0.7 1 Nov 2022;RelWithDebInfo)
SSL Library (Runtime): OpenSSL 3.0.7 1 Nov 2022
Copyright (c) all contributors on SoftEther VPN project in GitHub.
Copyright (c) Daiyuu Nobori, SoftEther Project at University of Tsukuba, and SoftEther Corporation.
All rights reserved.

Linux:
vpncmd command - SoftEther VPN Command Line Management Utility Developer Edition
SoftEther VPN Command Line Management Utility (vpncmd command)
Developer Edition
Version 5.02 Build 0   (English)
Compiled 2022/11/11 03:50:00 by vsts at fv-az310-262
(Linux-5.4.0-1094-azure;CMake 3.24.2;Ninja;GNU 7.5.0;OpenSSL 1.1.1  11 Sep 2018;RelWithDebInfo)
SSL Library (Runtime): OpenSSL 1.1.1  11 Sep 2018
Copyright (c) all contributors on SoftEther VPN project in GitHub.
Copyright (c) Daiyuu Nobori, SoftEther Project at University of Tsukuba, and SoftEther Corporation.
All rights reserved.

On Linux where runtime OpenSSL is different:
vpncmd command - SoftEther VPN Command Line Management Utility Developer Edition
SoftEther VPN Command Line Management Utility (vpncmd command)
Developer Edition
Version 5.02 Build 0   (English)
Compiled 2022/11/11 04:01:47 by root at debian
(Linux-5.10.0-19-amd64;CMake 3.18.4;Unix Makefiles;GNU 10.2.1;OpenSSL 1.1.1n  15 Mar 2022;RelWithDebInfo)
SSL Library (Runtime): OpenSSL 1.1.1f  31 Mar 2020
Copyright (c) all contributors on SoftEther VPN project in GitHub.
Copyright (c) Daiyuu Nobori, SoftEther Project at University of Tsukuba, and SoftEther Corporation.
All rights reserved.

Build environment string can accommodate up to 512 chars now.

I'm generally not a fan of fixed-sized strings when the actual possible length is unknown.

I'm generally not a fan of fixed-sized strings when the actual possible length is unknown.

I can change it to dynamic in Cedar as you wish. But that's only used in About.
For client/server info that are transferred through RPC. I think it has to have a size. Doesn't it?

https://github.com/SoftEtherVPN/SoftEtherVPN/blob/8ae0a932da1858abe26e07a4a03c18b04527ff64/src/Cedar/Admin.h#L135-L141
https://github.com/SoftEtherVPN/SoftEtherVPN/blob/ca996ed89accf358999a706085377452b5eb5ae6/src/Cedar/Admin.c#L12568-L12696