[WIP] - Add kernel signals exe_ino, exe_ino_ctime, exe_ino_mtime

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area API-version

/area build

/area CI

/area driver-kmod
/area driver-bpf
/area driver-modern-bpf

/area libscap-engine-bpf

/area libscap-engine-gvisor

/area libscap-engine-kmod

/area libscap-engine-modern-bpf

/area libscap-engine-nodriver

/area libscap-engine-noop

/area libscap-engine-source-plugin

/area libscap-engine-savefile

/area libscap-engine-udig

/area libscap

/area libpman

/area libsinsp

/area tests

/area proposals

Does this PR require a change in the driver versions?

/version driver-API-version-major

/version driver-API-version-minor

/version driver-API-version-patch

/version driver-SCHEMA-version-major

/version driver-SCHEMA-version-minor

/version driver-SCHEMA-version-patch

What this PR does / why we need it:
Dropping an implant, making the file executable and executing the implant is amongst one of the oldest tricks. While memory based cyber attacks mostly circumvent touching disk, reliably detecting drifts, that is, a suspicious new executable is executed is often considered a crucial baseline detection.
Falco's upstream rules "Container Drift Detected (chmod)" and "Container Drift Detected (open+create)" aim to detect the creation of a new executable in a container (drift). However, both rules are disabled by default, because those rules can be noisy in un-profiled environments and workloads. Finally, currently there are no easy or robust mechanisms to correlate above rules that are based on file operation events with the events where the executable is run (execve).
This PR attempts to address this gap via adding enhanced kernel signals to spawned processes. While the proposed signals won't replace the need to monitor file operation events, they can help reduce the search space for tracking spawned processes where for example chmod +x was run against the executable file on disk prior to execution (this causes ctime of inode to change, but we don't know if it was chmod related or a different status change operation). In addition, end users could use these fields for selected rules to augment information available for incident response.
"proc.exe_ino" "Inode number of executable image file on disk", "The inode number of the executable image file on disk. Can be directly correlated with file operation event's field fd.ino as alternative to using file paths."
"proc.exe_ino.ctime" "Last status change time (ctime - epoch ns) of exe file on disk", "Last status change time (ctime - epoch nanoseconds) of executable image file on disk (inode->ctime). Time is changed by writing or by setting inode information e.g. owner, group, link count, mode etc."
"proc.exe_ino.mtime" "Last modification time (mtime - epoch ns) of exe file on disk", "Last modification time (mtime - epoch nanoseconds) of executable image file on disk (inode->mtime). Time is changed by file modifications, e.g. by mknod, truncate, utime, write of more than zero bytes etc. For tracking changes in owner, group, link count or mode, use proc.exe_ino.ctime instead."
"proc.exe_ino.ctime_duration_proc_start" "Time delta in ns - proc clone ts minus ctime exe file", "Time delta between process clone ts and ctime exe file in ns, aka duration of time passed after modifying the status of the executable image and then spawning a new process using the changed executable image."
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Includes cleanup, mainly make sched_prog_exec_4 and execve_family_flags filler alike in terms of style. Refactored (no logic changes) get_exe_writable to avoid few redundant _READ()s on same kernel structures within the same filler (@LucaGuerra).
This PR is not yet ready. Hoping for some early feedback to make these new signals better :)
Checklist (this PR)

[ ] container.duration (how long has the respective container been running at event x) -> this as stand-alone could add a lot of value as a container only a couple of mins old is less likely to have been exploited
[ ] host.duration (how long since last host re-boot at event y for host events)
[ ] proc.exe_ino.ctime_duration_container_start (how much time has passed since container start until status change of exe file on disk) -> probably needed to filter changes due to new containers
[ ] add duration filters for mtime?
[ ] likely valuable to add some filtering wrt process ancestors? (know what events originated from an ancestor executable that was recently changed)
[ ] more ideas / different ideas
[ ] kmod
[ ] modern_bpf
[ ] scap file

Checklist (future PR)

Initial attempt for on host anomaly detection for container drift use case (expectation would be to first PR a proposal doc as this would mark a significant new feature)

Does this PR introduce a user-facing change?:

TBA

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: incertum
Once this PR has been reviewed and has the lgtm label, please assign gnosek for approval by writing /assign @gnosek in a comment. For more information see:The Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

This is a huge PR @incertum!
A couple of notes that must be addressed before leaving "wip" state:

apply same patch to modern-bpf and kmod too ;)
i think scap_procs must be updated fetching new info from proc, if we can, right?

Aside from this, it looks really cool, thank you!

Hello @incertum! The container drift detection seems to be something really significant, thanks for spending time on it and trying to detect this kind of behavior!
Since I see in your checklist that you are open to discuss also different ideas, although this one is really cool, I want to ask you what you think about this other approach that I came up with and  described here:
https://github.com/falcosecurity/libs/pull/287
The main problem of this approach is that it relies on overlayfs and so it cannot work with old kernels and container runtimes that do not use it. It also needs to be tested across a wider variety of kernels to be sure that it's working, since it was like an experiment for me. I would be happy to know what you think about it!

@FedeDP ty let me look into scap-procs - once feature complete will implement this for modern_bpf, scap file and kmod, always leave the kmod fun for the end :) Also still need to test this on more kernel versions and distros than just the one I was quickly developing on ...
@loresuso was actually lurking around that overlayfs PR a good while ago. Thanks for experimenting  :heart:! In general I believe more and stronger kernel signals just like the one you proposed are needed, let's chat more.
What is needed to merge it? I approve, really nice work and think this is an excellent feature that adds even more signal for the container use case and I think it's ok that it doesn't work for super old kernels etc. Besides containers would also be interested in nailing this for bare-metal hosts.
@loresuso more signals are needed for detecting memory attacks or RCE in a more general and robust way (executables are just one aspect), one step at a time though. And I saw you also refactored the get_exe_writable and created similar get_exe_inode lol, we can sync on how to merge this cleanup into one approach.
Also once all new kernel signals we can come up with at the moment are merged, wanna team up on creating a strong and robust userspace logic to nail it? Would be amazing if some rules come out at the other end that can be enabled by default aka they can work in unknown environments. Called it anomaly detection, but we can also call it advanced signal correlation etc :upside_down_face:.

Re fetching the container start time or pid namespace creation time works too still monkeying around if this is best implemented kernel side. Something like somehow fetching the start time of pid=1 as seen from process namespace or the creation ts of the pid namespace the process belongs too ... would you have any thoughts on this?

Hey folks, I'd like to add my thoughts to the discussion since I originally introduced the is_exe_writable flag for this purpose, discussed a lot with Lorenzo about its evolution is_exe_upper_layer and am very interested in basically catching suspicious executions. While it's true that attacks can be fully in memory (which would bypass any file-based rule of course) we all know that a defense-in-depth strategy needs to consider many cases. Also, I expect the most common attacks to be indeed file based. This is a bit of a larger discussion that we may want to expand somewhere.
Regarding attack scenarios, the proposed fields would allow us to add another way to filter events to try and reduce the noise from this kind of rules. I would love for Falco to be able to have a set of rules to deal with the standard "drop + execute" case. This is what comes to mind:

In containers you can do, depending on how your container is built, one of two things: you either use is_exe_writable with containers that runs as regular user but has executable files normally owned by root (this is the default if you run as user!) or is_exe_upper_layer which works with containers executed as root as well 😎  This alerts for new executables at all times.
On hosts in my opinion the best bet is is_exe_writable and inspect non-root users I think because root on a host does way too many things 😭 . Installing and updating software is common, downloading and running software happens often during normal deployments ...  So many regular actions would trigger drop+execute that may make this pretty useless :/ But in some deployments it's not expected for regular users to bring their own binaries, and that's what I would want to catch. Also, remember that true root can change mtime and ctime of all files if it wants.

@incertum 's idea I think is definitely clever, as it allows us to add the time dimension to the above. You can say "If a regular user is running an executable that they can modify AND it has been modified 'recently', then alert". This allows us to detect drops without drowning in noise caused by system-wide software updates and new deployments. Same goes for containers. In that case I like the stronger properties of is_exe_upper_layer because you can't easily evade it if you're inside a container. Even if you drop a file today and schedule its execution at some other time it will be caught.
In conclusion, I probably want all of these fields 😎 I actually wanted is_exe_upper_layer in 0.33.0 but there's so much content that is going into that release that we probably want to merge it right after the release so we have time to test it and see that it doesn't break too many things (every new thing happening at process start as you could see is a little tricky...). Does it make sense to you? The first step as you mentioned could be to refactor and generalize the exe inode data collection in the kernel and ebpf.

@LucaGuerra ❤ 😎 as always a fantastic summary and technical assessment of what the actual problem here is. Fully agree that all these signals combined will be super valuable in addition to existing metadata fields. It's nice to see three folks having come to similar conclusions, that is, (1) it is at process startup where we need to fetch better kernel signals and (2) this old problem "drop+execute" has not yet been well addressed.
Of course the "host" is the more tricky one, doesn't change the fact that I have been asked to fix / solve this ... so thinking we won't get away without determining a pattern of past behavior of the applications that are running, and analyze behaviors outside the past behavior. There will be both data modeling challenges and software implementation challenges, the good news is similar problems have been solved in the industry before and we can build upon this. Needless to say let's start more basic and iterate.
How about first merging @loresuso PR that features is_exe_upper_layer after the upcoming release freeze, I'll continue monkeying around a bit for next 2 weeks and see if maybe there are more kernel signals that could be valuable. Perhaps you stumble across something new as well 🙃 that would be cool.
After everything is merged we collaborate on a fresh PR that just does userspace modeling? Also happy to offer deploying a prototype to production to be able to better assess how well it may work and also check that Falco does not deteriorate in case we introduce some significant new userspace features.

... Also, I expect the most common attacks to be indeed file based. This is a bit of a larger discussion that we may want to expand somewhere.

Would you have ideas re what the best forum would be to expand on those Threat Modeling discussions?

Also, remember that true root can change mtime and ctime of all files if it wants.

Yeah you can never just have nice things in security, hence why I am a big fan of multi-signal correlations.

Thanks @incertum @LucaGuerra, this conversation is getting more and more interesting!
I strongly agree that all these signals combined together are needed to improve the detection capabilities of the drop+execute pattern. So, soon after the release, I'll try my best to get the exe_upper_layer merged. Some help in testing it better before the merge would be really appreciated!
Also, I wanted to say that I am thrilled to team up altogether to discuss how to improve the detection capabilities of Falco with these new signals.
I also believe that we have to expand the conversation (maybe in Slack?) to other attack patterns. I think we may want to research a bit on fileless execution and post container escapes behaviors (like accessing files outside overlayfs from not mounted fs). I think these patterns are widespread too nowadays and I have some ideas that I would love to share with you!

Thanks @incertum @LucaGuerra, this conversation is getting more and more interesting! I strongly agree that all these signals combined together are needed to improve the detection capabilities of the drop+execute pattern. So, soon after the release, I'll try my best to get the exe_upper_layer merged. Some help in testing it better before the merge would be really appreciated! Also, I wanted to say that I am thrilled to team up altogether to discuss how to improve the detection capabilities of Falco with these new signals.

I'll help testing it yay, please ping me once ready after rebase!

I also believe that we have to expand the conversation (maybe in Slack or a Github issue?) to other attack patterns. I think we may want to research a bit on fileless execution (especially the one implemented with memfd_create. Execution from tmpfs) and post container escapes behaviors (like accessing files outside overlayfs from not mounted fs). I think these patterns are widespread too nowadays and I have some ideas that I would love to share with you!

https://github.com/falcosecurity/libs/issues/615
Funny you mentioned memfd_create, during one of my latest hacking sessions looked into it just to realize it is as hopeless as mmap or write syscalls and super noisy, unless we find better kernel side modeling capabilities. Looked into userland exec a bit https://grugq.github.io/docs/ul_exec.txt and actually deployed rules looking for LD_PRELOAD env variable or accessing proc maps of a different process - of course a bit of failed experiments, so am still going a bit in circles re what could be a good signal here ... there was a great talk at BlackHat 2022 USA around it https://github.com/dsnezhkov/elfpack.

Kernel side solution for robustness reasons: Add pid namespace init task start ts to generically approx container or host start ts and compute time deltas useful for detections, such as container duration or duration between pidns start ts and ctime exe file if pidns ts predates ctime. A general detection use case can be that if suspicious events happen in multiple containers of a deployment near container start it's more likely to be "normal". The longer a container runs the longer it is "exposed".
What questions do you have re the proposed approach to solve above? Would it be possible to check soundness of this approach? This would be much appreciated. Initial experimentation seemed correct ts values for various scenarios, but will continue testing.

Another kernel side signal that would like to look into and possibly add to this PR would be:
"Interpreter scripts" aka text files with execute permissions (see https://man7.org/linux/man-pages/man2/execve.2.html)
For example chmod +x a.sh && ./a.sh or chmod +x a.sh && exec ./a.sh  is currently logged as "proc.exepath":"/tmp/a.sh","proc.name":"a.sh","proc.cmdline":"a.sh ./a.sh", but the interpreter was configured as #! /bin/sh and we wouldn't know what interpreter binary ran the script directly or that it was not a binary without inferring from extension if even available and we know how fragile that is.
Please note, not talking about the use case where you run the interpreter and pass the script, like /bin/sh a.sh  would give "proc.exepath":"/bin/sh","proc.name":"sh","proc.cmdline":"sh a.sh".
Any thoughts on above? @LucaGuerra @loresuso @FedeDP @Andreagit97
After that this PR should be feature complete and can start finalizing it, followed by code optimization review.

This PR is not ready for review.

@LucaGuerra and @loresuso in addition to VM tests, deployed these changes and @loresuso changes from https://github.com/falcosecurity/libs/pull/287 is_exe_upper_layer to production (eBPF only). Has been running for 2 weeks now and seems stable, also no unwanted CPU or memory usage increases.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: incertum
Once this PR has been reviewed and has the lgtm label, please assign gnosek for approval by writing /assign @gnosek in a comment. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ei @incertum could you rebase this on master?

Thanks for the review @Andreagit97 . Hope I got all the minor improvements properly implemented. In addition, updated the modern_bpf tests.
To the best of my knowledge and based on empirical perf measures in prod clone/execve* families of syscalls seem super light and are not moving the needle in terms of perf overhead compared to the heavy / high frequency syscalls. Perhaps it's fair to even anticipate that adding another 5-7 params for clone/execve* families should be no problem and they would have the potential to really boost detection capabilities and accuracy. For example, near-term @loresuso plans to work on "memfd+exec" execve flag. Also in bpf we still need to fix uid etc lookups as they do not account for user namespaces yet, so we may need to jump then anyways to another tail call.
Suggestion: What would you think of redefining clone/fork/execve* schemas? -> balance params a bit so that we can try to get rid of a few lookups that are performed a few times and in addition implement pushdown optimizations, such as that if pid <-> vpid is the same can also skip the pidns start ts lookup kernel side, right now those are in different fillers .. and there may be more such minor optimizations possible? If we were to do that, probably worth to check what other params are needed before changing the schema.

To the best of my knowledge and based on empirical perf measures in prod clone/execve* families of syscalls seem super light and are not moving the needle in terms of perf overhead compared to the heavy / high frequency syscalls.

To be honest, I thought these process syscalls were high-frequency in a production system, good to know this is not your case :)

Suggestion: What would you think of redefining clone/fork/execve* schemas? -> balance params a bit so that we can try to get rid of a few lookups that are performed a few times and in addition implement pushdown optimizations, such as that if pid <-> vpid is the same can also skip the pidns start ts lookup kernel side, right now those are in different fillers .. and there may be more such minor optimizations possible? If we were to do that, probably worth to check what other params are needed before changing the schema.

This could be an excellent idea if we are able to detect some useless parameters or some huge optimization. BTW in this direction, we have already a possible work with https://github.com/falcosecurity/libs/pull/526, before this, we have just to define a little bit better scap-file format to not break compatibility with old captures

Hi @incertum

Thanks for the review @Andreagit97 . Hope I got all the minor improvements properly implemented. In addition, updated the modern_bpf tests.

That's great! I already gave your PR with the latest changes a run on s390x and all tests passed successfully!

To the best of my knowledge and based on empirical perf measures in prod clone/execve* families of syscalls seem super light and are not moving the needle in terms of perf overhead compared to the heavy / high frequency syscalls.

To be honest, I thought these process syscalls were high-frequency in a production system, good to know this is not your case :)

I think it depends on the workload running on those systems and how much the systems are being utilized.

@Andreagit97 will follow up with opening a ticket re brainstorming for future optimization ...
Also noticed I missed param 21 in sched fork in kmod, now fixed.

ei @incertum thank you for the changes I will take another look ASAP
in the meanwhile, there is an issue compiling the kernel module version 2.6.32 as you can see from the CI job, the problem should be here
2022-11-30T11:31:25.3809926Z DEBU /tmp/driver/ppm_fillers.c:1157:9: error: incompatible types when assigning to type 'long long unsigned int' from type 'struct timespec'
2022-11-30T11:31:25.3810711Z DEBU     time = child_reaper->start_time;
2022-11-30T11:31:25.3811816Z DEBU          ^
2022-11-30T11:31:25.4193261Z DEBU ������Mmake[2]: *** [scripts/Makefile.build:230: /tmp/driver/ppm_fillers.o] Error 1
2022-11-30T11:31:25.4200441Z DEBU ������)make[1]: Leaving directory '/tmp/kernel'
2022-11-30T11:31:25.4202657Z DEBU ������:make[1]: *** [Makefile:1448: _module_/tmp/driver] Error 2

I will try to fix broken tests in the next few days :)

I will try to fix broken tests in the next few days :)

Thanks so much Andrea, this is very much appreciated :rocket: !!!

/milestone 0.11.0

Thanks a lot for the great work refactoring this PR @incertum and @Andreagit97 ! I went through the e2e flow again (I know there is some ongoing refactoring in the kmod). I only have a concern about the use of get_host_boot_time_ns which I'm going to detail in the code.

The last commit should fix windows CI, just removed scap_get_host_boot_time_ns() helper since we already have scap_get_boot_time() :)

The last commit should fix windows CI, just removed scap_get_host_boot_time_ns() helper since we already have scap_get_boot_time() :)

lol classic, thanks for keeping the new more reliable method to get a constant boot ts :)

LGTM label has been added.  Git tree hash: 7f7068dcd291f5ed76d4ec430bd06adb37f263bf

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, incertum
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, incertum, LucaGuerra
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP,LucaGuerra]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Add notifications for Linux using libnotify

This change will (try to) show a desktop notification using the libnotify gem on Linux.
If it fails, the error will be logged but not raised.

Thank you for implenting this. I love this feature and it works.
If you can fix minor things I have pointed out, lets merge.

I am going to fix in a branch and open another PR.

Sure, as long as the feature ends up in master, I'm happy :)


Exception processing latest go-plus.owl with owltools

Hi.
We've noticed that the latest go-plus.owl can't be processed by owltools.  The version from before Saturday or so were fine.  We're getting the file from http://purl.obolibrary.org/obo/go/snapshot/extensions/go-plus.owl
If this is an owltools problem I'll open an issue there.
Thanks!
owltools go-plus.owl -o -f obo out-file.obo
Exception in thread "main" org.semanticweb.owlapi.model.OWLOntologyStorageException: org.obolibrary.oboformat.model.FrameStructureException: multiple name tags not allowed. in frame:Frame(CHEBI:67208 id( CHEBI:67208)def( A double stranded ribonucleic acid.)synonym( double stranded ribonucleic acid RELATED)name( double-stranded RNA)name( dsRNA)is_a( CHEBI:33697)is_a( CHEBI:23367))
        at org.semanticweb.owlapi.oboformat.OBOFormatRenderer.render(OBOFormatRenderer.java:90)
        at org.semanticweb.owlapi.oboformat.OBOFormatStorer.storeOntology(OBOFormatStorer.java:42)
        at org.semanticweb.owlapi.util.AbstractOWLStorer.storeOntology(AbstractOWLStorer.java:155)
        at org.semanticweb.owlapi.util.AbstractOWLStorer.storeOntology(AbstractOWLStorer.java:119)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.saveOntology(OWLOntologyManagerImpl.java:1525)
        at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.saveOntology(OWLOntologyManagerImpl.java:1502)
        at owltools.io.ParserWrapper.saveOWL(ParserWrapper.java:289)
        at owltools.io.ParserWrapper.saveOWL(ParserWrapper.java:209)
        at owltools.cli.CommandRunner.runSingleIteration(CommandRunner.java:3712)
        at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:76)
        at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:68)
        at owltools.cli.CommandLineInterface.main(CommandLineInterface.java:12)
Caused by: org.obolibrary.oboformat.model.FrameStructureException: multiple name tags not allowed. in frame:Frame(CHEBI:67208 id( CHEBI:67208)def( A double stranded ribonucleic acid.)synonym( double stranded ribonucleic acid RELATED)name( double-stranded RNA)name( dsRNA)is_a( CHEBI:33697)is_a( CHEBI:23367))
        at org.obolibrary.oboformat.model.Frame.checkMaxOneCardinality(Frame.java:424)
        at org.obolibrary.oboformat.model.Frame.check(Frame.java:405)
        at org.obolibrary.oboformat.model.OBODoc.check(OBODoc.java:390)
        at org.obolibrary.oboformat.writer.OBOFormatWriter.write(OBOFormatWriter.java:183)
        at org.semanticweb.owlapi.oboformat.OBOFormatRenderer.render(OBOFormatRenderer.java:88)
        ... 11 more
[Exit 1 ]

There was a change that is probably the cause of this: https://github.com/geneontology/go-announcements/issues/56
I assume it's because there is content in the file now that was previously not saved to OBO, since it was in an external import.
@cmungall is there an owltools option that will let this through?

In the meantime I will try to see why we're getting two labels.

The extra label is coming from the PRO import.

@kimrutherford out of curiosity do you require OBO format for the next step in your workflow? If you save to OWL instead you shouldn't have an issue.

Unfortunately our workflow depends on OBO.

I've got a fix for the double CHEBI labels coming from PRO. However then this reveals that the OBA import contains double labels for OBA terms, which is actually that way in OBA. Will try to sort that out next.

@kimrutherford I just realized there is a --no-check option for owltools:
owltools go-plus.owl -o -f obo --no-check out-file.obo
should succeed. The question then is whether multiple labels causes you any other problems.

I just realized there is a --no-check option for owltools

That works.  Thanks!

The question then is whether multiple labels causes you any other problems.

It looks like it's all fine.

Looks like this is solved. If the issue persists, please reopen.


Dampings on edges

Currently, we apply all dampings to local contact matrices. If and to which extent these same dampings should be applied to edges should be discussed further. It is activated in https://github.com/DLR-SC/memilio/blob/main/cpp/models/secir/parameter_studies.h as it was done paper 1. Since commuter testing effects had to be studied in paper 2, this was deactivated there to avoid side effects.

@WadimKoslow This has to be considered if #74 makes it to the main.


2019.4.4 LTS?

Hi
update for 2019.4 LTS plz
or can I proceed with the current app?

well its working as 2019.3/2020.1 add it there
ie 2019.3/2019.4/2020.1


CLI Enhacements

Hi,
I use CLI a lot, as a calculator. Actually, instead of using calculator, I would like to use CLI always. My problem is, there is no easy access to CLI.
(1) I cannot pin it to the taskbar, because when I pin it, it is a terminal, not CLI.

(2) The best thing would be having a Windows wide shortcut. That would be amazing.
I know CLI is not a priority, but opening a Calcpad for quick calculations is an overkill.

Hi, Demir!
I am glad that at least one person is using it. :)
Probably, you pin the running application on the taskbar itself. That is why it pins the terminal instead.
You have an icon on the desktop named 'Cli'. Right click on the icon, click 'Show more options' if on Windows 10/11 and then 'Pin to taskbar'. This will pin the actual Cli module rather than Terminal.

Perfect, thanks! It opens as a seperate window from the shortcut, but it works.
Maybe you can think about having a icon for next release, a small CLI written on the existing icon? Admin users can change it of course by themselves.


Update progress bar width when terminal width changes.

I use a tiling window manager, so my term width changes a lot.
If you don't want to do it this way for performance reasons or whatnot, perhaps it could be made an option?
This reverts commit 2ae59f8aaf20df5b0f60050487d1c93ecac8e43b.

I also used to use a tiling window manager (awesome), and that was my concern originally. However, the performance was abysmal, I suspect that the Ruby lib I use to determine the shell width is shelling out every time.
I have a couple solutions that might work:

Revert it entirely, and add docs for when there is a huge max and a very fast increment, to only update the bar peroidically, using #increment(100) or whatever. However, I dislike this because it forces the user to keep track of counter themselves, and do modulo arithmetic to know if they should update the bar.
Add a timer, and only re-measure the width every 250ms or so.
Instead of a timer, use the rate to know if we're updating the bar more than a few times a second, and only re-measure the width every "n" updates automatically.

I'm not sure which I prefer out of #2 or #3. #2 requires us to keep track of a timer and last update: Time.now.to_f - @previous_width > 0.25. The code for #3 is a bit more complicated, but might not be too bad.
I'd except pull requests for either one, since you have the tiling window manager, perhaps you could try and see which works better for your situation?

I added a timer that makes it update once per second, and that works fine for my purposes. If you still consider it too much of a performance hog I don't think slightly raising the delay would be too bad.


Replace qt foreach keyword for range-based loops

As titled.
This brings the codebase closer to native C++ standards and algorithms.

https://doc-snapshots.qt.io/qt6-dev/foreach-keyword.html

Note: The foreach keyword was introduced before the C++11 range-based loops existed. New code should prefer C++11 range-based loops.

Updated changeset to fix clazy warnings.

Hi, is there anything else needed / requested for this PR to be merge? I'm happy to close the PR and merge in my origin mirror if merging with (this) upstream is unfortunately not possible...

Sorry for not responding, Your last comment got lost in the deluge of github emails I get related to work. :(
Much appriecated!


Unable to read the Android NDK

When trying to build for Android (using Windows)
It's looking for "platforms" but NDK no longer uses that directory.
Unable to read the Android NDK at C:/Users/X/AppData/Local/Android/Sdk/ndk/22.0.7026061, the path may be invalid. Is the path in android_ndk_repository() or ANDROID_NDK_HOME set correctly?
Expected directory at C:/Users/X/AppData/Local/Android/Sdk/ndk/22.0.7026061/platforms but it is not a directory or it does not exist. Unable to read the Android NDK at C:/Users/X/AppData/Local/Android/Sdk/ndk/22.0.7026061, the path may be invalid.

Thank you for reporting the issue. I updated README.
First, bazel does not seem to support NDK r22 yet, so please try with NDK r21.
Second, building mediapipe apks isn't possible on Windows and so is this plugin. Probably building on WSL2 will work.


Provide more explicit error if dim1 not supported

Types of changes

[ ] Docs change / refactoring / dependency upgrade
[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Motivation and Context / Related issue
#675

How Has This Been Tested (if it applies)

Checklist

[x] The documentation is up-to-date with the changes I made.
[x] I have read the CONTRIBUTING document and completed the CLA (see CLA).
[x] All tests passed, and additional code has been covered with new tests.

@teytaud doubledoerr_discrete_mutation does not work in dimension 1
Should it be updated to work?
If not how should Shiwa be updated in this case?

It should definitely be updated for working in dimension 1, no idea why it does not...

No idea why dim 1 does not work, but for this degenerate case, maybe we just switch to portfoliodiscretemutation ?


[ISSUE] Default-Size button not working as expected ?

Version of Archi

4.9.0
Archi Plug-ins

Not Relevant for this issue (IMHO :-)
Operating System

Windows 10
Expected Behaviour

When changing the default size of an element on a view (changing its default width and/or its default height), the [Default Size] button in the toolbar (which is greyed-out by default) should become active/available to the user, to "reset" element's dimensions to default.
Actual Behaviour

Actually, the [Default Size] button in the toolbar only become active when you change the element's default height. If you manage to change only its default width, the button keeps grayed-out and is not made active/available to the user.
Steps to Reproduce the Behaviour

Place an element on a view
Change the element's default width
Try to restore the element's default dimensions (height and width) by clicking the [Default Size] button
That's all :-)

Hi, I can't reproduce this.
What element are you using?

Change the element's default width

Do you mean by resizing it?


Reload Buttons

Implement Buttons for reloading the single panels.

Add all the little green buttons
:green_heart:


Create symlink to current InfluxDB CLI

@pauldix

@corylanou

CLI was already being installed, just needed the post-install script to set up the symlink.

+1 from @pauldix


BUG: Route mybe no desc field. And code not go fmt

as below picture show, Route mybe no desc field
/apisix/admin/routes

@chzhuo
Yes, I will add a check here
However, it is recommended that when using ingress-controller, only define route through CRD, do not directly use apisix dashboard or apisix admin api, this will lead to inconsistent state between apisix and k8s, ingress-controller only does one-way synchronization from k8s to apisix , Two-way synchronization is more difficult.
Define route to use CRD (in this way, route will automatically generate name)
You can view it through the apisix dashboard.

Solved since #9 merged.


Add bottom pane to SplashScreen

Add a bottom pane to the splash screen with buttons to add a file, delete a file, or create a new file.

Added buttons and serialization / deserialization (via JSON) of Fluke objects.


[BUG] Golang sarama integeration tests are easily to fail

Describe the bug
Golang sarama integeration tests are easily to fail because the container failed to start, it's because the golang-sarama container is fast and might complete quickly and then stop so that GenericContainer cannot catch the logs and wait for the specific starting to produce or starting to consume log.
01:53:04.668 [TestNG-method=simpleProduceAndConsume-1:org.testcontainers.containers.GenericContainer@484] ERROR 🐳 [streamnative/kop-test-golang-sarama:latest] - Could not start container
 org.rnorth.ducttape.TimeoutException: Timeout waiting for result with exception
     at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:54) ~[duct-tape-1.0.8.jar:?]
     at org.rnorth.ducttape.unreliables.Unreliables.retryUntilTrue(Unreliables.java:100) ~[duct-tape-1.0.8.jar:?]
     at org.testcontainers.containers.startupcheck.StartupCheckStrategy.waitUntilStartupSuccessful(StartupCheckStrategy.java:35) ~[testcontainers-1.15.1.jar:?]
     at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:432) ~[testcontainers-1.15.1.jar:?]
     at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:325) ~[testcontainers-1.15.1.jar:?]
     at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81) [duct-tape-1.0.8.jar:?]
     at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:323) [testcontainers-1.15.1.jar:?]
     at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:311) [testcontainers-1.15.1.jar:?]
     at io.streamnative.pulsar.handlers.kop.KafkaIntegrationTest.simpleProduceAndConsume(KafkaIntegrationTest.java:204) [test-classes/:?]

To Reproduce
Just run the KafkaIntegrationTest in local env or you can see it in CI.
Expected behavior
We should find a not flaky way to wait for the container start.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.

Currently I'll ignore these two tests in https://github.com/streamnative/kop/pull/628


feat: initial implementation of metabase group migration into guardian

@singhvikash11 @rahmatrhd After provider import is done. Can we move this to use that contract and update the PR?

@rahmatrhd @bsushmith Is there any plan to pick this up?


ARMA3 info_status.sh command not found

Gday,
When doing an update to grab the newer version of arma3 server today, i noticed an error thrown by update_check.sh
arma3server@droid118:~$ ./arma3server u
fetching update_check.sh...OK
[  OK  ] Update arma3-server: Checking for update: SteamCMD
Update available:
Current build: 1100267
Available build: 1154214
`https://steamdb.info/app/233780/`

Applying update...
/home/arma3server/lgsm/functions/update_check.sh: line 172: info_status.sh: command not found
The file update_status.sh doesn't exist. I am part way through the update with no other obvious problems however.
just thought id let you know.
Cheers.
Doug

Thanks for info.
Corrected with https://github.com/dgibbs64/linuxgsm/commit/b321f33ea8a98bddc8b86c70ebdcdaffd38d0218
Please, run ./arma3server update-functions and try again.
Let me know if the issue is solved.

no problem, i have another linux box that needs updating, ill update the script before doing the server update, report back in 20 mins or so :)

All good now,
my second server fetched the updated functions and performed the arma3 update from steam with error message.
cheers mate, have a good one

also, sorry i didn't think to check in the closed issues, i now see i created a duplicate :)

np, thanks for your report :)


[RFC] Feature: ParametricUMAP PyTorch implementation

In the spirit of the PR #578 which allows dependencies not to interfere too much with the test suite, I was wondering whether you would consider adding an extra dependency 😅
More seriously, I started working to an alternative PyTorch-based implementation for ParametricUMAP (will soon sync the branch on my fork)
If you would generally appreciate the idea and the contribution, I will elaborate better here my UEP (UMAP Enhancement Proposal) to have the two implementations co-existing, with the less impact possible in the API for backward compatibility.

I would like to get @timsainb 's opinion on this. Personally I can see a lot of value in supporting other deep learning frameworks, and since dependencies can be optional I don't see and extra potential dependency as a problem. I do know that other have asked about a PyTorch version, so there is certainly interest. I think it would be a very exciting addition. The key would be to ensure it syncs up with @timsainb 's implementation in TF and doesn't cause (too many) weird interactions as to what happens under the different backends. It sounds like you have thought about that, so please feel free to elaborate on that -- I would like to hear more.

I think that would be a great addition.

If Tim is on board, then I think we can all this a go! Really looking forward to seeing this in place.

Fantastic!! 🙌
Thanks so much @timsainb and @lmcinnes for getting back to me about this!
Will submit my initial PR very soon so that we can start discussing design options there.
I will be sharing quick ideas here as well while I do work on it, so that we could potentially upvote/downvote options very quickly - will be starting soon in my next post!
Thank you so much :)

Q: How do we allow multiple DL backend for Parametric UMAP (so far, tf and torch)
iow, what implementation will I import whenever I will do:
from umap.parametric_umap import ParametricUMAP

My idea would be in favour of a Convention of Configuration based approach - so no configuration file a-la-keras to choose the backend
General Idea

If torch is available but not tf --> go with torch
if tf is available but not torch --> got with tf
if tf and torch are both installed in the environment --> go by default with @timsainb / tf implementation (unless specified - as in importing the default torch-based class/implementation)

Similarly, whenever a custom encoder &| decoder parameter(s) are provided in the constructor, the choice of the backend will be directly derived from the type of the object.
Again, the goal is to be as less disruptive as possible with current APIs
What do you think? :)

Sounds ideal to me. I really like the idea of having the least friction for users, and this would meet that goal. It may be worth providing a warning, or some kind of feedback to the user, in the fallback/default case, so that they know that a decision has been made without their input. That is, if both tf and torch are installed, and no custom encoder or decoder is provided, let the user know that tf will be used, and how they can force the torch option if they would prefer.
In practice I assume the main reasons to have a strong preference is that the user has one but not the other installed, or because they want to define their own network architecture (encoder/decoder) in their preferred framework. Are there other reasons to have a significant preference?

@lmcinnes I'm currently running into problems with tensorflow's protobuf size limits when I try to run on larger datasets. Maybe pytorch won't have this issue? I haven't been able to find an easy workaround so far.

@cyrusmaher I have the impression that this may depend from many different factors: e.g. available hardware architecture, or dataset size just to mention the two most obvious.
One thing for sure is that torch does not use protobuf so if you think your issue is related to that, it might not have the same issue.
That being said, It is very hard to think of possible workaround or solution without having a clearer idea of the actual problem and errors.
So, my personal piece of advice here, would be to open a specific issue on that - if at all possible - specifying the environment used to replicate the issue.
In this way, it would be easier to understand what and where is the problem :)
HTH

hey @leriomaggio . Is this still something you're thinking about? I've been playing around with pytorch and I'm sold on it's advantages over tensorflow. It seems like it might be a good idea to even switch the primary backend over to pytorch. From a user standpoint pytorch is just a bit easier to deal with. And it seems like pytorch lightning allows us the same level of abstraction as keras.

Sounds great. I'm trying to tie up my thesis at the moment so I relate to juggling a whole lot of things at once.

I am happy that you share the same view on PyTorch, and I'd be happy to discuss any idea you might have (here, or anywhere you think would be best fit) about design. I was indeed keen on supporting pure PyTorch first, as I don't see lightning integration as necessary as Keras would be for Tensorflow (also in terms of adoption). However, happy to discuss about this too.

Sounds good to me, I'm new to pytorch so I'm sure you'd know better than me. If we can get this working without making people install anything new, all the better.

As for where I left: I think I started working on what is perhaps the most "intricate" part, as in porting the covariance calculation for the distance_loss_correlation as done in tensorflow_probability.
There is no equivalent to tensorflow_probability for PyTorch, so I decided to resort re-implementing it from scratch: not entirely a big deal, only some headaches to have tensor dimensions in place, and making sure the same operations (and assumptions on data) would be the same.
The implementation should be in place, and some tests are there as well, although I was planning to complete the test and then merging the branch in my fork into a similar branch here to continue working on the implementation.

That makes sense. I suppose I got lucky in finding tensorflow had already solved this. Your implementation looks great so far!

Hi @leriomaggio and @timsainb, I just came across this because I was looking for exactly this feature. I need some more flexibility with the embedding topology (e.g. I need to embed on a torus but also learn the torus dimensions) and I'm used to working with PyTorch. So this would be a killer feature for me :)
Is this still in progress? Is there maybe even a beta version that I could try out? :)

Hi @robert-lieck. I'm still trying to get my thesis finished and don't personally have the time to work on this. Very interested in seeing a torch implementation happen though!
For a basic implementation, all you need to do is translate this self-contained notebook to torch:
https://colab.research.google.com/drive/1WkXVZ5pnMrm17m0YgmtoNjM_XHdnE5Vp?usp=sharing
Probably every tensorflow function used there has a direct translation into torch, so I don't expect it would take too long. Writing your own implementation will give you the best flexibility too.
For the getting a version working in this repo: @leriomaggio 's implementation of the functions that tensorflow relies on tensorflow_probability for (from the comment above) look fantastic. The tensorflow_probability part is only needed for the global loss. To get a version that matches the tensorflow version, the next challenge is writing a robust data iterator for training, as well as an optimization loop to replace keras.fit. The rest of those tensorflow functions have a pretty direct translation in torch.
Beyond that, it

Hi both @robert-lieck @timsainb
First off, thanks very much @robert-lieck  for reviving the topic: there is no day that passes that I wish I could get some time to get back at this! I very much want this to happen!
I've been recently contributing to two other open-source projects and this unfortunately got a bit delayed, but I do really hope I could get back at where I left very soon.
Just bear with me another little bit: I promise I do really want this integration to happen! 💪
That said: all that @timsainb already mentioned is very correct and a starting point for sure!

Thanks for your replies @timsainb and @leriomaggio
@timsainb thanks for linking the notebook and good luck with the thesis writing 💪
@leriomaggio This is great! I will be off in a couple of days until end of September but I might be able to do some work on this once I'm back.
One question for clarification: Does ParametricUMAP also allow for parameterising the output_metric? It looks like this is not the case at the moment. Having that part implemented (and thus trainable) via PyTorch would be really cool: (1) It would allow to learn the embedding topology and (2) the gradient computations would come for free (which now need to be manually implemented via numba).
I haven't dived deep enough into the code yet, but I guess there should be quite some synergies in tackling this in the same go, right?

Hi everyone! This thread has been dead for a while but I've recently ported a simplified parametric umap to pytorch for necessity using pytorch lightning for the training loop. Here is the repo: https://github.com/elyxlz/umap_pytorch. I did this just by following the notebook and translating to pytorch. However, I encountered a problem on my first test using MNIST. I successfuly do get some class separation, but not nearly as good as is shown in the notebook, this is despite using exactly the same hyperparameters. Could anyone that has more knowledge than me help me figure out why? Interestingly my loss converged much earlier than it did for the notebook (around 0.2 instead of 0.1). For reference here is the plot:

Awesome to see someone working on this.
I've just looked briefly, but it looks like the convert_distance_to_probability differs from the original?

I fixed the probability computation and it looks much better now: https://colab.research.google.com/drive/1CYxt0GD-Y2zPMOnJIXJWsAhr0LdqI0R6?usp=sharing

Thank you so much! That's amazing news that it works. So strangely enough the original notebook you shared has some differences to the implementation on GitHub, such as the distance to probability function and how cross entropy is computed. Maybe I should be comparing more to the github version to find why the loss isn't converging correctly. Thanks again! :)

+1 this thread / idea. Would love to get it integrated with the core UMAP package. I hope this really happens.

Really cool incremental development. I am also waiting for a pytorch implementation desperately. Rooting for you folks!

Hi all, thanks for sharing the collab notebook, I think a  pytorch implementation would be a wonderful addition.
One comment/question about the code:
In the UMAPDataset class, the length is defined as int(self.data.shape[0]), which is the 60,000 MNIST train digits - but shouldn't the dataset be the size of self.edges_to_exp? Otherwise during training we only ever iterate over the first 60,000 sampled pairs from the shuffle mask. You probably don't want to iterate over the whole shuffle mask, but would like to sample a different subset each epoch.
I don't think this solves the above mentioned loss discrepancy though.

Thanks so much @elyxlz @timsainb for this PyTorch implementation of Parametric UMAP.  I'm in the process of adding it to our library (156 commits in to this branch :see_no_evil:), and having an implementation to adapt from has been really helpful.
Just want to add that I think @jh83775 is right -- you will want __len__ of the dataset to return self.edges_to_exp.shape[0], instead of self.data.shape[0].
You can verify this by setting __len__ to return some arbitrary small number (e.g. return 10) and seeing that training stops quickly.
I wonder if this "truncation" of the dataset explains the loss not converging all the way to 0.1 like you saw @elyxlz? Basically because we're not iterating through the entire dataset, so we can't drive the loss all the way down.
Also, this is a total nitpick, but I'm not sure if it's necessary to shuffle here?
https://github.com/elyxlz/umap_pytorch/blob/04d3cc6923ee783fa69806728ee165b92cd6af73/umap_pytorch/data.py#L40
        shuffle_mask = np.random.permutation(np.arange(len(self.edges_to_exp)))
        self.edges_to_exp = self.edges_to_exp[shuffle_mask].astype(np.int64)
        self.edges_from_exp = self.edges_from_exp[shuffle_mask].astype(np.int64)

This might be a case of translating literally from keras idioms--I'm pretty sure PyTorch / lightning will handle the shuffling as long as shuffle is set to True for the Dataloader`, as it is here:
https://github.com/elyxlz/umap_pytorch/blob/04d3cc6923ee783fa69806728ee165b92cd6af73/umap_pytorch/main.py#L91
I haven't tested this by removing the shuffling though. Please correct me if there's something I'm missing.

What is the current status on ParametricUMAP PyTorch?


KVM time sync. after downtime

Description
In the past, the libvirt introduced the domtime XXXX --sync command to instruct QEMU guest agent to synchronize the guest time from RTC. Although in general NTP server is recommended to run on both host and guest, the big time differences can't be automatically fixed by ntpd, or the fix doesn't happen fast enough.
Even in the oldest systems we support (Debian 9, Ubuntu 16.04), this feature is already supported.
https://forum.opennebula.org/t/fixing-windows-time-on-resume-in-kvm/3033
It might be good to follow the service libvirt-guests approach and optionally trigger domtime XXX --sync after resume, or migration. Although this doesn't fix all the time problems with the VMs, it might improve situation for some.
As described in /etc/sysconfig/libvirt-guests:
# If non-zero, try to sync guest time on domain resume. Be aware, that
# this requires guest agent with support for time synchronization
# running in the guest. For instance, qemu-ga doesn't support guest time
# synchronization on Windows guests, but Linux ones.
#SYNC_TIME=1

Proposed steps:

in remote's kvmrc, introduce SYNC_TIME (default: no) parameter to allow global configuration of this new feature

adapt KVM actions to trigger domtime XXXX --sync if SYNC_TIME=yes, but ignore exit code (operation might and will fail as it requires the domain to have qemu-ga running & RTC support !!!, this is just only helper, not a solution of all cases):

restore
maybe also migrate and migrate_local

test the feature in readiness and context

break time
VM suspend / resume
check time is correct

Progress Status

[ ] Branch created
[ ] Code committed to development branch
[ ] Testing - QA
[ ] Documentation
[ ] Release notes - resolved issues, compatibility, known issues
[ ] Code committed to upstream release/hotfix branches
[ ] Documentation committed to upstream release/hotfix branches

PRs to merge in master:
code: https://github.com/OpenNebula/one/pull/3874
docs: https://github.com/OpenNebula/docs/pull/816
tests: https://github.com/OpenNebula/development/pull/779


Can not run module:migrate-rollback [module] with specifying database connection

I can not run php artisan module:migrate-rollback [module] with --database=[dbConnection] option.
It ignores --database option and use default database connection.

I am having this same issue as well. Any update on this?

Seems like, that the defined options are not implemented.
Can't see anything in the code.
In the Migrate Command it's defined, but not in the MigrateRollback.


chore(deps): upgrade dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-main" workflow

Codecov Report

Merging #208 (c781e7a) into main (2974dfe) will increase coverage by 1.33%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #208      +/-   ##
==========================================
+ Coverage   83.33%   84.67%   +1.33%
==========================================
  Files           2        2
  Lines         126      137      +11
  Branches       31       34       +3
==========================================
+ Hits          105      116      +11
  Misses         21       21

Impacted Files
Coverage Δ

example/src/main.ts
90.62% <100.00%> (+2.16%)
:arrow_up:

src/index.ts
82.85% <100.00%> (+0.85%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ddb11f0...c781e7a. Read the comment docs.


feature: touchscreen/touchpad support

it would be nice to be able to move between windows with touchpad (and maybe touch screen) using 3/4 fingers
i have a touchscreen laptop with detachable keyboard and using paperwm with touchscreen/pad make for pretty enjoyable workflow

I don't have a way to develop/test this. I use a desktop computer with a mouse, and mostly the keyboard.

if you would be interested there are projects on getting arch/debian linux on arm64 chromebooks which are pretty cheap and interesting devices to run linux on

But let me know what kind of gestures you would like to have, and I will leave this issue open for future reference and possibly comments from other people. Maybe at some point I can do something about this.

just like with paperwm
3 fingers on trackpad used to move all windows left or right
and same or touch screen or moving your finder over waybar like a slider (like paperwm does, but it might be harder to do idk)

Have you checked there is no way you can bind the touchpad to dispatchers directly (bindm) for a few tricks?

it's less about keybind and more of guesture thing
like
# https://wiki.hyprland.org/Configuring/Variables/#gestures
gestures {
    workspace_swipe = false
}

but instead of workspaces with windows

Touchpad gestures are implemented in 44601b8c1c29f0d307686e1ae4885b8e05f91bb9.
Please, let me know how this works for you so we can close this issue.

it works
there are some graphical glitches but i think these are building hyprland from git related

Cool, thanks, closing then...


azure-cli: Update to version 2.62.0 and use zip file instead

What type of Apps related PR is this? (check all applicable)

[ ] ✨ Add Manifest
[x] ♻ Refactor
[x] 🐛 Bug Fix
[ ] 👷 Optimization
[ ] 🚩 Other

Description
Update azure-cli to latest version and use the zip file instead
Related Tickets & Documents
N/A
[optional] What gif/emoji best describes this PR or how it makes you feel?
:egg: or :bug: => :see_no_evil:
Did you verify the followings
Add/Modify a description in the manifest?

[ ] 👍 yes
[ ] 🙋 yes, but I need help
[x] 🙅 no, because they aren't needed

Check URLs?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

Update to the latest version?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

Update Hashes?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

Format the json manifest?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

Did you identify extra steps
Notes in the manifest?

[ ] 👍 yes
[ ] 🙋 yes, but I need help
[x] 🙅 no, because they aren't needed

Application Dependencies?

[ ] 👍 yes
[ ] 🙋 yes, but I need help
[x] 🙅 no, because they aren't needed

Application Suggestions?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

Pre-installation steps to perform?

[ ] 👍 yes
[ ] 🙋 yes, but I need help
[x] 🙅 no, because they aren't needed

Post-installation steps to perform?

[ ] 👍 yes
[ ] 🙋 yes, but I need help
[x] 🙅 no, because they aren't needed

File persistence to perform?

[ ] 👍 yes
[ ] 🙋 yes, but I need help
[x] 🙅 no, because they aren't needed

All changes look good.
Wait for review from human collaborators.
azure-cli

[x] Description
[x] License
[x] Hashes
[x] Checkver
[x] Autoupdate


Add a psplot command to plot and compare stats from multiple runs of psrecord

This allow to compare multiple runs of psrecord on one plot. I have added a new command since it is easier than playing with argparse's sub-parsers.
It takes a list of log files from psrecord, with options to plot only mem or cpu. I don't know what is the best for colors and linestyles to distinguish easily all the lines (suggestions are welcome ;)). Currently it uses mpl's color cycle, so mem and cpu for a file will have the same color (and linestyle).
I didn't modify the plot from main.py, it could probably use the same code, but I'm waiting for your feedback before going further ;-).
usage: psplot [-h] [-m] [-c] [-o OUTPUT] [files [files ...]]

Plot stats from psrecord

positional arguments:
  files                 List of files to plot

optional arguments:
  -h, --help            show this help message and exit
  -m, --mem             Show memory values
  -c, --cpu             Show cpu values
  -o OUTPUT, --output OUTPUT
                        Save the plot to a file

@saimn - sorry for never replying to this. In principle I like this, but I'm wondering if psrecord-compare might be a better command name, so that it's clearer that it's associated with psrecord. Or it could potentially be a special mode of psrecord, e.g. psrecord --compare. Could you rebase this and we can go through a round of comments?

@astrofrog - I had also forgotten this PR ! 😄  Ok for the rebase.

@astrofrog - I pushed the rebased branch with a few updates. About the command interface, I think I chose psplot because it can be useful also to plot one log file, so not only for comparison. There are also several options for the interface:

a new command,
a specific option, --compare or whatever, but it will possibly conflict with the exiting options and make the interface more difficult to understand
subcommands: psrecord plot for instance. This suppose that the existing psrecord command  is moved to a subcommand psrecord run or something. And then I think it is possible to specify a default subcommand, to keep the compatibility.

So... better late than never, am I right? ;)
I think my preferred option would be psrecord plot, so a sub-command.
Is there a way we can re-use the existing plotting code so that if someone does psrecord plot activity.txt they get the same plot that psrecord would have generated originally? I know this also adds the ability to compare, but it would be good to make it so that plotting a single one is consistent.

@astrofrog - Ok ;), I will have a look.

Closing this, anyone interested feel free to reuse this code or not and make a new PR :)


The test error curve on Cifar10.

Can someone show me the test error curve of the Ladder on Cifar10 dataset with 4000 labeled training? I wrote the code with Tensorflow, and got the accuracy of 78% with 20 epoch, and I can almost get the accuracy of 75% after 6 epoch. So, I just want to check if the results are similar. Many Thanks. The Figure shown bellow is my result.
ladder.pdf

Do you know where is the test error on mnist?


added default timeout env variable

It would be nice to pass timeout variable via system properties, so can configure for egxample debug environment with long/infite timeouts


chore: icons with annotation in components

:tada: This PR is included in version 1.0.33 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


powerlogHelperdFix stopped working

powerlogHelperdFix used to work great when I first downloaded it on my IOS 13.5 iPhone 8 because I couldn't see my battery info in settings. But started to not work around a week ago, and I've looked to reddit for help. Some have told me to use choicy, and others system info. They said to use choicy by disabling tweak injection then run ldrestart, so I did. Nothing changed. Turns out ldrestart unjailbroke my phone so I had to rejailbreak. My unjailbroken state brought back the battery info, stopped battery drainage, and cooled down my phone from about 120°F to around 80°F. Felt a big difference. I took that opportunity to rejailbreak and redownload every tweak I had before and double check back in settings if my battery info had disappeared. But even just 1 tweak no matter what it was caused my battery info to disappear and phone started to heat up again.
I have system info as of right now, but it hasn't made any difference if any. The only useful information it's given me was battery capacity, battery health, and CPU temperature. I'm starting to wonder if I could fix my problem with powerHelperdFix, I don't have to worry about downloading anything else and it not working. I'm grasping at straws and don't know what to do.

Yeah nothing works atm except for editing the jetsam plist, the next Substrate update will fix this. Here is a link to the beta: https://github.com/checkra1n/BugTracker/issues/343#issuecomment-659273220

Substrate update is out now... Closing


CI workflows

Issue:
Create CI workflows for test automation. This should include building the plugin and running unit/integration tests.
More on Plugin standards here

CI workflow fails for #17 and #18. The local build pass. This should be resolved by updating the github workflow environment.


cg-detection pipeline is failing due to incorrect pool usage

Failing pipeline run: https://dev.azure.com/dnceng/internal/_build/results?buildId=2534703&view=results
  ##[error]The pipeline is not valid. Could not find a pool with name $(default1ESInternalPoolName). The pool does not exist or has not been authorized for use. For authorization details, refer to https://aka.ms/yamlauthz.

/cc @ellahathaway, perhaps something was missing from https://github.com/microsoft/dotnet-framework-docker/pull/1167?

It looks like this pipeline isn't importing the common variables template where default1ESInternalPoolName is specified. I think that this pipeline should import the template and use the 1es official template too


chirpstack v4 decoder

Can someone post updated script for chirpstack v4 where the call is based on: decodeUplink(input)

Hi, this repo is now out of date. Please use Tektelic's decoders from here: https://github.com/TektelicCommunications/data-converters


Add attribute for renaming properties, remove support for Newtonsoft.Json and System.Text.Json attributes

Motivation:

Using another library attribute for controlling behavior how properties are transformed during wf build phase does not make a lot of sense
JsonPropertyAttribute also contains other properties that can affect behavior of serialization, user should not be allowed to control other aspects of serialization
User should not care(and know) which serializer is used by the library. This feature will also allow us to easily swap Newtonsoft.Json for System.Text.Json if we decide to do it in future

Decided against this for now


Added update branch API

Fixes #1183

@gmlewis Could you please review this PR?

@gmlewis Addressed reviews.
Hope the commits are squashed while merging.

Thank you, @vaibhavsingh97 for the LGTM.
Merging.


DRAFT: Measuring impact of increased resource on database time Min / max / average

Draft
This is a draft issue, and it will need to clarify specs and formalize what
counts as "done" before work begins on it.
Check with the issue's creator before starting work on this issue.
Overview
A clear description of the feature request.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Additional Context
Optional. Add any other context, screenshots, gifs, etc about the feature request here.

Findings are collected here https://docs.google.com/spreadsheets/d/152cB38JL9DD95Pwrh48e4wcGwNC8V9ovqhmjJP9_ngY/edit#gid=0


ring buffer data structure

Write a ring buffer data structure in datastructs/ring.c.  The header file provides an outline.
/* A ring buffer for multi-threaded programs. A single producer pushes
* objects into the ring, while a single consumer pops them off for
* processing. If the ring is full (i.e., the consumer is slower than the
* producer), then the push operation fails.
*/

Feel free to modify the declarations in the header file if you have a better idea.

@huyang1988  @pradeep-k @wenhuizhang

Hi

code merged in #26.


[Overlay] Split out AutoLockScrolling

One component, one feature.
AutoLockScrolling is useful on it's own.

[ ] PR has tests / docs demo, and is linted.
[ ] Commit and PR titles begin with [ComponentName], and are in imperative form: "[Component] Fix leaky abstraction".
[ ] Description explains the issue / use-case resolved, and auto-closes the related issue(s) (http://tr.im/vFqem).

This is very interesting :+1: :+1:

There are issues with AutoLockScrolling in the current implementation with nested dialogs, when they unmount out of order.  This component doesn't make it better or worse, but would be great to consider it as well when thinking about this.

@chrismcv tried a fix :)

gentle ping :)

@chrismcv Is right, the AutoLockScrolling is not handling correctly multiple mounted instances. Sorry @cgestes but I don't think that your fix is perfect :grin:. I think that the scroll show be locked as soon as one < AutoLockScrolling /> is asking for it.
This issue really remembers issues with concurrences access to memory.
What do you think about using a mutex?
Each token would represent the access to the lock ressource. We would only unlocked the screen once all token are returned.

@cgestes What do you think about just extracting the AutoLockScrolling component in this PR, then tackling the nested issue?

What do you think about using a semaphore?

That's really interesting :+1: I think it's the best approach. just keep an integer ( increment with mount decrement with unmount ) and then set with non-zero and set with zero :+1:

@alitaheri & @oliviertassinari I did not know the locker was not supposed to be the unlocker.
Will implement it with a counter, thanks for the idea :)

sorry - been a busy day... I agree, in that I don't think the suggested fix would work.  In my head, moving the lock variable outside the component this scope would be adequate, but counter approach works well too - again - I'm pretty sure it needs to be outside the this scope.

updated the PR with a semaphore :)

This looks good to me :)

Reviewing the commit again...
I dont think it is...
I'll add inline comments

I'll add an extra security.
I believe if we have lock = false we decrease the counter without having incremented it.
Please dont merge now.

I believe this time everything is ok.
It will only unlock if it has been locked.
What do you think?

I'm having a new look at this PR. I don't this that we need the this.locked variable.
I think that lockingCounter is enought.
Can we add unit test to this component? Each line of code matters here. Regression are very likcky.

from what I understand we do.

if lock is false the counter get decremented in CWU but never incremented
the code is safer to reason about

The code is not gonna evolve a lot. it's just a simple component doing one thing with no possible side effect.
It's nice to have test, but that's not critical for that kind of code from my own experience.
I'll try to add some when I'll come back from ski in 10 days.
I'am not sure how to do unittest which have impact on the DOM.

I'll come back from ski in 10 days.

@cgestes Enjoy :snowflake:!

rebased :)

@cgestes Looks like rebasing went a bit haywire, can you try again?

fixed. sorry for the inconvenient.

@cgestes That seems to work correctly :+1:. I'm fine with merging this PR.
But I have some remarks:

I think that we can handle the logic without the locked variable. I can't see casses where allowScrolling or disallowScrolling would not increment / decrement correctly.
We could be using the ES7 class properties for the locked and not calling the constructor()
Some test would be awesome if we want to refactor this component later.

this.locked is extra safety and code readability.
For example when you mount with lock=false and you unmount this ensure that the counter is not decremented.

rebased on @mbrookes changes :)

For example when you mount with lock=false and you unmount this ensure that the counter is not decremented.

True, we have this line to prevent this issue.

rebased on @mbrookes changes :)

Thanks.

second rebase done :)


Mainnet SS signs PSBT created for testnet3, displays mainnet addresses.

Posting this for informational purposes only.  I'm not convinced that anything "needs" be changed.  Can close this after discussions so it will still be searchable in the future.
I believe this has been reported in the past, but before today I was not able to read-enough into the complaint to be able to reproduce it.
In telegram, it was reported:
Daniel, [7/13/23 2:52 AM]
Hello! We were yesterday testing some multisig in our local meetup and we found something awkward. I don’t know if it’s a bug or intended behavior:

Created multisig 2-3 in sparrow and nunchuk, testnet. I’m some moment the power was off on the seed signer and it reseted on mainnet, we restore the 3 signatures and make a transaction. Then we signed those transactions with the seedsigner in mainnet, the address shown on the seedsigner were different than the one we have on sparrow/nunchuk. The transaction was equally signed and broadcasted without issues.

Daniel, [7/13/23 2:53 AM]
The only problem is the screen on the seedsigner showing a different address than sparrow. Also tried the seedsigner in regtest and signed transactions in testnet… this is intended behavior?

Daniel, [7/13/23 2:53 AM]
Is there a way to alert or know if the transaction is on a different network? Thank you! 🙏

Indeed, I was able to reproduce this by setting seedsigner to "mainnet" and creating similar 2of2 PSBT transactions from Specter Desktop and Sparrow.
Seedsigner loads the PSBT, does not warn, displays mainnet addresses for the correct derivation path and address index, correctly identifies change similarly for a mainnet address, and signs the transaction.
I've concluded that at this time, seedsigner did nothing wrong because there is no information in the PSBT (except for perhaps hinting that the derivation path is "non-standard" while it would still be perfectly valid) that indicates the transaction is for one network or another.  The resulting signed transaction would appear to be of valid construction for any network, and would only fail once the node realizes that the inputs are not valid unspent utxos... which is out of reach for any airgapped signing device.  It is up to the user to review the payment/self-transfer and change addresses ON THE SEEDSIGNER to know that something is wrong.  But if they didn't, since the utxo being spent do not exist on the wrong network, the resulting signed PSBT could not be broadcasted, and no harm would be done.
Parsing a Sparrow SpecterDesktop PSBT for a 2of2 Native Segwit wallet on testnet3
A PSBT can be thought of a string of bytes representing (magic, global-map, input-map, output-map).
Nothing I saw "screams" that this transaction is for one network or another, but there was a hint in derivation paths on inputs and outputs.
magic
"psbt" + 0xff, just as expected
global-map
PSBT_GLOBAL_UNSIGNED_TX containing a rawtransaction w/o signature/witness data.
0x00 just as expected to terminate the global-map
input-map
PSBT_IN_NON_WITNESS_UTXO the complete segwit rawtransaction previously confirmed in the blockchain -- the only input being spent.
PSBT_IN_WITNESS_UTXO amount and scriptPubKey of the input.
PSBT_IN_WITNESS_SCRIPT the witnessScript for the input
2 PSBT_IN_BIP32_DERIVATION values, one for each pubkey that must sign, w/ master fingerprint and derivation path of the input being spent
this might be useful if we'll hint that "maybe on wrong network" or "non-standard path".
0x00 just as expected to terminate the input-map
output-map
PSBT_OUT_WITNESS_SCRIPT the ouput witnessScript for the payment leaving the wallet.
2 PSBT_OUT_BIP32_DERIVATION values, one for each pubkey in the wallet, w/ master fingerprint and derivation path of the change coming back.
this might be useful if we'll hint that "maybe on wrong network" or "non-standard path".
0x00 as expected to terminate the output-map
0x00 as expected to terminate the psbt.

Gory details of the PSBT sample I was using, broken down below:
sparrow psbt 2of2 native-segwit spending 100,000 sats to 10,000 sat p2sh output and rest back to original wallet w/ 201s fee.

raw base64 psbt: cHNidP8BAH4CAAAAAakI7tv64ZoVu+wCDJyaS6CVhswD9/0WSADPZKE6kb+FAAAAAAD9////AsdeAQAAAAAAIgAgMcuwXUdM9TLvGHUsUtRIkSXbd2SRSc3gV5CJ9w5/NuAQJwAAAAAAABepFKuw5crTHMj5gcddhMoEnbcQJOtAhyGVFQAAAQB9AgAAAAEDBJ3MVsOE9Wl+2mp056YpCDI7f4Skk2XcdAW+ztzTKQAAAAAA/f///wKghgEAAAAAACIAIA00YGDExVb1hP3imMDzJMfzBQYhI8bue5cjsX7e3BWlshgGAAAAAAAWABTNZPpG+a5KwxfL+CSWxAYVvSa5fCGVFQABASughgEAAAAAACIAIA00YGDExVb1hP3imMDzJMfzBQYhI8bue5cjsX7e3BWlAQVHUiEDJ6LYAkzJsbypkD7x/f7mpZlvECGjVU0ieiMm7WEX9H8hA3a+aK2TxUJgyW0DjNoSCRsr9LFZ6Hud2BadCLHgmK4aUq4iBgMnotgCTMmxvKmQPvH9/ualmW8QIaNVTSJ6IybtYRf0fxwoZ+cfMAAAgAEAAIAAAACAAgAAgAAAAAABAAAAIgYDdr5orZPFQmDJbQOM2hIJGyv0sVnoe53YFp0IseCYrhoczvQ2BDAAAIABAACAAAAAgAIAAIAAAAAAAQAAAAABAUdSIQOLt87nFXZbkOhtNjIEso8bRK6H7FOaNbXnEcz+0Y6ymSED8RoLwytMPuZ4JQycK01oFWoX+l++Sbs6Wqwz9Oe2tmBSriICA4u3zucVdluQ6G02MgSyjxtErofsU5o1tecRzP7RjrKZHM70NgQwAACAAQAAgAAAAIACAACAAQAAAAMAAAAiAgPxGgvDK0w+5nglDJwrTWgVahf6X75JuzparDP057a2YBwoZ+cfMAAAgAEAAIAAAACAAgAAgAEAAAADAAAAAAA=

b64decode(psbt).hex():
70736274ff01007e0200000001a908eedbfae19a15bbec020c9c9a4ba09586cc03f7fd164800cf64a13a91bf850000000000fdffffff02c75e01000000000022002031cbb05d474cf532ef18752c52d4489125db77649149cde0579089f70e7f36e0102700000000000017a914abb0e5cad31cc8f981c75d84ca049db71024eb4087219515000001007d020000000103049dcc56c384f5697eda6a74e7a62908323b7f84a49365dc7405becedcd3290000000000fdffffff02a0860100000000002200200d346060c4c556f584fde298c0f324c7f305062123c6ee7b9723b17ededc15a5b218060000000000160014cd64fa46f9ae4ac317cbf82496c40615bd26b97c2195150001012ba0860100000000002200200d346060c4c556f584fde298c0f324c7f305062123c6ee7b9723b17ededc15a501054752210327a2d8024cc9b1bca9903ef1fdfee6a5996f1021a3554d227a2326ed6117f47f210376be68ad93c54260c96d038cda12091b2bf4b159e87b9dd8169d08b1e098ae1a52ae22060327a2d8024cc9b1bca9903ef1fdfee6a5996f1021a3554d227a2326ed6117f47f1c2867e71f30000080010000800000008002000080000000000100000022060376be68ad93c54260c96d038cda12091b2bf4b159e87b9dd8169d08b1e098ae1a1ccef43604300000800100008000000080020000800000000001000000000101475221038bb7cee715765b90e86d363204b28f1b44ae87ec539a35b5e711ccfed18eb2992103f11a0bc32b4c3ee678250c9c2b4d68156a17fa5fbe49bb3a5aac33f4e7b6b66052ae2202038bb7cee715765b90e86d363204b28f1b44ae87ec539a35b5e711ccfed18eb2991ccef43604300000800100008000000080020000800100000003000000220203f11a0bc32b4c3ee678250c9c2b4d68156a17fa5fbe49bb3a5aac33f4e7b6b6601c2867e71f3000008001000080000000800200008001000000030000000000

broken-down:

70736274ff = magic ("psbt" + 0xff)

# global-map
01 00 = keytype: PSBT_GLOBAL_UNSIGNED_TX
7e<PHONE_NUMBER>a908eedbfae19a15bbec020c9c9a4ba09586cc03f7fd164800cf64a13a91bf850000000000fdffffff02c75e01000000000022002031cbb05d474cf532ef18752c52d4489125db77649149cde0579089f70e7f36e0102700000000000017a914abb0e5cad31cc8f981c75d84ca049db71024eb408721951500 = value

00 = end of global-map

# input-map
01 00 = keytype: PSBT_IN_NON_WITNESS_UTXO
7d<PHONE_NUMBER>03049dcc56c384f5697eda6a74e7a62908323b7f84a49365dc7405becedcd3290000000000fdffffff02a0860100000000002200200d346060c4c556f584fde298c0f324c7f305062123c6ee7b9723b17ededc15a5b218060000000000160014cd64fa46f9ae4ac317cbf82496c40615bd26b97c21951500 = value

01 01 = keytype: PSBT_IN_WITNESS_UTXO
2b a0860100000000002200200d346060c4c556f584fde298c0f324c7f305062123c6ee7b9723b17ededc15a5 = value

01 05 = keytype: PSBT_IN_WITNESS_SCRIPT
47 52210327a2d8024cc9b1bca9903ef1fdfee6a5996f1021a3554d227a2326ed6117f47f210376be68ad93c54260c96d038cda12091b2bf4b159e87b9dd8169d08b1e098ae1a52ae = value

22 06 = keytype: PSBT_IN_BIP32_DERIVATION
0327a2d8024cc9b1bca9903ef1fdfee6a5996f1021a3554d227a2326ed6117f47f = keydata: this pubkey
1c 2867e71f 30000080 01000080 00000080 02000080 00000000 01000000 = value: fingerprint 48h 1h 0h 0 1

22 06 = keytype: PSBT_IN_BIP32_DERIVATION
0376be68ad93c54260c96d038cda12091b2bf4b159e87b9dd8169d08b1e098ae1a = keydata: this pubkey
1c cef43604 30000080 01000080 00000080 02000080 00000000 01000000 = value: fingerprint 48h 1h 0h 0 1

00 = end of input-map

# output-map
01 01 = keytype: PSBT_OUT_WITNESS_SCRIPT
47 5221038bb7cee715765b90e86d363204b28f1b44ae87ec539a35b5e711ccfed18eb2992103f11a0bc32b4c3ee678250c9c2b4d68156a17fa5fbe49bb3a5aac33f4e7b6b66052ae = value

22 02 = keytype: PSBT_OUT_BIP32_DERIVATION
038bb7cee715765b90e86d363204b28f1b44ae87ec539a35b5e711ccfed18eb299 = keydata: this pubkey
1c cef43604 30000080 01000080 00000080 02000080 01000000 03000000 = value: fingerprint 48h 1h 0h 1 3

22 02 = keytype: PSBT_OUT_BIP32_DERIVATION
03f11a0bc32b4c3ee678250c9c2b4d68156a17fa5fbe49bb3a5aac33f4e7b6b660 = keydata: this pubkey
1c 2867e71f 30000080 01000080 00000080 02000080 01000000 03000000 = value: fingerprint 48h 1h 0h 1 3

00 = end of output-map

00 = end of psbt

All of that manual parsing yesterday was for a SpecterDesktop PSBT.  My statements above remain my opinion for SpecterDesktop PSBTs... but since realizing that I had not addressed the PSBT from the original complaint, I spent time implementing a half-finished bip174 psbt parser, that will make looking at psbts much easier in the future.
SPARROW DOES INDEED HAVE ENOUGH INFO, via an optional field, for seedsigner to improve/warn the user that they're on the wrong network.
The PSBT global-map has, optionally since version 0, keytype:PSBT_GLOBAL_XPUB, which contains keydata that is a complete 78 byte bip32 xpub where the first 4 bytes are the network magic for mainnet or testnet.  The value is less interesting, it's the master-wallet-fingerprint + 4byte little endian path elements, concatenated.
I will need to take a look at our psbt parser and see where we might make changes to take advantage of this extra field that exists in Sparrow, but not SpecterDesktop xpubs.  As well, worth looking at similar from our other supported coordinators so that when a change is made, we'll know that it helps users of x,y,z coordinators.

@jdlcdl it may also make sense to make PRs into embit and Specter Desktop or at least see if there have been any discussion of that field in those repos.

I've since learned that nunchuk also has the PSBT_GLOBAL_XPUB present in at least single-sig native segwit PSBTs.
Also that at a glance, embit seems to faithfully support bip174 (either outright else PSBT objects will have inputs.unknown and outputs.unknown so that unsupported keytypes will get passed through.)  For that which is fully supported by embit, which includes PSBT_GLOBAL_XPUB, can do it the embit friendly way via PSBT.xpubs... but I'll have to verify/play.

I've poked around in embit and have a sample script that I believe illustrates PSBT_GLOBAL_XPUB can be accessed the 'embit' way via embit.PSBT.xpubs which behaves like a dict whose key is an embit.HDKey, which will have a ".version" attribute set to the network (main or test) that the wallet was created for (signet and regtest are same as test).
from base64 import b64decode
from embit.psbt import PSBT
from embit.networks import NETWORKS

if __name__ == '__main__':

    psbts = {
        'main': 'cHNidP8BAHECAAAAAdCEXX0XBXxR6G0dD95gpJhHbIT/lF60ThGfrg7my8AIAQAAAAD9////AhAnAAAAAAAAFgAUrqizVmmVhneVbnOSCadUp+hpTEQxANwAAAAAABYAFJNP3T/N7XdJnCTEsdK4zTNlr022AAAAAE8BBIiyHgMMRg9ggAAAABg5cK8myN4on9odN8WutS/eUTTRSGzSr0ROX6xjaeM/Az24ab/atYzPxNKwag3Sjowqk0rf5frw7z1X+mDIjeglEA55sJ1UAACAAQAAgAAAAIAAAQByAgAAAAHLKVpQmEIPT7rcZN1Ep/ucMTCGJ88GJb2gibOBJsdHCQAAAAAA/f///wJAS0wAAAAAABepFBfnfQKWZZ5DrrBD5SXD2zb5Ugkih84n3AAAAAAAFgAUJtD2n29RhvcJR+meZzz4H6dqC/whlRUAAQEfzifcAAAAAAAWABQm0Pafb1GG9wlH6Z5nPPgfp2oL/CIGAlc/HJvHl4CaJdIDHFJw8JLNavfO6HD+2o/UN82xoydAGA55sJ1UAACAAQAAgAAAAIABAAAAAwAAAAAiAgPj1ZMkmMtFkxy9dA5V8diy7EQBFw1njqZIOwt7+9wGLhgOebCdVAAAgAEAAIAAAACAAAAAAAkAAAAAIgICi586gJLYLYS2nojGdgII8F/lCij39uaO8p03Ihyng8EYDnmwnVQAAIABAACAAAAAgAEAAAAGAAAAAA==',
        'test': 'cHNidP8BAHECAAAAAdCEXX0XBXxR6G0dD95gpJhHbIT/lF60ThGfrg7my8AIAQAAAAD9////AhAnAAAAAAAAFgAUrqizVmmVhneVbnOSCadUp+hpTEQxANwAAAAAABYAFJNP3T/N7XdJnCTEsdK4zTNlr022AAAAAE8BBDWHzwMMRg9ggAAAABg5cK8myN4on9odN8WutS/eUTTRSGzSr0ROX6xjaeM/Az24ab/atYzPxNKwag3Sjowqk0rf5frw7z1X+mDIjeglEA55sJ1UAACAAQAAgAAAAIAAAQByAgAAAAHLKVpQmEIPT7rcZN1Ep/ucMTCGJ88GJb2gibOBJsdHCQAAAAAA/f///wJAS0wAAAAAABepFBfnfQKWZZ5DrrBD5SXD2zb5Ugkih84n3AAAAAAAFgAUJtD2n29RhvcJR+meZzz4H6dqC/whlRUAAQEfzifcAAAAAAAWABQm0Pafb1GG9wlH6Z5nPPgfp2oL/CIGAlc/HJvHl4CaJdIDHFJw8JLNavfO6HD+2o/UN82xoydAGA55sJ1UAACAAQAAgAAAAIABAAAAAwAAAAAiAgPj1ZMkmMtFkxy9dA5V8diy7EQBFw1njqZIOwt7+9wGLhgOebCdVAAAgAEAAIAAAACAAAAAAAkAAAAAIgICi586gJLYLYS2nojGdgII8F/lCij39uaO8p03Ihyng8EYDnmwnVQAAIABAACAAAAAgAEAAAAGAAAAAA==',
    }

    for my_net in ['main', 'test', 'regtest', 'signet']:
        print("\nSigner intends to use network: %s" % my_net)

        for psbt_net, psbt in psbts.items():
            psbt = PSBT.parse(b64decode(psbt))

            xpub_versions = set([k.version for k in psbt.xpubs])
            if NETWORKS[my_net]['xpub'] in xpub_versions:
                print('processing a %s psbt while signer on network:%s...' % (psbt_net, my_net))
            else:
                print('WARNING: psbt network:%s while signer on %s!' % (psbt_net, my_net))

@jdlcdl it may also make sense to make PRs into embit and Specter Desktop or at least see if there have been any discussion of that field in those repos.

On SpecterDesktop, I have changed my device from "seedsigner" to "coldcard" and exported a PSBT which does contain the desired PSBT_GLOBAL_XPUB keytype.   As we get closer to merging support for this in seedsigner, I think that the solution will be to request a configuration change for seedsigner signing devices so that the optional param "xpubs=True" is passed to the Wallet.fill_psbt() method.  However, I don't suspect that there would be any problem if SpecterDesktop did this before we supported it... we're simply ignoring it even though embit has parsed this info and made it easily available.
https://github.com/cryptoadvance/specter-desktop/blob/8f9c95fe1b45e638d2997e1392857a96b391f09b/src/cryptoadvance/specter/wallet/wallet.py#L1756
I will refrain from adding a new issue to the cryptoadvance/specter-desktop repo until others have taken a look at its resolving pr.  Once others are ok, perhaps I'd word it as follows:

Feature Request: PSBT_GLOBAL_XPUB for PSBTs exported to SeedSigner
PSBT_GLOBAL_XPUB is not an included keytype for PSBTs exported to seedsigner; it is available when device is "coldcard"
While seedsigner does not yet take advantage of this field, it would be available via embit in PSBT.xpubs and would be useful to inform the user when they have setup seedsigner to use testnet but are about to sign a PSBT intended for mainnet.  Without this information, seedsigner processes the PSBT all the way to a valid signature but shows testnet addresses instead of mainnet addresses.


feat: support analyzing images

Description
Support analyzing images
Related issue (if any): fixes #issue_number_goes_here
Checklist

[x] I have followed this repository's contributing guidelines.
[x] I will adhere to the project's code of conduct.

Additional information

Anything else?

@xieshenzh great job!
it's missing the documentation part to be added to README.md. Please add it ( for the new 2 methods added to the API, including usage with an example), including related configuration as environment variables/system properties, and unless you need some more time for checking/testing the integration of the api client with exhort backend , then i think it will be enough to be merged. ( again, really very good job!)
Please pay attention, that because snyk was removed, i had to change the integration tests, please rebase your work on top of the main branch, to overcome failure of integration tests.
Thanks!.

Thanks @zvigrinberg . I have updated the README.md with docs for the two new methods.


Future Request

Is your feature request related to a problem? Please describe.
I want to make an account
Describe the solution you'd like
Add the feature to make accounts

For the time being, I am using GitHub Pages to deploy the website and build the UI. GitHub-Pages only supports static sites, meaning it can only render what is in the HTML code, and there is no user interaction. While static sites are the ideal choice for blogs and portfolios, they have their limitations when creating a website that requires multiple users, databases, and user interaction. That is why I will use AWS (Amazon Web Services) to deploy the Woof and use python to handle the accounts once finished with the UI.


GEODE-6975: Reply with exception if fail to persit sender's new id.

Reply with ReplyException when processing PrepareNewPersistentMemberMessage
failed.
Retry again if remote member failed to persist the new persistent member id.
Make sure atomicCreation flag is reset, so that redundancy could be satisfied
during bucket creation retry if previous attempt failed.

I think it might be better to simply not rely in the case of these cancel exceptions - or maybe wait to process the member departure on the sending side of this PrepareNewPersistent...Message?
You added some retries in the case of this exception, but that looks like it's only for when we are creating the bucket for the very first time? What about when we are recovering the bucket from disk? We don't want that operation to fail if one of our peers is shutting down at the time we are recovering - and I don't think that has a retry.

You are right, retry for recovering case only occurs in getFromOne during GII. We do need to handle the case when recovering without GII.


Remove scheme when not present

I'm not sure if this is a bug or it was intentional. When using Xamarin builder the Scheme is left empty (for obvious reasons)
Scheme should not be shown unless one exists

It's not intentional. This step was initially a closed-source tool and we at Netguru don't develop in Xamarin, which means we always have a present Xcode scheme.

Scheme is also missing when building Android app.


refreschCache param doesnt work with Service Worker

We are using a Service Worker to cache GraphQl responses for 2 hours. Sadly this has the side effect that Rendertron is using this Service Worker caching as well, even when using the refreshCache param.
I would have expected the refreshParm to also stop this from happening.

Could you share a test case for this? And if you want to give fixing it a go, let me know!

@AVGP
I have the following Scenario:

We are using a graphql call to obtain the contents of a page.
We have a service-worker.js which caches the graphql call.
Sometimes the graphql call, causes a error on fetching the data. This is now cached by Rendertron.
Using the refreshCache=true parameter does not flush the service-worker.
Restarting Rendertron does remove this cache.

Would you be able to reproduce this on a super simple web page that uses a service worker that loads and caches some simple data?


[unpack] Golang pure unpack

Added support for decompressing tar.xz and tar.gz files, without exec(tar)
Allows to unzip .tar files in windows
Details

To decompress tar.gz the standard library is used
To decompress tar.xz files, github.com/ulikunitz/xz was used, which is a 100% go library, it should work on a windows installation

test

Linux (tar.gz, tar.xz)
Windows (tar.gz)

Curlie works on windows, although the recipe needs to be modified, please add a related pull request

Thanks for adding this change. I experimented with this as well when I was working with extraction. The main issue I had with them was that the decompression algorithms seem to run much slower than what runs with tar -xf  [FILE]. Hopefully they will improve sometime in the near future.
I like the change though... do you think the slowdown is worth the improved compatibility?
The main upside I see with this is that we could now do cross-platform installation validation/testing. From Windows, Linux, or Mac, we could test an installation for each of those platforms.

the loss of performance I also considered, the compatibility seemed to me to be a better advantage.
we might consider checking if 'tar' is installed and if so run it. in this way we would have a compatible alternative and a high performance one. and the compatible way can have a progress bar, this will help the user to know what is happening in the background which gives me a very big advantage
@candrewlee14

Yeah, I decided the compatibility was worth it too. It would be a massive improvement to have testing for all binaries for all platforms, so that's something I'll be working on. The progress bar for decompression is a great idea if you'd like to work on that.


workbench.action.showEditorsInGroup keybinding triggers wrong query

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.85.1
OS Version: Darwin x64 23.2.0

Steps to Reproduce

Trigger workbench.action.showEditorsInGroup
Trigger workbench.action.showEditorsInActiveGroup

Expected
Same dialog but different order and query strings: 1) edt vs 2) edt active.
Actual
Both edt active.
Problem
Passes constant from other command:
https://github.com/microsoft/vscode/blob/3ddf196d27d71d2f5539db607b1a10adb3d0800c/src/vs/workbench/browser/parts/editor/editorCommands.ts#L1565
Note
It also isn't translated (including English).

The described behavior is how it is expected to work. If you disagree, please explain what is expected and what is not in more detail. See also our issue reporting guidelines.
Happy Coding!

That is intentional. Can you show where something is not translated?

The actions aren't translated in the keybinding UI. They're rendered as the action identifier. Same for the related quick commands.

That is intentional.

Is there any difference at all?
Can 1 be removed?
At the very least it's confusing.
The behavior I'm looking for/expected is specifically not MRU.

@bpasero @chrmarti I don't believe this works as intended unless they're intentionally duplicated. Can you please confirm?

I do not understand what the bug is here: both should show editors of the active group, what am I missing?

My expectation was 1 is MRU and the other in visual (or at least stable) order. I'm not implying it should behave like I want. But I assume different commands should have distinct (vs identical) behavior. I think 1 should either be adjusted or removed and either/both have a language string to it/them.

Yes, its not unusual to have multiple commands doing the same, they might be used internally in different contexts or with arguments.
To get edt showing, you can use workbench.action.showAllEditors


[Event Request] Page 35516 Cash Receipt Journal FactBox - OnAfterUpdateInfoBoxRepeat

Hi Team,
In Page 35516 we need possibility to have our additional checks . Having event in the procedure UpdateInfoBox, would be a perfect solution :

[IntegrationEvent(false, false)]
local procedure OnAfterUpdateInfoBoxRepeat("Posting Date": Date; PostingDate: Date; CustLedgEntry: Record "Cust. Ledger Entry"; PmtDiscDate: Date; DueDate: Date; AgeDays: Integer; PaymDiscDays: Integer; DueDays: Integer)

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


Additional information in job status change Qiita emails

(Not sure if this is the right place for this!)
It would be beneficial to have additional information in "Job Status Change" emails regarding the artifact that is moving through the workflow pipeline (as well as error emails). This could include prep # (& data type) and Qiita IDs affected and can be located before the "New Status" line:

This is helpful when you are getting multiple emails from multiple jobs you are running, or if you are coming back from out of office and need a refresher on what jobs and projects are being referred to.

Thank you for opening @mmbryant23; the issue should be in qiita and I just moved it.

As a reminder, there are 3 types of input artifacts that a job can use:

processing (it has a prep id linked): it should display the study id (link), prep ids, data_type and command name
analysis (it has an analysis linked): it should display the analysis id (link)  and command name
jobs without input artifact (admins jobs): the command name and parameters submitted

@antgonza @mmbryant23 Just an FYI, the line 'New status: Error' line is the subject line of the email, so it's not possible to put something above it. We could change the subject line itself to include the info Antonio mentions above, or make it the first line of the body of the email, above "Error:'.


Can It Install on New HarmonyOS 2.0

I try to install using nonroot, waiting about 1 hr process install vanced microG, just see bar animation.

Are you still facing this issue?

Are you still facing this issue?

I try many times to install on Huawei MatePad 11 , not success install it .
I remember it could run YT Music , but not YT.
After that, I just Uninstall all, just use firefox shortcut to open it.
They are any member / new Vanced update for install correctly on HarmonyOS 2.0?

Harmony OS probably just don't like vanced. There's nothing we can do.

Closing this issue due to inactivity.


feat: support RBAC model in permission

Fix: https://github.com/casdoor/casdoor/issues/981
Confused about implementing domains, but feel free to test the RBAC model.
Signed-off-by: Yixiang Zhao<EMAIL_ADDRESS>

@KrishnaPG plz review