How can I update the version of CFLint installed with the ColdFusion Builder extension?

I looked at the post here: https://github.com/cfjedimaster/CFLint-Extension.  The instructions indicate:

"To update the CFLint bits, simply download the release and copy the jars into the extension's cflint_lib folder"

I'm not sure exactly where/which jars I'm supposed to copy after I download CFLint.

Great question.  I don't use CF Builder myself, and that extension hasn't been updated in a while.  But you should be able replace ALL the jars in cflint_lib with the all jar from maven.
https://search.maven.org/remotecontent?filepath=com/github/cflint/CFLint/1.1.0/CFLint-1.1.0-all.jar

Thanks very much for the quick response and suggestion.  I replaced all the jars with the one from above.  When I run CFLint from CFBuilder I receive the attached error.  If you have any suggestions that'd be great.  If not I totally understand.

Try changing line 4 of runlinter.cfm from:
linter = createObject("java","com.cflint.CFLint");
to
linter = createObject("java","com.cflint.CFLint").init(javacast('null','');
That should fix it.  CFLint has grown significantly since that extension was implemented.

@ccsimmons When you get it going, could you PR the necessary changes to Ray's project and would you mind writing up a brief setup documentation for CFBuilder / CFLint on our wiki?

@TheRealAgentK sure thing... @ryaneberly suggestion gets it running (after adding a closing parenthesis)... but I don't think it is actually finding "issues".  See the attached little example where it should have returned avoid-using-cfdump-tag and avoid-using-cfabort-tag.  CFLint in Atom returns them.  I poked around in Ray's runlint.cfm but didn't really see anything that was obvious to me.

Is runlint.cfm specifically setting certain rules to be used/not used maybe?
On Sun, 2 Jul 2017 at 12:13 PM, Chris Simmons<EMAIL_ADDRESS>wrote:

@TheRealAgentK https://github.com/therealagentk sure thing...
@ryaneberly https://github.com/ryaneberly suggestion gets it running
(after adding a closing parenthesis)... but I don't think it is actually
finding "issues". See the attached little example where it should have
returned avoid-using-cfdump-tag and avoid-using-cfabort-tag. CFLint in Atom
returns them. I poked around in Ray's runlint.cfm but didn't really see
anything that was obvious to me.
[image: screen shot 2017-07-01 at 8 02 36 pm]
https://user-images.githubusercontent.com/1590940/27766257-851cc7b6-5e98-11e7-90ec-2700fd6ccf29.png
[image: screen shot 2017-07-01 at 8 02 53 pm]
https://user-images.githubusercontent.com/1590940/27766260-885504b6-5e98-11e7-9f8b-991000ed8c3c.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/cflint/CFLint/issues/327#issuecomment-312462637, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AATrRDbLuh0VF6KwoGABZ7lX_bPMpAfXks5sJuCRgaJpZM4OLWhO
.

That's kind of what I wondered but I don't see anything obviously specifying it.
`
linter.scan(selection.path);
bugs = linter.getBugs();
//Do some massaging on the bugs to make it easier below
bugStruct = bugs.getBugList();
//this is a struct of issues, convert to one array
bugArr = [];
`
It looks like this bit above does the scan and then there is a loop over the bugStruct to append to bugArr (which is looped over to display the issues).
If I dump bugStruct it's empty.

Hmm... I don't have CFBuilder - hard to properly test. I'll try to whip up a simple CFML page trying to run the essence of calling CFLint through CFML and see if I can reproduce it.

@ccsimmons
In a nutshell: the issue is that there's no empty CFLint constructor anymore. You'll need to pass in a configuration object and because CF is really clunky when it comes to calling Java, it's a bit messy.
I believe something like this should work from a code point of view (at least as a starting point):
<cfscript>
    configUtils = createObject("java","com.cflint.config.ConfigUtils");

    pluginInfo = createObject("java","com.cflint.config.CFLintPluginInfo");
    pluginInfo = configUtils.loadDefaultPluginInfo();

    defaultConfig = createObject("java","com.cflint.config.CFLintConfig") ;
    defaultConfig.setRules(pluginInfo.getRules());

    myConfig = createObject("java","com.cflint.config.CFLintChainedConfig").init(defaultConfig);

    linter = createObject("java","com.cflint.CFLint").init(myConfig);
    WriteDump(linter);

    linter.scan("/Users/kai/Documents/Code/cupid-work/556503/ukrainedatecom/html/com");

    bugs = linter.getBugs();
    WriteDump(bugs);
</cfscript>

However, when I run it in my ACF 11 locally, I get an exception in my logs that point towards some logging issue/conflict:
Jul 02, 2017 1:12:12 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [CfmServlet] in context with path [] threw exception [Servlet execution threw an exception] with root cause
java.lang.NoClassDefFoundError: org/apache/logging/log4j/LogManager
	at net.htmlparser.jericho.LoggerProviderLog4J.getLogger(LoggerProviderLog4J.java:35)
	at net.htmlparser.jericho.LoggerProviderLog4J.getSourceLogger(LoggerProviderLog4J.java:41)
	at net.htmlparser.jericho.Source.newLogger(Source.java:1685)
	at net.htmlparser.jericho.TagType.getLogger(TagType.java:699)
	at net.htmlparser.jericho.TagType.register(TagType.java:131)
	at cfml.parsing.cfmentat.tag.CFMLTags.register(CFMLTags.java:61)
	at cfml.parsing.CFMLSource.<init>(CFMLSource.java:26)
	at com.cflint.CFLint.process(CFLint.java:242)
	at com.cflint.CFLint.scan(CFLint.java:209)
	at com.cflint.CFLint.scan(CFLint.java:198)
	at com.cflint.CFLint.scan(CFLint.java:173)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at coldfusion.runtime.StructBean.invoke(StructBean.java:508)
	at coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:2553)
	at cfjenkins12ecfm437854318.runPage(/Applications/ColdFusion11/cfusion/wwwroot/jenkins1.cfm:15)
	at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:246)
	at coldfusion.tagext.lang.IncludeTag.handlePageInvoke(IncludeTag.java:736)
	at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:572)
	at coldfusion.filter.CfincludeFilter.invoke(CfincludeFilter.java:65)
	at coldfusion.filter.IpFilter.invoke(IpFilter.java:45)
	at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:487)
	at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:42)
	at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
	at coldfusion.filter.PathFilter.invoke(PathFilter.java:142)
	at coldfusion.filter.LicenseFilter.invoke(LicenseFilter.java:30)
	at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:94)
	at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
	at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
	at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:58)
	at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
	at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
	at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)
	at coldfusion.CfmServlet.service(CfmServlet.java:219)
	at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
	at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at coldfusion.inspect.weinre.MobileDeviceDomInspectionFilter.doFilter(MobileDeviceDomInspectionFilter.java:121)
	at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:450)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)

Don't know if this helps... but it works if I copy the jar from the Atom plugin (.atom/packages/linter-cflint/bin/CFLint-0.10.0-all.jar) to cflint_lib instead of the one from https://search.maven.org/remotecontent?filepath=com/github/cflint/CFLint/1.1.0/CFLint-1.1.0-all.jar

Sorry, I must have been writing my comment as you were posting yours.

@ccsimmons Re the older version - I'm pretty sure that'd be the case because the way how CFLint gets executed and setup seems to have changed quite a bit between 0.10 and 1.0/1.1. See my other post re the setup - but I don't know what to do about the logging exception. That might be an issue on CF11 specifically, not sure how CFBuilder executes this and deals with library requirements.

It seems that if I put CFLint-1.1.0-all.jar in cfusion/runtime/lib instead of cfusion/lib it'll execute just fine and not throw the exception.
You will need to add bugStruct = bugs.getBugList(); to my code from above.
Also: What I've pieced together up there is just a very, very simple default config. If you want to mimic what all the configuration options can do (with .cflintrc for directories, command-line config etc) I'm pretty sure you'd need to do more than that.
It might be worthwhile to go through CFLintMain (.main() as well as .execute()) to see and understand how that's all pieced together now.
I strongly recommend you do that in the dev branch as we're releasing 1.2.0 from it very soon.

I was just about to post that I integrated what you gave me into runlint.cfm and it worked!  I'm just starting to fiddle around with .cflintrc and configuration options.

If you could manage to put together something that's working and configurable based on the code I gave you, I wonder if we could put together "an entry" point into the CFLint system that can be called in an easier way from CFBuilder.
For instance com.cflint.ide.CFLintCFBuilder --- in a similar way to how there is a CFLintTask as an entry point for Ant. @ryaneberly  ???

Why was the no-argument constructor removed from com.cflint.CFLint? The commit message just says "clean up, decomposing the CFLint monolith a bit, remove unused methods". So I guess maybe because it was unused in this project, but that discounts external usage. I think having it as a kind of default constructor makes sense if you don't care about custom configuration.

@TheRealAgentK so would the parsing of .cflintrc need to be something done in the ColdFusion Builder extension, or would ColdFusion Builder check for the existence of .cflintrc in the project and pass the file into CFLinter for parsing (and if the file does not exist use the code above for the default settings)?

I think rather than CFLintCFBuilder,  we just need to generically support an api, just like we support the CLI.

@ccsimmons I'm not 100% sure how CFBuilder works - does it trigger cflint for the current file or for a path? I think in general cflintrc would be checked by CFLint and not by the extension, but I'm pretty sure that the code I've pulled together would not deal with anything else but a default config.

@ryaneberly yes!

The CFBuilder extension checks to see if the "selection" is a file or folder but really only for the purpose of customizing the header of the resulting output to the user.  It uses linter.scan(selection.path); whether the path is a folder or a file and passes either the folder path or the file's path depending on what was selected.  I did also notice if you select multiple files the only one that gets scanned is the first one that was clicked.

@ccsimmons If I'm understanding the scan method correctly, it will only load the .cflintrc if it's scanning a folder. Unfortunately, for an individual file it doesn't even check. You would be able to pass in the config to the constructor, but that too is not so simple as the loadConfig method I found, which would be used for parsing that, is private.

How are you going with this?

Sorry for the delay.  I submitted a PR to Ray a few minutes ago for an update that uses the new jar and the default configuration.  What Wiki page would you like me to put the "brief setup documentation for CFBuilder / CFLint" on?

We just talked about where to put those IDE integrations and we prefer to - in general - get people to include/host them with the respective project. The CFLint Wiki will eventually go away and all the content will be delivered via the README and associated .md files.
So, ideally a paragraph with some basic info would continue to live on our end here and ideally the instructions on how to get CFBuilder going with CFLint would live on the CFBuilder-CFLint-Extension README. Maybe you could build it into the README and PR that to Ray too?

I see the PR was accepted by Ray - closing ticket now.

Been meaning to comment on this.  Unrelated to the CFB extension, the next version of the eclipse plugin for CFLint (that CFEclipse uses) can be used with CFB too.  It's nice in that it has a UI for configuring rules, and eventually will have "quick fixes" that you can do to add ignores and stuff, the way you can do when editing Java.

@denuno Thanks! I was going to ask about this. I installed it from here, but didn't see how it was supposed to work.
I'm very interested in this "quick fixes" too. I've seen other IDE linters have this functionality as well. I will have to check it out once you've done it to see if it can be integrated into the Atom or VS Code plugins since those platforms appear to support that.

Each IDE has it's own way of doing the "quick fix" stuff (of course-- standardization is for the weak and timid!) but I'll aim to keep the implementation as agnostic as possible.
You can also do teeny Eclipse programs that only have what they need to run, which would sorta give us a GUI of sorts with what's there now, but I'd love to support more than just Eclipse, and thus make "quick fixes" and such as plug-in-able to other stuphs as possible.

That would be a pretty interesting thing to have in CFLint itself, actually, along with a mode where you could enable auto-fixing (or maybe auto-ignoring)... e.g. instead of just warning about an unscoped var, optionally have it var scope it... yeah, something like enabling "suggestions", where it says what rule is broke and then also (if it is easy to figure out) shows a proposed fix.
Something to think about at least.  For sure would want it to be optional, as something like that would slow things down a bit I reckon.

@denuno Exactly! Perhaps you could create a new issue with your thoughts on this feature.

Autofix would be interesting.  Some scenarios would be easier to assume the right answer than others.

Sure, and perhaps in some cases you might even have more than one suggestion. Not all rules would be conducive to having that at all. But this should really have its own issue.

Agree.  #374
On Thu, Jul 27, 2017 at 8:09 PM, KamasamaK<EMAIL_ADDRESS>wrote:

Sure, and perhaps in some cases you might even have more than one
suggestion. Not all rules would be conducive to having that at all. But
this should really have its own issue.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/cflint/CFLint/issues/327#issuecomment-318520443, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AFtZu5JFq0C6n1ByJtZYKS_H5aDhTXfPks5sSSacgaJpZM4OLWhO
.


Have ui_manager.process_events() return consumed_event boolean

Found that when a dropdown menu covers a portion of the game that handles mouse events, the game will handle mouse events intended for the drop down menu and already consumed by ui_manager, causing unexpected behaviors.
Returning the consumed_event boolean allows the game logic to skip consumed events.
Updated doc string and verified that docs build.
Resolves #238.

Doy, and I was looking out for that very mistake!


Add a CLI flag to set GRPC address

Otherwise it's not possible to run multiple drone instances on one server (without docker).

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

as i know new drone does not have grpc

I don't think this is required anymore, upcoming version will drop grpc replaced by an rest api. Besides that, officially done is only supported via docker installation, and within the docker container nobody needs to change the grpc port.

upcoming version will drop grpc replaced by an rest api
What is the timetable here?

officially done is only supported via docker installation, and within the docker container nobody needs to change the grpc port
Hm, trought so. However I then wondered why the HTTP port is changeable.

closing since 1.0 no longer uses grpc


monaco: init at unstable-2012-06-03

Motivation for this change
A monospaced sans-serif typeface by Apple Inc.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

This seems to provide a download of an illegal copy of the font.

Oh, so.. i will close this PR.


Fix linker warning when checking for duplicated symbols

When we build archive files for mlkem-native, we check for duplicated
symbols by linking the target library against a dummy .c file. The
ad-hoc compiler invocation used for this did not use and -z noexecstack
flag which led to linker warnings.
This commit addresses this by passing the same CFLAGS to the dummy compilation
as we use in the rest of the build, which in particular includes -z noexecstack. The
only exception is -static, which is part of the CFLAGS for the library build, but dropped
for the dummy compilation.

Fix #590

I have tried this fix on Intel/Ubuntu, Graviton/Ubunto and M1/macOS and looks good.


Extract cluster-name from client go APIs

Extract cluster-name in kubearmor using client go API
General Information

Any k8s environment

To Reproduce

Deploy KubeArmor in any k8s cluster
Now run karmor log --logFilter all in a new terninal.
Observe the logs

In output of karmor logs, the cluterName for each log is set to default
Expected behavior
The log/alerts to have a valid cluster-name. For ex: in the attached screenshot, the cluster name is prod-cluster-onboarding.
But it is set as default.
Screenshots

This might a bit tricky and it's related to this issue
https://github.com/kubernetes/kubernetes/issues/44954

Hi
I want to work on this issue. Could you please assign it to me, it would be beneficial for my academics too.
Thanks

I was able to reproduce the bug. May i know where i can get the necessary files that i can modify to resolve the issue, i checked the whole repository.
BTW i reproduced it on docker driver.

This discussion thread on k8s group makes it sufficiently clear that there is no way to get a cluster name using k8s client go APIs.
Our option is to allow the ability to pass the cluster-name as an option during installation and as part of config and the telemetry generator uses the name from there.
This should be applicable in the context of systemd mode of installation as well where cluster name could be used to refer to "instance groups" for example.
The point is, we simply pick the cluster name from the config entry and set it.
Tasks involved:

[ ] check if it is possible to set cluster name using configuration. If not, make a provision for it.
[ ] Extend kubearmor-client to set this configuration based on argument in command line option.


fcos3d visualization

Hello, recently I am inference with fcos3d in nuscenes dataset. I am using tools/test.py and tried the images similar to Fig5. in the paper. The visualize result is like the following:

But when I look into it carefully, I found that the yaw of front-left camera were not correct, but in the paper:

The yaw was very good.
I am using fcos-finetune model, and the score thres=0.2. To correct the yaw problem, I used the mono_cam_box2vis function before draw pred bbox in the show_results function in model.detectors.single_stage_mono3d.py.

Do you use the latest master and corresponding checkpoints? We have just refactored the hack codes about yaw and dimension recently. Please refer to the compatibility documentation.

Thanks for the quick reply.
I am using the latest master (using git clone https://github.com/open-mmlab/mmdetection3d.git), mmdet==2.14.0, mmseg==0.14.1, mmcv-full==1.3.8.
The checkpoint is "fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_nus-mono3d_finetune_20210717_095645-8d806dc2.pth".
The input image was "fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_nus-mono3d_finetune_20210717_095645-8d806dc2.pth".
I am using the demo/mono_det_demo.py and the only modification was adding a line in
" show_bboxes = mono_cam_box2vis(show_bboxes)" under mmdet.3d.api.inference.show_proj_det_result_meshlab(). It should be near line 432.

We have removed the hack used in the visualization of monocular 3D detection results. So you don't need to add that line.


Fix a PS command argument

I think it still works on -PermissionsToKeys (that's likely what I ran at the time), but I agree with the comment on the topic that it should be -PermissionsToSecrets. @anurse Can you review/confirm?
Thanks to @Olav_Tollefsen1 for point this out.

@anurse  quickie review please.

@anurse IIRC it worked as "Keys" and not "Secrets" when we did this topic. AFAICT the commenter is correct ... it should be "Secrets."
@Rick-Anderson I'd feel more comfortable with this if we could get an Azure person who knows Key Vault inside-and-out to TAL. Any chance of that?

@SyntaxC4 can you recommend someone to review a one line change to Azure key vault?

@anurse Checkout your bullet 4 sub-bullet: https://github.com/aspnet/Docs/issues/1993#issue-184308289
The strange thing is that I when I ran the setup myself I very likely used -PermissionsToKeys list,get. I didn't have problems. Whatever I did, it did :sparkles: Just Work :sparkles:. However, you put "Secrets" in bold as though you found out that if you don't use -PermissionsToSecrets it will :boom:.
@Rick-Anderson @SyntaxC4 I think it's best that I re-run the topic and confirm that it must be "Secrets" there. I'll check it out next week and see what's what and report back. In the meantime, I'll mark this WIP.

@Rick-Anderson Sorry for the delay here. I'll take care of this as soon as IStartupFilter merges and IHostingStartup goes into review. Should be soon.

@Rick-Anderson Tested and confirmed ... -PermissionsToSecrets :+1: ... u can merge this.


Bug: Active Manage Bed button in Location Management Page is greyed out

Describe the bug
The manage bed button in the location management is currently greyed out, even though the button is active and clickable.
To Reproduce
Steps to reproduce the behavior:

Go to facility card
Click on view facility on any card
click on manage facility and click on the location management
See error

Expected behavior

[ ] The manage bed button should be similar to the rest of the buttons ("edit" and "delete" buttons) in the card

Screenshots

hey @nihal467  i would like to try to work on this issue

Hey @abhip161, would it be alright if I took on this issue? I can submit a PR right away.

@AdityaJ2305 I am working on it

@AdityaJ2305 I am working on it

ok , no problem

hey @nihal467
it is okk

@abhip161 can you share which users type was used, when you took the screenshot, because I am confused why the delete button is grayed out now ?

@nihal467 I was logged in by doctordev  username it shows only admin can delete
If I logged in by admin delete button will not graded

@abhip161 it looks good, can you push the PR


adjust collision shape on descending kick and troubleshoot state transition

descending kick disengages before sprite pixels appear to make contact. Adjust the area2d collision shape to match the foot placement.
Also, a successful descending kick should reverse direction and switch state to double jump, so the player vaults away.

see also: duplicate bug #20


Implemented an override for  get_form_prefix to append form key to fo…

Updated get_form_prefix making it append form key to the prefix so that django can generate different management_forms for different formsets. See issue #12

Current coverage is 63.49% (diff: 100%)

Merging #13 into master will increase coverage by 0.35%

@@             master        #13   diff @@
==========================================
  Files             7          7
  Lines           510        515     +5
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits            322        327     +5
  Misses          188        188
  Partials          0          0

Powered by Codecov. Last update c23ecec...00bee01


[BUG] Metrics(Requests,Dependencies) was sampled when moved to AzureMonitorOpenTelemetry

Library name and version
Azure.Monitor.OpenTelemetry.AspNetCore 1.2.0
Describe the bug
When i move my project to AzureMonitorOpenTelemetry, i found requests and dependencies overall count is incorrect.Perhaps due to data sampling 2% in server side. If I configure it to 100% there is no problem

Expected behavior
insight's metrics should not be sampled
Actual behavior
sampled
Reproduction Steps
services.AddOpenTelemetry().UseAzureMonitor()
enable server side data sampling and set to 2%

Environment
No response

Even if sampling is enabled, the standard metrics automatically produced by the distro should be accurate. Any change you can validate that the standard metrics are as expected?

@luox78 - Could you please check if the counts match if you view standard metrics from metrics explorer. Want to confirm if it's an UI issue.
Go to metrics explorer > select namespace Application Insights standard metrics > Select request count metric
https://learn.microsoft.com/en-us/azure/azure-monitor/app/media/pre-aggregated-metrics-log-metrics/002-metric-namespace.png#lightbox

Hi @luox78, is it possible that you're not gracefully shutting down your application?
Sample App
I made some changes to your sample app:

I changed it to generate exactly 3k Requests.
I called app.StopAsync() to shutdown gracefully and allow the SDK to flush it's buffer.

namespace ConsoleApp45582
{
    using System.Net.Http;
    using System.Threading.RateLimiting;
    using Azure.Monitor.OpenTelemetry.AspNetCore;
    using Microsoft.AspNetCore.Builder;
    using Microsoft.AspNetCore.Http;
    using Microsoft.Extensions.DependencyInjection;

    internal class Program
    {
        private const string TestServerUrl = "http://localhost:8888/";

        static async Task Main(string[] args)
        {
            // SETUP WEBSERVER W/ OTEL & AZURE MONITOR
            var builder = WebApplication.CreateBuilder(args);

            builder.Services.AddOpenTelemetry().UseAzureMonitor(c =>
            {
                //c.SamplingRatio = 0.02f;
                c.ConnectionString = "...";
            });

            var app = builder.Build();

            app.MapGet("/1", c => c.Response.WriteAsync("1"));
            app.MapGet("/2", c => c.Response.WriteAsync("1"));
            app.MapGet("/3", c => c.Response.WriteAsync("1"));
            app.MapGet("/4", c => c.Response.WriteAsync("1"));
            app.MapGet("/0", c => c.Response.WriteAsync("1"));

            _ = app.RunAsync(TestServerUrl);

            // GENERATE REQUESTS IN A LOOP
            Console.WriteLine("Start");

            var httpClient = new HttpClient();

            var rateLimter = new SlidingWindowRateLimiter(new SlidingWindowRateLimiterOptions()
            {
                PermitLimit = 20,
                Window = TimeSpan.FromSeconds(1),
                SegmentsPerWindow = 5
            });

            var count = 0;
            while (count <= 3000)
            {
                await rateLimter.AcquireAsync();
                Console.WriteLine($"{DateTime.UtcNow} - {count}");
                await httpClient.GetAsync($"{TestServerUrl}{count % 5}");
                count++;
            }

            await app.StopAsync();
        }
    }
}

Results

Test 1: I ran the test with no sampling.
Test 2: I ran the test with ingestion based sampling (2%)
Test 3: I ran the test with SDK based sampling (2%) c.SamplingRatio = 0.02f;

You'll see in the bottom left that I got exactly 9k.


Design a Mascot for Our Project

Pull Request Details:

Title: Design a Mascot for Our Project
PR Link: #15
Issue: #14

Next Steps:
I'm ready for feedback and any necessary revisions. Please review at your earliest convenience, and feel free to provide comments or suggest improvements.
Thank you for your time and consideration!

Hi User,
Thank you for your pull request. I appreciate your contribution to our project.
I have reviewed your changes and everything looks good. Please be patient, and I'll get back to you as soon as possible.
If you have any questions or need further assistance, don't hesitate to ask.
Best regards,
Jayesh


QASC: incomplete training set

Describe the bug
The training instances are not loaded properly.
Steps to reproduce the bug
from datasets import load_dataset

dataset = load_dataset("qasc", script_version='1.10.2')

def load_instances(split):
    instances = dataset[split]
    print(f"split: {split} - size: {len(instances)}")
    for x in instances:
        print(json.dumps(x))

load_instances('test')
load_instances('validation')
load_instances('train')

results
For test and validation, we can see the examples in the output (which is good!):
split: test - size: 920
{"answerKey": "", "choices": {"label": ["A", "B", "C", "D", "E", "F", "G", "H"], "text": ["Anthax", "under water", "uterus", "wombs", "two", "moles", "live", "embryo"]}, "combinedfact": "", "fact1": "", "fact2": "", "formatted_question": "What type of birth do therian mammals have? (A) Anthax (B) under water (C) uterus (D) wombs (E) two (F) moles (G) live (H) embryo", "id": "3C44YUNSI1OBFBB8D36GODNOZN9DPA", "question": "What type of birth do therian mammals have?"}
{"answerKey": "", "choices": {"label": ["A", "B", "C", "D", "E", "F", "G", "H"], "text": ["Corvidae", "arthropods", "birds", "backbones", "keratin", "Jurassic", "front paws", "Parakeets."]}, "combinedfact": "", "fact1": "", "fact2": "", "formatted_question": "By what time had mouse-sized viviparous mammals evolved? (A) Corvidae (B) arthropods (C) birds (D) backbones (E) keratin (F) Jurassic (G) front paws (H) Parakeets.", "id": "3B1NLC6UGZVERVLZFT7OUYQLD1SGPZ", "question": "By what time had mouse-sized viviparous mammals evolved?"}
{"answerKey": "", "choices": {"label": ["A", "B", "C", "D", "E", "F", "G", "H"], "text": ["Reduced friction", "causes infection", "vital to a good life", "prevents water loss", "camouflage from consumers", "Protection against predators", "spur the growth of the plant", "a smooth surface"]}, "combinedfact": "", "fact1": "", "fact2": "", "formatted_question": "What does a plant's skin do? (A) Reduced friction (B) causes infection (C) vital to a good life (D) prevents water loss (E) camouflage from consumers (F) Protection against predators (G) spur the growth of the plant (H) a smooth surface", "id": "3QRYMNZ7FYGITFVSJET3PS0F4S0NT9", "question": "What does a plant's skin do?"}
...

However, only a few instances are loaded for the training split, which is not correct.
Environment info

datasets version: '1.10.2'
Platform: MaxOS
Python version:3.7
PyArrow version: 3.0.0

Hi @danyaljj, thanks for reporting.
Unfortunately, I have not been able to reproduce your problem. My train split has 8134 examples:
In [10]: ds["train"]
Out[10]:
Dataset({
    features: ['id', 'question', 'choices', 'answerKey', 'fact1', 'fact2', 'combinedfact', 'formatted_question'],
    num_rows: 8134
})

In [11]: ds["train"].shape
Out[11]: (8134, 8)

and the content of the last 5 examples is:
In [12]: for i in range(8129, 8134):
    ...:     print(json.dumps(ds["train"][i]))
    ...:
{"id": "3KAKFY4PGU1LGXM77JAK2700NGCI3X", "question": "Chitin can be used for protection by whom?", "choices": {"text": ["Fungi", "People", "Man", "Fish", "trees", "Dogs", "animal", "Birds"], "label": ["A", "B",
 "C", "D", "E", "F", "G", "H"]}, "answerKey": "D", "fact1": "scales are used for protection by scaled animals", "fact2": "Fish scales are also composed of chitin.", "combinedfact": "Chitin can be used for prote
ction by fish.", "formatted_question": "Chitin can be used for protection by whom? (A) Fungi (B) People (C) Man (D) Fish (E) trees (F) Dogs (G) animal (H) Birds"}
{"id": "336YQZE83VDAQVZ26HW59X51JZ9M5M", "question": "Which type of animal uses plates for protection?", "choices": {"text": ["squids", "reptiles", "sea urchins", "fish", "amphibians", "Frogs", "mammals", "salm
on"], "label": ["A", "B", "C", "D", "E", "F", "G", "H"]}, "answerKey": "B", "fact1": "scales are used for protection by scaled animals", "fact2": "Reptiles have scales or plates.", "combinedfact": "Reptiles use
 their plates for protection.", "formatted_question": "Which type of animal uses plates for protection? (A) squids (B) reptiles (C) sea urchins (D) fish (E) amphibians (F) Frogs (G) mammals (H) salmon"}
{"id": "3WZ36BJEV3FGS66VGOOUYX0LN8GTBU", "question": "What are used for protection by fish?", "choices": {"text": ["scales", "fins", "streams.", "coral", "gills", "Collagen", "mussels", "whiskers"], "label": ["
A", "B", "C", "D", "E", "F", "G", "H"]}, "answerKey": "A", "fact1": "scales are used for protection by scaled animals", "fact2": "Fish are backboned aquatic animals.", "combinedfact": "scales are used for prote
ction by fish ", "formatted_question": "What are used for protection by fish? (A) scales (B) fins (C) streams. (D) coral (E) gills (F) Collagen (G) mussels (H) whiskers"}
{"id": "3Z2R0DQ0JHDKFAO2706OYIXGNA4E28", "question": "What are pangolins covered in?", "choices": {"text": ["tunicates", "Echinoids", "shells", "exoskeleton", "blastoids", "barrel-shaped", "protection", "white"
], "label": ["A", "B", "C", "D", "E", "F", "G", "H"]}, "answerKey": "G", "fact1": "scales are used for protection by scaled animals", "fact2": "Pangolins have an elongate and tapering body covered above with ov
erlapping scales.", "combinedfact": "Pangolins are covered in overlapping protection.", "formatted_question": "What are pangolins covered in? (A) tunicates (B) Echinoids (C) shells (D) exoskeleton (E) blastoids
 (F) barrel-shaped (G) protection (H) white"}
{"id": "3PMBY0YE272GIWPNWIF8IH5RBHVC9S", "question": "What are covered with protection?", "choices": {"text": ["apples", "trees", "coral", "clams", "roses", "wings", "hats", "fish"], "label": ["A", "B", "C", "D
", "E", "F", "G", "H"]}, "answerKey": "H", "fact1": "scales are used for protection by scaled animals", "fact2": "Fish are covered with scales.", "combinedfact": "Fish are covered with protection", "formatted_q
uestion": "What are covered with protection? (A) apples (B) trees (C) coral (D) clams (E) roses (F) wings (G) hats (H) fish"}

Could you please load again your dataset and print its shape, like this:
ds = load_dataset("qasc", split="train)
print(ds.shape)

and confirm which is your output?

Hmm .... it must have been a mistake on my side. Sorry for the hassle!


Errors when use 1.17.0 to build apiserver-aggregation

I use apiserver-boot 1.17.0 to build an apiserver-aggreation. I will create a REST resource and only use the Get and List method.
So when  run apiserver-boot create group version resource, I only type y for resource(n for both controller and admission controller).
When run the apiserver-aggregation, I can see tons of these errors,
E0515 17:07:05.664439       1 reflector.go:307] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to watch *v1.ConfigMap: unknown (get configmaps)
E0515 17:07:05.664840       1 reflector.go:307] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to watch *v1.ConfigMap: unknown (get configmaps)
E0515 17:07:05.918040       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.ValidatingWebhookConfiguration: the server could not find the requested resource
E0515 17:07:06.059478       1 reflector.go:153] pkg/mod/k8s.io/client-go@v0.17.3/tools/cache/reflector.go:105: Failed to list *v1.MutatingWebhookConfiguration: the server could not find the requested resource

For now the Get/List work as expected, but I need to confirm the root cause of the errors.
Can someone help me about this?

the errors from list-watching [mutating|validating]webhookconfigurations is a duplicate of #500. switching off these plugins should work. while as for list-watching configmap, there's a new internal controller ConfigMapCAController introduced in the latest releases https://github.com/kubernetes/kubernetes/blob/323f34858de18b862d43c40b2cced65ad8e24052/staging/src/k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go#L39-L41 it is basically used to manage in-cluster CA setups, which is irrelavant w/ the aggregated apiserver. i suppose adding a flag --authentication-skip-lookup=true should get rid of that error..

Duplicate with #500

@yue9944882 I need to reopen this issue since I found some other issues.
Let me give you some background information about this issue and #500 .
I use apiserver-boot to add a apiserver-aggregation layer for k8s and I only add a rest resource called MeteringReport(there are detail steps in #500 ).
My expected behavior is that user can use kubectl to get/list the resource MeteringReport.
Before I add --authentication-skip-lookup=true and --disable-admission-plugins= NamespaceLifecycle,MutatingAdmissionWebhook,ValidatingAdmissionWebhook you tell me, the behavior is that:

the apiserver-aggregation can be started with no errors
but after some hours(maybe 10+ hours) there existing lots errors like what I show before. At this time, user can not use kubectl get meteringreport to list/get rest resource.

After I add the two flags, the behavior is that:

the apiserver-aggregation can be started with no errors
but after some hours(maybe 10+ hours), user cannot use kubectl get meteringreport to list/get rest resource. At this time the apiserver-aggregation throws NO ERRORS.

Below is the error logs when user use this command kubectl get meteringreport -v=8
I0521 19:11:19.696062    9527 loader.go:375] Config loaded from file:  /root/auth/kubeconfig
I0521 19:11:19.701133    9527 round_trippers.go:420] GET https://api.ouzos.os.fyre.ibm.com:6443/apis/metering.ibm.com/v1?timeout=32s
I0521 19:11:19.701165    9527 round_trippers.go:427] Request Headers:
I0521 19:11:19.701178    9527 round_trippers.go:431]     Accept: application/json, */*
I0521 19:11:19.701189    9527 round_trippers.go:431]     User-Agent: kubectl/v0.0.0 (linux/amd64) kubernetes/$Format
I0521 19:11:19.701204    9527 round_trippers.go:431]     Authorization: Bearer <masked>
I0521 19:11:19.930325    9527 round_trippers.go:446] Response Status: 503 Service Unavailable in 229 milliseconds
I0521 19:11:19.930353    9527 round_trippers.go:449] Response Headers:
I0521 19:11:19.930362    9527 round_trippers.go:452]     Cache-Control: no-cache, private
I0521 19:11:19.930368    9527 round_trippers.go:452]     Content-Type: text/plain; charset=utf-8
I0521 19:11:19.930373    9527 round_trippers.go:452]     X-Content-Type-Options: nosniff
I0521 19:11:19.930378    9527 round_trippers.go:452]     Content-Length: 20
I0521 19:11:19.930383    9527 round_trippers.go:452]     Date: Fri, 22 May 2020 02:11:19 GMT
I0521 19:11:19.952703    9527 request.go:968] Response Body: service unavailable
I0521 19:11:19.958529    9527 request.go:1171] body was not decodable (unable to check for Status): couldn't get version/kind; json parse error: json: cannot unmarshal string into Go value of type struct { APIVersion string "json:\"apiVersion,omitempty\""; Kind string "json:\"kind,omitempty\"" }
I0521 19:11:19.958593    9527 cached_discovery.go:78] skipped caching discovery info due to the server is currently unable to handle the request

I0521 19:11:20.799094    9527 request.go:968] Response Body: service unavailable
I0521 19:11:21.165636    9527 request.go:1171] body was not decodable (unable to check for Status): couldn't get version/kind; json parse error: json: cannot unmarshal string into Go value of type struct { APIVersion string "json:\"apiVersion,omitempty\""; Kind string "json:\"kind,omitempty\"" }
I0521 19:11:21.165686    9527 cached_discovery.go:78] skipped caching discovery info due to the server is currently unable to handle the request
F0521 19:11:21.172909    9527 helpers.go:114] error: the server doesn't have a resource type "meteringreports"

Do yo know the root cause of this?

@xjtustt can you check if there's error status on your registered APIService resource? 503 normally suggests something's wrong w/ the APIService configuration (instead of apiserver itself)

@yue9944882
Thank you for your feedback. Since I have restarted the container so now the status of APIService looks good. Could you help me to check whether the configuration is correct?
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  creationTimestamp: "2020-05-21T06:05:36Z"
  labels:
  name: v1.metering.ibm.com
spec:
  group: metering.ibm.com
  groupPriorityMinimum: 1000
  insecureSkipTLSVerify: true
  service:
    name: metering-report
    namespace: <ns-name>
    port: 443
  version: v1
  versionPriority: 15
status:
  conditions:
  - lastTransitionTime: "2020-05-22T02:20:04Z"
    message: all checks passed
    reason: Passed
    status: "True"
    type: Available

Besides, I want to describe some more about how I do the authentication.
At the first step, I followed this document to generate the installing yaml file(including APIService, Service, Deployment, etc.). I noticed that the APIService contains an CABundle field.
But I found that this will be expired in one year then I found metrics-server do the authentication like this. So I use the same way.
This will be work in the k8s 1.12 but some days before I need to upgrade to go mod so I use apiserver-boot 1.17 to regenerated my project. After that I found some logs when the apiserver-aggregation starts,
I0522 02:18:35.349742       1 serving.go:306] Generated self-signed cert (/tmp/apiserver.crt, /tmp/apiserver.key)
I0522 02:18:35.349913       1 start.go:359] loading in-cluster loopback client...
W0522 02:18:36.150462       1 configmap_cafile_content.go:102] unable to load initial CA bundle for: "client-ca::kube-system::extension-apiserver-authentication::client-ca-file" due to: configmap "extension-apiserver-authentication" not found
W0522 02:18:36.150564       1 configmap_cafile_content.go:102] unable to load initial CA bundle for: "client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" due to: configmap "extension-apiserver-authentication" not found
I0522 02:18:36.275179       1 configmap_cafile_content.go:205] Starting client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0522 02:18:36.275235       1 shared_informer.go:197] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0522 02:18:36.275275       1 configmap_cafile_content.go:205] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0522 02:18:36.275314       1 shared_informer.go:197] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0522 02:18:36.275703       1 dynamic_serving_content.go:129] Starting serving-cert::/tmp/apiserver.crt::/tmp/apiserver.key
I0522 02:18:36.276051       1 secure_serving.go:178] Serving securely on [::]:7443

Not sure that the logging shows unable to load initial CA bundle is an expect behavior or not.

Could you help me to check whether the configuration is correct?

@xjtustt ur apiservice should work as far as i can tell. and the status shows that the connectivity is good.

Not sure that the logging shows unable to load initial CA bundle is an expect behavior or not.

W0522 02:18:36.150462       1 configmap_cafile_content.go:102] unable to load initial CA bundle for: "client-ca::kube-system::extension-apiserver-authentication::client-ca-file" due to: configmap "extension-apiserver-authentication" not found
W0522 02:18:36.150564       1 configmap_cafile_content.go:102] unable to load initial CA bundle for: "client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" due to: configmap "extension-apiserver-authentication" not found

i think these lines are just mild warnings. it's thrown from a newly-introduced controller inside the apiserver framework which enables the apiserver to perform in-flight certificate rotation. it wont break anything if the controller didn't start properly. you always have the option to upgrade your certs by manual certs replacing ;)

@xjtustt can you check if there's error status on your registered APIService resource? 503 normally suggests something's wrong w/ the APIService configuration (instead of apiserver itself)

@yue9944882 ,
I reproduced this issue again, at this time, APIService status is GOOD,
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  creationTimestamp: "2020-05-20T22:58:40Z"
  labels:
  name: v1.metering.ibm.com
  ownerReferences:
  resourceVersion: "1086706"
  selfLink: /apis/apiregistration.k8s.io/v1/apiservices/v1.metering.ibm.com
spec:
  group: metering.ibm.com
  groupPriorityMinimum: 1000
  insecureSkipTLSVerify: true
  service:
    name: metering-report
    namespace: ibm-common-services
    port: 443
  version: v1
  versionPriority: 15
status:
  conditions:
  - lastTransitionTime: "2020-05-22T02:08:59Z"
    message: all checks passed
    reason: Passed
    status: "True"
    type: Available

When I use kubectl get meteringreport -v=8 I found some errors:
I0522 09:28:02.061487   32016 loader.go:359] Config loaded from file /root/auth/kubeconfig
I0522 09:28:02.075242   32016 round_trippers.go:416] GET https://api.<host-name>:6443/apis/metering.ibm.com/v1?timeout=32s
I0522 09:28:02.075274   32016 round_trippers.go:423] Request Headers:
I0522 09:28:02.075286   32016 round_trippers.go:426]     Accept: application/json, */*
I0522 09:28:02.075295   32016 round_trippers.go:426]     User-Agent: kubectl/v0.0.0 (linux/amd64) kubernetes/$Format
I0522 09:28:02.075309   32016 round_trippers.go:426]     Authorization: Bearer yHLjXyDVkkchCTBVLvhsWN9HxNug-codJnS_49f2O0Y
I0522 09:28:02.412077   32016 round_trippers.go:441] Response Status: 500 Internal Server Error in 336 milliseconds
I0522 09:28:02.412121   32016 round_trippers.go:444] Response Headers:
I0522 09:28:02.412144   32016 round_trippers.go:447]     Content-Type: text/plain; charset=utf-8
I0522 09:28:02.412161   32016 round_trippers.go:447]     Date: Fri, 22 May 2020 16:28:02 GMT
I0522 09:28:02.412174   32016 round_trippers.go:447]     X-Content-Type-Options: nosniff
I0522 09:28:02.412186   32016 round_trippers.go:447]     Content-Length: 190
I0522 09:28:02.412198   32016 round_trippers.go:447]     Cache-Control: no-cache, private
I0522 09:28:02.421716   32016 request.go:942] Response Body: Internal Server Error: "/apis/metering.ibm.com/v1?timeout=32s": Post https://<IP_ADDRESS>:443/apis/authorization.k8s.io/v1beta1/subjectaccessreviews: http2: no cached connection was available
I0522 09:28:02.429359   32016 request.go:1145] body was not decodable (unable to check for Status): yaml: mapping values are not allowed in this context
I0522 09:28:02.429455   32016 cached_discovery.go:78] skipped caching discovery info due to an error on the server ("Internal Server Error: \"/apis/metering.ibm.com/v1?timeout=32s\": Post https://<IP_ADDRESS>:443/apis/authorization.k8s.io/v1beta1/subjectaccessreviews: http2: no cached connection was available") has prevented the request from succeeding

This line of error Post https://<IP_ADDRESS>:443/apis/authorization.k8s.io/v1beta1/subjectaccessreviews: http2: no cached connection was available appears multiple times in different scenarios. Do you know what it means?

https://<IP_ADDRESS>:443/apis/authorization.k8s.io/v1beta1/subjectaccessreviews: http2: no cached connection was available

i found similar issues in the kubernetes upstream repo:
https://github.com/kubernetes/kubernetes/pull/53318#issue-144103905
https://github.com/kubernetes/kubernetes/issues/33557
it seems like a scalability issue in http2 go libraries.. not sure how to deal w/ the issue, let me take a further look


improvement(dingz): read name from system configuration api

Hello
Thanks a lot for your work on this plugin. I'm very keen to help with this project. Just let me know if you like things done differently or in a specific way, so I can adjust the PR.
This PR is related to Issue #18
There is an undocumented API endpoint: /api/v1/system_config
Since it's not documented, It's probably not a safe bet.
Todos:

[ ] get in touch with the manufacturer and ask what's their reasoning for not documenting it.
[ ] make it more robust, in case of problems with the newly added request.
[ ] write tests
[ ] all tests pass

Kind regards

Thanks a lot for this contribution! I'll have a look at it, so far it looks good to me (and I even spotted a thing or two to improve my existing code :smile:). As you correctly note, the API endpoint you use is not documented – however it's not the only one I'm using and since the dingz App and WebUI seem to depend on these as well, it's reasonably safe to use them. 🙈

Awesome, thanks for merging. Thanks again for the plugin.
I'll play it a little more. Maybe I can help some more. Cool stuff! 🚀

Your contribution is most welcome. I’ll let you know how you could contribute, there are definitely many loose ends to tie together. I think this was in fact the first time someone created a pull request for one of my projects so I first need to figure out how to handle that. 🙃🤔
I’m still learning. The Promise.all() you added was something I’d ignored so far, I grew up with JavaScript callback hell a decade or so ago. 😂🤪
Better use of promises is definitely something that could be integrated in at least two additional places where I’ve chained API calls instead of collecting them with a Promise.all(). 🙃🤓
Generally speaking, before adding new features I would like to clean up and stabilize the code base at least a little bit. If you have any suggestions in that area let me know. And obviously the most important are users that use the plugin with different dingz configurations.


Allow NS records for non-origin records

We have a requirement to be able to set NS records for subdomains that will be externally managed - for example making aws.domain.co.uk be managed by route53
Currently this doesn't work,  because NS and SOA records are stripped.  To keep the dont-break-your-domain functionality could we filter out NS and SOA records at the root level, but not for subdomains?  Or possibly only filter out NS records that refer to mythic beasts?

The other option I can think of is to say that if you have any NS records in your zone file, your existing NS records will be stripped. If you have none, the ones in place will be kept. So you either specify all of them or none of them. Effectively if you choose to add NS records, we assume you know what you're doing, and on your own head be it if you break the zone.

I guess a --dont-strip-ns-records flag might work too?

Yes, though I think strip-ns-records would probably be safer as that requires you to explicitly enable the dangerous option which most people won't want. I'll see if I can trial that in the next few days with one of my less important domains.

Ah, yes.
I agree that the 'safe' way is the correct default, though i was thinking not stripping from the file as opposed to not stripping from the domain.  Maybe its a bad name.
--include-dangerous-records

Working on this now.

I've tested this, and it works perfectly for our purposes :)

I think I've found a bug either in the code logic or the API, using the following RFC zone file:
$ORIGIN dnstest.uk

@ 300 A <IP_ADDRESS>
@ 300 NS ns1.mythic-beasts.com.
@ 300 NS ns2.mythic-beasts.com.
@ 300 NS ns3.mythic-beasts.com.

delegated 300 NS ns1.mythic-beasts.com.

Two things happen if I try and upload this using the following command:
pipenv run python3 -- ./sync-zone.py --credentials-file ../mythic-dns-data/credentials.json --zone dnstest.uk --rfc-file ../mythic-dns-data/zones/dnstest.uk --include-dangerous-records
Error 1:
* Warning: deleting dangerous record:
@ 86400 NS ns1.mythic-beasts.com.
* Warning: deleting dangerous record:
@ 86400 NS ns2.mythic-beasts.com.
DELETE @ 86400 NS ns1.mythic-beasts.com.
DELETE @ 86400 NS ns2.mythic-beasts.com.
DELETE @ 300 A <IP_ADDRESS>
DELETE delegated 300 NS ns1.mythic-beasts.com.
ADD @ 300 A <IP_ADDRESS>
ADD @ 300 NS ns1.mythic-beasts.com.
ADD @ 300 NS ns2.mythic-beasts.com.
ADD @ 300 NS ns3.mythic-beasts.com.
ADD delegated 300 NS ns1.mythic-beasts.com.
Traceback (most recent call last):
  File "./sync-zone.py", line 455, in <module>
    print("* Error:", sync_response.code, sync_response.reason)
AttributeError: 'Response' object has no attribute 'code'

If I comment out line 455, error 2:
* Warning: deleting dangerous record:
@ 86400 NS ns1.mythic-beasts.com.
* Warning: deleting dangerous record:
@ 86400 NS ns2.mythic-beasts.com.
DELETE @ 86400 NS ns1.mythic-beasts.com.
DELETE @ 86400 NS ns2.mythic-beasts.com.
DELETE @ 300 A <IP_ADDRESS>
DELETE delegated 300 NS ns1.mythic-beasts.com.
ADD @ 300 A <IP_ADDRESS>
ADD @ 300 NS ns1.mythic-beasts.com.
ADD @ 300 NS ns2.mythic-beasts.com.
ADD @ 300 NS ns3.mythic-beasts.com.
ADD delegated 300 NS ns1.mythic-beasts.com.
NDELETE @ 86400 NS ns1.mythic-beasts.com.: No such record
ERR Entire transaction aborted due to earlier errors
ERR Entire transaction aborted due to earlier errors
ERR Entire transaction aborted due to earlier errors
ERR Entire transaction aborted due to earlier errors
ERR Entire transaction aborted due to earlier errors
ERR Entire transaction aborted due to earlier errors
ERR Entire transaction aborted due to earlier errors
ERR Entire transaction aborted due to earlier errors

It looks like the API doesn't like me deleting ns1.mythic-beasts.com., even though I'm going to add it back and it appears if I access the API directly using httpie:
http --form POST https://dnsapi.mythic-beasts.com/ domain='dnstest.uk' password='mypass' command='LIST'

I've just tested deleting the ns1.mythic-beasts.com. via the API instead of the script and get:
http --form POST https://dnsapi.mythic-beasts.com/ domain='dnstest.uk' password='mypass' command='DELETE @ 86400 NS ns1.mythic-beasts.com.'
HTTP/1.1 400 Bad Request
Access-Control-Allow-Headers: origin, content-type
Access-Control-Allow-Methods: GET, POST
Access-Control-Allow-Origin: *
Connection: close
Content-Type: text/plain
Date: Tue, 10 Mar 2020 15:02:28 GMT
Server: Apache/2.4.25 (Debian)
Transfer-Encoding: chunked

NDELETE @ 86400 NS ns1.mythic-beasts.com.: No such record

First error has been fixed by #31.

Testing and discussion with @nowster shows that if I leave off the --include-dangerous parameter, everything works as expected and the @ NS records aren't deleted (but the subdomain ones are synced as expected). Will close this issue for now and produce a release.


RAIInsights Causal - Heterogeneity Model unclear

It is not clear what the valid values are for the heterogeneity_model argument to the Causal manager. Seeing the definition in the code:
        heterogeneity_model: str = ModelTypes.LINEAR,

I assumed that anything from ModelTypes would work:
class ModelTypes:
    """Model type constants."""
    AUTOML = 'automl'
    LINEAR = 'linear'

However, I've just had an error back from running the manager:
[stderr]  File "create_causal.py", line 117, in <module>
[stderr]    main(args)
[stderr]  File "create_causal.py", line 75, in main
[stderr]    rai_i.causal.add(
[stderr]  File "/azureml-envs/azureml_831cf987908ea38a7b7c810bd17d4c02/lib/python3.8/site-packages/responsibleai/_managers/causal_manager.py", line 184, in add
[stderr]    self._fit_causal_analysis(analysis, X_train, y_train,
[stderr]  File "/azureml-envs/azureml_831cf987908ea38a7b7c810bd17d4c02/lib/python3.8/site-packages/responsibleai/_managers/causal_manager.py", line 257, in _fit_causal_analysis
[stderr]    causal_analysis.fit(X, y)
[stderr]  File "/azureml-envs/azureml_831cf987908ea38a7b7c810bd17d4c02/lib/python3.8/site-packages/econml/solutions/causal_analysis/_causal_analysis.py", line 644, in fit
[stderr]    assert self.heterogeneity_model in ['linear', 'forest'], (
[stderr]AssertionError: The only supported heterogeneity models are 'linear' and 'forest' but received automl

I'm going to fix my component, but at the very least the docstring should list the valid values.

I have updated the linked PR with the relevant doc strings. @riedgar-ms could you take a look?

It's good to see the allowed values listed, but it would also be helpful to have a line as to what 'heterogeneity' and 'nuisance' features are. I know that the docstrings aren't for in-depth discussion, but it would be good to give a one line explanation as to what those are about.

So I talked to the econml maintainer. The nuisance model tries to estimate the treatment features and the outcome variable from the other features in the data.
The heterogeneity model  helps to estimate the treatment effect based on the heterogeneity features.
The econml maintainer will add better doc strings for these in econml package. I can then bring those details in our responsibleai package.


Add helper for downloading all submissions in a subreddit between two timestamps (Implements: #337)

Hi, I've used this trick quite a lot, so I took some time and implemented a helper for praw. I hope you'll find it useful!

Reddit seems to use created attribute for "timestamp" queries. created seems to be a sort of broken timestamp. I think this is very confusing, and I always forget to convert time.time() output to "reddit timestamp". That's why I chose to make highest_timestamp and lowest_timestamp to be normal unix timestamp(even though it is inconsistent with r.search)

I am not a native speaker, so sorry in advance for possible grammatical errors in the docstring.

The basic algorithm is fairly simple, it just slides a dynamic length window over reddit timestamps and tries to keep 30..99 results within the window. Whenever this condition breaks, it increases or decreases the window length by a factor of 2x. I never really bothered to optimize these parameters, they are good enough for me. Maybe it is a good idea to expose them as function parameters.

Naming! TBH, I am not sure if all_submissions is a good name.

Any feedback is appereciated!

I've updated the diff and addressed some of the feedback from @13steinj . I also realized that I haven't actually made use of lowest_timestamp, so I fixed this(and tested the fix)

Bump!

This addition looks awesome. I really like this helper. There are a few order based things I asked questions about otherwise I think the next step is to add some tests. Given that there is a lot of code in this function, I'd really like to see 100% branch coverage. Great work!

Hi, I've address some of your feedback! I am not sure how to achieve 100% branch coverage though with betamax, since the time is constantly going forward and the recorded requests will become outdated the next second.

@eleweek betamax should be caching any and all requests made. Meaning; if you say, search via new and ensure that the first post is the newest post, and you do it on a small subreddit (like one of the api test subreddits as listed in PrawTest's configure), there shouldn't be a change in what is listed. Any further times the test is run, instead of making the request, betamax would be used to read from the cache.

Any further times the test is run, instead of making the request, betamax would be used to read from the cache.

Unfortunately, this isn't going to work: time goes forward => time.time() return value changes => the set of requests changes.

@eleweek You can mock out the calls to time so that it always return the values you want them to return when you run the tests. It's a little tricky if you've never done it before, a very useful tool to have for writing automated tests. There is some information here if you're up to it:
http://stackoverflow.com/questions/4481954/python-trying-to-mock-datetime-date-today-but-not-working

@bboe Thanks for the pointer, I've tried to make it work, but unfortunately it leads to this output when using nose:
Ran 261 tests in -422558.602s
I'm mocking time as:
import time

def mock_time():
    return<PHONE_NUMBER>
time.time = mock_time

Actually, mocking time in setUp() and de-mocking it in tearDown() seems to work, although this leads to incorrect time reported if you press Ctrl+C during the time this test case is run. Hopefully this is acceptable.

Okay, actually, I managed to resolve the issue with incorrect running time values! I've updated the PR

Hey
As far as I remember from way back the created value is the time in the servers timezone. Since reddit may move servers from one timezone to another at any point and may have servers in multiple distinct timezones, the created value should not be trusted to be a consistent difference from utc. As such only created_utc should ever be used, since it is the only reliable timestamp. IIRC the created attribute is a legacy value that is only kept around, because removing it would be a backwards incompatible change that may break an unknown number of apps that expect this value to exist. Also, due to the openness of reddits API, there is no known way of being able to contact all consumers of the API to inform of a backwards incompatible change. In other words, removing created was considered such a complicated, dangerous change that it was postponed in favor of getting more pressing changes out. With that in mind created should be considered deprecated, dangerous and not to be used.
I'm not 100% on this information, it's been quite a while since I've been in the nitty gritty of PRAW/reddit. But I wanted to pipe in, since it looks like this PR assumes a fixed relationship between created and unix time, which could change at any time and not even be guaranteed to be a constant. This could prove to be a very serious maintenance headache years in the future if reddit ever arbitrarily decides to change the servers timezone.

I did another quick read through and it looks like the server expects the timestamp arguments in the timezone of created which is what causes the conversions between unix time and reddit time. In which case, and if my memory of the above is correct, then what is being done seems to be the only way to do this from PRAW's POV. But it seems quite fragile for something that is part of PRAW and not an external script.
The proper solution would then be to update the respective endpoint at reddit to accept utc instead of timezone specific time or as an alternative. This would embrace the deprecated nature of created and allow server timezone to be changed arbitrarily without breaking stuff. PRAW could then use created_utc safely in the knowledge that it will be consistent forever.

@Damgaard I do agree with you that it would be better to use created_utc rather than deprecated and confusing created. The issue was raised before. Alas, reddit admins aren't particularly responsive.

The proper solution would then be to update the respective endpoint at reddit to accept utc instead of timezone specific time or as an alternative.

Personally, I find reddit admins extremely unresponsive. I've reported multiple bugs(got zero response) and tried to start a discussion on slightly changing a different endpoint(again, zero response), so even though I don't mind contributing to reddit(I actually fixed a bug in reddit once), I have little desire trying to make a breaking changes or significant modifications.

Bump! Is there any chance that this could be merged without changing reddit's source?

Is there any chance that this could be merged without changing reddit's source?

Yes, while I agree with @Damgaard that UTC timestamps should be used, I think something that works initially is better than nothing.
Regarding timing, it may be a few days until I can take a solid look before merging. Maybe @Damgaard or @voussoir can before then? If there's no activity before Monday, please bump again, and thanks for the contribution.

It's a shame the API isn't using the UTC timestamp, but it used to be much worse! I don't know when the created-created_utc offset became 28,800 globally but it must have been recent.
I haven't run any of your code, but it looks great to me, and the tests seem to agree. I might try to get a little more playful with the window_size modifiers instead of using a hard x2 multiplier though. For quick results, we want to get as close to our search_limit as we can on every loop, and avoid dawdling in the 30-90 range if possible.
There's probably no perfect solution to this, but maybe you can increase and decrease by a certain percentage, and only get aggressive after repeated failures.

As for the function name, maybe just timestamp_search? I'm having a hard time coming up with something descriptive and concise, haha.

For bonus points, do you think this function can accept some other search parameters? Cloudsearch is kind of annoying to format, but it's like a prefix expression I guess:
if username is not None:
    query = '(and author:"%s" timestamp:%d..%d)' % (username, lower, upper)
else:
    query = 'timestamp:%d..%d' % (lower, upper)

This opens up a lot of possibilities, so if the formatting doesn't get too crazy it may be worth the effort.
Sorry I haven't been too active recently, this is a great contribution and I wish I had participated more.

@eleweek Overall, I think we're just about ready. Perhaps we can make a simplification to the test with respect to directly comparing lists of subreddit objects, and rename the method. Then it should be good to go!
Thanks for the persistence and great work.

@voussoir @bboe Thanks for your feedback, I was busy last week, so I didn't get to updating the PR. I'll probably do this over the next couple of days.

I'll probably do this over the next couple of days.

Awesome! Please mention us when you've done that so we know when to take a look.

@bboe @voussoir I've updated the PR! Changed window sliding params + added extra filtering by other cloud search fields

Looks awesome. Just two bits of feedback, then I think we're good to go.

@eleweek I'm really glad you added the additional cloudsearch parameter support, thanks a lot! All of the changes get a thumbs up from me.
I'm still seeing extra_cloudsearch_fields={} in the latest commit, 601fb8a, though. Can you make sure that gets changed to None before we merge? Following the timestamps it looks like the commit was made after Bryce's comment so I think you forgot to change it.

@voussoir It was late where I live, and I figured I could push a commit that fixes CI problems and fix those issues later.
I fixed it now though.
Also pinging @bboe just in case

Great, thanks! Just had to double check.
@bboe, I'm happy with the PR when you are.

Awesome work @eleweek! Thanks for reviewing @voussoir. :clap:

Thank you @bboe , @voussoir and @13steinj . This was fun :)


Update Devnet Logo aligned with Starknet new logo

I have made a new logo of Starknet Devnet. The Starknet logo in the center is changed into the new one.
Thank you.

This PR doesn't just change the logo, it also adds an unwanted grey background, as can be seen here: https://media.discordapp.net/attachments/1178603291005947954/1224611637512638524/image.png?ex=661e1f76&is=660baa76&hm=b1a9ae7c636780cc07a19a9823543d9eb6c7e576da733701e968af0ac1a7c739&=&format=webp&quality=lossless&width=1282&height=623

Hey,
I think I have updated it here without grey background. I am going to modify the color later.
Thank you,

@FabijanC it shows no backgroun in my forked repo. Is it right?

I think I should upload the logo here.

There is another one modified the keyboard frame color into light orange to align with the new logo color.


Add support for camel-rabbitmq

As suggested in https://camel.apache.org/camel-quarkus/latest/list-of-camel-quarkus-extensions.html

In case you are missing some Camel feature in the list:

Upvote an existing issue or create a new one so that we can better prioritize our work

For the time being, I'll try my luck

You may try your luck using the given camel component on Quarkus directly (without an extension). Most probably it will work in the JVM mode and fail in the native mode. Do not hesitate to report any issues you encounter.

I would like to be able to compile my project down to native eventually. If I can help in any way (other than writing the extension and reporting errors), please let me know.

I can look at this

Sorry, I've been too busy this entire month. I might have some time next next week but I'm not sure with my current issues.

Any roadmap when it will be available for native and considered stable (instead of JVM + Preview)?

Created a separate issue https://github.com/apache/camel-quarkus/issues/1531


refactor: [M3-6320] - MUI v5 Migration - Components > DialogTitle

Description 📝
Refactors src/components/DialogTitle/DialogTitle.tsx to remove JSS

Note: The close button aspect of this component is not actually used anymore. I figured that I would keep it in for the time being.

Major Changes 🔄

Updated references for named export

Preview 📷

Before
After

If we were to activate the close button:

How to test 🧪

yarn dev
Go to http://localhost:3000/linodes
Attempt to delete any linode and observe no differences
N/A

The heading now matches what we use for confirmation/deletion dialogs using DialogTitle which is the standard.

Light
Dark
Prod


How to deactivate filter when using search API ?

Hello,
I'm using the new feature Custom Search / Search API from the version 2.9.0, thanks for this addition.
Sadly I have a problem with the data displayed.
Here is my workflow:
When I receive an event with keyup, I send a request who fetch Members (for example with the name Smith) from my database. Once I receive this list, I create a new formatted array which creates groups with Members and Companies.
For example:
[
    {
        "id": 10,
        "itemName": "Company ABC",
        "category": "Companies"
    },
    {
        "id": 11,
        "itemName": "Company XYZ",
        "category": "Companies"
    },
    {
        "id": 41,
        "itemName": "Member John Smith - Company ABC",
        "category": "Members"
    },
    {
        "id": 42,
        "itemName": "Member Agata Smith - Company XYZ",
        "category": "Members"
    }
]

Then I put this array as parameter data to the component. However, as my search was Smith, the results showing are only the Members, and not my full formatted array.
So, is it possible to disable the search filter if keyup method is used ?
Thanks !

The search filter is automatically disabled, when you use the search template. Can you check again . If it still doesn't work, please reproduce the same with an example at https://stackblitz.com/ and post. Definitely will look into it.

Also, please refer to example at https://cuppalabs.github.io/angular2-multiselect-dropdown/#/customSearchAPI


⚠ Bofrost.at has degraded performance

In 21faa67, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1410 ms

Resolved: Bofrost.at performance has improved in da4a8ca after 6 minutes.


68 add injection initializer

Changes introduced in these commits do not alter the functionality of HMS, instead making the interface more flexible and moving some of the logic away from demes.

I moved the initializer to config as suggested. Also, I decoupled the creation of initializer and providing the sampler with runtime context, which simplified the code. Now you can add a new, custom initializer and as long as it does not require any additional context-dependent information, you do not need to rewrite anything. If you needed it, there is a new function for that.
Conflicts with main are resolved. There was also a test_tree_format test file, which was not being detected by unittest discover, but was part of GitHub actions test suite, so I added a test class to it.


feat: native apple silicon

This is exploration into better support of Apple Silicon.
electron-builder supports producing either universal package or one for each architecture.
In case of Desktop, things are more complicated: we also bundle go-ipfs binary.
This means that unless the go-ipfs binary  is also universal or per-arch, we can't improve much.
Due to the way we depends on  go-ipfs via  npm-go-ipfs universal binaries all the way sound like way easier route (downside being the disk size), however dist.ipfs.io already has  amd64 and arm64  for darwin.
Universal binary test
Main cost of universal package is size, even with out go-ipfs (still being amd64) it adds ~75 MB:

195M   IPFS Desktop-0.16.0-universal.dmg (electron amd64&arm64 + go-ipfs arm64)
119 MB IPFS-Desktop-0.16.0.dmg (amd64, M1 will use Roseta)

TODO

[ ] decide if we want to provide single universal binary or one for amd64 (legacy intel)  and one for arm64 (new apple silicon)

Data point from Brave:

atm IPFS integration  is using amd64: go-ipfs binary is amd64, it runs on M1 via Rosetta. Brave plans on using  arm64 in the near future, as they already provide arm64 version of Brave itself.. In that contenxt universal binary is wasteful (IPFS component is fetched on-demand, so small size matters).

[ ] based on previous decision, decide how we handle go-ipfs dependency (right now it downloads go-ipfs only for arch of CI worker, which is amd64).

@lidel, do we know where the difference in size comes from, specifically? We could find where they are and try something to minimize the impact of shipping the universal package.

@lidel, do we know where the difference in size comes from, specifically? We could find where they are and try something to minimize the impact of shipping the universal package.

Not a dev, but I'm pretty sure the size increase is just the binaries about doubling in size due to basically containing two apps.

Hello, any update?
I've committed fix for go-ipfs https://github.com/ipfs/npm-go-ipfs/pull/43

It work fine with me, what error that you got?

I think that Universal would be easier for the end users despite the storage size.

Have we looked into https://github.com/tauri-apps/tauri at all?

Regarding universal app vs rosetta specific.. the speed increases you get by running M1 native is significant. I don't think we should decide to just force rosetta forevermore. If the decision is to wait for some period until we have more bandwidth, I think sticking with rosetta and the smaller bundle for now is fine. Ultimately, we should be moving to the type of app bundle the platform we're targeting recommends.
For Apple/macOS, that is the universal build.

@SgtPooki an issue you may want to look at too is #867

Have we looked into https://github.com/tauri-apps/tauri

I think we looked at it briefly, but it was very alpha back then.  IIRC the main downside was lack of tooling for end-to-end autoupdate solution, which is raison d'être of ipfs-desktop.  It seems they work on it, but it is still undocumented? https://github.com/tauri-apps/tauri/discussions/2776
Long term, we want to move away from fetching updates from centralized HTTP server like github and leverage IPFS+IPNS  instead (see old notes in https://github.com/ipfs/ipfs-desktop/issues/789), so if we have working setup for that, we could move away from Electron (literally the only value it brings is https://www.electron.build/auto-update working on all three platforms: macOS+Windows+Linux(AppImage))

75 MB won't probably affect many people.

Usually I would be against doubling the size of download, but we talk Apple here – it is fair to assume that  if someone uses mac they most likely don't care about additional download cost.
The caveat here is that we don't have "universal go-ipfs binary" – iiuc this PR (in current form) still bundles the intel version of go-ipfs.
Figuring out how to include both binaries  (arm64 and amd64) for darwin  is the key to unblocking this.

lo

I wrote a plan for unblocking this in https://github.com/ipfs/distributions/issues/770#issuecomment-1329178981.
Not a macOS user myself, but if someone wants to work on this, I'm around to advise.

I read that comment and the approach seems reasonable and clear. Thanks for the backlink!

There are two separate topics at play (mind I am not macOS user, below is my rough understanding on Apple Silicon situation)

architecture of go binary (kubo daemon)

What we did on dist.ipfs.tech is we provide separate Kubo binaries for amd64 (intel) and arm64 (arm/apple silicon) (https://github.com/ipfs/distributions/pull/838, https://github.com/ipfs/distributions/pull/841).
Brave downloads correct binary at runtime.
There is no universal binary for Kubo, but you could build one as noted in https://github.com/ipfs/distributions/issues/770#issuecomment-1329178981 – afaik nobody is doing that atm.

IPFS Desktop DMG bundles NPM version pulled in during the build – afaik this is still amd64, so it runs emulation.
IPFS Desktop needs to either stop embeedding and fetch correct binary at runtime, or include both and run correct one on macOS.

architecture of electron app (ipfs desktop)

this feels unresolved, but maybe not a big deal, since the majority of work happens in kubo?
iiuc what happens on macOS is that we produce DMG that is amd64 and when run on Apple Silicon it runs emulation

I agree we should close this, and if we want to use native arch on macOS, need to fill new issue and make a new plan for both topics above.

please see https://github.com/ipfs/ipfs-desktop/issues/2681 for a continuation of this work

Reopening because of https://github.com/ipfs/ipfs-desktop/issues/2813
I will prototype Universal DMG that includes Universal Kubo binary as well.

Fetching both x86_64 arm64 and merging them into universal Kubo binary is working (build log):
→ lipo info after merging binaries Architectures in the fat file: /Users/runner/work/ipfs-desktop/ipfs-desktop/node_modules/kubo/kubo/ipfs are: x86_64 arm64
Remaining work is to upstream kubo+0.28.0.patch to npm-kubo repo and test DMG from this PR on both Intel and ARM macOS.

Also tested Tested https://dweb.link/ipfs/bafybeidw62sobej3qlfsmxxw5bbnczz6ithimvixf3xg7rwquur3iww66q/ipfs-desktop-0.35.1-mac.dmg on ARM and it worked. Can test on a MacOS Intel tonight.

@SgtPooki yes, this will be released as 0.36.0 (2e891f3).
The crash is concerning.

Are you on Intel or Apple Silicon machine?
Is it crashing every time?
Was it crashing with RC 2 (.zip of build here)?

Are you on Intel or Apple Silicon machine?

Apple silicon

What is the version of macOS?

Sonoma 14.5 (23F79)

Is it crashing every time?

Yep.

Was it crashing with RC 2 (.zip of build here)?

I can get my hands on M1 macMini tomorrow, but having second data point would be useful.

that version is also crashing for me immediately. I remember a previous version you asked me to test a month ago or so worked fine. If that's the same one, it's not now and the problem is my system.

Thanks, sounds like it.
FWIW I've just tested both on M1 macMini and both work ok. Let's merge and ship.
If crashes of IPFS Desktop binary (electron) rather than kubo itself become common, then the first thing we should do is to upgrade to supported version of Electron (#2775)


back.pysim performance improvements

I'm not sure how high a priority the Python simulation backend is or whether it's worth making the structural changes to hdl.ast to optimize hashing, but I figured I would share what I've figured out so far in case it's useful. Please let me know what you think.
The changes here fall roughly into the following categories:

Fixing what I believe to be the intent behind curr/next comparisons, and avoiding doing work where (I think) it's known there is no work to be done
Optimizing the hashing of Value/Signal/Statement key objects, since this is done in a few very frequently traversed code paths
Caching compilation results (from hdl.ast to what I believe is Python bytecode)
Micro-optimizations. I think the only one that made it through was the VCD hash table lookup, and it's pretty marginal at this point

I used the following simulation to make performance measurements: sim_perf_test.py. I have perf data for each commit in the chain, both with and without VCD output enabled. I can see about uploading them somewhere if there's interest, but in the meantime here's a summary:
With no VCD output:

Commit
Description
Time elapsed (s)
Reduction (incr.)
Reduction (cum.)

bb1bbcc5
hdl.mem: fix source location of ReadPort.en.
116.8
--
--

52e30bc8
back.pysim: Fix curr=next optimizations
90.77
22.29%
22.29%

6902c99a
hdl.ast: Intern value/statement keys off the hot path
80.74
11.05%
30.87%

642e4774
back.pysim: Cache compilation results
32.17
60.16%
72.46%

d1fe3c63
back.pysim: Reuse clock simulation commands
25.29
21.39%
78.35%

65405683
back.pysim: Eliminate duplicate dict lookup in VCD update
24.88
1.62%
78.70%

With VCD output enabled:

Commit
Description
Time elapsed (s)
Reduction (incr.)
Reduction (cum.)

bb1bbcc5
hdl.mem: fix source location of ReadPort.en.
152.6
--
--

52e30bc8
back.pysim: Fix curr=next optimizations
127.4
16.51%
16.51%

6902c99a
hdl.ast: Intern value/statement keys off the hot path
118
7.38%
22.67%

642e4774
back.pysim: Cache compilation results
66.94
43.27%
56.13%

d1fe3c63
back.pysim: Reuse clock simulation commands
57.16
14.61%
62.54%

65405683
back.pysim: Eliminate duplicate dict lookup in VCD update
53.25
6.84%
65.10%

I did run the nMigen unit tests, but I haven't figured out yet how to set up sby on Windows so I don't know whether those tests are broken by my changes.

Fixing what I believe to be the intent behind curr/next comparisons

Curr/next comparisons implement the two-phase simulation algorithm that is deterministic in presence of asynchronous code. I looked at your fix and it will lead to incorrect simulation results.

Optimizing the hashing of Value/Signal/Statement key objects, since this is done in a few very frequently traversed code paths

I looked at your changes and while the intent seems reasonable, the patch seems fairly invasive at first glance. I'll take another look later.

Caching compilation results (from hdl.ast to what I believe is Python bytecode)

Extremely good change, one I was going to make for some time but never got around to.

Micro-optimizations

Also looks all reasonable.

To summarize, 642e477, d1fe3c6 and 6540568 are mergeable as-is, 6902c99 needs more discussion, and 52e30bc is unsound.

Thanks!

Curr/next comparisons implement the two-phase simulation algorithm that is deterministic in presence of asynchronous code. I looked at your fix and it will lead to incorrect simulation results. I can explain the algorithm if necessary.

I'm not sure I understand, but it's easy enough to revert. Would you prefer I modify the history to keep it linear or layer revert commits etc. on top of what's already here?

I looked at your changes and while the intent seems reasonable, the patch seems fairly invasive at first glance. I'll take another look later.

Yeah, I wasn't entirely sure how to make this less invasive. A big part of the performance gain was from eliminating glue logic from what was previously ValueKey/SignalKey.

To summarize, 642e477, d1fe3c6 and 6540568 are mergeable as-is, 6902c99 needs more discussion, and 52e30bc is unsound.

As it is, 642e477 depends on 6902c99 -- that was the motivation for the changes to ValueKey. I actually started with repr(value) as the key, but there was still a fair bit of overhead there.

A patch dedicated to optimization should not introduce a new public API.

Should that specific method be named _castable? Or should it go somewhere else?

I'm not sure I understand, but it's easy enough to revert.

The basic idea is that:

eval only looks at curr and only modifies next;
commit is the only one who can assign from next to curr;
the eval/commit loop continues while there are any changes.

The reason it's arranged like that is because rules (1) and (2) ensure that no matter in which order eval and commit are run (i.e. no matter how any particular netlist is translated, how it's arranged into modules, etc), each full eval/commit cycle will have the exact same result. Rule (3), by iterating to fixed point, ensures that all updates in a netlist fully propagate before advancing time--for example, consider a case where you have two domains with asynchronous reset chained, and you reset the first one. This will require two delta cycles (eval/commit loops).

Would you prefer I modify the history to keep it linear or layer revert commits etc. on top of what's already here?

I strongly prefer keeping the history linear.

Yeah, I wasn't entirely sure how to make this less invasive. A big part of the performance gain was from eliminating glue logic from what was previously ValueKey/SignalKey.

Should that specific method be named _castable? Or should it go somewhere else?

I'm not sure yet. I'll need to understand your proposed changes in detail to see if there is a less invasive way to do the same thing, or if there's really no other choice.

I reverted the unsound curr/next change and rearranged the interning API to conserve the existing ValueKey and SignalKey classes. I think all the new abstractions should now be hidden.

Thanks. I've cherry-picked the second two commits. I expect it will take a while to convince myself that the proposed interning changes are the way forward. I appreciate that you rearranged them to have no impact to the public API, but the bulk of my concern has to do with the increased internal complexity as well as soundness in face of mutability of AST nodes: I am hesitant to introduce any sort of persistent hashing while properties of most AST nodes remain writable.
Have you considered adding fast paths to the simulator, for the case where bench code operates on simple signals alone? This removes the need for caching bytecode, is an easy to understand change with minimal impact, and should yield most of the performance benefit.

I expect it will take a while to convince myself that the proposed interning changes are the way forward

Understood. I'm not necessarily expecting to get this specific design upstreamed, it's just where I ended up after probing for performance improvements.

the bulk of my concern has to do with the increased internal complexity as well as soundness in face of mutability of AST nodes: I am hesitant to introduce any sort of persistent hashing while properties of most AST nodes remain writable.

That makes sense. Do you anticipate the AST becoming immutable?

Have you considered adding fast paths to the simulator, for the case where bench code operates on simple signals alone? This removes the need for caching bytecode, is an easy to understand change with minimal impact, and should yield most of the performance benefit.

I'm not sure I follow. Are you thinking along the lines of falling back to a simpler scheduler if only the default sync and comb domains are used? Or did you have something else in mind?

The basic idea is that [...]

Thank you for explaining this, BTW. The eval/commit/loop approach makes perfect sense to me. I think I see at least one way the unsoundness of 52e30bc could manifest: if one invocation sets next != curr, and a second invocation sets it back to curr, it will be left with the wrong value for next. That's important because a single fragment could set next repeatedly.
I retraced my steps and realized that the call to pending.clear was the only part of that commit that actually mattered for performance. The rest of the changes were a result of trying to micro-optimize _SignalState.commit. If I'm not (still) very much mistaken, calling pending.clear should be correct since at that point in _SimulatorState.commit, every pending update from the current eval/commit loop has been effected, and the next eval step will set pending if necessary. What do you think?

Do you anticipate the AST becoming immutable?

This is a bit of a complex issue. At the very beginning, I designed nMigen so that it never mutates ASTs (and it still doesn't). When it needs to change an AST, it emits a new one with the necessary elements changed (the FragmentTransformer stuff). I knew this could be a performance issue when writing it, but I didn't realize how just how slow it would be.
This slowness could be fixed in two ways: a simpler but more flawed one--which is to copy the entire AST once, eliminating any sharing, and then mutating it; and a more complex but principled one--which is to not change ASTs in first place (with or without mutation), and instead treat ClockSignal, ResetSignal, domain names, etc similar to variable bindings. That is, instead of having DomainRenamer alter the actual AST, it would alter a binding table carried along with it, with this table specifying which domain becomes which after every transformation is applied.
The principled approach potentially has very significant benefits. It would dramatically improve performance on all designs but especially large generic ones. It would eliminate multiple complicated corner cases from the internal machinery. It would bring the internals of nMigen closer to well understood and well behaved compiler development principles, namely using variable bindings for late bound names (there's nothing wrong in principle with term rewriting it's currently doing, but this term rewriting is very ad-hoc and hard to analyze). It's also a lot of work.
There is a separate issue, which is the user mutability of ASTs, whether interior or exterior. For most expression nodes (everything except Signal, ArrayProxy, and UserValue), this is clearly undesirable. For ArrayProxy and UserValue, it's just interior mutability, and there's already code to cope with it, which should be largely sufficient. For Signal, the story is a bit complex--I believe it's never been "officially" supported, but you can mutate Signals and the results are currently correct, provided you never use them in expressions before you stop mutating them. This is used internally in nMigen in the FSM generator code, so it seems like a valuable capability and we might want to enforce that invariant in a future immutable AST.
To summarize, I think the AST should be immutable, but we're not quite there yet, and there is at least one corner case that currently presents an API with a soundness issue that would likely need to be handled explicitly.

That's important because a single fragment could set next repeatedly.

Yup, and there are many cases where this happens--for example each time you assert reset.

If I'm not (still) very much mistaken, calling pending.clear should be correct since at that point in _SimulatorState.commit, every pending update from the current eval/commit loop has been effected, and the next eval step will set pending if necessary. What do you think?

Could you show a patch that does what you want? I'm not quite sure exactly what change you're referring to, and it seems like preparing a patch isn't a lot of effort here.

back.pysim: Clear pending updates after they are effected

Oh... did I just completely forget to remove elements from self.pending anywhere at all? Well that would make things a lot slower than they should be!!

I'm curious, could you measure the perf impact? I'm really glad you noticed this, it's a really silly mistake with severe consequences.

I also cherry-picked it onto master:
* 555c23c8 back.pysim: Clear pending updates after they are effected
| * d891a794 back.pysim: Clear pending updates after they are effected
| * daa616f0 back.pysim: Eliminate duplicate dict lookup in VCD update
| * 1b60dbda ...
| * d698e6f4 ...
| * 3f466e89 ...
|/
* bb1bbcc5 hdl.mem: fix source location of ReadPort.en.

Here's what I got with VCD output disabled:

Commit
Description
Time elapsed (s)
Reduction (incr.)
Reduction (cum.)

bb1bbcc5
hdl.mem: fix source location of ReadPort.en.
152.6
--
--

...
...
...
...
...

daa616f0
back.pysim: Eliminate duplicate dict lookup in VCD update
47.24
...
69.04%

d891a794
back.pysim: Clear pending updates after they are effected
25.05
46.97%
83.58%

Commit
Description
Time elapsed (s)
Reduction (incr.)
Reduction (cum.)

bb1bbcc5
hdl.mem: fix source location of ReadPort.en.
152.6
--
--

555c23c8
back.pysim: Clear pending updates after they are effected
87.41
42.72%
42.72%

I've cherry-picked commit d891a79, thank you!

I'm not sure I follow. Are you thinking along the lines of falling back to a simpler scheduler if only the default sync and comb domains are used? Or did you have something else in mind?

Something else. You can notice that most simulator commands are along the lines of yield sig or yield sig.eq(123). Which means that by special-casing just these two patterns we can dramatically speed up simulation without any of the interning complexity.

I integrated the cherry-picked changes to clean up the PR in case you decide to do something with it. One thing that occurred to me that I haven't bothered to change is that the compilation cache should probably be scoped to the simulator rather than the coroutine process.

Something else. You can notice that most simulator commands are along the lines of yield sig or yield sig.eq(123). Which means that by special-casing just these two patterns we can dramatically speed up simulation without any of the interning complexity.

That makes a lot of sense. Speaking of the simulator interface, something I've thought of requesting is a way to wait asynchronously for a signal change from a coroutine process. It sounds from #228 like you already have some idea of how you'd like to do that? FWIW I found Value hashing useful for hacking together (emphasis on "hack") an implementation using _FragmentCompiler.

It sounds from #228 like you already have some idea of how you'd like to do that?

Internally, this is easy to do. The external API needs to be designed.

As per discussion above, I am closing this PR; I would encourage you (or anyone else) to submit another PR with a less invasive approach I suggested above. If it turns out that (contrary to my expectations) the simpler approach does not bring similar performance benefits, then I'm going to reevaluate the approach in this PR. Thanks for doing this work!


Add a Tilt support for rapid development

Looking at https://github.com/kinvolk/inspektor-gadget/blob/main/docs/CONTRIBUTING.md, it seems we are currently relying on pushing images to a registry during development. This can greatly slow down development.
Tilt allows rapid development by automatically updating local code changes on a remote k8s cluster, thus saving the need to rebuild a container image, push it and re-deploy it to the cluster. It is free and open source.
P. S. Not sure if "local gadget" addresses this concern. Still, even if it does, there are probably some cases where there is no substitute for running against a real k8s cluster, in which case Tilt support can be useful.

Good idea. Do you know how Tilt works with compiled programs? (Golang & C for the eBPF code)
Yes, local-gadget partially addresses this concern (I use it for faster local development), but as you say not everything can be tested with local-gadget.

Do you know how Tilt works with compiled programs? (Golang & C for the eBPF code)

Tilt is a Python-based wrapper for whatever, so you can invoke go or clang or whatever to generate binaries.


Define all table to case sentitive where possible

Reason: make even faster.
Create a filter to make impossible write upper case strings.

Setting is using this #61

Analysing here, all changes by now, was already done.


testCapsule ignoreEntity array + getEntityIsFreeAimingAt

https://wiki.rage.mp/index.php?title=Raycasting::testCapsule
ignoreEntity: Array of entities handle or object
https://wiki.rage.mp/index.php?title=Player::getEntityIsFreeAimingAt
Returns RAGE:MP Object or World-Object Handle if it found an entity in your crosshair within range of your weapon.. Returns undefined if no entity found.

Please open the pull request in the main branch. Branch types-[side] are for CD (Continuous delivery) only.


reset Context with subscription

we kinda support subscriptions by allowing subscription type to be set in the schema via https://github.com/lolopinto/ent/pull/1553
related task https://github.com/lolopinto/ent/issues/1544
ideally, we should automatically clear the context cache before each subscribe/resolve method since we could end up with long-lived subscriptions which can keep increasing the data in the context cache which is against the spirit of what it's designed for: was designed to live per-http-request

after https://github.com/lolopinto/ent/pull/1658, we don't want to clear context cache but reset the context so that each time, it's a new one