Manim 2.0?

There seems to be quite a lot of maintainability issues with the current Manim and areas which don't make full use of features of the python language but fixing them would basically require a rewrite of the codebase. Here are a few of the potential areas for improvement I have found:

There's lot of getter and setter functions in which should be properties

Using a python based latex compiler such as PyLaTeX would help decrease non-python dependencies. It would be nice if we could eliminate all non-python deps to simplify the install process although I'm not sure if that's practical.

Using the new pipfile standard to manage python dependencies.

imports.py should be __init__.py and __init__.py should be __main__.py

#609 split it modules. A lot of Manim is only required for specific use cases which is inevitable for it's use but that code should be split into modules based on functionality which the user can chose whether to be imported or not. E.g. if #764 is merged, then it should have it's own module called physics. This would also allow for large files to be split up (not to pick on #764 but it's good as it shows how much effort currently needs to be done to implement new objects) e.g psychics.py could be split into particle.py and pendulum.py inside the physics module.

Configuration management could be done much the same way as flask, through a single object called Config which parses say a yaml file then all the other modules can import the config and use it like a dictionary e.g config['CAMERA_QUALITY'] to get values instead of having functions such as get_camera_configuration().

The end goal of all of this is to abstract and automate as much of the internal stuff away from the module writers as well as users to make the code clean and readable.
Although these changes would break a lot of existing code using the library thus something that needs to be decided by the community whether it's worth it.
If you have any other suggestions for changes that would be suitable for a 2.0. Please comment on this issue. It would be nice to start compiling a list.

You raise some good points. I also noticed that manim has a lot of maintainability as well as convenience problems and started working on a full rewrite of manim in an entirely different language, javascript, at v2.eulertour.com:
 github link
It's still in the works, but when it's finished this version will allow for real-time previews, a user interface for visually tweaking animations and mobjects, interactivity, portability, and of course all of the features that are in this version of manim.
There's a lot of work to be done before this is ready, so if you're interested in helping please get in touch with me on the manim discord.

I absolutely agree with the issue at hand. When adding the new function for customizing the LaTeX template, i struggled a lot with the structure. Also, pull requests are pending for a long time and do not seem to be merged regularly. This is frustrating, especially for small improvements.
@eulertour Your work is impressive. I will have a look at it and see whether I am able to contribute, albeit rather small things.

@eulertour Whilst eulerv2 is very cool but I don't know javascript.
I also don't think that just forking and rewriting, whilst a valid option, is particularly a good idea as I feel like were taking the project away from Grant. I don't particularly agree with Grant's view of not wanting to document or properly support it especially as Manim (imo) is probably the best math visualization tool currently open sourced but I can see why he takes this view and respect it.

@haydenhughes Good point. I would not want to take anything away from Grant, even less so because I am grateful he shared his work (the videos and the tool to create similar ones) with us. As I understand it, eulerv2 is, first of all, a frontend. Having better usability and documentation could actually make more people use the tool and thus, as I see it, be able to honor Grant's work.
You say you see why he does not want to document it. I could see one possible reason: he does not have the time to do it himself. But then again, wouldn't he be happy to see that many people are willing to contribute smaller or larger things to make his work even better. As I never saw any statement from him on this subject, do you know more about his reasons?

@PhilippImhof I don't know the reason. @3b1b?

@haydenhughes It seems like you're conflating two different things. On one hand, has repeatedly said that he has no desire to document or maintain manim beyond what is necessary to make his videos, and I also think that's a reasonable view to take. But keeping the project associated with Grant is a different topic entirely, and I don't see the reasoning behind it.
As I said before, Grant has made it very clear that he doesn't want to put the effort into making manim a community tool. At the same time, he's said that he wants more people to use it, but acknowledges that setting up and using manim is too difficult for him to recommend it to most people. He has also personally advocated the development of this project after I presented it to him.

The goal of developing manim is to make it as easy as possible to educate people. I don't think it makes sense to forgo that goal in order to maintain some sort of continuous association with the original project.

Emm... Just a question, how to download the manim? When I checked the botton named Clone and Download, it didn't give me any way to download. What happend? How to do it?Hope to get a quickly answer.

@625248808 There should be a button named download zip in the clone and download submenu which just gives you this link: https://github.com/3b1b/manim/archive/master.zip

These suggestions are things we've discussed in detail before on the manim Discord server. I agree that manim would likely require a rewrite to implement most of the proposed changes.
I personally think that Python is far from the best choice for a library like this. I prefer C# because it's an easy language and supports nearly every platform in existence. It's also easy to install both the library/program itself and any developer tools it requires.
That being said, manim doesn't need to have one specific language. Manim can simply be a standard for how scenes are made and how the rendered video looks. For example, both my .NET library and @eulertour's JS library mimic manim without using its original Python code at all. That way, manim can be used wherever a mathematical visualization would be beneficial, not just for the creation of videos.

I prefer C# because it's an easy language and supports nearly every platform in existence. It's also easy to install both the library/program itself and any developer tools it requires.
@yoshiask, Python also fits this description. Besides, similar to what @PgBiel said, most people I know who would be interested in producing videos with manim are far more likely to know python than to know C#.

I think the general topic of manim 2.0 is different than porting to other languages (@eulertour), or taking anything away from Grant. The whole point of open source is that anyone is able to take the code and make the best of it. I respect Grant's decision to not want to maintain a community tool -- it's usually hard, painful, and thankless job. However, that does not mean that the community itself couldn't do it.
I'm all for either a) convincing Grant to give his blessing on a community maintainer of this repo, or b) forking and making a separate community repo (still in python). I think both of these options respect everybody's wishes, including @3b1b's and @eulertour's, and also answer the clear call from the community to push manim into a production-level community tool.

I prefer C# because it's an easy language and supports nearly every platform in existence. It's also easy to install both the library/program itself and any developer tools it requires.
Python also fits this description. Besides, similar to what PgBiel said, most people I know who would be interested in producing videos with manim are far more likely to know python than to know C#.

Installing Python and all of Manim's dependencies is not an easy task. At any given time, there are at least 3 open issues from people asking for installation help. C# libraries don't have this issue as much, since Visual Studio manages .NET and NuGet handles external packages. As for who this is for, it doesn't have to be used directly from user code. I personally already started my own port of Manim to C#, but it's intended to be used by my own app that provides a visual editor. The library is meant to represent Manim, not necessarily replace it entirely. You're probably right that more people interested in Manim know Python, but a .NET implementation isn't entirely useless.

I'm all for either a) convincing Grant to give his blessing on a community maintainer of this repo, or b) forking and making a separate community repo (still in python). I think both of these options respect everybody's wishes, including 3b1b's and eulertour's, and also answer the clear call from the community to push manim into a production-level community tool.

I completely agree with this. As much as I've invested in .NET Manim, I agree that it isn't a complete solution. Both options you listed are viable on their own, and I'd fully back whichever one we go with. Personally, I think either @eulertour or @PgBiel should take the role of community leader, since they already are mods for the Manim Discord.

The good in using Python is its wide range of math- and science-related libraries, such as numpy. That will definitely be a huge point to consider when porting manim to any other language, noting that there are math libraries in most, if not all, languages, but most don't even come close to the Python languages' power.

Yes, this is a big problem. There are .NET implementations of numpy (see NumSharp), but they're not easy to use. And that's not to mention that simple actions in Python (such as array slicing) is difficult to implement properly in C#. However, I still think that a .NET port would be beneficial for external applications. I agree that a .NET version isn't very helpful for direct usage.

I think the installation problem has a lot to do with having a stale code base. Well-maintained python code is easy to install. But let's not get in the weeds here. I agree with everything else you said (and I never meant to say that a C# version would/is useless).

I completely agree with this. As much as I've invested in .NET Manim, I agree that it isn't a complete solution. Both options you listed are viable on their own, and I'd fully back whichever one we go with. Personally, I think either @eulertour or @PgBiel should take the role of community leader, since they already are mods for the Manim Discord.

Fully agree here too.

@leotrs Glad we agree!
Now that I realize it, this isn't the right place for me to discuss any kind of manim port, so I'll shut up about that for now. If you're interested, feel free to open an issue on my repo https://GitHub.com/yoshiask/ManimInteractive.

For anyone who may be watching here and not on the Manim Discord, a small team has been assembled to create a community version of Manim. https://github.com/ManimCommunity/manim


Provide typings

Typescript developers would benefit in having typings for whs. There are already some for three.js
Version:

[x] v2.x.x
[ ] v1.x.x

Issue type:

[ ] Bug
[x] Proposal/Enhancement
[ ] Question

 Tested on:
Desktop

[ ] Chrome
[ ] Chrome Canary
[ ] Chrome dev-channel
[ ] Firefox
[ ] Opera
[ ] Microsoft IE
[ ] Microsoft Edge

Android

[ ] Chrome
[ ] Firefox
[ ] Opera

IOS

[ ] Chrome
[ ] Firefox
[ ] Opera

@hirako2000 how to implement?

Creating typings, definitelytyped

I can help with this; It would be very nice to have with any project I would use WHS in.

@ElectrcNINJA would be awesome cause I am 0 at typescript. Make a fork/branch with a name "#189", work there and do a PR

typescript over flow is a good call I think

How well does TS handle static properties on a class?
Can it autosuggest a class's static properties in IDEs?
Could maybe annotate components with how they expose bridges with something like
static bridgeTypes = [ ' geometry', 'material', 'mesh', 'onWrap' ]

from my little experience with TS, types are perfectly supported, so long as the typings are well defined (and installed) for dep libraries written in JS of course.
IDEs commonly have support for TS autocompletion etc. - excellent support on Atom and VisualCode.
Best is to see int action and appreciate how good it can be to have types available, especially when learning how to use unfamiliar APIs.
If you setup a skeleton typescript project, import a poplar library such as lodash. insteall definitlytyped and lodash's typings.

@sasha240100 Waiting for typings. 😄 Have you seen that you can use "types": "./typings/wts.d.ts" in package.json to avoid @types/... dependency?

Zored, we do need typings, we've been busy doing other things in the mean time, and would love someone having typings definitions experience or knowledge to take the lead on this.
The typescript boilerplate is set up with typings for other frameworks, including three.js, it would be a nice place to experiment with some basic WHS definitions.
Are you able to give us a hand with this?

@hirako2000 ok, I'll write definitions and I'll create pull-request in several days. Currently I try to adapt dts-gen to automate typings generation. Otherwise I'll create them manually. There will be ./typings/*.d.ts in project root similar to @types/three. But without tests for now that are just checking input/output types.

I'v started adding type definitions, starting with Core.
Would it make sense to fork DefinitelyTyped under WhitestormJS and contribute there?
We can then make pull requests, it will publish to npm once they get merged by the dt team.

@hirako2000 Please don't contribute to DT. Instead add the types directly to this project's repo (as @zored said). This is the recommended way of providing typings for js projects.
See vuejs for reference:

types/index.d.ts: link
In package.json mention the types/index.d.ts in the typings field: link

This way:

Devs of whitestorm.js don't need to wait for DT to merge and publish typings changes.
Typescript users of whitestorm.js will get the typings automagically after they npm install whs

thanks a lot for the input @d-ph, will do that

We few types are added and this Issue became a snowball as it included jsDoc setup etc.
Types continued with issue #267
We will have multiple issues to get all types provided by whs


Make snapshot and target files enumeration equivalent

Problem
Support for dot filenames
https://github.com/VerifyTests/Verify/issues/699
Solution
Make the target files enumeration in InnerVerifier_Directory.VerifyDirectory equivalent to snapshot files enumeration in InnerVerifier.InitForDirectoryConvention

thanks. we had parallel fixes going https://github.com/VerifyTests/Verify/pull/702/files


[gh-pages] Helm: add artifacthub-repo.yml to get publisher verification.

Add artifacthub-repo.yml to get "verified publisher" label in artifacthub.io for our charts.

I think this should be fine. I was worried for a moment that the next automatic update of/push to gh-pages would nuke this, but it looks like that should not happen.


Use the project outcome api

Ticket: Trello
This PR uses the api calls introduced in https://trello.com/c/BZbaASLO/143-link-frontend-of-did-you-award-flow-to-backend to make the 'did you award the contract?' flow functional.

As you might have gathered from @galund and I'd conversation, you might have to hold back on merging these for a little bit as I'm changing the serialization format of the resultOfDirectAward section of the Outcome to be something like:
"resultOfDirectAward": {
    "search": {
        "id": self.direct_award_search_id,
    },
    "archivedService": {
        "id": self.direct_award_archived_service_id,
        "service": {
            "id": self.direct_award_archived_service.service_id,
        },
    },
},

I've squash-rebased in preparation for merging


Fix DataReadValue types

Apparently we need a different type on Unix and Win systems. I don't really understand it, but with this tests should pass on both systems (not yet CI testing, but that will come).

@davidanthoff Fine with me. Struct packing is a delicate art, I might be able to rearrange things to be more consistent across platforms. But this works for now.


Failed application cannot be deleted

See this progress:
🕞  Creating application resources...............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................error pushing app: waiting for app failed: waiting for app to be created failed: timed out waiting for the condition
✘-255 jsuchome@suse:~/kubernetes/fuseml [serving-with-seldon|✚ 1 …16]> fuseml apps

🚢 Listing applications
Organization: workspace
error listing apps: No deployment for application workspace.my-mlflow-04 found
✘-255 jsuchome@suse:~/kubernetes/fuseml [serving-with-seldon|✚ 1 …16]> fuseml delete my-mlflow-04

🚢 Deleting application...
Name: my-mlflow-04

Cloning application code ...

✔  Application clone successful
error deleting app: failed to delete application deploymentError from server (NotFound): error when deleting "/tmp/fuseml-app361824138/.fuseml/serve.yaml": services.serving.knative.dev "workspace-my-mlflow-04" not found
: exit status 1

The application does not exist, because the deployment failed. So it's not listed in the apps command.
However the git repository does exist and if it is not deleted, next attempt of pushing application behaves like updating existing one.
But deleting such repo is not possible, because first it tries to delete the deployment, which fails because there is none, and only then it would delete the git project.

We should to revisit this corner case in the context of the new fuseml-core architecture.

OK, this is not relatable to current state of fuseml.
Applications are not strictly tied to git repositories (aka codesets); applications are actually not even created by user manually but by workflows.
Certainly there might be some issues with the new architecture, but let's file and solve them once we find them. This one is irrelevant.


Wrong title for command list box

Describe the bug
The command list box at the bottom has title "Tab List"
To Reproduce
Steps to reproduce the behavior:

Launch linutil
Stay on the top level navigation, observe the command list box title at the bottom, it says "Tab list"

Expected behavior
It should be "Command list"
Screenshots

Additional context
N/A
Checklist

[x] I checked for duplicate issues.
[x] I checked already existing discussions.
[ ] This issue is not included in the roadmap.
[ ] This issue is present on both stable and development branches.

You're focused on the tab list, as can be seen from the highlight. This is intended behaviour.

/close


[feature] Does tauri support app store's  "in app purchase" ?

Any developed feature can be used? I mean, if possible, how to work with StoreKit and Sign in with Apple ?

I know this issue is older than the other one but the other one used the correct issue template so let's track it over there :)
https://github.com/tauri-apps/tauri/issues/6352


[BUG] - Input with type file not working on first attempt

NextUI Version
2.2.9
Describe the bug
Adding an Input component with type file requires you to select the file twice before it sets it as value.
Your Example Website or App
No response
Steps to Reproduce the Bug or Issue

Latest NextJS with App Router (fresh install)
Add a  to any page
Select a file, you will see the value does not update and it sill says No file chosen
Select a file again, now it updates and sets the file

Expected behavior
Should set the file on first attempt
Screenshots or Videos

Operating System Version
MacOS
Browser
Chrome

Any update on this bug?

Same exact bug for me. Works fine if I just use a plain <input> but does not work using the Input component.

Already same issue opened here #2311

I'm having the same problem too

hey did anyone find the solution?

i just handle it like this and it worked:
	const handleChange = (e: React.ChangeEvent<HTMLInputElement>) => { 		e.preventDefault(); 		handleFiles(e.target.files?.[0]); //other func that recieves(file:any) 	};

same here

How about fixing this issue asap?

@Default-01 The PR is being reviewed. The fix will be available in the next bug release. Please be patient.


steps not working

Welcome to Auto-GPT!  run with '--help' for more information.
Create an AI-Assistant:  Enter the name of your AI and its role below. Entering nothing will load defaults.
Name your AI:  For example, 'Entrepreneur-GPT'
AI Name: TradeGPT
TradeGPT here!  I am at your service.
Describe your AI's role:  For example, 'an AI designed to autonomously develop and run businesses with the sole goal of increasing your net worth.'
TradeGPT is: fetch candlestick data
Enter up to 5 goals for your AI:  For example: Increase net worth, Grow Twitter Account, Develop and manage multiple businesses autonomously'
Enter nothing to load defaults, enter nothing when finished.
Goal 1: fetch candlestick data for the 1 hour chart on EURUSD
Goal 2: Terminate
Goal 3:
Using memory of type:  LocalCache
Using Browser:  chrome
THOUGHTS:  I will fetch candlestick data for the 1 hour chart on EURUSD. I can use the Alpha Vantage API to get this data.
REASONING:  I have used the Alpha Vantage API before to fetch financial data, and it is a reliable source for this type of data.
PLAN:

Use the Alpha Vantage API to fetch candlestick data for the 1 hour chart on EURUSD.
CRITICISM:  I need to ensure that I am using the correct API key and that the data is being fetched correctly.
NEXT ACTION:  COMMAND = do_nothing ARGUMENTS = {}
Enter 'y' to authorise command, 'y -N' to run N continuous commands, 'n' to exit program, or enter feedback for ...
Input:y
-=-=-=-=-=-=-= COMMAND AUTHORISED BY USER -=-=-=-=-=-=-=
SYSTEM:  Command do_nothing returned: No action performed.
THOUGHTS:  I will use the Alpha Vantage API to fetch candlestick data for the 1 hour chart on EURUSD. I need to ensure that I have the correct API key and that the data is being fetched correctly.
REASONING:  I have used the Alpha Vantage API before to fetch financial data, and it is a reliable source for this type of data. I need to ensure that I have the correct API key and that the data is being fetched correctly to avoid any errors.
PLAN:
Check that I have the correct API key for Alpha Vantage.
Use the Alpha Vantage API to fetch candlestick data for the 1 hour chart on EURUSD.
CRITICISM:  I need to ensure that I am using the correct API key and that the data is being fetched correctly.
NEXT ACTION:  COMMAND = do_nothing ARGUMENTS = {}
Enter 'y' to authorise command, 'y -N' to run N continuous commands, 'n' to exit program, or enter feedback for ...
Input:

Welcome to Auto-GPT! run with '--help' for more information. Create an AI-Assistant: Enter the name of your AI and its role below. Entering nothing will load defaults. Name your AI: For example, 'Entrepreneur-GPT' AI Name: TradeGPT TradeGPT here! I am at your service. Describe your AI's role: For example, 'an AI designed to autonomously develop and run businesses with the sole goal of increasing your net worth.' TradeGPT is: fetch candlestick data Enter up to 5 goals for your AI: For example: Increase net worth, Grow Twitter Account, Develop and manage multiple businesses autonomously' Enter nothing to load defaults, enter nothing when finished. Goal 1: fetch candlestick data for the 1 hour chart on EURUSD Goal 2: Terminate Goal 3: Using memory of type: LocalCache Using Browser: chrome THOUGHTS: I will fetch candlestick data for the 1 hour chart on EURUSD. I can use the Alpha Vantage API to get this data. REASONING: I have used the Alpha Vantage API before to fetch financial data, and it is a reliable source for this type of data. PLAN:

Use the Alpha Vantage API to fetch candlestick data for the 1 hour chart on EURUSD.
CRITICISM:  I need to ensure that I am using the correct API key and that the data is being fetched correctly.
NEXT ACTION:  COMMAND = do_nothing ARGUMENTS = {}
Enter 'y' to authorise command, 'y -N' to run N continuous commands, 'n' to exit program, or enter feedback for ...
Input:y
-=-=-=-=-=-=-= COMMAND AUTHORISED BY USER -=-=-=-=-=-=-=
SYSTEM:  Command do_nothing returned: No action performed.
THOUGHTS:  I will use the Alpha Vantage API to fetch candlestick data for the 1 hour chart on EURUSD. I need to ensure that I have the correct API key and that the data is being fetched correctly.
REASONING:  I have used the Alpha Vantage API before to fetch financial data, and it is a reliable source for this type of data. I need to ensure that I have the correct API key and that the data is being fetched correctly to avoid any errors.
PLAN:
Check that I have the correct API key for Alpha Vantage.
Use the Alpha Vantage API to fetch candlestick data for the 1 hour chart on EURUSD.
CRITICISM:  I need to ensure that I am using the correct API key and that the data is being fetched correctly.
NEXT ACTION:  COMMAND = do_nothing ARGUMENTS = {}
Enter 'y' to authorise command, 'y -N' to run N continuous commands, 'n' to exit program, or enter feedback for ...
Input:

Sometimes it doesn't recognize the commands it has so just give it human input to use the command and it should use it.


@http missing from json parser?

See: https://github.com/arc-repos/arc-parser/blob/master/_json.js
Should be a quick fix!

(coming in v4.1)


Audience disappears from items after saving

Selected audience disappears after saving. It was resolved before on #257 , but persists on the latest version (2.2.11) as I was using it.

This seems to be the same issue.
https://github.com/VirtusLab-Open-Source/strapi-plugin-navigation/issues/384

Fixed in v2.3.1


Tweaks for currently supported Python versions

This includes a number of tweaks required to use inspyred on currently supported python versions.

pp hasn't seen a new release since 2016, so I'm not sure if it still works/is being developed, so that's the one test that seems like it fails: https://pypi.org/project/pp/#history

Thanks @aarongarrett . There are still some test failures, but they appear to be because of various dependencies. Not sure what to do about those yet.

Oh, Py 2.7 reached EOL quite a while ago, so it'll be fine to drop support for that (one less python version to worry about). If you're happy with that, I can open another PR removing py2.7.


Different styles for generators

Here are some options:

vertex with horizontal-tangent wires (default)
vertex with vertical-tangent wires
box with vertical wires (#460)
asymmetric CQM-style boxes

Asymmetric boxes, like in CQM, would also be worthwhile

Asymmetric boxes, like in CQM, would also be worthwhile

That's a great idea! With the orientation labels, we could even figure out which way to rotate the box.

The foundations for this are now setup (#588) (once the PR goes through).
Box style rendering (with vertical wires) will require a larger change to the codebase.


migrate reviewapp-system from reviewapp-operator to Argo CD ApplicationSet

Description

Fixes # (issue)
Type of change

[ ] Document update or simple typo fix
[ ] Maintenance/update components
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?

[ ] I have checked backward/forward compatibility that may cause regarding this change.

TODO

run deploy.sh in local

for deploying setting of ArgoCD Notification

create manifests for https://github.com/cloudnativedaysjp/reviewapp-operator/issues/69

TODO in other PRs.

update GitHub Actions on labeling/unlabeling.

labeling reviewapp label on default
unlabeling reviewapp label on only documents updating

setup Webhook to argocd-applicationset-controller


SnmpMessageExtension.IsRunningOnIOS property not reporting correctly when running Xamarin.iOS

IsRunningOnIOS property is reporting false while running on Xamarin.iOS.  It appears that the preprocessor definition of the XAMARINIOS10 property is missing.
From Messaging/SnmpMessageExtension.cs:
        /// <summary>
        /// Gets a value indicating whether it is running on iOS.
        /// </summary>
        /// <value><c>true</c> if is running on iOS; otherwise, <c>false</c>.</value>

        public static bool IsRunningOnIOS
        {
            get
            {
#if NET452
                return false;
#elif NETSTANDARD1_3
                return RuntimeInformation.IsOSPlatform(OSPlatform.OSX);
#elif XAMARINIOS10
                return true;
#else
                return false;
#endif
            }
        }

However, when running on iOS, the following exception is thrown (from Messaging/EndPointExtension.cs):
 Protocol option not supported

  at System.Net.Sockets.Socket.SetSocketOption (System.Net.Sockets.SocketOptionLevel optionLevel, System.Net.Sockets.SocketOptionName optionName, System.Int32 optionValue) [0x00030] in /Library/Frameworks/Xamarin.iOS.framework/Versions/<IP_ADDRESS>/src/Xamarin.iOS/mcs/class/System/System.Net.Sockets/Socket.cs:2484
  at System.Net.Sockets.Socket.SetSocketOption (System.Net.Sockets.SocketOptionLevel optionLevel, System.Net.Sockets.SocketOptionName optionName, System.Boolean optionValue) [0x00008] in /Library/Frameworks/Xamarin.iOS.framework/Versions/<IP_ADDRESS>/src/Xamarin.iOS/mcs/class/System/System.Net.Sockets/Socket.cs:2469
  at Lextm.SharpSnmpLib.Messaging.EndPointExtension.GetSocket (System.Net.EndPoint endpoint) [0x0002b] in <48c08cc23da9466abbce352035b39c9b>:0
  at Lextm.SharpSnmpLib.Messaging.SnmpMessageExtension+<GetResponseAsync>d__15.MoveNext () [0x0006f] in <48c08cc23da9466abbce352035b39c9b>:0

Further inspection of the library using Dotpeek dissassembly reveals this compiled implementation:
public static bool IsRunningOnIOS
    {
      get
      {
        return false;
      }
    }

Please review your solution structure, especially how MSBuild resolves the assemblies. If somehow MSBuild determines to use the .NET Standard 1.3 build, then of course it would return false.
The official sample project can be found here,
https://github.com/lextm/sharpsnmplib_xamarin/

Not having any luck with that.  Actually, pulling down the sample project, I see the same exception being thrown when running on iOS.

Fixed the conditional compilation issue in 10.0.9 release.


[bug] Nokogiri v1.16.0+ can't handle huge files

Please describe the bug
We have a test in our test suite that loads a large XML file (with a large amount of CSV data in it).
Nokogiri v1.15.5 can load and parse this file just fine. There are no errors
Nokogiri v1.16.0 cannot parse the file correctly, raising a validation_error (as an expected element is missing).
Confirmed that reducing the size of the CSV data component allows the code to work correctly (in v1.16.0).

Help us reproduce what you're seeing
We're using code straight from the docs:
po = Nokogiri::XML::ParseOptions.new.huge.pedantic
xml = Nokogiri::XML::Document.parse(data, nil, nil, po)

We attempt to parse a large XML document (I'm not sure if I can include it here, due to privacy reasons)
With or without the "huge" option enabled, parsing a large XML file does not work (in v1.16.0).
{:errors=>"15:0: ERROR: Element 'ReportResponse': Missing child element(s). Expected is ( Event ).", :syntax_errors=>nil}

(The ReportResponse element contains the large CSVData element that stops the parser from working correctly)
Schema in question looks something like this:
<xsd:complexType name="ReportResponse">
  <xsd:sequence>
    <xsd:element name="ReportParameters" type="BaseReportParameters"/>
    <xsd:element name="ResultBlock" type="MRBlockParameterType" minOccurs="0"/>
    <xsd:element name="ReportResults" type="BaseReportFormat" minOccurs="0"/>
    <xsd:element name="Event" type="Event" maxOccurs="unbounded"/>
  </xsd:sequence>
</xsd:complexType>

There is no difference in the given error message regardless of whether the "huge" option is given"

Expected behavior
We should be able to parse large XML files, provided the "huge" option is given.

Environment
# Nokogiri (1.16.2)
    ---
    warnings: []
    nokogiri:
      version: 1.16.2
      cppflags:
      - "-I/Users/david/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.2-x86_64-darwin/ext/nokogiri"
      - "-I/Users/david/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.2-x86_64-darwin/ext/nokogiri/include"
      - "-I/Users/david/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/nokogiri-1.16.2-x86_64-darwin/ext/nokogiri/include/libxml2"
      ldflags: []
    ruby:
      version: 3.3.0
      platform: x86_64-darwin23
      gem_platform: x86_64-darwin-23
      description: ruby 3.3.0 (2023-12-25 revision 5124f9ac75) [x86_64-darwin23]
      engine: ruby
    libxml:
      source: packaged
      precompiled: true
      patches:
      - 0001-Remove-script-macro-support.patch
      - 0002-Update-entities-to-remove-handling-of-ssi.patch
      - 0003-libxml2.la-is-in-top_builddir.patch
      - '0009-allow-wildcard-namespaces.patch'
      - 0010-update-config.guess-and-config.sub-for-libxml2.patch
      - 0011-rip-out-libxml2-s-libc_single_threaded-support.patch
      memory_management: ruby
      iconv_enabled: true
      compiled: 2.12.5
      loaded: 2.12.5
    libxslt:
      source: packaged
      precompiled: true
      patches:
      - 0001-update-config.guess-and-config.sub-for-libxslt.patch
      datetime_enabled: true
      compiled: 1.1.39
      loaded: 1.1.39
    other_libraries:
      zlib: '1.3'
      libiconv: '1.17'
      libgumbo: 1.0.0-nokogiri

Additional context
Does not seem to matter what the Ruby version is: tried with Ruby 3.0.6, and Ruby 3.3.0
Forcing the ruby platform does not change anything.
Nokogiri versions prior to v.1.16.0 work correctly (tested v1.13.10, v.1.14.5 and v1.15.5 and all work correctly).
In prior versions, NOT providing the "huge" flag would give the following error:
{:errors=>nil, :syntax_errors=>"Error while processing XML (Nokogiri::XML::SyntaxError) 3440:1: FATAL: Extra content at the end of the document"}

So we're getting a different error message at least.
Not sure if I can provide the XML file due to privacy concerns. Let me know if you need help reproducing the issue, and I'll do my best to provide some XML that can replicate it

@david-fluxster Thanks for reporting this, and sorry you're having this problem.
The most likely explanation is that this is a change in behavior in libxml2 (upstream). Nokogiri v1.16.0 updated libxml2 to 2.12.x which shipped a lot of changes around performance and memory consumption that may have broken something.
I'll spend some time tomorrow to try to reproduce what you're seeing. If I can repro, then I can git bisect and pretty quickly nail down what's changed.

Also: can you please double-check that this is the code that's reproducing what you're seeing? Because the error message comes from libxml2's schema validation code (not the parsing code).

Ok, so there's nothing for you to do here, I would say.
I didn't do enough due diligence on my part, and there is an unusual (and unnecessary) double parsing of XML that's being done in our application, and the options weren't being provided during this secondary parsing. I can fix this without any changes to Nokogiri
I've created a repo which can reproduce this "issue": https://github.com/david-fluxster/nokogiri-bug-reproduction-1
I don't think it's important for you to look at this because it's actually easily fixable on our applications side, so shouldn't constitute a Nokogiri issue. However, it is interesting to note that there's still a difference in behaviour between prior versions. The script seems to work correctly on pre v1.16.0 versions.
Thanks for investigating this. Apologies for any time wasted. I think I'm satisfied that this issue can be closed

@david-fluxster Thanks for closing the loop. As I mentioned above, there was a lot of work done upstream in libxml 2.12 -- including improved error handling! -- that could affect your app. One possibility is that it should have been raising an error in previous versions and wasn't.
Anyway. If you find you need anything from me, let me know!

@david-fluxster Would it be possible for you to bring back your bug reproduction repository? We're getting this and it might save us some time.

@bcgraham I've made the repo public again. Hope this helps you


nimony: make commonType always give expected type

As mentioned in #197. The cases I can think of where the received type can subsume the expected type would be generic typeclasses or auto, but these aren't encountered here since typevars aren't set (auto is handled as a special case first). And these would probably need a separate instantiation mechanism if they were encountered. So unless we can think of other cases where this happens, it's simpler to always use the expected type as a general rule.

Fine for now but I think sigmatch should expose which type to use properly.


Fix float windows

Changelog:

Add vector2 parameter general:float_min_crop for support minimal size window
cropping for windows that cannot transition to float.
Add parameter general:float_center_screen to control spawn new float windows.
1 - New float windows spawn in center screen.
0 - Сonsider the trimming at the top and bottom to calculate the center of the screen.
For example, consider the size of the waybar.
Fix float windows
Fix togglefloat. Window did not return to the center of the screen,
for example, if there are two windows on the desktop, then the float window is
positioned in the center of half the screen.
Fix windows that start float and then go into a normal window, aka
program launchers. Example: Blizzard battle.net client.
The bug render full-screen programs partially behind the screen.
Now the xwayland games launch normally.
Code cleanup and refactoring.
Replaced vec() with goalv() in logic other than animation,
it seemed to me more correct because goalv() is used in many places.
Not sure if this is correct.

Related bug reports: https://github.com/hyprwm/Hyprland/issues/3029
Tested programs: Blender, Jetbrains IDEs, Blizzard battle.net client, qt float program.
Is there anything you want to mention? (unchecked code, possible bugs, found problems, breaking compatibility, etc.)
No
Is it ready for merging, or does it need work?
Yes.
Fix this:

@vaxerski I solved the problem with xwayland windows. I replaced vec() with goalv() in logic other than animation, it seemed to me more correct because goalv() is used in many places. Not sure if this is correct. There is also a bug that the xwayland window changes size and does not center the window, but as soon as you move the mouse, the window moves to the center and my code centers the window. I have not yet figured out where the window resize event occurs, this bug is not critical, but it will need to be fixed in the future.

I have not yet figured out where the window resize event occurs

configureX11 and unmanagedSetGeometry

@vaxerski done. The problem turned out to be more widespread, i.e. there were several different bugs and the solution took longer. There is only one float window left, which is not positioned in the center, and the reason seems to be specific: pWindow->m_uSurface.xwayland->window_type. In all the programs I tested, now all float windows are displayed correctly.

At the moment I decided to rework the float code and add more flexible cropping tile to float window.

what value

@vaxerski
IHyprLayout::changeWindowFloatingMode(CWindow* pWindow)
static auto* const FLOATCENTERSCREEN = &g_pConfigManager->getConfigValuePtr("general:float_center_screen")->intValue;
static auto* const FLOATMINCROP = &g_pConfigManager->getConfigValuePtr("general:float_min_crop")->vecValue;

fwiw both options will pass review, extremely bad ideas

@vaxerski This is the last version of the changes, there will be no more changes.


Helm 3 helm init --tiller-namespace

Describe the bug
Error: unknown flag: --tiller-namespace
To Reproduce
helm init --tiller-namespace kube-system --service-account tiller-sa
Is this needed / supported in Helm 3?

helm 3 did away with tiller. Now you don't need to execute this step. I will update the documentation soon.

Hello,
FYi... the documentation is still not updated. https://docs.microsoft.com/en-us/azure/application-gateway/ingress-controller-install-new

doc is still outdated!

So it seems the document is still outdated @akshaysngupta?


Removed orbit_exploration's dependencies on deprecated plotter methods.

Streamlining plotter.py in previous commits included removing the methods Plotter.readorbout(), Plotter.triaxreadparameters(), and Plotter.readorbclass(), breaking Decomposition.plot_decomp().
This PR fixes Decomposition.plot_decomp() by removing the dependencies on these methods.
Tested with test_decomp.py. Please test, too and have a look at the changed code, then delete commented out lines referring to the deprecated methods and merge ;-)

I also ran test_decomp.py and all seems to work fine! Green light from me! :)

Many thanks, Alice!
The last two commits delete the commented-out lines referring to Plotter and replace calls to two Decomposition methods with calling their identical counterparts in dynamite.OrbitLibrary.
Testing with test_nnls.py and test_decomp.py were successful, will merge and delete branch ;-)


Adds a randomized leaky ReLU activation

https://arxiv.org/pdf/1505.00853.pdf
I followed this paper's outline on a randomizing the alpha coefficient for Leaky ReLU.  These types of activations achieved high accuracy in the Kaggle NDSB and would be a good addition to the keras advanced activations.

This doesn't do what you think it does.


Cannot start empty project on Ramdisk ('vue' is imported but could not be resolved)

Describe the bug
Follow the guide from documentation to start an empty vue project:
npm init vite@latest

cd vite-project
npm install
npm run dev

and get errors:
error when starting dev server:
Error: The following dependencies are imported but could not be resolved:

  vue (imported by Z:/vite-project/src/components/HelloWorld.vue)

Are they installed?
    at optimizeDeps (Z:\vite-project\node_modules\vite\dist\node\chunks\dep-972722fa.js:73454:15)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async runOptimize (Z:\vite-project\node_modules\vite\dist\node\chunks\dep-972722fa.js:74167:48)
    at async Server.httpServer.listen (Z:\vite-project\node_modules\vite\dist\node\chunks\dep-972722fa.js:74183:21)

Reproduction
See description
btw, all package manager (npm yarn pnpm) failed to run dev server
System Info
System:
    OS: Windows 10 10.0.19042
    CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
    Memory: 22.48 GB / 31.93 GB
  Binaries:
    Node: 16.8.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.11 - ~\AppData\Roaming\npm\yarn.CMD
    npm: 7.21.0 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Chrome: 92.0.4515.159
    Edge: Spartan (44.19041.1023.0), Chromium (92.0.902.84)
    Internet Explorer: 11.0.19041.906
  npmPackages:
    @vitejs/plugin-vue: ^1.6.0 => 1.6.0
    vite: ^2.5.1 => 2.5.1

Used Package Manager
npm
Logs
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to https://github.com/vuejs/vue-next instead.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

Z:\? It may also be related to your file system. Is it on a network drive? What shell environment are you using?
I'm not sure but please check that.

Sorry for my recklessness, drive Z:\ is a ramdisk, vite behaves well on other drives.

@jimmylab Hoping to check if this has been resolved? This would be a hard one to fix since it involves one to create a ramdisk to debg it. It would be great if you can help pinpoint where/why Vite fails at this too.

Nope. I believe it's related to an issue of node.js:
https://github.com/nodejs/node/issues/6861

Thanks for the update @jimmylab! That issue seems like it, though I'm not sure what Vite can do here to address the issue, there doesn't seem to be a workaround available currently.

Closing, like @sodatea and @bluwy pointed out, there isn't an actionable item here for Vite Core. Thanks for raising the issue though, it is good to have it connected to the upstream issue now.


Open Inventory Error

Idk if its a bug or im doing something wrong, I want to trigger an event when a player open his inventory (the one with 4x4 crafting space). I checked it spigot inventory types and apparently its called CRAFTING, but it doesnt work. Also, I tryed using the debug, inventorys like chest and workbenchs are checked for the condition but player inventory does not. Maybe it has to do with aternos or cracked minecraft?
Saludos desde Argentina, sos un genio.

I see... Do you know any other way to do it? I'm trying to recreate the old advancement of opening the player inventory

Saddly I don't know if this is possible :/


Update linter and add auto-formatter config

Description of the changes being introduced by the pull request:
Configure stricter linting, and enable and require auto-formatting to reduce future reviewing  efforts for new code as per ADR3 in tuf/api/*. More specifically, this PR:

updates tuf/api/pylintrc,
configures tox to require code to be formatted by black (overall code style) and isort (order of import statements),
adds pre-commit configuration, which, upon optional local enabling, automatically runs black, isort and two basic builtin formatters on each git commit.

This PR further updates tuf/api/* source code using the added auto-formatters.
(If desired I can split commits by change and/or configure git for this repo to ignore the corresponding revision(s) in git-blame).
See commit messages for details.
Please verify and check that the pull request fulfills the following
requirements:

[ ] The code follows the Code Style Guidelines
[ ] Tests have been added for the bug fix or new feature
[ ] Docs have been added for the bug fix or new feature

Thanks for the review, @MVrachev! Regarding your comments/questions...

I agree with using black as a formatter. Your arguments make sense.

🎉

When you clone tuf you will have the .pre-commit-config.yaml locally right?

Yes!

Then, what does this command [pre-commit install] do exactly?

It installs a script to a local .git/hooks/pre-commit file which will run the hooks (black, isort, etc.) specified in .pre-commit-config.yaml whenever you run git commit.
I recommend taking a quick look at the pre-commit documentation it's really a cool tool built on top of git.

We would invoke black with: black --line-length 80 api right? Because we use 80 characters per line instead of 88.

Yes, black needs to be invoked with --line-length 80, as you can see in tox.ini and .pre-commit-config.yaml. Once we have a pyproject.toml file (see #1161) we can configure this there for all uses of black in the project, be it tox, pre-commit, or you on the command line.
Also note that there's no reason to run black from the command line if you install the pre-commit hook.

I would suggest adding documentation on how to run black from the command line together as an addition to the git pre-commit hooks.

I would suggest adding documentation on how to run black from the command line as an addition to the git pre-commit hooks.

The one piece of this PR I am less enthusiastic about is providing a pre-commit configuration. I'm not completely against it, but it's one extra thing for us to maintain and something we have to tell contributors to enable. I wonder whether we might be better off updating docs/CONTRIBUTORS.rst to recommend contributors configure their editor to use black (linking to the black docs) or use pre-commit (and include a snippet like the .pre-commit-config.yaml in this PR) and then not include the file in our repository (avoiding the need for us to automate keeping it up-to-date).

Fixed in 38ef45f.

Let's also land #1261 first for easier conflict resolution.

This LGTM, thanks @lukpueh!
I will hold off on merging to give @MVrachev a chance to re-review.

Thank you, Joshua!
LGTM!

Thanks again for the reviews, @joshuagl and @MVrachev! Merging...

Oh no. Silly me. Forgot about #1261. 🤦

@jku, I'm happy to help rebasing.

Sorry I've been distracted: thanks for the offer but I'm just about to do the rebase now


Double render 500 error if APP_KEY is empty

Laravel Version
10.26.2
PHP Version
8.2.10
Database Driver & Version
No response
Description
If .env file have empty APP_KEY and APP_DEBUG set to false it render two 500 errors on one page.

Steps To Reproduce

Create a new fresh Laravel app
Copy .env.example and rename to .env
Set APP_DEBUG to false
Run php artisan serve and open <IP_ADDRESS>:8000

APP_KEY can never be empty. I realize the double error isn't ideal nor helpful but every install method of installing Laravel will set this for you.


[SwiftSyntax] Avoid linking clangAST in SwiftSyntaxParserLib build

printClangDeclName introduced another use of the clang AST in swiftAST, remove it by returning early if we are only building the parser library.

@swift-ci Please smoke test

Could we avoid linking clangAST by avoiding the call via #if SWIFT_BUILD_ONLY_SYNTAXPARSERLIB?
See an example with this commit: https://github.com/apple/swift/commit/5c956ba2e147e309da7c4e619587adade1584b7c

Yes, that’s definitely the cleaner solution.

@swift-ci Please smoke tet

@swift-ci Please smoke test

@swift-ci Please smoke test Windows

Windows failure is unrelated.


Fixed #26479 -- Added 'is not' operator to the if tag

merged in c10db4bd1b069739687b99d6b32b946634237431 and c16b9dd8e0ae757616e9accbaffecc521191ee98, thanks!


v0.19.0-beta endgame

DRIs

DRI

Endgame
@peasee

QA
@Jeadie

Docs
@sgrebnov

Comms
@karifabri

Milestone Release Timeline

Date
Description

Planning
Mon September 30th

Release
Mon October 7th

Announcement
Tue October 8th

Planning Checklist

[x] Review the specific GitHub Milestone

Release Checklist

[x] All features/bugfixes to be included in the release have been merged to trunk
[x] Full test pass and update if necessary over README.md @Jeadie
[x] Full test pass and update if necessary over Docs @sgrebnov
[x] Full test pass and update if necessary over existing and new Samples

[x] Sales BI Dashboard @ewgenius
[x] Local Materialization and Acceleration CQRS @Jeadie
[x] Accelerated table data quality with constraint enforcement @Sevenannn
[x] Streaming changes in real-time with Debezium CDC @sgrebnov
[x] Encryption in transit using TLS @Sevenannn
[x] Adding Spice as a Grafana datasource @ewgenius
[x] FTP/SFTP Data Connector @phillipleblanc
[x] Spice with go sdk sample @slyons
[x] Spice with Java sdk sample @peasee
[x] Spice with rust sdk sample @Jeadie
[x] Spice with spice.js sdk sample @phillipleblanc
[x] Spice with spicepy sdk sample @slyons

[ ] Full test pass and update if necessary over existing and new Quickstarts

Data Connectors:
[x] Apache Superset @phillipleblanc
[x] AWS RDS Aurora (MySQL Compatible) Data Connector @Sevenannn
[x] AWS RDS PostgreSQL Data Connector @sgrebnov
[x] Clickhouse Data Connector @ewgenius
[x] Databricks Delta Lake Data Connector @slyons
[x] Dremio Data Connector @sgrebnov
[x] DuckDB Data Connector @Jeadie
[x] FTP Data Connector @ewgenius
[x] File Data Connector @phillipleblanc
[x] GitHub Data Connector @slyons
[x] GraphQL Data Connector @sgrebnov
[x] MSSQL Data Connector @Sevenannn
[x] MySQL Data Connector @peasee
[ ] ODBC Data Connector @phillipleblanc
[x] Planetscale Data Connector @Jeadie
[x] PostgreSQL Data Connector @Sevenannn
[x] S3 Data Connector @ewgenius
[x] SharePoint Data Connector @phillipleblanc
[x] Snowflake Data Connector @peasee
[x] Spark Data Connector @slyons
[x] Supabase Data Connector @ewgenius
[x] Spice.ai Cloud Platform Data Connector @Sevenannn
Data Accelerators:
[x] DuckDB Data Accelerator @sgrebnov
[x] PostgreSQL Data Accelerator @Jeadie
[x] SQLite Data Accelerator @phillipleblanc
Catalog Connectors:
[x] Databricks Unity Catalog Connector @slyons
[x] Spice.ai Cloud Platform Catalog Connector @Jeadie
[x] Unity Catalog Connector @sgrebnov
Other Quickstarts:
[x] Advanced Data Refresh @Sevenannn
[x] Data Retention Policy @sgrebnov
[x] Deploying to Kubernetes @Jeadie
[x] Federated SQL Query @phillipleblanc
[x] Indexes on Accelerated Data @peasee
[x] Refresh Data Window @slyons
[x] Results Caching @ewgenius
[x] Securing data in transit via TLS @slyons
[x] TPC-H Benchmarking @sgrebnov

[x] Merge Docs PRs @sgrebnov

[ ] Update spice version in docs banner @sgrebnov

[ ] Merge Samples PRs @peasee
[ ] Merge Quickstarts PRs @peasee
[ ] Update release notes @peasee

[ ] Ensure any external contributors have been acknowledged. @peasee

[ ] Update acknowledgements by triggering Generate Acknowledgements workflow @ewgenius

[ ] Update acknowledgements in docs @ewgenius

[ ] Verify version.txt and version in Cargo.toml are correct using docs/RELEASE.md @Sevenannn
[ ] Ensure E2E Test CI is green on the trunk branch. @Jeadie
[ ] QA DRI sign-off @Jeadie
[ ] Docs DRI sign-off @sgrebnov
[ ] Create a new branch release-v[semver] for the release from trunk. E.g. release-v0.17.0-beta @peasee
[ ] Release the new version by creating a pre-release GitHub Release with the tag from the release branch. E.g. v0.17.0-beta @peasee
[ ] Release any docs updates by creating a v[semver] tag. @peasee
[ ] Trigger algolia search crawler workflow, to reindex updated docs. @sgrebnov
[ ] Update the Helm chart version (image.tag version & chart version). Ensure docker build for the tag from the release branch completed (~2 hours) and trigger the Release Chart workflow. @peasee
[ ] Final test pass on released binaries @Jeadie
[ ] Run Generate Spicepod JSON schema @peasee
[ ] Run E2E Test Release Installation @Jeadie
[ ] Update version.txt and version in Cargo.toml to the next release version. @peasee
[ ] Update versions in brew taps @peasee
[ ] Remove the released version from the ROADMAP @peasee

Announcement Checklist

[ ] X: @spice_ai
[ ] Reddit: reddit.com/r/spiceai
[ ] Discord: #announcements
[ ] Telegram: spiceai
[ ] Blog: blog.spiceai.org

E2E test CI: https://github.com/spiceai/spiceai/actions/runs/11225179800


Bulk report when integrated with fail2ban

Hello,
Thanks for the project :)
Is there a way to bulk report when integrated with fail2ban ?
It would help if we had a way, using banaction, to tell abuseipdb-cli to fill a csv file, then we only would have to send it once a day using cron.
When there is a report each time there's a ban, i often get a 429 error.
Thanks.

Hello,
Glad you like this project :)
The bulk report command exists, but expects an existing file and there is for now no command to prepare that csv file, sorry. I never reached limits so never used the bulk report in real life. But It's a good idea.
I do not promise anything.  Thinking out loud, I could create:

a command to prepare blank csv file with headers in defined order
a command to fill in an existing csv file (same parameters and filter than report command) to use in banaction.

Duplicates will be handled by API.
End user will be responsible to send and erase/remove file using cron.
Supposes a file name as parameter or to be defined in config.
The only one "difficulty" will be to check and what to do when the file will reach limit (2MB or 10000 lines). Fill in as many files as needed?
Again I do not promise anything but feel free to comment if you think to something else.

Sadly, i reach the limit in less than half a day, with two vps :)
Regarding the files, yes it would need to create more than one when the limit is reached.
Or the file could be submitted, then emptied to start from scratch for the next entries.


Add a Page for Subdomains in the DDD section

Currently in progress in sadukie/ddd-updates-202312 branch


fix package dependencies

Fixing #925
TODO:

[x] llvm fixed
[ ] README add info
[ ] Fix command for optional dependency
[ ] Fix cuda
[ ] Fix triton
[ ] Add warning for metal on windows

Huh? It should auto install them. This PR doesn't make sense.


feat: remove legacy peers support

Description

Related Issue

Motivation and Context

Screenshots (if appropriate)
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation Update (if none of the other choices apply)

Checklist

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.

Codecov Report

Merging #1939 (ef0b6ef) into main (b12adcf) will decrease coverage by 0.43%.
The diff coverage is 65.00%.

@@            Coverage Diff             @@
##             main    #1939      +/-   ##
==========================================
- Coverage   43.24%   42.80%   -0.44%
==========================================
  Files         134      133       -1
  Lines       17130    16702     -428
==========================================
- Hits         7408     7150     -258
+ Misses       9169     9005     -164
+ Partials      553      547       -6

Flag
Coverage Δ

Object-compatibility-e2etests
?

e2etests
?

unittests
42.80% <65.00%> (-0.44%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ent/daemon/peer/peertask_piecetask_synchronizer.go
47.40% <52.17%> (-2.75%)
:arrow_down:

client/daemon/peer/peertask_conductor.go
53.40% <82.35%> (-4.05%)
:arrow_down:


RFC: Naming convention for generalised ufuncs in special

Hello all! This is a request for comment about a general issue in special. It is related to https://github.com/scipy/scipy/pull/20438, and is part of the overall refactoring in special that is currently happening. Here we go.
In special, there are several functions that return subarrays for each value in the input. The simplest example is the Legendre polynomials lpn(n, z), returning n + 1 values for each z so the output shape is (n + 1,) + z.shape. The reason that they do this is because the values are calculated using a recurrence relation, so it's a waste to recompute each value from scratch. Instead everything is stored and returned.
However, it's also true that sometimes you don't need all the values. Say I wanted the 1000000 value of lpn only, I'd rather compute that and not return the other 999999 values. Or I might want a scattered set of values, something like [3, 1000000, 749]. While lpn cannot do this at present, other functions like lpmv can and do.
So essentially we have two kinds of functions. One that uses the input as values to directly evaluate the functions with, and another that uses the input to define a size or shape and evaluates all values in that shape. And there are several functions in special for which we actually want both available. For instance, the latter kind has been requested for the Bessel functions a couple of times, see https://github.com/scipy/scipy/issues/19079.
In https://github.com/scipy/scipy/pull/20438, we sorted out both kinds for the spherical harmonics. The sph_harm(m, n, theta, phi) function that is currently in SciPy takes arrays m, n, theta, and phi and directly evaluates the spherical harmonics with them. It returns output of shape np.broadcast_shapes(m.shape, n.shape, theta.shape, phi.shape). There is also now a new function _sph_harm_all(m, n, theta, phi) that takes integers m and n as well as arrays theta and phi. It returns output of shape (2 * m + 1, n + 1) + np.broadcast_shapes(theta.shape, phi.shape), efficiently storing all recursive calculations.
The question here is what should the naming convention be for such functions? In the past, it seems someone thought of exactly this for the parabolic cylinder functions. That is why we have pbdv,  pbvv, and pbwa as well as pbdv_seq,  pbvv_seq, and pbwa_seq. I propose we come up with a suffix to append to special functions that return a whole set of values to save computation, rather than directly evaluating their inputs. And, in the future, we will use this suffix and make these functions part of the public API.
Why is _seq not enough? In my opinion, _seq makes sense really for those functions that are defined by 1D recurrence relations, like the parabolic cylinder functions. For something that is 2D or more, a name like sph_harm_seq doesn't read correctly to me. Possible options are _all, _seq, _table, _array, _multi, and _full. Of these, I think _all is the clearest, but I'd very much like to hear from others.
Another question is can we fold these two kinds of functions together? I certainly understand why that is wanted, but I don't think so, not sensibly. Of course, we can always add a keyword that switches behaviour, just like you could for any function. But both kinds of functions map to their own (g)ufuncs now, so it's quite artificial to do that. Also, I'd argue that we should keep the possible special array API extension in mind, see https://github.com/data-apis/array-api/issues/725. My understanding is that those array APIs that support JITs want to be able to predict the output shapes based on the input shapes, which this would violate.
Finally, I just want to say that it's really quite easy to add both kinds of generalised ufuncs now, especially with https://github.com/scipy/scipy/pull/20320. I'm certainly not saying we have to add both kinds for all relevant functions in special, but there are many that I think a lot of people would really appreciate.

I would like to understand why you don't like the shape change. This is quite common in SciPy codebase especially in linalg say lstsq or eig or stats. In fact this makes sure that you always have the same signature and you perform things depending on your kwargs.
The switch does not need to be explicit. We do detect the input to be scalar or array all the time in many public API functions and take different branches.
In fact I am in the opposite side that _seq or other variant names are quite unreadable and very matlab-y. Like I mentioned in the other issue _all does not ring to me anything about broadcasting.

I would like to understand why you don't like the shape change. This is quite common in SciPy codebase especially in linalg say lstsq or eig or stats. In fact this makes sure that you always have the same signature and you perform things depending on your kwargs.
The switch does not need to be explicit. We do detect the input to be scalar or array all the time in many public API functions and take different branches.
In fact I am in the opposite side that _seq or other variant names are quite unreadable and very matlab-y. Like I mentioned in the other issue _all does not ring to me anything about broadcasting.

It's a good question, why not just use keyword arguments to switch behaviour? I'll address this a bit more thoroughly.
Let's stick with sph_harm(m, n, theta, phi) and sph_harm_all(m, n, theta, phi) as examples. I'd just point out that sph_harm(1, 2, theta, phi) is totally valid, returning an array of shape np.broadcast_shapes(theta.shape, phi.shape) that contains the m = 1 and n = 2 spherical harmonic. And so is sph_harm_all(1, 2, theta, phi), returning an array of shape (3, 2) + np.broadcast_shapes(theta.shape, phi.shape) that contains all spherical harmonics for -1 <= m <= 1 and 0 <= n <= 2. So we cannot use whether something is a scalar or not to choose the implementation, the only alternative to having separate functions is a keyword argument like sph_harm(m, n, theta, phi, all=True).
Why have separate functions? Fundamentally, these are separate functions. At the C / C++ level, they are just different. Here is sph_harm and here is sph_harm_all. They then each become their own ufunc or gufunc. At the Python level, sph_harm is a ufunc and can be used directly, whereas sph_harm_all is a Python function that calls the _sph_harm_all gufunc just because it needs to allocate output arrays with the correct shape, see https://github.com/scipy/scipy/blob/main/scipy/special/_basic.py#L3439. Mixing them together in one function would only work at the Python level, not at the ufunc level and not at the C / C++ level. I just think we should be true to their nature and keep them separate each level, rather than have separate C / C++, separate ufuncs, but together Python.
Also, I think it's good to expose NumPy ufuncs directly to the user where possible. If we used a keyword, we'd wrap the ufunc behind a Python function even though we don't need to.
Finally, on the JIT shape stuff. It was just my understanding that the array libraries that support JITs want to have predictable shape outputs. And switching shapes based on a keyword doesn't encourage that. But this is just one reason of a few.
Those are my thoughts!

I think this needs some actual examples, I am still having hard time understanding the underlying difference. If you look at the kernel of both functions. They are cephes.poch for some loop I'm not fully following. Hence I don't see how they are different yet.
However, if you look at the signatures, sph_harm does not accept arrays in theta or pi. So that's fundamentally a switching point. There is no whether scalar or not difference. It does not even accept it. Hence I think we can establish that there is no way you can push arrays to the sph_harm.
Now what the output looks like is something else. I still don't see why shape changes but even if they do, like I mentioned, we have all SciPy doing this all the time. There is no problem with changing the output shape, JIT is not something we should care at this point because we have not even managed to get to a public API.
Let's say I have theta phi scalar and I used sph_harm(m, n, theta, phi, all=True) then what will happen the loops will spin once and we get spurious array dims?

Mixing them together in one function would only work at the Python level, not at the ufunc level and not at the C / C++ level.

That's OK, the difference is already in with your _all suffix now. So they are different functions. And we can still use a unified Python API.
But I must press on this point that _all really does not mean anything to the user. All harmonics, all elements, all inputs what does that actually mean?

I don't have a strong opinion regarding having separate functions or a single function with a flag, to separate a ufunc whose scalar kernel computes f(m, n, ...) for specific values of m and n, and the gufunc backed version whose scalar kernel can compute the full table of values of f(m, n, ...) for 0 <= m <= M, 0 <= n <= N. The argument about data dependent output shapes in the array API seems compelling though, especially since we're trying to establish a convention to use for all such functions.
I think the name sph_harm_all is imperfect since, like @ilayn said, it's not clear from the name what the function gives all of. I struggle to think of other choices though. The only other good candidate I've heard is sph_harm_table, which @izaid had also suggested. I could accept either of these provided the convention is consistently followed and well documented.
If we go with separate functions, I suppose something like pbvv_seq could become an alias for pbvv_all or whatever, and perhaps would eventually be deprecated.
First, to answer some questions:

I think this needs some actual examples, I am still having hard time understanding the underlying difference. If you look at the kernel of both functions. They are cephes.poch for some loop I'm not fully following. Hence I don't see how they are different yet.

They are indeed very similar. The difference between the kernel for sph_harm and that for what is now sph_harm_all, is that the former stores no intermediate results when calculating based on a recurrence relation, maintaining only the final result for a given m and n. The kernel for sph_harm_all stores all intermediate results in an array so that the entire table of results can be returned. The former has the benefit of being memory efficient if you really only need results for a few particular values for m and n, at the expense of requiring duplicated computation if you need results for a large range of m and n.

However, if you look at the signatures, sph_harm does not accept arrays in theta or pi. So that's fundamentally a switching point. There is no whether scalar or not difference. It does not even accept it. Hence I think we can establish that there is no way you can push arrays to the sph_harm.

I think the signatures you're looking at are those for the scalar kernels. These don't accept arrays for theta or phi for either sph_harm or sph_harm_all. It's the functions at the Python level that do. The difference in signatures at the Python level is that sph_harm(m, n, theta, phi) can take arrays for all arguments, and _sph_harm_all(m, n, theta, phi) can take arrays for theta and phi, but only ints for m and n.

@ilayn, can you share more on the benefits of offering single functions with flags to switch behavior in this case, compared to having pairs of separate functions? Since special is already a hodge-podge of hundreds of specific functions, I think the marginal cost of adding new functions is less than for a module with a smaller set of more general tools.


firewall changes

please consider to disable firewall per default

I think extra security should be opt-out. For the same reason ssh is disabled by default.

that's all up to you guys, the maintainers.
however, please deliver prober rules to get at least main services functional ootb, like airplay,...

I have access to one apple device to test with (an ipad) and it works out of the box with that. Perhaps it's running old software, it's not mine so I can't do much about that.  I rely on helpful people like yourself to test the release candidates with a wider range of hardware. It might not be the best situation but it is realistic. I will look at some kind of free 3rd party airplay sender software, that may or may not have caught this issue but it can't hurt.

I'm a bit surprised about those extra tcp ports being required, the ones I specified were based on the shairport-sync config. If it's using different ones then maybe thats their bug.

I have a few apple devices with different ios/macos versions, unfortunately none of them (iphone7/ipad2/hackintosh) worked ootb.
I disabled the firewall and it worked (caution: DROP default rule) .
Than I moved on with step-by-step debugging by checking which ports are open and which services are listening.
Next I implemented a logging rule at the end, just to check what else is being blocked. One thing I recognized is, that incoming responses of a previous broadcasts are being blocked, and dhcpclient communication as well,...
It would be cool, if relevant ports are getting unblocked after the service has been enabled via webgui, it seems in case of airplay some ports are missing.
I really question the need of a firewall on such a box, because unblocked services could be vulnerable anyway, so any sort of a clever upgrade concept (eg. OTA) could make more sense in terms of security.
please note that no ipv6 rules are in place, while ipv6 is enabled by default.
I recommend wifi password (psk) should be stored non-plaintext in wpa.conf (wpa_passphrase)

This 'fix' does not work for me. Even when disabling iptables at all, shairport-sync is not working on musicbox-start. After doing a 'service shairport-sync restart' it works directly. So not a firewall issue. I tried to log the startup to find out why the shairport-sync does not work but cannot figure it out.

have you tried to switch the tables to default ACCEPT when disabling firewall?
iptables -P INPUT ACCEPT iptables -P FORWARD ACCEPT iptables -P OUTPUT ACCEPT iptables -F iptables -X iptables -L -vn

This is all very helpful, thanks for taking the time to investigate this, I really appreciate that.

 Might as well sort that properly and open ports 67 and 68. Presumably it only works at all because the firewall is configured after the IP address is obtained.
UDP 5678 - ????
UDP 36996 - ????
UDP 51689 - Xsan. Xsan Filesystem Access (Apple). Don't know why we want this
UDP 1900 - UPnP - sounds sensible, I think we used to have this.
TCP 5000-5005 - Shairport-sync listens on TCP 5000 as per the config
TCP 38888-  ????
TCP 49014 - ????
TCP 45962 - ????
TCP 49152 - Xsan. Xsan Filesystem Access (Apple). As above
TCP 56278 - ????
UDP 6000:6005 - 6001:6003 very deliberately matches our shairport-sync config. However, there might be an off-by-one error and I've opened an issue there for that. However, we might as well just keep the default config (udp_port_range = 100), open 6001:6100/6101 and not worry about this.
UDP 60458 - Xsan. Xsan Filesystem Access (Apple). As above

I don't see which of these were breaking Airplay support (unless it was related to ports 600x) and I'm reluctant to add a bunch of random ports that I do not understand. I'm curious to know what was using those random high ports.
Sure, any of the allowed services could be vulnerable but it's still fewer points of entry; I don't think that's a good argument for removing it. The fact that any additional services you might install will probably require you to mess around with all this isn't great from a usability standpoint. At least now having the ability to easily disable the whole thing makes debugging it easier; or one can actively choose to disable the firewall if they want to.
My v1.0 work uses ufw to handle the ipv6 side of things. I don't really want to have to mess around with that separately, maybe we should just turn it off? net.ipv6.conf.all.disable_ipv6 = 1 ?
I don't think we've ever looked into wpa_passphrase, that's interesting. I guess we could convert their plaintext password into a passphrase and then replace the wifi password value in settings.ini with that. Then we can keep the same logic. That sounds good.

That seems to be generic description of the ports, hard to tell if proprietary stuff really needs it, and if it is version related. also, could well be that other apple services try to connect to it, nowadays even os'es are chatty,...
my personal view on this is: it's a cool media box and I'd see it in private lan's which are mostly secured via NAT or firewall from outside, so it can have firewall disabled.
maybe you can implement a switch for privat/public use that triggers corresponding firewall rules.
if you need to secure the box, pls. think of a good security concept.
I'd suggest:

disable IPv6 , if not possible, force it to link-local only
run it as user and chroot environment (disable root)
secure any privacy and abusable information
no cleartext passwords,.. (3rd party account such as spotify,  wifi-password, etc,..) so, in case the box gets hijacked the information is useless

thanks for your effort and cooperation!

without airplay client connected:
root@MusicBox:~# netstat -tulpena
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode       PID/Program name
tcp        0      0 <IP_ADDRESS>:22              <IP_ADDRESS>:*               LISTEN      0          9865        2959/dropbear
tcp        0      0 <IP_ADDRESS>:445             <IP_ADDRESS>:*               LISTEN      0          9855        2946/smbd
tcp        0      0 <IP_ADDRESS>:5000            <IP_ADDRESS>:*               LISTEN      0          10124       2980/shairport-sync
tcp        0      0 <IP_ADDRESS>:139             <IP_ADDRESS>:*               LISTEN      0          9857        2946/smbd
tcp        0    368 <IP_ADDRESS>:22         <IP_ADDRESS>:55427      ESTABLISHED 0          10150       2996/dropbear
tcp6       0      0 :::22                   :::*                    LISTEN      0          9866        2959/dropbear
tcp6       0      0 :::445                  :::*                    LISTEN      0          9851        2946/smbd
tcp6       0      0 :::5000                 :::*                    LISTEN      0          10125       2980/shairport-sync
tcp6       0      0 :::139                  :::*                    LISTEN      0          9853        2946/smbd
udp        0      0 <IP_ADDRESS>:5353            <IP_ADDRESS>:*                           106        9039        2312/avahi-daemon:
udp        0      0 <IP_ADDRESS>:32808           <IP_ADDRESS>:*                           0          9680        2902/dhclient
udp        0      0 <IP_ADDRESS>:68              <IP_ADDRESS>:*                           0          9699        2902/dhclient
udp        0      0 <IP_ADDRESS>:38736           <IP_ADDRESS>:*                           106        9041        2312/avahi-daemon:
udp        0      0 <IP_ADDRESS>:123             <IP_ADDRESS>:*                           0          10363       3004/ntpdate
udp        0      0 <IP_ADDRESS>:137       <IP_ADDRESS>:*                           0          9794        2937/nmbd
udp        0      0 <IP_ADDRESS>:137        <IP_ADDRESS>:*                           0          9793        2937/nmbd
udp        0      0 <IP_ADDRESS>:137             <IP_ADDRESS>:*                           0          9790        2937/nmbd
udp        0      0 <IP_ADDRESS>:138       <IP_ADDRESS>:*                           0          9796        2937/nmbd
udp        0      0 <IP_ADDRESS>:138        <IP_ADDRESS>:*                           0          9795        2937/nmbd
udp        0      0 <IP_ADDRESS>:138             <IP_ADDRESS>:*                           0          9791        2937/nmbd
udp6       0      0 :::60854                :::*                                106        9042        2312/avahi-daemon:
udp6       0      0 :::5353                 :::*                                106        9040        2312/avahi-daemon:
udp6       0      0 :::14690                :::*                                0          9681        2902/dhclient
udp6       0      0 :::123                  :::*                                0          10364       3004/ntpdate

with airplay client connected:
root@MusicBox:~# netstat -tulpena
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode       PID/Program name
tcp        0      0 <IP_ADDRESS>:22              <IP_ADDRESS>:*               LISTEN      0          9865        2959/dropbear
tcp        0      0 <IP_ADDRESS>:6680            <IP_ADDRESS>:*               LISTEN      1001       11153       3055/python
tcp        0      0 <IP_ADDRESS>:445             <IP_ADDRESS>:*               LISTEN      0          9855        2946/smbd
tcp        0      0 <IP_ADDRESS>:49152      <IP_ADDRESS>:*               LISTEN      108        11165       2970/upmpdcli
tcp        0      0 <IP_ADDRESS>:5000            <IP_ADDRESS>:*               LISTEN      0          10124       2980/shairport-sync
tcp        0      0 <IP_ADDRESS>:139             <IP_ADDRESS>:*               LISTEN      0          9857        2946/smbd
tcp        0    780 <IP_ADDRESS>:43124      <IP_ADDRESS>:4070     ESTABLISHED 1001       11114       3055/python
tcp        0    268 <IP_ADDRESS>:22         <IP_ADDRESS>:55427      ESTABLISHED 0          10150       2996/dropbear
tcp        0      0 <IP_ADDRESS>:46984         <IP_ADDRESS>:6600          ESTABLISHED 108        11160       2970/upmpdcli
tcp        0      0 <IP_ADDRESS>:5000       <IP_ADDRESS>:55521      ESTABLISHED 0          10961       2980/shairport-sync
tcp        0      0 <IP_ADDRESS>:47812      <IP_ADDRESS>:80         ESTABLISHED 1001       11117       3055/python
tcp6       0      0 :::22                   :::*                    LISTEN      0          9866        2959/dropbear
tcp6       0      0 :::445                  :::*                    LISTEN      0          9851        2946/smbd
tcp6       0      0 :::49152                :::*                    LISTEN      108        11166       2970/upmpdcli
tcp6       0      0 <IP_ADDRESS>:6600            :::*                    LISTEN      1001       11149       3055/python
tcp6       0      0 :::5000                 :::*                    LISTEN      0          10125       2980/shairport-sync
tcp6       0      0 :::139                  :::*                    LISTEN      0          9853        2946/smbd
tcp6       0      0 <IP_ADDRESS>:6600          <IP_ADDRESS>:46984         ESTABLISHED 1001       11161       3055/python
udp        0      0 <IP_ADDRESS>:5353            <IP_ADDRESS>:*                           106        9039        2312/avahi-daemon:
udp        0      0 <IP_ADDRESS>:32808           <IP_ADDRESS>:*                           0          9680        2902/dhclient
udp        0      0 <IP_ADDRESS>:68              <IP_ADDRESS>:*                           0          9699        2902/dhclient
udp        0      0 <IP_ADDRESS>:38736           <IP_ADDRESS>:*                           106        9041        2312/avahi-daemon:
udp        0      0 <IP_ADDRESS>:1900            <IP_ADDRESS>:*                           108        11169       2970/upmpdcli
udp        0      0 <IP_ADDRESS>:6001       <IP_ADDRESS>:*                           0          10964       2980/shairport-sync
udp        0      0 <IP_ADDRESS>:6002       <IP_ADDRESS>:*                           0          10965       2980/shairport-sync
udp        0      0 <IP_ADDRESS>:6003       <IP_ADDRESS>:*                           0          10966       2980/shairport-sync
udp        0      0 <IP_ADDRESS>:56199         <IP_ADDRESS>:*                           108        11167       2970/upmpdcli
udp        0      0 <IP_ADDRESS>:137       <IP_ADDRESS>:*                           0          9794        2937/nmbd
udp        0      0 <IP_ADDRESS>:137        <IP_ADDRESS>:*                           0          9793        2937/nmbd
udp        0      0 <IP_ADDRESS>:137             <IP_ADDRESS>:*                           0          9790        2937/nmbd
udp        0      0 <IP_ADDRESS>:138       <IP_ADDRESS>:*                           0          9796        2937/nmbd
udp        0      0 <IP_ADDRESS>:138        <IP_ADDRESS>:*                           0          9795        2937/nmbd
udp        0      0 <IP_ADDRESS>:138             <IP_ADDRESS>:*                           0          9791        2937/nmbd
udp6       0      0 :::60854                :::*                                106        9042        2312/avahi-daemon:
udp6       0      0 :::5353                 :::*                                106        9040        2312/avahi-daemon:
udp6       0      0 :::14690                :::*                                0          9681        2902/dhclient

install lsof:
apt-get install lsof
list all network ports:
root@MusicBox:~# lsof -i
COMMAND    PID     USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
avahi-dae 2312    avahi   13u  IPv4   9039      0t0  UDP *:mdns
avahi-dae 2312    avahi   14u  IPv6   9040      0t0  UDP *:mdns
avahi-dae 2312    avahi   15u  IPv4   9041      0t0  UDP *:38736
avahi-dae 2312    avahi   16u  IPv6   9042      0t0  UDP *:60854
dhclient  2902     root    7u  IPv4   9699      0t0  UDP *:bootpc
dhclient  2902     root   20u  IPv4   9680      0t0  UDP *:32808
dhclient  2902     root   21u  IPv6   9681      0t0  UDP *:14690
nmbd      2937     root    9u  IPv4   9790      0t0  UDP *:netbios-ns
nmbd      2937     root   10u  IPv4   9791      0t0  UDP *:netbios-dgm
nmbd      2937     root   11u  IPv4   9793      0t0  UDP MusicBox.local:netbios-ns
nmbd      2937     root   12u  IPv4   9794      0t0  UDP <IP_ADDRESS>:netbios-ns
nmbd      2937     root   13u  IPv4   9795      0t0  UDP MusicBox.local:netbios-dgm
nmbd      2937     root   14u  IPv4   9796      0t0  UDP <IP_ADDRESS>:netbios-dgm
smbd      2946     root   27u  IPv6   9851      0t0  TCP *:microsoft-ds (LISTEN)
smbd      2946     root   28u  IPv6   9853      0t0  TCP *:netbios-ssn (LISTEN)
smbd      2946     root   29u  IPv4   9855      0t0  TCP *:microsoft-ds (LISTEN)
smbd      2946     root   30u  IPv4   9857      0t0  TCP *:netbios-ssn (LISTEN)
dropbear  2959     root    3u  IPv4   9865      0t0  TCP *:ssh (LISTEN)
dropbear  2959     root    4u  IPv6   9866      0t0  TCP *:ssh (LISTEN)
upmpdcli  2970 upmpdcli    5u  IPv4  11160      0t0  TCP localhost:46984->localhost:6600 (ESTABLISHED)
upmpdcli  2970 upmpdcli    6u  IPv4  11165      0t0  TCP MusicBox.local:49152 (LISTEN)
upmpdcli  2970 upmpdcli    7u  IPv6  11166      0t0  TCP *:49152 (LISTEN)
upmpdcli  2970 upmpdcli    8u  IPv4  11167      0t0  UDP localhost:56199
upmpdcli  2970 upmpdcli   10u  IPv4  11169      0t0  UDP *:1900
shairport 2980     root    4u  IPv4  10124      0t0  TCP *:5000 (LISTEN)
shairport 2980     root    5u  IPv6  10125      0t0  TCP *:5000 (LISTEN)
shairport 2980     root    9u  IPv4  10961      0t0  TCP MusicBox.local:5000->airplayclient.local:55521 (ESTABLISHED)
shairport 2980     root   10u  IPv4  10964      0t0  UDP MusicBox.local:x11-1
shairport 2980     root   11u  IPv4  10965      0t0  UDP MusicBox.local:x11-2
shairport 2980     root   12u  IPv4  10966      0t0  UDP MusicBox.local:x11-3
dropbear  2996     root    6u  IPv4  10150      0t0  TCP MusicBox.local:ssh->airplayclient.local:55427 (ESTABLISHED)
mopidy    3055   mopidy   14u  IPv4  11114      0t0  TCP MusicBox.local:43124->lon2-accesspoint-a13.lon2.spotify.com:4070 (ESTABLISHED)
mopidy    3055   mopidy   15u  IPv4  11117      0t0  TCP MusicBox.local:47812->a2-16-194-217.deploy.akamaitechnologies.com:http (ESTABLISHED)
mopidy    3055   mopidy   17u  IPv6  11149      0t0  TCP *:6600 (LISTEN)
mopidy    3055   mopidy   18u  IPv4  11153      0t0  TCP *:6680 (LISTEN)
mopidy    3055   mopidy   26u  IPv6  11161      0t0  TCP localhost:6600->localhost:46984 (ESTABLISHED)

From all those it looks like just the following are wrong/missing:

UDP 1900 (upmpdcli)
UDP 67 & 68 (dhclient)

Which still doesn't explain what was breaking shairport-sync! I've spent some time looking through the shairport-sync code and it seems pretty clear that it's only 5000 TCP and then 3 per client (starting from UDP 6001) that are used. Avahi is already open (UDP 5353) but I don't know what avahi is doing with 38736. So I'm really confused, and I'm leaning towards having the firewall disabled by default. Raspbian doesn't include one, presumably because it just gets in the way more than helps.  Life is too short!


[azure] New azure pack

It comes with an initial set of compute actions. In the future I will also look into adding some object storage actions.
+------------------+-------+------------+----------------------+
| ref              | pack  | name       | description          |
+------------------+-------+------------+----------------------+
| azure.create_vm  | azure | create_vm  | Create a new VM.     |
| azure.destroy_vm | azure | destroy_vm | Destroy a VM.        |
| azure.list_vms   | azure | list_vms   | List available VMs.  |
| azure.reboot_vm  | azure | reboot_vm  | Reboot a running VM. |
+------------------+-------+------------+----------------------+

Also added some storage actions and updated the description.


[Issue] no kind "CronTab" is registered for version "stable.example.com/v1"

I made a copy from cnat to crontab, below is its definition of CRD
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  # name must match the spec fields below, and be in the form: <plural>.<group>
  name: crontabs.stable.example.com
spec:
  # group name to use for REST API: /apis/<group>/<version>
  group: stable.example.com
  # list of versions supported by this CustomResourceDefinition
  versions:
    - name: v1
      # Each version can be enabled/disabled by Served flag.
      served: true
      # One and only one version must be marked as the storage version.
      storage: true
      additionalPrinterColumns:
      - name: schedule
        type: string
        jsonPath: .spec.cronSpec
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                cronSpec:
                  type: string
                image:
                  type: string
                replicas:
                  type: integer
  # either Namespaced or Cluster
  scope: Namespaced
  names:
    # plural name to be used in the URL: /apis/<group>/<version>/<plural>
    plural: crontabs
    # singular name to be used as an alias on the CLI and for display
    singular: crontab
    # kind is normally the CamelCased singular type. Your resource manifests use this.
    kind: CronTab
    # shortNames allow shorter string to match your resource on the CLI
    shortNames:
    - ct

the code of types.go
package v1

import (
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

type CrontabSpecV1 struct {
	CronSpec string `json:"cronSpec"`
	Image    string `json:"image"`
}

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

type Crontab struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec CrontabSpecV1 `json:"spec,omitempty"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

type CrontabList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`

	Items []Crontab `json:"items"`
}

and the code of register.go
// Define your schema name and the version
var SchemeGroupVersion = schema.GroupVersion{
	Group:   "stable.example.com",
	Version: "v1",
}

var (
	SchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes)
	AddToScheme   = SchemeBuilder.AddToScheme
)

// Resource takes an unqualified resource and returns a Group qualified GroupResource
func Resource(resource string) schema.GroupResource {
	return SchemeGroupVersion.WithResource(resource).GroupResource()
}

func Kind(kind string) schema.GroupKind {
	return SchemeGroupVersion.WithKind(kind).GroupKind()
}

// Adds the list of known types to the given scheme.
func addKnownTypes(scheme *runtime.Scheme) error {
	scheme.AddKnownTypes(SchemeGroupVersion,
		&CrontabList{},
		&Crontab{},
	)
	metav1.AddToGroupVersion(scheme, SchemeGroupVersion)
	return nil
}

after the commandgenerate-groups.sh "deepcopy,client,informer,lister" be executed, following code raise an error
	client, e := versioned.NewForConfig(config)
	if e != nil {
		panic(e.Error())
	}
	crontab, err := client.StableV1().Crontabs("default").Get(context.TODO(), "cron-1", metav1.GetOptions{})

the error message was
no kind "CronTab" is registered for version "stable.example.com/v1" in scheme "pkg/crd/typedClient/client/clientset/versioned/scheme/register.go:15"

however the kind CronTab has already been registered in register.go, not sure why it happened. is there anyone who can help on that? thanks.

Thanks for raising this @yang-wang11 and I'm unsure I follow what exactly your steps were. You mentioned generate-groups.sh (I assume you're referring to https://github.com/kubernetes/code-generator?) Maybe @sttts has a better view?

@mhausenblas  @sttts  yes, I follow the book step by step(chapter 4-5). I did the following changes:

upgrade to v1 instead of v1alpha1
remove the sub-resource status

you may see below code generated from generate-groups.sh, it seems to look good.
var Scheme = runtime.NewScheme()
var Codecs = serializer.NewCodecFactory(Scheme)
var ParameterCodec = runtime.NewParameterCodec(Scheme)
var localSchemeBuilder = runtime.SchemeBuilder{
	stablev1.AddToScheme,
}
var AddToScheme = localSchemeBuilder.AddToScheme

func init() {
	v1.AddToGroupVersion(Scheme, schema.GroupVersion{Version: "v1"})
	utilruntime.Must(AddToScheme(Scheme))
}

I tried to force execute its init function. the error does seem not to disappear.
_ "typedClient/client/clientset/versioned/scheme"

Note the different spelling: CronTab vs. Crontab. The scheme has the latter, the clients wants the former.

no kind "CronTab" is registered for version "stable.example.com/v1" in scheme "pkg/crd/typedClient/client/clientset/versioned/scheme/register.go:15"

Is this a copy-and-pasted error message? Am not sure where client-gen gets the CronTab type from. It is based on the Go type, not the CRD yaml. Hence, it should print a "Crontab" error, not "CronTab" 🤔

@sttts thanks for the help.
I'm a little confused, is that mean I need to rename my go type from Crontab to CronTab?  like below
func addKnownTypes(scheme *runtime.Scheme) error {
	scheme.AddKnownTypes(SchemeGroupVersion,
		&CronTabList{},
		&CronTab{},
	)
	metav1.AddToGroupVersion(scheme, SchemeGroupVersion)
	return nil
}

If you add a type to a scheme, its Go type name is used as kind (via reflection).
Now I also understand what is going on. You are getting a CronTab from the server and the client tries to find a CronTab in its scheme. It only finds a Crontab, which is not helpful. So the server returns an unexpected object. Those two have to match.
Where Crontab is added to the scheme in register.go, you can give a custom name with the capital T. Then it will work too.

no kind "CronTab" is registered for version "stable.example.com/v1" in scheme "pkg/crd/typedClient/client/clientset/versioned/scheme/register.go:15"

Is this a copy-and-pasted error message? Am not sure where client-gen gets the CronTab type from. It is based on the Go type, not the CRD yaml. Hence, it should print a "Crontab" error, not "CronTab" 🤔

yes, this error message copy from the console.

@sttts  @mhausenblas thanks for the help. it works.


beginSegment() should perform sampling if the caller haven't done any sampling decision

Problem
The beginSegment() method doesn't perform sampling if the caller of this method hasn't done any sampling decision.
Background
Currently sampling for a given incoming request is performed by shouldTrace(). By default, this method starts pollers in the background to fetch sampling rules from the X-Ray console and sampling is performed based on fetched sampling rules.
Usecase
Currently, shouldTrace() is only called by AWSXRayServerFilter middleware. In the scenario where a user doesn't want to use AWSXRayServletFilter middleware and instrument application using just beginSegment() and endSegment(), no sampling would be performed. This is due to limitation of beginSegment not performing sampling, if the caller of this method has not performed sampling decision. As a result, all the segments are sampled and sampling rules configured through X-Ray console doesn't have any effect.
Proposed solution
beginSegment should do sampling if the caller hasn't done any sampling.
Reference
AWS X-Ray SDK for .NET/Core has necessary implementation.
Note :

This should be backwards compatible change.

Added new API for sampling.
https://github.com/aws/aws-xray-sdk-java/pull/196


Remove text box behaviour just have a styled select?

Hi just wondering if this is possible using any of these options as I want to not using the default select input arrow and need a custom style. selectize looks great but can you remove the text input feature and just have it clickable like a button/normal select?

You can use search-enabled=false to get that effect.


Added deploy command for docker

There's an official docker image of apache airflow on docker hub, can we try using that?
Also, a gif with demo?

Let's create a contributing guide as well, would be easier for code reviewers as well to understand the code.

Better start writing unit tests sooner, would be difficult to add them later and might need to restructure code to make it testable.

There's an official docker image of apache airflow on docker hub, can we try using that?
Also, a gif with demo?

This is just for testing purpose. @sunasra will be creating a docker-compose file that'll be actually used.
Yes, sure will attach a gif.

Let's create a contributing guide as well, would be easier for code reviewers as well to understand the code.

Will start creating it.

Better start writing unit tests sooner, would be difficult to add them later and might need to restructure code to make it testable.

Next few PRs will of tests. Will be setting up the pipeline now since this is blocked till git integration.

@chattarajoy
Deploy on Qubole via analyze coming soon.

There's an official docker image of apache airflow on docker hub, can we try using that?
Also, a gif with demo?

That docker image is used by airflow opensource team to run the test I guess. It also installs Hadoop and some other packages which doen't require to run airflow. I am referring https://hub.docker.com/r/apache/airflow
Let me know if any other official image is available.


PyPi release for 0.3.1?

Hi there, will there be a 2021 0.3.X release to pypi ?  many thanks!

This seems to be done, right?


Scripts that load the current values from isobus.net

Today I had the idea of a script that automatically loads all current values (Manufacturer Codes, PGNs, DDIs, Device Class / Functions) from isobus.net during the build (within the build.rs) and thus always provides this information completely and up-to-date by generating the respective enums from a code generator. I know that this is a lot of work that is certainly not relevant at the moment, but I wanted to keep this idea for a later date.

The C++ stack already does this for DDI's at least, so maybe it can be inspiration: https://github.com/Open-Agriculture/AgIsoStack-plus-plus/blob/main/scripts/code_gen_ddi_database.py

Oh that's neat, I hadn't even seen it yet. But I think I have to go a different way except for the DDIs, as there are apparently only files for the DDIs.


chore: add gleam lang lsp

gleam is missing from filetype and server mapping.

@williambotman /fixup