Enhancement: Rust, C or C++ bindings for path resolution

Starlite currently resolves paths using a tree structure, but in python. This is quite slow. It would be very helpful if we could create custom bindings in a low level and substantially faster language to implement the tree and its required parsing methods.
The data structure that is often used for this purpose is called a "radix prefix tree". This is not necessarily the only option we have - but its a strong one.
PRs are welcome, as are discussions.

Some links to the existing code in starlite would be helpful to start.
Also, have you considered/tried compiling starlite/parts of it with mypyc?

Also, have you considered/tried compiling starlite/parts of it with mypyc?

100% this. I'd love to see what the solution looks like in modern cython (i say modern because the last time I built anything of consequence with cython was before typing was really a thing) and mypyc.
Thinking about this issue on the next meta level up, an overall efficiency analysis of the two main things Starlite does is something that I'm keen to do. From my understanding so far, these two main things are:

serve requests, sounds obvious, the route from request entry to response return would benefit from some diagrams and docs, and even the smallest of optimizations along that path are +1 from me.
the handler -> controller -> router -> app walk on startup - the whole library is pretty much built on this and so it should be pretty optimal I think. I reckon it also might be neat to make this walk a feature of library startup and let users register hooks onto it. For example, I might want to register a hook onto the walk that logs the number of routes that I have nested under each router/controller and how many of them are unauthenticated and authenticated. Just an example, but it would be a neat and I think unique feature that we could promote.

Don't get me wrong, I'd also like to see what the solution looks like with py03 and rust too though :laughing:
It would be good for developer docs to have a diagram of that "radix prefix tree" pattern included. I'm glad you mentioned that @Goldziher as the code is a bit hard to follow (not a criticism but just by nature) - is that the algo in StarliteASGIRouter.__call__()?

I did some trials with cython and mypyc. Mypyc Didn't like async and some other stuff, so it could handle some parts of the codebase and not others. Cython required quite a few changes to the code to get the compiled code to actually work.
I'm not against this, but for money these solutions are a compromise - offering a performance boost, but less so than native code.
As to code snippets, diagrams etc. Sure, I'll do this later today.

As to code snippets, diagrams etc. Sure, I'll do this later today.

Thanks!

I did some trials with cython and mypyc. Mypyc Didn't like async and some other stuff, so it could handle some parts of the codebase and not others. Cython required quite a few changes to the code to get the compiled code to actually work.

Was this trying to cythonize/mypycize the whole lib? Or just targeted stuff?

I did some trials with cython and mypyc. Mypyc Didn't like async and some other stuff, so it could handle some parts of the codebase and not others. Cython required quite a few changes to the code to get the compiled code to actually work.

Was this trying to cythonize/mypycize the whole lib? Or just targeted stuff?

Both.
I had more success with cython, but the performance boost wasn't great. The main issue for compiling the entire lib is pydantic - pydantic has cython, so if you install cython as part of your dependencies it will compile, giving you a speed boost. But any code that uses pydantic cannot be compiled itself.

So, lets start with code. The most updated version is the one in #173, so I will explain the code there below, which will also help by and by with the review of that PR.
There are two files that interest here - starlite/app,py and starlite/asgi.py. Lets start with whats happening inside the Starlite class:
class Starlite(Router):
    # quite a bit of code....
    def construct_route_map(self) -> None:
        """
        Create a map of the app's routes. This map is used in the asgi router to route requests.

        """
        if "_components" not in self.route_map:
            self.route_map["_components"] = set()
        for route in self.routes:
            path = route.path
            if route.path_parameters or path in self.static_paths:
                for param_definition in route.path_parameters:
                    path = path.replace(param_definition["full"], "")
                path = path.replace("{}", "*")
                cur = self.route_map
                components = ["/", *[component for component in path.split("/") if component]]
                for component in components:
                    components_set = cast(Set[str], cur["_components"])
                    components_set.add(component)
                    if component not in cur:
                        cur[component] = {"_components": set()}
                    cur = cast(Dict[str, Any], cur[component])
            else:
                self.route_map[path] = {"_components": set()}
                self.plain_routes.add(path)
                cur = self.route_map[path]
            if "_path_parameters" not in cur:
                cur["_path_parameters"] = route.path_parameters
            if "_asgi_handlers" not in cur:
                cur["_asgi_handlers"] = {}
            if "_is_asgi" not in cur:
                cur["_is_asgi"] = False
            if path in self.static_paths:
                cur["static_path"] = path
                cur["_is_asgi"] = True
            asgi_handlers = cast(Dict[str, ASGIApp], cur["_asgi_handlers"])
            if isinstance(route, HTTPRoute):
                for method, handler_mapping in route.route_handler_map.items():
                    handler, _ = handler_mapping
                    asgi_handlers[method] = self.build_route_middleware_stack(route, handler)
            elif isinstance(route, WebSocketRoute):
                asgi_handlers["websocket"] = self.build_route_middleware_stack(route, route.route_handler)
            elif isinstance(route, ASGIRoute):
                asgi_handlers["asgi"] = self.build_route_middleware_stack(route, route.route_handler)
                cur["_is_asgi"] = True

    def build_route_middleware_stack(
        self, route: Union[HTTPRoute, WebSocketRoute, ASGIRoute], route_handler: BaseRouteHandler
    ) -> ASGIApp:
        """Constructs a middleware stack that serves as the point of entry for each route"""
        asgi_handler: ASGIApp = route.handle
        for middleware in route_handler.resolve_middleware():
            if isinstance(middleware, StarletteMiddleware):
                asgi_handler = middleware.cls(app=asgi_handler, **middleware.options)
            else:
                asgi_handler = middleware(app=asgi_handler)
        if self.allowed_hosts:
            asgi_handler = TrustedHostMiddleware(app=asgi_handler, allowed_hosts=self.allowed_hosts)
        if self.cors_config:
            asgi_handler = CORSMiddleware(app=asgi_handler, **self.cors_config.dict())
        return asgi_handler

    def register(self, value: ControllerRouterHandler) -> None:  # type: ignore[override]
        """
        Register a Controller, Route instance or RouteHandler on the app.

        Calls Router.register() and then creates a signature model for all handlers.
        """
        routes = super().register(value=value)
        for route in routes:
            if isinstance(route, HTTPRoute):
                route_handlers = route.route_handlers
            else:
                route_handlers = [cast(Union[WebSocketRoute, ASGIRoute], route).route_handler]  # type: ignore
            for route_handler in route_handlers:
                self.create_handler_signature_model(route_handler=route_handler)
                route_handler.resolve_guards()
                route_handler.resolve_middleware()
                if isinstance(route_handler, HTTPRouteHandler):
                    route_handler.resolve_response_class()
                    route_handler.resolve_before_request()
                    route_handler.resolve_after_request()
            if isinstance(route, HTTPRoute):
                route.create_handler_map()
            elif isinstance(route, WebSocketRoute):
                route.handler_parameter_model = route.create_handler_kwargs_model(route.route_handler)
        self.construct_route_map()

So, lets start with the register method. This method is called in the init method for the application via a super call to the underlying Router class (Starlite router, not Starlette!). It basically iterates through all the routers, controllers and route handlers, processes them and in the end calls self.construct_route_map().
Note: This method is exposed to allow dynamic registeration of routes - even after app initialization.
The construct_route_map method is where the Starlite routing logic is created. Its very different from whats happening in Starlette and other frameworks (read Django, FastAPI etc.), because these frameworks rely on regex matching. What they do is basically iterate through all the registered routes, trying to match them with a regex, and once there is a success or partial success, continue. The problem with this approach, which is btw also the same in the popular NodeJS framework Express and other popular non-python frameworks, is that it doesn't scale well. The more routes you have, the longer matching can take. In other words - time complexity is not good. The one advantage this method does have though is that the python regex engine is implemented in C, so its quite fast in comparison to native python code.
What does Starlite do? basically we create a "tree" structure, or whats called trie. I decided to implement this using python dictionaries and combine it with sets, because of the particulars of how python implements these - dictionary key lookup is fast because it uses sets behind the scene, and sets are fast because they are implemented natively, with set lookup being exceptionally fast in python (respective of python that is).
How does the code look like?
def construct_route_map(self) -> None:
        """
        Create a map of the app's routes. This map is used in the asgi router to route requests.

        """
        if "_components" not in self.route_map:
            self.route_map["_components"] = set()
        for route in self.routes:
            path = route.path
            if route.path_parameters or path in self.static_paths:
                for param_definition in route.path_parameters:
                    path = path.replace(param_definition["full"], "")
                path = path.replace("{}", "*")
                cur = self.route_map
                components = ["/", *[component for component in path.split("/") if component]]
                for component in components:
                    components_set = cast(Set[str], cur["_components"])
                    components_set.add(component)
                    if component not in cur:
                        cur[component] = {"_components": set()}
                    cur = cast(Dict[str, Any], cur[component])
            else:
                self.route_map[path] = {"_components": set()}
                self.plain_routes.add(path)
                cur = self.route_map[path]
            if "_path_parameters" not in cur:
                cur["_path_parameters"] = route.path_parameters
            if "_asgi_handlers" not in cur:
                cur["_asgi_handlers"] = {}
            if "_is_asgi" not in cur:
                cur["_is_asgi"] = False
            if path in self.static_paths:
                cur["static_path"] = path
                cur["_is_asgi"] = True
            asgi_handlers = cast(Dict[str, ASGIApp], cur["_asgi_handlers"])
            if isinstance(route, HTTPRoute):
                for method, handler_mapping in route.route_handler_map.items():
                    handler, _ = handler_mapping
                    asgi_handlers[method] = self.build_route_middleware_stack(route, handler)
            elif isinstance(route, WebSocketRoute):
                asgi_handlers["websocket"] = self.build_route_middleware_stack(route, route.route_handler)
            elif isinstance(route, ASGIRoute):
                asgi_handlers["asgi"] = self.build_route_middleware_stack(route, route.route_handler)
                cur["_is_asgi"] = True

The above code creates a tree structure with each level of the tree being identified by a path component and each node containing data about its children as well as any routes belonging to this level.
Lets consider an example from the tests:
def test_handler_multi_paths() -> None:
    @get(path=["/", "/something", "/{some_id:int}", "/something/{some_id:int}"], media_type=MediaType.TEXT)
    def handler_fn(some_id: int = 1) -> str:
        assert some_id
        return str(some_id)
    with create_test_client(handler_fn) as client:
       # ....

If we inspect in the debugger the kind of mapping that is created above, we see this:
{
   '_components':{
      '/'
   },
   '/':{
      '_components':{
         'something',
         '*'
      },
      '_path_parameters':[

      ],
      '_asgi_handlers':{
         'GET':<bound method HTTPRoute.handle of <starlite.routes.HTTPRoute object at 0x106592ea0>>
      },
      '_is_asgi':False,
      '*':{
         '_components':set(),
         '_path_parameters':[
            {
               'name':'some_id',
               'type':<class'int'>,
               'full':'some_id:int'
            }
         ],
         '_asgi_handlers':{
            'GET':<bound method HTTPRoute.handle of <starlite.routes.HTTPRoute object at 0x106593760>>
         },
         '_is_asgi':False
      },
      'something':{
         '_components':{
            '*'
         },
         '*':{
            '_components':set(),
            '_path_parameters':[
               {
                  'name':'some_id',
                  'type':<class'int'>,
                  'full':'some_id:int'
               }
            ],
            '_asgi_handlers':{
               'GET':<bound method HTTPRoute.handle of <starlite.routes.HTTPRoute object at 0x106593a00>>
            },
            '_is_asgi':False
         }
      }
   },
   '/something':{
      '_components':set(),
      '_path_parameters':[

      ],
      '_asgi_handlers':{
         'GET':<bound method HTTPRoute.handle of <starlite.routes.HTTPRoute object at 0x1065937d0>>
      },
      '_is_asgi':False
   }
}

This brings us to the last method in the Starlite app that is of interest here - the method called build_middleware_stack, that is called as part of the mapping logic above - for each route handler:
    def build_route_middleware_stack(
        self, route: Union[HTTPRoute, WebSocketRoute, ASGIRoute], route_handler: BaseRouteHandler
    ) -> ASGIApp:
        """Constructs a middleware stack that serves as the point of entry for each route"""
        asgi_handler: ASGIApp = route.handle
        for middleware in route_handler.resolve_middleware():
            if isinstance(middleware, StarletteMiddleware):
                asgi_handler = middleware.cls(app=asgi_handler, **middleware.options)
            else:
                asgi_handler = middleware(app=asgi_handler)
        if self.allowed_hosts:
            asgi_handler = TrustedHostMiddleware(app=asgi_handler, allowed_hosts=self.allowed_hosts)
        if self.cors_config:
            asgi_handler = CORSMiddleware(app=asgi_handler, **self.cors_config.dict())
        return asgi_handler

This method creates the ASGIApp for each individual combination of route + route handling function, with the last component in the middleware stack to be called being the route.handle method and the first being the CORSMiddleware if existing. This is stored under _asgi_handlers with the following type of keys GET, POST, PUT etc. and asgi / websocket.
This now brings us to how these paths are resolved. This happens in the ASGIRouter class's __call__ method, which is called by the Starlite app for each asgi request. This is how its called from the Starlite class:
    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        scope["app"] = self
        if scope["type"] == "lifespan":
            await self.asgi_router.lifespan(scope, receive, send)
            return
        try:
            scope["state"] = {}
            await self.asgi_router(scope, receive, send) # <-- this is the call
        except Exception as e:  # pylint: disable=broad-except
            await self.handle_exception(scope=scope, receive=receive, send=send, exc=e)

And this is whats happening there:
    def parse_scope_to_route(self, scope: Scope) -> Dict[str, Any]:
        """
        Given a scope object, traverse the route mapping and retrieve the correct "leaf" in the route tree.
        """
        path_params: List[str] = []
        path = cast(str, scope["path"]).strip()
        if path != "/" and path.endswith("/"):
            path = path.rstrip("/")
        if path in self.app.plain_routes:
            cur: Dict[str, Any] = self.app.route_map[path]
        else:
            cur = self.app.route_map
            components = ["/", *[component for component in path.split("/") if component]]
            for component in components:
                components_set = cast(Set[str], cur["_components"])
                if component in components_set:
                    cur = cast(Dict[str, Any], cur[component])
                    continue
                if "*" in components_set:
                    path_params.append(component)
                    cur = cast(Dict[str, Any], cur["*"])
                    continue
                if cur.get("static_path"):
                    static_path = cast(str, cur["static_path"])
                    if static_path != "/":
                        scope["path"] = scope["path"].replace(static_path, "")
                    continue
                raise NotFoundException()
        scope["path_params"] = (
            parse_path_params(cur["_path_parameters"], path_params) if cur["_path_parameters"] else {}
        )
        return cur

    @staticmethod
    def resolve_asgi_app(scope: Scope, route_mapping: Dict[str, Any]) -> ASGIApp:
        """
        Given a scope, retrieves the correct ASGI App for the route
        """
        asgi_handlers = cast(Dict[str, ASGIApp], route_mapping["_asgi_handlers"])
        is_asgi = cast(bool, route_mapping["_is_asgi"])
        if is_asgi:
            return asgi_handlers[ScopeType.ASGI]
        if scope["type"] == ScopeType.HTTP:
            if scope["method"] not in asgi_handlers:
                raise MethodNotAllowedException()
            return asgi_handlers[scope["method"]]
        return asgi_handlers[ScopeType.WEBSOCKET]

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        """
        The main entry point to the Router class.
        """
        try:
            route_mapping = self.parse_scope_to_route(scope=scope)
            asgi_handler = self.resolve_asgi_app(scope=scope, route_mapping=route_mapping)
        except KeyError as e:
            raise NotFoundException() from e
        await asgi_handler(scope, receive, send)

As you can see, the call method first calls parse_scope_to_route passing to it the ASGI scope. This method in turn. does the following (I will annotate the code):
 def parse_scope_to_route(self, scope: Scope) -> Dict[str, Any]:
        """
        Given a scope object, traverse the route mapping and retrieve the correct "leaf" in the route tree.
        """
        path_params: List[str] = [] # stored path params, we append them as we find them when traversing
        path = cast(str, scope["path"]).strip()
        if path != "/" and path.endswith("/"): # normalize the path - we do not care for trailing slashes
            path = path.rstrip("/")
         # the path is a "plain_route", meaning it has no path params, e.g. "/companies"
         # we therefore do not need to traverse the map and can simply use the normalized path as a key
        if path in self.app.plain_routes:
            cur: Dict[str, Any] = self.app.route_map[path]
        else: # we got here, which means the path we got is parameterized
            # we traversing the map via looping, each time we move a level, cur is going to be updated
            cur = self.app.route_map
            components = ["/", *[component for component in path.split("/") if component]]
            # for "/foo/123/buzz" we will have here ["/", "foo", "123", "buzz"]
            for component in components:
                # we are taking the component set we have, because its the fastest lookup
                components_set = cast(Set[str], cur["_components"])
                # if the component we have here is not a parameter, it should match, e.g. ["/", "foo", "buzz"]
                if component in components_set:
                   # update cur to the next level
                    cur = cast(Dict[str, Any], cur[component])
                    continue
                # we have an "*" in the component_set, we therefore allow for a parameter
                if "*" in components_set:
                    # we got to this point with the value "123", and we appent it to the path components
                    path_params.append(component)
                    # update cur to the next level
                    cur = cast(Dict[str, Any], cur["*"])
                    continue
                # special case for static paths
                if cur.get("static_path"):
                    static_path = cast(str, cur["static_path"])
                    if static_path != "/":
                        scope["path"] = scope["path"].replace(static_path, "")
                    continue
                # no matches, raise 404
                raise NotFoundException()
        # we use a helper function to transform url params, which are always strings, into their values.
        scope["path_params"] = (
            parse_path_params(cur["_path_parameters"], path_params) if cur["_path_parameters"] else {}
        )
        return cur

Once the map has been traversed to the correct point - and if no exception has been raised, the tree node or leaf is returned - this is actually an object of this type:
{
   '_components':{
      'something',
      '*'
   },
   '_path_parameters':[

   ],
   '_asgi_handlers':{
      'GET':<bound method HTTPRoute.handle of <starlite.routes.HTTPRoute object at 0x106592ea0>>
   },
   '_is_asgi':False,
   '*':{
      '_components':set(),
      '_path_parameters':[
         {
            'name':'some_id',
            'type':<class'int'>,
            'full':'some_id:int'
         }
      ],
      '_asgi_handlers':{
         'GET':<bound method HTTPRoute.handle of <starlite.routes.HTTPRoute object at 0x106593760>>
      },
      '_is_asgi':False
   },
   'something':{
      '_components':{
         '*'
      },
      '*':{
         '_components':set(),
         '_path_parameters':[
            {
               'name':'some_id',
               'type':<class'int'>,
               'full':'some_id:int'
            }
         ],
         '_asgi_handlers':{
            'GET':<bound method HTTPRoute.handle of <starlite.routes.HTTPRoute object at 0x106593a00>>
         },
         '_is_asgi':False
      }
   }
}

Additionally, the resolved path params have been injected into the scope object, which will allow them to be used later when we need to parse kwargs for the handling function and dependencies.
Finally, the actual handler is resolved in the method resolve_asgi_app:
    def resolve_asgi_app(scope: Scope, route_mapping: Dict[str, Any]) -> ASGIApp:
        """
        Given a scope, retrieves the correct ASGI App for the route
        """
        asgi_handlers = cast(Dict[str, ASGIApp], route_mapping["_asgi_handlers"])
        is_asgi = cast(bool, route_mapping["_is_asgi"])
        if is_asgi:
            return asgi_handlers[ScopeType.ASGI]
        if scope["type"] == ScopeType.HTTP:
            if scope["method"] not in asgi_handlers:
                raise MethodNotAllowedException()
            return asgi_handlers[scope["method"]]
        return asgi_handlers[ScopeType.WEBSOCKET]

This method simply uses data in the scope and the node that have been passed to it to determine the correct handler key to retrieve - i.e. if its an @asgi function, it will use the ScopeType.ASGI, if its an http route it will use the method from scope (e.g. GET) and otherwise it will default to WebSocket. If a key error is raised, this is translated into a 404.
Summary
So as you can see above route resolution happens using a tree structure. The advantage of this approach is that route handlers are grouped in branches, with shared path components serving as shared prefixes.
I tried to make this as efficent as i knew how - by using optimized python constructs on the one hand, and by processing and caching during application bootstrap. Still, the traversal of the paths is limited by the runtime speed of python. This can probably be enhanced using a native binding, but I cannot say how much etc.

Wow @Goldziher - that's comprehensive! I'll take the time to review when I have some time and if I have any questions I'll ping you in discord.
Cheers.

@vrslev read up my explanation above, it will clarify the code you were looking at

Thanks!

I think I might want to give a crack at this.
Which do you think would be better, making this part of starlite or placing the tree code in a serperate repository?
Additionally, do we want to just use straight C++ classes with something like PyBind11, or use Python.h with C for better performance at the expense of some readability?
Here are some example Radix trees: Rax and radix_tree.

Go for it.
How drastic will the perf difference be? If it's small, say 10-20 percent, I'd say maintainability is paramount and thus c++ would be better.
Please use the same repository. We will need to create a testing setup for this code as well I guess.

Could you give some insights on why is chosen C++ over Rust? The later seems like a default modern choice now and I'm surprised to see that it's not.
orjson is written in Rust so is pydantic-core.

I think it's just byproduct of the domain knowledge of the people sticking up their hands to have a crack at it.
No bias against rust, nor any requirement that we only make one implementation before we lock something in.

I'll give writing the bindings in Rust a shot.

To clarify, I'm envisioning something like:
AnyRoute = Union[WebSocketRoute, ASGIRoute, HTTPRoute]

class RouteMap:
    def __init__(self): ...

    def add_routes(self, routes: Collection[AnyRoute]): ...

    def resolve_route(self, scope: Scope) -> ASGIRoute: ...

implemented as a compiled extension. Does that seem accurate?

Pretty much

I had a go at trying to extract out such an interface in python first: https://github.com/Dr-Emann/starlite/blob/refactor_route_map/starlite/route_map.py
Unfortunately, I did find that I needed access to a Starlite instance for .static_paths and .build_route_middleware_stack() when adding routes, I added it as a param to __init__ in my code.
Is there an existing way to benchmark the routing, ideally by itself, or if not, an easy set-up?
Finally, a note about the difference between regex matching and this trie method: while the standard way of matching many regex-en against a string would be to try each regex in sequence, it's possible, depending on the regex engine, to compile a whole set of regexes together, and match on them all simultaneously, returning which one(s) matched. Given how optimized regex engines tend to be, I'd be interested in a comparison of trying to resolve the routes in that way, as well.

I had a go at trying to extract out such an interface in python first: https://github.com/Dr-Emann/starlite/blob/refactor_route_map/starlite/route_map.py
Unfortunately, I did find that I needed access to a Starlite instance for .static_paths and .build_route_middleware_stack() when adding routes, I added it as a param to __init__ in my code.
Is there an existing way to benchmark the routing, ideally by itself, or if not, an easy set-up?
Finally, a note about the difference between regex matching and this trie method: while the standard way of matching many regex-en against a string would be to try each regex in sequence, it's possible, depending on the regex engine, to compile a whole set of regexes together, and match on them all simultaneously, returning which one(s) matched. Given how optimized regex engines tend to be, I'd be interested in a comparison of trying to resolve the routes in that way, as well.

That's great!
So there is no test setup for this at present I'm afraid. And yes, testing compiled regexes is a good idea .

I've ported the logic for the route map to a Rust extension very directly using the same trie method, but with a Rust struct containing HashMaps and HashSets to represent each of the attributes in the original Python node. The code is available at my fork of Starlite here.
The code passes all pytest tests, but I haven't benchmarked it vs the original implementation, I think maybe I can work more on that part soon / others can jump in and help? I believe the route map code is translated more or less correctly, but please take a look as well and let me know if there are any issues with it.
As for the package/release side of things, the Rust-Python interop uses PyO3, and builds with Maturin. I added an empty build script (though I'm not sure if it's necessary?) and set it up to run in pyproject.toml. Running maturin build in a Poetry shell will output different python version / platform-specific wheels for the extension to ./target/wheels, so maybe there is a way to directly include these in the end package through Poetry, but I'm not familiar with the settings for that so I didn't spend more time looking for it.
One potential issue with the current Rust implementation is that it needs handles to the function parse_path_params and types HTTPRoute, WebSocketRoute, and ASGIRoute, so they need to be directly passed in when calling Rust. I'm not sure if PyO3 has a simpler way to do this. For exceptions, there is a simple macro that can pull them in directly from the Python world.
The Python interface for the route map is here. Please let me know if there are any issues / things that need to be added.

I've ported the logic for the route map to a Rust extension very directly using the same trie method, but with a Rust struct containing HashMaps and HashSets to represent each of the attributes in the original Python node. The code is available at my fork of Starlite here.
The code passes all pytest tests, but I haven't benchmarked it vs the original implementation, I think maybe I can work more on that part soon / others can jump in and help? I believe the route map code is translated more or less correctly, but please take a look as well and let me know if there are any issues with it.
As for the package/release side of things, the Rust-Python interop uses PyO3, and builds with Maturin. I added an empty build script (though I'm not sure if it's necessary?) and set it up to run in pyproject.toml. Running maturin build in a Poetry shell will output different python version / platform-specific wheels for the extension to ./target/wheels, so maybe there is a way to directly include these in the end package through Poetry, but I'm not familiar with the settings for that so I didn't spend more time looking for it.
One potential issue with the current Rust implementation is that it needs handles to the function parse_path_params and types HTTPRoute, WebSocketRoute, and ASGIRoute, so they need to be directly passed in when calling Rust. I'm not sure if PyO3 has a simpler way to do this. For exceptions, there is a simple macro that can pull them in directly from the Python world.
The Python interface for the route map is here. Please let me know if there are any issues / things that need to be added.

First off - many thanks.
It seems there are now two possible implementations in place.
What I suggest is that you add a PR with your changes.
We will review it, and get it merged into a special branch in Starlite other than main. This will our baseline for the rust bindings and we can then then work on benchmarking.
We need build scripts and unit tests as well.
What do you guys think? @Dr-Emann @nramos0 @peterschutt

I think I might want to give a crack at this.
Which do you think would be better, making this part of starlite or placing the tree code in a serperate repository? Additionally, do we want to just use straight C++ classes with something like PyBind11, or use Python.h with C for better performance at the expense of some readability?
Here are some example Radix trees: Rax and radix_tree.

Here's my implementation (in C): https://github.com/MrAwesomeRocks/starlite/blob/NinoMaruszewski/issue-177-c-bindings/starlite/routing/.
The pure-python part (route_map.py) is done, the C implementation is still WIP.

I think I might want to give a crack at this.
Which do you think would be better, making this part of starlite or placing the tree code in a serperate repository? Additionally, do we want to just use straight C++ classes with something like PyBind11, or use Python.h with C for better performance at the expense of some readability?
Here are some example Radix trees: Rax and radix_tree.

Here's my implementation (in C): https://github.com/MrAwesomeRocks/starlite/blob/NinoMaruszewski/issue-177-c-bindings/starlite/routing/.
The pure-python part (route_map.py) is done, the C implementation is still WIP.

Thanks @MrAwesomeRocks. We will be going with Rust though, and work on the open PR as a basis. It's the best course of action for two reasons- 1. That implementation has a complete draft, and 2. Rust is easier to maintain for us in the long run and is as such future proofing.
You're welcome to chime there if course.

It appears that there is not sufficent benefit from this.

What happened here?

The performance gains were not (might not have been?) significant enough to justify the switch.

To expand on this: This still holds true today. The performance impact of the routing layer is negligible, and due to the nature of the router's architecture (radix based vs. prefix + regex map), it scales very well with a large number of routes as well. There's simply too little gain for too much effort.

Do you happen to remember what the performance gain of the Rust-based router was?

I don't remember the specifics of the difference between the rust/python stuff, but I think I remember something along the lines of after profiling how much time is taken by the router part as part of benchmarks, and it was something like even if we reduced the time taken by that part to 0, it would have made less than a 1% difference in total time or something.


Project license

Hi, could you add a license to the project? Without an explicit license it's implicitly full "all rights reserved" copyrighted so forking and expanding is a bit problematic. (Unless that's on purpose, that's ok too, but a license would clarify things.)

I've now added the MIT License to the project, feel free to do whatever you want with the project!


[AutoDiff] NFC: shorten test name.

differentiation_activity_analysis.swift -> activity_analysis.swift
The path makes it clear that the test is related to differentiation.

@swift-ci Please smoke test

@swift-ci Please smoke test


Explore clean-up

Briefly describe the feature or bug that this PR addresses.
Clean up of structure and style in the explore page and some related components to make it easier to read and understand
Please link the Airtable ticket associated with this PR.
https://airtable.com/appT8tnLbGJV6v81V/tbli2lWQHAqBa6ZcI/viwKscR1ZvgkUxVtA/recksec5TNqAphI6C?blocks=hide
If applicable, include screenshots of the feature/bugfix introduced by this PR (Please include both desktop and mobile if there is a significant UI change).
No visual changes compared to master.

@HimanshuRanka do you know what this PR is? Is it still relevant? Should we close this or test it still works and merge? Will leave to you to decide course of action


Changed message of exception to improve readability

Hey - I don't mean to be an annoying "You have a error" because I make those all the time; I simply wanted to help.
I got the following exception from this great framework:

AutoFixture.Idioms.GuardClauseException : An attempt was made to assign the value  to the parameter "redispatchSeries" of the method ".ctor", and Guard Clause prevented it, however the thrown exception contains invalid parameter name. Ensure you pass correct parameter name to the ArgumentNullException constructor.
Expected parameter name: redispatchSeries
Actual parameter name: unitId

And realized it lacked an  "a" between

AutoFixture.Idioms.GuardClauseException : [...] method ".ctor", and >a< Guard Clause prevented it[...]

So I wanted to fix those.
However, upon creating this pull request, my subscription to Grammarly  proposed a few other changes, which I've added in a separate commit, which contains those changes.
After doing these changes I tried to live by the Boy Scout Rule, and DRY the creation of these exceptions, as well as implement it for EmptyGuid, which I realize didn't check param name.

Hello @C0DK ,
Thank you for submitting the PR.
Indeed there are some typos in the codebase, that have crept in over time.
I'll try to have a look at your PR in the next couple days. Until then please have a look at the build errors on AppVeyor.
You can also have a look at the contributing guide, to learn about the code style and workflow used in the project.

@aivascu Thank you. I never got around to answering you, but I've fixed the issues now.
Love the framework, and felt like it was a good way to show it, by helping out a bit. I hope the PR doesn't break the SRP too much by both doing some boy scouting + proofreading. Luckily they are separate commits, and we can just drop some if you guys disagree.

@aivascu do you have time to check this? :)


Change from poetry to rye

This is a proof of concept showing how the project will change if we move from poetry to rye.

The tests will all fail :upside_down_face:


CI_GROUP_ID is not a Gitlab envir

There is no such thing as CI_GROUP_ID in the list of predefined environment variables. It is possible to get the group name with CI_PROJECT_NAMESPACE, but I don't think it is currently possible to get the group id.

looking at the code the use of this variable looks dead, plus I had a look and I don't think you can publish to a group space anyhow, my normal workflow for working with gitlab is publish to project and then specify the resolver for dependencies at group level. I'll confirm this behaviour at some point, the CI_GROUP_ID could always be manually specified by user in .gitlab-ci.yml


Update to Cesium 1.20

Fixes #73

@tomped would you like to test this by verifying that each example still works?

I found two issues.

Tilt Camera. I can't see the globe. http://localhost:8080/examples/cameraTilt.html
Smooth Animation with frameend. Not smooth; camera jumps around. http://localhost:8080/examples/frameAnimation.html

That is what I found. The others look good. 👍.

Thanks @TomPed!  I fixed those two examples

I think they are all working now.

I'm not that powerful yet, 😢.

Thanks @hpinkos @TomPed!


Incorrect Reference to API Scope location while granting permission to API scope

[Enter feedback here]
Feedback starts here
Under Register the ProxyIdentityExperienceFramework application section, while granting permissions to the API scope that we exposed earlier in the IdentityExperienceFramework registration:
The IdentityExperienceFramework API is available under the APIs my organization uses section but not under the My APIs section.
Please verify again and correct it accordingly in this document.
Feedback ends here

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: bee63035-00ee-f7fd-97a1-69dae5f808ef
Version Independent ID: a3447928-73cc-0290-6d46-b89234757e3b
Content: Tutorial - Create user flows and custom policies - Azure Active Directory B2C
Content Source: articles/active-directory-b2c/tutorial-create-user-flows.md
Service: active-directory
Sub-service: b2c
GitHub Login: @garrodonnell
Microsoft Alias: godonnell

@boga5
Thanks for your feedback! We will investigate and update as appropriate.

@boga5
Thanks for your feedback! I've assigned this issue to the author who will investigate and update as appropriate.

#please-close


External Libraires only show jdk lib.

Help~

And The package were not imported success.

probably it is the case od https://github.com/bazelbuild/intellij/issues/3112
I’ve created git patch for original repository, and locally use the patches version of intel’s bazel plugin. Probably could rebuild on top of master & share patched plugin at Friday

https://github.com/dpoluyanov/bazel-wsl/releases/download/v0.0.2/ijwb_bazel-fix-3112.zip rebuild it from master

https://github.com/dpoluyanov/bazel-wsl/releases/download/v0.0.2/ijwb_bazel-fix-3112.zip rebuild it from master

It's resoved extend lib problems?

Yep, it should resolve an issue with dependencies, but try next version of Bazel-wsl and ijwb https://github.com/dpoluyanov/bazel-wsl/releases/download/v0.0.3/ijwb_bazel-fix-3112.zip with additional fixes for run configurations

Thanks a lot,after  download，how to use it？

Download & install it into IntelliJ IDEA through: File -> Settings -> Plugins -> ⚙ (Manage repositories) -> Install plugin from disk (alternatively Help -> Find Action, type Install plugin from disk and press Enter), and choose downloaded zip archive as plugin for installation.

I tried idea 2021.2 and 2021.3.
The version is older or newer.

As for https://plugins.jetbrains.com/plugin/8609-bazel/versions/stable/149663 the minimum compatible range is 212.5080.55 (I've tested on IntelliJ 2021.2.3), but your version is 212.4746.92, so I'd suggest to update IntelliJ to 2021.2.3

great! Finally success on my wsl!

As for https://plugins.jetbrains.com/plugin/8609-bazel/versions/stable/149663 the minimum compatible range is 212.5080.55 (I've tested on IntelliJ 2021.2.3), but your version is 212.4746.92, so I'd suggest to update IntelliJ to 2021.2.3

By the way, is there any way support idea 2021.3 version?

https://github.com/bazelbuild/intellij/issues/3001 an issue still open in bazel-IntelliJ plugin, so I guess work in progress

OK，wait for a good news.


[DemoApp] Update space token screen

Prerequisites

[x] I have searched the backlog for duplicate or closed feature requests

As a
Developer
... I want to
illustrate all tokens
... so that I can
see the result
Motivation and context

On space screen : add missing tokens
On space screen: add illustration with text for all sections in screen

Proposed Solution
No response
Other
No response
Contact Details
No response

@ludovic35 where can I test this, I do not see any Alpha version associated to this issue?


BlobServiceClient missing on docs about signIn on Azure Blob Storage

it's missing one import in the docs.
Current:
from azure.identity import DefaultAzureCredential

Should be:
from azure.identity import DefaultAzureCredential
from azure.storage.blob import BlobServiceClient

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 15591874-317e-ce4c-0dff-20d06b625e40
Version Independent ID: be7fed9a-343a-0ec7-a94d-02c354d1ae33
Content: Quickstart: Azure Blob Storage client library for Python - Azure Storage
Content Source: articles/storage/blobs/storage-quickstart-blobs-python.md
Service: azure-blob-storage
GitHub Login: @pauljewellmsft
Microsoft Alias: pauljewell

@ropoko
Thanks for your feedback! We will investigate and update as appropriate.


opencv4 regex patch for x64-uwp-static-md triplet

The machine setup:
Windows 10 Pro 22H2
cmake 3.30
The steps to reproduce:
command : vcpkg.exe install opencv4:x64-uwp-static-md --allow-unsupported
The outcome you expected:
No error on installation
The actual outcome:
["I get a crash dialog"](error: building opencv4:x64-uwp-static-md failed with: BUILD_FAILED)
Logs :
vcpkg\buildtrees\opencv4\config-x64-uwp-static-md-out.log :

CMake Error at cmake/OpenCVUtils.cmake:1641 (if):
  Syntax error in cmake code at
    C:/NextBIM_NN/opencvtest/vcpkg/buildtrees/opencv4/src/4.8.0-f4e8005717.clean/cmake/OpenCVUtils.cmake:1641
  when parsing string
    ^ocv\.3rdparty\.
  Invalid escape sequence \.

Solution : Patch --> 0023-fix-regex-OpenCVUtils.patch

[x] Changes comply with the maintainer guide.
[x] SHA512s are updated for each updated download.
[x] The "supports" clause reflects platforms that may be fixed by this new version.
[x] Any fixed CI baseline entries are removed from that file.
[x] Any patches that are no longer applied are deleted from the port's directory.
[x] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[x] Only one version is added to each modified port's versions file.

@microsoft-github-policy-service agree

@MBSpie Please add the port version and update the version database via running ./vcpkg x-add-version. Thanks.

I suggest to postpone this to after review/merge of #38928.

why is the error present only in the triplet you tried? I cannot reproduce it in my usual triplets

why is the error present only in the triplet you tried? I cannot reproduce it in my usual triplets

You need to add port-version in vcpkg.json.
D:\a\_work\1\s\ports\opencv4\vcpkg.json: note: if<EMAIL_ADDRESS>is already published, update this file with a new version or port-version, commit it, then add the new version by running:
  vcpkg x-add-version opencv4
  git add versions
  git commit -m "Update version database"

was a reply to me? the version is not important. I need to understand how to reproduce the issue

anyway, I got what's going on but the fix here is totally wrong. The meaning of the regex is to escape the dot, not the backslash... I am adding a fix in my opencv 4.9 PR, please @MBSpie try that

Hello cenit,
Thank you for your time and effort. Indeed your opencv 4.9 PR fixes my issue. However I still get the same error on the master branch. I will close this PR due to the fact that it's not an actual fix but if you have hints on how you fixed it let me know !

This issue will be fixed in the #38928 update.

I got what's going on but the fix here is totally wrong. The meaning of the regex is to escape the dot, not the backslash.

The patch is right with regard to the the regex syntax error. First backslash to escape for CMake language, second backslash to escapte the dot for regex language.
(There is an alternative without backslash: [.])
The stray in in foreach is another bug. The correct modern replacement is IN ITEMS, and the better full alternative  is IN LISTS ARGN.

I got what's going on but the fix here is totally wrong. The meaning of the regex is to escape the dot, not the backslash.

The patch is right with regard to the the regex syntax error. First backslash to escape for CMake language, second backslash to escapte the dot for regex language. (There is an alternative without backslash: [.])
The stray in in foreach is another bug. The correct modern replacement is IN ITEMS, and the better full alternative is IN LISTS ARGN.

from the cmake guide it looks like the regex is perfectly fine if you want to match "ocv.3rdparty." as the beginning of a target name

from the cmake guide it looks like the regex is perfectly fine if you want to match "ocv.3rdparty." as the beginning of a target name

This is just the regex language. But it is embedded in CMake language. Note the doc's last sentence before your screenshot:

Note that two backslashes (\\1) are required in CMake code to get a backslash through argument parsing.

And the CMake error message isn't a fake, either.

so it still puzzles me why the error appears only in this triplet. I was sure it was related to a strange combination of targets that was triggering the error, and removing the bug in the foreach apparently fixing the problems really distracted me from the problem you were expressing.
Shall I fix also the regex in my PR?

IMO both lines should be fixed and submitted upstream. The fix is 100% clear.
However, it is really puzzling that it is hard to trigger. Due to the stray in, the error will always occur when the function is callled.

The rh of the MATCHES operator is a regex, not a string. Are you sure the syntax as written really requires an escape char if you need to match the dot? I am still unsure about the regex, sorry for insisting. I will write some tests to verify it

cmake_minimum_required(VERSION 3.16)

project(cmaketest)

list(APPEND listtest "ocv.3rdparty.libjpeg")
list(APPEND listtest "ocv.3rdparty.libpng")
list(APPEND listtest "ocv\\\.3rdparty\\\.libtiff")
list(APPEND listtest "ocv\\\.3rdparty\\\.libwebp")

message(STATUS "listtest: ${listtest}")

foreach(tgt in ${listtest})
  message(STATUS "tgt: ${tgt}")
  if(tgt MATCHES "^ocv\.3rdparty\.")
    message(STATUS "Matched first regex")
  elseif(tgt MATCHES "^ocv\\.3rdparty\\.")
    message(STATUS "Matched second regex")
  elseif(tgt MATCHES "^ocv\\\.3rdparty\\\.")
    message(STATUS "Matched third regex")
  else()
    message(STATUS "No match")
  endif()
endforeach()

foreach(tgt ${listtest})
  message(STATUS "tgt: ${tgt}")
  if(tgt MATCHES "^ocv\.3rdparty\.")
    message(STATUS "Matched first regex")
  elseif(tgt MATCHES "^ocv\\.3rdparty\\.")
    message(STATUS "Matched second regex")
  elseif(tgt MATCHES "^ocv\\\.3rdparty\\\.")
    message(STATUS "Matched third regex")
  else()
    message(STATUS "No match")
  endif()
endforeach()

when executed gives this result
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- listtest: ocv.3rdparty.libjpeg;ocv.3rdparty.libpng;ocv\.3rdparty\.libtiff;ocv\.3rdparty\.libwebp
-- tgt: in
-- No match
-- tgt: ocv.3rdparty.libjpeg
-- Matched first regex
-- tgt: ocv.3rdparty.libpng
-- Matched first regex
-- tgt: ocv\.3rdparty\.libtiff
-- No match
-- tgt: ocv\.3rdparty\.libwebp
-- No match
-- tgt: ocv.3rdparty.libjpeg
-- Matched first regex
-- tgt: ocv.3rdparty.libpng
-- Matched first regex
-- tgt: ocv\.3rdparty\.libtiff
-- No match
-- tgt: ocv\.3rdparty\.libwebp
-- No match
-- Configuring done (3.1s)
-- Generating done (0.0s)

i cannot understand why no matches for tiff and webp examples)

That's easy. Check the output: they come with two characters between words. The regex' match exactly one char.
-- tgt: ocv\.3rdparty\.libtiff
-- No match
-- tgt: ocv\.3rdparty\.libwebp
-- No match

I still cannot understand the failure nor your regex suggestion

I had to study your example. Long story short:
https://cmake.org/cmake/help/latest/policy/CMP0053.html

Now any non-alphanumeric, non-semicolon, non-NUL character may be escaped following the escape_identity production in the Escape Sequences section of the cmake-language(7) manual.

#cmake_minimum_required(VERSION 2.6) # syntax warning
#cmake_minimum_required(VERSION 3.0) # syntax error
cmake_minimum_required(VERSION 3.1) # no syntax issue reported, but...
if("foo_bar" MATCHES "^foo\.bar")
    message(FATAL_ERROR "OOPS, didn't want to match '_'")
endif()
if("foo_bar" MATCHES "^foo\\.bar")
    message(FATAL_ERROR "OOPS, didn't want to match '_'")
endif()
if("foo_bar" MATCHES "^foo[.]bar")
    message(FATAL_ERROR "OOPS, didn't want to match '_'")
endif()
message(STATUS HOORAY)
return()

Now note the problem with \.: It is a plain . after CMake language.

ok maybe got you.
maybe, since in any case we have to match ocv.3rdparty.*, not anything with a backslash in it, so the regex for me is still ok

the regex for me is still ok

It is okay if the . is to match any input character, but I think and you think that this is not the case. (That's the only reason to escape the ..)
But my test shows that \. is not okay if you want to match literal . only.

Thanks. Your test.cmake was clear about "^foo.bar" matching also unwanted "_"
But still, I cannot grasp the problem in its entirety. Sorry for abusing of your time and patience. This is the 2nd edition of my cenittest.cmake
cmake_minimum_required(VERSION 3.16)

list(APPEND listtest "ocv.3rdparty.libjpeg")
list(APPEND listtest "ocv\.3rdparty\.libpng")
list(APPEND listtest "ocv\\.3rdparty\\.libtiff")
list(APPEND listtest "ocv\\\.3rdparty\\\.libwebp")

message(STATUS "listtest: ${listtest}")

foreach(tgt in ${listtest})
  message(STATUS "Testing tgt: ${tgt}")
  if(tgt MATCHES "^ocv.3rdparty.")
    message(STATUS "Matched first regex")
  endif()
  if(tgt MATCHES "^ocv\.3rdparty\.")
    message(STATUS "Matched second regex")
  endif()
  if(tgt MATCHES "^ocv\\.3rdparty\\.")
    message(STATUS "Matched third regex")
  endif()
  if(tgt MATCHES "^ocv\\\.3rdparty\\\.")
    message(STATUS "Matched fourth regex")
  endif()
  if(tgt MATCHES "^ocv[.]3rdparty[.]")
    message(STATUS "Matched fifth regex")
  endif()
  if(tgt MATCHES "^ocv[\.]3rdparty[\.]")
    message(STATUS "Matched sixth regex")
  endif()
  if(tgt MATCHES "^ocv[\\.]3rdparty[\\.]")
    message(STATUS "Matched seventh regex")
  endif()
  message(STATUS "End of testing tgt: ${tgt}")
endforeach()

cmake -P cenittest.cmake gets this:
-- listtest: ocv.3rdparty.libjpeg;ocv.3rdparty.libpng;ocv\.3rdparty\.libtiff;ocv\.3rdparty\.libwebp
-- Testing tgt: in
-- End of testing tgt: in
-- Testing tgt: ocv.3rdparty.libjpeg
-- Matched first regex
-- Matched second regex
-- Matched third regex
-- Matched fourth regex
-- End of testing tgt: ocv.3rdparty.libjpeg
-- Testing tgt: ocv.3rdparty.libpng
-- Matched first regex
-- Matched second regex
-- Matched third regex
-- Matched fourth regex
-- End of testing tgt: ocv.3rdparty.libpng
-- Testing tgt: ocv\.3rdparty\.libtiff
-- End of testing tgt: ocv\.3rdparty\.libtiff
-- Testing tgt: ocv\.3rdparty\.libwebp
-- End of testing tgt: ocv\.3rdparty\.libwebp

I cannot explain the results. Can you?

also, if i just edit the opencv code in this way:
function(ocv_install_used_external_targets)
  message(STATUS "ARGN: ${ARGN}")
  if(NOT BUILD_SHARED_LIBS
      AND NOT (CMAKE_VERSION VERSION_LESS "3.13.0")  # upgrade CMake: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/2152
  )
    foreach(tgt in ${ARGN})
      message(STATUS "tgt: ${tgt}")
      if(tgt MATCHES "^ocv\.3rdparty\.")

adding some message(STATUS... to help debug, i have this result:
-- ARGN:
-- tgt: in
CMake Error at cmake/OpenCVUtils.cmake:1669 (if):
  Syntax error in cmake code at

    E:/code/vcpkg_cenit/buildtrees/opencv4/src/4.9.0-60c77fc213/cmake/OpenCVUtils.cmake:1669

  when parsing string

    ^ocv\.3rdparty\.

  Invalid escape sequence \.
Call Stack (most recent call first):
  modules/videoio/CMakeLists.txt:267 (ocv_install_used_external_targets)

CMake Error at modules/videoio/CMakeLists.txt:267 (ocv_install_used_external_targets):
  A logical block opening on the line

    E:/code/vcpkg_cenit/buildtrees/opencv4/src/4.9.0-60c77fc213/cmake/OpenCVUtils.cmake:1669 (if)

  is not closed.

while in my example, even if i make the list empty (like ARGN here), it doesn't result in a cmake spectacular syntax error, it just says that "in" (the stray argument) doesn't match the (broken) regex, that's it
I cannot really understand what's going on 😢

Your test has cmake_minimum_required(VERSION 3.16), so CMP0053 is set to NEW, so there is no error.
For the cmake code, the complaint is about the RHS (regex), not the LHS (variable), and we don't know the state of CMP0053 (and CMP0012). You may easily check the top-level cmake_minimum_required(), but it will be hard to find all nested cmake_minimum_required() and cmake_policy(), plus strange effects e.g. of macros.~ :cry: Just look at the top-level CMakeLists.txt: uwp translates to WindowsStore, and then the build runs with cmake_policy(VERSION 2.8), plus manual changes:
https://github.com/opencv/opencv/blob/08f7f13dfab327967f61a6a4242ed233dfeaad32/CMakeLists.txt#L18-L24
grep -R cmake_policy doesn't give hits for CMP0053, but there is one for CMP0012 in cmake/OpenCVGenPkgconfig.cmake. That's the right set for the problem.
The problem is unrelated to regex. It is about CMake parsing (quirks).


Set session_token cookies as Secure

This makes it so the cookie is only sent over HTTPS.

Are there any benefits to having SSL in development? If not, we should not make it a requirement (I'm leaning towards no SSL in dev)


Improve error message when missing queryGenerationFactories in the template.

Thank you for this great generator.
While Build Template it throws this error:
 Error: Cannot generate a random value for scalar 'smallint'.
  The random generator is not able to randomly generate a value for non-standard GraphQL scalars.
  You have to provide a custom factory by providing this in your config:
  {
  'smallint': () => generateRandomCustomScalar()
  }

Can you please provide an example on how to fix this? Documentation is not clear enough, where the  'smallint': () => generateRandomCustomScalar() should go in the config? Can you provide an example of generateRandomCustomScalar()?

Thank you for raising this issue. So this is an error thrown by the Query generator plugin, which is not entirely accurate when using the magidoc template directly. I will improve the error message for this special error, since I expect it may happen quite often.
To solve it, you need to provide a query generation factory in the magidoc config
export default {
  introspection: {
    // ...
  },
  website: {
    // ...
    queryGenerationFactories: {
      smallint: 42
    }
  }
}

More detail about it can be found [here](https://magidoc-org.github.io/magidoc/plugins/graphql-query-generator].

Oh.. sorry, my bad. I wrote this on my phone, should have opened my computer 😅. Simply wrap it in an options key:
export default {
  introspection: {
    type: "url",
    url: "http://localhost:9091/v1/graphql",
    headers: {
      "x-hasura-admin-secret": "myadminsecretkey",
    },
  },
  website: {
    template: "carbon-multi-page",
    options: {
      queryGenerationFactories: {
        smallint: 42
      }
    }
  },
};

yeah, you are right, on the docs link you provided above it's inside an options object but was hard to spot due to the many comments 😅.
Probably this is not possible: Can we first check for unknown scalars prior reaching the Build Template step? It takes lot's of time to build and throw an error if scalar is not known. We know witch scalars are not compatible before hand of course, just saying.
Thanks @pelletier197, keep it up!

I can try to look if I could find a way to do that, or at least show all the missing scalars at once in the error.
You're right, it's not necessarily easy, because the cli doesn't actually do anything with the options, it simply forwards them directly to the template. And since the factories support glob syntax as well, it adds a little bit of complexity.
I'll see what I can do, thanks for the idea!

That makes sense!
It would be great to do a check first, will save some time while building the config since we won't have to cross check our scalars against it.
Another option is to have them predefined, at least the most common ones like eg timestamp, date, uuid and let the user override them on options.queryGenerationFactories if desired.

There are some default factories, but very few. I'll add those you mentioned, they clearly seem like common things that could come back in many APIs.

great, these are some common Hasura scalars:
    {
        _text: "loremipsum",
        bigint: 1000,
        date: "2022-03-06T08:23:45.000Z",
        float8: 0.1,
        geography: { type: "Point", coordinates: [0, 0] },
        geometry: { type: "Point", coordinates: [0, 0] },
        jsonb: {},
        numeric: 1,
        smallint: 42,
        timestamp: "2022-03-06T08:23:45.000Z",
        timestamptz: "2022-03-06T08:23:45.000Z",
        uuid: "a12ce80c-42e3-4149-89cc-4afda30e5e7f",
      }

After adding all these scalars above it throws a new error:
  Error: Fatal error: A recursive input with a non-nullable circular reference was detected, which makes it impossible to generate a query. If you ever see this error, it means that your GraphQL uses an invalid schema.
  at generateInput (file:///private/var/folders/yz/dk5dbx8n2hqcfk1kdktxmknh0000gn/T/carbon-multi-page@2.0.0/node_modules/.pnpm/@magidoc+plugin-query-generator@2.0.0/node_modules/@magidoc/plugin-query-generator/build/generator/fakeGenerator.js:34:23)

ps. I'm using Hasura 2.7.0

Thanks. Those all seem reasonable to add to the default factories.
About your error, thanks for taking the time to check it.
I'll check it out as well when I get some time. Would it be possible for you to provide me with the _schema.json file for your API? If not possible, a minimal reproduction schema would be great otherwise.
I feel like this error message needs more detail as well. I'll improve that as well.

Sorry but I cannot share anything, it's a private repo. :(
It fails on the account_insert_input, this is how I found it:
 if (!isNullableType(input)) {
        console.log(
          "type: ",
          unwrappedType,
          "input: ",
          input,
          "name: ",
          unwrappedType.name
        );
        throw new Error(
          "Fatal error: A recursive input with a non-nullable circular reference was detected, which makes it impossible to generate a query. If you ever see this error, it means that your GraphQL uses an invalid schema."
        );
      }

this spits out type:  "account_insert_input" input:  "account_insert_input!" name:  account_insert_input, this input type has 0 non-null types and this table has some (ORM) table connections: eg:
input account_insert_input {
 about_me: String,
show_seen_sent: Boolean
    targeted_matches: match_arr_rel_insert_input
    time_format: String
    unban_reason: String
    unbanned_at: timestamptz
    unbanned_by: account_obj_rel_insert_input
    unbanned_by_id: String
    updated_at: timestamptz
    verified_by: account_obj_rel_insert_input
    verified_by_id: String
devices: device_arr_rel_insert_input

where device_arr_rel_insert_input type is:
"input type for inserting array relation for remote table \"device\""
input device_arr_rel_insert_input {
    data: [device_insert_input!]!
    "upsert condition"
    on_conflict: device_on_conflict
}

account_obj_rel_insert_input type is:
"input type for inserting object relation for remote table \"account\""
input account_obj_rel_insert_input {
    data: account_insert_input!
    "upsert condition"
    on_conflict: account_on_conflict
}

wish I could provide more info, best thing you can do is to run a hasura docker and create a simple database. Hasura has weird schema in general.

Okay, I think I get the issue. I'll open another bug for that. This is recursion with two layers, I'm gonna see how I can fix this, not an easy one.
In the meantime, you could probably do something like this if you're blocked and it's the only thing like this in your API.
{
  queryGenerationFactories: {
    device_arr_rel_insert_input: null,
    'device_arr_rel_insert_input!': {} // an example
   }
}

Thanks for all the help by the way, really appreciated!

You're welcome @pelletier197, and again, thanks for this great generator. If you need anything please tag me! If you want to get exactly the same experience like me then visit Hasura Getting Started.

The issue with the recursion should be fixed now in 2.1.0, along with a few other improvements. Let me know if you have other issues.
I have not implemented the default types of Hasura yet, cause I have a little bit of confusion looking at the types example you shared compared to Hasura's docs.  I'll need a little more research before implementing them. See #93 for more details.


emacs: slash-terminate comp-eln-load-path

Motivation for this change
Emacs just concatenates the version and hash to the string provided in comp-eln-load-path (see https://github.com/emacs-mirror/emacs/blob/feature/native-comp/src/comp.c#L4088), so directories must end with a slash (@AndreaCorallo, is this intended?). Otherwise, we get paths like "/nix/store/akx3lh6h8vyj63xgi396mylbhn3lmc4p-emacs-gcc-20201217.0/share/emacs/native-lisp28.0.50-x86_64-pc-linux-gnu-c50f2f5ede36309a94931f324ed27b53".
See https://github.com/nix-community/emacs-overlay/issues/74
Things done

Built on platform(s)

[x] NixOS

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Fits CONTRIBUTING.md.

I would consider this a bug to report upstream; comp-eln-load-path entries should always be treated as directory paths, so the routine to generate the absolute ELN path should append a directory separator if necessary. @collares if you would like me to file a bug report, let me know.

Filed http://debbugs.gnu.org/cgi/bugreport.cgi?bug=45462 upstream

Closing since this was fixed upstream.


bugfix in pull request

forgot to move fillErizos() inside the aws callack caused this pull request to be ineffective.

Thank you for the fix. However it seems you have forgotten to add fillErizos function in case no IP and not amazon are set by configuration (line https://github.com/ging/licode/pull/445/files#diff-e3cda258841aa4572d51c64b5078383cL247)

Ok, I'll try to do a major change for everything concern aws in another pull request.

switched to: https://github.com/ging/licode/pull/507


added open source flutter app "Ebook-App"

Fixes #133
@ashitaprasad please review and merge this PR

The github repo does not exist.


nu_plugin_plotters should have a way to set axis labels

When plotting data from a time series from the nu_plugin_prometheus plugin:
'rate(unpoller_device_port_transmit_bytes_total{port_id="some-port-id"}[5m])'
| prometheus query range -s home --start ((date now) - 1hr) --end (date now) --step 5min
| get values
| flatten
| rename y x

y
x

75867.61893734452
1728150300

121664.27478666068
1728150600

60296.01729080339
1728150900

96637.246886853
1728151200

113744.3007772066
1728151500

77804.2022084753
1728151800

78611.33469707123
1728152100

114471.74035087718
1728152400

333646.5026033993
1728152700

102914.6110389838
1728153000

80590.20828362502
1728153300

107095.817733316
1728153600

82750.42104672216
1728153900

The x axis values are seconds from the UNIX epoch. Plotting them as the raw value is not very useful for knowing the time span the plot covers.
It would be nice to attach an x (and y) label to a series that plotters-rs could use to give the axis labels as shown in several of the examples, perhaps:
# above command
| upsert x_label {|r|
  ($r.x | into int ) * 1_000_000_000
  | into datetime
  | format date "%+"
}

y
x
x_label

103804.44004673732
1728150697
2024-10-05T17:51:37+00:00

119285.249122807
1728150997
2024-10-05T17:56:37+00:00

55978.62157771563
1728151297
2024-10-05T18:01:37+00:00

75475.46051015754
1728151597
2024-10-05T18:06:37+00:00

78608.26529215892
1728151897
2024-10-05T18:11:37+00:00

98761.50626146942
1728152197
2024-10-05T18:16:37+00:00

148210.41477338516
1728152497
2024-10-05T18:21:37+00:00

299929.763187166
1728152797
2024-10-05T18:26:37+00:00

83996.41052631578
1728153097
2024-10-05T18:31:37+00:00

59713.041159526045
1728153397
2024-10-05T18:36:37+00:00

129653.7883944674
1728153697
2024-10-05T18:41:37+00:00

106094.0097469238
1728153997
2024-10-05T18:46:37+00:00

64033.876479793405
1728154297
2024-10-05T18:51:37+00:00

I dislike the x_label proposal, it gets hard to get it right when you use multiple series and each series could come with an x_label. I think a better alternative altogether would be to pass the chart a closure that formats the labels on the x-axis.


link AWSDNSRecord and LoadBalancerV2

This resolves issue #488
Links AWSDNSRecord to LoadBalancerV2.

I gave this a crack, but I'm not sure it's what you had in mind.
I added a test to route53 (felt appropriate as it's the route53 module that creates the links). I added a DNS record to the test data and then load both the EC2 and Route53 test data and check there's the expected link.
Let me know what you think.


The linux binary in the releases page is not working for MAC OS

Context
I have downloaded the cardano-addresses https://github.com/input-output-hk/cardano-addresses/releases/download/3.6.0/cardano-addresses-3.6.0-linux64.tar.gz . When I tried running in the macos, I'm getting an error exec format error: ./cardano-addresses

Infra
cardano addresses - 3.6.0
macos BigSur - 11.5.2

Unfotunately we don't have MacOS binary available at the moment on cardano-addresses release page.
It is available however on the cardano-wallet release page.
So cardano-wallet-v2021-09-09-macos64.tar.gz includes cardano-addresses inside that should work on MacOS.


Fix some typos, backticks, new clippy lint.

And yep, I already submitted the actual typos upstream.


Start inter-requiring actions

Take another step towards modularity by using require, not globals, to have actions call each other
Well, this may give a point to @kepta's idea of using ES6 modules :) - iD does a lot of circular dependencies, and that currently breaks this branch.
Other notes:

Rollup's umd and iife patterns don't mix with the global use strict in iD + phantomjs, so it's removed from start.js for now.
This upgrade phantomjs, making the bind polyfill unnecessary

Okay, ready for review @bhousel

@tmcw I wanted to know is adding babel to the build step to use the goodness of ES6 a good idea?

I wanted to know what do you think about adding a babel transpilation step to use the sugary ES6 🙃.

per chat - yes eventually but not today! 😄


codeql: improve Ubuntu dependency installation

Commit Message:
If apt can't update the package index, it doesn't exit with an error
by default, so add the --error-on=any flag to force an error.
Pass the --yes flag to the install to force non-interactive mode.
Additional Description: This change ought to make the CI error in #16541 more obvious.
Risk Level: Low
Testing: None
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

CI checks failed with this error:
ERROR: ./docs/root/version_history/current.rst:68: Version history not in alphabetical order (http vs crash support): please check placement of line

which I believe will be fixed by #16555

@jpeach its wierd that this didnt fail anyway - on the azure build it also hit this issue but did fail
relatedly im wondering whether we should remove (with sed i guess) the ms packages source - afaiaa its unused in both cases

@jpeach its wierd that this didnt fail anyway - on the azure build it also hit this issue but did fail
relatedly im wondering whether we should remove (with sed i guess) the ms packages source - afaiaa its unused in both cases

Yeh, I'm not sure why the other PR gets the failure. It looks like the apt install fails but still exits 0 and I'm not sure why that happens. I think the changes in this PR are a slight improvement, but not sure whether they will fix the issue fully.

im not sure that this change is necessary and/or will have any effect
my guess re the reason its not failing is that its not being run -e - if that is the case --error-on wont have any effect
regarding the --yes flag - im kinda surprised that it works normally without this - either way i dont think it will have any bearing on the failure

GitHub documents the bash shell as being run with -e.
I think this change is a slight improvement. It's more explicit and passes the right flags to get the behavior we want, but whether it fixes the actual problem seen in other PRs? Probably have to suck it and see :)

@phlax is worth giving this a try?

is worth giving this a try?

perhaps - the -y flag certainly wont hurt - and i guess neither will the --error-on flag - im just not convinced that it changes anything - i dont want to block it tho


Create identity-infra stack

Who is this for?
Developers who want to keep infrastructure handling PII separate from infra that does not.
What is it doing for them?
Creating a stack in the identity account for deploying the identity-admin account. This necessarily requires a different VPC, LB and developer access credentials. It allows for easily differing security groups by virtue of having a different LB from the rest of the FE apps.
This change create an experience and identity folder within the top level infrastructure folder, moves modules to the top level and repoints dependencies.
Note: This is deployed

For https://github.com/wellcomecollection/platform/issues/4811


Inner types in prototype style

Inner types are currently only partly functional in prototype style. For instance, this doesn't work:
class Outer() {
    class Inner() {
        shared String s = "hello";
    }
    shared String test() { return Inner().s; }
}

In the generated JS code test can't access Outer.Inner() properly because it is defined inside Outer and is not exported. That could be fixed of course. But perhaps the real problem is that we mix prototype style and capture style here: while methods are defined separately and added to the prototype, inner types are defined inside the body of the outer type, creating a closure. It seems that this mixing of concepts is not completely trouble-free.
It would be more consistent, and probably work out better in the long term, to define inner types in the same way as other class members in prototype style. It would also definitely be more efficient because currently inner types are completely initialized every time an instance of the outer class is created.
Implementing this is definitely not uncomplicated either though.

After playing around with some JavaScript code I think it should be possible to move the inner types out of the class body. I'll need to refactor the generated code a bit first.

Inner types in prototype style are now added to the prototype instead of being defined in the class body. All the tests still pass, and correct code is generated for the example given above. As a nice side effect the prototype style code is now more readable because the members are grouped properly.
But there are probably some cases that still don't work, for both prototype style and capture style, in particular if inner and outer types have identical names.

Ok, this seems to work pretty well now. The problems with identical names of inner and outer types are not directly related to this, so I'll open a separate issue for that.


removing unwanted files ++

Removed unwanted files
Data services' examples
etc.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[release-1.10] Add Traffic Verification to Helm Install Test

This is an automated cherry-pick of #31950

@googlebot I consent.


v2 release discussion

Creating this issue to have one place to discuss the v2 release.
I think that we should clean up the changelog. By that I mean remove from the changelog everything that is noise for the average formidable user. For example >" refactor: use hasOwnProperty in check against files/fields" is not useful information for someone that is trying to migrate from v1 to v2. If you agree I can make a PR for that.
First time since a long time that we are under 30 open issues 🥇 .
Ideally we also

fix the tests (they broke since I migrated to Jest a bit too fast).
create a migration guide

Agree. These are noise. The beginning is good. They have links to all the issues and PRs to see how much work is done.
Migration guide is nothing more than the exports change. Oh yea, and the plugins API and the cool fileWriteStream handling.

From my comment here: https://github.com/node-formidable/formidable/issues/635#issuecomment-813020126

I guess #689 was merged and then the filter PR #716 was also merged, right? I guess that means a v2 release sometime in the next few weeks?

Is this true? Would a v2 release be sometime in the next weeks?

fix the tests (they broke since I migrated to Jest a bit too fast).

Which tests are broken? Everything on GitHub Actions that's failing on the master branch?
https://github.com/node-formidable/formidable/runs/2249456170
Would be good to have a clear list of open tasks and how to attack them, just in case anyone (eg. @Songkeys) would like to contribute here.

Which tests are broken? Everything on GitHub Actions that's failing on the master branch?

Fixtures. When you try to run tests locally you can see.

Is this true? Would a v2 release be sometime in the next weeks?

Soon ™ lol

https://ko-fi.com/tunnckoCore Everyone get in

@tunnckoCore publish v2 whenever you are ready, I am going to start the v3 branch soon

ideally sooner than later because most people assume npm contains the last version and create issue like this https://github.com/node-formidable/formidable/issues/726

Expect to see v2 released in these days, and don't forget to upgrade @types/formidable as well.

Yeah, @types/formidable are currently broken with the 1.2.2 version, after the patch from @peterblazejewicz and @martin-badin :
https://github.com/DefinitelyTyped/DefinitelyTyped/pull/51364#issuecomment-812663849

@karlhorky
We can revert the 1.2.2 changes or the conflicting change can be fixed by PR. I'd nt say types are broken.

x-ref: DefinitelyTyped/DefinitelyTyped#52697

Hey there, sorry for the silence lately! :)
I'm sorry for saying it again, but... I'm really currently upgrading the machine (ahhh, damn tired of jumping around linux distros, it was great on ArchLinux for YEARS, the thing is I don't have the nerves to restore my awesome setup ;d).
I'm in terrible period, but it's now 15 days holidays here and I'm REAAALLY heating and getting up. Еxcuses, excuses, excuses... anyway.
Thanks!
PS: Even if everything is okay on the branches, i can't just run npm publish because don't have setup here.

As of<EMAIL_ADDRESS>the types are working again 🙌  I guess they should also work for Formidable v2 too. 👍
Thanks to @peterblazejewicz for the fix!
https://github.com/DefinitelyTyped/DefinitelyTyped/pull/52697#issuecomment-841247700

you're welcome! I've not been using this module a lot, so keep this in mind, when something surfaces, thx!

To install from github master in the meantime
npm i github:node-formidable/formidable

To install from github master in the meantime
npm i github:node-formidable/formidable

Without correct type definition

Just a heads up but the bulkiness of formidable has increased quite dramatically in the 2.x version.
Original 1.2 had zero dependency which was... very nice. The new "and improved" 2.x has quite a lot of junk dependencies:
`--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  | +--<EMAIL_ADDRESS>  | `--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  +--<EMAIL_ADDRESS>  | `--<EMAIL_ADDRESS>deduped
  `--<EMAIL_ADDRESS>    `--<EMAIL_ADDRESS>      +--<EMAIL_ADDRESS>      | +--<EMAIL_ADDRESS>      | `--<EMAIL_ADDRESS>deduped
      +--<EMAIL_ADDRESS>      | +--<EMAIL_ADDRESS>deduped
      | +--<EMAIL_ADDRESS>      | | `--<EMAIL_ADDRESS>deduped
      | `--<EMAIL_ADDRESS>      `--<EMAIL_ADDRESS>
Some of these are... quite unnecessary like once which hasn't been updated in over 5 years and is literally a few lines of code.
Not to mention dezalgo which is literally a few lines of codes with two more additional dependencies, that are also few lines of code, just to make a callback run on next tick? Feels HIGHLY overkill to have to install 3 dependencies just for a tiny functionality like that.
I know npm and javascript developers don't care about what kind of code they're installing with their library but some of us actually care. :(

the 3.x version already got rid of the qs dependency
For the rest you are right I agree , some could be removed, but it has to be done on a case by case basis, sometimes a 20 line of code library handles quite a few edge cases one would not necessarily think about it.
Also if you make a PR try to find out why a library is used, maybe there is a good reason.

If I may ask @GrosSacASac what's the difference between 1.x and 2.x and 3.x? I'm not finding a good change log

https://github.com/node-formidable/formidable/blob/3.x/CHANGELOG.md
if anything is missing tell me

Zero dependencies and 85.1 kB package size used to be definitely killer features of formidable comparing with multer.
Regarding the version 3.0, Is there any way to actually install it via npm?
I would like to switch from the abandoned multer and formidable 3.0 seems to be a great choice.
BTW, hopefully, the version 3.0 comes not only in the form of the ESModule but also with the async/await support out of box and  will be the callback-free.

v2 was supposed to be published a couple months ago already, unfortunately people with access to npm publish are absent nowadays ... Myabe you can try to tweet if you have twittter
There is an open issue for async await https://github.com/node-formidable/formidable/issues/685
To install 3.x with npm use
npm i node-formidable/formidable#3.x

To install 3.x with npm use
npm i node-formidable/formidable#3.x

I'm trying to install it via package.json, but there I can't fin a v.3.0 as a proposed version for formidable:

Is it possible to install v.3.0 via package.json?

v2 was supposed to be published a couple months ago already, unfortunately people with access to npm publish are absent nowadays ... Myabe you can try to tweet if you have twittter

Very similar story with multer, it reached the RC2 stage in March 2020, since that time no new builds…

@pubmikeb WHAT you show me is a screenshot of you trying the autocomple, which takes data from npm published versions. But 3.x is not on npm. Only on github. that is why I gave you the command. it is a shorthand shorthand for npm install github:/[#]:
if you want to change package.json directly and then install then
"formidable": "github:node-formidable/formidable#3.x",

I published v2 and v3 on my npm account
https://www.npmjs.com/package/@grossacasacs/formidable
For 2 reasons, I realized that github branches could be deleted instantly which would break all my project that use formidable and install it from github, and the wait is over
🤙

I take it that v2 and v3 will also be released under formidable (unscoped) soon too?

They (people with npm formidable write access) barely show sign of life in the past year, I don't think it will happen soon

Ah, you don't have access? I thought you did because of your commit rights in this repo here. @tunnckoCore would it make sense to give access to @GrosSacASac ?

Yea, sorry...
Ha, didn't know either. I think I have npm now, will try to add you.

@GrosSacASac, ready. But please release on separate dist-tags, not on latest.

I can't use 2fa on npm, you should publish.
Why not on latest ? People had enough time to prepare I think.

I can't use 2fa on npm, you should publish.

Duh. haha.
Is it active and required? Hm, can we disable it (temporarily) and where? haha. Why you can't? Some country restrictions or something like that?

Why not on latest ? People had enough time to prepare I think.

Yea, definitely. Yea, maybe v2 on the latest, and v3 as separate dist-tag or on the canary.
Basically, cuz I didn't looked more deeply and focused in half a year and don't won't to go wrong. Plus, we don't have metrics to know how many tried the canary, or using it currently. And we grew to 20M/month, which may be exactly because the v2-canary but still..

Tidelift requires 2fa.
I tried to use 2fa but it didn't work. (Empty error message) I can try again this weekend.

Tidelift requires 2fa.

Ah.
Have some problems with them in the past 1-2 months. I'm waiting a response and we'll see soon, and may disable it (i've seen how, haha)

(Empty error message)

oh...

One month later. No response from Tidelift.

I can try again this weekend.

any updates?
So... this leaves us in a strange state where I'm the only "active" (questionable, but not completely abandoned it and i'm looking from time to time trying to keep up) one who have npm access. Hell of a year, but I'm squeezing some time to publish v2 on latest and v3 as dist-tag. At least to get things going.

@GrosSacASac do the master branch have latest stuff from your v2? And is the v3 branch the actual/latest v3? Just asking to figure out what to publish where ;d

master is v2 yes, v3 is the latest v3 also yes,
My phone (too old I think) can't run google play store to download the 2fa authentication companion Npm recommends,
and apparently I could also do 2fa with a browser extension but I don't know how to get started

Hm. What npm recommends? I had Google Authenticator, but mainly use Authy including for npm too. I don't know through browser. If you're on linux, you can try passwordstore.org (on arch pass and pass-otp) which has 2FA support. I think it also has Android app, so you can try from there.

@GrosSacASac awesome! :tada:  So you can publish now? Try publishing to some dist-tag i guess.

Published 3.1.1-canary.20211030 from 3.x branch with npm tag 3.x
Published 2.0.0-canary.20210330 from master branch with npm tag canary
the canary version
npm install formidable@canary
3.x
npm install<EMAIL_ADDRESS>Should I publish 2.x as latest ?

Published 1.2.3 from branch 1.x with npm tag 1.x

Awesome. So..
I wrote some notes for the v1 README. Let me know if that's clear on how we are proceeding.

I'll update the 1.x readme, you just need to rename branches and publish the versions if you are okay with that plan.
Make sure to publish 1.2.4 to latest  AND v1 with updated readme first, then mark them both as deprecated. I'm not sure if we can deprecate a specific dist-tag.. but anyway. Hm, we should also make another file on the master branch with that notes, so we can link to it on the deprecation messages.
Deprecation message like Please upgrade to latest formidable@v2 or formidable@v3 versions. Check these notes: [link to the master notes file]
npm deprecate formidable@"< 1.2.4" "Please upgrade to latest formidable@v2 or formidable@v3 versions. Check these notes: [to the master notes file]"
npm deprecate<EMAIL_ADDRESS>"Please upgrade to latest formidable@v2 or formidable@v3 versions. Check these notes: [to the master notes file]"
npm deprecate<EMAIL_ADDRESS>"Please upgrade to latest formidable@v2 or formidable@v3 versions. Check these notes: [to the master notes file]"

We are deprecating all v2-canary to inform all that are using it to upgrade to latest or v2 or v3 dist-tags.
So, yea, we are publishing v2 to latest i guess :tada: :rocket:

Added VERSION_NOTES on the master branch
updated some badges and stuff on the README

Ok
Ok So master is 3.x So we merge https://github.com/node-formidable/formidable/pull/767 now ?
Dist tags are like pointers that point to a specific version, we can only deprecate versions.
Ok I will publish

Published 1.2.4
Published 2.0.0 as latest

Okay, great.
v3 is master, yea. But I don't know what is in v2 in git, what is in v2 in npm, what's the diff with v3, and what we should move as the v2 to a separate v2 branch. :rofl: But anyway, we will figure it out. Just publish v2 to v2 dist-tag too, and your v3 (#767?) to v3 dist-tag.

Oh and rename the dist-tags on npm, because notes mention v1, v2, v3 and not 1.x, 3.x.

Created a 2.x branch from master, then merged 3.x branch into master

Should I drop "canary" ? Since we already have tags ?

Published<EMAIL_ADDRESS>in v3 tag (with your VERSION_NOTES and updated readme)

Tried v2 tag but

Tags that can be interpreted as valid semver ranges will be rejected. For example, v1.4 cannot be used as a tag, because it is interpreted by semver as >=1.4.0 <1.5.0. See #6082.

I don't know why it worked with the v3 tag

We could do like npm https://www.npmjs.com/package/npm latest-1 latest-2 latest-3 etc

duh.. :rofl: well, okay. and instead of canary we will publish to next-2 and next-3 for preview versions.
but should update all the notes first haha..

Should I run this command ?
npm deprecate<EMAIL_ADDRESS>"Please upgrade to latest formidable@latest or formidable@v3 versions. Check these notes: https://github.com/node-formidable/formidable/blob/master/VERSION_NOTES.md"

So far we have dist-tags:
1.x: 1.2.4                  latest: 2.0.0
3.x: 3.1.1-canary.20211030  v3: 3.1.2-canary.20211030

Should I run this command ?

if <2.0.0 works to deprecate all v1, yea.. but i'm not sure.
plus you should<EMAIL_ADDRESS>to deprecate the canary usage

So far we have dist-tags:

delete 3.x and v3

Tried v2 tag but

By the way, did you tried v2.x or just v2 as dist-tag? It should work if don't have .x.. hmmm

Deprecated all the 1.x and canary versions

If I delete how are people going to install v3 ?

I tried just v2

Ok so npm i<EMAIL_ADDRESS>npm ERR! code ETARGET
npm ERR! notarget No matching version found for<EMAIL_ADDRESS>So 3.x is also invalid. and 2.x also.
I think it worked because the argument was passed from package.json (publishConfig)
So I rename to next-3 and latest-2 and latest-1 or ?

Hmm... their tag naming doesn't work well to us, hm.
v1
Move v1 to legacy dist-tag. Reserved for v1 specifically, formidable@legacy will always install just v1.
v2
Now on latest, AND v2-latest (should work cuz it's invalid version range).
Future not ready for release updates/bugfixes and things will be on v2-next, like the -canary until now.
v3
Move v3 to v3-latest and future not ready for release will be on v3-next, like the -canary until now.
We still can add the dates to the -next tags, like -next.20210312.

Renamed the branches to follow that. You should update your local copies with
for v1
git branch -m 1.x v1-legacy
git fetch origin
git branch -u origin/v1-legacy v2-legacy
git remote set-head origin -a

for v2
git branch -m 2.x v2-latest
git fetch origin
git branch -u origin/v2-latest v2-latest
git remote set-head origin -a

and for v3, copy the current master to v3-latest

Okay, i got that.
latest: 2.0.0
legacy: 1.2.4
v2-latest: 2.0.0
v2-next: 2.0.0
v3-latest: 3.1.1-canary.20211030
v3-next: 3.1.1-canary.20211030

I'll fix the notes and bump the versions. In the *-next dist-tags we will publish prereleases.

Switched all the tags and published all the versions. Seems like what you're saying is right, it works when it's on the publishConfig.tag so i switched them all to have such so we can stop thinking about --tag on npm publish. That probably should work with automation too. Well, i have some concerns cuz I built several tools that are around automation and they were using --tag. But anyway we won't have that much automation here, except Renovate which I should fix too, it seems.

Nice, but we should remove v1 v2 and v3 tags from npm , because you think that installing @v3 will install from that tag, but because it is also a valid version range (as per npm) it will convert it to @>3 & <4 (And it will be even more confusing since it will look like it works initially)

Ohhh yea, got that.. was just thinking about that.. ooooh fack.. :rofl:

Okay... as I thought before few hours.

for v1: legacy branch - legacy dist-tag
for v2: master branch - latest dist-tag
for v3: next branch - next dist-tag

When v3 should become latest on npm, we cut the master branch to v2 branch and merge next branch into master. And the next branch and dist-tag becomes for v4.

Okay lets close this, continue here https://github.com/node-formidable/formidable/issues/769


hombros failing on vagrant up - fix ssh

Hombros is failing on vagrant up:
Recipe Compile Error in /tmp/vagrant-chef/411882505a992344385486cbec4abb0b/cookbooks/hombros/recipes/default.rb
==> hombros: ================================================================================
==> hombros:
==> hombros:
==> hombros: Errno::ENOENT
==> hombros: -------------
==> hombros: No such file or directory @ rb_sysopen - /home/vagrant/.ssh/id_rsa
Apparently the key needs to be present on compile, but the assumption is that the shared default recipe would populate.
Going to refactor so that the default script is run on the base image, which is pretty well behaved and I think it's OK to make it a requirement that the base image be created first.
Not sure why this is only surfacing now, probably previous builds were not starting from as clean a slate as I thought (/mnt/shared/keys not empty)

should really refactor shared:default into something more explanatory and just for base

fixed in corresponding branch.


Fix #963 - Add extra friend logs to the end of the list instead of the front

Friend logs that were not already in the list of logs were being added to the front with use of position variable. Removing it adds the friend logs to the end.
If friend logs were found, but not already in the list, it is because they are older, so it is correct to add them to the end of the list.
Fixes #963

I'm still learning how to manage my commits. If I need to redo this to prevent so many commits showing up, please let me know.

When you prepare something for a pull request, the best way is to create a new feature branch based on upstream/master and then add your commit there.

Thanks for the fix - reverse order, damn'.


Split scheduling pass into scheduling and padding

Now functionality of scheduling pass is split into two pieces; scheduling (analysis) and padding (transform). In between these operations, one can insert non-transform operation such as alignment (i.e. reschedule).

Summary
This PR splits functionality of scheduling passes into scheduling (analysis) and padding (transform). This is beneficial since one can interleave alignment passes, i.e. rescheduling, in between these operation without regenerating DAG. In addition, scheduling operation creates time_slot dictionary in the property set of the pass manager. This allows us to easily track instruction start time without accumulating instruction duration from the begging of the DAG wire.
Details and comments
Note that this is breaking API change since now scheduling passes insert no delay. In addition, consecutive delays are now merged into a single delay, which is necessary to correctly evaluate alignment constraints (will be in follow-up PR). Because it is not easy to dynamically switch base class (analysis -> transform pass) just for backward compatibility (technically possible though), current approach is to allow this API change and to write enough information in the release note. Since scheduling pass is used only in the specific situation (e.g. noisy circuit simulation, evaluation of alignment constraints, dynamical decoupling), the impact of this change is expected to be limited.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

88 of 93 (94.62%)  changed or added relevant lines in 11 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.007%) to 83.444%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

qiskit/transpiler/passes/scheduling/base_scheduler.py
7
8
87.5%

qiskit/transpiler/passes/scheduling/padding.py
54
55
98.18%

qiskit/transpiler/preset_passmanagers/level0.py
2
3
66.67%

qiskit/transpiler/preset_passmanagers/level2.py
2
3
66.67%

qiskit/transpiler/preset_passmanagers/level3.py
2
3
66.67%

Totals

Change from base Build<PHONE_NUMBER>:
0.007%

Covered Lines:
52449

Relevant Lines:
62855

💛  - Coveralls

Thanks @itoko! It might be hard to grasp the entire picture of the refactoring only with this PR, but align measure and align pulse are the constraints on t0. The main target of time_slot (now it's renamed to node_start_time) is indeed these alignment passes in which currently t0 are computed by integrating duration of all ancestor nodes on the wire. Thus having start time rather than idle time makes the logic much faster.
I was also thinking to store (t0, t1) tuple in the property set, but I decided to store only t0 since t1 = t0 = node.duration (having both is bit redundant).

Oh, I missed that. You're right. Alignment constraints are about t0 (not duration), so we need to store t0s even if it is not directly used in Padding passes. Also, not storing t1 (and prev_node as well) sounds reasonable to me.


Call to undefined function Sober\Controller\add_action()

Just installed controller using
composer require soberwp/controller:9.0.0-beta.3
and getting the following error across the whole project
Error: Call to undefined function Sober\Controller\add_action() in /srv/www/myproj/current/vendor/soberwp/controller/controller.php on line 36

Hi @iljamm
That is strange, I will investigate today and let you know.

Sorry my bad, i installed in a wrong composer file. When I required it in the theme's composer all worked as expected. May be it could be made a tad more prominent in the documentation, which composer file should be used. I realised you said that in the following sentence:
"Please note that Controller is no longer an mu-plugin and is now a Composer theme depedency."
IMHO addition of dummy path to the file would prevent such errors in the future.

Thanks, noted, I will update the README file to make it more obvious.


Add demo URL

The README.md is great. Don't forget to add a URL to the playable demo.

[ ] Add URL to README.md
[ ] Add URL to GitHub repo description. Click Edit on the repo homepage to see the URL field.

The screenshot links to the demo. I Guess I need to make it more obvious

Ah, I think I did not expect to find the link there because images on GitHub READMEs by default link to the image file itself.


Image started failing on Fedora 32 since homeassistant/home-assistant:0.107.0

Some change occurred between homeassistant/home-assistant:0.106.0  and homeassistant/home-assistant:0.107.0  breaks the behavior on Fedora32 x86_64.
I've filed a bug report against podman but I'm not sure if the problem is on this or their side, considering that 0.106.0 starts up fine.
https://github.com/containers/libpod/issues/6185
https://gist.github.com/paoloantinori/94ef7334005f8efd650180570cc1c490
The issue might be in the base image though:
homeassistant/i386-base-python:3.7-alpine3.11 throws the same exact error.

Hi there, thanks for opening up an issue.
Unfortunately, we currently don't support podman. It might be caused by the rootless nature.
I hope you'll find an answer to your issue. 👍