[RFC] Structured Concurrency

Crystal has an great concurrency model based on Fibers and Channels, which can be used to pass messages around.
Fibers are conceptually pretty simple. If you spawn one, it takes off from the main context and runs concurrently for an indefinite amount of time, depending on what it does and how long it takes to do this. From the perspective of the main control flow, it's essentially fire and forget.
Real life problems typically ask for a more sophisticated way of handling concurrent tasks. Sometimes you need to wait for either one, some or all tasks to be finished before continuing the main scope. Error handling in concurrent tasks is also important and the ability to cancel the remaining tasks if others have finished or errored.
Fibers and Channels can be used to implement a model for structured concurrency. Given that this is a pretty common idiom, I'd like to see a generalized implementation in Crystal's stdlib.
What we have
HTTP::Server#listen uses a custom implementation of a wait group executing a number of tasks simultaneously and waiting for all to finish. Other examples are in the parallel macro or Crystal::Compiler#codegen_many_units. parallel is the only feature of structured concurrency currently available in the stdlib, but it is only suitable for a fixed number of concurrent tasks that are known at compile time.
A more generalized approach would help to make this concept easily re-useable.
It can be implemented based on the existing features that Fiber and Channel provide. The only thing that's missing is a way to deliberatly kill fibers and unwrap their stack (see #3561, and a proposed implementation in #6450).
Background
I recommend reading the articles referenced below. They both describe a model of structured concurrency which essentially restricts the execution of concurrent tasks to a specific scope and having tools to manage them. This contrasts with the model of go (Go) and spawn (Crystal) which just fires off a new fiber without caring about it's life cycle. This makes it hard to follow control flow: what happens where and when in which scope.
The main idea of this proposal is to understand that each fiber is limited to the scope it is executed in:

Every time our control splits into multiple concurrent paths, we want to make sure that they join up again.

This ensures that fibers don't get lost doing whatever stuff they might not even be supposed to do anymore.
I believe this concept can be applied to almost any real-life use case of fibers.
Having a structured flow of control also allows for a proper exception flow. Right now, unhandled exceptions within a fiber are just printed and ignored. When a fiber is scoped to some parent context, an exception can just be propagated there.
Prototype
I have implemented a simple prototype of a concurrency feature (based on Fiber.cancel from #6450). The idea is to have a coordination tool for running fibers, called a Spindle. It is used to spawn fibers and ensure to collect them. This particular implementation allows running multiple tasks concurrently and if one of them fails, it cancels all the others. This is of course just an example of behaviour, there are many different ways to react.
The code can be found at: https://gist.github.com/straight-shoota/4437971943bae7000f03fabf3d814a2f
I don't have a concrete proposal how this should be implemented in terms of stdlib API's but the general idea is to provide tools for running tasks concurrently. We could even think about removing unscoped spawn (it can be considered harmful after all), but that's not necessarily required and can probably be decided upon later.
References

Notes on Structured Concurrency, or: Go Statement Considered Harmful by Nathaniel J. Smith
Structured Concurrency (by Martin Sústrik)
Python Concurrency for Mere Mortals (talk) by Nathaniel J. Smith

Some examples of similar libraries:

Go: WaitGroup
Rust: CrossBeam
C: libdill
Python: Trio
Ruby: Polyphony
Kotlin: kotlinx.coroutines

Just a note that everything in the std related to concurrency, except spawn (that is, parallel, future, etc.) are just small examples to test whether concurrency work, and in my opinion they should also be deleted and rethought from scratch for a unified way to do concurrent programming in Crystal. I mentioned in the past that WaitGroup is on the roadmap (apparently someone edited that wiki and removed that, sigh), it's just that implementing all of these patterns without parallelism already there is almost a waste of time because implementing them with parallelism in mind is totally different, I think (at least if you want it to perform well; for example Go's WaitGroup doesn't use channels and it's super efficient.

I'd rather write a "good" Promise implementation and then have Promise.all/Promise.first etc. As far as i'm concerned the abstraction of a fiber with a return value, called a promise or future, is a better and more composable than waitgroups and can still be used as waitgroups pretty easily.

Promises are more about communication than structuring. You can do a lot of things in a similar way, but it feels like an inferior solution to me.
Running two methods concurrently would look like this with a promise:
promise = future ->do_some_stuff
do_some_other_stuff
promise.get

In my example implementation it looks like this:
concurrent do |spindle|
  spindle.spawn ->do_some_stuff
  spindle.spawn ->do_some_other_stuff
end

It's  little bit more verbose, but easier to follow control flow. Especially when it gets more complicated.
(Verbosity could actually be reduced when using with spindle yield, but I think I like the expressiveness better.)
One way or the other: a good promise implementation would need a way to cancel a fiber as well, wouldn't it?

@straight-shoota you could do it like that or you could use promises like this:
Promise.all(
  Promise.new ->func,
  Promise.new ->func2
)

which is both less verbose and more powerful since you can do
value1, value2 = Promise.all(
  Promise.new ->func,
  Promise.new ->func2
)

very easily to get the function's return values.
Changing it to "first fiber wins" aka a race becomes s/all/first/, just changing one function instead of having to refactor your code to use a different concurrency mechanism (consider the difference between a function taking a spindle as a param and returning a promise, thats a big refactor)

and JS promises have no way to cancel them, we don't need cancellation for them

These Promise.all and Promise.first are nice, but they only allow a static number of concurrent tasks. This is just like the parallel macro and not sufficient for a real-world application where tasks can be dynamically added at runtime (like connection handlers of a network server for example).

At least I would want to have a way to cancel a promise/fiber (and do it gracefully) in case its result/effect is no longer required. It shouldn't block resources for unnecessary work.

@straight-shoota No, .all and .first can easily take arrays.

How would that work with adding child tasks during runtime? You would have to append them to the array - somehow that could probably be made to work with .all and .first. But it makes a really ugly API using an array as registry for tasks.

Ah I see what you want. We could easily add block-based versions of Promise.first and Promise.all which do what you want, but I can't help but feel that adding tasks at runtime would be hard to track.

I was thinking more of a
Promise.all do |promises|
  promises << ...
end

interface in addition to the other promise interfaces to make it do wait groups as well.
I guess it's probably quite a hack though, and better to separate the two concepts.

@RX14 That doesn't seem much different from
concurrent do |spindle|
  spindle.spawn ...
end

So we pretty much want the same thing in this regard ;)

@straight-shoota yeah I like the concept I was just wondering if we could work it into the "promise" concept for simplicity.

I've been writing a Promise library for Crystal Lang and it's almost complete (core implementation complete with specs)
https://github.com/spider-gazelle/promise
It might not be "Structured Concurrency" in the most strict sense however it does simplify coordinating a bunch of async events and it's quite a popular paradigm. Would love to see it in the standard library if that's something you would consider.
I think my implementation is pretty neat in any case:
require "promise"

promise = Promise.new(Int32)
result = p.then { |result| result.not_nil! + 100 }.then { |result| "change #{result} type" }
promise.resolve(10)

# Can also grab the value via a future
result.value # => "change 110 type"

Any .then block can change promise types as it propagates down the chain.
.catch blocks can only propagate exceptions and are used to recover values back to the initiating promise value to continue.
Thanks to the Crystal type safety it puts most promise implementations to shame.

I don't think a promise library in crystal would look anything at all like a JS promise library. .then isn't really required at all. Libraries should expose synchronous APIs, and any uses of Promise should really stay internal to the application.

Well yeah (exposing synchronous APIs)
Not sure how you escape from using the equivalent of .then though it's kind of the je ne sais quoi of promises.  I'd like futures in the stdlib and promises perhaps can be a shard built on that.

I'm still not sure what the conceptual difference between a promise and future would be in crystal though :thinking:

I like to think of promises and futures as either end of a pipe, the promise is where I can put something in and the future is where I can wait and listen for the result.
Now taking this picture, I just described a Channel with a buffer capacity of one that can only be written to once. So perhaps for us Promises would actually be just redundant to channels.

Promises are really complimentary to channels. Channels for distributing async work and promises for returning the results.
For instance, I'm working on updating this influxdb library to the latest version of Crystal lang as the original maintainer doesn't have the time.
The original version just spawns to perform an async request (a channel here would be ideal) - however either way you can never know if the request succeeded and you don't always want to wait around at this point for the response.
A Promise in this case is the perfect solution.

You can ignore the result like the current implementation
You can be synchronous if you want to be Promise#value
You can use a call back to handle a success or failure condition

Basically promises provide the ultimate flexibility and are very simple to use

Well, regardless of the terminology what I want is a fiber which can return a value, and you can wait for it to complete with a value or error

@stakach I don't think a promise is the right tool in your example. The library method #write should just be blocking like any  Crystal IO method and directly call #send_write. It should be the caller's task to manage concurrency and execute this method in a separate fiber where appropriate. The library simply doesn't need to care about that.
This would be idiomatic Crystal with a simplified API by removing sync argument and a dependency on a promise implementation. It's consistent with other IO methods, including others on the same type (select, query, drop) don't seem to have an option to execute asynchronously.

@straight-shoota problem is running the write concurrently doesn't work on the current version of Crystal so it has to be via the channel to ensure serial writes.
Basically because the HTTP client response from influx is chunked, crystal yields the current fiber while it waits for IO mid HTTP response cycle.
Then I can start another request while the client is still processing the previous response which leads to weird errors. (not sure if this is an issue with the crystal lang HTTP::Client however if you take that influxdb library, make the minor changes to run on 0.25.1 and run the tests, it blows up) - this is a common issue with fibers and the library should definitely deal with this not the caller.

The problem of being called from a different fiber is already solved by using the channel - any fiber can call the code, the channel does the work and the promise returns the result.
Very little complexity vs locks and multiple client instances

@RX14 I deliver you "a fiber which can return a value, and you can wait for it to complete with a value or error"
https://github.com/spider-gazelle/promise#simple-concurrency-control

@straight-shoota thanks for the advice - I threw a lock around the influxdb HTTP client, you were right, was definitely the way to go.
Also watched the Trio video does seem pretty cool. Look forward to Spindles landing!

Notes on Structured Concurrency, or: Go Statement Considered Harmful by Nathaniel J. Smith

He overlooks the problem too much and the metaphors between one domain and the other are forced to match his worldview.

Structured Concurrency (by Martin Sústrik)

This article is nice.
Just two notes:
GC vs systems language
GC languages and systems-languages (i.e. languages where the sole runtime is the execution stack itself and you have to care about pointer and lifetimes) behave differently in such situations.
Thread-start functions (or, fiber-start functions) in GC'ed languages will be closures rather than actual plain functions and, if some value is captured from parent scope, these values won't be destroyed when the parent dies. So we don't usually worry about child threads/fibers overlapping the lifetime of the parent thread/fiber (just like what happens in Go's). If you need deterministic destruction of objects, just be explicit about it. pthread_cancel and pthread_join will do in C. Extending the use-case to the world beyond C, you'll probably follow Java's approach and change cancellation API to throw exceptions and triggers stack unwinding on the target thread/fiber. If you have C++, there will also be RAII scope guards properly handling cancellation requests (and I bet RAII is already outside of Crystal's scope).
Don't complicate the state machine
This author too fancies about abstractions. There are two strategies here: graceful vs ungraceful shutdown. Interruption/cancellation API is about graceful shutdown. Don't fancy about timeouts and stuff. Ask the thread/fiber to gracefully shutdown and give it its time.
Want ungraceful shutdown leaking resources and possibly leaving live locks and breaking invariants on data structures? As a rule of thumb this is bad design and you don't want to turn into an idiom culturally seeded on your language. Two approaches here would be:

Spawn a real process (os.exit() won't clean any resources) with a greater level of isolation and kill the process when it fails to satisfy an associated watchdog.
kill() that unschedules the fiber forever as soon as scheduler has a chance. Marked with a BIG BOLD “don't use it if you don't know what you're doing” in the documentation. No fancying about timeout, just a simple API. If you want to give time for the fiber to finish, it's cancel() + timedjoin() (the only API specifically designed for dirty shutdown here is kill() and all others are a fundamental vocabulary that emerged elsewhere).

Is this impacted (e.g. made easier, harder, or superseded) in any way by the proposed MT support? Even if it's orthogonal, as suggested here, it would be nice to have eyes on the MT proposal with compatibility (or non-incompatibility) with this proposal in mind.

@chocolateboy Structured concurrency does not depend on multithreading, but it can easily integrate it. A concurrency context allows to configure very specifically whether tasks can be executed in parallel, how many threads, error handling etc.

I suppose only a limited version of this would be possible after 1.0, as a unlimited ability to spawn without a lifetime restriction isn't really compatible with the concept.

@yxhuvud Not sure I understand.
I'm convinced that when a tool for structured concurrency is available, ideally there should really be no reason to use a random anywhere. Best case scenario would be to avoid that completely. But you should still be able to have both, structured and unstructured fibers.
I don't see a reason why unrestricted spawns wouldn't work with a concept of structured concurrency. Technically, the structure would be build upon the unstructured primitives anyway.
Removing spawn as top level method would be possible when there's a different tool for spawning fibers in context. But it's not strictly necessary.

The way I see it, spawn coupled with Channel and select are the low-level concurrency primitives in Crystal. In fact, they are the same in Go. I see them similar to Erlang/Elixir receive and ! (send). People don't usually use them, they use abstractions on top of them (like GenServer), but from time to time they are useful... especially for building more abstractions.
So we should keep those in Crystal, and build good abstractions on top of them. For instance, actors would be nice.

"I don't see a reason why unrestricted spawns wouldn't work with a concept of structured concurrency."
Because it defeats the point. Users should be forced to think about the lifetime of the things they spawn, and it should possible to see using visual inspection of the code what fibers can be running at any given point in the code. Or to quote the referred blogpost referred to above with the (quite telling) title notes-on-structured-concurrency-or-go-statement-considered-harmful :
The good news, though, is that these problems can all be solved: Dijkstra showed us how! We need to:

-   Find a replacement for go statements that has similar power, but follows the "black box rule",
-   Build that new construct into our concurrency framework as a primitive, and don't include any form of go statement.

The whole point of that whole post is a denouncement of the Go model with unbounded spawns!
And yes, fibers would obviously still be implemented using low level constructs. But probably used as often as Thread.new by end-user applications (ie never). The easiest way to create them (spawn) should not be the one with the worst long time maintainability properties. Hence my comment about a limited implementation (similar to how C still has goto).

Okay, I see what you mean. There's no technical reason, but it would make sense to advocate the higher level interface. And frankly, assuming its used in every API, there would be no reason for direct access to the low level interface.
I just feel that can only be achieved as a second step. The first step would be to design and implement a high-level interface for structured concurrency. Then wait for it being adopted. After that, the now (hopefully) unused features can be removed/reduced.

I would like to add presentation about structured concurrency in context of C++ to this conversation. Memory management doesn't apply to Crystal, but allocated resources could be external, e.g., database connection. Hence, the presentation might help people to see how resource allocation can be done safer with structured concurency.

Structured Concurrency: Writing Safer Concurrent Code with Coroutines... - Lewis Baker - CppCon 2019

In addition, I like how Martin creates connection between object lifetime management and fiber lifetime management (see Martin Sústrik's blog post linked in issue description).

Interesting. The concept of Executors confuse me. I can understand the wrap of the Job around a Task, but the way the Executors is expressed seems over-complex to me.
I've played with the Task in C#, very close to the concept of future in C++ materialized by the Async template. For the two, a lightweight process is spread in a thread-pool. We can grab the result later and continue the our work. The way Task is used in C# permit to organize the work (when it start, when we need to wait), perform check on the task and act on failure.
The other way is to have a long running process and use the classical channel / subroutine as in Go or Elixir (with the Mailbox / Job primitives). A bit light for more advanced management, but everything is possible.
A very interesting topic anyway :) Thanks a lot.

There's also discussion of go concurrency bugs and structured concurrency. Some bugs were caused by unbounded go statements in the golang stdlib.
I think structured concurrency should be provided by the language as primitives and the stdlib should always use it.
Kotlin coroutines is a very good implementation of structured concurrency. I think Crystal should have something similar to it.

Ruby implementation: Polyphony (discussion)

@straight-shoota even Java has Structured Concurrency now. Now there is one reason to use Java over Crystal, how long are we going to allow this?


fix(chart-data-api): improve chart data endpoint errors

SUMMARY
TL;DR: this makes viz plugins behave in the same way as legacy viz plugins when the request fails or returns no data.
This fixes the following problems in the chart data endpoint:

Errors were not picked up from QueryObjects when calling the chart data endpoint. This meant that erroneous requests were returned as 200s with an error-field that was not handled in any way, causing the chart to render in unexpected ways. Now the error field is is checked for each response dataset; if a single one is defined, a 500 is returned (these are mostly expected to be logical errors in the backend code).
if no data was returned, the chart would not render, leaving the old chart on screen, potentially even an incorrect chart type. This was due to the legacy endpoint returning a 200 with empty data, as opposed to the new endpoint returning a 200 response with an error field but without a data field. To harmonize behavior, QueryContext was updated to return a non-error response with an empty data field for empty result sets.
The error message from erroneous requests were not displayed, making it difficult to see in the frontend what the problem was.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
AFTER
When an incorrect request is performed, the error message is displayed, making debugging easier. Previously only "An error occurred." was displayed. Note that this error will be fixed by #10299

When no data is returned, the chart data endpoint returns a response that the Chart component can properly identify as an empty response. Previously this left the previous rendered state visible.

TEST PLAN
Local testing.
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

Codecov Report

Merging #10300 into master will increase coverage by 0.16%.
The diff coverage is 71.42%.

@@            Coverage Diff             @@
##           master   #10300      +/-   ##
==========================================
+ Coverage   70.42%   70.58%   +0.16%
==========================================
  Files         599      599
  Lines       32067    32083      +16
  Branches     3244     3245       +1
==========================================
+ Hits        22583    22646      +63
+ Misses       9380     9333      -47
  Partials      104      104

Flag
Coverage Δ

#cypress
55.53% <0.00%> (+0.06%)
:arrow_up:

#javascript
59.57% <0.00%> (-0.01%)
:arrow_down:

#python
69.76% <83.33%> (+0.26%)
:arrow_up:

Impacted Files
Coverage Δ

superset-frontend/src/chart/Chart.jsx
65.30% <0.00%> (-1.37%)
:arrow_down:

superset/charts/api.py
73.19% <75.00%> (+0.03%)
:arrow_up:

superset/common/query_context.py
80.12% <100.00%> (+0.38%)
:arrow_up:

superset/charts/schemas.py
100.00% <0.00%> (ø)

superset/connectors/sqla/models.py
88.95% <0.00%> (+0.14%)
:arrow_up:

superset/views/core.py
74.64% <0.00%> (+0.24%)
:arrow_up:

superset/db_engine_specs/base.py
83.98% <0.00%> (+0.28%)
:arrow_up:

superset/jinja_context.py
80.95% <0.00%> (+0.95%)
:arrow_up:

superset/views/database/views.py
62.98% <0.00%> (+1.65%)
:arrow_up:

... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 93753cf...f7f4c1b. Read the comment docs.


Provide support for services

Tasks

[ ] Implement service side rmw methods
[ ] Implement client side rmw methods
[ ] Update graph cache accordingly
[ ] Implement methods related to services introspection

Some comment related to zenoh queryables and ownerships:
Context
Following RMW implementation if using queryable:

The queryable is declared at rmw_create_service
A query is analized at rmw_take_request
A query is answered at rmw_send_response

Issue
In this example: https://github.com/eclipse-zenoh/zenoh-c/blob/master/examples/z_queryable.c#L28-L42
The callback for the queryable receives a z_query_t pointer.
The method that is used to send a reply is: z_query_reply (For what I saw from docstrings: "This function must be called inside of a Queryable callback passing the
query received as parameters of the callback function" )
After double checking with Zettascale folks: using the z_query_t outside of its callback is UB, as said query is destroyed. It is in the roadmap to support this API.
Alternative
We could move forward using publishers and subscriptions for service support, typically how it is done for the dds-based implementations, and upgrade to use Queryables later on when supported.

We could move forward using publishers and subscriptions for service support, typically how it is done for the dds-based implementations, and upgrade to use Queryables later on when supported.

I don't love the 2 topic solution (it leads to races), but this is an OK workaround for now.  Once we get the queryable-handle support later, we can switch to it.
The only thing I'll suggest is that we design the API so that we can easily switch to the queryable-based solution in the future.

It is in the roadmap to support this API.

Actually owned queries are coming very soon in zenoh-c: https://github.com/eclipse-zenoh/zenoh-c/pull/213
And will come later for zenoh-pico.

It is in the roadmap to support this API.

Actually owned queries are coming very soon in zenoh-c: eclipse-zenoh/zenoh-c#213 And will come later for zenoh-pico.

Awesome, thanks for tackling this quickly

@Yadunund , checking to see if services are now supported? I see #86 and #88, both of which seem promising.

@Yadunund and @francocipollone , checking to see if services are now supported? I see #86 and #88, both of which seem promising.

Sort of.  They work in their most basic form, but they are pretty buggy right now.  #90 adds in graph introspection for services.  I'm working on some additional fixes on top of #90 which improves the reliability of them.  Hopefully after that they will be closer to "done".

We've fixed all of the known issues with services right now, so closing this.  Any other problems we can ticket separately.


dcp: remove container after copying

Cleans up after copying succeed - uses the force option by default

Closes #16

I tried to write a test, but it's not entirely evident how to get the container id out from the logs. It's possible. I think it's ok to merge this one without a test.


Reproduce baseUrl + .js extension with .ts source

Besides, .cjs should try .cts, and .mjs should try .mts also.

https://github.com/privatenumber/get-tsconfig/blob/96a5c10bed2750794ac720ebf7a1dd7d5169a06e/src/paths/index.ts#L61
And why relative paths are always ignored? I'm not quite sure to understand.

./foo should also try foo/index.ts, ./foo.js should also try foo.js/index.ts 😂
For more test cases, please see https://github.com/alexgorbatchev/eslint-import-resolver-typescript/blob/master/tests/withJsExtension/test.js


Upgrade jQuery

On my fork of RIDE I have activated Whitesource-Bolt app, and it detected a vulnerability on JSQuery package included in robot library.
This was in robot version 3.1.1.
Please see Issue 53 in my RIDE fork.

Luckily doesn't look too serious. Should anyway upgrade jQuery to 3.4.0 that has this issue fixed in RF 3.2.

The security issue is reported as CVE-2019-11358. As I wrote above, it doesn't look that serious for us but is definitely worth fixing by upgrading jQuery. The current latest version is 3.4.1 and it seems to work fine with our logs and reports.


Errors in docs

There is no "create experiment" when right clicking

There is no option to "run locally" when right clicking

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d4fd070b-ec93-be29-9361-302d383d6724
Version Independent ID: b1854f9d-3426-104e-e939-b077075544ac
Content: Interactive debugging with Visual Studio Code - Azure Machine Learning
Content Source: articles/machine-learning/how-to-debug-visual-studio-code.md
Service: machine-learning
Sub-service: core
GitHub Login: @luisquintanilla
Microsoft Alias: luquinta

Thanks for the feedback! We are currently investigating and will update you shortly.

Hi @aronchick
Thanks for raising this issue. Sorry to hear you ran into an issue. I believe the reason you're running into this issue is the guide uses the 1.0 CLI and as of the latest update, the VS Code AML extension uses 2.0 CLI which does not yet support the local run / debug experience.
To switch to the 1.0 CLI, set the azureML.CLI Compatibility Mode setting in Visual Studio Code to 1.0. For more information on modifying your settings in Visual Studio, see the user and workspace settings documentation.
I have a PR which should clarify this point.

Thanks - just close this then.


Add filter on resolved/unresolved status to Inbox and Recent conversations

This should probably happen once we build a multi-select filter widget.

Shall I do this? Can you please assign this to me

@zulipbot  claim

@zulipbot claim


Typescript migration

This migrates all src lenis files to typescript and exports the types needed to define stuff like callbacks, etc.
I decided against making userData generic as its to loose to type and different data can be passed in on different occasions.
Some stuff still needs to be documented in the class instances but first we need to define what should be public and whats internal. Left TODOs on those. Else this should be good to go.

published in v1.1.10-dev.0


health checks markers

Description:
Describe in a couple of sentences what this PR adds
Merge checklist:

[ ] Changelog record added
[ ] Documentation updated (not only docstrings)
[ ] Commits are squashed

closing this PR as following two PRs has been created in favour of this:

PR 1 has checks only:
https://github.com/openedx/edx-repo-health/pull/461
PR 2has Configs only:
https://github.com/openedx/edx-repo-health/pull/462


IML changes to build from within IDEA v14.1.1

Marked src/14.1 as source folder, and marked src/14, src/13.1 as excluded folders
Removed stale entries to source/test folders 13.1.6, 14.0.1, build_tests

@EBatTiVo Can you review & approve?


Delete key binding

Add a Delete key binding for removing shapes

Hi, I did some workaround to try having a (Ctrl + x) key binding for this -
  componentDidMount() {
    document.addEventListener('keydown', this.handleKeyDown);
  }

  componentWillUnmount() {
    document.removeEventListener('keydown', this.handleKeyDown);
  }

  handleKeyDown = (event: KeyboardEvent) => {
    if (event.ctrlKey && event.key === 'x') {
      this.removeSelectedObject();
    }
  };

  removeSelectedObject = () => {
    const { selectedNodes } = this.state;
    if (selectedNodes.length === 0) {return;}

    const objectId = selectedNodes[0]; // Assuming only one object is selected

    const sharedModel = this.props.cpModel.jcadModel?.sharedModel;
    if (!sharedModel) {
      return;
    }

    const dependants = sharedModel.getDependants(objectId);

    let body: React.JSX.Element;
    if (dependants.length) {
      body = (

          {'Removing this object will also result in removing:'}

            {dependants.map(dependant => (

{dependant}

            ))}

      );
    } else {
      body =
Are you sure?
;
    }

    showDialog({
      title: `Removing ${objectId}`,
      body,
      buttons: [Dialog.okButton(), Dialog.cancelButton()]
    }).then(({ button: { accept } }) => {
      if (accept) {
        const toRemove = dependants.concat([objectId]);
        sharedModel.removeObjects(toRemove);
      }
    });

    this.props.cpModel.jcadModel?.syncSelected({});
  };

And it works, I just copied the body of onClick() inside a function removeSelectedObject, of the remove object button in the ObjectTree component & invoking that on KeyboardEvent. Again this is a rough approach which doesn't follow DRY principle yet but I can work on standardizing the same if this is what is expected by the delete key binding.

you might want to take a look at this PR https://github.com/QuantStack/jupytergis/pull/58. jupytergis and jupytercad share a lot of codebase

Thanks, this helps a lot.


Enable correct representation of components in stretched layer 2 VLAN

Issue type
[ ✓] Feature request
[ ] Bug report
[ ] Documentation

Environment

Python version:  2.7.6
NetBox version:  2.2.1

Description
Whilst netbox has the capability to represent "global" vlans, it lacks the ability to correctly represent a stretched layer 2 network, i.e. where one or more vlans live in one or more datacenters, and may share a vlan id with other local or stretched vlans from the same or other tenants at the same site (in totally physically segregated networks, of course).
This disability extends to the components living in the datacentres, e.g. its not possible to represent that a virtual machine lives in a cluster that exists in multiple sites on a stretched layer 2 vlan; instead you have to use workarounds like duplicate resources at each site or other permutations/combinations of entering in fake data.  From the location of a VM, the cluster it lives on, the IP network it lives on, the VLAN its in, etc. all permit only a 1:1 mapping with "site" and "tenant", therefore you're forced to duplicate entries to work around the problem.
The "global" vlan described by #235 doesn't exist in practical use (but would be supportable under this issue anyway), and unfortunately other attempts like #852 were incorrectly closed as a duplicate of the former.  The core difference to #235 is the 1:many relationships with sites, it's not just 1 site or "all sites".  For geolocation and other reasons this problem also extends to tenants.
TL;DR: sites and tenants should accept 1:many relationships when used in VMs, clusters, VLANs, prefixes, ...

ping @stefanjagger @Armadill0 @Censored3 @candlerb @rdujardin (and anyone else with more than one datacentre ;)

The fact that things can only be assigned to a single "Site" or "Tenant" (or to none) hasn't been a major stumbling block for me.  I would choose to create a single object associated with none, rather than duplicate the object.
Right now I am using "Tenant" to represent "major service group", and there are cases where one device/VM is part of two specific services, but not global.  Hence I could make use of many-to-many Tenant relations if they existed.
However for me, it would be much more useful if "Role" could change into something like a tag or label, where I could assign multiple Role labels to the same Device or VM.  There are a ton of uses I could find for that (e.g. which major services are affected by this device; what monitoring tools to configure for it...) and I'd no longer need to mis-use Tenant.  I don't know if there's a ticket for that idea.
I would not use many-to-many relations between (VLAN|Prefix) to Site.  In fact, in the current model, it annoys me that both VLANs and Prefixes both have to be separately associated with Site and Tenant, so there's lots of duplicate entry already, which would only get worse if this became many-to-many.
I would prefer that the Prefix can be associated just with a Tenant, and the VLAN can be associated just with Site(s).  But that's a different issue.

The trouble with not associating is that you can't drill down from a site level or perform any kind of automation based on that grouping via the API.  Its defeating the purpose of functionality like http://netbox.readthedocs.io/en/latest/data-model/extras/#export-templates
I have multiple sites, two of which have some spanned layer 2 networks.  I need to be able to represent that certain vlans are the same vlan at those two sites (and, ergo, all the other objects that exist thereon) however I can only duplicate everything at those sites, or use "global" vlan and add them to many other sites they don't exist at.
The Role => tag idea is solid, its only used for search so it makes sense and especially where you have a physical server host that might run segregated roles in containers.  I'd use this too!
For gateways and hosting providers, multiple tenants can be at the same site and duplicate vlan id's and IP networks (RFC1918 addressing) with another tenant as the physical network is segregated; this is why there's that superset - its not a duplicate for the same reason another person called Brian is not a duplicate of you, even though in smaller circles you can be the only Brian in the room.

multiple tenants can be at the same site and duplicate vlan id's and IP networks (RFC1918 addressing)

That's what VRFs are for: to distinguish distinct instances of the same prefix.  Duplicate vlan IDs are already supported.

@darthmdh would #150 solve your issues. You talk about assigning vlans to sites for the purpose of automation. For me, I only care that a vlan is assigned to a port. Being assigned to at least one port means the vlan is at the site. Prior to that assignment, I do not care whether or not it is "at the site" from an automation standpoint. There is nothing meaningful I can do from an automation perspective by knowing strictly that a vlan is assigned to a site.

@darthmdh To be clear, a global VLAN is one which is available at all sites. Your request is to support the assignment of a VLAN to a set of specific sites. How do you propose extending the data model to facilitate this?

Related to the discussion on multiple tags: #132

@lampwins unfortunately that's not related.  The reason this issue fails automation is that you're currently forced to create fake entries - iterating via the REST API will then have the orchestration system attempt to access / create / modify / document things that don't actually exist (or take actions multiple times unnecessarily on the thing that does exist).  I do support issue #150 however as it would be useful to relate a device to vlans its interfaces are on if for nothing else than documentation / CMDB.
@candlerb my problem is the reverse; I know I can create duplicate vlan id's and I explicitly do NOT want to do this when it is the same vlan.  Netbox needs to support a 1:1 relationship between a vlan object created in netbox and an actual vlan that exists.  Right now I'm forced to create duplicate vlan objects because of this issue where netbox will not let me associate the vlan object with the specific sites it exists at.  There is no vrf, this is not a distinct instance of the same vlan id at one or more sites, it is literally the exact same vlan.  There is a DCI circuit between the sites and multiple layer 2 networks are stretched between them.  You can arp devices at both sites on that vlan from either side.  It is not a "global vlan" as there are other sites which do not have these vlans - they might have a distinct instance (or ten) of the same vlan id, or not, but that's already supported as we both know.
@jeremystretch Going on the assumption Sites is already 1:many with VLAN Groups, Prefixes and VLANs, the relationship table needs to be extended to many:many so that these items can exist at multiple sites.  I should probably mock this up in a test instance.
Prefixes probably annoy me the most, I'm unable to associate them properly as the layer 3 network exists on a stretched layer 2 at multiple sites (but not every site), so the site and vlan fields are left blank & there's no ability to navigate up & down the association hierarchy.

@candlerb my problem is the reverse; I know I can create duplicate vlan id's and I explicitly do NOT want to do this when it is the same vlan.

I didn't say you should.  I said: "I would choose to create a single object associated with none, rather than duplicate the object."
The comment about duplicate IP subnets in different VRFs was in response to:

multiple tenants can be at the same site and duplicate vlan id's and IP networks (RFC1918 addressing) with another tenant as the physical network is segregated

Therefore, in this thread you have been talking about different things at different times:

a single vlan ID 1234 with subnet <IP_ADDRESS>/24, which spans multiple sites
vlan ID 1234 with subnet <IP_ADDRESS>/24 for tenant 1, and a separate instance of vlan ID 1234 with subnet <IP_ADDRESS>/24 for tenant 2.  These are separate broadcast domains, isolated from each other.

In Netbox, the first is modelled as a single VLAN object and single prefix object.  The second is modelled as two VLANs (both with ID 1234) and two prefixes (both with <IP_ADDRESS>/24, but with different VRF IDs to distinguish them as being different subnets)

Netbox needs to support a 1:1 relationship between a vlan object created in netbox and an actual vlan that exists. Right now I'm forced to create duplicate vlan objects because of this issue where netbox will not let me associate the vlan object with the specific sites it exists at.

No, you're not forced to do that.  What I suggest is you create a "global" VLAN (associated with no sites, or all sites, however you look at it), even though it may not propagate to all sites.  Then you put in the comments "this VLAN exists in site X and Y only".
It is not perfect, but in my opinion it is better than creating duplicate objects representing the same thing.
The long-term solution I think you want is to be able to associate a VLAN with an explicit list of sites.  I wouldn't really object to it, but for me it's a low priority.  You explicitly pinged me to ask for my opinion, and so here it is.
However this idea bears some similarity to the idea of tagging (#132) where an arbitrary set of tags can be associated with an object, and that would be much more useful to me.
The main problem I see with associating a VLAN with a list of sites, is that for consistency you would also need to associate a prefix with a list of sites, and that results in more duplicate data entry and hence increases the likelihood of inconsistencies.
To avoid this, as I said before, I think that a prefix should be associated with a tenant (or tenants), and a VLAN should be associated with a site (or sites).  The VLAN is the thing which has physical expression, in the sense of where its broadcast domain propagates.  Given a prefix which has been assigned to a VLAN, you can find out which sites it exists in by seeing where its associated VLAN propagates.

The main problem I see with associating a VLAN with a list of sites, is that for consistency you would also need to associate a prefix with a list of sites, and that results in more duplicate data entry and hence increases the likelihood of inconsistencies.

Where is the duplicate data, though?  If the prefix <IP_ADDRESS>/24 exists in separate vlans at (separate, or the same) site you simply have multiple prefixes that share the same name, but are otherwise completely different.  It is perfectly logical for multiple tenants to use RFC1918 addressing however they see fit.

To avoid this, as I said before, I think that a prefix should be associated with a tenant (or tenants), and a VLAN should be associated with a site (or sites). The VLAN is the thing which has physical expression, in the sense of where its broadcast domain propagates. Given a prefix which has been assigned to a VLAN, you can find out which sites it exists in by seeing where its associated VLAN propagates.

Prefixes can already be associated with a tenant, and I'm simply asking to fix that the vlan cannot be associated with multiple sites as per the common place layer 2 stretched vlan method.  The design philosophy clearly states that netbox is to accurately reflect a real-world network.  You can have prefixes that do not exist on any vlan (as the network is untagged), so its not accurate to map ip -> prefix -> vlan to discover the site (and the workaround for this would be to create a gazillion duplicate "vlan 1" objects, which is just horrid)

What I suggest is you create a "global" VLAN (associated with no sites, or all sites, however you look at it), even though it may not propagate to all sites. Then you put in the comments "this VLAN exists in site X and Y only".

Then you're not able to accurately navigate in the UI nor via the REST API.  In practical usage there's no such thing as a "global" vlan (though you could argue untagged id 1 fits the bill on a technicality, just to be argumentative).  Look at #329, its one of the many issues here wrongly closed as a duplicate of #235 before #235 was actually implemented in a manner that does not permit accurate reflection of stretched layer 2 networks.
I know what you're saying about random tags in #132 (this comment is killer!) and at first glance it would seem you could lump sites and tenants into this, but this would make it difficult to expose a sane api as the containing object would be /api/dcim/tag/<random_name> instead of /api/dcim/sites (and this would have a knock-on effect with the UI).  Whilst that's perfectly fine with arbitrary labels like "role" or "platform", it doesn't lend itself well to something that physically exists like a location.

Where is the duplicate data, though?

Sigh.

VLAN 1234 is associated with sites A, B, C, X and Y (which is what you propose).
Prefix <IP_ADDRESS>/14 is associated with VLAN 1234
Prefix <IP_ADDRESS>/14 is associated with sites A, B, C, X and Y (because prefixes can be assigned to sites)

Therefore, points (1) and (3) require duplication of the site linkage information.

You can have prefixes that do not exist on any vlan (as the network is untagged)

The prefix exists on a layer two domain.  If that domain passes through a switch, the switch will assign it a VLAN ID - even if it's untagged on all ports on that switch.  So to me, it makes sense to model all layer two domains as "VLANs".  But you're right, people today may be assigning prefixes directly to sites without bothering to model the layer two domain.

Closing this out as it never really went anywhere, and I'm not interested in upending the current data model at this point.


WSO2 profile deployment order incorrect in docs

Description:
As mentioned in [1], the IoT profile deployment order is incorrect. While enabling analytics and running the IoT Server before deploying the Analytics server gives and error as follows.
ERROR {org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker} -  Error while trying to connect to ssl://localhost:7712 {org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker}
[1] https://docs.wso2.com/display/IoTS310/Running+the+Product
Steps to reproduce:
Enable Analytics in Api-manager.xml
Run iot-server
Run analytics server

Thanks for reporting this issue. FYI, actually, this error is harmless. Databridge connection pool will eventually succeed and connect to the port 7712 when it is available. Have you experienced any issues with the functionality of the product when you startup the product as mentioned order?

Closing as the steps work as documented. You don't need to do any additional configurations, you only need to start the analytics profile.


Certificate for private ip

Up to late september i was using cloudflare dns to get certifacte for my homelab on my local network, 192.168.0.xxx.
But now when caddy want to refresh certificate i get the following error.
In cloudflare dashboard i had setup
A record for lab.example.com and ip to <IP_ADDRESS>
{"level":"error","ts":1668018453.8717525,"logger":"tls.renew","msg":"could not get certificate from issuer","identifier":"lab.example.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"HTTP 400 urn:ietf:params:acme:error:dns - no valid A records found for lab.example.com; no valid AAAA records found for lab.example.com"}

My Caddyfile
# vim: ft=caddyfile
(cloudflare) {
	tls<EMAIL_ADDRESS>{
        dns cloudflare (env.CLOUDFLARE_API_TOKEN)
        resolvers <IP_ADDRESS> <IP_ADDRESS>
	}
}

{env.DOMAIN} {
	import cloudflare
	encode zstd gzip
	handle_path /kosync/* {
		reverse_proxy kosync:8081
	}
	root * /srv/web
	handle {
		file_server {
			precompressed br zstd gzip
		}
	}
}

wallabag.{env.DOMAIN} {
	import cloudflare
	reverse_proxy wallabag:80
}

Let's Encrypt says that your domain doesn't have any valid A or AAAA records. Unfortunately you redacted your domain so I'm unable to help any further :(

my domain is lab.tokariew.xyz
and this is from my dashboard…
I was able to get the cert earlier with my setup.

Ok, that should probably work.
What changed from earlier to now?

Rebuild caddy podman/docker image.
Should i grab earlier release of docker image?

Can you provide more details?
It's not immediately clear to me what is going on, so I'll need your help to understand it better.
Ideally, we need to be able to reproduce the bug in the most minimal way possible. This allows us to write regression tests to verify the fix is working. If we can't reproduce it, then you'll have to test our changes for us until it's fixed -- and then we can't add test cases, either.
I've attached a template below that will help make this easier and faster! This will require some effort on your part -- please understand that we will be dedicating time to fix the bug you are reporting if you can just help us understand it and reproduce it easily.
This template will ask for some information you've already provided; that's OK, just fill it out the best you can. :+1: I've also included some helpful tips below the template. Feel free to let me know if you have any questions!
Thank you again for your report, we look forward to resolving it!
Template
## 1. Environment

### 1a. Operating system and version

```
paste here
```

### 1b. Caddy version (run `caddy version` or paste commit SHA)

```
paste here
```

### 1c. Go version (if building Caddy from source; run `go version`)

```
paste here
```

## 2. Description

### 2a. What happens (briefly explain what is wrong)

### 2b. Why it's a bug (if it's not obvious)

### 2c. Log output

```
paste terminal output or logs here
```

### 2d. Workaround(s)

### 2e. Relevant links

## 3. Tutorial (minimal steps to reproduce the bug)

Instructions -- please heed otherwise we cannot help you (help us help you!)

Environment: Please fill out your OS and Caddy versions, even if you don't think they are relevant. (They are always relevant.) If you built Caddy from source, provide the commit SHA and specify your exact Go version.

Description: Describe at a high level what the bug is. What happens? Why is it a bug? Not all bugs are obvious, so convince readers that it's actually a bug.

2c) Log output: Paste terminal output and/or complete logs in a code block. DO NOT REDACT INFORMATION except for credentials. Please enable debug and access logs.
2d) Workaround: What are you doing to work around the problem in the meantime? This can help others who encounter the same problem, until we implement a fix.
2e) Relevant links: Please link to any related issues, pull requests, docs, and/or discussion. This can add crucial context to your report.

Tutorial: What are the minimum required specific steps someone needs to take in order to experience the same bug? Your goal here is to make sure that anyone else can have the same experience with the bug as you do. You are writing a tutorial, so make sure to carry it out yourself before posting it. Please:

Start with an empty config. Add only the lines/parameters that are absolutely required to reproduce the bug.
Do not run Caddy inside containers.
Run Caddy manually in your terminal; do not use systemd or other init systems.
If making HTTP requests, avoid web browsers. Use a simpler HTTP client instead, like curl.
Do not redact any information from your config (except credentials). Domain names are public knowledge and often necessary for quick resolution of an issue!
Note that ignoring this advice may result in delays, or even in your issue being closed. 😞 Only actionable issues are kept open, and if there is not enough information or clarity to reproduce the bug, then the report is not actionable.

Example of a tutorial:
Create a config file:
{ ... }

Open terminal and run Caddy:
$ caddy ...

Make an HTTP request:
$ curl ...

Notice that the result is ___ but it should be ___.

i started to get working on it, but when I replaced variables in Caddyfile with my domain and api key, it started to work.
So I messed up a Caddyfile syntax. Sorry for problem :(


Add Redis as a Vector Store

Current Version of the Spring AI allows InMemoryVectorStore but RedisVectorStore is expected along with.
Expected Behavior
Redis should be able to use as VectorStore
Current Behavior
InMemoryVectorStore  is supported.
Context
I am working on document querying use-case with azure openai embedding stored in Redis. So i request the team to implement Redis as Vector Store

I am currently investigating using Redis as a VectorStore implementation in spring-ai.
Here is some relevant information on the topic that I am currently reading.

@jxblum , I am using Redis for GenAI app. Please let me know if we could connect on this.

I have used Redis as a vector store multiple times with LangChain and considering my Java experience, would like to contribute.
Can it be assigned to me?

@jxblum, any update on Redis VectorStore?

Hi @radhakrishna67 - Thank you for reaching out.
Sorry for the lapse in communication. I have been juggling multiple responsibilities and haven't been fully allocated/dedicated to working Spring AI just yet. My primary responsibilities are on the Spring Data Team, as the Spring Data Redis technical lead. I have also been busy preparing for Spring Data releases next week.
I have started crafting a PR for this ticket and hope to have something relatively soon.
Of course, anyone interested in this ticket is also free to experiment, contribute or simply share ideas. Nothing is stopping anyone from doing that.
Anyway, I will report back again when I am ready for some review/feedback.
Cheers!

@jxblum , Thanks for the response.
I do understand the busy days. I will wait for it.

@jxblum
I'm interested to do some experiment for redis,,
Is the open ai key is free ? or need billing ?

Hi @omarmahamid - Creating an OpenAI account at OpenAI and setting up an API Key is free. Once you have signed up and logged in, then you can even add additional keys and organizations to your account.

NOTE: I have not signed up for Azure OpenAI yet, so I am not familiar with their process.

Of course, not everything is 100% free, and OpenAI will rate limit your requests, naturally.


Put Monterey/WA to WA

Description of proposed changes
Fixed a potential typo

Hi @danrlu  - thank you so much! However, the reason this was changed to California is because we weren't able to find that there's a Monterey County in Washington - seems to just be in California. But tracking these things down can be tricky - do you know if there's a Monterey County in Washington?

Oh I see. No I have no idea. I thought it's b/c there is another Monterey County in WA that you guys appended CA to the Monterey County in CA to distinguish it!! There's only 1 sample on Gisaid from WA Monterey from Seattle Flu Study. I emailed them to confirm the location with them. Thanks Emma!

Thanks @danrlu ! Please feel free to reopen if you hear back from Seattle Flu Study and we should adjust this location to be more accurate!


How to define a custom knowledge base for Entity Linker ?

I'm going to leverage an augmented version of UMLS (by adding new aliases from external sources to help the algorithm to match better). It seems that candidate generation module leverages a pre-indexed version of UMLS and loads them from the cloud. May I ask you to help me to understand what's the best practice to load a custom version of UMLS or augment the current version with new aliases please?

Hi, see if the response to this issue works for you: https://github.com/allenai/scispacy/issues/331#issuecomment-803291644


[Enhance] Replace class-name.ts with clsx

Describe the bug
createCls([false]) will raise an exception.
I suggest using clsx since it's already handling those cases and it's a great package, and if so please let me know so I can open a PR.
Reproduction
createCls([false])
System Info
No response
Used Package Manager
pnpm
Validations

[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a ikun-ui issue and not a framework-specific issue. For example, if it's a Svelte SFC related bug, it should likely be reported to https://github.com/sveltejs/svelte instead.
[X] The provided reproduction is a minimal reproducible example of the bug.

createCls([false]) is not a common use case since createCls is an internal method, but clsx looks great and I think it can be used

I don't mean to use false inside array, but the array itself it doesn't work, while if you worked with @vuejs classes before you will see that it supports classes and other formats just like clsx does.


Make private repositories listable

Feature Description
Hello,
I would like to be able to make private repositories visible for everyone who are in the organization.
The user should only be able to see repository name and the owner(s), and then he could ask a owner to get access.
The idea is to avoid multiple creation of same repository in a organization.
Ideally, I would like to have a setting parameter for each organization to enable "show private repositories" (default: false).
Best regards
Screenshots
No response

What's the idea of private org repos then? It's useful if you have multiple different clients assigned to their respective teams in organizations. And each client can see only necessary repositories.
Wouldn't your idea break it?

Hi @KroMignon, I noticed that you said you cannot reopen this issue, it shouldn't happen. Could you please show me whether there's a button like this in your view? It will help me to adjust my work, thanks.

Here is what I have on GitHub, there is no such button

Thank you so much, I'll figure it out. Before that, let me help you to reopen this PR first.


Open the camera in 720p for the bandwidth test.

We should attempt to open the camera in 720p for the bandwidth test to better test the ramp-up time to higher bitrates.

@KaptenJansson - can you verify the behaviour in machines that do not support that resolutions?  and review this change?
Supposedly it should work since it is a maxWidth constraint.

I will test and report back.

Since we are using max constraints I think it would make sense to fetch the actual resolution from the stream using getStats, actually it would make sense in any case (regardless of constraints used) as it would be interesting to see if the capture resolution affects bandwidth performance (especially on weaker devices). This is out of scope for this pull request, filed issue #61.


Improve description for customers query

Problem
Current Description:
List of the shop's customers.

There is some underlying logic which should be described:

all users created by accountRegister mutation are displayed here
the query will also return staff users, if they used their account for making an order

1. General Assumptions

Acceptance Criteria

[ ]
[ ]

2. API changes
Mutations

Types

3. Database changes

4. UML diagrams

5. To Do list

### Tasks
- [ ] Add a draft title or issue reference here

6. To Test

[ ]
[ ]

I'd like to work on this issue. Seems like a good way to get into understanding the code base.

Hi, if this available I would like to work on this.


[Translatable] Query Hint: Fatal error: Nesting level too deep - recursive dependency?

Hi,
I'm using the Translatable, Timestampable and Loggable extensions from the 2.1.x tag together with Doctrine 2.1.5. Translation basically works fine, but as soon as I use the Query Hint, I get the following Fatal error:
Fatal error: Nesting level too deep - recursive dependency? in [...]/Doctrine/ORM/Query.php on line 603
Stack trace:
[...]
8	1.4853	34574280	Doctrine\ORM\AbstractQuery->getResult( )	../IndexController.php:24
9	1.4853	34574672	Doctrine\ORM\AbstractQuery->execute( )	../AbstractQuery.php:413
10	1.4853	34574512	Doctrine\ORM\Query->_doExecute( )	../AbstractQuery.php:607
11	1.4853	34574512	Doctrine\ORM\Query->_parse( )	../Query.php:234
12	1.4853	34574592	Doctrine\ORM\Query->_getQueryCacheId( )	../Query.php:207
13	1.4853	34574752	var_export ( )	../Query.php:603

The code causing the error looks like the follwing (just a test query, but same behaviour on more complex queries):
<?php
$select = "SELECT e FROM Entity\\Message e";
$query = $em->createQuery($select);
$query->setHint(
    \Doctrine\ORM\Query::HINT_CUSTOM_OUTPUT_WALKER,
    'Gedmo\\Translatable\\Query\\TreeWalker\\TranslationWalker'
);

Zend_Debug::dump($query->getDQL());
Zend_Debug::dump($query->getSQL());

$results = $query->getResult();
foreach ($results as $result) {
    Zend_Debug::dump(array($result['id'], $result['title'], $result['content']));
}

Any ideas?
Regards,
Mathias

hm, tell me if disabling query cache will solve the problem? etc $query->useQueryCache(false); in that case I will need to check what is the problem with cache

Thanks for the quick answer. Disabling the query cache works (i guess the _getQueryCacheId( ) method is not called in that case).

Ok, thanks for tips @l3pp4rd  !

and regarding translatable hint, use locale setter hint based on session, because query cache is based on hash from hints, ref: https://github.com/l3pp4rd/gediminasm.org/blob/master/src/Gedmo/DemoBundle/Controller/CategoryController.php#L243


build: cleanup build scripts, use lazy APIs, update to Gradle 8.8.

Can be reviewed commit-by-commit, although two of the final commits are rather large (very samesy though).

api check failure:
  --- /home/runner/work/misk/misk/misk-api/api/misk-api.api
  +++ /home/runner/work/misk/misk/misk-api/build/api/misk-api.api
  @@ -56,7 +56,7 @@
   public abstract interface class misk/scope/ActionScoped {
   	public static final field Companion Lmisk/scope/ActionScoped$Companion;
   	public abstract fun get ()Ljava/lang/Object;
  -	public abstract fun getIfInScope ()Ljava/lang/Object;
  +	public fun getIfInScope ()Ljava/lang/Object;
   }```

should I just update it per this suggestion?
```   You can run :misk-api:apiDump task to overwrite API declarations
``


Code Sprint Ideas

The overarching goal I think would be to make an actual release of intake-esgf. I was mistaken about it already existing in PyPI. In my mind, we should:

[ ] Create documentation. My contribution to the cookbook could work as a start. I like cf-xarray's style. Along the left side I would like to have little snippets that show the different features (basics, esgf1 support, model groups, session logs). I have this started in a branch.
[ ] Setup some process for releases, I like version numbers in the 2024.01 style.
[ ] Setup better testing. I have a couple but they should cover more.
[ ] Setup CI. Setting it up is not overly hard, but failures could easily be due to server issues. What should be tested and how?
[ ] Code review. I will do a first pass before, but it would be good to step through how it works and if it makes sense the way it is organized.
[ ] Should intake-esgf be more formally a intake Catalog? When I look at their class structure, I am not sure it makes sense, but then I see intake-esm implements something. Perhaps it is that intake catalogs do not have such a strict  definition?
[ ] Currently I have only tested CMIP6 output, how can we make it work for CMIP5 too?
[ ] My progress bars do not always work well in notebooks. I see that there is a from tqdm.notebook import tqdm_notebook, but how could I check internally which version to use?
[ ] Enable globus transfers. For this we need to think how this might break the paradigm of how this works. Could we trigger a transfer and then issue a block until the transfer is finished? Does globus give us the tools to make this work?
[ ] For non-xarray people, we need a to_filelist() operator that will download data and then give you the path of where it is found. This would be useful for me in ILAMB as well.
[ ] Local database #4

...

@nocollier - is there anything I can help with this week that would help setup for next week?

I cannot think of anything specific. In my mind we need to setup several things and decide what is in the first version--it is never done of course. I would benefit from talking to someone else who is experience in how things like CI/docs/releases are done to help make decisions. I also would love to have someone else understand the code-base and being able to work on features. That also assumes that you agree/like the way I have organized things.

Discussions about next steps
intake-esgf

[ ] Setup data for testing
[ ] Expand the search capability

[ ] Synonyms, cf-variables, etc.
[ ] Project agnostic searches (CMIP5)

[ ] Globus

[ ] transfers
[ ] compute
[ ] flows

Cookbooks

[ ] Publication-focused workflows

[ ] Comparing ESGF+ARM data
[ ] Reproducing key figures from IPPC report

[ ] Using with other packages

[ ] https://icclim.readthedocs.io/
[ ] https://xclim.readthedocs.io/

WPS

[ ] Get this running within OLCF/ALCF


⚠ Bofrost.lu has degraded performance

In c14d432, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 915 ms

Resolved: Bofrost.lu performance has improved in 7f18970 after 6 minutes.


Add SingleHandleMethodInvokeStrategy

The HandleCommandStrategy was rewritten to behave like it was contemplated for the SingleHandleMethodInvokeStrategy so this issue can be closed.


chore: bump deps and dedupe lockfile

Codecov Report

Merging #307 (4616df0) into main (8e7c25a) will not change coverage.
The diff coverage is n/a.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@           Coverage Diff           @@
##             main     #307   +/-   ##
=======================================
  Coverage   98.89%   98.89%
=======================================
  Files          52       52
  Lines        2355     2355
  Branches      313      313
=======================================
  Hits         2329     2329
  Misses         26       26

Flag
Coverage Δ

plugin-discord
97.63% <ø> (ø)

tagscript
99.60% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Use Cursor Pagination / remove Count() queries from backend

As a result of us spending time optimizing db indexes and queries, we see that one of the biggest reasons for slow queries seems to be large Count() queries. This is because in the current pagination architecture, we fetch the total count of results for every filter / query made to the backend. We need this to show total number of results returned as well as calculate the number of pages of results, etc.
Whilst I like being able to show total number of results and having explicit pagination, it does seem like we can get significant performance gains by changing our pagination method: never fetch total counts (perhaps unless the user explicitly requests for it), and use "Cursor Pagination" in the backend to "See More" to fetch more results for the user, without showing explicit page numbers.
What this would likely involve:

Implement Cursor Pagination in the DRF backend code: https://www.django-rest-framework.org/api-guide/pagination/#cursorpagination
Double-check SQL queries being executed and ensure there are no more Count() queries being executed
If needed, create a separate endpoint that only returns the Count for a filter, that can be called explicitly by a user if they want the count of their result-set
Handle new pagination style on the frontend:

Remove page numbers
Remove showing users total count of results (maybe convert to a button where users can explicitly request the count)
Enable a "See More" button for users to paginate / see more results from their result-set, which would show "No More Results" or so once the result-set has been exhausted

This is likely a bit of work, especially on the frontend.
@willemarcel do you think we could maybe do a quick attempt to implement Cursor Pagination on the backend and see if we could run some tests to measure the impact this has on performance and then see if this is worth pursuing?
cc @bitnerd @geohacker

Yes, it's quite simple to make that change in the backend. The frontend is the part that requires a lot of work to switch to the cursor pagination.


When LDK node is down, we don't show a way to reset in the UI

If we're keeping node start/reset in the UI for debuging purposes, we probably should show it when then node is unable to start, right now if the nodei s unavailable we don't show it.

@Kukks please take a look at this one.


[1.12] Automated Bump DC/OS UI v2.26.12

High-level description
What features does this change enable? What bugs does this change fix?
Corresponding DC/OS tickets (required)

DCOS-ID JIRA title / short description.

Related tickets (optional)

DCOS-ID JIRA title / short description.

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[ ] Change log from the last version integrated (this should be a link to commits for easy verification and review): example
[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain.
[ ] Test Results: [link to CI job test results for component]
[ ] Code Coverage (if available): [link to code coverage report]

Please update CHANGES.md file and apply Ready for Review Label.
* Bumped DC/OS UI to [1.12+v2.26.12](https://github.com/dcos/dcos-ui/releases/tag/1.12+v2.26.12)

You can edit CHANGES.md directly here: https://github.com/dcos/dcos/edit/dcos-ui/bump/2019-06-03-17-37-28/1.12+v2.26.12/CHANGES.md?pr=%2Fdcos%2Fdcos%2Fpull%2F5638


Add All Contributors bot + contributors' list on the readme file

I don't know if I will ever have other contributors to this repo, but just in case (and because I want to learn how to use it), I would like https://allcontributors.org installed and configured in this repo in order to display all possible contributors in a prominent place in the readme file.

@all-contributors please add ccamara for source code

Great!


redlock: added definitions and tests

https://github.com/mike-marcacci/node-redlock

@chrootsu Thanks mate!


Point selection method selects points in the water

The function 'slcstack.point_selection(threshold, method="amplitude_dispersion",chunks=5000)' selects also points in the water.
This is strange since these points have a very high amplitude dispersion.
See the image in the appendix for the selected points. The red points are points that are selected in the water

This problem is caused since there are points with nans in the slc stack. When we then apply the point selection method, these points are also selected since the algorithm treats these nan values as very small values

Thanks @wietskebrouwer for submitting this issue. Indeed it is weird to have points with such low amplitude dispersion on water. I will take a look at this ASAP.

Hi @wietskebrouwer,
I made this example notebook (you need up unzip it) to reproduce the problem, but to me it all works well, no points are selected on the water.
However, I do notice one thing which may cause the problem: in the Zarr file you created:  computed_slcs_s1_dsc_t037.zarr, there is only complex data but no amplitude. Maybe the way you add the amplitude changed the value of amplitude on those NaN locations, and making those points popping up.
In the attached Notebook, I was using the _get_amplitude function to add the amplitude as a delayed object. Then if the complex is np.nan at the location, the amplitude stays np.nan and won't pass the amplitude dispersion criterion. Because both np.nan<value and np.nan>value will return False.
My take away from this, is maybe we should make the two internal functions _get_amplitude and _get_phase public, since they are recommended ways to add these two fields.
Could you please check if your problem is solved by calling _get_amplitude, to compute amplitude?

Hi @rogerkuou,
Thanks for the update. When I run your notebook, I indeed do not get the points selected at the water. I will now try to implement this way of adding the phase and amplitude data to the stack in my own code to check whether it indeed works.

I think I already see where the problem is. In my original note I send to you, I needed to add the mother to the slc stack. See image:

Probably this is causing the issue. Because on the stack where I manually added the mother image (see my notebook) I still have to problem that points in the water are selected. Also when I apply the _get_amplitude and _get_phase functions you suggested.

Hi @wietskebrouwer, I found this is related to the behavior of xarray.DataArray.std and xarray.DataArray.mean. By default, they ignore the NaN values, but not returning NaN.
For the points on the water, it appears that they have zero amplitude value on reference image, and NaN on others. Then the NAD and NMAD are all 0, which passes the thresholding of "<threshold", hence they are selected.
I made a fix in PR#51. No extra excluding needed since NaN should not pass the thresholding. And I am not infavor of adding a thresholding specifically on NaN since it is redundant and may cause unexpected behavior.
For your implementation, I suggest to change it like this:
def point_selection(self, threshold, method, chunks=1000):

        ......
        match method:
            case "nad":
                #mask = self._nad() < threshold
                # mask = np.logical_and(~np.isnan(self._nad()), self._nad() < threshold)
                mask = self._nad() < threshold # This should be enough

            case "nmad":
                #mask = self._nmad() < threshold
                # mask = np.logical_and(~np.isnan(self._nmad()), self._nmad() < threshold)
                mask = self._nmad() < threshold
            case _:
                raise NotImplementedError
      ........

def _nad(self, chunk_azimuth=500, chunk_range=500):
        # Time dimension order
        t_order = list(self._obj.dims.keys()).index("time")

        # Rechunk to make temporal operation more efficient
        amplitude = self._obj.amplitude.chunk(
            {"azimuth": chunk_azimuth, "range": chunk_range, "time": -1}
        )

        nad = amplitude.std(axis=t_order, skipna=False) / (
            amplitude.mean(axis=t_order, skipna=False) + np.finfo(amplitude.dtype).eps
        )  # adding epsilon to avoid zero division

        return nad

    def _nmad(self, chunk_azimuth=500, chunk_range=500):
        """
        Estimate the NMAD for a xarray dataset.
        """
        # Time dimension order
        t_order = list(self._obj.dims.keys()).index("time")

        # Rechunk to make temporal operation more efficient
        amplitude = self._obj.amplitude.chunk(
            {"azimuth": chunk_azimuth, "range": chunk_range, "time": -1}
        )

        median_amplitude = amplitude.median(axis=t_order, skipna=False)
        mad = (np.abs(amplitude - median_amplitude)).median(dim="time", skipna=False)
        nmad = mad / (median_amplitude + np.finfo(amplitude.dtype).eps)

        return nmad

Could you please try if this works?


What is the difference between gpt2_non_optimized and gpt2_optimized in tests？

As shown in the code, gpt2_optimized has these optimizations enabled:
    config.use_flash_attn = True
    config.fused_bias_fc = True
    config.fused_mlp = True
    config.fused_dropout_add_ln = True
    config.residual_in_fp32 = True

Why is test_gpt2_optimized slower than test_gpt2_non_optimized? Is it normal?

It's testing correctness, not speed. The tests are also running other stuff. It doesn't measure speed.

But surely these operations such fused should speed up the inference of the model?

Yes, you can also write your own benchmark to measure the speed. You shouldn't rely on how long the tests take to finish.

Get,Thank you very much!


SMTPNotSupportedError at / SMTP AUTH extension not supported by server.

Im getting this error in django to send email using office 365

This isn't a Django repository, sorry.

This isn't a Django repository, sorry.

Okay sir, I had a question, how to send Email using sendGrid Template in Django ?
thank you

@obedsayyad123 I'm not sure why you are asking me Django questions, I really do not have much experience with it, I use Flask. Maybe this StackOverflow questions tells you what you need to know? https://stackoverflow.com/questions/30140940/how-to-use-designed-template-in-django-using-sendgrid-api.


Problem with dot character in token field

Hi,
I was surprised that a dot (.) seems not to be accepted in the token field. Did I miss here something or what is the reason for this behavior?

Ok, just found the solution which is to remove the default characters which will trigger a token creation:
tokenView.tokenizingCharacters = [];


Remove the Enrollment Type Dropdown in the Experiment Data Tab

The "Type" dropdown in the "Enrollment" section under the "Data" tab on the Experiment details page is somewhat ambiguous regarding what it means when a group type (e.g., classId) is selected.
We should either make it clearer in the UI (e.g., rename the "Group Enrollments Across Sites" label) or remove the dropdown to avoid confusion.
This should be applied to the redesigned UI (it might not be needed for the old experiment design).
@VivekFitkariwala @ppratikcr7 If anyone from Playpower Labs recalls the meaning or usage of this dropdown, please leave a comment.

So, when you select individual type, it will show you the complete enrollment count (i.e. I enrolled 3 students) as shown below:

And when you select Group Type, it will show the graph for the number of groups that have enrolled students for group type: "add-group1" as below: (so I enrolled 2 students in teacher1 group and 1 student in teacher2, so we see 2 enrolled groups in graph)

We should keep the dropdown and maybe rename it to "Groups Enrolled" for the dropdown and rename to "Grouped Enrolled Across Sites" for the label.

Rename the label to "Groups Enrolled" or "Unique Groups Enrolled"

What do you mean by "it will help us see different graphs as well"? Does the "Type" dropdown display different graphs? I think the dropdown just shows or hides the "Group Enrollments Across Sites"? (which we can always show if we remove the "Type" dropdown)

Yes, it shows different graphs as shown in my first comment.

OK, we discussed that we still need the "Type" dropdown and for viewing the 2 different graphs. We might want to update the enrollment counts label but I think it can be created as a separate issue. I will close this one.


Use latest firmware file when flashing instead of hard coding file names

The enhancement to the flash script should let users copy the firmware files directly from the repository without having to rename them (less questions and issues raised if updates fail).

Looks great! Thank you so much for contributing :)


Add Backend Dockerfile

Create Dockerfile to containerize the server application.

Great job. You've got the right idea with containerizing front and backend. There will be a third container eventually... but for now focus on getting the two to work locally with docker.


Add ability to filter by column values to Table View

Add individual column filtering to the Table View. See https://datatables.net/examples/api/multi_filter_select.html

Initially, I implemented this by populating all options when the data is initially loaded from a file. However, this increased the loading time by as much as 15s for ~4.7 mb SQLite file due to querying the DB, filtering, and sorting values for each column.
To fix this, I changed to lazy loading of the options when the user clicks the dropdown menu. This does add a little (<1sec) delay when the select is clicked, but has the benefit of not affecting file loading times, and means that we can do additional filtering (e.g. based on the jstree location) each time the user clicks the options.


Install on Odroid failing silently

I've installed ZFSBootMenu on an Odroid (https://www.odroid.co.uk/H3-Plus) and whilst the boot menu comes up fine and I can see and interact (as in use the recovery or chroot shells) fine, trying to boot a kernel displays a message about tearing down USB, and then reboots.
The menu screen is the same as my other machines showing the correct root dataset and showing the right kernel.  I've gone into the recovery shell and can see the zpool status is fine, and I can boot from a USB drive and everything on the pool works (I can write it, etc.)
I'm using Ubuntu 23.04 with a 6.2.0-20 kernel.  The kernel should support ZFS (although I'd have expected it to at least boot further even if it didn't!).
How do I go about debugging this?  I've tried increasing the loglevel on the kernel commandline (ctrl-E) but I'm at a loss as to what else I can do!

I have a suspicion that the kernel doesn't support kexec. The most straightforward test would be to drop to an emergency shell with Ctrl+R, mount_zfs pool/path/to/your/root/filesystem, change to the boot subdirectory of the path that it reports, load the kernel with
kexec -a -l ./your-kernel-here --initrd=./your-initramfs-here --cmdline="root=zfs:pool/path/to/your/root/filesystem other args here"

and finally try to jump into the kernel with kexec -e -i. Watch for any reported errors to see if anything useful is printed.

Assuming you mean "--command-line=..." as --cmdline doesn't work, that does the same thing as before, as in it simply reboots the machine :(

I've tried again with a USB stick rather than the mmcblk device.  I created this on another machine, and rebuilt as I've done for that one.  Again ZBM loads fine, imports the pool and finds the kernel.  However on attempting to boot (I assume this is done as a kexec as per doing the zkexec in the recovery shell) it comments on the USB teardown, and then returns to the "bios" screen after a few seconds.  Is there anything else I can try?  Without any error messages I'm struggling to think of new things to try :(
The build by the way is very simple following the ZBM Ubuntu instructions, so it's a 2 partition system, 1GB (I'm only on a 32GB system so don't want to use the full 2GB if possible) fat partition (mkfs.vfat -F32) and then a single device/vdev zpool created using the advised parameters.  I then go through the debootstrap process, and then chroot (actually I've been using systemd-nspawn of late, but I've tried with the standard chroot as well in case that was an issue) and install the kernel and few other utilities.  This feels like a specific issue to this hardware, but I don't know what to try and look for.

Upgraded the bios in case that would help, from 1.12 to 1.15 in case that'd help.  No change though in the symptoms that I can see.

Unfortunately, I think this is probably some hardward incompatibility. It might be worth trying kexec from within your boot environment (or a live environment) to see if the result is the same.

Ok, I'm up for testing that, but can you give me some instructions on how?  Also what in the hardware obstructs this?  Maybe there is a setting I can change?

If you do work out a viable solution to this issue, please feel free to note it here. We can then either adapt the solution to be a permanent part of ZFSBootMenu, or at the least officially document it. Barring that, there's nothing else actionable here for us.


Disable fields while saving

Fields are disabled, buttons are inactivated etc. while the form data is saved. Relies on setAsEnabled() being implemented properly in the field subclasses.

I have a use case where SelectList field is intentionally disabled in some cases (type_enabled = false), hence the first implementation of Field.setAsEnabled(). If I read the code correctly, the save process would now first disable all fields and then enable them, regardless of their initial states. This needs a fix.
I was already thinking about that Field.isTypeEnabled() was technically broken, as it does not change dynamically, and hence it becomes useless after the initial build process. Field.isFieldEnabled() is fine, but if a non-field component should eventually need enabled state as well, it would make sure to move isTypeEnabled() to component level and use that. In short, I don't like having both enabled methods at the same time - it becomes confusing. But this can be decided later when we have more information what is truly needed...

Yes, I figured that might sometimes be the case – fields being disabled depending on the values of other fields or something. I guess we need some extra variable to store the state prior to saving and then re-enable the field after saving only if the pre-save state was enabled.
I'll see if I can merge Field.isFieldEnabled()and Field.isTypeEnabled() while I'm at it.

I personally would suggest using an internal list of fieldList indexes that have enabled state initially true, and then re-iterating that list again when restoring states. We would however need to do this also on errors as well.... This is not a very global behavior (yet), so extending Tinycar.Api.call() parameters to receive and execute a custom failure callback handler alongside other internal error handling sounds logical  to me.
The isTypeEnabled conversion might be a bit tricky because the initial state in the model might not reflect the state in the field yet. The easiest way to bypass this would be to always use the internal variable and rely on start() to set it to match the model state (mainly to disabled) before it is needed anywhere else. I know that this is the exact the opposite of what the comment in start() says about enabled state, but it seems to make more sense now to optimize for the most common scenario....
So, a few changes for a simple UI effect :smile:


Status code 403

⚠ Please verify that this bug has NOT been raised before.

[X] I checked and didn't find similar issue

🛡 Security Policy

[X] I agree to have read this project Security Policy

📝 Describe your problem
Hello,

I can browse all my websites with all my devices. (https://alainpicard.ca)
However, I get for all of them a 403 in the panel.
They are on AWS EC2
CloudFlare as CDN

📝 Error Message(s) or Log
Request failed with status code 403
🐻 Uptime-Kuma Version
1.21.3
💻 Operating System and Arch
Debian 11
🌐 Browser
Chrome 113.0.5672.93
🐋 Docker Version
20.10.5+dfsg1
🟩 NodeJS Version
No response

OK in my side, but it is usually caused by Cloudflare's Browser Integrity Check.

Hi, thank you for this.

Best!

Hi, thank you for this.
Best!

thanks kinda dumb, that's in most cases the main point in using cloudflare


show text when in tablet mode?

I'm not finding the function of set show text or not in tablet mode?

According to material design guidelines, in tablet mode no text is visible (at least on the screenshot there is none). Additionally I don't really know where to show it, as only short or small in font-size labels will fit below the icons without enlarging the "bar's width".

Unfortunately, I'm not going to support showing titles on tablets when using the tablet layout with this library.


Make cupsDirRead on Windows act more like POSIX

FindFirstFile needs a pattern suffix added to the path.
Here's an example.
This one small fix improves #228 such that the config files are loaded, but something else is not quite right:

2021-05-25T13:02:25.559Z  Using default data directory "C:\Users\nick\AppData\Local\Temp/ippserver.37636".
2021-05-25T13:02:25.560Z  Using default spool directory "C:\Users\nick\AppData\Local\Temp/ippserver.37636".
2021-05-25T13:02:25.593Z  Using default listeners for localhost:8631.
2021-05-25T13:02:25.594Z  Loading printers from "test/system.conf".
2021-05-25T13:02:25.595Z  Skipping ".".
2021-05-25T13:02:25.596Z  Skipping "..".
2021-05-25T13:02:25.596Z  Loading printer from "foo.conf".
2021-05-25T13:02:25.602Z  [Printer foo] printer-uri="ipp://localhost:8631/ipp/print/foo"
2021-05-25T13:02:25.645Z  Loading printer from "infra.conf".
2021-05-25T13:02:25.645Z  Unknown directive "AuthProxyGroup" on line 2 of "test/print/infra.conf".
2021-05-25T13:02:25.645Z  [Printer infra] printer-uri="ipp://localhost:8631/ipp/print/infra"
2021-05-25T13:02:25.655Z  Loading printer from "ipp-everywhere-pdf.conf".
2021-05-25T13:02:25.658Z  [Printer ipp-everywhere-pdf] printer-uri="ipp://localhost:8631/ipp/print/ipp-everywhere-pdf"
2021-05-25T13:02:25.669Z  Loading printer from "ipp-everywhere-raster.conf".
2021-05-25T13:02:25.672Z  [Printer ipp-everywhere-raster] printer-uri="ipp://localhost:8631/ipp/print/ipp-everywhere-raster"
2021-05-25T13:02:25.683Z  Loading printer from "ippserver-data-type-test.conf".
2021-05-25T13:02:25.686Z  [Printer ippserver-data-type-test] printer-uri="ipp://localhost:8631/ipp/print/ippserver-data-type-test"
2021-05-25T13:02:25.698Z  Loading 3D printers from "test/print/ippserver-data-type-test.conf".
2021-05-25T13:02:25.699Z  Skipping ".".
2021-05-25T13:02:25.699Z  Skipping "..".
2021-05-25T13:02:25.699Z  Loading 3D printer from "foo3d.conf".
2021-05-25T13:02:25.702Z  [Printer foo3d] printer-uri="ipp://localhost:8631/ipp/print3d/foo3d"
2021-05-25T13:02:26.620Z  [Printer ippserver-data-type-test] Now using DNS-SD service name "ippserver-data-type-test (2)".

@sicklittlemonkey I'll need to merge this change into OP CUPS first, then merge back here... But looks reasonable. I'll also add "." and ".." removal, too, which is something the POSIX code path does and will make the two versions behave the same.

Thanks @michaelrsweet that sounds great.

@sicklittlemonkey
Hello, I have tried your modification plan. Although our modification method is different, the result should be the same, and the configuration file can be successfully read. After fixing the problem of reading the configuration file, the program can run to if (select(max_fd + 1, &input, NULL, NULL, &timeout) <0 && errno != EINTR)(Appears in the ServerRun function in the client.c ) file to get an error. I solved this problem by rewriting a new ServerRun function and replacing the original one.
I will upload an ippserver that can successfully run normally on windows later. But in order to investigate the problem, I added a lot of logs and tentative modifications made the code a bit messy, so I will re-add the necessary modifications to the original code to make the program work normally, and then upload it again. It may take some time.
I have some questions after reading the log you uploaded. I hope to get some insights for a better understanding of ippserver. I found that you can successfully read the contents of the configuration file when the startup parameter you use is -C test , and when I use this parameter, the if ((fp = cupsFileOpen(conf, "r") ) == NULL) （Appears in the load_system function in the conf.c file） will fail to read system.conf. I must use an absolute path like -C "D:\ippsample-master\test" as a parameter to successfully open the "system.conf" configuration file. What other operations did you do to succeed?

Hi @wangzhili-China

I must use an absolute path like -C "D:\ippsample-master\test" as a parameter to successfully open the "system.conf" configuration file. What other operations did you do to succeed?

I tried to make the test similar to running from the command line in Linux, so in Visual Studio I changed:
ippserver->Properties->Debugging->Working Directory
to
$(ProjectDir)..

[master 00181de] Merge changes from CUPS master@a5687bd03d3856de83ccdc01305f5ecd8804027e


Introducting Nexus Operator 0.1.0

Signed-off-by: Ricardo Zanini<EMAIL_ADDRESS>Introducing Nexus Operator 0.1.0 version:
Creates a new Nexus 3.x deployment in a Kubernetes cluster. Will help DevOps to have a quick Nexus application exposed to the world that can be used in a CI/CD process:

Deploys a new Nexus 3.x server based on either Community or Red Hat images
Creates an Ingress controller in Kubernetes (1.14+) environments to expose the application to the world
On OpenShift, creates a Route to expose the service outside the cluster

Full documentation: https://github.com/m88i/nexus-operator/blob/master/README.md

Thanks submitting your Operator. Please check below list before you create your Pull Request.

Flat operator directory structure is obsolete from 23-rd of October 2019, only nested directory structure will be accepted.

New Submissions

[x] Has you operator nested directory structure?
[x] Have you selected the Project Community Operator Submissions in your PR on the right-hand menu bar?
[x] Are you familiar with our contribution guidelines?
[x] Have you packaged and deployed your Operator for Operator Framework?

[x] Is your submission signed?

Updates to existing Operators

[x] Is your new CSV pointing to the previous version with the replaces property?
[x] Is your new CSV referenced in the appropriate channel defined in the package.yaml ?
[x] Have you tested an update to your Operator when deployed via OLM?
[x] Is your submission signed?

Your submission should not

[x] Modify more than one operator
[x] Modify an Operator you don't own
[x] Rename an operator - please remove and add with a different name instead
[x] Submit operators to both upstream-community-operators and community-operators at once
[x] Modify any files outside the above mentioned folders
[x] Contain more than one commit. Please squash your commits.

Operator Description must contain (in order)

[x] Description about the managed Application and where to find more information
[x] Features and capabilities of your Operator and how to use it
[x] Any manual steps about potential pre-requisites for using your Operator

Operator Metadata should contain

[x] Human readable name and 1-liner description about your Operator
[x] Valid category name1
[x] One of the pre-defined capability levels2
[x] Links to the maintainer, source code and documentation
[x] Example templates for all Custom Resource Definitions intended to be used
[x] A quadratic logo

Remember that you can preview your CSV here.
--
1 If you feel your Operator does not fit any of the pre-defined categories, file a PR against this repo and explain your need
2 For more information see here

@ricardozanini, please check https://operatorhub.io/preview as it doesn't show basic info after importing your CSV.

@J0zi many thanks reviewing this PR!
I don't understand, which basic info am I missing? In the Preview I can see the description of the CR, the summarized doc for what the operator do and all the links related.
Thanks!

@J0zi could you please help me understanding which basic info am I missing? The preview is fine for me importing the CSV from those PR. Many thanks!

@ricardozanini, sorry my fault. Preview showed some empty page containing just name of the operator. But I see you are right, all is there, so merging:)


community[patch]: fix hunyuan client json analysis (#22452)

Thank you for contributing to LangChain!

[x] PR title: "package: description"

Where "package" is whichever of langchain, community, core, experimental, etc. is being modified. Use "docs: ..."  for template changes, "infra: ..." for CI changes.
Example: "community: add foobar LLM"

[x] PR message: Delete this entire checklist and replace with

Description: a description of the change
Issue: the issue # it fixes, if applicable
Dependencies: any dependencies required for this change
Twitter handle: if your PR gets announced, and you'd like a mention, we'll gladly shout you out!

[x] Add tests and docs: If you're adding a new integration, please include

a test for the integration, preferably unit tests that do not rely on network access,
an example notebook showing its use. It lives in docs/docs/integrations directory.

[x] Lint and test: Run make format, make lint and make test from the root of the package(s) you've modified. See contribution guidelines for more: https://python.langchain.com/docs/contributing/

Additional guidelines:

Make sure optional dependencies are imported within a function.
Please do not add dependencies to pyproject.toml files (even optional ones) unless they are required for unit tests.
Most PRs should not touch more than one package.
Changes should be backwards compatible.
If you are adding something to community, do not re-import it in langchain.

If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.

@baskaryan Lgtm, seems like it resolves an error that was raised previously https://github.com/langchain-ai/langchain/issues/22452


Comment for Bitbucket stopped working

Apparently pronto still uses bitbucket v1.0 API for reading and posting comments. This API has just been removed (it now returns http error code 410).
We need to update to v2.0 API to keep this behaviour.

This bug could be closed after this pull request #352 is merged.

Is there any prediction on when the version with bitbucket API v2 support will be released?

The PR has been merged. I would like to get a few others merged in before cutting a new release.
Is it possible for you to point to GH for now?

Unfortunately not.
All my projects are stored on Bitbucket.
I totally understand the the wait time for other merges. I will try to make a fork and solo release to use on my projects.
Thanks 😁

@doomspork I pointed the pronto gem to use the master branch and there still appears to be error around comments. Now though it's an error with #parse_comments.
I'm very unfamiliar with the bitbucket api, but I'm wondering if how the responses are parsed needs to get updated as well.

See this PR, it should fix it #354
I had issues too