Training with GPU; requires replacing scikit-learn

There's very much a question of whether this is worth the leap for us in the short term, but either way it's worth documenting what this might involve.
First of all, we use scikit-learn for running training, whether it's MLP (multilayer perceptron) or LR (logistic regression) / SGD (stochastic gradient descent) training.
Second, scikit-learn is CPU-only and their MLP is apparently on the simple side:

scikit-learn FAQ:

Will you add GPU support?
No, or at least not in the near future. The main reason is that GPU support will introduce many software dependencies and introduce platform specific issues. scikit-learn is designed to be easy to install on a wide variety of platforms. Outside of neural networks, GPUs don’t play a large role in machine learning today, and much larger gains in speed can often be achieved by a careful choice of algorithms.

Note that MLP = neural network.

Deep learning and reinforcement learning both require a rich vocabulary to define an architecture, with deep learning additionally requiring GPUs for efficient computing. However, neither of these fit within the design constraints of scikit-learn; as a result, deep learning and reinforcement learning are currently out of scope for what scikit-learn seeks to achieve.
You can find more information about addition of gpu support at Will you add GPU support?.
Note that scikit-learn currently implements a simple multilayer perceptron in sklearn.neural_network. We will only accept bug fixes for this module. If you want to implement more complex deep learning models, please turn to popular deep learning frameworks such as tensorflow, keras and pytorch

scikit-learn neural networks page:

Warning
This implementation is not intended for large-scale applications. In particular, scikit-learn offers no GPU support. For much faster, GPU-based implementations, as well as frameworks offering much more flexibility to build deep learning architectures, see Related Projects.

A Stack Overflow Q&A:

SciKit Learn is a general machine learning library, built on top of NumPy. It features a lot of machine learning algorithms such as support vector machines, random forests, as well as a lot of utilities for general pre- and postprocessing of data. It is not a neural network framework.
... If you're doing deep learning, scikit-learn may still be useful for its utility part; aside from it you will need the actual deep learning framework [such as Keras or PyTorch] ...

Follow-up comment and reply:

You can build neural net classifiers in Sci-Kit though, right?
yes, you can but it's more of a toy feature for learning without hardware (GPU) acceleration

So, GPU is relevant for MLP training generally, and scikit-learn can't do GPU.
Next there's the question of how much work it'd take to support GPU-training in pyspacer:

Need to pick a different ML library which supports GPU training, and need to learn how to install and use that library.

The actual training calls happen in train() of train_utils.py, a function that's about 70 lines long (although somewhat dense). So the scikit-learn calls and logic in that function need to be replaced with calls/logic of a different ML library. I'm sure the inputs also need to be in a slightly different format, so there'll be some data manipulation work to be done.

classify_features needs to be reworked to accept classifiers created by the different ML library. The classifier format we currently use is a pickle of CalibratedClassifierCV, a scikit-learn class, so the current format's scikit-learn-specific. (Side note: it would be great to move away from library-specific classifier formats, regardless of whether that move is done as part of a move towards GPU or not)

For compatibility with existing coralnet classifiers, as well as for experimentation purposes, training should support: scikit-learn CPU training, non-scikit-learn GPU training, and probably also non-scikit-learn CPU training. Classification should support scikit-learn classifiers and non-scikit-learn classifiers (ideally a common interchange format for better future-proofing).

I'll make a follow-up comment on potential ML libraries for GPU training.

Potential libraries for GPU training
First of all, there are the low-level deep learning frameworks (PyTorch, Tensorflow, Google JAX), and then there are libraries which define more approachable interfaces on top of those frameworks (Keras, PyTorch Lightning, fastai). My general impression is that you want one of the more approachable interfaces unless you're doing heavy customization/research, which we aren't yet. But, the framework may or may not influence the choice of library.
So far, I don't know which framework is "better" overall, but quick notes on them:

Tensorflow is the oldest, then PyTorch, then JAX which is the newest.

Tensorflow was owned by Google, but it's FOSS under Apache license as of 2015; their engineers may still have a big part in development though. PyTorch was owned by Meta, but it's FOSS under modified BSD license as of 2022, and is now part of the Linux Foundation umbrella; their governing board apparently has reps from various corporations - Amazon, Google, Microsoft, Meta etc. JAX is owned by Google.

My notes on libraries so far - very preliminary stuff, partly based on word on the street at Stack Overflow / Reddit / etc., and obviously there are more libraries out there:

Keras: Originally Tensorflow based, but released support for PyTorch and JAX very very recently (version 3.0, 1-2 weeks ago!).

PyTorch Lightning: Apparently the high-level-ness is comparable to Keras; that is, seemingly balanced enough to work for most folks. First public release was 2019.

PyTorch Ignite: Part of PyTorch's GitHub org. I'm uncertain on its maturity as a library, as it's currently on version "0.4.13", but the first public release was 2018 which is not all that young.

fastai: PyTorch based, and inspired by Keras. Seems to be one of the most opinionated library choices, easy to get going and does use best practices, but might be painful to do even moderate customizations. Their quick-start guide starts with from X import * statements which is not great practice in Python. I'm particularly unimpressed by their landing page, with the feed of clickbait-headline posts, and the "Make ___ ___ again" header.

FWIW: I've used Pytorch lighting and it's good choice if we go down the
Pytorch route....
You could probably code this up on 4-8 hours for sigle threaded CPU
training.
Optimizing on GPU isn't that hard really, although there are some
hyperparameters that affect performance.
David
On Thu, Dec 7, 2023 at 3:41 PM StephenChan @.***> wrote:

Potential libraries for GPU training
First of all, there are the low-level deep learning frameworks (PyTorch,
Tensorflow, Google JAX), and then there are libraries which define more
approachable interfaces on top of those frameworks (Keras, PyTorch
Lightning, fastai). My general impression is that you want one of the more
approachable interfaces unless you're doing heavy customization/research,
which we aren't yet. But, the framework may or may not influence the choice
of library.
So far, I don't know which framework is "better" overall, but quick notes
on them:

Tensorflow is the oldest, then PyTorch, then JAX which is the newest.
Tensorflow was owned by Google, but it's FOSS under Apache license as
of 2015; their engineers may still have a big part in development though.
PyTorch was owned by Meta, but it's FOSS under modified BSD license as of
2022, and is now part of the Linux Foundation umbrella; their governing
board apparently has reps from various corporations
https://urldefense.com/v3/__https://pytorch.org/foundation__;!!Mih3wA!D7ZO3tozwHfKIi617O4WOVH8YIQJoNkmAOSTp8VUmMSMKyyJUmFW0z3UU5IFagV18zGOQ0dhGAfd_IeqPt_vXFL-$

Amazon, Google, Microsoft, Meta etc. JAX is owned by Google.

My notes on libraries so far - very preliminary stuff, partly based on
word on the street at Stack Overflow / Reddit / etc., and obviously there
are more libraries out there:

Keras: Originally Tensorflow based, but released support for PyTorch
and JAX very very recently (version 3.0, 1-2 weeks ago!).
PyTorch Lightning: Apparently the high-level-ness is comparable to
Keras; that is, seemingly balanced enough to work for most folks. First
public release was 2019.
PyTorch Ignite: Part of PyTorch's GitHub org. I'm uncertain on its
maturity as a library, as it's currently on version "0.4.13", but the first
public release was 2018 which is not all that young.
fastai: PyTorch based, and inspired by Keras. Seems to be one of the
most opinionated library choices, easy to get going and does use best
practices, but might be painful to do even moderate customizations. Their
quick-start guide starts with from X import * statements which is not
great practice in Python. I'm particularly unimpressed by their landing
page, with the feed of clickbait-headline posts, and the "Make ___ ___
again" header.
—
Reply to this email directly, view it on GitHub
https://urldefense.com/v3/__https://github.com/coralnet/pyspacer/issues/74*issuecomment-1846271077__;Iw!!Mih3wA!D7ZO3tozwHfKIi617O4WOVH8YIQJoNkmAOSTp8VUmMSMKyyJUmFW0z3UU5IFagV18zGOQ0dhGAfd_IeqPisAZh4a$,
or unsubscribe
https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABKA5AMOZHYUDAXAJ2ZXXBDYIJHZDAVCNFSM6AAAAABALZXSBCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBWGI3TCMBXG4__;!!Mih3wA!D7ZO3tozwHfKIi617O4WOVH8YIQJoNkmAOSTp8VUmMSMKyyJUmFW0z3UU5IFagV18zGOQ0dhGAfd_IeqPvNSeiIb$
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Good to know, as from what I've seen so far, I was also slightly leaning towards PyTorch Lightning. So I guess the migration from scikit-learn's pickled classifier format could really be the part that takes the most work.

+1 for Lightning. I think this might also be a decent case for setting up some kind of mlops for experimentation too.
Here's a quick way to add the gpu as an option to torch_utils.py
https://github.com/yeelauren/pyspacer/blob/train_coralnet/spacer/torch_utils.py

Ah interesting - offhand I can't tell if cuML can do MLP, but might be an option if we end up being flexible on the training algorithm.
I've also just heard about skorch but I'm also not sure what it could do for us yet.

A couple of thoughts on this.
If we go to deep learning, I'd also vote for PyTorch with some wrapper, like lightning.
However, I don't think it makes much sense to move to GPU based training for LR or 1-2 layer hidden MLPs. I think there is a lot we can do to set convergence criterion, thread data loading while computing gradients, and so on.
So the reason to go to GPUs, IMO, would be if we move to deep fine-tuning of the base networks.  There is no doubt that would lead to more accurate classifiers. In particular for larger sources.
The issue/complexity with deep fine-tuning is on the invoke (predict) step. The fine-tuned nets will be the same size (in bytes) as the original network. That means that there is lot more data to load up when the predict request arrives. Which will affect invoke latency.
Anyways, my 2 cents.

Thanks for chiming in, Oscar - yeah, I can at least confirm there is much more we can look at for optimizing classifier training, before moving to GPU (I'm currently looking into the data loading). As we do so, we should keep benchmarking and looking for where the bottleneck is.
I'm definitely not familiar enough with the network-tuning process to fully understand that part, but will keep your notes in mind!

I'm planning to work on a pytorch implementation for the MLP in the near future. I believe it will be beneficial for MERMAID and CoralNet. Additionally, I'm not satisfied with the limitations of sklearn regarding #98. Therefore, I think it would be better to work towards a pytorch implementation for the long-term goal or at the very least for the classification.


Autoskill fails click on animations which redistribute crit stars

Environment
Samsung Galaxy S9+
Ankulua 8.5.3 Pro2
Description
Anytime autoskill uses a skill which causes critical stars to redistribute, the next click will fail and the script usually halts there.
I have currently encountered this with both Gil and Raikou's star absorb skills as well as moving a unit in with plugsuit.
I have managed to "fix" the issue simply by adding a wait(1) in the CastSkill() function.

Oh I see! This explains why there are occasional issues with the CastSkill() function. I never noticed that it was specifically star redistribution skills that caused it. Thanks for the info! I have a similar fix implemented in one of my personal repos that is currently pending review in the Pull Requests section.


Reactor Stopped

I'm trying the new tokio2 branch with a simple subscriber.
I'm trying to subscribe to two topics:
let (mut client, receiver) = MqttClient::start(client_options);
let topics = vec![
    ("+/devices/+/activations", QoS::AtMostOnce),
    ("+/devices/+/up", QoS::AtMostOnce),
];
client.subscribe(topics).expect("Subcription failure");

thread::spawn(move || {
    println!("--> Listening!");
    for packet in receiver {
        println!("received {:?}", packet);
    }
}).join().unwrap();

When I start the client:
INFO 2018-02-03T01:04:16Z: rumqtt::client::connection: mqtt connection successful
DEBUG 2018-02-03T01:04:16Z: rumqtt::client::connection: Sending packet. Subscribe(Subscribe { pid: PacketIdentifier(1), topics: [SubscribeTopic { topic_path: "+/devices/+/activations", qos: AtMostOnce }, SubscribeTopic { topic_path: "+/devices/+/up", qos: AtMostOnce }] })
DEBUG 2018-02-03T01:04:16Z: rumqtt::client::connection: Received packet. "Suback(Suback { pid: PacketIdentifier(1), return_codes: [Success(AtMostOnce), Success(AtMostOnce)] })"
DEBUG 2018-02-03T01:04:16Z: rumqtt::client::connection: Sending packet. Suback(Suback { pid: PacketIdentifier(1), return_codes: [Success(AtMostOnce), Success(AtMostOnce)] })
ERROR 2018-02-03T01:04:16Z: rumqtt::client::connection: Reactor stopped. v = ()

Why is the reactor stopped?
And why is the thread not terminated (= the packet receiver closed) if the reactor is stopped?
 You have discovered a bug. The client is sending wrong suback 2nd time. Mosquitto seems to be lenient with this. Which broker are you using?
I've pushed a fix. Can you test now?

Which broker are you using?

I don't know, I'm communicating with TTN servers :)

I've pushed a fix. Can you test now?

Will try, hopefully tonight.

The double suback seems fixed, but I still lose connection...
 INFO 2018-02-03T19:37:29Z: rumqtt::client::connection: Address resolved to Some(V4(<IP_ADDRESS>:1883))
 INFO 2018-02-03T19:37:29Z: rumqtt::client::connection: mqtt connection successful
DEBUG 2018-02-03T19:37:29Z: rumqtt::client::connection: Sending packet. Subscribe(Subscribe { pid: PacketIdentifier(1), topics: [SubscribeTopic { topic_path: "+/devices/+/activations", qos: AtMostOnce }, SubscribeTopic { topic_path: "+/devices/+/up", qos: AtMostOnce }] })
DEBUG 2018-02-03T19:37:29Z: rumqtt::client::connection: Received packet. "Suback(Suback { pid: PacketIdentifier(1), return_codes: [Success(AtMostOnce), Success(AtMostOnce)] })"
received Suback(Suback { pid: PacketIdentifier(1), return_codes: [Success(AtMostOnce), Success(AtMostOnce)] })
ERROR 2018-02-03T19:37:32Z: rumqtt::client::connection: Reactor stopped. v = ()
 INFO 2018-02-03T19:37:32Z: rumqtt::client: Will sleep for Duration { secs: 10, nanos: 0 } seconds before reconnecting

 INFO 2018-02-03T19:37:42Z: rumqtt::client::connection: Address resolved to Some(V4(<IP_ADDRESS>:1883))
 INFO 2018-02-03T19:37:42Z: rumqtt::client::connection: mqtt connection successful
ERROR 2018-02-03T19:37:45Z: rumqtt::client::connection: Reactor stopped. v = ()
 INFO 2018-02-03T19:37:45Z: rumqtt::client: Will sleep for Duration { secs: 10, nanos: 0 } seconds before reconnecting

Can I debug something about this without access to server logs?

@dbrgn Broker is disconnecting the client for some reason. It's difficult to debug this with out logs. One way to cross check is to connect using some other mqtt client and see if it works.

It seems to work with paho-mqtt (python). There I'm connecting via TLS though. Maybe that's the issue? (I would have expected that subscriptions wouldn't even be acked if TLS is required by the server.)
So, how would I set up TLS? This is all I can find in the docs:

The code has a comment saying "ca and, optionally, a pair of client cert and client key". So, what is ca? Is it a string to a path to some kind of certificate? If yes, what format? PEM/DER? Or is it a string of the PEM itself?

@dbrgn Can you paste you MqttOptions here

Sure!
let client_options = MqttOptions::new("smartmail", "eu.thethings.network:1883")
        .unwrap_or_else(|e| {
            println!("Could not initialize MqttOptions: {}", e);
            exit(3);
        })
        .set_keep_alive(10)
        .set_reconnect_opts(ReconnectOptions::AfterFirstSuccess(10))
        .set_connection_method(ConnectionMethod::Tcp)
        .set_security_opts(SecurityOptions::UsernamePassword((
            conf.ttn_app_id.clone(),
            conf.ttn_access_key.clone(),
        )));

Even if commenting out the subscription and everything except the basic auth:
let client_options = MqttOptions::new("smartmail", "eu.thethings.network:1883")
        .unwrap_or_else(|e| {
            println!("Could not initialize MqttOptions: {}", e);
            exit(3);
        })
        .set_security_opts(SecurityOptions::UsernamePassword((
            conf.ttn_app_id.clone(),
            conf.ttn_access_key.clone(),
        )));

...it still fails.
This is not about the authentication though, if I change the basic auth password to "foo", connecting fails:
 INFO 2018-02-04T12:00:44Z: rumqtt::client::connection: Address resolved to Some(V4(<IP_ADDRESS>:1883))
ERROR 2018-02-04T12:00:49Z: rumqtt::client: Network connection failed. Error = MqttConnectionRefused(5), Connection count = InitialConnect

I tried TLS:
.set_connection_method(ConnectionMethod::Tls("mqtt-ca.pem".into(), None))

...where mqtt-ca.pem is this file in the current directory. (This is also used successfully for the connection through Python.)
When running this, connecting fails:
 INFO 2018-02-04T12:05:01Z: rumqtt::client::connection: Address resolved to Some(V4(<IP_ADDRESS>:1883))
ERROR 2018-02-04T12:05:01Z: rumqtt::client: Network connection failed. Error = Io(Custom { kind: Other, error: Error { code: ErrorCode(5), cause: Some(Io(Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" })) } }), Connection count = InitialConnect

I asked the server maintainers for logs, but I don't know yet if I'm going to get any.

@dbrgn If you are sure that the username and password that you are reading from config is right, this should've worked. I'll check if basic username password auth works with mosquitto broker

Here's a network trace:

The TCP connection is closed after 3 seconds by the server. I guess this is probably something on the server side.

Could the protocol name difference be relevant?

@dbrgn Ahhh yeahh..This library only supports the latest mqtt 3.1.1

@tekjar can we do something about it to add support for older versions? and if not, is there a way to show a better error message?

@dbrgn

can we do something about it to add support for older versions

I'm not very keen about this. Most of the brokers support latest version.

is there a way to show a better error message

Client can't do much as it is the one initiating the connection. Broker should send connack with proper error return code. I'll add a debug print and see the return code in connack

It's been very long (2014 to be precise) since 3.1.1 got released

Ah, I wasn't aware that it's that long. I'll ask the backend folks about it :)

The awesome @htdvisser pointed out to me that only one MQTT client with the same ID is allowed to be connected to the server at the same time, and I used a static client ID ("smartmail"). I had another version of the code (based on the non-async branch) running on the server. I assume that the local connection caused the server to drop my other instance, which then automatically reconnected, resulting in a connection drop of the async client a few seconds later 🙈
Maybe a note could be added to the docs about this. Otherwise, the issue can be closed. Thanks @tekjar for the debugging help! :)

@dbrgn

only one MQTT client with the same ID is allowed to be connected to the server at the same time
Ahh yeah, broker should do this to recover from halfopen connections. Would've been easier if you had access to broker logs (:


Make the checkbox checked for already included elements restrictions in assignment while editing.

Is your feature request related to a problem? Please describe.
Currently, if some elements restrictions are included and mentor have to edit the assignment, the checkbox for already included is not checked which may lead to confusing for mentors to reselect.
Describe the solution you'd like
Make the checkbox checked for already included elements restrictions in the assignment while editing.
Are you working on this? (Yes/No)
Yes, but if new contributors want to work they can work.

I would like to work on this issue. I request to provide a bit more detail on it please.

Hi! I would like to work on this issue. Can I contribute as well?

@Swayamshu thanks for your initiative but I have started working on the issue and will raise a PR soon. Kindly check other unassigned issues.

@Swayamshu thanks for your initiative but I have started working on the issue and will raise a PR soon. Kindly check other unassigned issues.

Ok, no issues. Thank you


Syntax not highlighted until exiting insert mode

I've recently run into an issue where anything I type in insert mode doesn't receive syntax highlighting until I return to normal mode.

Syntax highlighting works fine if I disable the extension. I also validated that it isn't caused by conflict with another extension by disabling all extensions except for vscode-neovim.
Versions
VSCode: 1.59.0
neovim: 0.5.0
vscode-neovim: 0.0.82

Try with empty init.vim. Maybe you turned on syntax in nvim.

@with-heart Treesitter was causing the issue for me, and following the config in https://github.com/vscode-neovim/vscode-neovim/issues/609#issuecomment-1042796958 worked:
    highlight = {
        enable = (not vim.g.vscode),
    },


Improve terraform documentation

One of early adopters reported following:
Few hiccups I experienced.

1. The terraform readme should include instructions for terraform.tfvars, it's not exactly clear what variables to be set

2. Before running terraform apply there is a need to run terraform init from a fresh installation

Linked to #40, if the documentation sufficient for now I think it is fine to close this issue.


DBFS is the default location for managed tables

A managed table is a Spark SQL table for which Spark manages both the data and the metadata. In the case of managed table, Databricks stores the metadata and data in DBFS in your account. Since Spark SQL manages the tables, doing a DROP TABLE example_data deletes both the metadata and data.

This statement is true if no location is specified for the schema in which that table is defined (ie if it is a managed schema as well).  If the schema is defined with a location, and if a table is defined within it without specifying the location, though that table will indeed be considered managed, the table objects will be stored in that external location.  This page incorrectly states that the table objects will be stored in DBFS.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ccfaeca7-133f-9c75-2cf9-9d6dd04b36b9
Version Independent ID: 54c71bbc-cbe8-f467-8c40-a38b8d76bf6d
Content: Databases and tables - Azure Databricks
Content Source: databricks/data/tables.md
Service: azure-databricks
GitHub Login: @mssaperla
Microsoft Alias: saperla

Thanks for the feedback! We have assigned the issue to the content author to further review this and update the document as appropriate.

#please-close


updated: Add xr curl

[x] All tests passed. If this feature is not already covered by the tests, I added new tests.
[x] This pull request is on the dev branch.
[x] I used gofmt for formatting the code before submitting the pull request.

Related to closed PR: https://github.com/jfrog/jfrog-cli-core/pull/88


Add CSI driver for GCP Filestore

What this PR does / why we need it:

Add GCP Filestore CSI driver addon
Automatically deploy addon if external cloud controller manager (.cloudProvider.external) is enabled
Add default StorageClass for GCP Filestore CSI driver to the default-storage-class embedded addon

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
CCM support is required to move forward with this PR. Currently, CCM for gce/gcp is not in a usable state. Please check cloud-provider-gcp for CCM status.
Does this PR introduce a user-facing change?:

* Add GCP Filestore CSI driver addon
* Automatically deploy addon if `.cloudProvider.external` is enabled
* Add default StorageClass for GCP Filestore CSI driver to the default-storage-class embedded addon

Closing this PR, we'll cover this in a fresh PR.


Signature Verification example using ring crate

Hi
I am trying to play around with some rust TA examples and I found that the signature_verification-rs example is using the ring module from a specific repo (https://github.com/veracruz-project/ring.git)
My questions are the following:

is it safe and secure to use external crate in the TA for cryptographic operations instead of tee api?
why use a specific repo for ring and not the official crate? (https://crates.io/crates/ring)
are there other crypto crate we could use in TA?

Thanks

Hi @syedelec

is it safe and secure to use external crate in the TA for cryptographic operations instead of tee api?

TEE Core APIs are defined in the standard specification and implemented differently in various TEE OS. OP-TEE uses libtomcrypt as its default implementation of the crypto APIs. It also supports adding a new software implementation.
It's okay to use the third-party crypto libraries in TAs. Its security depends on the security of the crypto library itself.

2. why use a specific repo for ring and not the official crate? (https://crates.io/crates/ring)

Ring in veracruz-project has been ported to OP-TEE (e.g. this commit).

3. are there other crypto crate we could use in TA?

Other crypto crates can be used in TA after they have been ported to OP-TEE's standard library. Ring is the only crypto crate supported now.

Thanks! That's way clearer now!


Introduce BuildController.sendIntermediate

Context

Contributor Checklist

[ ] Review Contribution Guidelines
[ ] Make sure that all commits are signed off to indicate that you agree to the terms of Developer Certificate of Origin.
[ ] Make sure all contributed code can be distributed under the terms of the Apache License 2.0, e.g. the code was written by yourself or the original code is licensed under a license compatible to Apache License 2.0.
[ ] Check "Allow edit from maintainers" option in pull request so that additional changes can be pushed by Gradle team
[ ] Provide integration tests (under <subproject>/src/integTest) to verify changes from a user perspective
[ ] Provide unit tests (under <subproject>/src/test) to verify logic
[ ] Update User Guide, DSL Reference, and Javadoc for public-facing changes
[ ] Ensure that tests pass sanity check: ./gradlew sanityCheck
[ ] Ensure that tests pass locally: ./gradlew <changed-subproject>:quickTest

Reviewing cheatsheet
Before merging the PR, comments starting with

❌ ❓must be fixed
🤔 💅 should be fixed
💭 may be fixed
🎉 celebrate happy things

According to @donat, IDEA already has a solution for streaming models and it's potentially better regarding backwards compatibility, we should reassess this solution after he had a chance to sync with the JB folks.

Turns out my initial reassessment was wrong, and IDEA uses protobuf to stream model but not to slice up the model building.

Superseded by https://github.com/gradle/gradle/pull/27095


[ja] Translate concepts/scheduling-eviction/scheduling-framework/ into Japanese

Resolve #31886

If I should translate "Scheduling Cycle" and "Binding Cycle" as 「スケジューリングサイクル」「バインディングサイクル」, please let me know. I will replace it and recommit.

/assign @nasa9084

Hi @kakts, Thank you for your review!

@196Ikuchil Thank you for your contribution.
/lgtm

sorry for late reviewing~
/approve


JSX

This adds support for the JSX AST tree in the walker.
This requires JSX parsing, e.g. with acorn-jsx.

We're not including node types our parser doesn't produce in the walker -- since that'd be an open-ended set. You can implement and distribute it as a separate module (mutating the walker object namespace from outside).


[MENFORCER-185] [WIP] Excludes take reactorProjects into account

TODO require IT
Enforce only when release
<plugin>
	<artifactId>maven-enforcer-plugin</artifactId>
	<execution>
		<id>enforce-no-snapshots</id>
		<goals>
			<goal>enforce</goal>
		</goals>
		<configuration>
			<rules>
				<requireReleaseDeps>
					<onlyWhenRelease>true</onlyWhenRelease>
				</requireReleaseDeps>
			</rules>
		</configuration>
	</execution>
</plugin>

Or
<plugin>
	<artifactId>maven-enforcer-plugin</artifactId>
	<execution>
		<id>enforce-no-snapshots</id>
		<goals>
			<goal>enforce</goal>
		</goals>
		<configuration>
			<rules>
				<requireReleaseDeps>
					<failWhenParentIsSnapshot>false</failWhenParentIsSnapshot>
					<excludes>
						<exclude>${project.groupId}:*</exclude>
					</excludes>
				</requireReleaseDeps>
			</rules>
		</configuration>
	</execution>
</plugin>

ref: https://stackoverflow.com/q/40591346/942671

Closed in favor of https://github.com/apache/maven-enforcer/pull/35


Use new template with capitalised case type

https://trello.com/c/C34ub2CM/171-case-created-notifications-have-incorrect-capitalisation-in-subject
Description
I have added a new template here which uses a variable for the capitalized case type in the subject.
https://www.notifications.service.gov.uk/services/b1dab0f5-4651-4af8-9f15-fa8143ff338d/templates/b5457546-9633-4a9c-a844-b61f2e818c24
And made a small change to the mailer to use the new template and added in the new variable.

Checklist:

[ ] Have you documented your changes in the pull request description?
[ ] Does the change present any security considerations?
[ ] Is any gem functionality overloaded? Eg: Devise controller methods being overloaded.
[ ] Has acceptance criteria been tested by a peer?
[ ] Has the CHANGELOG been updated? (If change is worth telling users about.)

General testing (author)

[ ] Test without JavaScript
[ ] Test on small screen

Accessibility testing (author)

[ ] Works keyboard only
[ ] Tested with one screen reader
[ ] Zoom page to 400% - content still visible
[ ] Disable CSS - does content make sense and appear in a logical order?

Codecov Report

Merging #1571 (3de1ec2) into master (c472429) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1571   +/-   ##
=======================================
  Coverage   89.73%   89.73%
=======================================
  Files         388      388
  Lines        8317     8317
  Branches     1218     1218
=======================================
  Hits         7463     7463
  Misses        679      679
  Partials      175      175

Impacted Files
Coverage Δ

app/mailers/notify_mailer.rb
93.18% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c472429...3de1ec2. Read the comment docs.


Implemented Distance Feature Query

Added the Distance Feature Query
See #1202

Seems like we were working on the same thing at the same time. Added this already in 13dd156dabeed67c79d06c66661533697c7c8912 and e4cd810912a7387a965286b6e18d06e12dba7f5f.


Exit from Kiosk

Can not exit from Kiosk mode. After pressing ctrl + alt + s , the screen goes blank. Nothing happens

Hi, Were you able to find the solution for this. I am stuck on this issue as well. Thank you.

Hi, Were you able to find the solution for this. I am stuck on this issue as well. Thank you.

Not yet. Seems, there were no solution. Also, the kiosk mode reset on every restart. It will only usable for static web sites.

This is not an issue, it's just an unimplemented feature.
"Kiosk mode" is part of the FydeOS enterprise solution offering. In this project, we are shipping a "proof of concept" and aim to demonstrate how this works in code level.


Reverts should be tested in Dynamic sale

We should make sure that reverts are triggered for the right reasons and the protect efficiently a right full buy

Here is an example of how I create helpers to test
https://github.com/closerdao/proof-of-presence/blob/99d298b5d8e5b328b02bcc38eacd5885435ac035/test/utils/diamond/adminHelpers.ts#L9-L117
Here are the tests using those helpers:
https://github.com/closerdao/proof-of-presence/blob/99d298b5d8e5b328b02bcc38eacd5885435ac035/test/diamond/MembershipFacet.test.ts#L24
The missing tests are here:
https://github.com/closerdao/proof-of-presence/blob/99d298b5d8e5b328b02bcc38eacd5885435ac035/test/DynamicSale.test.ts#L62


nixos/mobilizon: convert to mdDoc, remove obsolete override

Description of changes
Follow up on https://github.com/NixOS/nixpkgs/pull/119132#issuecomment-1716498185
Unfortunately I didn't manage to reproduce the errors, so couldn't check wether all of them went away.
Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

This look good, waiting until tomorrow in case Sandro wants to add anything.

I've just seen the warning while deploying it. If it is now fixed that's fine for me.

Okay cool, just ping me if you happen to see some leftover warning.


Currency module doc

Documentation to go along with https://github.com/prebid/Prebid.js/pull/1374

Added in a tweak to the DigiTrust page in the same modules area as well -- accounting for the code inspection comments from https://github.com/prebid/Prebid.js/pull/1428

Note that this PR also includes:

new user-sync config in the 0.27 version of the Rubicon adapter
tweak to the DigiTrust module instructions to support setConfig

Thanks @bretg!  Awesome docs.

@bretg These docs may need an update once prebid 1.0 is released since it looks like currency definition will become required: https://github.com/prebid/Prebid.js/issues/891#issuecomment-335268378
Regarding this note:

Publishers may continue to use the bidCpmAdjustment approach, or may begin using this optional module, gaining automatic updates as currency exchange rates fluctuate.

I'm not 100% clear on how this will affect publishers who already have bidCpmAdjustments in effect. Previously there was no assumption of USD, or defaulting to USD, so an existing hardcoded bidCpmAdjustment might need to be changed. While bidCpmAdjustments will still be supported it doesn't sound like the currency module is optional.

@bwoolcott - the currency conversion feature isn't turned on unless the page explicitly turns it on with a setConfig({currency: {}}). e.g.
pbjs.setConfig({
    "priceGranularity": "low",
    "currency": {
       "adServerCurrency": "JPY",
       "granularityMultiplier": 108
    }
});

So unless I'm misunderstanding your point, I don't believe there's a problem with existing bidCpmAdjustments -- if pubs would prefer to keep using bidCpmAdjustment, that's fine. If they want to move to the more dynamic currency feature, they would remove the bidCpmAdjustment at that time.


Integer-valued operators

Hi all, I'm super excited to start using the new lineax library to re-implement some of my own linear operator functionality in statistical genetics. We commonly use integer valued matrices to represent genotype data (0/1/2) and can represent a centered/scaled genotype matrix as a linear operator (I'll skip details for now). Currently, lineax enforces float-valued operators, as examplified by
        if not jnp.issubdtype(matrix, jnp.inexact):
            matrix = matrix.astype(jnp.float32)

Is there room for either an Int operator, or to use generic types to generalize the matrix operator by underlying numeric type?

Hmm, that is -- you want to solve an $Ax = b$ problem where both $A$ and $b$ are integers, and the construction of the problem is such that $x$ is known to have an integer value as well?
What kind of linear solver are you currently using to solve this?

Hi @patrick-kidger , I'm not suggesting that b must be integer, only A. Without going into too much detail, this is related to space/memory, where we can use small integer representation (e.g., int8), since we only need to cover 3 values (which can be further improved by leveraging sparsity of the 0s).

Ah, you're primarily just interested in representing the linear operator, not using it to solve a linear system? You could do that like this, then:
import equinox as eqx
import jax
import jax.experimental.sparse as js
import jax.numpy as jnp
import lineax as lx

class SparseQuantisedOperator(lx.AbstractLinearOperator):
    matrix: js.BCOO

    def mv(self, vector):
        return js.sparsify(lambda m, v: m @ v)(self.matrix, vector)

    def as_matrix(self):
        raise ValueError("Refusing to materialise sparse matrix.")
        # Or you could do:
        # return self.matrix.todense()

    def transpose(self):
        return SparseQuantisedOperator(self.matrix.T)

    def in_structure(self):
        _, in_size = self.matrix.shape
        return jax.ShapeDtypeStruct((in_size,), jnp.float32)

    def out_structure(self):
        out_size, _ = self.matrix.shape
        return jax.ShapeDtypeStruct((out_size,), jnp.float32)

#
# Example usage: solve a linear system.
#

@lx.is_symmetric.register
def _(op: SparseQuantisedOperator):
    return False

@lx.is_negative_semidefinite.register
def _(op: SparseQuantisedOperator):
    return False

@lx.linearise.register
def _(op: SparseQuantisedOperator):
    return op

matrix = js.BCOO.fromdense(jnp.array([[1.0, 0.0], [0.0, 1.0]]))
op = SparseQuantisedOperator(matrix)
sol = lx.linear_solve(op, jnp.array([1., 2.]), solver=lx.NormalCG(rtol=1e-5, atol=1e-5))
print(sol.value)

I think probably the generic approach. Otherwise you may have that operator.matrix.dtype != operator.dtype; it's generally better to make illegal data structures unrepresentable.


Set mower to idle state upon reaching daily limit

This pull request resolves #34 by setting the mower in idle state upon reaching daily maximum work capacity.
In accordance with the GARDENA Operator's Manual, distinct mower models have varying maximum work capacity that either is defined by a minimum daily standby time or a maximum cutting time within a day.
Previously, upon reaching the daily limit, the mower's state was set to error, which implied a need for manual intervention to rectify the situation. However, the daily limit reached, is only a temporary state and resets automatically at midnight and eliminating the need for manual intervention.
My GARDENA smart SILENO life 700 LONA seems to have a maximum cutting time within a day of ~6 hours (338 minutes of cutting and 14 minutes of returning to dock to be exact):

After ~6 hours of cutting and returning to the dock the state has been changed to idle, when using the changes in this commit.

Here's a snapshot of the state and the attributes after the mower has changed its state to idle.

Here's a snapshot from the GARDENA app for iOS:

Around midnight the state is automatically changed back to docked:

Hi @grm ,
slowly summer is coming up and I would be happy to have this merged.
Code lgtm. Is there anything that still has to be done before merging? Can I support you with that?

I have not seen this MR.
i merge it now and release a new beta !

Thx for the code :)


what's the difference between your method and classmix

Hello, I have read your paper, but I have a question of your paper.
My question is what's the difference between your method and classmix?
why not compare with classmix?

Hello, I have read your paper, but I have a question of your paper. My question is what's the difference between your method and classmix? why not compare with classmix?

Hi! Thanks for your good questions. The difference is that our method aims to alleviate the empirical distribution mismatch problem in semi-supervised learning by using labeled and unlabeled data symmetrically. In contrast, Classmix utilizes only labeled data through supervised loss and classmix within unlabeled data. There are too many methods exploring the potential of copy-paste, making it difficult and unrealistic for us to compare all of them.


V1.3.0 contributions BBDK

Ready for next iteration https://github.com/cryptosharks131/lndg/pull/94/commits/9cc8480cfe8992af4f6abe753f099ba8142a5d95


Can this work with Spring Boot also in the Unit Test

Using the below test I get this error
Caused by: java.lang.IllegalStateException: This must be done before route mapping has begun
This is because I have Spark Routes as Beans, which get initialized before the @ClassRule gets executed. I just cannot figure out the wiring to get them to play nice together.
Doubt it matters, but I also have a need to use dbunit to initialize H2 db, which is why you see that included.
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration
@TestExecutionListeners({DependencyInjectionTestExecutionListener.class,
        DbUnitTestExecutionListener.class
})
public class FooTest {

    @Configuration
    @ComponentScan(basePackages = {"com.jose.lotto"})
    static class SomeConfig {

        // getting called before @ClassRule
        @Bean
        public static PropertyPlaceholderConfigurer propConfig() {
            PropertyPlaceholderConfigurer ppc =  new PropertyPlaceholderConfigurer();
            ppc.setLocation(new ClassPathResource("application.properties"));
            return ppc;
        }
    }

    public static class TestControllerTestApplication implements SparkApplication {

        @Override
        public void init() {
            new SimpleTestController(); // a single Route to satisfy the test
        }
    }

    @ClassRule
    public static SparkServer2<FooTest.TestControllerTestApplication> testServer
            = new SparkServer2<>(FooTest.TestControllerTestApplication.class, 4567);;

    @Test
    public void sample() throws Exception {
        SparkClient.UrlResponse response = testServer.getClient().doMethod("GET", "/test", null);
        assertEquals(200, response.status);
        assertEquals("This works!", response.body);
        assertNotNull(testServer.getApplication());
    }
}

Closing since I dont think this problem is related to spark-test

Hi @leonj1. Sorry about my delay in the answer. I'm not an expert in Spring Boot, but I think is a problem with the order of execution of @ClassRule.
You can use the SparkServer without using the annotation @ClassRule (executing the methods before() and after() manually, although it may seem not very "pretty") or you can try to initialize your beans in a custom JUnit Rule and chain it after SparkServer using RuleChain

No worries. Thanks for the reply. With the help of StackOverflow folks I managed to sort things out. Here is that entry, which also contains a link to a working GitHub project: http://stackoverflow.com/questions/39919316/not-sure-why-this-unit-test-works-spring-configurationcontext-updating-or-creat


Recognition of Zealandia (Polypodiaceae) [NOT PASSED]

Author(s) of proposal
Ashley Field
Name of taxon
Zealandia Testo & A.R.Field
Rank of taxon
Genus
Approximate number of species affected
3
Description of change
Testo et al. (2019) described the genus Zealandia Testo & A.R.Field based on a molecular phylogenetic and morphological appraisal as part of a multi-genus approach to achieving monophyly from a previously paraphyletic Microsorum and Colysis. The abstract of Testo et al. (2019)  is reproduced below:
"The taxonomy of the Polypodiaceae subfamily Microsoroideae is highly problematic, especially with respect to the circumscription of the highly variable and non-monophyletic genus Microsorum. Using phylogenetic analyses and morphological evidence, we demonstrate that sixteen taxa typically treated in the genera Microsorum and Colysis are not closely related to those groups and instead belong to three clades that are successive sister groups to the Old-World ant-fern genus, Lecanopteris. We use the available genus name Dendroconche for one of these clades and propose the new genera Bosmania and Zealandia to accommodate the remaining two groups. We provide a description and identify morphological synapomorphies for each of the genera, make new combinations and designate lectotypes where necessary, and present keys and descriptions for all relevant species. We also discuss the evolution of ant-fern associations in the lecanopteroid ferns and highlight the need for additional taxonomic work in the subfamily. The following new combinations are provided: Bosmania lastii, B . leandriana, B . membranacea, Dendroconche ampla, D . latilobata, D . linguiforme, D . sayeri, D . scandens, D . varians, Zealandia novae-zealandiae, Z . powellii, Z . pustulata, Z . pustulata  subsp. howensis, and Z . vieillardii."
Reason for change
Multiple studies have shown that Microsorum and Colysis as adopted in PPG I (2016) were paraphyletic. This proposal is directed at achieving monophyly among relevant genera by recognition of three separate new genera for the clades presented by Testo et al. (2019) and also presented by Chen et al. (2020). This proposal should be read in conjunction with proposals for  Bosmania + Dendroconche + Lecanopteris s.s. sensu  Testo et al. (2019) (#25, #27).
This proposal affects species occurring primarily in Australia, New Caledonia and New Zealand. It was adopted by Australian Plant Census but not for the New Zealand Fern Flora. It has been superseded in World Ferns.
An alternative classification to achieve monophyly was presented by Perrie et al. (2021) by expanding the circumscription of Lecanopteris. This alternative affects the same number of taxa overall, and should be examined in the alternative PPG II proposals for Bosmania + Lecanopteris s.l. sensu Perrie et al. (2021) (#28).
Reference(s) for publication of the name
Testo W.L., Field A.R., Sessa E.B., Sundue M. (2019) Phylogenetic and Morphological Analyses Support the Resurrection of Dendroconche and the Recognition of Two New Genera in Polypodiaceae Subfamily Microsoroideae Systematic Botany 44(4): 1–16.
See also:
Chen C-C, Hyvönen J, Schneider H. 2020. Exploring phylogeny of the microsoroid ferns (Polypodiaceae) based on six plastid DNA markers. Molecular Phylogenetics and Evolution 143: 106665.
Field A.R. (2020) Classification and typification of Australian lycophytes and ferns based on Pteridophyte Phylogeny Group classification PPG I. Australian Systematic Botany 33(1): 1–102.
Perrie L.R., Field A.R., Ohlsen D.J., Brownsey P.J. (2021) Expansion of the fern genus Lecanopteris to encompass some species previously included in Microsorum and Colysis (Polypodiaceae) Blumea 66: 242–248.
List the numbers of any related issues
#25, #27, #28
Code of Conduct

[X] I agree to follow the PPG Code of Conduct

This proposal was voted on during PPG Ballot 3 (voting period September 2023). A total of 67 votes were cast. There were 43 'Yes' votes (64.2%) and 24 'No' votes (35.8%). The proposal does not pass.


Revert "Added basic steps for shrinking MPR"

Reverts mendix/docs#8488

This PR did not revert the right changes, closing it.


Clean up text-classification-training.py script

While the script works as intended, I think that it should be cleaned up.
As such, functions should split up in different util files depending on domain: this will make  the code easily readable.
E.g. functions gcs_data_to_docker  and training_output_to_gcs should be stored in a file_utils.py

I realised how poorly structured my repo was, in part due to this video.
As such, some major are made in b87c632f017ec839f85949f4a115f68d54d9aa22!

Dependencies are now pinned to a version; which makes the code reproducible.
The Docker image for inference installs a CPU-only version of PyTorch, which makes it much slimmer.
There is now only one trainer example; so users can make use of a GPU & Automatic Mixed Precision or not when training.
Makes proper use of modules, rather than one big script, this makes the code better legible and easier to add-on.


'/login' returns 500 error

I thought I had everything in and configured correctly but:
While Apache shows that the base url redirects correctly '/login' returns a 500 error.
My vhost for apache looks like this:
<VirtualHost *:80>         ServerAdmin webmaster@localhost         ServerName ubu1         DocumentRoot /var/www/BookStack/public         <Directory /var/www/BookStack/public>                 AllowOverride All                 Order allow,deny                 allow from all         </Directory>         ErrorLog ${APACHE_LOG_DIR}/error.log         CustomLog ${APACHE_LOG_DIR}/access.log combined </VirtualHost>
Any ideas?

Nothing immediately obvious to me, Is the 500 error from the application (in a white box) or does it look like a standard apache error message?
If it's from the application you could try changing APP_DEBUG to true in the .env file to get more information. If it's from apache try looking at the apache logs in /var/log/apache2 to get more information.

Oi, I missed the APP_DEBUG in the .env.
My apologies.
Got the info there that there was a rogue permission that the Apache Debug logs wasn't apparently catching.
Thanks.


feat: Disable 'Create Invoice' button until an amount is entered

Describe the changes you have made in this PR
The button is now disabled until the user enters a value in the 'Amount' field. This ensures that the user cannot create an invoice without specifying the amount first.
This change helps to prevent invalid or incomplete invoice creation.
Link this PR to an issue [optional]
Fixes #2802
Type of change

feat: New feature (non-breaking change which adds functionality)

Screenshots of the changes [optional]
Current  - ( Button is active by default )
https://github.com/getAlby/lightning-browser-extension/assets/100958893/68f2fb21-60bb-445d-b43e-9a55d30e1873
After PR -( Button is disabled by default and enabled when user enters a value )
https://github.com/getAlby/lightning-browser-extension/assets/100958893/45cf92a0-1a5a-47e7-803d-c68c724a5bef
Checklist

[x] Self-review of changed code
[x] Manual testing
[ ] Added automated tests where applicable
[ ] Update Docs & Guides
For UI-related changes
[x] Darkmode
[x] Responsive layout

So we make 0-amount invoices unusable for all connectors that support that currently for the sake of disabling a button?
I like the approach suggested here with the 0-amount invoice being shown by default on the receive screen (if there is no lightning address and the connector supports it) but until we have that I would prefer to keep this feature in there.


No "mobile app" when doing sspr in Azure China portal

Hi writer team,
We are using Azure China Environment.  Our customer configured mobile app for SSPR authentication method.

But there is no "mobile app" option for verification during SSPR. They are confused on it.

Can you help to let us know the reason and confirm whether "mobile app" authentication is available in Azur China?
Thanks.
Regards
Alex

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: cacc043b-aeec-5deb-e60f-b11e6381b0ba
Version Independent ID: aa2be87c-296f-b3ff-9549-4407c46746d4
Content: Enable Azure Active Directory self-service password reset - Microsoft Entra
Content Source: articles/active-directory/authentication/tutorial-enable-sspr.md
Service: active-directory
Sub-service: authentication
GitHub Login: @Justinha
Microsoft Alias: justinha

Hi writer team,
Any update?

Thanks for your feedback. We need to close all open issues at this time so we can migrate to a new Entra repository. I created a work item to have an expert review this feedback and update docs accordingly. #please-close


chore: migrate entrypoint of CLI to esm modules

🎉 Thanks for submitting a pull request! 🎉
Summary
As the esm module migration is such a big thing I try to split it up into small chunks and try to use both cjs and esm in a grace period to do a incremental migration.

For us to review and ship your PR efficiently, please perform the following steps:

[x] Open a bug/issue before writing your code 🧑💻. This ensures we can discuss the changes and get feedback from everyone that should be involved.
[x] Read the contribution guidelines 📖. This ensures your code follows our style guide and
passes our tests.
[x] Update or add tests (if any source code was changed or added) 🧪
[x] Update or add documentation (if features were changed or added) 📝
[x] Make sure the status checks below are successful ✅

A picture of a cute animal (not mandatory, but encouraged)

CI tests are currently failing. I am wondering if maybe the new tests setup will help with this?

Hi @lukasholzer, I left a small comment.
Also in order to trigger a release we'll need to merge this as fix or feat.

Always great to see more things migrating to ESM! Has this PR fallen under the radar?

@janosh sadly I had to put it on the side due to a shift of resources and priorities but If you want to continue feel free to!
I would love to see this effort going further!

I'm not familiar with the code base so I could only take over if it's minor stuff. What's left to do beside merge conflict resolution and addressing the comment by @erezrokah?

@janosh that should be all - having passing checks (e2e tests and unit tests) should provide the confidence

Basically we try to split it from top to bottom to migrate small parts for small parts as you can import commonjs in esm.
But vice verca is way more complicate through dynamic imports we can go from top to bottom and move the files to .mjs that import some commonjs.
The downside is that we have to move named imports then to default imports and destruct it later but that is all.

Closing as a lot of this is now already in the main branch


[HOLD for payment 2023-05-03] [$1000] Leading whitespaces aren’t truncated while saving email/phone number in  new contact method page

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Go to Settings -> Profile -> contact methods
Click on new contact methods
Add email with spaces at front e.g “<EMAIL_ADDRESS>Type in password and save
Now, it should redirect to contact methods page
Notice that the leading spaces aren’t removed while displaying email/phone number

Expected Result:
Leading or trailing spaces should be removed and email.phone number should be displayed accordingly for better UI
Actual Result :
Leading spaces makes improper indentation while display.
Workaround:
Can the user still use Expensify without this being fixed? Have you informed them of the workaround?
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: 1.3.3
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/43996225/233529701-d5974fa3-e27f-4ce8-b726-f489357e43a3.mov

Expensify/Expensify Issue URL:
Issue reported by: @ashimsharma10
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1681998814607449
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01b41c774d839dccd0
Upwork Job ID:<PHONE_NUMBER>356744704
Last Price Increase: 2023-04-22

Triggered auto assignment to @strepanier03 (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

Bug0 Triage Checklist (Main S/O)

[ ] This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
[ ] This bug is not a duplicate report (check E/App issues and #expensify-bugs)

If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead

[ ] This bug is reproducible using the reproduction steps in the OP. S/O

If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
If the reproduction steps aren't clear and you determine the correct steps, please update the OP.

[ ] This issue is filled out as thoroughly and clearly as possible

Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.

[ ] I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

Proposal
Posting proposal early as per new guidelines
Please re-state the problem that we are trying to solve in this issue.
Leading whitespaces aren not truncated while saving email/phone number in new contact method page.
What is the root cause of that problem?
We can see within onLoginChange() we do not trim login before save to state, so login value will be with leading and trailing space if entered.
https://github.com/Expensify/App/blob/4f7cfdb53216185ac7e825127b3be2f8238340fd/src/pages/settings/Profile/Contacts/NewContactMethodPage.js#L68-L70
Also during submitForm() we do not trim this.state.login as shown below,
https://github.com/Expensify/App/blob/4f7cfdb53216185ac7e825127b3be2f8238340fd/src/pages/settings/Profile/Contacts/NewContactMethodPage.js#L85-L92
So User.addNewContactMethodAndNavigate() method will save login value with leading and trailing space. This is the root cause of the problem.
What changes do you think we should make in order to solve the problem?
To solve this we can think to trim login while save to state at line 69 within onLoginChange().
But it is not advisable because user may enter Email Or Phone number within login field. So if user typing email then trim() during save to state will be ok, because generally blank space not comes within email so user will not try to enter blank space within email.
But if user enter Phone number then user have habit to enter phone number like "123 4567 8901" i.e. with space. So if we trim value during onChange, then as soon as user type space within phone number onLoginChange() will trim space during save to state. So it will make bad user experience and user will not able to enter space within phone number.
So better approach is to trim login within submitForm() function and use trimmed login value for lodashGet() and addNewContactMethodAndNavigate() api. So modified submitForm() function code looks as shown below.
submitForm() {
    // Trim leading and trailing space from login
    const login = this.state.login.trim();

    // If this login already exists, just go back.
    if (lodashGet(this.props.loginList, login)) {
        Navigation.navigate(ROUTES.SETTINGS_CONTACT_METHODS);
        return;
    }
    User.addNewContactMethodAndNavigate(login, this.state.password);
}

So this will solve the issue i.e. truncate leading/trailing white space from email or phone number, and allows to enter space within phone number. Result video attached below.
What alternative solutions did you explore? (Optional)
None.

Results
Truncate leading/trailing space from email.
https://user-images.githubusercontent.com/7823358/233578309-69a620dd-a2c2-4ce9-9abf-2213d26af09d.mov
Truncate leading/trailing space from Phone and allow space in between phone number.
https://user-images.githubusercontent.com/7823358/233578399-ebfc37b9-db7c-47b7-ad1c-073f595b47a1.mov
https://user-images.githubusercontent.com/7823358/233585221-7127fade-4899-4976-86a8-5a1c5dd86f2a.mov

Looks like something related to react-navigation may have been mentioned in this issue discussion.
As a reminder, please make sure that all proposals are not workarounds and that any and all attempt to fix the issue holistically have been made before proceeding with a solution. Proposals to change our DeprecatedCustomActions.js files should not be accepted.
Feel free to drop a note in #expensify-open-source with any questions.

Able to recreate following instructions but oddly I have the email listed twice, once with the extra spaced and once without.

Job added to Upwork: https://www.upwork.com/jobs/~01b41c774d839dccd0

Current assignee @strepanier03 is eligible for the External assigner, not assigning anyone new.

Triggered auto assignment to Contributor-plus team member for initial proposal review - @Santhosh-Sellavel (External)

Triggered auto assignment to @luacmartins (External), see https://stackoverflow.com/c/expensify/questions/7972 for more details.

@PrashantMangukiya's Proposal LGTM, straight forward change!

📣 @PrashantMangukiya You have been assigned to this job by @luacmartins!
Please apply to this job in Upwork and leave a comment on the Github issue letting us know when we can expect a PR to be ready for review 🧑💻
Keep in mind: Code of Conduct | Contributing 📖

@Santhosh-Sellavel @luacmartins If we allow space on the phone number then case I think the issue will remain the same for the phone right?

It won't create the current issue, we are just trying to trim out leading and trailing space. So we are good here.
If you are talking about space between the phone numbers, that's an invalid phone number case we got already covered with showing an error. So we are good to go here.
But restricting user input is bad UX.

@Santhosh-Sellavel  @luacmartins  Thank you. Applied to job on Upwork. I will prepare and submit PR within 3 hours asap.

Reviewing label has been removed, please complete the "BugZero Checklist".

The solution for this issue has been :rocket: deployed to production :rocket: in version 1.3.5-6 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

https://github.com/Expensify/App/pull/17817

If no regressions arise, payment will be issued on 2023-05-03. :confetti_ball:
After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

[ ] External issue reporter
[ ] Contributor that fixed the issue
[ ] Contributor+ that helped on the issue and/or PR

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

Merged PR within 3 business days of assignment - 50% bonus
Merged PR more than 9 business days after assignment - 50% penalty

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

[ ] [@Santhosh-Sellavel] The PR that introduced the bug has been identified. Link to the PR:
[ ] [@Santhosh-Sellavel] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
[ ] [@Santhosh-Sellavel] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
[ ] [@Santhosh-Sellavel] Determine if we should create a regression test for this bug.
[ ] [@Santhosh-Sellavel] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
[ ] [@strepanier03] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

Just getting back from out of office and working to catch up on this now. I'll have an update shortly.

@ashimsharma10 - I have hired you to a bug report job in Upwork.
@Santhosh-Sellavel and @PrashantMangukiya - I have hired you to the Upwork job as well.

@Santhosh-Sellavel - If you can finish the BZ checklist I'll update the comment and make any necessary reg test GHs.

I'll check in later today to pay what I can.

@strepanier03 Offer accepted on Upwork.

@PrashantMangukiya and @ashimsharma10, I have paid you both via Upwork and closed contracts, thank you both!

@strepanier03  Received, Thank you.

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

[x]  [@Santhosh-Sellavel] The PR that introduced the bug has been identified. Link to the PR: https://github.com/Expensify/App/pull/16919
[ ]  [@Santhosh-Sellavel] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
[ ]  [@Santhosh-Sellavel] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
[ ]  [@Santhosh-Sellavel] Determine if we should create a regression test for this bug.
[ ]  [@Santhosh-Sellavel] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
[ ]  [@strepanier03] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

Regression steps
From the PR itself looks good to me

Go to Settings -> Profile -> Contact methods
Click on "New contact method" button
Add email/phone with leading and trailing spaces e.g “<EMAIL_ADDRESS>”
Tap Add button
Now, it will go back to "Contact methods" page
Verify that leading/trailing spaces does not show while display email/phone number.

👍  or  👎
cc: @luacmartins or @strepanier03

@strepanier03 Accepted offer thanks!


Completed WriteLib typings, added _replicas typings for ReplicaService, updated README

Closed by https://github.com/Mixu78/replicaservice-typings/commit/05cb26ef2ff76c903ab3064cdab8541793416921


chore: prepare release 2023-03-14

Summary
This PR has been created using the yarn release script. Once merged, the clients will try to release their new version if their version has changed.
Version Changes

javascript: 5.0.0-alpha.52 -> prerelease (e.g. 5.0.0-alpha.53)
java: 4.0.0-SNAPSHOT -> minor (e.g. 4.0.0-SNAPSHOT)
php: 4.0.0-alpha.50 -> prerelease (e.g. 4.0.0-alpha.51)

Skipped Commits
(None)

will re-trigger the release with https://github.com/algolia/api-clients-automation/pull/1387 in it


weights = ef.max_sharpe() is not working some times

/usr/local/lib/python3.6/dist-packages/pypfopt/efficient_frontier.py in max_sharpe(self, risk_free_rate)
239         # max_sharpe requires us to make a variable transformation.
240         # Here we treat w as the transformed variable.
--> 241         self._objective = cp.quad_form(self._w, self.cov_matrix)
242         k = cp.Variable()
243
/usr/local/lib/python3.6/dist-packages/cvxpy/atoms/quad_form.py in quad_form(x, P)
225         return x.H @ P @ x
226     elif P.is_constant():
--> 227         return QuadForm(x, P)
228     else:
229         raise Exception(
/usr/local/lib/python3.6/dist-packages/cvxpy/atoms/quad_form.py in init(self, x, P)
35
36     def init(self, x, P):
---> 37         super(QuadForm, self).init(x, P)
38
39     def numeric(self, values):
/usr/local/lib/python3.6/dist-packages/cvxpy/atoms/atom.py in init(self, *args)
43         # Convert raw values to Constants.
44         self.args = [Atom.cast_to_const(arg) for arg in args]
---> 45         self.validate_arguments()
46         self._shape = self.shape_from_args()
47         if len(self._shape) > 2:
/usr/local/lib/python3.6/dist-packages/cvxpy/atoms/quad_form.py in validate_arguments(self)
51             raise ValueError("Invalid dimensions for arguments.")
52         if not self.args[1].is_hermitian():
---> 53             raise ValueError("P must be symmetric/Hermitian.")
54
55     def sign_from_args(self):
ValueError: P must be symmetric/Hermitian.

Hi @kman624,
from the trace, it looks like the covariance matrix is not symmetric. This can occur for a number of reasons, depending on how it was derived. You can try to fix it like this:
cov_matrix = np.maximum( cov_matrix, cov_matrix.transpose() )

If that does not work, or I did not interpret the trace correctly, please post a reproducible example, if possible.

Thank you for that, unfortunately it did not help. I will link my dataset and I am using the Efficient Frontier Optimization. My code works for other datasets, for some reason this particular one is causing problems. I am also using Google Colab, if that makes a difference.
Also, side question. For my other datasets that work and I get the optimal portfolio out, why do only 5-10 stocks out of the 50-100 I input make it through? Especially some I know from fundamental analysis are quite good?
https://docs.google.com/spreadsheets/d/13Qvrjrq_X11tkPYsIOT7hywNZ7PaZd9UkH-3xJEDEu0/edit?usp=sharing

Great if you are using Colab. That should not cause any issues by itself. Can you perhaps share the notebook or the code you use exactly? Otherwise, it is quite hard to reproduce the issue. One thing to mention is that max_sharpe() is generally a bit more complex compared to other optimizations, as the variables are transformed in the background. But that mostly only comes into effect when adding constraints (which you might or might not be doing).
Regarding your observation about concentration, indeed, this is a common behavior of classical mean-variance optimization.
There are different ways to deal with it, a common one is regularization, as described in the Docs.

I see, here is a link to the Colab. I changed the code to weights = np.array([ef.max_sharpe()]   and that sorta fixed things. I just have a new error now on the next line with one of my variables. You should be able to see it. Thank you so much for helping me with this
https://colab.research.google.com/drive/1yzEGEeJY7AJb2vU5xLJqbVxnmhZT5zCk?usp=sharing

Also, when I use the L2 Regularization, I get this error:
NameError                                 Traceback (most recent call last)
 in ()
1 ef = EfficientFrontier(mu, S)
----> 2 ef.add_objective(objective_functions.L2_reg, gamma=0.01)
3 weights = ef.max_sharpe()
4
5 #ef = EfficientFrontier(mu, S)
NameError: name 'objective_functions' is not defined
I do not know why, I thought it was in the package

Re the first error:
Some assets in df only have NaN observations.
Try:
df.dropna(how='any',inplace=True,axis=1)

Re your second error:
You need to import it first:
from pypfopt import objective_functions

the df.drop line worked, thanks! I tried something like that earlier but I do not think I wrote it like you did
I am currently trying the objective functions solution

If you were wondering, out of the Nasdaq 100, the program says to spend all your money on Tesla. How funny. I suppose it makes sense based on it's price history


⚠ GitLab Server has degraded performance

In e184efc, GitLab Server (https://gitlab01.its-telekom.eu) experienced degraded performance:

HTTP code: 200
Response time: 6394 ms

Resolved: GitLab Server performance has improved in 9104422.

Resolved: GitLab Server performance has improved in 9104422.


Downloading files during sync

Hello,
this might be a very stupid question, but should it be possible to download files from a private s3 bucket?
I managed to upload files to the desired s3 bucket, but none of the files present in the bucket seems to be downloaded when syncing (which I would expect to happen as part of a sync).
In addition I am very confused why in the example configuration the setting --acl public-read is used if one already specifies AWS credentials. The bucket I care about is not set to public.

Just a quick me too - am looking for a way to sync down (from AWS into the GH Action)


Command line option to overwrite dirs

I request some command line option that can overwrite existing directories.
For example,
rubex cat.rubex
vi cat.rubex # make some changes..
rubex -f cat.rubex # overwrite cat directory

Good thought.


Warning: React has detected a change in the order of Hooks called by WrapRoot.

Hello,
First of all, thank you for the plugin - it's exactly what I was looking for. However it seems to have a problem with rendering. Console displays the following error:
Warning: React has detected a change in the order of Hooks called by WrapRoot. This will lead to bugs and errors if not fixed. For more information, read the Rules of Hooks: https://fb.me/rules-of-hooks

   Previous render            Next render
   ------------------------------------------------------
1. useState                   useContext
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

    in WrapRoot (created by LocationProvider)
    in LocationProvider (created by Context.Consumer)
    in Location (created by WithLocationComponent)
    in WithLocationComponent
    in Suspense
    in _default (at app.js:67)

Which affects the build, I beleive, as after rendered page is loaded, it flickers to translation variables, which are replaced by translation in a fraction of a second.
Do you have any idea of what it can be?

Hi!
First of all, thank you for using my plugin and leaving feedback!
About the order of hooks issue, I get the error too. However, I am not sure why that happens as all the hooks are rendered the same way each time, and I haven't used neither useState nor useContext in WrapRoot. This leads me to think that something is internally using those to cause the problem. I will look into it immediately! Thank you for reporting!
About the flicker problem, did you set useSuspense and wait options in i18next to true? Refer to the example config here. Does this solve at least the flicker issue for you?

I think I managed to pinpoint the problem - useTranslation from react-i18next calls useContext conditionally. I opened an issue and made a PR. As soon as it is fixed, I will update react-i18next in this plugin and the issue should be resolved! :)
You can track the issue here  if you are interested.

Turns out there was also a goof up on my part. I fixed it and the issue should be gone now, but I will still update the package when/if my PR is merged.
Update to the latest version and tell me if the issue persists :)

Thank you for fast response! Yes, the issue was fixed for me, thank you! Now it's officially the best and easiest localisation plugin for gatsby! :)

Thank you, I am glad you find it useful :)
I will be closing the issue now, if you have any other problems or feature suggestions, don't hesitate to open another issue!


[Bug]: global-setup and esm do not work when importing relatively

Version
29.1.0
Steps to reproduce

clone https://github.com/hasezoey/ts-jest-esm-globalsetup-issue
install all packages (yarn install)
run yarn jest
observe error

Expected behavior
i expect the import to work like in commonjs
Actual behavior
error about not being able to import the given file
Debug log
TS_JEST_LOG=ts-jest.log yarn jest
yarn run v1.22.19
$ /mnt/projects/nodejs/ts-jest_test/node_modules/.bin/jest
Error: Jest: Got error running globalSetup - /mnt/projects/nodejs/ts-jest_test/test/globalSetup.ts, reason: Cannot find module '../src/somefile.js'
Require stack:
- /mnt/projects/nodejs/ts-jest_test/test/globalSetup.ts
- /mnt/projects/nodejs/ts-jest_test/node_modules/jest-util/build/requireOrImportModule.js
- /mnt/projects/nodejs/ts-jest_test/node_modules/jest-util/build/index.js
- /mnt/projects/nodejs/ts-jest_test/node_modules/jest-config/build/getCacheDirectory.js
- /mnt/projects/nodejs/ts-jest_test/node_modules/jest-config/build/Defaults.js
- /mnt/projects/nodejs/ts-jest_test/node_modules/jest-config/build/normalize.js
- /mnt/projects/nodejs/ts-jest_test/node_modules/jest-config/build/index.js
- /mnt/projects/nodejs/ts-jest_test/node_modules/jest-cli/build/init/index.js
- /mnt/projects/nodejs/ts-jest_test/node_modules/jest-cli/build/run.js
- /mnt/projects/nodejs/ts-jest_test/node_modules/jest-cli/build/index.js
- /mnt/projects/nodejs/ts-jest_test/node_modules/jest-cli/bin/jest.js
- /mnt/projects/nodejs/ts-jest_test/node_modules/jest/bin/jest.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1090:15)
    at Module._load (node:internal/modules/cjs/loader:934:27)
    at Module.require (node:internal/modules/cjs/loader:1157:19)
    at require (node:internal/modules/helpers:119:18)
    at Object.<anonymous> (/mnt/projects/nodejs/ts-jest_test/test/globalSetup.ts:1:1)
    at Module._compile (node:internal/modules/cjs/loader:1275:14)
    at Module._compile (/mnt/projects/nodejs/ts-jest_test/node_modules/pirates/lib/index.js:136:24)
    at Module._extensions..js (node:internal/modules/cjs/loader:1329:10)
    at Object.newLoader (/mnt/projects/nodejs/ts-jest_test/node_modules/pirates/lib/index.js:141:7)
    at Module.load (node:internal/modules/cjs/loader:1133:32)
error Command failed with exit code 1.

Additional context
already tried to use ts-jest-resolver as a resolver and also tried using preset ts-jest/presets/default-esm and setting the options manually, also tried various options from ESM Support
Environment
System:
    OS: Linux 6.3 Manjaro Linux
    CPU: (16) x64 AMD Ryzen 7 5800X 8-Core Processor
  Binaries:
    Node: 19.9.0 - /tmp/yarn--1683548400265-0.7148494764630906/node
    Yarn: 1.22.19 - /tmp/yarn--1683548400265-0.7148494764630906/yarn
    npm: 5.1.0 - /mnt/projects/nodejs/ts-jest_test/node_modules/.bin/npm
  npmPackages:
    jest: ^29.5.0 => 29.5.0

Worked for me adding compilerOptions: "allowImportingTsExtensions": true with "noEmit": true (that is required for the first one to work). Then importing from ../src/somefile.ts works.
In real world allowImportingTsExtensions should live in some tsconfig.test.json. Or so.

Or without any changes in tsconfig.json, but import from ../src/somefile.
(Indeed buggy. That should throw an error.)

but import from ../src/somefile.

that is not possible, then typescript complains that the extension is missing in esm

This is not something ts-jest can solve. The error was thrown out before reaching transforming step. The globalSetup file never reaches process method of the transformer.

Worked for me adding compilerOptions: "allowImportingTsExtensions": true with "noEmit": true (that is required for the first one to work). Then importing from ../src/somefile.ts works.
In real world allowImportingTsExtensions should live in some tsconfig.test.json. Or so.

@mrazauskas But it wont compile with noEmit (or the --noEmit flag) set to true. Furthermore, if i create one tsconig.json with  ... "allowImportingTsExtensions": true, "noEmit": true ..., and a tsconfig.build.json with
{
    "extends": "./tsconfig.json",
    "compilerOptions": {
        "allowImportingTsExtensions": false,
        "noEmit": false
    },
    "include": ["src/**/*"],
    "exclude": ["tests/**/*"]
}

It still doesnt compile with the tsconfig.build.json because i have imported everything like import { foo } from "bar.ts" (see my issue #4162) to satisfy jest/ts-jest. So practically i need two codebases, one with everything imported as import { foo } from "bar.ts" for testing and one with everything imported as import { foo } from "bar" to actually compile it. maybe Babel could help to add the extensions before running jest or ts-jest. I am not a babel wizard, maybe somebody could help me. This issue rendered my codebase useless for production and @kulshekhar isnt responding.
Any help is highly appreciated

I don't have references handy but for server-side TS code I think the consensus is now to add .js extensions to relative imports.  It's a hassle and I'm not sure the tooling is totally there yet.  I filed https://github.com/kulshekhar/ts-jest/issues/4013 to document this approach but I haven't had time to put together a PR -- I just got back to working on related projects this week.


Universal example small piece of redundat code

Just new to ES6.
Universal example, file common/api/counter
export function fetchCounter(callback) {
  // Rather than immediately returning, we delay our code with a timeout to simulate asynchronous behavior
  setTimeout(() => {
    callback(getRandomInt(1, 100))
  }, 500)

  // In the case of a real world API call, you'll normally run into a Promise like this:
  // API.getUser().then(user => callback(user))
}

Isn't the arrow function redundant? Do I miss something?
export function fetchCounter(callback) {
  // Rather than immediately returning, we delay our code with a timeout to simulate asynchronous behavior
  setTimeout(
      callback(getRandomInt(1, 100))
, 500)

  // In the case of a real world API call, you'll normally run into a Promise like this:
  // API.getUser().then(user => callback(user))
}

No, you would invoke callback synchronously when this line is executed and pass its return value to setTimeout.


Add limited guest role.

This PR adds limited guest role and accessible_user_dicts function which will be used to get only accessible users for limited guest users.

First commit adds limited_guest role value of 800 to the UserProfile model.
Second commit adds is_limited_guest.
Third commit is for including limited guests in the count returned by realm_user_count_by_role.
Fourth commit adds a limited guest user in development and test database.
Fifth and sixth commit are for adding helper functions to get accessible user dicts.

Related issue - #10970.

@timabbott can you review this? I will mention few points -

We show the Shylock user in guest in the login page for now.
There are a couple of bugs in dev server UI for the added user. Like role colmn is empty in the admin users list. Have not added this role to frontend yet.

Closing this since we have decided to implement this based on the groups permissions model #19525.


⚠ Bofrost.be has degraded performance

In 83f7e9b, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1546 ms

Resolved: Bofrost.be performance has improved in ec1b6ea.


Vercel shows internal server error for next-drupal node module

14:00:51.677
Error: Internal Server Error

14:00:51.677
at /vercel/path0/node_modules/next-drupal/dist/index.js:86:19

14:00:51.677
at processTicksAndRejections (internal/process/task_queues.js:93:5)

14:00:51.677
at async getStaticPaths (/vercel/path0/.next/server/pages/blog/[...slug].js:3196:17)

14:00:51.678
at async buildStaticPaths (/vercel/path0/node_modules/next/dist/build/utils.js:16:80)

14:00:51.678
at async /vercel/path0/node_modules/next/dist/build/utils.js:26:622

14:00:51.678
at async Span.traceAsyncFn (/vercel/path0/node_modules/next/dist/telemetry/trace/trace.js:6:584)

Can you give me a bit more info? I'll take a look at next-drupal/dist/index.js:86:19 but more context would be helpful. Thanks.

@arshad The error is appearing in Vercel build log, as provided below. Please let me know if you anything here indicative of something I did wrong on my side.
13:59:34.834  	Cloning gitlab.com/quest-king/test-site (Branch: master, Commit: 2964140)
13:59:44.293  	Cloning completed: 9.460s
13:59:45.220  	Analyzing source code...
13:59:46.704  	Installing build runtime...
13:59:48.744  	Build runtime installed: 2.040s
13:59:51.548  	Looking up build cache...
13:59:51.670  	Build cache found. Downloading...
13:59:53.092  	Build cache downloaded [25.36 MB]: 1421.673ms
13:59:54.418  	WARNING: You should not upload the .next directory.
13:59:54.420  	Installing dependencies...
13:59:55.529  	npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but package-lock.json was generated for lockfileVersion@2. I'll try to do my best with it!
13:59:56.553  	npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none is installed. You must install peer dependencies yourself.
13:59:56.561  	npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/fsevents):
13:59:56.562  	npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
13:59:56.562  	up to date in 1.693s
13:59:56.679  	51 packages are looking for funding
13:59:56.679  	  run npm fund for details
13:59:56.691  	Detected Next.js version: 10.2.0
13:59:56.693  	Running "npm run build"
13:59:56.903  	><EMAIL_ADDRESS>build /vercel/path0
13:59:56.903  	> next build
13:59:57.246  	info  - Loaded env from /vercel/path0/.env.production
13:59:57.525  	info  - Using webpack 5. Reason: future.webpack5 option enabled https://nextjs.org/docs/messages/webpack5
13:59:57.639  	info  - Checking validity of types...
14:00:00.969  	info  - Creating an optimized production build...
14:00:04.257  	info  - Compiled successfully
14:00:04.257  	info  - Collecting page data...
14:00:05.182  	Error: Internal Server Error
14:00:05.182  	    at /vercel/path0/node_modules/next-drupal/dist/index.js:86:19
14:00:05.182  	    at processTicksAndRejections (internal/process/task_queues.js:93:5)
14:00:05.182  	    at async getStaticPaths (/vercel/path0/.next/server/pages/[...slug].js:2669:17)
14:00:05.182  	    at async buildStaticPaths (/vercel/path0/node_modules/next/dist/build/utils.js:16:80)
14:00:05.182  	    at async /vercel/path0/node_modules/next/dist/build/utils.js:26:622
14:00:05.182  	    at async Span.traceAsyncFn (/vercel/path0/node_modules/next/dist/telemetry/trace/trace.js:6:584)
14:00:05.184  	Error: Internal Server Error
14:00:05.184  	    at /vercel/path0/node_modules/next-drupal/dist/index.js:86:19
14:00:05.184  	    at processTicksAndRejections (internal/process/task_queues.js:93:5)
14:00:05.184  	    at async getStaticPaths (/vercel/path0/.next/server/pages/blog/[...slug].js:3196:17)
14:00:05.184  	    at async buildStaticPaths (/vercel/path0/node_modules/next/dist/build/utils.js:16:80)
14:00:05.184  	    at async /vercel/path0/node_modules/next/dist/build/utils.js:26:622
14:00:05.184  	    at async Span.traceAsyncFn (/vercel/path0/node_modules/next/dist/telemetry/trace/trace.js:6:584)
14:00:05.197  	info  - Generating static pages (0/5)
14:00:05.499  	info  - Generating static pages (1/5)
14:00:05.505  	info  - Generating static pages (2/5)
14:00:05.506  	info  - Generating static pages (3/5)
14:00:05.928  	Error: Internal Server Error
14:00:05.928  	    at /vercel/path0/node_modules/next-drupal/dist/index.js:86:19
14:00:05.928  	    at processTicksAndRejections (internal/process/task_queues.js:93:5)
14:00:05.928  	    at async getStaticProps (/vercel/path0/.next/server/pages/index.js:3236:20)
14:00:05.928  	    at async renderToHTML (/vercel/path0/node_modules/next/dist/next-server/server/render.js:28:1689)
14:00:05.928  	    at async /vercel/path0/node_modules/next/dist/export/worker.js:26:6
14:00:05.928  	    at async Span.traceAsyncFn (/vercel/path0/node_modules/next/dist/telemetry/trace/trace.js:6:584)
14:00:05.964  	info  - Generating static pages (5/5)
14:00:05.965  	info  - Finalizing page optimization...
14:00:05.989  	Page                             Size     First Load JS
14:00:05.989  	┌ ● / (ISR: 1 Seconds)           1.15 kB        95.7 kB
14:00:05.989  	├   /_app                        0 B            84.5 kB
14:00:05.989  	├ ● /[...slug]                   1.67 kB        92.2 kB
14:00:05.989  	├ ○ /404                         3.03 kB        87.5 kB
14:00:05.989  	├ λ /api/exit-preview            0 B            84.5 kB
14:00:05.989  	├ λ /api/preview                 0 B            84.5 kB
14:00:05.990  	└ ● /blog/[...slug]              905 B          95.4 kB
14:00:05.990  	+ First Load JS shared by all    84.5 kB
14:00:05.990  	  ├ chunks/2.47f79c.js           17.7 kB
14:00:05.990  	  ├ chunks/commons.cd9bd9.js     13.4 kB
14:00:05.990  	  ├ chunks/framework.0529c1.js   42 kB
14:00:05.990  	  ├ chunks/main.e3016c.js        6.87 kB
14:00:05.990  	  ├ chunks/pages/_app.047fc0.js  3.47 kB
14:00:05.990  	  └ chunks/webpack.ce1b5f.js     993 B
14:00:05.990  	λ  (Server)  server-side renders at runtime (uses getInitialProps or getServerSideProps)
14:00:05.990  	○  (Static)  automatically rendered as static HTML (uses no initial props)
14:00:05.990  	●  (SSG)     automatically generated as static HTML + JSON (uses getStaticProps)
14:00:05.990  	   (ISR)     incremental static regeneration (uses revalidate in getStaticProps)
14:00:08.547  	Traced Next.js server files in: 2.459s
14:00:09.319  	WARNING: Unable to find source file for page next/dist/pages/_error.js
14:00:09.425  	Created all serverless functions in: 877.983ms
14:00:09.834  	Uploading build outputs...
14:00:10.625  	Deploying build outputs...
14:00:15.910  	Build completed. Populating build cache...
14:00:21.095  	Uploading build cache [25.36 MB]...
14:00:21.585  	Build cache uploaded: 490.13ms
14:00:22.101  	Done with "package.json"

Closing this issue out as I believe issue was due to pushing .next directory to vercel


19041.789 rdp check shows not correctly

The RDP check shows that it successfully connects to the host, but i can do nothing in it because it shows not correctly and the only thing i can do is controlling the host to shut down, sleep or restart.

can you provide a config file for 19041.789 ? there is listen but not support in my PC

Duplicate of https://github.com/stascorp/rdpwrap/issues/1317


Breaking Changes due to MetaTensor

Describe the bug
Due to the the incorporation of MetaTensor there are breaking changes in the model zoo format on certain transforms AddChannel. There  is an appropriate documentation for the reason of the error
To Reproduce
Steps to reproduce the behavior:

git clone the monai repo
Install using pip install -e '.[all]'
Run using the following command
python -m monai.bundle run training --meta_file ../model-zoo/models/spleen_ct_segmentation/configs/metadata.json --config_file ../model-zoo/models/spleen_ct_segmentation/configs/train.json  --logging_file ../model-zoo/models/spleen_ct_segmentation/configs/logging.conf

Additional context
The metadata.json file includes the monai version compatibility but I think that is wrong. Maybe it should be rolled back to 0.8 for now and once the configs are updated with the new transforms we can switch it back to 0.9.0

I just changed the monai version to 0.8.0 in the metadata.json file for the spleen_ct_segmentation, but I get the same errors. That tells me that we do not have a version compatibility check in the monai.bundle run function.

Hi @vikashg ,
Thanks for your feedback, we don't have check in the bundle run function so far.
I didn't see the AddChannel transform in the latest spleen bundle now:
https://github.com/Project-MONAI/model-zoo/blob/dev/models/spleen_ct_segmentation/configs/inference.json#L41
Could you please help take a look and try again?
Thanks in advance.

have you tried running both pip uninstall monai and pip uninstall monai-weekly a few times before pip install, sometimes the previous versions of the packages are not uninstalled properly

Hi @wyli I did uninstall monai but I didnt do pip uninstall monai-weekly I will do that.

Hi @wyli I did uninstall monai but I didnt do pip uninstall monai-weekly I will do that.

There is no monai.bundle before monai v0.9.0, and monai-weekly gets the latest dev version newer than 0.9.1 release.

Hi @vikashg , so far is the issue solved? It seems this ticket can be closed.

Yes I agree.


Working with custom datasets and SG detection formats

Hi,
I'm trying to train a yolox model using my own custom dataset.
The reason im using a custom dataset is becuase of the caching machenizem used in SG dataset objects which results in super long dataset loading time and Memory Error for (very) large datasets.
So far i've been successfull in implementing a somewhat uncached version ofo SG's DetectionDataset.
I'm also working with clearml datasets so i added the option of fetching a dataset from clearml server and loading the dataset metadata.
I've also used the plot function to plot my dataset samples which works well when not using any transforms.
But my issue is with the transformations (copied the transform pipeline from the yolox yaml config) and specifically with Mosaic (since it is also the first one and probably screws up my labels)
The transformations do work on the image part, i'm getting nice augmented images, but the labels don't make sense.
i tried working with different detection formats (LABEL_CXCYWH, LABEL_XYWH, LABEL_XYXY) but no luck.
when investigating the mosaic code it is vaguely documented but just above the operation that is done on the labels there's a comment
# Normalized xywh to pixel xyxy format - line 506.
which is uclear to me since in your coco detection dataset the XYXY_LABEL format is used, but it succefully transforms the label as well when using that dataset.
my question is what is the expected input format to the transforms?

Hi @wematan ,
Caching the annotations led to a Memory Error ?  We have a ticket to make the cache optional, but it was just never prioritized enough because, until now, it never led to this situation of crashing the code.
The transforms expect a sample {"image": <np.ndarray>, "target": <np.ndarray>}, except for mosaic which expects another field {"additional_samples": List[{"image": <np.ndarray>, "target": <np.ndarray>}]}. made of 3 samples.
Mosaic will combine the main sample with the additional 3 together into a single image/label.
FYI, we add the samples in our dataset here: https://github.com/Deci-AI/super-gradients/blob/b4608f618a60de1504211f75e14f9fc7ade2874a/src/super_gradients/training/datasets/detection_datasets/detection_dataset.py#L378-L393
Other information about our transforms

All of our transforms expect the target to be (x, y, x, y, label) - including Mosaic
The transforms are applied on a single sample, not batch

About YoloX
It expects

image to be of shape [C, H, W]
target to be padded using DetectionTargetsFormatTransform (required), which is [max_targets, 5] with max_targets a parameter to the Transorm and 5 for (label, cx, cy, w, h).

If this doesn't answer your question or if you have another problem, you can share the code you implemented, it could help me be more precise in my answer :)

@Louis-Dupont Thanks a lot for your quick and elaborate response.
I've finally got it to work now.
Regardig caching, yes it is an issue since most end2end training frameworks focus on implementing as many models as they can but only benchmark those on a specific type of datasets (which unfortunately are of small capacity), it became harder to train (very) large datasets (10M> objects).
BR,
Matan.


Bluetooth: controller: Fix bug in Encryption feature cond. compile

Fix a bug in Encryption Procedure feature conditional
compile which prevent any data transfer when the feature was
disabled.
Signed-off-by: Vinayak Kariappa Chettimada<EMAIL_ADDRESS>

Codecov Report

Merging #13488 into master will not change coverage.
The diff coverage is 0%.

@@           Coverage Diff           @@
##           master   #13488   +/-   ##
=======================================
  Coverage    52.5%    52.5%
=======================================
  Files         318      318
  Lines       46526    46526
  Branches    10754    10754
=======================================
  Hits        24429    24429
  Misses      17198    17198
  Partials     4899     4899

Impacted Files
Coverage Δ

subsys/bluetooth/controller/ll_sw/ull_conn.c
49.01% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f9e73c9...e94364f. Read the comment docs.