Too many array elements cause slow editor

Godot version
4.0-beta7
System information
win10
Issue description
Unable to load it and run print it

Steps to reproduce

Minimal reproduction project

Printing will always slow the editor down, the more the bigger the array is (more lines/more formatting work)
Use the debugger/remote scene tree instead of printing

I don't want to print, but I want to use it. As a result, I can only save the coordinates in a file and then read them, otherwise the editor will get stuck

Even if I save the coordinates in another script, loading and calling it will flash back

You don't have to print it to editor to use it. I am loading a long list of stars from CSV w/o printing it.
Also make sure you save the list to some variable once you've loaded, this way you don't recreate the list every time...

您不必将其打印到编辑器即可使用它。我正在从CSV加载一长串星星，而无需打印它。
还要确保在加载后将列表保存到某个变量中，这样您就不会每次都重新创建列表......

According to my test, when I write an array variable on the editor, it will be stuck if there are more than 2000 elements, and it will flash back when I read it

For the last time, please upload a minimal reproduction project in your bug reports. Otherwise, we can't afford to spend time trying to recreate the potential issues that you claim exist without having any details on how to reproduce those issues.
If you say that the editor is slow when an array contains too many elements, please make a simple project which demonstrates this (single scene with a single script that has a big array which slows the editor) and attach it here as a zip.
Otherwise, we'll simply have to ignore your issues going further as they're all too low quality to work on.

I made a reproduction project for what it's worth..
slow_editor_recreation.zip
tried printing a file full of vectors with size of 140kb (just like OP). print just fine, can't reproduce

Thanks for testing! Closing then. We can reopen if a minimal reproduction project that actually triggers the issue is provided.

you are right because the RAM usually be 16 GB and this file probably is 10 MG so it not so much


Chapter9 File Download examples assume that the Project's directory is equal to System.getProperty("user.dir"), but sometimes this is not the case

In the kazurayam21 branch, the sample code of the Chapter9 "File Download" section explains several approaches where to locate the output files from the test cases.
based on the User's home directory
https://github.com/kazurayam/selenium-webdriver-java/blob/kazurayam21/selenium-webdriver-junit4/src/test/java/io/github/bonigarcia/webdriver/junit4/ch09/download/DownloadChromeJUnit4Test.java
        targetFolder = new File(System.getProperty("user.home"), "Downloads");

based on the Current Working Directory
https://github.com/kazurayam/selenium-webdriver-java/blob/kazurayam21/selenium-webdriver-junit4/src/test/java/io/github/bonigarcia/webdriver/junit4/ch09/download/DownloadFirefoxJUnit4Test.java
        targetFolder = new File(".");

and
https://github.com/kazurayam/selenium-webdriver-java/blob/kazurayam21/selenium-webdriver-junit4/src/test/java/io/github/bonigarcia/webdriver/junit4/ch09/download/DownloadHttpClientJUnit4Test.java
...
        File pngFile = new File(".", "webdrivermanager.png");
...
        File pdfFile = new File(".", "webdrivermanager.pdf");
...

The new File(".") means the Current Working Directory, which is equal to System.getProperty("user.dir"). The document wrotes:

We use the local project folder.

However, this is not necessarily right. As I explained in the #8, The Current Working Directory is NOT necessarily equal to the local project folder.

I will modify https://github.com/kazurayam/selenium-webdriver-java/blob/kazurayam21/selenium-webdriver-junit4/src/test/java/io/github/bonigarcia/webdriver/junit4/ch09/download/DownloadHttpClientJUnit4Test.java so that it uses com.kazurayam.unittest.TestHelper#getProjectDir().
I will do this change in the kazurayam8i branch.


App is closed when delete button is long pressed

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Open app
Insert emoji from google keyboard
Press back delete button

Expected Result:
Emoji should be removed, and the app should not exit
Actual Result:
App is exited
Workaround:
unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native - POCO X3

Version Number: 1.2.40-3
Reproducible in staging?: Need reproduction
Reproducible in production?: Needs reproduction
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos:
https://user-images.githubusercontent.com/43996225/209336256-ce4c1f92-0697-4816-9ad4-f6b756753a6d.mov
Expensify/Expensify Issue URL:
Issue reported by: @ekramkhan66
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1671535081000989
View all open jobs on GitHub

I can't seem to reproduce this one, I attempted on multiple android devices via Browser stack. Can you post in Slack to see if others might be able to reproduce this? Otherwise let's close this.


Some error outputs while using roiware_pool3d operations

Hey guys,
Thanks for your amazing work!
I am trying to use your roiaware_pool3d operations in my project. However I found that the output of points_in_boxes_gpu function looks incorrect.
I input 3D point clouds and bounding boxes([x, y, z, l, w, h, rz]) in lidar coordinates, but the outputs of point_in_boxes_gpu seem to be rotated in a wrong angle.
I will attach some output here, hoping it will help you to figure out my error.

@HilbertXu
Hi, thanks for your interest in mmdet3d. I checked the functions and found that according to the definition of the LIDAR coordinate system, point_in_boxes_gpu does rotate the bbox using the wrong angle.
Unfortunately, this is a legacy issue because earlier we followed the coordinate system convention of SECOND. Due to the large amount of code related with this issue, we cannot simply fix point_in_boxes_gpu. We are planning a refactor of the coordinate systems to resolve this issue. Sorry for the inconvenience.
As you mentioned, you were only using the CUDA ops, so a quick fix could be taking the opposite of the rotation angle in the CUDA ops in your code.
For example:
boxes = torch.tensor(
    [[[0.0, 0.0, 0.0, 1.0, 20.0, 1.0, 0.523598]]], dtype=torch.float32).cuda() # 30 degrees
pts = torch.tensor(
    [[[4, 6.928, 0], [6.928, 4, 0], [4, -6.928, 0], [6.928, -4, 0],
      [-4, 6.928, 0], [-6.928, 4, 0], [-4, -6.928, 0], [-6.928, -4, 0]]], dtype=torch.float32).cuda()
point_indices = points_in_boxes_gpu(points=pts, boxes=boxes)
print(point_indices)

The original output is
tensor([[ 0, -1, -1, -1, -1, -1,  0, -1]], device='cuda:0', dtype=torch.int32)

Take the opposite of rz here.
Then the output is
tensor([[-1, -1,  0, -1,  0, -1, -1, -1]], device='cuda:0', dtype=torch.int32)

which is the correct behaviour under the LIDAR coordinate system defined in the docstring.

Closed due to inactivity.
p.s. A refactor is ongoing now.

Just updated the docstrings to correctly describe the current coordinate systems.


UI Improvements

Changed default colors.
Centered the main logo.
Improved typography, font sizes, and line heights.
Styled language nav buttons.

Hello @nahuelcandia ! I've been looking around for people making changes to the default CSS files from this project and you seem like the right person to ask about : D
I've applied a couple of changes of my own to some CSS files:
/source/stylesheets/_variables.scss
/source/stylesheets/print.css.scss
/source/stylesheets/screen.css.scss
To make the FE look like this project TradeGecko
I can see the results applied in my local environment but when I commit and deploy those changes I don't see any change at all in production (Github page: https://github.com/juanroldan1989/movie-quotes-api) : (
I've already tried re-building the build folder with $ middleman build --clean command but I still don't see any changes.
Any thoughts on this ? I would really appreciate any feedback mate.
Thanks in advance !

I've just fixed this issue. I post my conclusions for anyone else out there with the same issue.
It turned out my gh-pages branch went out of sync with my work in master branch, and some commits (including the one with my changes to the FE) were not tracked properly within the gh-pages branch.
Since Github Pages fetch their content from gh-pages branch, whenever I tried the ./deploy.sh script, the content pushed and actually deployed to the server was the old one in the gh-pages branch and no changes were made to the server.
Now I've removed the gh-pages branch and created it again from master. Pushed the new gh-pages branch to my repository and everything is working as expected now : D
Cheers  !

@juanroldan1989 sorry for the delay, I just read this! Glad you were able to solve the issue :)


8.7 auth upgrades

Improved authentication using authorization code with PKCE
Simplification of sdk-config.json file
Revoke tokens on logout
Browser reload without reauthentication

It looks like something isn't quite right when I reload /embedded after completing a run through an /embedded use. Perhaps one of the bShowPega, bShowTriplePlayOptions settings isn't getting set back correctly. As shown in the attached screenshot, I can get it in the state when - after reload - clicking a Shop Now button shows the buttons and the page where the Pega flow should be. At point, I have to click the Shop Now button again.

A .har file is useful if you encounter such issues.  Until re-authentication is supported, you limited to the length of time specified by the access token within the OAuth 2.0 client registration record.  Curious whether the 15 minutes had expired since you did the initial authorization?
After which reloading the screen should be fine, more work will happen in future around handling the token expirations and refreshes.

The 15 minutes hadn't expired. I went through the flow to the end and then refreshed the page (my normal use). I'll grab a .har file but will approve the PR for now since it seems like an edge case. Thanks!


replace rusoto_s3: part 2 (last part)

Why
step 2 of https://github.com/roapi/roapi/issues/240: replace rusoto_s3 (in use_memory_table: true path) with S3 client from object_store (s3://)
rusoto_s3 is used in columnq::io::s3 and columnq::io::s3 is used in two places:

columnq::partitions_from_table_source
columnq::table::delta

we would like to remove rusoto_s3 and switch to object_store, so can unify S3/GCS/Azure in future
How
following part 1 (https://github.com/roapi/roapi/pull/244), this PR (part 2) will focus on migrating columnq::table::delta and delete columnq::io::s3
this PR also remove crate rusoto_xxx completely and close https://github.com/roapi/roapi/issues/243
Test
Rely on existing unit tests in
https://github.com/roapi/roapi/blob/1e3644ec36e1a9080dac3d5f50e3f24990f4cae7/columnq/src/table/delta.rs#L131-L132
https://github.com/roapi/roapi/blob/1e3644ec36e1a9080dac3d5f50e3f24990f4cae7/columnq/src/table/delta.rs#L151-L152

One thing I forgot to mention in the previous PR review, we should enable the aws-profile feature in the object store crate because using profile to inject credential is a very common pattern in AWS land.

One thing I forgot to mention in the previous PR review, we should enable the aws-profile feature in the object store crate because using profile to inject credential is a very common pattern in AWS land.

good point, improve here: https://github.com/roapi/roapi/pull/247


Add release annotation to App Insights post deployment

Context
AB#214348 AB#214347
Change proposed in this pull request
Modified deployment pipelines for Web and Platform to annotate App Insights post-release.
Checklist

[x] Work items have been linked (use AB#)
[ ] Your code builds clean without any errors or warnings
[ ] You have run all unit/integration tests and they pass
[x] Your branch has been rebased onto main
[x] You have tested by running locally

Need to review why the current deployment tasks are not add the annotations, before moving to a custom script.

Updating the app setting ensured automatic release annotation.


replace all references to rtfeldman/roc with roc-lang/roc

Github redirects this automatically but we might as well clean this up.

This is done!


ValueError: Attempting to unscale FP16 gradients when using algo=full

Hi, I'm having trouble training a LyCORIS using the "full" algorithm. This is the error I get:
Traceback (most recent call last):
  File "/content/kohya-trainer/train_network.py", line 1033, in <module>
    trainer.train(args)
  File "/content/kohya-trainer/train_network.py", line 849, in train
    accelerator.clip_grad_norm_(params_to_clip, args.max_grad_norm)
  File "/usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py", line 2040, in clip_grad_norm_
    self.unscale_gradients()
  File "/usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py", line 2003, in unscale_gradients
    self.scaler.unscale_(opt)
  File "/usr/local/lib/python3.10/dist-packages/torch/cuda/amp/grad_scaler.py", line 307, in unscale_
    optimizer_state["found_inf_per_device"] = self._unscale_grads_(
  File "/usr/local/lib/python3.10/dist-packages/torch/cuda/amp/grad_scaler.py", line 229, in _unscale_grads_
    raise ValueError("Attempting to unscale FP16 gradients.")
ValueError: Attempting to unscale FP16 gradients.

I'm wondering if there is anything obviously wrong in my config file (below). It works fine with other algorithms such as lora and lokr which is strange.
[Basics]
pretrained_model_name_or_path = "/content/drive/MyDrive/ComfyUI/models/checkpoints/SD15/sd-v1-5-pruned-noema-fp16.safetensors"
train_data_dir = "/content/drive/MyDrive/Loras/AveryCharacter/dataset"
resolution = "768"
seed = 23
max_train_steps = 10000	# This is overwritten by max_train_epochs anyway
max_train_epochs = 10
clip_skip = 1

[Save]
output_dir = "/content/drive/MyDrive/Loras/AveryCharacter/output"
output_name = "AveryCharacter"
save_precision = "fp16"
save_model_as = "safetensors"
save_every_n_epochs = 1
save_every_n_steps = 5000
save_state = false
save_last_n_steps_state = 1	# basically saving the last + final state if save_state set to true
# save_last_n_epochs_state = 1
# save_n_epoch_ratio = 10
# save_last_n_epochs = 10
# save_last_n_steps = 100000

[SDv2]
v2 = false
v_parameterization = false
scale_v_pred_loss_like_noise_pred = false

[Network_setup]
network_dim = 64
network_alpha = 1
dim_from_weights = false
network_dropout = 0
network_train_unet_only = false
network_train_text_encoder_only = false
resume = false
# network_weights = 'path/to/network_weights'
# base_weights = 'path/to/base_weights'
# base_weights_multiplier = 1

[LyCORIS]
network_module = "lycoris.kohya"
network_args = [ "preset=attn-mlp", "algo=full", "train_norm=True", ]

[Optimizer]
train_batch_size = 1
gradient_checkpointing = false
gradient_accumulation_steps = 1
optimizer_type = "AdamW8bit"
unet_lr = 0.0004
text_encoder_lr = 0.0004
max_grad_norm = 1.0
optimizer_args = [ "weight_decay=0.1", "betas=0.9,0.99",]

[Lr_scheduler]
lr_scheduler_type = ""
lr_scheduler = "cosine"
lr_warmup_steps = 5
lr_scheduler_num_cycles = 1
lr_scheduler_power = 1.0	# Polynomial power for polynomial scheduler
# lr_scheduler_args = ...

[Training_preciscion]
mixed_precision = "fp16"

[Further_improvement]
min_snr_gamma = 0
# noise_offset = 0.05   # cannot be set with multires_noise
# adaptive_noise_scale = 0
multires_noise_discount = 0
multires_noise_iterations = 0
# scale_weight_norms = 1

[ARB]
enable_bucket = true
min_bucket_reso = 320
max_bucket_reso = 960
bucket_reso_steps = 64
bucket_no_upscale = true

[Captions]
shuffle_caption = false
caption_extension = ".txt"
keep_tokens = 0
caption_dropout_rate = 0
caption_dropout_every_n_epochs = 0
caption_tag_dropout_rate = 0.0
max_token_length = 150
weighted_captions = false
token_warmup_min = 1
token_warmup_step = 0

[Attention]
mem_eff_attn = false
xformers = true

[Data_augmentation]
color_aug = false
flip_aug = false
random_crop = false

[Cache_latents]
cache_latents = true
vae_batch_size = 1
cache_latents_to_disk = true

[Sampling_during_training]
sample_sampler = "ddim"
# sample_every_n_steps = 5000	# overwritten by n_epochs
# sample_every_n_epochs = 1
# sample_prompts = "sample_prompts.txt"

[Logging]
logging_dir = "logs_training"
log_with = "tensorboard"
log_prefix = "AveryCharacter_"
# log_tracker_name = ?
# wandb_api_key = ?

[Dataset]
max_data_loader_n_workers = 8
persistent_data_loader_workers = true
dataset_repeats = 1	# Not sure how this is used
# dataset_class = package.module.Class
# dataset_config = ...

[Regularization]
# This is not really needed because you can do regularization by putting everything in train
# reg_data_dir = "/path/to/reg"
prior_loss_weight = 1.0

[Huggingface]
save_state_to_huggingface = false
resume_from_huggingface = false
async_upload = false
# There are more arguments

[Debugging]
debug_dataset = false

[Deprecated]
use_8bit_adam = false
use_lion_optimizer = false
learning_rate = 0.0004

[Others]
lowram = false
# in_json = "/path/to/json_metadata"
# face_crop_aug_range = 2.0
# vae = "/path/to/vae"
training_comment = ""

Extra Info

Using Google Colab T4 instance
Using the below command to install requirements

!pip install --upgrade accelerate==0.25.0 transformers==4.26.0 ftfy==6.1.1 albumentations==1.3.1 opencv-python==<IP_ADDRESS> einops==0.6.0 diffusers[torch]==0.25.0 pytorch-lightning==1.9.0 bitsandbytes==0.41.3.post2 tensorflow==2.14.0 safetensors==0.4.1 toml==0.10.2 voluptuous==0.13.1 lion_pytorch==0.0.6 dadaptation==3.1 prodigyopt==1.0 lycoris_lora xformers==0.0.22.post7

Let me know if you need any extra info, I'm quite new to training LoRAs / LyCORIS so I might be doing something silly

This should be fault in kohya.


WIP: centosci-tasks-nbjp7: Update translations from Fedora Zanata

Update translations from Fedora Zanata

[ ] po-refresh

po-refresh in progress on centosci-tasks-nbjp7.
Log: https://<IP_ADDRESS>:8493/logs/po-refresh-598-20190320-060404/


RFC: WithPushedOrigin instruction

Summary
The proposed change is the introduction of a WithPushedOrigin instruction.
The instruction allows the XCVM to "push" a new origin for just a few instructions and then return to the original origin when those instructions are done.
It can be seen as pushing a new origin on a stack for the execution of a block of instructions and then popping it back when the block is done.
The new origin can only be empty (clear the origin) or a child of the current origin.
The goal is to give developers more flexibility and a better experience when handling origins in their messages.

Starting the FCP (Final Comment Period), it will last 10 calendar days. After that, this RFC will be either accepted, rejected or explicitly postponed for a future time.

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:
https://forum.polkadot.network/t/final-comment-period-for-two-xcm-rfcs/3516/1

This makes a lot of sense to me!
Just a thought, but I wonder about simplifying the name, as the Pushed feels a little unnecessary. Maybe WithOrigin, as it is IMO clear by the struct that it only executes the contained XCM with the specified origin. Alternately, if that is unclear, maybe ExecuteWithOrigin.


New URL path /help

Is your feature request related to a problem? Please describe.
no
Describe the solution you'd like
Add path to help
Describe alternatives you've considered
N/A
Additional context
N/A

Hi @ulibn do we have any new on this issue?

@ulibn is there any update on this issue?

@jarryfull The change you request is now available in the partition under the private environment.


Quick pointer for Selenium?

First, this is a great project and it works excellent with AWS Lambda functions right out of the box. Nice work.
What would you like to have implemented?
I was hoping to use this with Selenium instead of Puppeteer.
Why would it be useful?
It would be useful for those that have large projects already using Selenium.

Do you know a quick fix to get this working? I saw selenium-webdriver/chromium, but switching to it produces an error.
Without looking into the nuts and bolts of @sparticuz/chromium, should this be possible with a few adjustments or do you know if there something in the build that would stop this from working? Thanks.
Ex:
require("chromedriver");
const chromium = require("@sparticuz/chromium");
let swd = require("selenium-webdriver");
const fs = require("fs");
let { username, password } = require("./credentials.json");
var chrome = require("selenium-webdriver/chrome");
let browser = new swd.Builder();
var options = new chrome.Options();
options.setBinaryPath("/tmp/chromium");
options.addArguments('--headless');
let tab = browser.forBrowser("chrome").setChromeOptions(options).build();
...

I've not used selenium so I'm not sure.
This package's main goal is getting a chromium binary for you to use. So as long as you call executablePath() you should get a chromium binary in/tmp/chromium
The other functions are just helpers (args).
If I get a change I'll take a look at selenium, if you figure it out, let me know, or add a PR for the examples directory.

Try calling chromium.executablePath() before the selenium web driver and it may work

Hey @Sparticuz, I just got back and tested it out. I see how it's being rolled out and took a look at how this package is extracting chromium. Even after waiting for some reason this isn't playing well with Selenium. I tested by changing the path to my local installation of Chrome as a quick test and that worked. I was able to test yours with Puppeteer in AWS and it worked just fine. Not sure what the problem is atm, but thinking there is a param that needs to be set for Selenium to use chromium.

Hi, is there any solution for this? It seems like selenium will require this amazing feature as well. Thankss


Make Markdown the default

Reorder to make Markdown the default out-of-the-box experience.

Also changing so that clipboard is the default as well.


Upgrade Conventional Commits CI to support GitHub webapp commit messages

I guess we need to upgrade our conventional commits CI to support GitHub webapp commit messages.
Originally posted by @pavlovcik in https://github.com/ubiquity/ubiquity-dollar/issues/586#issuecomment-1450904927

How did you manage to create such commit message Update packages/contracts/scripts/deploy/manager/DiamondDeployHelper.ts via github UI ?
I tried both editing with codespaces and "in place editing" and everywhere there was an option to write a proper commit message, but Update packages/contracts/scripts/deploy/manager/DiamondDeployHelper.ts seems to be auto generated.

Looks like @zgorizzo69 did that and can provide a clarification here.
I know there is one more way to edit via the GitHub UI that he might have done. In my review, there is a "suggestion" function where I can "suggest" to change the code directly. Then the reviewee can easily press a button to write the changes all from the PR review UI.

i just accepted and commited the changes it was not clear how / what test will be used for the commit message

/assign


using TrajGWAS with VCF in float type

Thank you for developing TrajGWAS package. I have the issue when trying to load VCF file( the dosage in VCF is float). The error messages implies that calculation is not working for Matrix{Float 64}.

Do you have a workaround about it? Because we would really need it to be orginal float type VCF, like this:

Many thanks ahead!

Thanks for your interest in the package. It seems like a vector of integers is placed when a vector of floating-point numbers is required (second line from the bottom of the first image). However, without seeing the code, I cannot tell where this is happening.


Allow pre-allocated Ctx

For directbuffer variants with dict for lower allocation effect.
Note that the measurements of allocations do not track the native ones, which includes the new Ctx malloc'ed when not provided.

Codecov Report

Merging #99 into master will increase coverage by 1.1%.
The diff coverage is 71.83%.

@@             Coverage Diff             @@
##             master      #99     +/-   ##
===========================================
+ Coverage     60.79%   61.89%   +1.1%
- Complexity      138      152     +14
===========================================
  Files            10       13      +3
  Lines           755      790     +35
  Branches        121      127      +6
===========================================
+ Hits            459      489     +30
- Misses          178      179      +1
- Partials        118      122      +4

Hi,
Sorry for the late reply, I was traveling.
LGTM but I have a few comments.

I would prefer to keep the existing Zstd static method signatures as they are. With the proposed changes we will break customer code as we add new parameters to existing methods.

I was also thinking how to add support for the new APIs that reuse the compression/decompression context between requests. I think we should not add the new compression/decompression methods as static methods on Zstd (it's already quite heavy) but as proper methods on ZstdCompressCtx and ZstdDecompressCtx. For example, instead of Zstd.compress(..., cctx) we will have cctx.compress(...)

Yes, you're absolutely right.
I'll rework to your suggested API, and force-push a change.

Great, thanks!

Just curious will this feature be integrated soon?

Just curious will this feature be integrated soon?

I still plan on doing the suggested changes to the API before merging. No ETA.

I've refactored the API.
This has no changes to any existing API.
Rather it adds new methods on the new context classes.

LGTM. I will review in detail and merge early next week.

Looks great. Thanks a lot for the contribution. I will follow next week with small tweaks, e.g. fix the shared log exception messages not to refer to dictionaries.

@mortengrouleff , I have pushed a continuation of the contexts support, still just for ByteBuffers. Can you look if it will fit your use case? The difference is that the dictionary is attached to the context and not supplied on each compression or decompression request.
If it works for you, I am inclined to move forward only with these new methods. Here are my reasons:

passing parameters to compression/decompression leads to combinatorial explosion of variants - Zstd class looks quite confusing for this reason.
most of the old low-level functions have limitations, e.g.:

/*! ZSTD_compress_usingCDict() :
 *  Note : compression level is _decided at dictionary creation time_,
 *     and frame parameters are hardcoded (dictID=yes, contentSize=yes, checksum=no) */

attaching params to context compose better.

Any feedback is welcome,
luben

Yes. That sounds like a good plan.
From the commits it looks like it's possible to switch to another dict reusing the same context. That should ensure the same low allocation load as the initial PR allowed, while being easier to use.
Having the loadDict(ZstdDictDecompress dict) is the crucial point I guess.
BTW: Why not have those be a void functions, as they can only return 0 or throw an exception.

Good point about making them void. Yes, dictionaries can be switched and removed dynamically.

@mortengrouleff , I pushed a cleanup commit and switched tests. I will release soon, as I already imported zstd-1.4.4 the weekend, just waiting for them to announce the release.

why is there a level on the c_ctx, isn't that set in stone when creating the dict to use?

@mortengrouleff it looks the level is picked by the CCtx. I was not expecting in neither, but also makes sense.

Ahh, okay. Makes sense then.
I wonder if the levels need to match, or if there may be a point in using a higher level when creating the dict than when applying it, as that may take a bit of extra time for the dict-build, but then be more efficient on the (say) 1000 invocation of compress that follow...

Looking forward to trying out this new release!

Looks the levels don't need to match. Also I don't see any difference in speed or compression ration depending on the dictionary compression level.

Just pushed it. It should be available on Maven central soon.


HDDS-8638. Support CryptoOutputStream and CipherOutputStream in createStreamKey

What changes were proposed in this pull request?
Currently, ClientProtocol.createStreamKey returns OzoneDataStreamOutput which is not an OutputStream. Therefore, it does not support CryptoOutputStream and CipherOutputStream since they requires OutputStream.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8638
How was this patch tested?
Added new tests.

@jojochuang , thanks a lot for reviewing this!


className mark option should allow multiple classes (separated by spaces)

It seems that the className option for marks will only accept one class but should probably accept more as it is quite common for approaches like tailwind to use many classes. Example here:
https://observablehq.com/d/25a81df102bdaa88

Yes, the validation of this option is currently too strict. We should relax it to allow multiple classes.


Unzipping zip archives that were created on Windows with 7Zip

Dear Mr. Roy Marmelstein
This is for sure not a bug. But I have not found a solution for the following problem. Maybe you can give me a quick advice.
If I create a zip archive with the OS X default compression feature that's integrated in Finder and extract this archive again with Zip.unzip then unzipping works.
If I create an archive on Windows with 7Zip then Zip.unzip throws an exception 'Error.UnzipFail' in file 'zip.swift' at line 92 because 'unzOpenCurrentFile' returned with -103 (UNZ_BADZIPFILE).
I have tryed all zip compression algorithms that are offered by 7Zip: Deflate, Deflate64, BZip2, LZMA and PPMd.
Can you tell me the algorithm to use for compression, to successfully extract archives with your library on iOS?
Yours sincerely
Tobias Miller

Hey @TobiasWGDE, that's an interesting issue. Zip is built on top of minizip (https://github.com/nmoinvaz/minizip). Maybe it's missing something to deal with Windows-produced zip files. I'm afraid I won't be able to look into this until Tuesday though...

i've investigated the problem further and could track this behavior down to a case where it seems that I can reproduce the bug, but not reliably. Whether the bug occurs or not, seemingly depends on randomly moving some files or folders out of or into the directory structure that gets zipped.
One thing I haven't mention, is that the files were originally created on OS X 10.11.2 and then transferred with dropbox to a Windows 10 system.
Maybe this is some kind of random CRC-Bug caused by 7Zip, that minizip can't handle. Or there are file system level encoding issues between Mac FS, dropbox cloud storage and NTFS.
Tomorrow I will try to attache some example archives.

Here are the archives. The archives were created with 7Zip. Algorithm: 'Deflate', All other compression settings were left in their defaults.
original_content.zip contains the files and folders, which should be zipped:
It has a top level folder, that contains text files, images and a 'gallery' folder. The 'gallery' folder contains again text files, images and a 'thumbs' folder. The 'thumbs' folder contains images only.
These are working:
without_thumbs_folder.zip: Everything but the 'thumbs' folder
thumbs_folder_only.zip: Only the 'thumbs' folder
These don't:
original_content.zip
original_content_copied_some_files_rm.zip: Copied to an other location, then I removed some files in each folder.
original_content.zip
original_content_copied_than_some_file_rm.zip
thumbs_folder_moved_one_level_up.zip
thumbs_folder_only.zip
thumbs_folder_removed.zip
thumbs_folder_moved_one_level_up.zip: Same as original_content.zip but 'thumbs' folder moved one level up.

Hi @TobiasWGDE, thanks for adding the files.
I ran some tests and found the issue. In all the files that don't work, the top-level-folder contains a special character - for example, in original_content.zip, the top-level folder is
test_flie·text
It looks like 7zip doesn't encode the character · correctly and Zip is unable to cast it to String/NSString. That is what is causing the failure.
Other compression tools (tested with Apple's Archive utility) can encode the character · without an issue.


[11.x] Replaced array_push with direct array syntax

According to PHP documentation, using $array[] = to add a single element is preferred over array_push() due to reduced overhead from function calls.

@xurshudyan Maybe all other occurrences of array_push() should be replaced as well? A quick search revealed only 6 other files that still contain array_push().


hint-text and beep config

There should be an option to disable the "beep" that sounds when a barcode is scanned as well as the option to remove the "Place your barcode inside the viewfinder rectangle to scan it" text.
Taken from the old repo.
The ngCordova documentation links to the old repo. Should I open an issue in ngCordova repo to fix the link?

+1 would be great to be able to disable the beep

+1 the beep makes this completely unusable for me

+1

I was able to overwrite the beep sound with a silent ogg file of my own (effectively silencing it but not eliminating the audio playback) using a build hook:
#!/usr/bin/env node

var fs = require('fs');

var rootdir = process.argv[2];

if(rootdir) {
	var filename = 'beep.ogg';
	if(fs.existsSync(filename)) {
		fs.rename(filename, 'platforms/android/res/raw/' + filename, function(e) {
			console.warn(e);
		});
	} else {
		console.warn(filename + ' does not exist');
	}
}

In config.xml:
<platform name="android">
	<hook type="before_compile" src="overwrite_beep.js" />
</platform>

Beep disabling would be useful

+1

it is possible to set the hint-text to nothing:
cordova.plugins.barcodeScanner.scan(
      function (result) {},
      function (error) {},
      {
          "prompt" : "" // supported on Android only
      }
   );

+1

I don't want to keep the beep too... What a pity that it is not customizable in the plugin...
However, following the @cookieguru 's idea, with Ionic 2, I have done :

create a silence.ogg (with Audacity, generate Silence, export in .ogg)
ionic hoooks add, if you have not the hooks structure with after_prepare
create a new file in hooks/after_prepare/030_overwrite_beep_scanner.js

#!/usr/bin/env node

var fs = require('fs');

var rootdir = process.argv[2];

if (rootdir) {
    var filename = 'silence.ogg';
    var dirPart = __dirname + '/resources/';
    if(fs.existsSync(dirPart + filename)) {
        fs.createReadStream(dirPart + filename).pipe(fs.createWriteStream('platforms/android/res/raw/beep.ogg'));
    } else {
        console.warn(dirPart + filename + ' does not exist');
    }
}

chmod 755 030_overwrite_beep_scanner.js
copy silence.ogg in hooks/after_prepare/resources/

Hope it helps!

For those who uses meteor I finaly found how to do that by just putting a new beep.ogg (silent of course) in a special top folder that meteor provide.
So first u have to create a folder named cordova-build-override, then inside create a platforms/android/res/raw folder and place ur new beep.ogg inside.
this url gave me the idea : https://documentation.onesignal.com/docs/customize-notification-sounds


Ruby 1.9.3

FYI, this is now working on Ruby 1.9.3 - might be worth updating the readme!

Ah, removed one notice but not the other. Thanks again


[Bug]:  bulkWalkCmd introduces loop when host unreachable

Expected behavior
When trying to iterate over the generator returned by bulkWalkCmd, we'd expect to see an exception raised in case the target host is unreachable or not responding to the getBulk request.
Actual behavior
Instead we see a loop occuring inside the generator, where a getBulk message is repeatedly being sent out.
Detailed steps

create a new bulkWalkCmd request
call next()
loop occurs

We first found this issue on our application code base, but this can be replicated with the sample code provided in https://docs.lextudio.com/pysnmp/docs/hlapi/asyncio/manager/cmdgen/bulkwalkcmd since the demo.pysnmp.com host is unreachable.
Python package information
6.1.2
Operating system information
Ubuntu 22.04.4 LTS
Python information
3.10.12
(Optional) Contents of your test script
from pysnmp.hlapi import *
from pysnmp import debug

# use specific flags or 'all' for full debugging
debug.setLogger(debug.Debug('all'))
g = bulkWalkCmd(SnmpEngine(),
   CommunityData('public'),
   UdpTransportTarget(('demo.pysnmp.com', 161)),
   ContextData(),
   0, 25,
   ObjectType(ObjectIdentity('SNMPv2-MIB', 'sysDescr')))
next(g)

Relevant log output
2024-04-30 12:24:27,460 pysnmp: StatusInformation: {'errorIndication': RequestTimedOut('No SNMP response received before timeout')}
2024-04-30 12:24:27,460 pysnmp: processResponsePdu: sendPduHandle 4922693, statusInformation {'errorIndication': RequestTimedOut('No SNMP response received before timeout')}
2024-04-30 12:24:27,461 pysnmp: sendPdu: securityName s2108015491253068459, PDU
GetBulkRequestPDU:
 request-id=16078184
 non-repeaters=0
 max-repetitions=25
 variable-bindings=VarBindList:
  VarBind:
   name=<IP_ADDRESS>.<IP_ADDRESS>
   =_BindValue:
    unSpecified=

2024-04-30 12:24:27,461 pysnmp: sendPdu: current time 20 ticks, one tick is 0.5 seconds
2024-04-30 12:24:27,461 pysnmp: sendPdu: new sendPduHandle 4922694, timeout 2.0 ticks, cbFun <bound method CommandGenerator.processResponsePdu of <pysnmp.entity.rfc3413.cmdgen.BulkCommandGenerator object at 0x7609e98268c0>>
2024-04-30 12:24:27,461 pysnmp: prepareOutgoingMessage: PDU request-id 16078184 replaced with unique ID 16169135
2024-04-30 12:24:27,461 pysnmp: prepareOutgoingMessage: using contextEngineId <SnmpEngineID value object, tagSet <TagSet object, tags 0:0:4>, subtypeSpec <ConstraintsIntersection object, consts <ValueSizeConstraint object, consts 0, 65535>, <ValueSizeConstraint object, consts 5, 32>>, encoding iso-8859-1, payload [0x80004fb8056465...732d766d38998ec0]> contextName b''
2024-04-30 12:24:27,461 pysnmp: generateRequestMsg: using community <OctetString value object, tagSet <TagSet object, tags 0:0:4>, subtypeSpec <ConstraintsIntersection object, consts <ValueSizeConstraint object, consts 0, 65535>>, encoding iso-8859-1, payload [public]> for securityModel <SnmpSecurityModel value object, tagSet <TagSet object, tags 0:0:2>, subtypeSpec <ConstraintsIntersection object, consts <ValueRangeConstraint object, consts -2147483648,<PHONE_NUMBER>>, <ValueRangeConstraint object, consts 0,<PHONE_NUMBER>>, <ValueRangeConstraint object, consts 1,<PHONE_NUMBER>>>, payload [2]>, securityName <SnmpAdminString value object, tagSet <TagSet object, tags 0:0:4>, subtypeSpec <ConstraintsIntersection object, consts <ValueSizeConstraint object, consts 0, 65535>, <ValueSizeConstraint object, consts 0, 255>>, encoding utf-8, payload [s2108015491253068459]>, contextEngineId <SnmpEngineID value object, tagSet <TagSet object, tags 0:0:4>, subtypeSpec <ConstraintsIntersection object, consts <ValueSizeConstraint object, consts 0, 65535>, <ValueSizeConstraint object, consts 5, 32>>, encoding iso-8859-1, payload [0x80004fb8056465...732d766d38998ec0]> contextName b''
2024-04-30 12:24:27,462 pysnmp: generateRequestMsg: Message:
 version=1
 community=public
 data=PDUs:
  get-bulk-request=GetBulkRequestPDU:
   request-id=16169135
   non-repeaters=0
   max-repetitions=25
   variable-bindings=VarBindList:
    VarBind:
     name=<IP_ADDRESS>.<IP_ADDRESS>
     =_BindValue:
      unSpecified=

2024-04-30 12:24:27,462 pysnmp: sendPdu: MP succeeded
2024-04-30 12:24:27,462 pysnmp: sendMessage: sending transportAddress ('<IP_ADDRESS>', 161) outgoingMessage
00000: 30 28 02 01 01 04 06 70 75 62 6C 69 63 A5 1B 02
00016: 04 00 F6 B8 AF 02 01 00 02 01 19 30 0D 30 0B 06
00032: 07 2B 06 01 02 01 01 01 05 00
^CTraceback (most recent call last):
  File "/home/rmarques/devops/lldp-parser/app/from pysnmp.py", line 12, in <module>
    next(g)
  File "/home/rmarques/.local/share/virtualenvs/lldp-parser-63Lywrb8/lib/python3.10/site-packages/pysnmp/hlapi/asyncio/sync/cmdgen.py", line 679, in bulkWalkCmd
    result = loop.run_until_complete(run())
  File "/usr/lib/python3.10/asyncio/base_events.py", line 636, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
    self._run_once()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 1871, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/lib/python3.10/selectors.py", line 469, in select
    fd_event_list = self._selector.poll(timeout, max_ev)
KeyboardInterrupt

After some debugging on pysnmp code, I could see that the issue is that the once inside  ./hlapi/asyncio/cmdgen.py
It seems that nothing is really handling the requestTimedOut exception:
lines 958 - 966
        if errorIndication:
            yield (
                errorIndication,
                errorStatus,
                errorIndex,
                varBindTable and varBindTable[0] or [],
            )
---->           if errorIndication != errind.requestTimedOut:
                return

After this nothing in the code caughts it and since this is in a run_until_complete loop I guess it causes the loop
I've changed this to
lines 958 - 966
        if errorIndication:
            yield (
                errorIndication,
                errorStatus,
                errorIndex,
                varBindTable and varBindTable[0] or [],
            )
---->           if errorIndication == errind.requestTimedOut:
                return

just to test it out and the loop stops, but not sure what's the impact this might have to the rest of the code....

Since we retired the sync API in release 6.2, this issue will not be worked on.


chore: address and delete many porting notes across the library

I went hunting for random porting notes and deleted / addressed ones that were relatively easy to deal with.

bors merge


failed to set power state macosx ventura

Required Info

Camera Model
D435i

Firmware Version
5.15.1

Operating System & Version
MacOS 13.5 Apple Silicon

Platform
Mac Realsense Viewer

SDK Version
v2.50.0

I built the SDK for mac as instructed in the documentation, however whenever I open realsense viewer with sudo realsense-viewer, I am bombarded with errors stating UNKNOWN in rs2_get_option(options:0x6000036dc960, option:Stereo Baseline): failed to set power state except replace Stereo Baseline with many different features each time the error pops up I click "ok" and a new one shows up.
What I've tried
I tried building libusb separately as stated in https://lightbuzz.com/realsense-macos/
I've seen posts telling me to copy 99-realsense-libusb.rules, but Im not sure where to copy it for MacOS.
Additionally,
When I try to see if the camera is recognized in pyrealsense2-macosx, it states that one realsense device is connected, but it gives me an error saying no device is found when i try to use it in Python.

Hi @CG-Spectre  A RealSense user with MacOS Ventura at https://github.com/IntelRealSense/librealsense/issues/11815#issuecomment-1620581786 also had the Failed to set power state error when using the RealSense Viewer tool and problems with the Python wrapper.
An alternative approach to the Lightbuzz guide for installing librealsense on Apple Silicon for Ventura is to use the command brew install librealsense
https://formulae.brew.sh/formula/librealsense
One RealSense user suggested using brew install libusb first before doing brew install librealsense.

I am having the same issues on m2 mac with Ventura. I know using sudo is meant to fix it but the USB power issues are still there, albeit it intermittently does work at a very low frame rate. I have tried installing with homebrew with no such luck. Any other ideas?

Hi @MartyG-RealSense. Thanks so much for the quick reply! I have tried using both a USB c to c cable and a USB A to C adaptor, without success.
In the past I was able to get the same camera reliably working on an Intel Mac with this USB a to c adaptor (for this to work I had to always use sudo).
I have been forced to upgrade to the m2 and now cannot get it to reliably work unfortunately. Sometimes it runs with sudo but most of the time not.

Hi @CG-Spectre  A RealSense user with MacOS Ventura at https://github.com/IntelRealSense/librealsense/issues/11815#issuecomment-1620581786 also had the Failed to set power state error when using the RealSense Viewer tool and problems with the Python wrapper.
An alternative approach to the Lightbuzz guide for installing librealsense on Apple Silicon for Ventura is to use the command brew install librealsense
https://formulae.brew.sh/formula/librealsense
One RealSense user suggested using brew install libusb first before doing brew install librealsense.

Thanks I'll try doing these things as soon as I can, though I already have tried installing libusb beforehand.

@CG-Spectre  I look forward to your next update.  Good luck!
@jen-co  If you were using an Intel Mac before and switched to M2 then the problem may be with Apple Silicon in general rather than the M2 version of Apple Silicon specifically.  Which installation method did you use to install the librealsense SDK on your Mac, please?

@CG-Spectre I look forward to your next update. Good luck!
@jen-co If you were using an Intel Mac before and switched to M2 then the problem may be with Apple Silicon in general rather than the M2 version of Apple Silicon specifically. Which installation method did you use to install the librealsense SDK on your Mac, please?

Installing via Brew, unfortunately, didn't have much of an effect, however, I'm now trying to test to see if it works with my own XCode project. I was able to link the header files, but even after linking the dylib file that was generated after building librealsense, I'm getting the error Library 'realsense<IP_ADDRESS>' not found. Even though the dylib file does appear under the Frameworks folder of my project.

@MartyG-RealSense thanks for clarifying. I installed via brew.

The librealsense SDK has not officially supported MacOS since 2017's 'High Sierra' version, though the SDK has continued to work well with modern Macs up until Apple Silicon models and MacOS Monterey.
Monterey was very difficult to get working with RealSense.  The RealSense MacOS community exchanged advice with each other to work through the problems, and eventually the Lightbuzz installation guide was created.
Ventura also has problems with librealsense, and the solution is again likely to come from the community of Ventura users as Intel no longer provides Mac support for RealSense and has no Mac computers to test with unfortunately.

Hi @CG-Spectre  You are very welcome.  Thanks very much for the update.

Case closed due to no further comments received.

Facing the same issue with macos sonora 14.1
Tried almost everything, no matter what it simply ends-up with segmentation fault or failed to set power state
The lib installed from brew (brew install librealsense) gives following logs:
 15/11 15:28:52,992 INFO [0x1e6145ec0] (context.cpp:336) Found 1 RealSense devices (mask 0xff)  15/11 15:28:53,068 INFO [0x1e6145ec0] (rs.cpp:2697) Framebuffer size changed to 1209 x 781  15/11 15:28:53,068 INFO [0x1e6145ec0] (rs.cpp:2697) Scale Factor is now 1 [1]    59744 segmentation fault  sudo realsense-viewer
Another approach was to build everything from source using the script here (does almost the same suggested here: https://lightbuzz.com/realsense-macos/) -- https://github.com/yugasun/pyrealsense2-mac/blob/main/build.sh
This solution faces the issue of failed to set power state.
If anyone have any solution to make this work on macos sonoma with apple m1-m3 processors please let me know

@kirienkomaxym  I wonder if you may have found a solution to your issue? I am faced with the same problem currently. Thanks

Woww.. Can't believe it's so difficult to get working with M1.


Make sure generics are correct for generified return types

Fixes https://issues.apache.org/jira/browse/GROOVY-7846

Would be nice to integrate in 2.4.7.

+1 please get this into 2.4.7


The infrastructure/DEPLOYMENT.md instructions do not work

Problem description
The instructions in infrastructure/DEPLOYMENT.md did not work and the CDK_EOAPI_DB_PGSTAC_VERSION is outdated.
Specifically:

The command CDK_EOAPI_STAGE=staging CDK_EOAPI_DB_PGSTAC_VERSION=0.7.1 npm --prefix infrastructure/aws run cdk -- synth returns AssertDescription: CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI.. I run npm --prefix infrastructure/aws run cdk bootstrap and after re-run the commands but nothing changed.
The command CDK_EOAPI_STAGE=staging CDK_EOAPI_DB_PGSTAC_VERSION=0.7.1 npm --prefix infrastructure/aws run cdk -- deploy eoAPI-${CDK_EOAPI_STAGE} returns No stacks match the name(s) eoAPI-staging
I guess CDK_EOAPI_DB_PGSTAC_VERSION should become equal to 0.7.10

Expected Output
Deploy eoAPI in aws.
Environment Information

Os: Ununtu
Python: 3.10.11
Node: v18.16.1
npm: 9.5.1
CDK version: 2.76.0

@charalamm thanks for opening the issue and also a PR 🙏
when writing the documentation I assumed we could have the env variable provided in a onliner
CDK_EOAPI_STAGE=staging echo ${CDK_EOAPI_STAGE}
>>

But as shown in ☝ , it doesn't work 🤦 so to fix this you need to do 👇
CDK_EOAPI_STAGE=staging CDK_EOAPI_DB_PGSTAC_VERSION=0.7.1 npm --prefix infrastructure/aws run cdk -- deploy eoAPI-staging

For the bootstrap part, I'm not sure what's going on but I'll have a look

@vincentsarago It works now with the changes in infrastructure/aws. Thank you!

Noted. Thanks!


new aws account request for ERIC

Environment details
Application Name
ERIC

Description of application
LAA use this application for sharing Management Information with Legal Aid providers, currently only one report has been released. Effectively this is the frontend of the MI service

GitHub team slug

Environments

Environment
Access level

[ ] Development
[ ] view-only[ ] developer[ ] sandbox

[ ✓] Test
[ ] view-only[ ] developer

[ ] Preproduction
[ ] view-only[ ] developer

[✓ ] Production
[ ] view-only[ ] developer

Tags

tag
value

application

business-unit

infrastructure-support

owner

Networking options
Subnet sets

[ ] General - this application will use the same subnet as other applications in your business unit
[ ] Isolated - this application has highly sensitive data which must have its own isolated subnet with no network connectivity

How do users connect to the application?

[ ] Over the public internet
[ ] With a purple cabled device (please give details)
[ ] With a MoJ Official device

Connectivity to other applications or external parties

Additional features

[ ] Additional VPC Endpoints
[ ] Extended DNS Zones
[ ] Other - please specify

Other information

Definition of done

[ ] account created
[ ] customer informed

duplicate sorry


Adding pinned Repositories

Please provide an option to add pinned repositories instead of starred or recently added

Now you can customize the projects you want to display in manual mode. https://github.com/arifszn/gitprofile?tab=readme-ov-file#external-projects


Use of VKPP_CALL() does not match definition in vkpp master

In vkpp master, VKPP_CALL() takes three parameters, but in the usage in vpp, only one parameter is given. Compilation therefore fails.

I just merged the right branch into master here, it compiles for me now. Please post the compiler output if it still doesn't work for you.
Thank you for making me fix builds on master for all my repos btw, appreciated :D

My pleasure, or perhaps masochism. :-)
vpp now compiles cleanly on my side including intro_glfw.
intro_ny also compiles though with a single warning:
Dependency ny from subproject subprojects/ny found: YES
Dependency glfw3 found: YES (cached)
Build targets in project: 6
Found ninja-1.9.0 at /usr/bin/ninja
[36/37] Compiling C++ object 'docs/exa...50135fc<EMAIL_ADDRESS>In file included from ../docs/examples/intro_ny.cpp:32:
../subprojects/ny/include/ny/keyboardContext.hpp:15:7: warning: ‘class ny::KeyboardContext’ has virtual functions and accessible non-virtual destructor [-Wnon-virtual-dtor]
   15 | class KeyboardContext {
      |       ^~~~~~~~~~~~~~~
[37/37] Linking target docs/examples/intro_ny.

Great! No on to rvg....Which still fails on the vkpp output.cpp bug. I guess you haven't refreshed it yet.

Good to hear!


Pulumi package publishing error

Hello @muhlba91
I am one of the maintainers of the Pulumi registry. There are some issues getting the latest version of your package (v5.0.0) published to our registry as it seems it is unable to locate a schema file. It looks like in your previous version there was one located there. This seems to no longer be the case with 5.0.0 and our publishing pipeline is giving an error since it is unable to locate a schema file. Not sure if there were some recent changes in the latest release that need to be adjusted here. Feel free to reach out and happy to assist in any way that I can!

Hi,
thank you for your report! Yes, I have gitignored all auto-generated files, and, hence, also the schema files.
It's back in starting with v5.0.1.


Add Yueyun Shu to Contributors list

Hi  @rsandx, I'm quite elated about your pull request. I wanna evolve this project to addresses various problems faced by first-time contributors. I'd love to learn about your journey in open source community, the problems, pain points you had etc.
Could you explain how you felt when you went through the tutorial, made a pull request and learned that I merged it?
We’ve recently added social share  to our web app. Could you please go to https://roshanjossey.github.io/first-contributions/#social-share and share your first contribution to open source?
Also, check out projects with easy issues while you’re there.

The tutorial is quite straightforward, I had no problem to follow it and made the first pull request.

I’d urge you to continue contributing to other open source projects.
We also have a slack team to help/mentor people trying to contribute to open source projects.
You can join the team by going to https://goo.gl/PM1Jdh


No "run all above" on first cell and don't start history if not sending any code.

For #4743

[X] Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR)
[X] Title summarizes what is changing
[X] Has a news entry file (remember to thank yourself!)
[ ] Has sufficient logging.
[ ] Has telemetry for enhancements.
[X] Unit tests & system/integration tests are added/updated
[ ] Test plan is updated as appropriate
[ ] package-lock.json has been regenerated by running npm install (if dependencies have changed)
[ ] The wiki is updated with any design decisions/details.

when statements for package.json commands don't have a good way to check what line you are currently on. But instead I just changed the moment that we started up history to just before we send code, so now we don't get that spurious startup when running an empty line.

Maybe put a comment in one of the runcell commands saying why the history is acquired each time?

In reply to: 474003187 [](ancestors = 474003187)

Sounds good. I'll add it in on the first of the commands.

In reply to: 474008932 [](ancestors = 474008932,474003187)


Class list in alphabetical order

Attribute Terminologies and Data Elements CodeSets are displayed in Mauro in alphabetical order:

However, Classes are not. This makes it difficult to find the item you are looking for.
James has said it is possible to change the Class order to alphabetical to match the others.

Some of the bugs in the ordering code have now been fixed.  However, we still need to add functionality for explicitly deciding whether the contents of a class should be ordered or not, and present the contents through the APIs accordingly.

James thinks this fixed.

@jamesrwelch  - above you said that you thought this was fixed.
Classes are in alphabetical order but when a new one is created it goes to the bottom of the list.
Do you know if this will be fixed as part of the OCC work?

This is inconvenient, yet is not preventing you from creating data dictionary items, so I disagree with the "Showstopper" label. This will have to be an issue the Mauro team need to fix due to the remaining work System C need to do to complete the migration to Mauro.

@jamesrwelch - can you do anything about this please?
I do not agree with Peter that it is not important.
This was raised in 2022 and is still an issue.

30/07/24 - James will look into this and see how big a fix this is.
It is only Classes this fails on, every other items appear in alphabetical order once created.


feat: add kind to local scope

Add kind to local scope and use to distinguish different scope types. It is also used to distinguish l/r values in schema expr to provide more accurate completion in lsp.

1. Does this PR affect any open issues?(Y/N) and add issue references (e.g. "fix #123", "re #123".):

[ ] N
[x] Y #747

2. What is the scope of this PR (e.g. component or file name):
kclvm/sema/src/advanced_resolver/mod.rs

3. Provide a description of the PR(e.g. more details, effects, motivations or doc link):
Add kind to local scope and use to distinguish different scope types. It is also used to distinguish l/r values in schema expr to provide more accurate completion in lsp.

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[x] Other

4. Are there any breaking changes?(Y/N) and describe the breaking changes(e.g. more details, motivations or doc link):

[x] N
[ ] Y

5. Are there test cases for these changes?(Y/N) select and add more details, references or doc links:

[x] Unit test
[ ] Integration test
[ ] Benchmark (add benchmark stats below)
[ ] Manual test (add detailed scripts or steps below)
[ ] Other

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 87.787%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2293

Relevant Lines:
2612

💛  - Coveralls


getFiles() in react native with relayjs mutations

Hi!
I need to download multiple files from react native app throw relayjs mutation. Files array looks like:
[
    {uri: 'file:///path/to/file.jpg', mime: 'image/jpg'},
    {uri: 'file:///path/to/file2.jpg', mime: 'image/jpg'},
]

I found getFiles() func for mutations, but I don't understand what format of the data it should return?

the format is a map
{
'filename1': File1,
'filename2': File2,
}


WIP: HTTP Client abstraction

motivation: incoming features use HTTP to access remote resources: package collection and package registry. we should abstract given complexy around error handling and non-apple platform support
changes:

create HTTPClient utility in Basics modules

TODO:

verify concept makes sense
add tests
discuss our approach to non-apple platforms (given FoundationNetworking dependency on OpenSSL)

cc @yim-lee @mattt

to be clear: this is pretty rough, its just to see if the concept makes sense and we should continue to invest energy in this direction

@neonichu aware of Downloader in TSC, once we feel good about this we should discuss consolidating

Candidly, I'm not entirely sure what problem this solves, and I'm skeptical of the benefit of building our own networking abstraction here.
If the concern is cross-platform compatibility, that seems like something that should either be solved at the Foundation layer or avoided by adopting another networking stack vis-à-vis AsyncHTTPClient.
If the hope is to unify the various components that use networking, my instinct is to say that "duplication is far cheaper than the wrong abstraction" — especially when we're talking about N ≤ 3.

I believe the motivation is to be able to use Foundation on Apple platforms and AsyncHTTPClient on others, right?

Candidly, I'm not entirely sure what problem this solves, and I'm skeptical of the benefit of building our own networking abstraction here.
If the concern is cross-platform compatibility, that seems like something that should either be solved at the Foundation layer or avoided by adopting another networking stack vis-à-vis AsyncHTTPClient.
If the hope is to unify the various components that use networking, my instinct is to say that "duplication is far cheaper than the wrong abstraction" — especially when we're talking about N ≤ 3.

hi @mattt, all excellent points. The reason I bout this up and created https://github.com/apple/swift-package-manager/pull/3040 is that there are at least 4 places that we now need an http client in SwiftPM: Downloader, Registry, Collections x 2.
Downloader gets away with using FoundationNetworking since its supporting a feature that only works on Apple platforms (binary dependencies) but this is a problem we know we need to address.
using AsyncHTTPClient is indeed very compelling but it means Xcode will gain a dependency on it via libSwiftPM which is non-trivial.
You are correct to point out that this should be solved by fixing FoundationNetworking to not depend on OpenSSL. One way to do that which was discussed in length on other forums was the idea to "rebase" URLSession in non-Apple platforms on namespaced copy of AsyncHTTPClient or SwiftNIO, but this is non-trivial work and AsyncHTTPClient does not yet support everything that URLSession needs.
Given that there is no immediate solution to fixing FoundationNetworking such that it does not depend on OpenSSL, I feel a unified abstraction in SwiftPM that deals with HTTP requests could be compelling until one is put in place. This will allow us to move this complexity away from the call sites and evolved it over time.

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test linux

@swift-ci please smoke test

@swift-ci please smoke test

This seems to have caused a build failure on Windows: https://dev.azure.com/compnerd/3133d6ab-80a8-4996-ac4f-03df25cd3224/_apis/build/builds/42746/logs/46


Add new optional parameter newMethodName to ChangeMethodTargetToStatic recipe

What problem are you trying to solve?
I want to replace some static method from a class to another class with potentially another method name.
Ex : migrate Guava to commons-lang (or the contrary), this kind of method exists in spring or log4j notably
com.google.common.base.Strings.Strings.isNullOrEmpty -> org.apache.commons.lang3.StringUtils.isEmpty
Describe the solution you'd like
Add new optional parameter newMethodName defaulted to methodName by default to the ChangeMethodTargetToStatic recipe.
Have you considered any alternatives or workarounds?
No.
Thanks in advance.

Hi @alexandrenavarro ; You can already achieve such a migration by combining two recipes together, as for instance seen here:
https://github.com/openrewrite/rewrite-static-analysis/blob/6d9048c86852119be4d552907b404e788fc2efc6/src/main/resources/META-INF/rewrite/java-best-practices.yml#L44-L53
type: specs.openrewrite.org/v1beta/recipe
name: org.openrewrite.staticanalysis.ReplaceValidateNotNullHavingSingleArgWithObjectsRequireNonNull
displayName: Replace `org.apache.commons.lang3.Validate#notNull` with `Objects#requireNonNull`
description: 'Replace `org.apache.commons.lang3.Validate.notNull(Object)` with `Objects.requireNonNull(Object)`.'
recipeList:
  - org.openrewrite.java.ChangeMethodName:
      methodPattern: org.apache.commons.lang3.Validate notNull(Object)
      newMethodName: requireNonNull
  - org.openrewrite.java.ChangeMethodTargetToStatic:
      methodPattern: org.apache.commons.lang3.Validate requireNonNull(Object)
      fullyQualifiedTargetTypeName: java.util.Objects

This way each recipe focuses on just one aspect, while together they accomplish a complete migration. Not how the second recipe matches on the non-existing Validate requireNonNull, yet still is able to change the type.
We prefer this approach of combining recipes, rather than adding additional options to each recipe for combinations of operations to apply. We have added a recipe builder to aid in composing such existing recipes together: https://app.moderne.io/recipes/builder
I hope that helps! And if you compose and want to share any such recipes to got from Guava to Commons-Lang, those would fit right in here: https://github.com/openrewrite/rewrite-migrate-java/tree/main/src/main/java/org/openrewrite/java/migrate/guava

Thanks you, I did not realize you can do it with 2 recipes.


AJ-955: just the backup changes from #280

This PR started by extracting the changes to the backup APIs from #280. It does NOT include anything from that other PR about WDS->WDS communication when in clone mode or integration with Leo. It's just the changes to the backup APIs.
Then, I responded to my own comments in #280 and tweaked the implementation to address those comments. This PR looks different than #280, most obviously:

I changed the API endpoint signatures
I changed the API responses to use a generic Job response; this sets us up for future work in making these APIs and snapshot-import, PFB-import APIs asynchronous. This change cascaded a bunch of Java model changes.

Objective of this PR is to keep the backup changes and the cloning changes separately reviewable. Hopefully we can merge this one first and then that other PR becomes smaller!

a few minor comments but otherwise look good! thanks for getting this in - I agree it will help make my PR easier to wrap up!


IO streaming for sub-processes

Right now stdout and stderr are printed only when the streams close, which means the user is kept waiting during long running sub-processes. One solution would be a new logging strategy based on events. Events would also improve logging for concurrent tasks, i.e. serve.

This will be easily possible once #53 is done.


--tsv ignores --nodeclined

--nodeclined inhibits display of declined events, except when combined with --tsv. --tsv should respect --nodeclined.

https://github.com/insanum/gcalcli/commit/c1234389658bb6c5c4808f66642616d080b40d69 is an analogous fix for --nostarted.

Fixed as part of 757cb0f.


Add 'setuptools.sub_command' entry-points

This PR expands on the work done in #2591, to support plugins that wish to add custom steps to the build process.

Summary of changes
The method setuptools.Command.get_sub_commands is extended to load sub-commands from setuptools.sub_commands.* entry-points. (It also automatically insert items to distribution.cmdclass, so there is no need to specify extra distutils.commands entry points).
The entry-points should correspond to a setuptools.Command subclass when loaded.
There are 3 optional attributes these subclasses can define to customise when/if the sub-commands will run:

after OR before -- a string with the name of another already defined sub-command (by default the new sub-command will run after the last of the existing ones).
condition -- a string with the name of a predicate method in the parent command class.

Closes (potentially?) #2591
Things purposefully not considered (future work)
The changes in this PR improve the ergonomics for setuptools plugins writers, but not for end-users (via setup.py or setup.cfg files).
Currently there is an asymmetry on how plugin writers and end-users can customize the setuptools build process. Plugin writers can define any entry-point, while end users are restricted to cmdclass.
I believe that this asymmetry should be considered in a separated proposal, and then, the ergonomics defined here can be extended to end-users.
Pull Request Checklist

[x] Changes have tests
[x] News fragment added in changelog.d/.
(See documentation for details)

disclaimer: the following feedback is made without any in-depth analysis of the problem or solution.

after OR before -- a string with the name of another already defined sub-command (by default the new sub-command will run after the last of the existing ones).

When I see after and before, it feels like an imperfect ordering with possible irreconcilable parameters. How about instead allowing for a priority or order that's a numeric value, allowing plugins to declare a position in an ordered set?

sub_commands

As a nitpick, let's use subcommand or some other single-word moniker if possible (at least in the public interface). Having an underscore-separated indicator (sub_commands) creates ambiguity (get_sub commands vs. get sub_commands). Maybe step or operation.

Hi @jaraco thank you very much for the comments and sorry for the long time it took me to reply.

When I see after and before, it feels like an imperfect ordering with possible irreconcilable parameters. How about instead allowing for a priority or order that's a numeric value, allowing plugins to declare a position in an ordered set?

Do you mean having 2 parameters? One for the priority order (i.e. the order with which the plugins will be loaded) + one for the position? I think that even if we add such kind of parameter there will always be the chance that the position in the ordered set is not consistent across multiple environments.
Depending on which dependencies the final user adds to their [build-system] requires (e.g. if they add multiple plugins defining extra build-steps), plugins will experience different orderings.
In my opinion the best we can do is to guarantee the relative position of the 3rd party in relation to the "built-in build steps".
Would it be better if instead before or after we have append or prepend, meaning that the command should be run after all "builtin build steps" or before all of them?

As a nitpick, let's use subcommand or some other single-word moniker if possible (at least in the public interface). Having an underscore-separated indicator (sub_commands) creates ambiguity (get_sub commands vs. get sub_commands). Maybe step or operation.

I agree with you in this point, however distutils already uses sub_commands and get_sub_commands, so the implementation in the PR tried to promote consistency. I am happy to change that if you prefer.

I feel like it isn't possible to guarantee any consistent complete ordering when considering multiple plugins that don't know about each other. finalize_distribution_options support an order propery to specify a preferred priority. But there is no mention of how setuptools breaks ties. This would leave us in the same place as being discussed here, which seems OK. It seems unlikely that there would be two plugins would have an interaction where one doesn't operate correctly when running after the other. However, I could see a plugin that specifically needs to be run before build_clib, as an example. So I guess I'm leaning towards supporting both before/after and priority.
before & after can be used for when the plugin specifically needs to be executed relative to first party setuptools sub-commands. This allows a plugin to control what bucket they are placed in. A plugin that registers for after X is in a distinct bucket that always executes before a plugin that registers for before x+1. If you "just want to run when build executes", don't specify anything and it will be placed in the last bucket (maybe a special bucket that comes after the after bucket of the last first party sub-command?).
order (used to be consistent with finalize_distribution_options) defaults to 0 and is only used to sort the ordering of custom sub-commands within each specific bucket.

In the keyring project, the plugins specify a priority (with some coded heuristics about the meaning) and some of them use the priority of another to specify a relative priority. That approach has its downsides, but it does provide a straightforward way to order the plugins.
I don't feel strongly about it. Happy to proceed with whatever approach Anderson prefers.

Hi @jaraco, after giving some thought to this problem, I decided to modify the PR in the following ways:

Renamed the entry-point from setuptools.sub_commands.build to setuptools.build_steps after your comment.

Added a priority customization. This attribute is used to sort the entry-points before they are added to the list of build-subcommands. Defaults to 0.
If entry-points have the same name, the one with higher priority wins.

Removed the confusing after and before attributes.

Added an insert_build_step customization.
By default, custom build steps are appended to the build.sub_commands list.
Higher priority commands will be appended "first".
This should cover the majority of use cases.
If this is not enough for a specific use case, developers can implement the insert_build_step class/static method to fine tune in which order the custom build step will be executed.

Please feel free to bike-shed this names, I am not very sure myself.


Fileclass Settings won't save after recent insider update

It seems that after the recent insider update, Fileclass Settings are not responsive. I am experiencing issues with the Button Icon (which will not change and after saving goes back to default) and parent file classes which I can select but are not acknowledged. In the Release Note of the 1.5.5 Obsidian insider update, I read: 'File Explorer and Tags pane both received a small code refactor. This might affect some plugins that rely on the internal JS structure'. Could that cause the problem?

Could you test with 0.8.0-beta-4 please? Do you still encounter this issue?
https://github.com/mdelobelle/metadatamenu/releases/tag/0.8.0-beta-4

I faced the same problem. Icons assigned to properties in a class are not displayed. Everything works by default. I also checked it with the beta version. All the same.

I can't reproduce this issue.
Could you paste some screen shots of what is happening? (and any error console that may be printed when you try to change an icon?)

Setting Class properties

view window setting

or from context menu

This is not a bug. Unlike the icon for the fileclass button, the icons for field actions (that you have underlined in your screenshots) have always been fixed and cannot be customized.

Thank you! This helped clarify the point. But why is there no information about this in the interface itself? (for example - Icon settings for the mobile version)

fair point. I'll improve that in the next version

CHanged in 0.8.1


[ZZO] Add   in post date/reading time

A coffee icon in a separate row looks somehow odd on mobile.
Along the way I also increased spacing around · in the post view and unified it. Feel free to remove the second change if you don't like it.

Ok, I'll check it on mobile

A coffee icon in a separate row looks somehow odd on mobile.

I meant before my change. After   it is moved to the next row with the text which looks nicer. There is everything ok with the icon itself :).


New measure entry error

When clicking on the exp bar it doesn't add the new entry even with the in game tool tip showing

In menu there is option Open Data Folder. Can you show me content of runtime.log?

runtime.log
resolution is 2560x1440 and it is the same with game and app

I saw screenshot is in 2544x1436 resolution.
Is this screenshot was cut?

Yes just cut out second monitor

Ok. Unfull resolution affects scanning region and it cuts out part of text from tooltip.
Add to settings.json in Data Folder this section under "layouts":
"close_to_2560x1440": {
    "info_board_x": 734,
    "info_board_bottom": 1409,
    "control_region_x": 734,
    "control_region_y": 1412,
    "control_region_width": 1092,
    "control_region_height": 28,
    "in_game_exp_bar_x": 734,
    "in_game_exp_bar_y": 1422,
    "in_game_exp_bar_width": 1092,
    "in_game_exp_bar_height": 8,
    "in_game_exp_tooltip_x_offset": 64,
    "in_game_exp_tooltip_y": 1339,
    "in_game_exp_tooltip_width": 603,
    "in_game_exp_tooltip_height": 101
},

Then From menu Layout > close_to_2560x1440.
And uncheck detect on start.
Should work.

If this doesn't work. Try do second screenshot when you point cursor exactly at 50% marker in exp bar.
It looks like, something is wrong with detection box for text.

Try with this in "layouts" section in settings.
"half_x_offset_2560x1440": {
    "info_board_x": 734,
    "info_board_bottom": 1409,
    "control_region_x": 734,
    "control_region_y": 1412,
    "control_region_width": 1092,
    "control_region_height": 28,
    "in_game_exp_bar_x": 734,
    "in_game_exp_bar_y": 1422,
    "in_game_exp_bar_width": 1092,
    "in_game_exp_bar_height": 8,
    "in_game_exp_tooltip_x_offset": 32,
    "in_game_exp_tooltip_y": 1343,
    "in_game_exp_tooltip_width": 587,
    "in_game_exp_tooltip_height": 97
},

That worked!
Thank you so much. Sorry for how much of a pain that was.

It's fine. For some reason offset between cursor and in-game exp tooltip is half of it's supposed normal length. Hard to tell why. Might be existence of second monitor making game to do this. But this new settings should work for your setup now :)


Create a Json File for welcome channel

Can you make a Json File for welcome channels so it can send message in many server not only in one server.

No, this bot isn’t meant to be multi server.


[zero3] how to get the model reconstructed for saving?

Under zero2 self.model.state_dict() returns a fp16 version of the model, under zero3 it returns some placeholder: with all the weights being just tensor([1.],, so how can we get the trained model out of deepspeed?
This is related to https://github.com/microsoft/DeepSpeed/issues/800 but there under zero2 we at least had the fp16 version save-able, now there is no way at all. This is a total lock-in, unless I'm missing on some API that was added with zero3.
Ideally it'd be awesome if it were to reconstruct it directly on the disk, since that will ensure that there will be enough memory to do so.
To summarize different requests so far - users have 3 different needs:

being able to leave DeepSpeed after training with it. So they should be able to take a DeepSpeed checkpoint and recover the full consolidated fp32 model in a single file
same as (1) but fp16. Surely once can go from (1) to (2) easily, but I think it'd be much faster to get to (2) directly - I could be wrong. If I'm wrong than (1) is enough.
being able to call deepspeed.consolidate_weights() in the rank0 process which would give users full non-partitioned weights back (perhaps with a bool arg of whether they want the fp16 or fp32 version). So now they can just save the model as they do with any other pytorch tools. This would only be practical for small-ish models. The key here is that while this would be somewhat costly they will be able to use their code almost w/o any change if they train in various ways and not just with deepspeed. I think this must happen on cpu, since it's unlikely gpus will have the memory for that. It probably will have to return a copy of the model with the consolidated weights, so that the user can continue training the original model. So probably something along the lines of https://github.com/microsoft/DeepSpeed/pull/882 but in addition the partitioning will need to be removed too. The result of this call will provide users an equivalent of what they have under zero2 at this moment (if it's fp16).

I think all 3 would be more or less the same code, with just different ways of using it - (3) using the existing deepspeed engine and not needing to access the filesystem, (1) and (2) w/o needing an engine and relying exclusively on the filesystem.
Thank you!
@jeffra, @tjruwase

@stas00, do you have repro steps?

If you have a huggingface repo already from its root:
PYTHONPATH=src deepspeed --num_gpus 2 examples/seq2seq/run_translation.py --model_name_or_path t5-small --output_dir /tmp/zero3 --overwrite_output_dir --max_train_samples 10 --max_val_samples 10 --max_source_length 12 --max_target_length 12 --val_max_target_length 12 --do_train --do_eval --num_train_epochs 1 --per_device_train_batch_size 2 --learning_rate 3e-3 --warmup_steps 8 --predict_with_generate --logging_steps 0 --save_steps 2 --eval_steps 1 --group_by_length  --adafactor --dataset_name wmt16 --dataset_config ro-en --source_lang en --target_lang ro --source_prefix "translate English to Romanian: " --deepspeed examples/tests/deepspeed/ds_config_zero3.json

and the config file is: examples/tests/deepspeed/ds_config_zero3.json with contents:
{
    "fp16": {
        "enabled": true,
        "loss_scale": 0,
        "loss_scale_window": 1000,
        "initial_scale_power": 16,
        "hysteresis": 2,
        "min_loss_scale": 1
    },

    "zero_optimization": {
        "stage": 3,
        "cpu_offload": true,
        "cpu_offload_params": true,
        "cpu_offload_use_pin_memory" : true,
        "overlap_comm": true,
        "contiguous_gradients": true,
        "stage3_max_live_parameters": 1e9,
        "stage3_max_reuse_distance": 1e8,
        "stage3_prefetch_bucket_size": 2e5,
        "stage3_param_persitance_threshold": 1e5,
        "reduce_bucket_size": 3e6,
        "prefetch_bucket_size": 3e6,
        "sub_group_size": 1e6
    },

    "optimizer": {
        "type": "AdamW",
        "params": {
            "lr": 3e-5,
            "betas": [0.8, 0.999],
            "eps": 1e-8,
            "weight_decay": 3e-7
        }
    },

    "scheduler": {
        "type": "WarmupLR",
        "params": {
            "warmup_min_lr": 0,
            "warmup_max_lr": 3e-5,
            "warmup_num_steps": 500
        }
    },

    "steps_per_print": 2000,
    "wall_clock_breakdown": false
}

then:
ls -l /tmp/zero3/checkpoint-2/
total 868K
-rw-rw-r-- 1 stas stas 1.4K Mar 18 17:27 config.json
drwxrwxr-x 2 stas stas 4.0K Mar 18 17:27 global_step2/
-rw-rw-r-- 1 stas stas   12 Mar 18 17:27 latest
-rw-rw-r-- 1 stas stas  61K Mar 18 17:27 pytorch_model.bin
-rw-rw-r-- 1 stas stas 1.8K Mar 18 17:27 special_tokens_map.json
-rw-rw-r-- 1 stas stas 774K Mar 18 17:27 spiece.model
-rw-rw-r-- 1 stas stas 1.9K Mar 18 17:27 tokenizer_config.json
-rw-rw-r-- 1 stas stas  346 Mar 18 17:27 trainer_state.json
-rw-rw-r-- 1 stas stas 2.4K Mar 18 17:27 training_args.bin

Some variation of this will eventually be part of DeepSpeed API, but if you need it sooner, here is how you get the consolidated fp16 state_dict:
self is the deepspeed engine object.
def zero3_consolidated_fp16_state_dict(self):
    """

    This function restores a full non-partitioned state_dict with fp16 weights

    similar to nn.Module.state_dict (modelled after _save_to_state_dict), but:

    1. consolidates the weights from different partitions on gpu0
    2. works on one layer at a time to require as little gpu0 memory as possible, by
    moving the already consolidated weights to cpu
    3. takes care to keep the shared params shared

    Returns:
        a consolidated fp16 ``state_dict`` on cpu on rank 0, ``None`` on other ranks

    """

    import deepspeed
    from collections import OrderedDict

    if not self.zero_optimization_partition_weights():
        raise ValueError("this function requires ZeRO-3 mode")

    state_dict = OrderedDict() if torch.distributed.get_rank() == 0 else None
    shared_weights = {}
    def get_layer_state_dict(module, prefix=""):
        # gather one layer at a time to be memory-efficient
        with deepspeed.zero.GatheredParameters(list(module.parameters(recurse=False))):
            if torch.distributed.get_rank() == 0:
                #module._save_to_state_dict(state_dict, prefix, keep_vars=False)
                for name, param in module.named_parameters(recurse=False):
                    if param is None:
                        continue
                    key = prefix + name
                    # for shared weights we want to make sure not unshare them when copying to cpu
                    data_ptr_id = param.storage().data_ptr()
                    if data_ptr_id in shared_weights:
                        # shared weights
                        #print(f"`{key}` is shared with `{shared_weights[data_ptr_id]}`")
                        state_dict[key] = state_dict[ shared_weights[data_ptr_id] ]
                    else:
                        state_dict[key] = param.detach().cpu()
                        shared_weights[data_ptr_id] = key

                # now buffers - not sure if need to take care of potentially shared weights here
                for name, buf in module.named_buffers(recurse=False):
                    if buf is not None and name not in module._non_persistent_buffers_set:
                        state_dict[prefix + name] = buf.detach().cpu()

        for name, child in module.named_children():
            if child is not None:
                get_layer_state_dict(child, prefix + name + ".")

    #see_memory_usage("before get_layer_state_dict", force=True)
    # XXX: not sure about starting prefix? see pretrained load
    get_layer_state_dict(self.module, prefix="")
    #see_memory_usage("after get_layer_state_dict", force=True)

    return state_dict

This will be mostly resolved by:

https://github.com/microsoft/DeepSpeed/pull/893
https://github.com/microsoft/DeepSpeed/pull/892


Update documentations for UG, DG

Did a general update for our UG, DG.
We still need to update and reformat the following according to the feedback provided:

[ ] Front matters
[ ] Organise TOC / Collapsible
[ ] Improved glossary/definitions of terms (eg CLI, GUI)
[ ] Usage of 'YOU' language, promotional value proposition
[ ] Adding UI Screenshots (Before & After running a command)
[ ] Tutorial on how to get started
[ ] Making UG more user-friendly
[ ] Sectioning the UG
[ ] Hyperlinks to improve navigability
[ ] FAQ
[ ] Update use cases in DG

Codecov Report
Base: 57.06% // Head: 55.19% // Decreases project coverage by -1.87% :warning:

Coverage data is based on head (e076b0e) compared to base (47ffdcd).
Patch coverage: 0.00% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff              @@
##             master     #188      +/-   ##
============================================
- Coverage     57.06%   55.19%   -1.88%
  Complexity      635      635
============================================
  Files           116      120       +4
  Lines          2590     2678      +88
  Branches        304      316      +12
============================================
  Hits           1478     1478
- Misses          987     1075      +88
  Partials        125      125

Impacted Files
Coverage Δ

...va/seedu/studmap/logic/commands/AddTagCommand.java
0.00% <0.00%> (ø)

...va/seedu/studmap/logic/commands/DelTagCommand.java
0.00% <0.00%> (ø)

...eedu/studmap/logic/parser/AddTagCommandParser.java
0.00% <0.00%> (ø)

...eedu/studmap/logic/parser/DelTagCommandParser.java
0.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


doesn't work now

Error in response to storage.get: TypeError: Cannot read property '0' of null
at upload_problem (chrome-extension://eomonjnamkjeclchgkdchpabkllmbofp/leetcode-github.js:335:37)
at null.callback (chrome-extension://eomonjnamkjeclchgkdchpabkllmbofp/leetcode-github.js:49:9)
at HTMLDocument. (chrome-extension://eomonjnamkjeclchgkdchpabkllmbofp/leetcode-github.js:29:25)
at fire (chrome-extension://eomonjnamkjeclchgkdchpabkllmbofp/lib/jquery.js:3119:30)
at Object.fireWith as resolveWith
at Function.ready (chrome-extension://eomonjnamkjeclchgkdchpabkllmbofp/lib/jquery.js:3443:13)
at completed (chrome-extension://eomonjnamkjeclchgkdchpabkllmbofp/lib/jquery.js:3474:10)

Same here, can't commit the LC questions and answers to github repo now.

Same error message here, "Add quesiton" and "Add README.md" disappeared and code could not be pushed to Github.

same here

Sorry it's long time since my last update, so it may not match new leetcode UI. I will update ASAP.

Same here :)

same here :)

Same problem here. Can't commit to github.

Github commit function is really useful, it helped me review my codes conveniently. If possible, could you fix that first?

could you teach me how to upgrade this plugin , it's really useful, thanks

@binarylu

@Nicksxs Sure, emailed to youl

Any updating plan for this project? Or this project has been deprecated? @binarylu


⚠ Young Discoverers has degraded performance

In aa45a67, Young Discoverers (https://youngdiscoverers.org.au) experienced degraded performance:

HTTP code: 200
Response time: 5518 ms

Resolved: Young Discoverers performance has improved in 1563b93.


Version 0.1.2 tells me it is 0.1.0 and that there is an update

Hi Erik and thank you for your work!
I would like to report a bug on version 0.1.2. After I unzipped the EXE file and run it, it shows the "Update available" link bottom right. After I click the link and then the link to the GitHub releases page, I do not see newer version then 0.1.2.
Could you please help me with this issue, as I am not sure whether I am using the newest version.
See the screenshot below.

Hey!
My apologizes for the confusion. I was just testing the release process, but have not officially released the app yet.
Nevertheless, the app is ready now. I still got a few small improvements on my list, but since there is already interest I am happy to make a proper first release now. Please bear with me, I am planning to make that release today.

Alright, I made the first official release. Please check it out and let me know if that fixes your issue.
I would also be very interested for your feedback in general and please let me know if you encounter any other issues.

Look perfect now! My 5 MB JSON file with 417 elements was loaded in a fraction of a second. Really great job!

Thank you very much! I am glad you like the app.
I will close this issue now as it is solved. Please let me know if you encounter any other issues.


WIP fix build for gfortran

I have manually changed lib ifcore to lgfortran. This will need to fixed properly for intel and gnu compiler and not hard-coded.
this is a temporary fix.
The order of libs is also incorrect for XIOS_LIBRARY_LIST. xios depends on netcdf so it should come second.
There was an erroneous ::Xios for the constructor when creating xios_handler.
fixes #413

I have generalized the fortran runtime lib (ifcore/lgfortran) to work for ifort or gfortran.


Reload/Refresh Inline Script During Runtime

Reload/Refresh Inline Script During Runtime
We might want to analyze the data dynamically and get the response from the inline analysis directly, just like using ipython during runtime.
Features

[x] Getting error messages from Python and informing users, instead of terminating the whole process.
[x] Load the script and interact dynamically during the code runtime.
[x] Changes maintain throughout the rest of the process
[x] Export the current functions.
[x] Can determine and see what functions will run in the following steps.

Enhancement
~- [ ] Colorful python prompt terminal~
~- [ ] Indent~

[x] Parsing traceback errors
[ ] string or char? change to string.

Working Procedure
When to enter and exit interactive mode

Run user script in try --> stop if it goes wrong
Detect LIBYT_STOP file --> stop if detected

In interactive mode

Each inline function execute results/status:* Inline function execute status:
  * yt_inline() ...... finished!
  * yt_inline_arg() .. failed
    * Traceback message ...

Enter interactive mode >>>:>>> if a == "something":
...     print("run somthing")

TODOs
Problems

[x] Scope problem: originally, every inline function are done in inline script's namespace. When we dynamically load and update these functions, we are in global namespace, but we need to be in module's namespace.

Possible solution:

[x] use inline_script namespace, for every input from user.
[x] use exec and pass in sys.modules["inline_script"].__dict__

[ ] yt.enable_parallelism() might run multiple times, if reloading script does not set correctly.

It's OK to run multiple times, but it's waste of time to do this.
Possible solution:

[ ] probably do some checks to see if this MPI is launched already.
~- [ ] Inline script halt once one rank hits an error.~
~- Possible solution~
~- [X] [Fast fail with mpi4py]~(https://stackoverflow.com/questions/49868333/fail-fast-with-mpi4py)
~- [ ] coroutine~

[x] Indentation, everything must have the same indentation with the inline script.

Possible solution:

[x] if it test each line of code to see they compile or not, then we don't have this problem.

[x] How to tell if it is a valid input?

Possible solution:

[x] Test can these line of code can compile or not.
[ ] IPython?

[ ] Cannot change previous line inputs. (put it to the last to solve)

Possible solution:
~- [ ] ncurses~
~- [ ] Erase the current printed console line and To input up arrow key~

Makefile

[x] Add compile option INTERACTIVE_MODE
~- [ ] Cleaner Makefile.~

yt Finalize

[X] Python objects (freed by Py_Finalize())
[X] Function Status Vector (Don't need to.)

Try, Except, Finally

[x] try: execute inline script
[x] except: only root prints full traceback msg, the other ranks prints no error msg.

[x] store error msg to somewhere else, so that we can print it in yt_interactive_mode.
[X] parse traceback msg, make it more readable for user

[x] finally: sync status?
[x] Use %libyt exit for exit. (temp)

libyt command

Should start with %libyt, like %libyt ...
[x] %libyt exit: exit interactive mode, and enter next iteration of simulation.

Tests

[X] Errors raise by libyt module

[X] C
[X] Python

[X] Errors raise by yt
[ ] Test in cluster

Done!


Convert gradle scripts to kts

Conversion of Groovy code to Kotlin of Gradle files. Work in progress. This description needs updating

We've agreed that this work stream is blocked by external factors. Closing this pull request to reflect that.


"Source Code" button not working as intended

Small issue:
When you hover over the "Source Code" button for the "Melumi" card, there is a small white bar on the far left of it. This does not happen with any of the other buttons.

The button seems to work on both my laptop and phone. On both, all of the buttons look the same, which would be expected as the code should be exactly the same for all of the buttons.


Add manual pagination to the datasets page

Datasets page components/datasets/datasets/ must have manual pagination regulated by URL params pageNumber and pageSize. You can get some inspiration from the traces page or from the dataset page

Hi, Dinmukhamed! Could u please assign this task for me?

@sanxzhar go ahead

@skull8888888 @dinmukhamedm i would like to work on this issue Could u please assign this task for me?

Looks like you've pinged them by editing the message, I am not sure if that notifies. @sanxzhar are you working on this issue?


Replace Moment.js with Day.js

Fixes #3133
Description
For Moment.js is now in legacy/maintenance mode. Consider replacing it with other libraries.
There are several other alternatives. Including luxon, Day.js and date-fns
Why we choose Day.js. There are several considerations:

Small. It only 2.9K in size (without extension).
API is highly compatible with Moment.js. So, it is easy for SDK to convert.
Great support for internationalization.
Rich and comprehensive functions and extensions.

Detailed introduction and comparison：https://github.com/you-dont-need/You-Dont-Need-Momentjs/blob/master/README.md
Affected projects

libraries/adaptive-expressions
libraries/botbuilder-repo-utils
libraries/botbuilder
tools

Pull Request Test Coverage Report for Build 495356117

0 of 0   changed or added relevant lines in 0 files are covered.
64 unchanged lines in 12 files lost coverage.
Overall coverage increased (+0.3%) to 84.638%

Files with Coverage Reduction
New Missed Lines
%

libraries/adaptive-expressions/src/builtinFunctions/getNextViableDate.ts
1
95.18%

libraries/adaptive-expressions/src/builtinFunctions/getPreviousViableTime.ts
1
92.86%

libraries/adaptive-expressions/src/builtinFunctions/startOfDay.ts
1
87.5%

libraries/adaptive-expressions/src/builtinFunctions/startOfHour.ts
1
87.5%

libraries/adaptive-expressions/src/builtinFunctions/startOfMonth.ts
1
87.5%

libraries/botbuilder-lg/src/templateExtensions.ts
1
84.48%

libraries/adaptive-expressions/src/builtinFunctions/convertFromUTC.ts
2
90.48%

libraries/adaptive-expressions/src/builtinFunctions/getNextViableTime.ts
2
90.28%

libraries/adaptive-expressions/src/builtinFunctions/getPreviousViableDate.ts
2
92.77%

libraries/adaptive-expressions/src/functionUtils.internal.ts
5
96.19%

Totals

Change from base Build 486704763:
0.3%

Covered Lines:
17844

Relevant Lines:
20142

💛  - Coveralls

Pull Request Test Coverage Report for Build 495356117

0 of 0   changed or added relevant lines in 0 files are covered.
64 unchanged lines in 12 files lost coverage.
Overall coverage increased (+0.3%) to 84.638%

Files with Coverage Reduction
New Missed Lines
%

libraries/adaptive-expressions/src/builtinFunctions/getNextViableDate.ts
1
95.18%

libraries/adaptive-expressions/src/builtinFunctions/getPreviousViableTime.ts
1
92.86%

libraries/adaptive-expressions/src/builtinFunctions/startOfDay.ts
1
87.5%

libraries/adaptive-expressions/src/builtinFunctions/startOfHour.ts
1
87.5%

libraries/adaptive-expressions/src/builtinFunctions/startOfMonth.ts
1
87.5%

libraries/botbuilder-lg/src/templateExtensions.ts
1
84.48%

libraries/adaptive-expressions/src/builtinFunctions/convertFromUTC.ts
2
90.48%

libraries/adaptive-expressions/src/builtinFunctions/getNextViableTime.ts
2
90.28%

libraries/adaptive-expressions/src/builtinFunctions/getPreviousViableDate.ts
2
92.77%

libraries/adaptive-expressions/src/functionUtils.internal.ts
5
96.19%

Totals

Change from base Build 486704763:
0.3%

Covered Lines:
17844

Relevant Lines:
20142

💛  - Coveralls

This is awesome! What a huge bundle size saving 👍

This is awesome! What a huge bundle size saving 👍


Problems with versions of dependencies when installing

When trying to install graph-app-kit as explained in "Install and import" (in the README.md) with npm install graph-app-kit --registry https://neo.jfrog.io/neo/api/npm/npm, we're getting the following errors:
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none is installed. You must install peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none is installed. You must install peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS><= 15 but none is installed. You must install peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS><= 15 but none is installed. You must install peer dependencies yourself.

The required versions are somehow contradictory to each other.

Hi,
That looks a bit confusing, I agree.
Everything works in React 16 though, so you could ignore those warnings for now.
We'll have a look at this.


Bad indentation after '?'

A newline after the ? operation is not indented; e.g. rust-mode-20171208 formats code like
fn foo() -> Result<()> {
    let bar = a
        .method()?
    .another_method()
        .a;
}

The .another_method() should be on same level like the surrounding lines.

Can't reproduce any more.