Diffusion update breaks train_dreambooth scripts

Describe the bug
Last night I was running the dreambooth fine-tuning scripts on colab fine but today after Diffusion release 0.9.0 landed, the training script breaks.
Nothing in my code has changed except now when I launch the training script on accelerate I see the error below.
Running %pip install -qq git+https://github.com/ShivamShrirao/diffusers seems to install version Diffusion 0.9.0 and all I can think is that this has introduced a breaking change in the training script.
Is there anyway I can pin diffusers version to 0.8.1 for now when installing this github?
Reproduction
Try running the fine-tuning example notebook.
Logs
Traceback (most recent call last):
  File "train_dreambooth.py", line 822, in <module>
    main(args)
  File "train_dreambooth.py", line 458, in main
    revision=args.revision
  File "/usr/local/lib/python3.7/dist-packages/diffusers/pipeline_utils.py", line 680, in from_pretrained
    model = pipeline_class(**init_kwargs)
  File "/usr/local/lib/python3.7/dist-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py", line 143, in __init__
    feature_extractor=feature_extractor,
  File "/usr/local/lib/python3.7/dist-packages/diffusers/pipeline_utils.py", line 149, in register_modules
    library = module.__module__.split(".")[0]
AttributeError: 'list' object has no attribute '__module__'
Traceback (most recent call last):
  File "/usr/local/bin/accelerate", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/dist-packages/accelerate/commands/accelerate_cli.py", line 43, in main
    args.func(args)
  File "/usr/local/lib/python3.7/dist-packages/accelerate/commands/launch.py", line 837, in launch_command
    simple_launcher(args)
  File "/usr/local/lib/python3.7/dist-packages/accelerate/commands/launch.py", line 354, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python3', 'train_dreambooth.py', '--pretrained_model_name_or_path=runwayml/stable-diffusion-v1-5', '--pretrained_vae_name_or_path=stabilityai/sd-vae-ft-mse', '--output_dir=/content/drive/MyDrive/stable_diffusion_weights/@potion', '--revision=fp16', '--with_prior_preservation', '--prior_loss_weight=1.0', '--seed=1337', '--resolution=512', '--train_batch_size=1', '--train_text_encoder', '--mixed_precision=fp16', '--use_8bit_adam', '--gradient_accumulation_steps=1', '--learning_rate=2e-6', '--lr_scheduler=constant', '--lr_warmup_steps=0', '--num_class_images=200', '--sample_batch_size=4', '--max_train_steps=2000', '--save_interval=250', '--save_guidance_scale=20', '--save_infer_steps=250', '--save_sample_prompt=fire @potion with dragon inside bottle, perfectly ornated, intricate details, 3d render vray, uhd, beautiful, trending', '--concepts_list=concepts_list.json', '--not_cache_latents']' returned non-zero exit status 1.

### System Info

diffusers==0.9.0
OS: Google Colab Pro (Linux)
GPU: Tesla T4

After installing the github repo, `!pip show diffusers` returns:

Name: diffusers
Version: 0.9.0.dev0
Summary: Diffusers
Home-page: https://github.com/huggingface/diffusers
Author: The HuggingFace team
Author-email<EMAIL_ADDRESS>License: Apache
Location: /usr/local/lib/python3.7/dist-packages
Requires: filelock, regex, Pillow, requests, importlib-metadata, huggingface-hub, numpy
Required-by:

Same issue here using:https://colab.research.google.com/github/ShivamShrirao/diffusers/blob/main/examples/dreambooth/DreamBooth_Stable_Diffusion.ipynb?authuser=0&pli=1#scrollTo=gW15FjffdTID
All I could do was import my model to a different colab and run it there. I have not tried to train a new one yet but I assume would cause the same issue generating samples after training. Did you find a fix yet?

@rmac85 No because I was in the middle of fine-tuning lots of different models :/
Since it's a fork, it's tracking all the updates directly from diffusers branch. We might need to pin the pip install directly toa commit prior to  v0.9.0 for now. Will update if I find a commit hash that works.

Also ran into this issue where the DreamBooth Stable Diffusion Collab no longer works.
Any ideas of this is something that can be fixed? Following this for updates.
In the meantime trying out the fast-dreamboth collab.
Thanks for all your hard work.

Hey, Sorry I didn't catch that during the update. Will fix it in a bit.

Thanks, pease update here when you do.

Beautiful Thanks ShivamShriaro. Hoping things run smooth. My fiance and I are working on a project and have been in a good creative flow thanks to the collab. Thanks again.

I specified
!wget -q https://github.com/ShivamShrirao/diffusers/raw/main/examples/dreambooth/train_dreambooth.py
!wget -q https://github.com/ShivamShrirao/diffusers/raw/main/scripts/convert_diffusers_to_original_stable_diffusion.py
%pip install -qq git+https://github.com/ShivamShrirao/diffusers@7684518377180879a6e45611820e92c9dec7d4b6
%pip install -q -U --pre triton
%pip install -q accelerate==0.12.0 transformers ftfy bitsandbytes gradio natsort
This is maybe Okay.

I specified
!wget -q https://github.com/ShivamShrirao/diffusers/raw/main/examples/dreambooth/train_dreambooth.py !wget -q https://github.com/ShivamShrirao/diffusers/raw/main/scripts/convert_diffusers_to_original_stable_diffusion.py %pip install -qq git+https://github.com/ShivamShrirao/diffusers@7684518377180879a6e45611820e92c9dec7d4b6 %pip install -q -U --pre triton %pip install -q accelerate==0.12.0 transformers ftfy bitsandbytes gradio natsort
This is maybe Okay.

Where to put that and does it need to replace anything?

@rmac85 try pinning the diffusers install to the commit just prior to v0.9.0dev merge into main:
%pip install -qq git+https://github.com/ShivamShrirao/diffusers@cecdd8bdd1c0ac902483e464f40ebdaa91f3fe13
This is working for me.
You would change this line in the training notebook: https://github.com/ShivamShrirao/diffusers/blob/7dc594706b15f51bcc2a69bf6a5ad17e39959774/examples/dreambooth/DreamBooth_Stable_Diffusion.ipynb
As per Naoyaikeda's comment , it looks like any commit prior to the update should work.

Quick side question - What Sample Method does this collab use?

Okay, thanks so much. I'm sure Shivam will get it fixed for us soon as he said either way. I am an AI addict though so not my patience. LOL Will give it a try.

Should be fixed in 4affee020e85eecff56e002be77270daf4e5cba7.

@MysticSpiderman DDIM by default but u can use others like Euler, euler a, DPM.

Shivam, I am kind of a dummy, do you own the colab notebook or is it just somebody else's based on your code? I'm just wondering if you can provide detailed instruction on how to import the other schedulers in the colab, like exactly what line to put and where. Sorry to be a pest, you must be very busy. Thanks so much for your great work.

In inference I tried:
from diffusers import StableDiffusionPipeline, EuleraScheduler - doesn't work.
I'm not sure the right way to put it.

Okay I got it myself, it's called EulerAncestralDiscreteScheduler
Had to remove clip_sample and set_alpha_to_one


Make githubsource's SecretToken optional

From https://github.com/knative/eventing-contrib/issues/297
Right now SecretToken appears to be required:
https://github.com/knative/eventing-sources/blob/master/pkg/apis/sources/v1alpha1/githubsource_types.go#L66
I think it would be nice if it were made optional, and if not provided then the system just created something to use (like a UUID). This value has no meaning outside of this subscription so I don't need the requirement to force the user to specify it. However, if someone does want to do so they can.

/remove-lifecycle stale

/remove-lifecycle stale

/remove-lifecycle stale

/remove-lifecycle stale


Files: show CumulativeSize for directories

Right now (v2.11.4) ipfs-webui reads Size value from ipfs.files.stat('/path/in/mfs').
Unfortunately, the Size of a directory is 0, which results in missing size for dirs:

I believe in this case makes more sense to  show total size of directory, including the metadata.
So the fix is to detect Type being directory and use CumulativeSize instead.
cc @rafaelramalho19  mind prioritizing this? should be easy win :pray:

I tried multiple ways to debug this, but it appears that in the codebase this is already handled:

Are you sure you're using the latest main branch version?

I tried multiple ways to debug this, but it appears that in the codebase this is already handled:

Are you sure you're using the latest main branch version?

Hm... indeed! It took me a while to reproduce it. Turns out sizes are missing only if... your directory has more than 100 items :trollface:
https://github.com/ipfs-shipyard/ipfs-webui/blob/9ccb2e5d7b27416369eb5c63a27463af49242ab2/src/bundles/files/actions.js#L579
I suspect this limit was introduced a long time ago as a fail-safe when listing huge directories.
While we should have a limit, its too low atm. Perhaps:

short term: open a PR that bumps the number to 1000
long term: refactor to execute files.stat for the first 1000k items, and then fetch sizes on-demand when user scrolls past that point.

@rafaelramalho19 I'm ok with closing this by bumping limit to 1k.
Making it smarter could be a time sink, so the long term fix could be part of redoing Files UI,
which  afaik  @jessicaschilling plans to do that at some point anyway.

Hm... indeed! It took me a while to reproduce it. Turns out sizes are missing only if... your directory has more than 100 items :trollface:
https://github.com/ipfs-shipyard/ipfs-webui/blob/9ccb2e5d7b27416369eb5c63a27463af49242ab2/src/bundles/files/actions.js#L579
I suspect this limit was introduced a long time ago as a fail-safe when listing huge directories.
While we should have a limit, its too low atm. Perhaps:

short term: open a PR that bumps the number to 1000
long term: refactor to execute files.stat for the first 1000k items, and then fetch sizes on-demand when user scrolls past that point.

@rafaelramalho19 I'm ok with closing this by bumping limit to 1k.
Making it smarter could be a time sink, so the long term fix could be part of redoing Files UI,
which  afaik  @jessicaschilling plans to do that at some point anyway.


Add noto sans.

CJK fonts are born to be big as there're 10s of thousands of glyphs.
This is noto sans: https://fonts.google.com/noto/specimen/Noto+Sans+SC?subset=chinese-simplified&noto.script=Hans
It's the default Chinese font for Google's Android so it's most likely the widest used open font.

Fonts is no longer accepting new fonts due to file size concerns. Please download the latest patch for an easy way to add a custom font.


Feature/shoping list

Project features

Follow the layout of the wireframes provided. Personalize the rest of the design including colors, typographies, spacings, etc.

Login page and registration page:

Built with Devise.

Recipes list:

Display a list of recipes created by the logged-in user as in the wireframe.
Lead to recipe details.

Public recipe list:

Display a list of all public recipes ordered by newest as in the wireframe.
Lead to recipe details.
If the user is the owner of the recipe, he's allowed to delete it.

Recipe details:

Display a toggle button that allows for a recipe to be made public or private.
If the recipe is public or the user is the owner of the recipe, should display the recipe details as in the wireframe.
If the user is the owner of the recipe, should lead to the form that allows the user to add new food.

Create a navigation menu that allows users to open all of the pages you created.

Write unit and integration tests

Food list:

Display a list of food added by the logged-in user as in the wireframe (display also quantity of a given food).
Lead to a form that allows users to add new food.

General shopping list view:

Show the list of food that is missing for all recipes of the logged-in user.
Count the total food items and total price of the missing food.

Thank you @brainconnect93.


bug:  when use:  ES5 var RBAC = require('rbac').default;

it's undefined

@reboy Just clashed with the same. Took a look at the export and you can use this:
const RBAC = require("rbac").RBAC;

@reboy @KIvanow even shorter:
const {RBAC} = require('rbac');


Identities management becomes increasingly complex and error-prone; should be refactored for ease of continuing development

TBD

?!


sixel_encoder_prepare_palette: kill use-after-free

The only place sixel_encoder_prepare_palette() is called from is sixel_encoder_encode_frame(). If we have a SIXEL_FAILED result inside the former, we unref the palette, potentially freeing it. We then goto the error path for
sixel_encoder_encode_frame(), where we unref it again, in a classic double-free. Remove the internal unref. As noted,
this is a single call site, so removing the unref can't cause a memory leak.
Closes #27, reported by a4865g ("WuLearn").

@a4865g


force protobuf-java version as 3.21.9

Signed-off-by: Yaliang Wu<EMAIL_ADDRESS>Description
force protobuf-java version as 3.21.9
Issues Resolved
[List any issues this PR will resolve]
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #588 (5a45421) into 2.x (32f0e6b) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##                2.x     #588   +/-   ##
=========================================
  Coverage     79.57%   79.57%
  Complexity      929      929
=========================================
  Files            92       92
  Lines          3540     3540
  Branches        326      326
=========================================
  Hits           2817     2817
  Misses          601      601
  Partials        122      122

Flag
Coverage Δ

ml-commons
79.57% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


It would be ~awesome~ to be able to update the notes and tags on a transaction

I am working on a personal project to sync my Amazon purchase info into the transaction notes for each transaction in Mint that is from Amazon so I can better sort them.  I would love a way to update the notes and labels/tags on a transaction.  Right now, it's just a scraping API, but if I could do updates, THAT would be FANTASTIC :)

Thanks! I certainly welcome any clean PRs to address this, but otherwise isn't in the scope/intentions of this project.


Change Detection Relative Atmosphereic Correction

There are tools in the Change detection toolbox that are pathed to the project and not pathed to the Change detection toolbox on Github.  Tools in the project toolbox should be removed/deleted and a step added to added the change detection toolbox from github.

We already tried three versions of the project where we pulled down tools from Github and this didn't work because of this issue with referencing the paths (even after recent updates). Hence I was advised to package the tools within the project. This traditionally didn't work either until I embedded the scripts within the tools. This seemed to work and other machines could open and load the tools from the project package. However there is still a path somewhere that is pointing back to Github. Have potentially sorted this, will try again today on Natalie's machine.

Tool are all found in the project folder. Not all of them work but at least there is only one set of the.


Sets up Github Workflows to run CI on Ubuntu + Windows

title, using a matrix solution for the OSes and for the target/toolchain

Not sure why that linking is failing at the end.


Added examples for usage in a middleware

The examples are checked in the CI, not built. Should be enough?
Yes, rebased.


fa_icon is undefined for views when running specs

[X] Yes, I have tried all 10 debugging steps outlined in the Contribution Guidelines.
I have been using font-awesome-rails for a while with no issues, however after installing the ammeter gem (helps in writing specs for rails generators, so shouldn't be related to FontAwesome...), I started having issues with the fa_icon being undefined when called from views when running specs only (everything works fine in development and production). I have submitted an issue to them, but I'm starting to wondering if this is either an issue with font-awesome-rails only, or a combination of the two, or possibly an issue rspec-rails and how they include helper methods in the view and helper objects of the respective spec types.
I saw that this gem has had quite a few issue submitted regarding fa_icon being undefined, so I'm hoping that you'll be able to point us in the right direction. A few notes:

I've bundle install'd and restarted the server and such.
As far as I can tell, I don't have any other gems that use FontAwesome.
font-awesome-rails is not in my assets group
Here is the output of the bundle list command:

awesome_print (1.6.1)
coffee-rails (4.1.1)
factory_girl_rails (4.7.0)
font-awesome-rails (<IP_ADDRESS>)
haml-rails (0.9.0)
jquery-rails (4.1.1)
rails (4.2.6)
rails-deprecated_sanitizer (1.0.3)
rails-dom-testing (1.0.7)
rails-erd (1.4.7)
rails-html-sanitizer (1.0.3)
rails_12factor (0.0.3)
rails_serve_static_assets (0.0.5)
rails_stdout_logging (0.0.5)
rspec-rails (3.4.2)
sass (3.4.22)
sass-rails (5.0.4)
sprockets-rails (3.0.4)

I made a copy of the project I'm working on, which you can find here: https://github.com/crawfoal/metanoia-view-helper-issues. The README gives more details about the issues I'm seeing.
The issue I created for the ammeter gem is here: https://github.com/alexrothenberg/ammeter/issues/53

I'm not an rspec advocate (in fact, I'm the complete opposite), and I'm not an advocate of "view specs"; however, it looks like you're fighting a limitation of rspec-rails. I don't know how this relates to ammeter (I didn’t play with that permutation).
I submitted you a PR that helps remedy the test suite of your project to behave as you would expect.
https://github.com/crawfoal/metanoia-view-helper-issues/pull/1

Including FontAwesome::Rails::IconHelper in your ApplicationHelper should never be necessary.
You must declare which helpers you want to include when you run view specs using rspec-rails.  Why? I have no earthly idea, but rspec-rails documents this.


Failed to annotate Object type in JavaScript

It seems that VS Code can't annotate Object type for JavaScript files, while other types like number and string work well.
Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.63.0
OS Version: Windows 10 21H2 (OS build 19044.1526)

Steps to Reproduce:

Create a new JavaScript file.
Write the code below:

/** @type {object} */
let obj;

Check the hover information of this variable which will read: "let obj: any".

TS treats object as any in JS because they are essentially the same for type checking. Add a more specific type if you want specific typings


Improve security for the token flow and add Web API protection

As per @vibronet's comments during BUILD 2016, the following improvements will be made in order to comply with security requirements on Azure AD/OpenID.
This also requires different AccessToken class handling as described in https://github.com/thephpleague/oauth2-client/issues/513.

[x] Always validate the id_token signature

When we obtain new access_token, id_token is provided as well. The signature should be always validated whenever constructing the AccessToken object instead of checking it when calling $provider->getResourceOwner($token);. If invalid, an exception should be returned.
At the moment, the resource owner is obtained from the access_token which is wrong due to the fact, that with the 2.0 Azure AD authentication endpoints, it is not guaranteed that the access_token is still going to be a JWT (as per @vibronet's advice during BUILD 2016).

[x] Always validate following id_token claims

[x] Audience (aud) claim - to verify that the id_token was intended to be given to your app
[x] Not Before (nbf) and Expiration Time (exp) claims - to verify that the id_token has not expired
[x] Issuer (iss) claim - to verify that the token was indeed issued to your app

This also requires the ability to turn off the validation and use a custom one when multitenant (/common/ endpoint) application scenario is used
In case of a single tenant application in use with v2.0 endpoint, it will also allow for easier tenant filtering.

[x] Ability to protect your web API with this library - going to introduce a function like validateAccessToken($token) which is going to return either the parsed token which your app received or throw an exception with the error.
[x] Add documentation and example of protecting your Web API with validateAccessToken($token)
[x] Add support for urn:ietf:params:oauth:grant-type:jwt-bearer grant type (use setGrant to register it), also reflect it in the README.md

Will be added as TheNetworg\OAuth2\Client\Grant\JwtBearer for shorter name and referenced as jwt_bearer in $provider->getAccessToken(...)

[x] Add method $provider->getLogoutUrl($post_logout_redirect_uri) to make logging out more streamlined.

[x] Implement
[x] Add documentation to README.md

[x] Update composer.json to support version 1.4.0 and higher of thephpleague/oauth2-client

The full implementation is going to require quite a lot of testing, because this could potentially break plenty of things if implemented wrong, so it might take a while before it gets released.

Almost forgot, I also modified the https://login.windows.net/common/discovery/keys url to use the v2.0 keys since Firebase JWT was giving me an error for a missing KID value.


fix csv import

csv import confuses abstensions with rejections and does not set indexes of answers, lists and theses properly

data.json must be replaced now


NPM backend not finding updates

Describe the bug
Packages installed using the npm backend do not update even though they are set to version "latest"
I've only discovered this because I use eas-cli which notifies me whenever it's out of date.
To Reproduce

Install a npm package using mise with the version set to "latest"
Check if npm has an update for that package
Try to run mise outdated or mise upgrade
The npm packages updates are not detected and do not get updated

Expected behavior
npm packages to be updated when there are updated available
mise doctor output
version: 2024.8.13 macos-arm64 (2024-08-26)
activated: yes
shims_on_path: yes

build_info:
  Target: aarch64-apple-darwin
  Features: DEFAULT, NATIVE_TLS
  Built: Mon, 26 Aug 2024 17:55:13 +0000
  Rust Version: rustc 1.80.1 (3f5fd8dd4 2024-08-06) (Homebrew)
  Profile: release

shell:
  /opt/homebrew/bin/fish
  fish, version 3.7.1

dirs:
  data: ~/.local/share/mise
  config: ~/.config/mise
  cache: ~/.cache/mise
  state: ~/.local/state/mise
  shims: ~/.local/share/mise/shims

config_files:
  ~/.config/mise/config.toml

backends:
  cargo
  core
  go
  npm
  pipx
  spm
  ubi
  vfox

plugins:
  usage  https://github.com/jdx/mise-usage.git#fe3888a

toolset:
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
  java@openjdk-20
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

env_vars:
  MISE_FISH_AUTO_ACTIVATE=1
  MISE_NOT_FOUND_AUTO_INSTALL=0
  MISE_SHELL=fish

settings:
  activate_aggressive = false
  all_compile = false
  always_keep_download = false
  always_keep_install = false
  asdf = true
  asdf_compat = false
  cargo_binstall = true
  color = true
  disable_default_shorthands = false
  disable_tools = []
  experimental = true
  go_default_packages_file = "~/.default-go-packages"
  go_download_mirror = "https://dl.google.com/go"
  go_repo = "https://github.com/golang/go"
  go_set_gopath = false
  go_set_goroot = true
  go_skip_checksum = false
  http_timeout = 30
  jobs = 4
  legacy_version_file = true
  legacy_version_file_disable_tools = []
  libgit2 = true
  node_compile = false
  not_found_auto_install = false
  paranoid = false
  plugin_autoupdate_last_check_duration = "7d"
  python_default_packages_file = "~/.default-python-packages"
  python_pyenv_repo = "https://github.com/pyenv/pyenv.git"
  raw = false
  trusted_config_paths = []
  quiet = false
  verbose = false
  vfox = false
  yes = false
  ci = false
  debug = false
  trace = false
  log_level = "info"
  python_venv_auto_create = false

  [status]
  missing_tools = "if_other_versions_installed"
  show_env = false
  show_tools = false

No warnings found
No problems found

Additional context
mise config
[tools]
node = "lts"
bun = "latest"
java = "openjdk-20"
zig = "latest"
usage = "latest"
"npm:eas-cli" = "latest"
"cargo:krabby" = "latest"

mise ls
bun           1.1.26      ~/.config/mise/config.toml latest
cargo:krabby  0.1.8       ~/.config/mise/config.toml latest
java          openjdk-20  ~/.config/mise/config.toml openjdk-20
node          18.6.0
node          20.16.0
node          20.17.0     ~/.config/mise/config.toml lts
npm:eas-cli   10.2.2      ~/.config/mise/config.toml latest
usage         0.3.0       ~/.config/mise/config.toml latest
zig           0.13.0      ~/.config/mise/config.toml latest

When I manually install the latest version using mise install<EMAIL_ADDRESS>the output of mise ls says it's outdated
bun           1.1.26             ~/.config/mise/config.toml latest
cargo:krabby  0.1.8              ~/.config/mise/config.toml latest
java          openjdk-20         ~/.config/mise/config.toml openjdk-20
node          18.6.0
node          20.16.0
node          20.17.0            ~/.config/mise/config.toml lts
npm:eas-cli   10.2.2
npm:eas-cli   11.0.0 (outdated)  ~/.config/mise/config.toml latest
usage         0.3.0              ~/.config/mise/config.toml latest
zig           0.13.0             ~/.config/mise/config.toml latest

It might well be that caching plays its part here. From my quick try to reproduce i could resolve this by running a mise cache clear before a mise ls which then showed the outdated indicator properly. Maybe the cache freshness threshold for latest versions should be lowered.


Check organization and role when your opens or see details of an object.

Check organization and role when your opens or see details of an object.

Written by @xqliu
Created at 2016-04-21T23:12:07.000+0800
Link to http://betterlife.atlassian.net/browse/BE-221

Role check is build in, but organisation check is not.

Written by @xqliu
Created at 2016-04-22T00:55:51.720+0800


Add support for defining notes

Hi!
I really love your project!
However, the examples you have, have pretty simple short relationships between components.
In some cases, especially using dynamic diagrams, I really would like to put more long form explanations, like Talks to IDP, cache JWT token in memory/redis for 24h, inject to Header. However, such long text does not fit well to diagram, when attached to relationship. For sure, it can be simplified by adding separate "IDP" system, "cache" component and "header injection" component, but then the diagram becomes a bit too bloated. I want to give more details without going too deep on component level.
Do you have plans to add support for adding notes, like PlantUML, or maybe there are some tricks which I can use right now?
Thanks!

Thanks! You can do something like this (enable tooltips on the diagram viewer to see the detail when hovering the mouse pointer over the relationship).
workspace {

    model {
        a = softwareSystem "A"
        b = softwareSystem "B"

        a -> b "Talks to IDP" {
            properties {
                "Notes" "Cache JWT token in memory/redis for 24h, inject to Header"
            }
        }
    }

    views {
        systemLandscape "key" {
            include *
            autolayout lr
        }
    }

}

I keep meaning to add an extended description field, which will show up better in the tooltip, but that's still on the todo list.

Thanks!
Well dynamic diagram is hard to embed in some cases, like Github Readme, so I would rather find some graphical solution.

Since you plan to add notes to the PlantUML output, I'm going to close this issue.

To be frank I would love having it in structurizr since one of the key features for me is ability to use manual placing instead of auto layout. PlantUML is more a plan B with a lot of downsides.

There are no plans to add a graphical solution I'm afraid. I'd generally recommend using the interactive versions of the diagrams; either via the paid cloud service/on-premises installation, or the free Structurizr Lite Docker container.


Symbol properties are ignored

Symbols can be used as property keys, but they are ignored.
const merge = require('deepmerge');

const mySymbol = Symbol('mySymbol');

const x = { value: 42, [mySymbol]: 'hello' };

console.log(x);
// { value: 42, [Symbol(mySymbol)]: 'hello' }

const y = { other: 33 };
const z = merge(x, y);

console.log(z);
// { value: 42, other: 33 }
// expected: { value: 42, [Symbol(mySymbol)]: 'hello', other: 33 }

Makes sense.  We should be calling Object.getOwnPropertySymbols) as well as Object.keys.
I would welcome a pull request with a failing test!

The solution should check that getOwnPropertySymbols exists before calling it, for backwards compatibility with ancient environments.

Closed by #151.  Published as 3.3.0


Run tests for mysql 5.x without a github ubuntu-18.04 image

ubuntu-18.04 image is deprecated already. We use this image for mysql 5.7 image.

We should use another way to run old MySQL versions.

closed by 779

Unfortunately, this it not be done with #779, because 20.04 and 22.04 images use only mysql8.0.32: 20.04 and 22.04.


[R] The amount of "USD EQUIV" is not showing on role-assignment

Expected: Show the amount B1-7 on the form
Actual: No data is shown

Where exactly does it not show?

Sorry, my comment got lost somewhere, check the role-assignment for "ROLECAP TEST", there is not USD EQUIV in the header.

OK it was not on role assignment but on the role assignment proposals :)


peedDependacy change to allow bootstrap version 5.2.0-beta1

I'm not sure if possible, but if you could modify it so npm will allow me to add bs-toaster with bootstra 5.2.0-beta1 installed this would be great.

I need to test it and if it works I will make a release. :)

should be no problem anymore since 5.2.0 is released


go_1_22: 1.22.8 -> 1.22.9

Changelog: https://go.dev/doc/devel/release#go1.22

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 354123

x86_64-linux

:white_check_mark: 31 packages built:

acme-dns
ali
baidupcs-go
bettercap
bitmagnet
devdash
gitmux
go_1_22
grafana-agent
hof
honeytrap
ivpn
ivpn-service
juicefs
nomad (nomad_1_8)
nomad_1_7
peroxide
photofield
pingu
prometheus-dnsmasq-exporter
sensu-go-agent
sensu-go-backend
sensu-go-cli
teleport (teleport_16)
teleport.client (teleport_16.client)
teleport_15
teleport_15.client
tempo
warp-plus
wireguard-go
zincsearch


Fish - Seasonal

Bugs already has 4 seasons available as options.  Same thing should be done for Fish.  Will require sorting JSON though.

Completed


Create more items

Requested Items:

[X] Trap
[ ] Coins

Feel free to add more @Monkey-bored .

Items were moved to separate issues.


[ISSUE #1528]modify unit test of producer

What is the purpose of the change
work for this issue.
Brief changelog
modify unit test

@keranbingaa it seems something wrong with this PR, could you continue to follow with it:)

@keranbingaa it seems something wrong with this PR, could you continue to follow with it:)

ok, I will follow it until it solves.

Coverage increased (+0.3%) to 50.645% when pulling b8e1be8a7749bc2546b678b7d66b7ddd78ef320b on keranbingaa:unit-test into 5a8c4ede16628e5afc38b9e1b5298fdbd63c3435 on apache:develop.


Not clear that users should click 'Allow all the time'

When we are asked about allowing location tracking in the permissions dialog (when running android R) the most obvious options are:

While using the app,
Only this time, and
Deny.

Lots of people will think they're doing the right thing by clicking 'While using the app'. In fact, the user has to click a link 'Allow in setting' and then 'Allow all the time' in order to do the right thing.
To Reproduce
Steps to reproduce the behaviour:

Click continue, enter postcode, click continue.
Click 'Enable Permissions' and you get this:

If you choose the top option you get:

Expected behaviour
Before being presented with the 'Enable Permissions' button the user should be told that for the app to work they have to click the Allow in setting link and then click [ ] Allow all the time.
(Ideally show a couple of photos of what these screens the user will be presented with will look like)
Smartphone (please complete the following information):

Device: emulated Pixel XL
OS: Android R

Have tested this on an emulated Pixel XL with Android R.
After clicking the top permission button ("While using the app"), the NHS COVID-19 app realises that it is the wrong permission setting and alerts the user, so no need to worry about users thinking it's fine without allowing in the background :)
Screen recording:

I like that - very hard to ignore or get wrong.
Sent with GitHawk

I'm not an Android app developer (I'm a database man) but this request (from the app) would surely lead the end-user to think the app is going to access their location, when it doesn't need to do that... can this permission request not be removed?

can this permission request not be removed?

Unfortunately not. Android - for whatever reason - requires location permissions in order for the app to have access to Bluetooth LE. The following links have more information:
https://developer.android.com/guide/topics/connectivity/bluetooth#Permissions
https://stackoverflow.com/questions/33045581/location-needs-to-be-enabled-for-bluetooth-low-energy-scanning-on-android-6-0
https://issuetracker.google.com/issues/37065090

Hmm, i wonder if a change / fix for this will be included in Google's next Android O/S update...

You require the location permission to deal with Bluetooth LE, because you could work out a user's location if you know the physical location of the BLE object they have interacted with.
Let's say I build an app for a coffee shop that uses BLE beacons. When a user with the app installed detects the BLE beacon, the app will send a push notification with the coffee menu.
Now whenever the user enters the coffee shop and triggers the beacon, the app now knows that the user is in the coffee shop, and I have acquired their location without ever asking for that permission.
The problem is that Android has no way of knowing if the BLE object is a static beacon that you could know the location of, or if it is a phone that is someone's pocket that doesn't have a fixed location. You could quite easily set up an Android phone that is always plugged in on the coffee shop counter as your 'beacon', and to Android it appears the same as the one moving around in someones pocket.
Therefore, Android requires you to ask for the location permission from the user when you are handling BLE.

I'm pasting this message in every active GitHub issue, so you may receive duplicate notifications.
Today, I'm happy to announce that NHSX has released the full git commit history for the Isle of Wight Beta apps.

Android - https://github.com/nhsx/COVID-19-app-Android-BETA
iOS - https://github.com/nhsx/COVID-19-app-iOS-BETA

As discussed, we have redacted API keys, sensitive domain names, and some of the developers' personal details. I am still waiting on final approval to publish the server-side code.
I would like to personally thank the community for your comments, bug reports, and vulnerability disclosures. They all went into helping the development process.
The beta trial of this app has now ended and we've moved to the next phase of app development. It is our intention to publish the source code of future apps as the binaries are released to the public.
Once again, thank you for being part of this.
Terence Eden
Head of Open Technology - NHSX


How to compare Dates?

I need to compare particular date with my input DateTime property. It can be ==, >, <, >=, =<.
I have not found any sample of date comparison in your demo application.

@pmahit05 Comparing DateTime is the same as regular csharp and yes, uses the standard operators.
Here's a fiddle with a few examples:
https://dotnetfiddle.net/v7GPat

Thanks a lot Alex.
I will try and let you know.
Thanks & Regards,Prasad Mahit
609-775-7639
On Thu, Sep 30, 2021 at 8:26 PM Alex Reich @.***> wrote:

@pmahit05 https://github.com/pmahit05 Comparing DateTime is the same as
regular csharp and yes, uses the standard operators.
Here's a fiddle with a few examples:
https://dotnetfiddle.net/v7GPat
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/RulesEngine/issues/245#issuecomment-931821982,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APEGXNRIV7O4OBVKKIZKF6TUEUE3RANCNFSM5FDD7NLQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


Fix T.type_alias signature in sorbet gem shim

Motivation
It seems this shim wasn't updated when the signature of T.type_alias was changed.
Test plan
No tests.

Btw, I forgot to note that this was resulting in some end-user visible problems like:
$ bundle exec srb rbi hidden-definitions
Running command: hidden-definitions
Generating: sorbet/rbi/hidden-definitions/
Requiring all of your code
Traceback (most recent call last):
	15: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/bin/srb-rbi:237:in `<main>'
	14: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/bin/srb-rbi:224:in `main'
	13: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/bin/srb-rbi:232:in `block in make_step'
	12: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/lib/hidden-definition-finder.rb:38:in `main'
	11: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/lib/hidden-definition-finder.rb:43:in `main'
	10: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/lib/hidden-definition-finder.rb:64:in `require_everything'
	 9: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/lib/require_everything.rb:19:in `require_everything'
	 8: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/lib/require_everything.rb:39:in `load_bundler'
	 7: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/lib/gem_loader.rb:579:in `require_all_gems'
	 6: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/lib/gem_loader.rb:579:in `each'
	 5: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/lib/gem_loader.rb:581:in `block in require_all_gems'
	 4: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/lib/gem_loader.rb:562:in `require_gem'
	 3: from /Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/lib/gem_loader.rb:562:in `require'
	 2: from /Users/ufuk/src/github.com/Shopify/bundix/lib/bundix.rb:9:in `<top (required)>'
	 1: from /Users/ufuk/src/github.com/Shopify/bundix/lib/bundix.rb:23:in `<module:Bundix>'
/Users/ufuk/.gem/ruby/2.6.5/gems/sorbet-0.4.4969/lib/t.rb:28:in `type_alias': wrong number of arguments (given 0, expected 1) (ArgumentError)

after a bundle update sorbet in a repo.


keybase_service_base: use refreshers to bust service and local team caches

Both KBFS and the service have team caches.  Cached info is generally invalidated by lazy messages sent via the server, but these may not arrive in time or at all in some scenarios.  In those cases, the client can specify exactly what information it is trying to validate, and if that info doesn't match what's in the cache, it can force the service to check with the server to get the most up-to-date information.
In particular:

When trying to decrypt a team TLF MD encryted with a particular keygen, we need to check whether that's a known keygen yet.
When trying to verify that a user + verifying key is a valid writer for a team TLF, we need to verify both that the verifying key is currently valid for the user, and that the version of the user that contains that verifying key is a member of the team (using the "eldest seqno" currently associated with the user).  This means we have to plumb the verifying key into the IsTeamWriter call, which is kind of annoying, but it'll make future Merkle checks a bit easier I think.
When trying to verify that the current signed-in user is a valid reader for a team TLF, we need to verify that the current version of the user is currently a reader for the team.  In this case, we don't need to know the eldest seqno, we can just set it to 0 and let the service use the current cached eldest seqno for that user.
We also need to cache-bust based on the expected role of the user, in case the user role gets changed.

Issue: KBFS-2308

Suggested reviewing order:

interfaces.go
keybase_service_base.go
kbpki_client.go
*metadata*.go
Everything else.


Add non-default schema support

This change will support having user permissions in schema's other than public. Puppet will create the schema based on the hash passed in for the users permissions.

Looks good CJ. Thanks for addressing the ACL unless clause.  Merge away.


where is apk file saved

After running buildozer it shows apk is saved in bin folder but no bin folder is present and I can't find the apk file. Where is it saved?

Hello. You can get the path of saved file by using 'filename' output.
https://github.com/ArtemSBulgakov/buildozer-action#outputs

Try to run echo "${{ steps.buildozer.outputs.filename }}"


Move persistent and volatile server state into the log

This PR moves persistent and volatile server state into the log. The reasoning behind this is because the Raft algorithm specifies that term and votedFor should both be written to disk. So, we add a LogDescriptor to the log and write term and election information to that file. In order to ensure the API isn't spread around too much, I decided to simply move all the state for a server into the Log. This doesn't totally feel right, but there's bound to be some awkwardness since only some server state is persisted and some is held in memory only.
I don't intend to merge this pull request immediately, but I want to start some discussion on the design since it feels a bit dirty, but maybe that's all in my head. The new Log methods need more tests, and tests for recovering term and vote should be added to recovery tests as well.

My first reaction is that this info does not belong to the Log's domain. My justification it simply based on idea that Log in its entirety is a unit of replication. The protocol's goal is to keep the Log identical on all its members and therefore it might be cleaner track member specific non-volatile state separately outside of the Log.
Do you see challenges with tracking this state outside?

I definitely agree. I dislike it. I think the log should store only individual mutations and some other mechanism should store server state. It doesn't feel right that volatile server state is stored here.
So, i can scrap this branch and revert to the previous log. In that case, rather than adding server state methods to the log (term, leader, vote) maybe we should create A separate class for storing server state that's also managed by Storage but is not the Log. So, the Log and sever metastore will be separate, but both can still be managed by Storage to use the same directory/StorageLevel configurations. That actually feels a lot nicer to me.

On Nov 3, 2015, at 8:14 PM, Madan Jampani<EMAIL_ADDRESS>wrote:
My first reaction is that this info does not belong to the Log's domain. My justification it simply based on idea that Log in its entirety is a unit of replication. The protocol's goal is to keep the Log identical on all its members and therefore it might be cleaner track member specific non-volatile state separately outside of the Log.
Do you see challenges with tracking this state outside?
—
Reply to this email directly or view it on GitHub.


New component: Fancy select

Will look at @chriswrightdesign's lib for this or just not bother and recommend people use that instead?

Have a look, see if it works for you. I have to test it in JAWS still.

Gonna close this, any custom form things I'd use: http://chriswrightdesign.github.io/customforms/.


MED-1681: add S3 support

Add S3 Support for Binary. Files are now saved to s3 instead of inline base64 strings
Add AWS_S3_BUCKET env variable

NOTE: this PR may introduce breaking changes with the current version of medeo:

currently files are saved as base64 string but this feature is no longer supported with the new provider
There is a PR required in medeo/medeo for updating the file management
A migration tool should be written to update the documents already in production

here is a gist that could be useful for the migration
https://gist.github.com/Chzzs/53feb542fcdbeb92cf35a68748684fb8

:tada: This PR is included in version 1.3.0 :tada:
Your semantic-release bot :package::rocket:


Escaped models are randomly failing at generation

@sebastienlevert we're going to start enforcing fines for issues like these.
What language? What description? Can you consistently reproduce it?...

We can reproduce in Typescript, pretty much on every generation with Graph.
@koros please share some background.

Is that issue for the smoke test in kiota typescript that has been falling for a while now?

After a long investigation on the subject we have a race condition happening which either results:

model is an interface, with static factory, serialization and deserialization functions (expected)
model is a class, with static factory, serialization and deserialization functions (unexpected, it's kind of halfway through the transformation to an interface)

In both cases the code interface declaration writer is being called, in the former after, in the later before the code class declaration writer. We effectively have BOTH the class and the interface in the DOM as the TypeScript transformation from class to interface is not cleanly pulling the classes. Why is that more present on classes that have been escaped, I'm not sure yet.
This is due to multiple factors:

We build the Code Dom in parallels so the order is never guaranteed.
Multiple places in the code rename the interface by simply affecting the code interface name, but not always removing the code class.
When the rename is done, the element entry name in the DOM hasn't been updated.

This issue manifests itself much more often on larger descriptions.
Here is what I have started:

All rename calls to go through the dedicated code class method (so if we have an issue, it's at one place)
Call AddRange systematically during permutations (so conflicts are handled instead of failing silently)
Throwing when a remove is called but nothing was found (same reason)

Although the problem isn't solved yet, we're getting closer.


Fix: getIntDate for within-time-switch

use local variable value and not every time msg.ts

Thank you for the pull request!
I will integrate this into the 2.0 release on which I'm currently working on (see dev branch).


Token Expired

When your token expires, it is not easy to figure out what happened or how to fix it. This probably needs better error messages.

This needs to go to CredentialManager. It's a won't fix for FIM.


Bump verison to 0.11.1

Bumping the version to allow us to start working on updating the web interface by sending PRs.

Could we make this 0.11.1? It's not clear that this is a release that has a coherent set of functionality changes, which is kinda what we've been using for the middle version number.

Ah sure, I was following semantic versioning and regarded "Add --skip-failed-analyses option" (https://github.com/neulab/ExplainaBoard/pull/418) as new functionality, but it makes sense to follow what we've been using!


Added copy colors, player colors and blending to the renderer

big commit!!
also contains TerserPlugin. idk just ignore

Gdrenderw? More like GdrenderPOG


B11009049

Checks

[ ] Your YAML file has required fields as listed in Specs
[ ] All checks have passed yaml-lint passed
[ ] You agree to contribute your work under MIT license

reviewed, closing.
however be aware of extra contents in PR (you're submitting .vscode, which is wrong).


Write a UI

Initially just display the current bear. Fancy it up later.
Try VueJS... maybe.

Added with aa2c8a4. Still need to deploy it though.


Research ResizeObserver-based modifiers

I would like to see... 🙋♀🙋♂
ember-did-resize-modifier relies on element-resize-detector, which uses a custom solution to detecting element resize (it was backed by research).
The addon works well. There has also been a growing development in a modifier that uses native ResizeObserver API. Let's investigate if now is a good time to switch the resize modifier to one that uses native JavaScript API.
Why and how 💬
Modifiers help us develop a composable solution. If I separated concerns for a container query correctly, we should be able to switch the resize modifier without making a breaking change.
Research what these solutions do and don't do well:

element-resize-detector
ember-on-resize-modifier
ember-resize-modifier
ember-resize-observer-modifier

Since a resize modifier is an integral dependency, let's look into their commitment to stability (documentation, testing, and maintenance).
Additional context ➕
I imagine that, in a real-world app, O(10¹) resizes can happen at the same time (not just 1 or 2 resizes). Rather than eyeballing each modifier's performance, see if we can reliably measure the resize performance in testing and do a health check in CI.

TracerBench

Hey, I am currently looking for a way to make container queries happen. Personally, I would just replace ember-did-resize-modifier with ember-on-resize-modifier, which should give you all the benefits of using a resize observer. I have been using ember-on-resize-modifier for a while and it seems to work wonderfully.

Hi, @st-h. Thanks for providing feedback that container queries matter to you and your apps. 🙂
I haven't recently worked on investigating the use of 3 modifiers listed above, but would be glad to try them out again (ember-on-resize-modifier, in particular). Would you like to help out by making a pull request?

Around July 2020, I tried replacing ember-did-resize-modifier with the 3 modifiers. For every one of them, I saw a performance issue in resize (or rerender, I suppose) in the Dashboard page—I believe, due to the D3 component (Widget 2).
At the time, I wasn't sure if I coded how to rerender D3 chart incorrectly, or if I used the 3 modifiers incorrectly, or something else. I wasn't also sure how to measure performance so that I could quantitatively describe the impact of replacing ember-did-resize-modifier.
If you could fork this repo and try replacing ember-did-resize-modifier and ember-on-resize-modifier, I think that would make a good start to solving this issue. 👍🏼

Will you let me know what you think of the suggested path forward? Thanks again for providing your feedback!

Thanks for your response. Actually, I wrote a naive implementation of container queries myself using ember-on-resize-modifier and a few css classes. Yet, it turned out, it was going to be quite difficult to nest multiple instances of it.
My intention was to try to replace ember-did-resize-modifier as soon as I can find a solution that works reliably in our app. I was running into a few scenarios where ember-container-query would get into an infinite loop, constantly swapping out the currently active feature. I think this can be worked around by being careful where to actually place the <ContainerQuery>. Also I found the default styles that are applied to cause some issues in some cases.

I've been trying out using ember-on-resize-modifier and the fork seems to be working fine within our app. Not sure atm why some test are failing. Seems like some of them are testing immutability, which I am not sure why this is an issue now.

Oh, what do you think about making the default unit ems instead of px? Probably this should be configurable, or both should be supported in some way, but I feel often ems are more useful than pxs...

Hi, @st-h. I haven't had time to review your pull request yet, sorry. I will try to find time to do so this weekend.
With regards to em, I think it's good to keep the default unit in pixels. I don't use em myself (rem, I do) so I'm currently unsure what is a particular use-case that you would like to solve with em and container query. Could you explain your use case to me?

Honestly, I am surely not an expert regarding sizing and their units. But, afaik the main thing is that when changing the base font size of the body (which apparently is a thing some users actually do), the whole page will scale if built using em. If one uses px everywhere, usually the font size changes, but everything that is specified using px will not scale (so things that aligned in its default state will no longer align). From what I have been researching and reading within the last years, it seems to be pretty common and recommended practice to mainly use em. I was using px a lot in the past, but so far have been moving things over to em where there is no reason to use px and it's been working out really well.
I have been using my fork for few days for local dev and on our beta frontend, and so far it seems to be working without issue.

Ok, I just came across this https://css-tricks.com/em-based-media-queries-are-based-on/ after running into some issues with my media queries. So, essentially it's not possible to reliable convert em to px within media queries, as I don't know how to differentiate if the font size is set on the body or using the browser settings. So using ems for media queries just seems to introduce more issues than it solves. (I was also under the impression, that the font size set on the body would be applied in any case, as this is the case with ems otherwise)


fix: pass missing props to children when mapping

[NO-JIRA]
Description
picassoMapper was not implemented correctly and it wasn't passing child.props when creating the React Element and therefore, links were not working as they were loosing href since it wasn't being passed down to the Link component
How to test

RichText component should be working as expected and links also have their href and redirecting to the appropriate URL
CI should be 💚

Development checks

[x] Add changeset according to guidelines (if needed)
[x] Read CONTRIBUTING.md and Component API principles
[x] Make sure that additions and changes on design follow Toptal's BASE design, and it's been already discussed with designers at #-base-core
[x] Annotate all props in component with documentation
[x] Create examples for component
[ ] Ensure that deployed demo has expected results and good examples
[x] Ensure the changed/created components have not caused accessibility issues. How to use accessibility plugin in storybook.
[x] Self reviewed
[x] Covered with tests

Breaking change

[ ] codemod is created and showcased in the changeset
[ ] test alpha package of Picasso in StaffPortal

All development checks should be done and set checked to pass the
GitHub Bot: TODOLess action

PR commands

List of available commands:

@toptal-bot run package:alpha-release - Release alpha version
@toptal-anvil ping reviewers - Ping FX team for review

PR Review Guidelines

When to approve? ✅
You are OK with merging this PR and

You have no extra requests.
You have optional requests.

Add nit: to your comment. (ex. nit: I'd rename this variable from makeCircle to getCircle)

When to request changes? ❌
You are not OK with merging this PR because

Something is broken after the changes.
Acceptance criteria is not reached.
Code is dirty.

When to comment (neither ✅ nor ❌)
You want your comments to be addressed before merging this PR in cases like:

There are leftovers like unnecessary logs, comments, etc.
You have an opinionated comment regarding the code that requires a discussion.
You have questions.

How to handle the comments?

An owner of a comment is the only one who can resolve it.
An owner of a comment must resolve it when it's addressed.
A PR owner must reply with ✅ when a comment is addressed.

@toptal-anvil ping reviewers

@alexvcasillas https://picasso.toptal.net/fix-use-rich-text-hook/?path=/story/components-richtext--richtext#html-from-fe-for-live-editing-preview Does not seem to be redirecting to https://toptal.com 🤔

This one seems to be working fine for me. It's true that "HTML from FE for live-editing preview" link seems not to be working and I'll investigate further :)

@toptal-anvil ping reviewers


Fix: resolve possible symlink in "Show File at Commit"

repo_path always returns a "realpath", so we also need to resolve filepath.

The fix depends on #470, so the fix is included in #470 instead.


Request to bump connect requirement for 2.x

As of 2.5.2, the connect requirement is set to 1.8.x. This was fine until connect 1.8.2 introduced a breaking API change for file uploads. I would like the connection version to be set to ~1.8.2 for the 2.x branch, since using express I cannot really control if the installed connect is pre- or post-1.8.2.

connect 2x separates the three parsers into separate middleware so it's easier to mix-and-match but for now you can just delete the multipart one http://senchalabs.github.com/connect/middleware-bodyParser.html

Yes, but I want to use the multipart parser, but the multipart parser is completely different between connect 1.8.1 and 1.8.2, so my application using express can't know which version of the multipart parser it is going to get without checking
require('express/node_modules/connect').version

it just changed to req.body to req.files, which was a security flaw

Yea, I know. But since express may have been installed on a machine a while ago, it may be connect 1.8.0 or 1.8.1, in which case there is no req.files property and the files are just intermingled in req.body. It is really just a problem because the code may be deployed to all kind of machines and even when I have my package.json set to the latest version of express, npm install will not upgrade the inner connect within express if it is already express 2.5.2. npm update may or may not update the inner connect, depending on if that machine's local npm cache state (since they are not connected to the Internet). Basically, there is no way I can make npm install or npm update fail if express's inner connect is not at least 1.8.2.
Anyway, I'll just take your answer on this. I just wanted to let you know why I was requesting the version bump (plus, connect changing some public API in the last dot release is kind of annoying, since it cause all my file uploading to stop working when some machines got connect 1.8.2).

well npm update doesn't work properly, so that's more of an npm/deployment problem. I agree it's very annoying, typically not something I would do but 90% of the users were having tons of difficulties with uploads. Unfortunately sometimes you have to be annoying to progress faster

I agree. Anyway, I just wanted to suggest it. For now I'll just version detect express's connect and access the correct property for the version while I wait for my deployment environment to get updated.

why not just update it? npm update is kinda busted but i often end up just doing rm -fr node_modules && npm install

That's pretty much what I have to do.

yeah that's npm being busted. if there's a new release that is acceptable it should be grabbing it with update but it doesnt


Built-in operators affecting each others types

Shared logic variables causes different operator calls to affect each others' types.

Fixed by 53f11391e552b221c00fc6d140515f469d620547


Question: how to insert oauth2Session.accessToken into the body?

I'm trying to do the following:
POST https://clientcenter.api.bingads.microsoft.com/Api/CustomerManagement/v13/CustomerManagementService.svc
Authorization: oauth2 code
Content-Type: text/xml
SOAPAction: GetUser

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:v13="https://bingads.microsoft.com/Customer/v13">
<soapenv:Header>
   <v13:DeveloperToken>{{devToken}}</v13:DeveloperToken>
   <v13:AuthenticationToken>{{oauth2Session.accessToken}}</v13:AuthenticationToken>
</soapenv:Header>
<soapenv:Body>
   <v13:GetUserRequest>
      <v13:UserId xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>
   </v13:GetUserRequest>
</soapenv:Body>
</soapenv:Envelope>

{{
  console.info(oauth2Session);
}}

oauth2Session is logged OK (if I remove <v13:AuthenticationToken> line), but it gives error on attempt to use it in the mentioned line.
Without this SOAP header I get an auth error from the remote server. Is there any way to fix it?

There is an order problem in my program. Currently the variables in the body are replaced first and then the header variables. But only by replacing the Authorization Header the variable oauth2Session is defined. I did not consider this case that in the body also the token could be needed. I'm adjusting the order because it makes more sense that way.

Thx for your quick replay and Happy New Year!
On the topic: I'm not sure, but if it is not straightforward how to fix it, maybe it's possible to create some ugly workaround?
For instance, I tried to use OPTIONS request to obtain headers first and then pass oauth2Session to the next request.
But I was not able to fill global variable from postscript block of OPTIONS request. Maybe I missed something.

Happy New Year to you too:-)
Yes, there is already a workaround. The approach with the OPTIONS request is good. The reason is probably that I have inserted an explicit variable scoping and you have to explicitly reference variables of other areas (see @ref. This should work (not tested).
# @name token

OPTIONS https://clientcenter.api.bingads.microsoft.com/Api/CustomerManagement/v13/CustomerManagementService.svc
Authorization: oauth2 code
...

{{
  exports.token = oauth2Session.accessToken;
}}
...
###
# @ref token
POST https://clientcenter.api.bingads.microsoft.com/Api/CustomerManagement/v13/CustomerManagementService.svc
Authorization: Bearer {{token}}
Content-Type: text/xml
SOAPAction: GetUser

...

I released a version with v5.9.0 where the original request should work. Have fun trying it out

Works like a charm, thanks!


Regenerate yarn.lock after storybook updates

Description
This PR just regenerates the yarn.lock file after I pushed through a bunch of storybook 6.1.21 dependency updates this morning.  I've seen this happen before where old versions in yarn.lock don't get deleted.  I'm guessing it's some timing thing with one dependency PR taking it out but another that still is based on the old version of the package putting it back in.

Warnings

:warning:
Please add the JIRA issue key to the PR title (e.g. MB-123)

  Generated by :no_entry_sign: dangerJS against 030d60996df4ade94599bbd8e9fd47fefdb47739


Address AppLifecycle comments from Push Notifications PR

https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647720668
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647722057
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647723191
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647723677
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647738627
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647748642
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647749318
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647753265
https://github.com/microsoft/ProjectReunion/pull/823#issuecomment-857077610
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647755955
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647758069
https://github.com/microsoft/ProjectReunion/pull/823#issuecomment-857207980
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647829683
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647831383
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647832079
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647832786
https://github.com/microsoft/ProjectReunion/pull/823#issuecomment-857218420
https://github.com/microsoft/ProjectReunion/pull/823#issuecomment-857224140
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647838155
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r647839502

A few more comments from latest review:
https://github.com/microsoft/ProjectReunion/pull/823#issuecomment-859905716
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r650260518
https://github.com/microsoft/ProjectReunion/pull/823#discussion_r650264034

https://github.com/microsoft/ProjectReunion/pull/823#issuecomment-862515669

https://github.com/microsoft/ProjectReunion/pull/823#discussion_r652992371

working through the list.  in my current topic branch I have all except these changed:
https://github.com/microsoft/WindowsAppSDK/pull/823#discussion_r647838155
https://github.com/microsoft/WindowsAppSDK/pull/823#issuecomment-857224140
https://github.com/microsoft/WindowsAppSDK/pull/823#issuecomment-857218420
https://github.com/microsoft/WindowsAppSDK/pull/823#discussion_r647831383
https://github.com/microsoft/WindowsAppSDK/pull/823#discussion_r647829683
https://github.com/microsoft/WindowsAppSDK/pull/823#discussion_r647755955
https://github.com/microsoft/WindowsAppSDK/pull/823#issuecomment-857077610
https://github.com/microsoft/WindowsAppSDK/pull/823#discussion_r647738627
https://github.com/microsoft/WindowsAppSDK/pull/823#discussion_r647722057
https://github.com/microsoft/WindowsAppSDK/pull/823#discussion_r647720668


Inconsistencies in Bar Chart Heights During Dynamic Data Management

Hello,
When the bar chart data is managed dynamically, inconsistencies occur in the heights of the bars.
I fetch the incoming data from the service filtered according to specific time periods. There are no issues during the initial render, but when re-rendering multiple times, distortions in the bars occur.
I believe the main problem happens during the re-rendering process of the bar chart.
Example:
// Mock data based on filters
const mockData = {
1: {
items: [
{ changePercent: 55, code: 'AAPL' },
{ changePercent: 22, code: 'GOOGL' },
{ changePercent: 33, code: 'AMZN' },
{ changePercent: 77, code: 'MSFT' },
],
},
2: {
items: [
{ changePercent: 100, code: 'AAPL' },
{ changePercent: 55, code: 'GOOGL' },
{ changePercent: 67, code: 'AMZN' },
{ changePercent: 12, code: 'MSFT' },
],
},
3: {
items: [
{ changePercent: 166, code: 'AAPL' },
{ changePercent: 28, code: 'GOOGL' },
{ changePercent: 41, code: 'AMZN' },
{ changePercent: 55, code: 'MSFT' },
],
},
};
const mockFilters = [
{ value: 1, shortName: '1Y' },
{ value: 2, shortName: '3Y' },
{ value: 3, shortName: '5Y' },
];
const InvestmentBar: React.FC = () => {
const [selectedFilter, setSelectedFilter] = useState(mockFilters[0]);
const renderFilterButtons = useMemo(
() =>
mockFilters.map(x => (
<Button
key={x?.value}
onPress={() => setSelectedFilter(x)}
text={x?.shortName}
/>
)),
[]
);
const renderData = useMemo(() => {
return mockData[selectedFilter.value]?.items.map((item, index) => ({
value: item.changePercent,
label: item.code,
frontColor:
mockData[selectedFilter.value].items.length - 1 === index
? colors.secondary500
: colors.primary500,
}));
}, [selectedFilter]);
return (

{renderFilterButtons}

);
};
export default InvestmentBar;

Hi @dogankablan  👋
The easiest way to fix this issue is to use animated charts. You can make the chart animated by passing the isAnimated prop.
The other way (without using an animated chart) is to force re-render the Chart component.


Use C# 11 feature (raw string literals) for C# widget provider page

All is in the title ^^.
Close #4682

@drewbatgit Any news for this PR ?

@Mysterious-Dev  There is a VS bug that causes issues when changing the project to target .NET 7. We don't want to cause frustration for developers just to utilize these new features. There is a bug fix that has been committed, but not yet released. I have scheduled to follow up on this issue in December.
https://developercommunity.visualstudio.com/t/Using-C-project-properties-to-change-Ta/10220321?ref=native&refTime=1697668375573&refUserId=3ecae279-b085-41e5-8176-afdfb2972795

Thanks for your contribution. This change has been made in an internal PR that also provides guidance for updating the default project template to support .NET 8.0, so I'm closing this PR, but your changes have been incorporated into the docs.


Getting array theme values using theme func

The docs show an example of getting at a value from the theme. How can I get at a value if it is part of an array? (space values for example).

Quick question regarding this. I'd love to mainly use the theme getter function. Reading the docs, I was under the impression that it would convert the values in the arrays—e.g. the font size scale— to px, just like the fontSize utility. However, this does not work:
theme.js
------
const fontSizes = [12, 14, 16, 20, 24, 32, 48, 64, 72]
export default { fontSizes }

Component.js // doesn't work
------
export default styled.p`
    font-size: ${theme('fontSizes.1')};
`

If I add px manually (see below), it works. Is that intentional or did I miss something?
Component.js // works
------
export default styled.p`
    font-size: ${theme('fontSizes.1')}px;
`

@gregoralbrecht,
I think that issue has to do with your browser requiring a unit of measure for the font-size property. It needs to know whether you want rem, em, px, etc.
If you updated your array to look like this:
['12px', '14px', '16px', '20px', '24px', '32px', '48px', '64px', '72px']

You could use font-size: ${theme('fontSizes.1')};

@alanbsmith Thanks for the response! I'm aware we need a unit measure here, just trying to figure out how to best use this library. In the documentation the font size scales are provided as:
// typographic scale
const fontSizes = [
  12, 14, 16, 20, 24, 32, 48, 64, 96, 128
]

And about the fontSize utility it also says: "Numbers from 0-8 (or theme.fontSizes.length) are converted to values on the font size scale."
Personally, I would have loved to use the theme function exclusively because I don't end up with a ton of props on each component. So I'm just trying to understand how to properly define my (responsive) scales :)

If you're only wanting to use the theme values in styled-components without props, you can wrap the themeGet and util.px functions for your own use. Something like:
import { themeGet, util } from 'styled-system'
const themePx = (...args) => props => util.px(themeGet(...args)(props))


FEAT-#2491: optimized groupby dictionary aggregation

What do these changes do?

 That was done to easily translate function name in aggregation dictionary to its proper map and reduce functions.
Now, when PandasQueryCompiler.groupby_agg obtains dictionary as a function, it checks that all function exists in the groupby_reduce_functions dictionary, if it is - going to new added PandasQueryCompiler._groupby_dict_reduce method, otherwise continues in groupby_agg.
Added PandasQueryCompiler._groupby_dict_reduce method to properly dispatch function names to its map+reduce implementations.
Refactored data_management/functions/groupby_function.py to reduce the amount of unused code and split map and reduce phase in different methods. Overall logic of GroupbyReduceFunction doesn't changed much
Added partitions enumeration at reduce phase to fix #2512, now we're inserting 'by' columns in case of as_index=False only if partition_idx == 0
Added apply_indices parameter to BaseModinFrame.groupby_reduce method, it allows us to do not touch partitions that do not contains columns to aggregate

Performance
Number of groups: 100
Times in tables: median of 10 runs
Function: 3 reduction function applied to 3 different columns (sum, prod, count)
Since now we're applying 2 functions to partitions instead of 1 (map + reduce), we can see a little regression in dictionary aggregation for small datasets (~200k rows) because we're getting more overhead of partitioning than profit. But when the number of rows becomes bigger (1kk+) we see a significant increase in performance.
 16 cores

(200.000, 256)
len(by) == 1
len(by) == 2
len(by) == 6

pandas
0.008s
0.012s
0.023s

modin #2534
0.14s
0.17s
0.21s

modin master
0.14s
0.15s
0.18s

(1.000.000, 256)
len(by) == 1
len(by) == 2
len(by) == 6

pandas
0.029s
0.057s
0.11s

modin #2534
0.13s
0.17s
0.24s

modin master
0.26s
0.31s
0.41s

(10.000.000, 20)
len(by) == 1
len(by) == 2
len(by) == 6

pandas
0.31s
0.6s
1.18s

modin #2534
0.37s
0.5s
0.62s

modin master
1.29s
1.77s
2.86s

 112 cores

(200.000, 256)
len(by) == 1
len(by) == 2
len(by) == 6

pandas
0.008s
0.012s
0.023s

modin #2534
0.36s
0.5s
0.7s

modin master
0.4s
0.47s
0.63s

(1.000.000, 256)
len(by) == 1
len(by) == 2
len(by) == 6

pandas
0.029s
0.057s
0.11s

modin #2534
0.37s
0.49s
0.69s

modin master
0.57s
0.64s
0.81s

(10.000.000, 20)
len(by) == 1
len(by) == 2
len(by) == 6

pandas
0.31s
0.6s
1.18s

modin #2534
0.41s
0.59s
0.78s

modin master
1.73s
2.06s
3.16s

 Script to measure
import numpy as np
import os
import pandas
from modin.utils import try_cast_to_pandas
from timeit import default_timer as timer

def generate_data_file(
    nrows=200000,
    ncols=256,
    ngroups=100,
    filename=None,
    force=False,
    rand_low=-100,
    rang_high=100,
):
    if filename is None:
        filename = os.path.abspath(
            f"int_dataset-{nrows},{ncols},{rand_low},{rang_high},{ngroups}.csv"
        )

    print("generating", filename)

    if os.path.exists(filename) and not force:
        return filename

    data = {
        f"col{i}":
        # Generates ngroups for groupby in each column
        list(np.random.randint(rand_low, rang_high, ngroups)) * (nrows // ngroups)
        for i in np.arange(ncols)
    }
    print("dict generated!")
    df = pandas.DataFrame(data)
    print("dataframe created!")
    df.to_csv(filename)
    print("csv ready!")
    return filename

def measure_fn(fn, n):
    times = []
    for _ in range(n):
        t1 = timer()
        fn()
        times.append(timer() - t1)
    return np.min(times), np.median(times), times

def measure_time(df, by, n, **kwargs):
    print(f"\n======= Testing {kwargs['operation'][0]} =======")
    # Printing parameters:
    kwargs.update(
        {"Frame shape": df.shape, "len(by)": len(by), "ncores": pd.DEFAULT_NPARTITIONS}
    )
    op = kwargs.pop("operation")[1]

    print("Parameters:")
    for k, v in kwargs.items():
        print(f"\t{k}: {v}")

    as_index = kwargs["as_index"]

    def measure():
        md_res = op(df.groupby(by, as_index=as_index))
        # shape to trigger materialization
        md_res.shape

    min_t, median_t, times = measure_fn(measure, n)

    print(f"Min time: {min_t}s | Median time: {median_t}s")

if __name__ == "__main__":
    import modin.pandas as pd
    from modin.pandas.test.utils import df_equals

    pd.DEFAULT_NPARTITIONS = 16

    nrows = [200_000, 1_000_000]

    by_cols = [
        ["col0"],
        ["col0", "col1"],
        ["col0", "col1", "col2", "col3", "col4", "col5"],
    ]
    operation = [
        ("agg", lambda grp: grp.agg({"col6": "sum", "col7": "prod", "col8": "count"})),
    ]
    as_index = [True]
    ngroups = 100

    for nrows_ in nrows:
        fname = generate_data_file(nrows_, ngroups=ngroups)
        md_df = pd.read_csv(fname)
        for op in operation:
            for by in by_cols:
                for as_ind in as_index:
                    measure_time(
                        md_df,
                        by,
                        operation=op,
                        ngroups=ngroups,
                        as_index=as_ind,
                        n=10,
                    )

ASV runs:
MODIN_CPUS=16 asv continuous src/master HEAD -b TimeGroupBy --launch-method=spawn -a number=5
BENCHMARKS NOT SIGNIFICANTLY CHANGED.

MODIN_CPUS=112 asv continuous src/master HEAD -b TimeGroupBy --launch-method=spawn -a number=5
BENCHMARKS NOT SIGNIFICANTLY CHANGED.

# full log with times: https://pastebin.com/raw/x3z50sfe

@dchigarev TimeGroupBy suite does not contain perf tests with dictionary aggregations, can you add the ones?

@anmyachev added new benchmark suits, also updated PR comment with new ASV results

@YarShev @gshimansky need your reviews


Mint from website

see: http://blog.adnansiddiqi.me/build-your-first-solidity-dapp-with-web3-js-and-metamask/

code of the example can be found here: https://github.com/kadnan/SolidityPythonTutorial/blob/main/contract_demo.html


bug: "ERROR: logging before flag.Parse" in all log messages

All log messages from kube-keepalived-vip are prefixed with "ERROR: logging before flag.Parse".
For example:
ERROR: logging before flag.Parse: I0412 14:14:03.266668       1 keepalived.go:168] reloading keepalived

Most likely it comes from https://github.com/golang/glog/blob/master/glog.go#L680. Can it be fixed in kube-keepalived-vip?
Thanks in advance.

Closing. Please update to 0.27


cherry-pick signle commit 43a5e58dfa: Clean up logic in CdbTryOpenTable.

This commit removes the function UpgradeRelLockAndReuseRelIfNecessary and clean up the implementation of CdbTryOpenTable. Now the logic is:

open the relation using some lockmode
if need correct lockmode (AO|AOCO with GDD enabled), then close and reopen it using upgraded lockmode
if need lockUpgraded information, deduce this using actually mode with request mode.

Also we remove the post-check for AO table not upgrade, because under the new implementation that's impossible to happen and NOTE Greenplum does not support change table's storage kind yet (Even support some day, there will not be any problem).
The other place that uses UpgradeRelLockAndReuseRelIfNecessary is gpdbwrappers.cpp:GPDBLockRelationOid. Before this commit, it passes nullptr to UpgradeRelLockAndReuseRelIfNecessary, thus will open, lock, close the relation, and then lock the relation oid again. This commit implement this by just calling LockRelationOid since there is no need to test lock-upgrade here:

Greenplum only need to upgrade lock level for target relation of DELETE | UPDATE statement
Parse-Analyze stage will always handle root partition correctly and store the lockmode in RTE
ORCA does not support DML on partiton table yet, and in future when implementing this, ORCA should use the lockmode in RTE of root so no need to deduce lock-upgrade
Index does not have lock-upgrade issue.

fix #ISSUE_Number

Change logs
Describe your change clearly, including what problem is being solved or what feature is being added.
If it has some breaking backward or forward compatibility, please clary.
Why are the changes needed?
Describe why the changes are necessary.
Does this PR introduce any user-facing change?
If yes, please clarify the previous behavior and the change this PR proposes.
How was this patch tested?
Please detail how the changes were tested, including manual tests and any relevant unit or integration tests.
Contributor's Checklist
Here are some reminders and checklists before/when submitting your pull request, please check them:

[ ] Make sure your Pull Request has a clear title and commit message. You can take git-commit template as a reference.
[ ] Sign the Contributor License Agreement as prompted for your first-time contribution(One-time setup).
[ ] Learn the coding contribution guide, including our code conventions, workflow and more.
[ ] List your communication in the GitHub Issues or Discussions (if has or needed).
[ ] Document changes.
[ ] Add tests for the change
[ ] Pass make installcheck
[ ] Pass make -C src/test installcheck-cbdb-parallel
[ ] Feel free to request cloudberrydb/dev team for review and approval when your PR is ready🥳

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

close as dup of https://github.com/cloudberrydb/cloudberrydb/pull/688/


Query Regarding threshold variable in clarkson_woodruff_transform

Abstract:
There is a threshold variable 0.1 in the clarkson_woodruff_transform demo and in the documentation, which according to my overall understanding should not be there because it has never been used in the calculations as such.
Details:
I found clarkson_woodruff_transform through GSOC18 and started going through the research paper as well as the code present in the scipy.
There was a threshold variable, which confused me because we are setting threshold and sketch_size on our own(it's because the sense in which threshold is mentioned is confusing). Because According to my understanding of the whole algorithm, threshold depends on sketch_size and no relation has been calculated for those.
from scipy import linalg
n_rows, n_columns, sketch_n_rows = (2000, 100, 100)
threshold = 0.1
tmp = np.random.normal(0, 0.1, n_rows*n_columns)
A = np.reshape(tmp, (n_rows, n_columns))
sketch = linalg.clarkson_woodruff_transform(A, sketch_n_rows)
sketch.shape
(100, 100)
normA = linalg.norm(A)
norm_sketch = linalg.norm(sketch)

Can anyone please help clearing this confusion??

also this example will not pass the test case, written for clarkson transform
for that alpha has to be reduced from 0.1 to around 0.01 (somewhat near to that)
if u need that test case, i can share (i didnt share because it was very big)

pinging @jomsdev

thanks for pinging me @ilayn
@kdhingra307 I am not actually at home, I will replay tomorrow :)

@jomsdev thanks
no issues

Hello @kdhingra307 I am glad to see that you are interested in contributing to improve the randomized numerical linear algebra methods in scipy.
If I am not wrong you are talking about the tests for the Clarkson transform (CWT). As you may have seen in the paper, there are two related  terms in the transformation: the size of the sketch and the error.
In a real world application you would have to first calculate the size of the sketch given the error bound that you want to have. However, for testing it, we decided to fix the size of the matrices, and put a threshold that is big enough to don't fail (knowing that our matrices are random.normal with mu=0, sigma=0.01) if the transformation is performed correctly.
# Big dense matrix dimensions
n_matrix_rows = 2000
n_matrix_columns = 100

# Sketch matrix dimensions
n_sketch_rows = 100

# Error threshold
threshold = 0.1

The API does not let you yet get the right size given a desired error, neither the error given a size of the sketch but it is something that would be nice.
Not sure if this is what you were asking for, let me know if I can help you clarifying anything else.

@jomsdev that's what I wanted to know
because it was little misleading initially and in the docs(not in test cases), the threshold variable needs to be changed a little bit (based on my calculations)
It was written in the docs about threshold and sketch size but I didn't find any code to find good sketch size (that's why I put this query)
should I add on to this, because I feel its important to find the right sketch size

@kdhingra307 it would be a good contribution.
I have a proposal for this GSoC for implementing Blendenpik and one of the sub tasks is to implement a method for this.
If you want to learn more about sketching errors, scikit has two implementatios of random projections.
In it's documentation you can find some useful links for the The Johnson-Lindenstrauss lemma
And other interesting contribution could be having two different sketch methods implemented the CWT (adapted to sparse matrices) and the Fast Johnson-Lindenstrauss trasnformation for dense ones. Then our algorithms (low rank, square solvers etc) could take advantage of each algorithm.

@jomsdev Are u mentoring GSoC project for Sketches?? Because I got to know about Clarkson Woodruff transformation from there only.
thanks for scikit-learn link, I will go through those.
I did read the research paper about Johnson-Lindenstrauss transformation but shouldn't we sparsify clarkson woodruf transform first??

I am  not mentoring but applying as student too (I am currently finishing my master in discrete mathematics so this was my last chance for participating to GSoC :D)
As you said, one of the main advantages of CWT is that you could compute the sketch in nnz(A), where nnz is the number of non zeroes entries in the matrix. CWT was my first contribution to Scipy and given that I wasn't sure about how much support sicpy has for sparse matrix I decided to go step by step.
There is no problem in having both methods: CWT and FJLT implemented only for dense matrices and then improve the support of CWT for sparse matrices or do it in inverse order, first have CWT for sparse matrices and then implement the FJLT.

@jomsdev same here, I am in the final year too :P
I think, we should go by implementation in the inverse order, because then for FJLT we don't have to sparsify too. We can then have direct sparsify implementation of FJLT.
plus also I think, if we implement algorithm for dense matrix first, then it would be redo of some of the work later on.
What do you say?

Well, FJLT can be used for dense and CWT for dense and sparse.  This is how it should look like after everything is implemented. From there, I do not see a a huge advantage about having the sparse version first because even without it we could implement Blendenpik and other algorithms on top of it but not taking advantage of sparsity.
But yes, I don't see any problem in having CWT for dense and sparse before FJLT.

Let's go with the sparsity of CWT first and finish this to a good point, then we should go for FJLT.

@kdhingra307 time for writing the proposals, do you still being interested in RNLA? I am glad to help in finding something interesting/useful.
In fact, we may end up doing something complementary :)
Let me know if you need something.

@jomsdev yaa I am, I have studied whole CWT and majority of FJLT in the meantime.
I was not available last week due to a minor surgery.😅😅
We will surely,
Of course, I will start adding stuff now, related to CWT and its sparsity.

I have created a pr with #8579
Also, I will also share my proposal and proposed workflow during the GSoC timeline

@cdipaolo, @jomsdev  just checking, can we close this issue or is there something of interest left here?

@rgommers Yes I believe this issue can be closed since the reference to a threshold in tests has been removed in #9884. Also, we provide some guidance on what an actual threshold should be in the docstring now.
That said I think it would be reasonable looking forward to add methods into scipy.linalg or scipy.sparse.linalg which use sketching under the hood to perform routine numerical linear algebra tasks like solving least squares problems approximately [1] or finding approximate low rank factorizations [1,2]. If/when these methods do get added (which is also worthy of discussion) it will be very important to include correct tolerance/threshold parameters. This could require some relatively extensive empirical investigation to get right, since many upper bounds in the theory don't give correct constant factors (and sometimes even asymptotics) as far as I'm aware. A recent paper by Dobriban and Liu [3] looks like it might be useful here though.

That said I think it would be reasonable looking forward to add methods into scipy.linalg or scipy.sparse.linalg which use sketching under the hood to perform routine numerical linear algebra tasks like solving least squares problems approximately [1] or finding approximate low rank factorizations [1,2].

@cdipaolo yes I think so.

If/when these methods do get added (which is also worthy of discussion) it will be very important to include correct tolerance/threshold parameters. This could require some relatively extensive empirical investigation to get right, since many upper bounds in the theory don't give correct constant factors (and sometimes even asymptotics) as far as I'm aware. A recent paper by Dobriban and Liu [3] looks like it might be useful here though.

What would also be very useful is an overview of what is available. E.g. there's some randomized low-rank approximation methods in scipy.linalg.interpolative, and there's https://ristretto.readthedocs.io/en/latest/ which includes some sketching methods. It's pretty hard for non-experts to figure out what they need here, or the relative performance of the options.

closing since this issue was addressed


Draft cheat sheet for UI developers

Hi @joepavitt,
Since the widget loading is now getting mature, I have tried to digest the documentation.  Although it is very well written, I have not a full understanding yet but mostly due to a severe lack of free time.
So I started drawing to create my personal cheat sheet, to support my old brain ;-)
It is FAR from complete, and it will most probably contain incorrect stuff.  But I need to get started somewhere...
I thought it might be interesting for other developers to share my draft version here, to help them getting started.

If anybody has some constructive input about this draft diagram, that is more than welcome!
Everybody is welcome to start adding stuff with a free hand paint tool on top of this image, and share it here.
I will update my drawing afterwards with the feedback.
Bart

Will digest this fully tomorrow when at my desk, but this is astounding Bart.
The only thing I can point out immediately is that my "state" and "props" have some inconsistencies in their implementation atm, which I'm working on refinements in with some the ui-control work I'm doing atm

Thanks!!  No need to hurry, because I know you are very busy...
I had a other few things which were not clear to me in the documentation:

Why does the dashboard scan the packages for the keyword "node-red-dashboard-2" to find widgets, while the widgets also call group.register.  Isn't the latter one enough?

In the widget example code snippet, it would be help to show how to call/use myMethod and myComputedProp.

In the same example code snippet I see props: ['id', 'props', 'state'],.  The same statement contains two times props, while they refer to something else?  Is that correct?  If so, it would help if the second occurence would be called e.g. 'properties'.

Not sure where the mapstate and messages is located in the frontend part.  Similar to the data tracker and Vuex.

Thanks!!

I was stuck last night with a couple of things on my client side, so I have added some details about that side:

Hopefully this helps me now in determining what I am doing wrong.


Implement addToCart facade/action/effect, etc.

Facade: Enhance configuration commons service with addToCart method
Action: new action for AddToCart
Reducer: remove the link between product and configuration in storage
Effect: new addToCart effect
Adapter: new addToCart API in conifugator-commons.adapter
implementation in occ-configurator-variant.adapter
Before execution of addToCart respect configuration update queue.

Commited in branch feature/GH-5386_add_to_cart


Build(deps): Bump keycloak-js from 16.1.1 to 17.0.0 (#2745)

Pull request opened by the merge tool on behalf of #2745

Merged in cb255b4a8017503a3a0ef6dbe17793148210b01c


[INS-2079] Request collection modals PreRelease tests

Closes INS-2079
Test locally:

Clone this branch
Run npm run bootstrap
Run npm run watch:app in one terminal
Run CI=1 npm run test:dev --prefix  packages/insomnia-smoke-test -- collection-interactions

closing this one for now


Multiple days selections?

Version of flutter_calendar_carousel
Expected behavior
Is there a way to do multiple days selections, I mean  for instance , in the same month select multiple days , but in the next month no?

This wasn't in our requirement. We'll think about it!


Would it be possible to extend this include a MoveSensor?

Pardon my naive question, but would it be difficult to extend this component to include a MoveSensor that would fire any time an element's position changes with respect to its parent?

Sorry, but this functionality has nothing to do with this library.


generateTilingTexture console.log left in Phaser 2.4.6 release JS

I was following Emanule Feronato's character selection tutorial and noticed during my implementation that a message was being logged in Chrome's console from line 22487 in phaser.js (2.4.6):

console.log('generateTilingTexture', texture, frame);

I assume this was supposed to be commented out and just got missed? Testing against 2.4.4's JS doesn't log this message.
Simple example:
<!DOCTYPE html>
<html>
<head>
    <title></title>
	<meta charset="utf-8" />
</head>
<body>
	<script src="lib/phaser.min.js"></script>
	<script>
		var game = new Phaser.Game(400, 300, Phaser.AUTO, 'phaser-example', { preload: preload, create: create });

		function preload() {
			game.load.image('button', 'assets/block.png');
		}

		function create() {
			var scrollingMap = game.add.tileSprite(0, 0, game.width, game.height, 'button');
		}
	</script>
</body>
</html>

This is fixed in the dev branch (and was some time ago). Will be part of 2.4.7 next week.


Fix include

Signed-off-by: Pablo Garrido<EMAIL_ADDRESS>

Codecov Report

:exclamation: No coverage uploaded for pull request base (galactic@266248d). Click here to learn what that means.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             galactic     #221   +/-   ##
===========================================
  Coverage            ?   49.39%
===========================================
  Files               ?       43
  Lines               ?     1660
  Branches            ?      478
===========================================
  Hits                ?      820
  Misses              ?      617
  Partials            ?      223

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 266248d...247cd0c. Read the comment docs.

@mergify backport foxy main