Update docker image for tests

update docker image for tests
add teamcity test reporting
add missing dependency for python module (_lzma)

The idea behind this change is to get faster feedback from CI by:

having a build configuration on teamcity (as opposite to travis/appveyor run tests on single platform as quick as possible)
add tests reporting feature to teamcity build (show number of successful/failed tests on overview screen)
make available docker image with all dependencies to run ixmp without need to install anything on host system (apart docker), e.g. use it as base to run server integration scripts

Hi @zikolach. Given that our current CI already installs and runs tests successfully, I would suggest mapping the Dockerfile to use a much of that already-existing infrastructure. For example, we for sure do not need to build python itself from source - you can instead use miniconda and make sure everything is stable in separate environments. See e.g., the start of our ci installation here

@gidden the idea of changing Dockerfile was to have working base docker image based on debian having up-to-date java and python (+ maybe gams). Nothing more (especially related to testing) as base image is intended to be used for production (e.g. running integration scripts as well). I am not sure if the way to install Python is the best one (just had troubles to install java on top of base official python image, so found this way working) and where Dockerfile for such an image should be located. I think we should discuss it offline/in person as there are other pieces in ci shell scripts (e.g. windows/macos support) which was most probably not intended to be used with docker.

sounds good!
On Mon, Jul 29, 2019 at 10:25 AM Nikolay Kushin<EMAIL_ADDRESS>wrote:

Hi @zikolach https://github.com/zikolach. Given that our current CI
already installs and runs tests successfully, I would suggest mapping the
Dockerfile to use a much of that already-existing infrastructure. For
example, we for sure do not need to build python itself from source - you
can instead use miniconda and make sure everything is stable in separate
environments. See e.g., the start of our ci installation here
https://github.com/iiasa/ixmp/blob/master/.travis.yml#L36
@gidden https://github.com/gidden the idea of changing Dockerfile was
to have working base docker image based on debian having up-to-date
java and python (+ maybe gams). Nothing more (especially related to
testing) as base image is intended to be used for production (e.g. running
integration scripts as well). I am not sure if the way to install Python is
the best one (just had troubles to install java on top of base official
python image, so found this way working) and where Dockerfile for such an
image should be located. I think we should discuss it offline/in person as
there are other pieces in ci shell scripts (e.g. windows/macos support)
which was most probably not intended to be used with docker.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/iiasa/ixmp/pull/176?email_source=notifications&email_token=AAKUAEKV6IPNRVLMDCGWMBTQB2SP5A5CNFSM4IHCAPUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD277GNA#issuecomment-515896116,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAKUAEPFOWBJGM5WDPVAFXTQB2SP5ANCNFSM4IHCAPUA
.

the way to install Python is the best one (just had troubles to install java on top of base official python image, so found this way working)

@gidden I updated Dockerfile to use latest debian (buster) distro. Now python3 is installed from repository (not sources).
I am not sure I fully understand how dependency caching is set up, e.g. https://github.com/iiasa/ixmp/blob/master/ci/env.sh#L3 - seems should work for Travis, but does it work for Appveyor?

hey @zikolach apologies for the delay here. I see that there is a run_tests.sh, but am not sure where/how these tests are being run. Do we need to add a teamcity hook on the repo?

@gidden no, there is a build configuration on teamcity which has build step running following script in container built from Dockerfile:
#!/bin/bash

source /opt/python3/bin/activate

pip install .[tests,ci]

py.test tests

Until this PR is merged I use a workaround (another build step containing exactly same Dockerfile content as this PR).
run_tests.sh is just for convenience to run tests locally (in docker) if one doesn't want to setup local python environment.

Usually webhooks are added from teamcity (there is UI for that), but with latest release either github plugin or something else got broken - so it works now without webhooks just polling once in a while repository for changes.

ok, great. Is it possible to put these files not in the root directory? For example in the ci folder or (even better) ci/teamcity? We should also make specific folders for the other CIs..

Thanks for suggestions @gidden and @khaeru.
I don't fully agree as I treat Dockerfile to be used not only for tests/CI, but as delivery package/option. run_tests.sh is showing how to run tests if you have no python environment set up (but have docker one) - like having sample for py.test tests shell call in README (which is now removed from there).
Nevertheless I updated PR to get it finally merged.

Dockerfile to be used not only for tests/CI, but as delivery package/option

This would be a nice additional option to have. However, if it were to truly be a first-class install option, then it would have to be added to the installation page in the documentation—explaining when a user might want to choose it, other requirements, and then how to install. We would also need to consider whether it made best sense to put efforts into supplying a Docker image for ixmp, or instead maybe message_ix or even the entire MESSAGEix-GLOBIOM model, shipping with GAMS, input data, etc. etc.
Another PR that adds that would be welcome. The Dockerfile could be 'promoted' out of the ci/ directory at the same time.


offline mode?

Hi,
I have been experimenting with the Chord Free version and so far I am enjoying the visualizations. However, I run into an issue with the "Too many requests made to chord free" message. If I am working on a project that utilizes a chord visualization, and am running it and testing it as I write the code for other parts of the project, this message shows up after a while. Now from your response to another user's issue, it seems this is a feature (not a bug) introduced to limit requests to your AWS service that generates the visualizations, and that this blocking mechanism should last for up to one hour. This makes it quite hard to work with the free version since if I want to make any changes to my code and run it, it uses up one of the "free requests" allotted me before the blocking happens. Is it feasible for you to provide an offline version so that I can generate the visualizations locally? Does the pro license allow this? I imagined that once I installed the package, it would work like any other python visualization library (like matplotlib or seaborn) and would run locally and offline (if needed), and that the pro version came with extra options (something I would consider if I found myself using these types of charts often, since I usually like to customize things if I can).
If there is no way to bypass this blocking feature, would you consider reducing the blocking time down from 1 hour? If it was meant to mitigate requests down from the level of hundreds per second, having a blocking period (or better yet, a waiting period instead of returning to the user the "too many requests" message) of say, one second, should easily smooth out the requests. Is AWS really unable to handle the traffic?

Hi @evan-wes !
Happy to see you're enjoying the visualisations :)
Maintaining chord-pro and chord-free ended up being quite onerous as they were two different codebases. To address that I merged them into one, with chord-pro allowing the use of additional customisations and modes (e.g. those here https://datacrayon.com/posts/statistics/data-is-beautiful/chord-pro-features-for-chord-diagrams/), some of which rely on server-side extras.
As you saw in a previous issue somewhere (or on our discord), I discovered some users were using chord-pro to generate chord diagrams as part of their own service/software. This meant one license was being used by hundreds of users - but the real problem was that modest instance (single VPS) was unable to cope with all the requests. The chord-pro rate limit is 200 an hour, so it's unlikely you will ever hit it when using it for data visualisation.
Saying that - I just rebooted the chord service which resets the limits, so you should be able to try a few more when you get this reply.
If you're interested, chord-pro is a (relatively) small one-time cost. I'm sure if I dumped more money into AWS I could remove the rate limits entirely (especially for free users), but I'm not able to fund any philanthropy just yet!
There are some nice alternatives too, like https://plotly.com/python/v3/filled-chord-diagram/.
All the best


[Bug]: Balance not adding up, too-high Blocktank fees, Blocktank takes many attempts

Describe the bug
A few possibly related bugs:
Part 1 - "It looks like 14,793 Sats have gone missing somehow in setting up lightning. These numbers don't add up. I sent myself (received) 57,840. Then set up lightning. Now I only have 43,047 left. Where is the other 14,793 sats? If they went for fees then it needs to explain that somewhere. "
Part 2 - "Note: It costs $123 to add $2 to my lightning channel."
Part 3 - "It took 9 attempts over 2 days to get the Lightning channel open. It kept throwing an error about outputs exceeding inputs."
Reproduce

Go to '...'
Click on '....'
Scroll down to '....'
See error

Screenshots / Recording

Operating system
Android?
Bitkit version
No response
Log output
No response

Part 2 was fixed here.
Part 3 is being addressed here. Fix incoming.


Performance of downstream tasks could not be reproduced

Hello, thanks for your great work! I'm interseted in the work, but can't get the similar result when reproducing the batch intregration and preturb prediction task by using the tutorial code and pretrained checkpoints. It seems still has some gap with the best result in the preprint. Is there any tricks or matters needing attention during the  downstream tasks?

Hi, thank you for the question. We did intend to make the notebook run smoothly as it is, and there are supposed to be no additional tricks needed after a proper installation. The tutorial notebooks were tested before we updated them. We tried to set random numbers in most places. There is still some uncontrollable randomness brought by different software versions and even CPU or GPU specs. Overall, technically, you should get the same or similar results as shown by running the notebooks.
So, could you point to me which notebook you are running, and do you mind sharing some results you have now if possible? If you are running multiple ones, it could be easier to show one of them first. I would guess it could be some env settings. I'll try to see what is going on and help with reproducing the results in the notebook.

Hi, thank you for the question. We did intend to make the notebook run smoothly as it is, and there are supposed to be no additional tricks needed after a proper installation. The tutorial notebooks were tested before we updated them. We tried to set random numbers in most places. There is still some uncontrollable randomness brought by different software versions and even CPU or GPU specs. Overall, technically, you should get the same or similar results as shown by running the notebooks.
So, could you point to me which notebook you are running, and do you mind sharing some results you have now if possible? If you are running multiple ones, it could be easier to show one of them first. I would guess it could be some env settings. I'll try to see what is going on and help with reproducing the results in the notebook.

Thanks for your reply! The intregration result I produced is Avg_batch 0.984(higher than the preprint) Avg_bio 0.757(lower than the preprint). And I found that the lower loss in test means higher Avg_batch but not higher avg_bio.
I wonder which part of loss is mainly related avg_bio. It seems ECS is related, but  after  removing it the Avg_bio is still near 0.75.

Hi, I see. I think for that experiment in the tutorial, you would normally see avg_bio above 0.81. Please see these two runs I recently reproduced https://wandb.ai/scformer-team/scGPT-public/runs/3lbdx364, https://wandb.ai/scformer-team/scGPT-public/runs/3b7pxf1g . On the other hand, the evaluation metrics for integration can fluctuate a bit since metrics like NMI and ARI are sensitive to recognized clusters. It may occasionally throw out different metric scores. I would suggest (1) having a try with different random seeds (2) as long as the clusters are visually well separated and align well with the cell type annotations. That indicates the workflow works properly.

Thanks for your reply!

I'm having trouble reproducing Tutorial_Integration.ipynb, the only changes I made to the code was disabling flash attention (couldn't get it to install). I've run the nb twice and got the following results:

This is with the scGPT_human pretrained model available for download in the google drive. I might try the script next examples/finetune_integration.py

Hi @wconnell , currently you will need flash attention to load the pretrained model weights correctly. This is because the names of the parameters differ from naive pytorch MHA. We are working on it and will support running with naive pytorch very soon in the coming days. Please have a look at #39 as well, I will mark the todo items there once the updates complete

using flash attention (only code change was reducing the batch size), trouble reproducing
seed=42

seed=41

Hi @wconnell , what is the batch size you currently use? Choosing an appropriate batch size can be important for fine-tuning, I would recommend using the default value as provided if memory allows. On the other, you may also have a look at the UMAP results, and see the comment here as well https://github.com/bowang-lab/scGPT/issues/64#issuecomment-1683413837

I used a slightly smaller batch size, 56 for the first run and 48 for the second run.
Looking back at these runs, the first model in fact achieved the reported avg_bio about halfway through training. 👏

loss (I think val_loss is used to select best model) does not seem to be well correlated with avg_bio performance though

there seems to be a bit of brittleness in a fine tuning scenario, wondering how this can be mitigated 🤔
going to keep benchmarking, hopefully this is helpful!

I think its also quite important to point out the metrics on the PCs of preprocessed data

Hi @wconnell , regarding your last two comments:

I used a slightly smaller batch size, 56 for the first run and 48 for the second run.
Looking back at these runs, the first model in fact achieved the reported avg_bio about halfway through training. 👏

For a self-supervised learning scenario like this, the "vla_loss" indicates how well the model can reconstruct the data. So I think in general it reflects how well the learned cell embeddings are, and thus used this score to select the best model. If you look at the logged umap at 10 and 15 epochs in wandb, I would guess they may look similar for this data, and the umap at 15 epoch may have more subtle structures like subclusters. On the other hand, the reported scores like avg_bio are computed by matching the results with human labels, so even if the subtle structures may actually reveal more biological signals such as subcelltype clusters, it may lead to lower scores.
This is related to the points made in the comment https://github.com/bowang-lab/scGPT/issues/64#issuecomment-1716701492 . I think a proper evaluation for data integration is still a challenge in the field. Therefore, we listed both the scores and all the umaps when doing the comparison in the manuscript. A combined view of both should give a better evaluation. I would strongly recommend looking at the umap as well, and as long as the clusters are visually well separated and align well with the cell type annotations, it should indicate a proper integration result.

It is true this tutorial data is of rather small size and of high sequencing quality. PCs may also work fine since there is small batch effect in the data. Scores on this data by other integration methods such as scvi and harmony are also listed in the online manuscript. In general, the value of integration methods can be better demonstrated on more challenging datasets with complex cell types and batch effects. We did experiments on a diverse range of datasets and you can find the results in the manuscript supplementary figures. We also plan to release a separate repo for generating the results on these other datasets soon

I think a proper evaluation for data integration is still a challenge in the field

I heartily agree, and successful application to downstream tasks is really the best proof IMO

subtle structures may actually reveal more biological signals such as subcelltype clusters

I concur here, and I wonder if there is a good experiment here biasing some fraction of cells in a cell type cluster (with biologically/technical noise) and observing if the model corrects. I expect it would!
To be clear, I think the results are strong and supported by good evidence, thanks for engaging ✅

Thank you and thanks for the great suggestions!


Remove the docker build process

Make sure we don't publish this non-official build

Wrong branch, but mark already caught it!


tell what you look for in a company / position right on top of About

I no longer have an "about" page and my landing page describes who I am and what I want to do. (Still needs plenty but no longer needs what I'm looking for)


Add an option to expose generated swift classes to Obj-C

Hi,
Could it be possible to add an option to generate classes with @objc classes ?
Thanks.

See #194 (and #609)


Add Shibarium L2

Please help in added Shibarium L2 🙏
Go DeFi 🫡
https://github.com/shibaone/static/blob/main/network/mainnet/shibarium/index.json
https://docs.shib.io/docs/shibarium/welcome/welcome-overview


Fixed gutters issue

When gutter is set as fixed, gutters are added as paddings instead of margins.
So when you set your last column as last with @include float-span(4, last); for example last column loses its gutter but occupies whole available space.
I described this issue in detail on stackoverflow:
https://stackoverflow.com/questions/46179004/singularity-grid-fixed-gutters-issue

@dekisha, did you ever find a solution to this problem? Fixed gutters isn't really viable as it is now because the sizing is inconsistent.

@cameronbprince no, I haven't find a solution to my issue. I understand that sizing is inconsistent, but design requirements were to always have fixed gutter of 20px. IMO there shouldn't be an option to use fixed gutters in singularity if it's impossible to align left and right edge with container.
Nevertheless I am using flexbox for layout now, so this is issue is no longer that much relevant to me. But I would really want to know if this is a bug or I am doing something wrong?
Stackoverflow question hasn't been answered as well as you can notice.

Thanks for the bug report and yes it sounds like it’s either a bug with the code or the documentation.
However Singularity is no longer being maintained and we recommend using CSS grid or flexbox instead as they do many of the same things more simply and efficiently.


Samsung Galaxy S2 problem

Error during backup/restore:
java.security.InvalidAlgorithmParameterException:trustAnchors.isEmpty()

Maybe issue with the cacert bundled on your S2 phone? Have you tried copying the JRE certs from a working phone, or re-creating it manually?

I've got exactly the same issue on my SGS2.
Could you pls specify how to 'copy the JRE certs , or re-creating it manually' ?

Btw, don't have the error with the original SMS backup app

same problem with cyanogenmod and htc desire :(


[WIP]Add HMAC authentication for http api.[skipci]

Description:
Add hmac auth support for http component.
Related issue (if applicable): fixes #
Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.github.io#<home-assistant.github.io PR number goes here>
Example entry for configuration.yaml (if applicable):
http:

Checklist:

[ ] The code change is tested and works locally.

If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.github.io

If the code communicates with devices, web services, or third-party tools:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] New dependencies have been added to the REQUIREMENTS variable (example).
[ ] New dependencies are only imported inside functions that use them (example).
[ ] New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[ ] New files were added to .coveragerc.

If the code does not interact with devices:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] Tests have been added to verify that the new code works.

ref:
https://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

We should not merge this in the current state. We should create a generic authentication system that can have different authentication backends.
I suggest you start by opening an RFC in the architecture repo: https://github.com/home-assistant/architecture/issues


How to run the train_ego_pred.py?

I find the train_ego_pred.py use the data from HEVI_dataset/ego_motion. But I don't have the HEVI Dataset. Is there any way to train without HEVI Dataset? @MoonBlvd

@Flawless1202 If you would like to use your own dataset, just write your own dataloader by following the HEVIDataset in lib/utils/fol_dataloader.py.  If you would like to use HEVI, I think Honda is planning to release the dataset to universities and academy in the recent 2 weeks. If you just want the ego motion data from the visual odometry, please find the data I process using ORBSLAM2 for train and validation. Each pickle file is a dictionary corresponds to one car's trajectory, and it has corresponding ego motions.

Thanks for your reply. And I met another problem. I try to use ORB_SLAM2 to generate the ego motion data from the SA Dataset. But the videos have no camera calibration and distortion parameters. So how do you generate the ego motion data with ORB_SLAM2? @MoonBlvd

@Flawless1202 I didn't find intrinsic matrix neither, so I just used [fx fy cx cy] = [938.0666, 843.003, 643.298, 366.1718], which was for HEV-I. This will make the visual odometry not accurate for sure.

@MoonBlvd Thanks for your reply. And these days I try to generate the ego motion with ORB_SLAM2, but the sequences is to short and often initialize failed. How do you to solve the problem? And when I try to generate the tracking result, it is not always continuous. Has this problem you ever met and how do you solve it? Look forwarding to your reply.

@Flawless1202 Yes, the ORB_SLAM2 won't work perfectly with video. I used linear interpolation which you can find in lib/utils/data_prep_utils.py

@MoonBlvd  Thanks. I also try to use linear interpolation and I also find it in lib/utils/data_prep_utils.py. But I am not really sure the data format which you called 'ORB_SLAM_FORMAT'. When I run the orb_slam2, I got the trajectory file format is 'timestamp x y z qx qy qz w', only 8 numbers in every line and can not use your data prepare scripts. What the data format you get from orb_slam2?

I find the train_ego_pred.py use the data from HEVI_dataset/ego_motion. But I don't have the HEVI Dataset. Is there any way to train without HEVI Dataset? @MoonBlvd

I think if you want to try on HEVI dataset, you could change the HEVIEgoDataset loader to load the same processed pkl files like HEVIDataset loader and return ego motion only (neglecting other information).
@MoonBlvd correct me if I am wrong. It should be sufficient right for starting (even with out downloading and processing the dataset from HEVI website) ?

hey @Flawless1202  did you manage to solve this problem?

Hey guys, I'm sorry if I have

Hey guys, I'm sorry if I have closed the issue by mistake.
What I got from ORB_SLAM2 was text files. Each line of the text file was a flattend Tcws matrix. What I did here  was just to load the lines and reshape it to a 4*4 matrix.

@MoonBlvd Thanks a lot. I have solved the problem, but the ORB_SLAM2 works too bad on my dataset. I don't think the odometry is accurate enough for the ego motion prediction task. Do you think the odometry is necessary?

@Flawless1202 could you please explain how you solved this problem. Because right now I am also stuck at the same point that I have the 'timestamp x y z qx qy qz w' format and don't know how to convert it into the format used by @MoonBlvd.

@SateeshKumar21 I think the load_vo function in lib/utils/data_prep_utils.py doesn't work well for the text file generated by ORB_SLAM2. So I use the formula to convert the quaternion to transformation matrix by myself.

@MoonBlvd could you please specifically describe how you run ORB slam. I ran Orb-slam using the mono_TUM( https://github.com/raulmur/ORB_SLAM2/tree/master/Examples/Monocular/mono_tum) executable and I receive the text file of the format 'timestamp x y z qx qy qz w' which is not compatible with the scripts you provide.

@MoonBlvd Hi auther, first of all thank you for your great work!
As asked by others, could you please let us how to get the ORB result?

SLAM.SaveKeyFrameTrajectoryTUM() -> outputs 'timestamp x y z qx qy qz qw', so it seems not. In addition, it output only key frames, not all the frames.
SLAM.SaveTrajectoryTUM(); -> outputs all the frames, but not 4x4 matrix.
Or did you add some codes at the end of System::SaveTrajectoryTUM() like below?

        cv::Mat Tcw_1d = Tcw.reshape(1);
        for (int i=0; i<16; i++)
        {
            f << Tcw_1d.at<float>(i) << " ";
        }
        f << endl;

Hey guys, I'm sorry if I have closed the issue by mistake. What I got from ORB_SLAM2 was text files. Each line of the text file was a flattend Tcws matrix. What I did here was just to load the lines and reshape it to a 4*4 matrix. @SateeshKumar21 @Flawless1202

@kimimasa-tamura-TRI Do you have any updates on this issue? I'm facing the same issue here.


chore: add root package.json with workspaces

This is mainly to install dev deps using npm i instead of installing them separately.

not needed anymore as most packages got moved out of the repo


Lists: deleteSurroundingText tries to delete the whole composition on a list item

Given:

If I try to delete the 1st character with a backspace press in any list item, InterceptInputConnection.deleteSurroundingText(beforeLength: Int, afterLength: Int) will be called by the OS with beforeLength being 1 (the character before the selection) and afterLength being the length of the rest of the word (4 in this case), so effectively the OS is telling us to delete the whole word for some reason.
This is not what should happen and we need to either investigate why this is happening and fix it or replace the current implementation of that method with one that just deletes whatever the current selection is, although this might lead to some other side effects in unknown conditions.

Tested on the latest and this works now, probably fixed during the paragraphs/lists re-work.


Scaler updater check

Codecov Report

Merging #368 (f9799e6) into master (849fa35) will decrease coverage by 0.33%.
The diff coverage is 9.09%.

@@            Coverage Diff             @@
##           master     #368      +/-   ##
==========================================
- Coverage   41.75%   41.41%   -0.34%
==========================================
  Files          32       32
  Lines        4771     4817      +46
==========================================
+ Hits         1992     1995       +3
- Misses       2779     2822      +43

Impacted Files
Coverage Δ

assemblyline_core/scaler/scaler_server.py
29.55% <0.00%> (-0.61%)
:arrow_down:

...mblyline_core/scaler/controllers/kubernetes_ctl.py
12.01% <6.25%> (-0.20%)
:arrow_down:

assemblyline_core/scaler/controllers/docker_ctl.py
12.38% <9.52%> (-0.80%)
:arrow_down:

assemblyline_core/scaler/controllers/interface.py
53.65% <66.66%> (-0.19%)
:arrow_down:

assemblyline_core/dispatching/dispatcher.py
76.60% <0.00%> (+0.13%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 849fa35...f9799e6. Read the comment docs.


ci: rename release-pr.yaml to release-docs.yaml file

rename release-pr.yaml to release-docs.yaml file as it updates docs vs updating actual binary
Closes #113

Codecov Report
Patch and project coverage have no change.

Comparison is base (74ae03e) 35.17% compared to head (e686163) 35.17%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #119   +/-   ##
=======================================
  Coverage   35.17%   35.17%
=======================================
  Files          12       12
  Lines        1143     1143
=======================================
  Hits          402      402
  Misses        720      720
  Partials       21       21

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

@reetasingh thanks for the PR! steps also needs updating so we basically run "make version-docs ..." in the release that was cut.

#113

@sozercan do we want

@reetasingh thanks for the PR! steps also needs updating so we basically run "make version-docs ..." in the release that was cut. (current logic is using vNext tag)

@sozercan  I'm unsure if I have a complete understanding of the requirement. From the analysis of the workflow, it appears that when the workflow is manually triggered (indicated by github.event_name being equal to 'workflow_dispatch'), the NEW_VERSION value is determined based on the release_version input provided. However, when a new version tag is pushed (indicated by github.event_name being equal to 'push'), the current logic uses the vNext tag for NEW_VERSION.
Do we want for the second case (when a new version tag is pushed), to not increment the minor version but rather to use it as it is?
Regenerate response


feature: More highlight groups for popup window

Did you check the docs?

[X] I have read all the lazy.nvim docs

Is your feature request related to a problem? Please describe.
There are a lot of color options using the provided highlight groups, but not all GUI elements are covered.
Describe the solution you'd like
I want to be able to change the color of everything on the GUI.
Describe alternatives you've considered
I don't know how to do this other than with adding hi lines in my colorscheme file. For example,
...
hi LazyButtonActive cterm=NONE ctermfg=white ctermbg=black
hi LazyButton ctermbg=black ctermfg=brown
hi LazyValue ctermbg=lightgray ctermfg=darkgray
    hi link LazyCommit LazyValue
...

Additional context
I have been able to change the colors of nearly everything, except I can't change the colors of these parts

The "already up to date" message is always blue and the "✔[task]" part is always green.

those are the default diagnostics groups. See :h diagnostic-highlights
Please use discussions next time for user questions.


Ensure singleton types always compare identical

This PR ensures the "singleton" types (any, unknown, string, number, boolean, bigint, symbol, void, undefined, never, and object) always compare identical when their type kind matches, regardless of type object identity. This in particular ensures that the undefined used to represent a missing property compares identical to the "regular" undefined.
The PR also ensures that TypeFlags.Boolean is consistently set on any union of the boolean literal values true and false. We weren't setting this flag on type alias declarations for true | false.
Fixes #44388.

Will this also fix https://github.com/microsoft/TypeScript/issues/44453?

@weswigham Except for error messages in one case (for an indexed access type [number][boolean] where we just report an error for boolean instead of true and false separately) we have no compiler behaviors that change based on whether TypeFlags.Boolean is set or not, so adding a test isn't super useful. My main concern was external dependencies such as linters.


Added Examples and Options to document

Added Examples to document (some of them still need verifications)
Added description of options (It is a draft to start, still needs improvement)

Added necessary options to document. Deleted max_soft_clip. Set max_soft_clip default to 0.

Updated options.
Added Reference links. (Please check the links)

Updated examples.

Updated Example4.


Not able to log in into Concourse deployed with AWS CPI

After running bucc up --lite --cpi aws bosh instances which hosts the ConcourseCI is created and it has its public IP address. When I hit the public address it opens the ConcourseCI user interface but once when I try to log in it makes a redirect to the instance private IP and the login attemp fails. How could I resolve this, may you help please?
For the sake of testing I'm doing this exercise with EC2 located in public subnet. Should I do this in private subnet and using load balancer which target group should hold the bosh instance in the private subnet ?

I managed to resolve the issue.


Add [Jose account] #custodian GD7473…VYFXVV

Address: GD7473LSCR4Z4WP3BIR3XLN2OLI4WMTPMB46GGCBC2LYXY2ZQCVYFXVV
Tags: custodian
Title: Jose account
Requested by @jvilla29xlm

I GCNFF44NIYWJCXNX4ZUC3YQI4XLIERCB73HULRNNOCWBNWHABYY2YOJW account holder sent 26,154 xlm to GD7473LSCR4Z4WP3BIR3XLN2OLI4WMTPMB46GGCBC2LYXY2ZQCVYFXVV
How do i get my xlm back

I believe I been scammed from an email about staking my XLM. Please someone help

Any help with this? I believe I been scammed. Here is the email I got see
below. I sent over 26k in XLM to
GD7473LSCR4Z4WP3BIR3XLN2OLI4WMTPMB46GGCBC2LYXY2ZQCVYFXVV
Please see transaction ID
https://stellar.expert/explorer/public/tx/3af7a1e890f0ce3887a99b5f5bb5b57d25bff94b6a058b2ef0aaf890ed8dfd0d
I hope someone can retrieve my funds I should of knew better
On Sun, Dec 18, 2022 at 10:47 AM stellar-expert-deploy <
@.***> wrote:

Address: GD7473LSCR4Z4WP3BIR3XLN2OLI4WMTPMB46GGCBC2LYXY2ZQCVYFXVV
Tags: custodian
Title: Jose account
Requested by @jvilla29xlm https://github.com/jvilla29xlm
I GCNFF44NIYWJCXNX4ZUC3YQI4XLIERCB73HULRNNOCWBNWHABYY2YOJW account holder
sent 26,154 xlm to GD7473LSCR4Z4WP3BIR3XLN2OLI4WMTPMB46GGCBC2LYXY2ZQCVYFXVV
How do i get my xlm back
—
Reply to this email directly, view it on GitHub
https://github.com/stellar-expert/public-directory/issues/310, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/A42Z7P354SD3D7WJKLSBGETWN4WXXANCNFSM6AAAAAATCSW4RE
.
You are receiving this because you were mentioned.Message ID:
@.***>

@jvilla29xlm If you just sent your assets to this account and you don't have the private key then yes you were scammed.
There is no XLM staking system through Stellar.

First thing I also must warn you about is DO NOT GIVE YOUR PRIVATE KEY TO ANYONE, especially someone who contacts you.
When someone creates an account only the creator knows the public and private keys and the wallet is not active until it is funded. When it is funded it will look like the funder is the one who “created” the account. So if I am the first to send XLM to fund the account it will look like I started the account. But only the actual creator actually knows the private key, unless they tell you the private key and then in that case both people know the private key.
That is the way people get scammed into "creating" a "special secure" wallet or using new fraudulent exchanges. Many scammers imitate people/influencers you may know and walk you into the scam.
The scammers create the wallet (knowing the private key) they then convince you to fund "your special secure wallet". When you do it looks like you created it, but you don't even have the private key. Some scammers take the assets right away or try to convince you to "secure" more. Most scammers use a Binance or ChangeNow to off ramp the assets they stole. They only want XLM or USDC since that is what they can sell and get their loot in other exchanges. This stealing and selling off chain is part of the crap that drives the price of XLM down since they just sell at the low price on the books.

@jvilla29xlm  If you do know the private key I would suggest setting up a new excange account with some reputable exchange like Lobstr, StellarX, etc and then use that private key and publick key to connect that wallet and then you can send it backe to yourself.

@jvilla29xlm
Stellar Expert is only a blockchain explorer tool for people to use on the Stellar Blockchain. This is not a Stellar product/company but I do try to give answers if I can. But like I said you can also become a target of a secondary scam contacting you and offering to help if you give then your private key or pay them to help.

@jvilla29xlm please feel free to reopen this issue with new information.


decision: Semantic naming for Cortex Engines?

Goal

We should have more semantic naming format for Cortex engines
e.g. llamacpp-engine instead of cortex.llamacpp

Tasklist

Discussion: https://github.com/janhq/cortex.cpp/discussions/1167
Make changes once team has consensus

IMO, maybe keep it as llamacpp, onnx and tensorrt

@tikikun @nguyenhoangthuan99 Can I check: if we are using "engine names", should it be:

llamacpp (format: GGUF)
onnxruntime (format: ONNX)
tensorrt-llm? (format: TensorRT engines)

Lack of decision causing issues here: https://github.com/janhq/cortex.cpp/issues/1283

@nguyenhoangthuan99 @vansangpfiev adding this as a priority to decide for consistency before the release

To choose the best option for both user experience and development experience, let's analyze the two proposed options and the current state:

Option 1: llama-cpp, onnx-runtime, tensorrt-llm
Option 2: llamacpp, onnxruntime, tensortllm
Current: llama.cpp, ONNXRuntime, TensorRT-LLM

Analysis:

Consistency: Option 2 is the most consistent, using all lowercase and no hyphens. This can be beneficial for both users and developers as it reduces cognitive load and potential for errors in typing or remembering names.

Readability: Option 1 is more readable due to the use of hyphens, which can help in quickly distinguishing between words. However, this advantage is minor.

Familiarity: The current naming uses the official names of the projects (llama.cpp, ONNXRuntime, TensorRT-LLM). This might be more familiar to users who know these projects.

Consistency with current implementation: Option 1 is closer to the current implementation, which might make the transition easier for existing users.

I prefer Option 1 to update our engine naming convention to llama-cpp, onnx-runtime, and tensorrt-llm. This change brings consistency across our engine names while maintaining their familiarity. The new format improves readability and aligns with common practices in the open-source community, enhancing both user experience and code maintainability. This standardized approach will also help us smoothly integrate any future engines or technologies.
❯ cortex-nightly engines list
+---+---------------+-------------------+---------+--------------+
| # | Name          | Supported Formats | Version | Status       |
+---+---------------+-------------------+---------+--------------+
| 1 | onnx-runtime  | ONNX              | 0.0.1   | Incompatible |
+---+---------------+-------------------+---------+--------------+
| 2 | llama-cpp     | GGUF              | 0.0.1   | Ready        |
+---+---------------+-------------------+---------+--------------+
| 3 | tensorrt-llm  | TensorRT Engines  | 0.0.1   | Incompatible |
+---+---------------+-------------------+---------+--------------+

@nguyenhoangthuan99 @vansangpfiev @namchuai I think we should go with how the upstream authors name it, i.e.

onnxruntime
llama-cpp
tensorrt-llm

For the Repos, I recommend we add an *-engine suffix to differentiate us from the base inference backend.

Old name
New name

cortex.tensorrt-llm
tensorrt-llm-engine

cortex.llamacpp
llamacpp-engine

cortex.onnx
onnxruntime-engine

Marking as done! Engines renamed for consistent experience

Renaming Repos task - moving to a discussion for Sprint 22

Reopen this epic, we have not rename upstream engines repo yet


fix: lookupServerUrl before deleteSubscription

r+ @s12f @daleiz ,
as for
  @Test
  @Timeout(20)
  void deleteNonExistSubscription() throws Exception {
    Assertions.assertThrows(
        Throwable.class,
        () -> {
          try {
            hStreamClient.deleteSubscription("emacs");
          } catch (Throwable e) {
            e.printStackTrace();
            logger.info("get error {}", e.toString());
            throw e;
          }
        });
  }

after this patch
fromJust, called at src/HStream/Server/Handler/Subscription.hs:109:32 in hstream-<IP_ADDRESS>-0ac718e5d4db9620244c34d716ef700833dd5d5c6e1feb37809a4b1728910923:HStream.Server.Handler.Subscription

before
io.grpc.StatusException: ABORTED: Subscription is bound to a different node

I had fix the server issue, the message would be okay after image updated.


Upgrading to Assemble 0.6.0

When installing this via npm I managed to miss the "not ready for use" bit. Oops. But now that it's installed, working great my Grunt setup, Assemble 0.6.0 looks useful. Can you recommend a way I could update Assemble to that version before I try something irresponsible?

Upgrade the grunt plugin to work with Assemble v0.6.0? We haven't even begun work on that yet. My recommendation would be to use v0.6.0 directly (without grunt or gulp). It's much more powerful

Cool. In that case I'll just stick with the version of Assemble in the grunt-assemble plugin, as the grunt integration is pretty important for me right now. Thanks for the reply.

as the grunt integration is pretty important for me right now. Thanks for the reply.

yeah, no problem. we do plan to update the plugin. we'll updated this when we do

+1

Waiting for this update. thanks in advance.

+1

+1

Waiting for the grunt plugin as well.

We use grunt for our complex build process and want to use it in the future too. Is a branch available where the integration for assemble 0.6.0 is developed?

no, but @doowb and I have been talking a lot about this lately. we both want to get this implemented be we've been crunched on time lately so we're trying to stay focused on assemble and docs

actually, I just remembered I started working on this a couple weeks ago, I might be able to finish it up quickly. I'll report back when I have a chance to look. feel free to ping me if you don't hear from me for a couple days!

Is that "not ready for use" comment still valid? People seem to be using /assemble/grunt-assemble instead of assemble/assemble. I would like to use assemble with handlebars 3.x, is this the best version to use or install assemble0.6 instead.

@gunnx Start using grunt-assemble. You will get version 4.x of assemble but with handlebars 3.x support.

I'm running into this exact issue. It has been marked as fixed in 0.6.0 so I'm looking forward to grunt-assemble getting updated.

Closing since this isn't really an issue and based on age. If you have handlebars, or grunt-assemble related issues, it's best to discuss those on the grunt-assemble repo or handlebars-helpers, or the repo that is most related.

.. unless it is related to grunt-assemble, in which case we should create a new issue for it


#309 create helper functions

Description
Helper functions to allow taking the average or boundary value of a constant
Fixes #309
Type of change

[x] New feature (non-breaking change which adds functionality)

Key checklist:

[ ] No style issues: $ flake8
[ ] All tests pass: $ python run-tests.py --unit
[ ] The documentation builds: $ cd docs and then $ make clean; make html

You can run all three at once, using $ python run-tests.py --quick.
Further checks:

[ ] Code is commented, particularly in hard-to-understand areas
[ ] Tests added that prove fix is effective or that feature works
[ ] Any dependent changes have been merged and published in downstream modules

Codecov Report

Merging #311 into master will increase coverage by 0.04%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #311      +/-   ##
==========================================
+ Coverage   96.86%   96.91%   +0.04%
==========================================
  Files          53       53
  Lines        3257     3276      +19
==========================================
+ Hits         3155     3175      +20
+ Misses        102      101       -1

Impacted Files
Coverage Δ

pybamm/expression_tree/unary_operators.py
91.45% <100%> (+1.45%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9253377...122b565. Read the comment docs.


Layered Profile Blueprint

This PR is not related to any issue. This is a blueprint to propose layered profiles a simple way to assign profiles to Delivery Services and Servers.

Which Traffic Control components are affected by this PR?

Documentation

What is the best way to verify this PR?
Go through the blueprint and make sure it looks correct.
PR submission checklist

[x] This PR has tests
[x] This PR has documentation
[x] This PR has a CHANGELOG.md entry
[x] This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY (see the Apache Software Foundation's security guidelines for details)

LGTM, I assume that if users still want to use a single monolith profile that will still be supported?  This is important for migration purposes.

One other thing I just thought of: how is backwards compatibility handled for objects with layered Profiles? If I have a server with:
{
    "id": 7,
    "profiles": ["foo", "bar"]
}

visible in APIv5, then if I request GET /api/4.0/servers?id=7 do I see
{
    "id": 7,
    "profile": "foo"
}

or
{
    "id": 7,
    "profile": "bar"
}

? I assume it'd never choose a middle Profile and would always choose consistently, but should we use the "bottom" Profile foo because as the base Profile it's assumed to contain the bulk of the configuration and changed relatively the least? Or should it choose the "top" Profile bar because that's the one that is never overridden, and all of its Parameters definitely apply?
If I update a server changing its Profile through API v4 or earlier when it has been configured with multiple Profiles, does that change only the entry that would be chosen for display in a GET response, or does it replace all Profiles with the single one specified? That is, if I have
{
    "id": 7,
    "profiles": ["foo", "bar"]
}

and do a PUT /api/4.0/servers?id=7 with
{
    "id": 7,
    "profile": "testquest"
}

in the body and later do a GET /api/5.0/servers?id=7, do I see
{
    "id": 7,
    "profiles": ["foo", "testquest"]
}

or
{
    "id": 7,
    "profiles": ["testquest", "bar"]
}

or
{
    "id": 7,
    "profiles": ["testquest"]
}

?

One other thing I just thought of: how is backwards compatibility handled for objects with layered Profiles? If I have a server with:

I put a lot of thought into this in my initial proposal. I came up with some options, but they're really hacky.
Honestly, I think the best solution is probably to return a 4xx code from /servers and /deliveryservices if a user has assigned more than 1 Profile. And to warn users when they do, that this will break old clients. Which is mostly ok in practice, it mostly means people just have to upgrade all ATC components and clients before they start using Layered Profiles.
I don't suggest that lightly. IMO breaking backward compatibility is one of the worst things we can do, in terms of usability, maintaining users, I could go on. But I honestly think the alternative is probably worse here.
See https://cwiki.apache.org/confluence/display/TC/Layered+Profiles for some of my compatibility ideas.

I assume it'd never choose a middle Profile and would always choose consistently, but should we use the "bottom" Profile

We can't return any single real profile. If someone assigned multiple Profiles, those Parameters, in that exact order, are necessary. Anything that looks at a single profile for params will be broken. If you look at that wiki link, most of my ideas are around creating a "fake" profile. But IDs and names make that hard, and you still can't edit it, so POST/PUT still return error codes. Unless you did even more hacks.

What if we returned a null Profile when there are multiple? That's still breaking, but somewhat less so. It means that eg. my script that does toget servers | jq '.response[]|.id still gets me all the server IDs instead of failing. In that case, I'd also vote that a PUT or POST from API versions 4 and earlier do what you'd expect - set the server's Profile to the single specified Profile.
That's harder for Delivery Services, where profile: "null" is already totally legal, which makes it much more of a lie.

What if we returned a null Profile when there are multiple? That's still breaking, but somewhat less so

That makes clients work who don't need the profile/params, but my fear is it's even worse if they do. Especially DSes, which are allowed to not have profiles. But even Servers might not have error checking, that they were guaranteed not to need before.
I feel like forcibly breaking and making people upgrade is much safer here. If we don't, they can operate on that data, and do any number of wrong and broken things, potentially catastrophically breaking their CDN or deleting data. And they won't even know, and it could be a nightmare to track down.
Returning an HTTP error code makes it as easy and obvious as possible to fix, and doesn't potentially irrevocably destroy data, or create an impossible-to-debug scenario.

I'd also vote that a PUT or POST from API versions 4 and earlier do what you'd expect - set the server's Profile to the single specified Profile.

Same issue: if there were multiple, now they just wiped them out, destroying data. I feel like the valid use cases of legitimately wanting to replace multiple profiles with a single are extremely unlikely.
Again, if there are 0 or 1 assigned, letting API <4.0 set it to a different single value is totally fine. But if it has 2+ Profiles, it seems highly unlikely to be correct, and potentially catastrophic. I think the only safe thing to do is return an error code and disallow the entire request or modification.

... if there were multiple, now they just wiped them out, destroying data.

sure, but that's what you'd expect to happen, same as if you did a PUT with "profiles": ["a single profile"].
It's just that these are probably two of the three most-used endpoints in the API. I'm not sure just a warning alert is enough when you're breaking probably nearly every single client of the API in existence already by the time you see it. That should definitely come with a confirmation dialog or something in TP, though I don't think there's an equivalent for raw API calls.

Should the API disallow multiple uses of the same Profile within the list for a single server or Delivery Service (or potentially Origin)?

Yes. Because it doesn't ever make sense. Since Parameters are applied one profile at a time, including the same profile twice will just override any of the same parameters in-between the two, and have the same effect as only having that profile in the later position.

Catching up on the conversation about backwards-compatibility, it seems like using some kind of tool outside of TO to do the profile layering is sounding better and better. Or, since it seems like we never want to stop adding to the monolith, we could just update the profiles API to produce a "layered" profile. For example:
POST /api/4.1/profiles
{
    "name": "my-new-abc-def-profile",
    "type": "ATS_PROFILE",
    "description": "my new layered profile that is composed of the abc and def profiles",
    "cdn": "mycdn",
    "routingDisabled": false,
    "layeredProfiles": [
        "abc",
        "def"
    ]
}

{
    "response": {
        "name": "my-new-abc-def-profile",
        "type": "ATS_PROFILE",
        "id": 123,
        "description": "my new layered profile that is composed of the abc and def profiles",
        "cdn": "mycdn",
        "routingDisabled": false,
        "layeredProfiles": [
            "abc",
            "def",
        ]
        ...
    }
}

This new profile would then just be the same as any other profile except that you can't assign parameters directly to it, you can only assign parameters to its individual layers. So it only breaks backwards compatibility in terms of assigning parameters to profiles when they're layered, rather than breaking our bread and butter servers API. A server would still only have one profile.

it seems like using some kind of tool outside of TO to do the profile layering is sounding better and better

-1, It fundamentally doesn't solve the problem. The goal here is to make operating ATC/TO/TP easier. Right now, our Profiles are an unmaintainable mess, they're so huge, and 95% duplicated, the Parameters are constantly wrong and nobody knows it. Doing some kind of outside "generation" doesn't solve the problem. If anything, it makes it worse, because now operators aren't allowed to actually use TO/TP to modify things, so operating becomes even harder. Or if they do, the generation is out of sync, making it even more dangerous and error-prone.

since it seems like we never want to stop adding to the monolith

I'm a pretty big proponent of getting rid of the monolith. But TO is our data store. The schema we have today is not operationally workable, and that's what Layered Profiles fixes. Generating what amounts to a denormalized blob and shoving that in the database is even more difficult and error-prone to operate.

we could just update the profiles API to produce a "layered" profile. For example:

    "abc",
    "def"
]

I'm not seeing how that solves the backward-compatibiliity issue? That would still break existing clients

> So it only breaks backwards compatibility in terms of assigning parameters

So instead of breaking getting things with profiles, we break changing the profiles or assigning things to them? I'm not seeing how that's a win, they both seem about the same.

> A server would still only have one profile.

Except that we redefined what a "profile" is, and it can now be a "profile" or a "profile of profiles."

Maybe it's just me, but that seems a lot harder to understand and work with. Breaking change aside, the idea that a server/ds has a list of Profiles applied in order seems pretty straightforward. Where the idea that a server/ds has a profile, that can be a single or a "profile of profiles" seems to add another layer of abstraction and indirection, seems like it fundamentally adds more to think about. Where "a list" isn't fundamentally another layer of abstraction.

So instead of breaking getting things with profiles, we break changing the profiles or assigning things to them? I'm not seeing how that's a win, they both seem about the same.

It's a win because we wouldn't be breaking one of our most-used APIs entirely for users of prior API versions. Instead, we're only breaking profile-param assignments, when the profile is layered, which you're not even supposed to do. So it "breaks" something that you're not supposed to do, rather than breaking something that you are supposed to do (updating a server).
Additionally, I would argue that placing the ordering of the profiles on the server object itself is bad, because, in general, all servers with the same characteristics are going to share the same order of layered profiles. My moving the order into the profile object itself, you don't have to specify the order on every single server you're using it for. Instead, you only have to get the order right once, and you can't accidentally change that ordering from the servers page in the UI.
As an operator, using the "profile list" idea, I still have to look at what profiles are in the list and figure out which profile to update. Using the "profile of profiles" idea, it could be presented exactly the same in the UI (as a list of profiles), so there wouldn't be any extra steps or pages to click through. You have to design your "profile of profiles", which would be the same amount of work as defining the order of the "profile list", except you don't have to worry about the ordering when assigning it to servers.

It's a win because we wouldn't be breaking one of our most-used APIs

I'm not sure "most-used" is true. In terms of operation, basically everything non-trivial needs Profiles, Parameters, DeliveryServices, and Servers. For operational scripts that are trivial, I'm not sure there are more little scripts that just need the DS or Server, than there are little scripts that just need Profiles or Parameters.

Additionally, I would argue that placing the ordering of the profiles on the server object itself is bad, because, in general, all servers with the same characteristics are going to share the same order of layered profiles. My moving the order into the profile object itself, you don't have to specify the order on every single server you're using it for. Instead, you only have to get the order right once, and you can't accidentally change that ordering from the servers page in the UI.

That seems like a disadvantage, not an advantage. One of the primary use cases Layered Profiles solves is the 95% identical Profiles, that just have a few small changes. Layered Profiles allows e.g. ["ATS9", "DATACENTER-FOO", "my-one-off-hack", "CDN-X"']. If every combination of that requires a profile-of-profiles, now we're back to the combinatorial explosion of having to create and assign unique profiles for every 95%-identical thing with a few small changes. Sure, we "isolated" the Parameters on my-one-off-hack which is kind of nice; but you still have to create, assign, and manage the Profiles ["ATS8", "DATACENTER-FOO", "my-one-off-hack", "CDN-X"'] and ["ATS9", "DATACENTER-BAR", "my-one-off-hack", "CDN-X"'] and ["ATS8", "DATACENTER-BAR", "my-one-off-hack", "CDN-X"'] and ["ATS9", "DATACENTER-FOO", "my-one-off-hack", "CDN-Y"'] and ["ATS8", "DATACENTER-FOO", "my-one-off-hack", "CDN-Y"'] and ["ATS8", "DATACENTER-BAR", "my-one-off-hack", "CDN-Y"'] and ["ATS8", "DATACENTER-BAR", "my-one-off-hack", "CDN-Y"'].

If every combination of that requires a profile-of-profiles, now we're back to the combinatorial explosion of having to create and assign unique profiles for every 95%-identical thing with a few small changes.

That might be the case, but the combinatorial explosion doesn't also include parameters in the mess, which is the real issue here, since only the "legacy" profiles have parameters. You're still only editing the parameters of "legacy" profiles. Plus, since the ordering is important, it makes sense to not have the possibility of different orderings on every server in the same "set" by having the ordering be defined only once in the profile object.

Defining the order in the profile object rather than the server also reduces the chances of making a mistake when creating a new server. Instead of having to replicate the list exactly on the new server to match another server in the same set, you just have to assign it the same profile. Less button clicks and no chance of messing up the order seems like a win for me. Plus, we've been using example profile lists of just a few profiles each. The chance of making a mistake in the ordering increases with the size of the list.

I'm not really a big fan of those two approaches either. The second one where you redefine what a Profile is somewhat more promising, but I share Rob's reservations about it. It'd be nice if there was some way to just deprecate the old endpoints and introduce entirely new ones, so that they could operate totally independently, but that's not only a lot more work but difficult (for me, anyway) to decide how to design.

On other hand, I'm with Rawlin on Servers and Delivery Services being our two most requested objects/endpoints. Maybe not in terms of "things that use the API" but almost certainly in terms of "requests that hit the API". It's also very troubling that if even one server uses a layered Profile GET /servers would fail completely. Could the API instead feign ignorance of things with layered Profiles in lower API versions than the one that supports it? like if you do GET /api/4.0/servers it returns a list of all servers that don't have layered profiles, and maybe a warning alert that tells you some were omitted (maybe even which, by hostname or ID) and why. PUTs can still fail for layered-profile servers, POSTs can tell the truth if you try to create one that doesn't appear in the results of a GET - that the server actually exists but you're not seeing it because your API version isn't high enough to support its structure.

how about this? if you try to update a server that has a layered profile (1+ profiles) with a non-5.x endpoint, ignore the profile value on PUT (don't make the profile change) and return alert.type=warning and alert.message='server updated but server has a layered profile and changes to a layered profile must be made using api 5.x'

That sounds like a reasonable approach to me, @mitchell852.

I'm with Rawlin on Servers and Delivery Services being our two most requested objects/endpoints

Do you have data to back that up?
I'm looking at our production requests for these endpoints over an arbitrary period, and I see about 10,000 profile requests, about 15,000 deliveryservice requests, about 125,000 parameter requests, and about 20,000 server requests. The data I'm seeing suggests none of these 4 endpoints are greatly more than the others, though parameters seem to be approaching order-of-magnitude.
Moreover, I maintain it also makes conceptual sense. My intuition is that most large scripts will need all 4, and that small scripts likely need profiles and parameters about as often as delivery services and servers. The data I see supports that.

If every combination of that requires a profile-of-profiles, now we're back to the combinatorial explosion of having to create and assign unique profiles for every 95%-identical thing with a few small changes.

That might be the case, but the combinatorial explosion doesn't also include parameters in the mess, which is the real issue here

Er, that's not the issue this solves. Parameters aren't duplicated today. You can't create two Parameters with the same name, config file, and value; they're required to be unique. The problem Layered Profiles solves is the Profile combinatorial explosion, it doesn't affect the Parameters that will exist at all.
And again, a Profile-of-Profiles doesn't solve that, because that same combinatorial explosion of ATS8_CDN-Y_DATACENTER-FOO_HACK-2 still exists.
We do have a lot of Parameters, and we could come up with ideas to fix that. But this doesn't, that's a completely different problem.

Defining the order in the profile object rather than the server also reduces the chances of making a mistake when creating a new server.

And increases the chance of making a mistake when creating the Profile-of-Profiles, and when assigning it when there are so many very similar ones. The chance of operator error, like the endpoints on old APIs, are just different not better or worse.

Could the API instead feign ignorance of things with layered Profiles in lower API versions than the one that supports it? like if you do GET /api/4.0/servers it returns a list of all servers that don't have layered profiles,

That would be extremely dangerous. If an operator has a script that needs to get all profiles, or any subset of profiles, they almost certainly actually need all profiles. And arbitrarily excluding some, even with a warning that will frequently be ignored, will result in whatever they're doing with the data not applying to many servers or delivery services. This solution seems to me like it will make operators do the wrong thing 99% of the time.

ignore the profile value on PUT (don't make the profile change) and return alert.type=warning and alert.message='server updated but server

That is also extremely dangerous and error-prone. If someone posted something, they almost certainly expect it to be applied completely or not at all. Worse, people shouldn't ignore warnings, but they frequently do, especially in scripts.
Applying part of what a user said to apply is far, far more dangerous than returning a real error and telling the user they need to upgrade their client.

breaking prior versions seems like a non-starter.

I don't understand your position here. As above, the two proposed solutions do break prior versions. They just pretend not to. Which is exponentially worse.
But the original design doesn't break prior versions, unless a user chooses to use the new feature.
It seems to me the original design breaks prior versions far less, and far less dangerously, than the proposed alternatives.
I mean, most new features inherently require new clients to work with. The only difference here is that choosing to use the feature requires upgrading all old clients, not just the old clients that need to work with the new feature.
Which I agree is unfortunate, but I don't see an alternative that doesn't require either a great deal of hackery, or fundamentally doesn't solve the problem this is solving, or else pretends not to fail but actually fails secretly and catastrophically.
Or else just not doing this, and continuing to live with our enormous number of often-wrong Profiles, which also seems worse.

Okay, rather than "our two most used" they are merely "two of our most used".

That would be extremely dangerous. If an operator has a script that needs to get all profiles, or any subset of profiles, they almost certainly actually need all profiles... This solution seems to me like it will make operators do the wrong thing 99% of the time.

Well their alternative is to get none at all, doing nothing 100% of the time, and a script that no longer works, though. By "profiles" I assume you mean "servers", right? This shouldn't require any actual changes to Profiles, as I understand the blueprint.
Not that those are our only two choices, but I see them as at least equally unacceptable.

Applying part of what a user said to apply is far, far more dangerous than returning a real error and telling the user they need to upgrade their client.

I agree with this; it seems like if you just ignore the Profile then the changes requested were not actually successfully made, and saying otherwise is just lying to the client. Although we have

"When an object is replaced, the response body MUST contain a representation of the object after replacement."

in our API guidelines, in practice I don't think most endpoints actually respect that to the letter, and I'm sure most clients don't even look at the response.

But the original design doesn't break prior versions, unless a user chooses to use the new feature.

That is true, it's not like upgrading to a TO version that supports layered Profiles is going to immediately cause breakages. I think my main concern is just that the bar for breaking tens of thousands of requests after that upgrade is extremely low. Is there any way we can make it harder to make a change that's going to shatter the API? Like, idk, forcing an admin user to make the first Profile layering, before anyone else can? I'm not tied to that particular idea, but just some kind of gatekeeping so that everyone associated is ready for the change and everyone is very sure of that readiness.

We need to find a happy medium here. How about this:

if a server has multiple profiles assigned:

pre-5.0 GETs have profileId as the 1st profile in the list
pre-5.0 PUTs that change the profileId will return a 4xx saying that the server has multiple profiles and must be changed via 5.0+

if a server has one profile assigned:

pre-5.0 GET and PUT behavior is the same as it is today (you are free to change the server's single profile)

That way, we're only breaking older clients that want to change profiles and only when those servers have multiple profiles.

We need to find a happy medium here. How about this:

I'm ok with most of those bullet points, but I have concerns with

pre-5.0 GETs have profileId as the 1st profile in the list

causing production issues when operational scripts get partial data and apply it, potentially missing critical Parameters.
I'd like to propose an alternative, that I think addresses both our concerns:

In API 5.0, we change the /servers object profile and profileId fields to be arrays, but a PUT or POST with multiple values returns a 400 error. The UI may or may not display a list (which can only add 1), the client implements handling multiple, and the API is documented to potentially return multiple and how their Parameters must be applied.
In API 6.0 (or possibly 5.0 in the following TC major version wherein API 4.0 is removed), we actually implement the feature, and multiple profiles can be assigned, are displayed in the UI, etc.

The disadvantage is that it will take us a little bit longer to get the feature.
But the advantage is that it solves both our concerns: clients on supported APIs are never given 4xx errors even after the feature is in-use, and we never return or apply partial data that could produce wrong results in Production stuff.
I'd also note, we can still work on the development. We can add things like database multi-profile support, TP UI things, etc that just won't be "turned on."

That works for me, but for this:

In API 5.0, we change the /servers object profile and profileId fields to be arrays

it might be nice to just have a single array of profile names, since they have to be unique anyways, and get rid of profile IDs on the server altogether. I believe the TO API generally has both name-based and ID-based profile parameter routes to use.

Currently we restrict servers such that no two may share a Profile and a service IP address - how will that constraint change when servers have multiple Profiles? Will they be disallowed to share a service IP and any of their Profiles? Any of their Profiles in the same order? Only the exact same sequence of Profiles? Or should the constraint be lifted (which would be my vote immediately without thinking through the implications - that could be a bad idea, idk)?

Currently we restrict servers such that no two may share a Profile and a service IP address - how will that constraint change when servers have multiple Profiles? Will they be disallowed to share a service IP and any of their Profiles? Any of their Profiles in the same order? Only the exact same sequence of Profiles? Or should the constraint be lifted (which would be my vote immediately without thinking through the implications - that could be a bad idea, idk)?

I've said for years, the key should be (Host, domain, port) or (ip, port); probably both.
It'll probably take some testing to make sure everything is ok with that, but that should be ok, and is really the right answer anyway.

should the constraint be lifted

I don't think it should be removed entirely, I think we really need some kind of constraint to prevent duplicate servers. Namely the real unique network identifier: the IP-port tuple and/or hostname-port tuple.
Alternatively, if we don't want to incur the dev cost of that right now, we could probably make a database or API constraint (ideally both) for profiles-in-same-order+IP. It's probably a pretty ugly db constraint, but I think it's possible.
I don't have a strong opinion either way.


Add equal_nan toggle for NaN values in array_equal

Fixes #4201
Relates to https://github.com/numpy/numpy/pull/16128, https://github.com/numpy/numpy/issues/9229

Jenkins, test this please

Jenkins CI test (for commit e60e4e32afac6332e536ee4071e2d66504a310dc, target branch master) succeeded!


Overhaul uploading - decouple upload logic from fragments/activities

As mentioned at #1060 , this probably needs to be done prior to implementing multiple uploads. Currently upload logic is coupled to various entry points - ShareActivity.java, MultipleShareActivity.java, and soon to be NearbyMapFragment and PlaceRenderer as well when direct uploads from Nearby are implemented. We would like to get the direct Nearby uploads out first so that users can use the feature, but after that is done we should buckle down and get this sorted.
Scope of this task:

Remove upload logic from the various entry points and have it all in one reusable module

@whym @psh or anyone else: Suggestions on how we can best organize the upload logic, especially in preparation for #604 ?

Also, I think this should go on a feature (upload-overhaul) branch, so that we can get a complete implementation of the following, before we roll out to users:

Upload code overhaul (this)
Multiple uploads from within the app (#604 )
Upload tests (#705 )

@maskaravivek do you still want to have it? If yes, I will assign.

@neslihanturan I think Vivek mentioned on Hangouts that wanted to swap this with me (so he is doing Wikidata edits now and I am ding this) :)

Looking through ShareActivity.java today, these are my plans at the moment:

To simplify permission logic, I don't think it's necessary to have Storage permissions requested via both Snackbar AND dialog (when user taps the submit button). The Snackbar should be used solely for optional permissions like Location, IMO.
Functions like Zoom and Find other images should have their own classes.
Both  and  intent filters should be directed to ShareActivity.java, and within ShareActivity.java itself we will handle things differently depending on whether 1 or many images are selected.

@misaochan There was a discussion a little while ago: #1128 - "Thinking about a combined share GUI" - with design mockups online.
If that's the direction you're going with creating a "combined" GUI that merges the single and multiple uploads thenI have a large portion of it already coded and working that you could use as a baseline to start from - see my "combined-upload" branch which implements a new UploadActivity following a Model-View-Presenter style.  Happy to drop that as a PR onto a feature branch.  I extracted a "categories model" from the categorization fragment as I wanted to share code with my new upload, that helped a lot with code complexity.
Where I stopped -

Integrating "quality" checks (picture too dark for instance) and adding a "yes/no" for including bad pictures in the upload batch
Snagging GPS coordinates from other pictures - the whole implementation is in view code right now and needs to be teased apart
Pulling data directly from incoming shared content:// URIs into a local cache to support "offline" uploads and restarting failed uploads

Also, looking at my repo, I think I need to push code from my laptop at home.  I'll rebase on master and push tonight when I get home.

@psh That would be wonderful, thanks so much! :) I was initially planning for the new upload UI to be a future enhancement (the aim of this issue was just to unify the upload entry points), but given that you appear to have already done a substantial amount of work on it, I could indeed try to build on what you've done and perhaps the goal of #1128 might be realized sooner than expected. :) I'll take a look at the PR as soon as you submit it.
Have the xml files for the new layout already been added as well, or only the backend?
@maskaravivek @neslihanturan What do you guys think? This might add a bit of work for us, but it makes more sense than overhauling the old code, when the new mockups are much better IMO.

It would be great to have @psh's contribution in overhauling the upload flow. Building upon his changes would be quite nice.

I agree that it would be too difficult to build on @psh 's work at this point of time - the scope for his changes is much larger than what we originally intended, and it will be difficult to get it to work within our grant deadlines. (I probably titled this issue wrongly at the start - we are not "overhauling" the uploads per se, we just want upload logic in 1 reusable module to unify upload entry points before we can enable multiple uploads).
However, I'd like to float the possibility of picking it up in the next grant. The upload UI and process is very much due an overhaul, and that PR provides a good foundation.


Update domain lisk.io to lisk.com

What was the problem?

The lisk domain was changed from lisk.io to lisk.com, which was not reflected in codebase

How was it solved?

Updated all the domain references

How was it tested?

Click on new domain links and check if they navigate to correct page

There's still quite a few spots with .io suffix. I need to confirm with DevOps which are already updated before updating them and merging.

Let me know the missing places, I will update them 👍🏼

@ManuGowda I've updated the docs & configs, can you please have a final look?


 THX

What is the point of this? The original uses 7 chars which is consistent with the GitHub short format.

@yyx990803
It may create a non-unique commit id by slicing 7 lengths. (Although the current probability is very unlikely)

you can ask git rev-parse --short for the shortest and yet unique SHA1

Further read: https://stackoverflow.com/questions/18134627/how-much-of-a-git-sha-is-generally-considered-necessary-to-uniquely-identify-a

After looking at where we are using these short commit ids, I think we should keep this consistent with GitHub’s format.

@yyx990803 @sxzz  @Justineo  Updated.
Is this commit necessary any more ?


Why are directories implemented this way ?

Hi,
I don't get why you have such a troublesome directory implementation.
Why don't you use a flag in metadata to say the file pointed to is a directory ?
That way, the directory will be described inside a file's content and since you already have atomic file content's replacement, you'd have atomic directory's content replacement.
This would also save a lot of memory.
You can also add some bit in each file's descriptor in the directory content to indicate if the corresponding file is valid (actually, you'd set the bit to 0 atomically when the file is deleted or moved).
That way, you can append to the directory content only the changes to the directory without actually allocating/muxing blocks. Provided you set a sentinel to 0xFFFFFFFF (all bits set), you can append to the directory's content block to add new file or modified file description.
When reading directory, you just read the "log" and report only the valid items (that will be unsorted, but that's a really small cost to pay for the gains).
After a directory content block is too much modified (typically, because too many files have changed that the "tail" containing the changelist is now bigger than the initial content) you apply the file content modification algorithm you've already designed to write a "clean" directory log and reclaim the free blocks. Since this is atomic too, you are always safe even in case of powerloss.
Usually, on embedded FS, you don't have thousands of files so this should work well.
That way, you'd have atomic move, small footprint, and good boot performance.

Hi @X-Ryl669, these are good thoughts. Heres some quick notes:
One important note is that the file structure in littlefs doesn't actually provide atomicity by itself. Files are copy-on-write (COW), which means new revisions can use parts of the older revisions, but the actual atomic update is pushed upwards.
Usually COW filesystems (btrfs for example) push atomic updates all the way up to the root of the filesystem, where they use a little log to atomically update the entire filesystem. However, this presents a challenge for wear-leveling, since those updates all get focused into a single block.
For littlefs, we basically put that log at the metadata level. Counterintuitively, one log vs many logs takes up the same code size. It does cost more storage memory (2x for metadata), however storage is cheap and metadata is small.
Combining the metadata + directory structure lets us minimize the wasted space for logging by combining the metadata from multiple files.
I'll have to think more about what it'd look like if the dir logs were also stored in files, it's an interesting idea.

There's also a bunch of changes going on right now around the directory pairs planned for the 2.0 update: https://github.com/ARMmbed/littlefs/pull/85. Most notably, the directory pairs become true logs.
Though a lot of complexity comes from the fact that littlefs is trying to support more than just flash. This means a couple of things:

We may not be able to clear bits atomically. NAND would need to fetch and rewrite a page of flash. SD/eMMC would need to fetch, erase, and then rewrite the flash, which is definitely not atomic. And several forms of internal flash just don't support masking bits after programming them once.
Not all storage supports writes < block size. For SD/eMMC block size == prog size. This means atomic updates relying only on incremental logs would break down and devolve into a full COW system such as btrfs.

It makes sense when supporting non word-modifiable storage (or at least when the driver don't support it).
To come back to the directory as file idea:

Suppose you have a word A in the metadata that say if the pointed item is a file or a directory.

Then to create a directory, you allocate a file (like any other file) with an header storing the word A and a revision

To modify a directory, you allocate a new file/dir and store the word A and the updated revision in it (no need to change the old directory, so any directory operation is thus atomic). File link, size, attribute, etc... should be stored here. A special file state should be stored here too, with (at least) 3 possible value (file, directory, movelink)

To search in a directory, you'll have to resolve which directory is the lastest (as you have to do for new file) using the metadata pair (I think it's were old directory reclamation could happen, since when the older directory can no longer be logically selected, it's safe to reclaim)

To move a file from one directory to another directory (the only operation that can't be atomic by itself) is also possible via this trick:
a. You first change the source directory to a new version with the file marked as a movelink (also storing the destination of the move).
b. You change the destination directory with the file in its new place
c. You finally change back the source directory actually removing the file.

With this scheme, any powerloss would not corrupt the filesystem. If the move is interrupted before a, the file is still at its old position. If it's interrupted before b, you can finish the operation next time to read this directory transparently. If it's interrupted before c, if you access the destination directory you'll see the file in there (so it appear as if the operation succeeded). If you access the source directory, you'll see the "movelink" and when checking the destination directory since the file is present in, you can safely remove the movelink.
One point here is that the "movelink" should also store the destination directory's revision, so if some other process increased the destination directory's revision, then it's safe to remove the movelink.


Use react particleJs

Lets make the cool background with react particleJs.

can i ?

doing

Lets make the cool background with react particleJs.

done

#32 Thanks all

can i ?

Thanks for your interesting. Basically you can take it where the issue is still open, maybe we will have new feature to be added soon.


🐛 BUG: AssertionError when running experimental local dev server with remote KV

Which Cloudflare product(s) does this pertain to?
KV, Wrangler
What version of Wrangler are you using?
2.17.0
What operating system are you using?
Windows 10
Describe the Bug
When running a development server locally with experimental and remote KV, the server errors and doesn't listen for requests. This problem seems to happen just after installing @miniflare/tre but is not resolved by installing it manually.
I'm using nodejs_compat.
Command
wrangler dev --experimental-local --experimental-local-remote-kv
Log
⛅ wrangler 2.17.0
--------------------
Using vars defined in .dev.vars
Your worker has access to the following bindings:
- KV Namespaces:
  - FORMS: *****
  - TIMEOUTS: *****
  - LINKS: *****
- Vars:
  - FROM_ADDRESS: "(hidden)"
  - MAILGUN_API_KEY: "(hidden)"
  - MAILGUN_API_BASE_URL: "(hidden)"
  - SECRET_SIGNATURE: "(hidden)"
  - STORAGE_IMPLEMENTATION: "(hidden)"
X [ERROR] local worker: AssertionError [ERR_ASSERTION]: false == true

      at transformMf2OptionsToMf3Options
  (***\node_modules\wrangler\wrangler-dist\cli.js:132242:37)
      at async startLocalWorker
  (***\node_modules\wrangler\wrangler-dist\cli.js:131757:28)
  {
    generatedMessage: true,
    code: 'ERR_ASSERTION',
    actual: false,
    expected: true,
    operator: '=='
  }

This is solved by passing --persist at the same time. Looking at the assertion that is failing, this is intentional behaviour. This isn't documented anywhere I could find and there is no indication in the command help screen. This error message could be more explanatory than false == true. I'm renaming this issue accordingly.

The --experimental-local-remote-kv flag was removed in Wrangler v3, so I'm going to close this issue for now.


return ... if ...

Allow reverse if including return. This would work similar to the let/if statement but have the following signature:
return val if expr;

This would allow for both easy parameter checking and one line return statements.
Some examples:
return Err(new Error("There must be at least 5 apples")) if apples < 5;
return Ok(None) if !ip_well_formed(&ip);

The problem with this is that it is incongruous with the way all other Rust code is written. Well it may be familiar to people coming from Ruby, I imagine programmers coming from other languages would be confused and surprised by it.

You can have one line return statements today already:
if expr { return val; }

This would add another way of doing the same thing without being more useful.

Adding on to what @mglagla said, we have:
return if expr { val } else { other_val };

And if the line is the last line of a function, the explicit return can be omitted.
Overall, this feels like it's solving an ergonomics problem that Rust doesn't have.

Wait @AndrewBrinker, so does that code already exist, I have not heard of it

@Nokel81 if blocks are expressions in Rust rather than statements, so they can be used in the same places as any other expression, e.g. let x = if true { 1 } else { 2 }; is also valid.


Where's the documentation?

The problem (description, use case, screenshots...)
On the Sprout Documentation there's this link: Google Recaptcha for Sprout Forms.
Where in the READ ME  of the sprout-forms-google-recaptcha plugin, there's nothing more.
The solution (that feature you'd like to see)
Make the documetation accessible.

I'm working on simplifying the docs as we get upgraded to Craft v4 and ReCaptcha will be integrated and no longer be a separate plugin.
The documentation linked to is for developers who are adding custom captcha integrations. Captchas are extensible in Sprout Forms so once you install the a Captcha Integration (like the Google ReCaptcha plugin does), the ability to enable/disable it and any settings appear on the Sprout Forms Spam Protection settings page.
For installed captchas, the more appropriate doc page is this one:
https://sprout.barrelstrengthdesign.com/docs/craft-v3/forms/spam-protection.html


results array

I was able to install component with no issues. Now uploading to a local server and returning results (uploading file to read content).  I created a custom class and i wanted to assign the results to an array for later use (ngfor).
`
items: IFileData[] = [];
ngOnInit(): void {
this.zone = new NgZone({
enableLongStackTrace: false
});
    this.basicOptions = {
        url: 'http://localhost:57017/api/FileReader'
    };

}

handleUpload(data: any): void {
    this.zone.run(() => {
        this.response = JSON.parse(data);
        console.log("how many records: " + this.response.length);
    });
}`

but now it fails to return.  I was looking in the docs how i would assign to my array.  Any recommendations would be helpful.

ok. looks like i did resolve it (i should've known)
replace the variable
response: any = {};
to
myObj: myInterface[] = [];
and on function handleUpload
 handleUpload(data: any): void {         if (data && data.response) {                         data = JSON.parse(data.response);             this.myObj = data;         }     }
i can refactor but this is pretty much done.


Document priority level of how we develop

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/374
Originally created by @bencodezen on 2019-02-11 18:54:44

serious regressions are a priority over new functionality

View 2 previous comments from the original issue on GitLab


Add figure decorator.

lcdblib version: NA
Python version: NA
Operating System: NA

Description
I have been thinking about streamlining my plotting workflow. I am proposing a decorator to plot functions allow outputting different contexts and formats.
What I Did
My current decorator:
from functools import wraps

def make_figs(fname=None, styles=None, formats=None):
    def _plot_all(func, styles=styles, formats=formats):
        @wraps(func)
        def wrapper(*args, styles=styles, formats=formats, **kwargs):
            def plot_style(style, formats):
                if isinstance(formats, str):
                    formats = [formats]
                elif formats is None:
                    formats = ['png', 'eps']

                with plt.style.context('seaborn-' + style):
                    fn = fname + '_' + style
                    func(*args,  **kwargs)
                    plt.tight_layout()
                    if style != 'notebook':
                        for f in formats:
                            plt.savefig('{}.{}'.format(fn, f))
                        plt.close()

            if isinstance(styles, str):
                styles = [styles]
            elif styles is None:
                styles = ['notebook', 'talk', 'poster', 'paper']

            for style in styles:
                plot_style(style, formats)

        return wrapper
    return _plot_all

To use you can call it like so:
@make_figs(fname='test_plot', styles=['notebook', 'poster', 'paper'], formats=['png', 'svg'])
def plot(df):
    fig, ax = plt.subplots(1, 1)
    df.plot('X', 'Y', kind='scatter', ax=ax)
    ax.set_title('test')
    return fig

This would then output:

test_plot_poster.png
test_plot_poster.svg
test_plot_paper.png
test_plot_paper.svg

The notebook style is displayed.
@daler you probably already have something that does this kind of thing. Just seeing if it is generally useful and should be be added to lcdblib. Or if I should just keep it in my personal library.

Oooo that looks really nice. I generally have separate code blocks for different styles (or separate modules entirely just for manuscript figures). But I'd start using this instead.

I would be curious how you do figures, it would be nice to have some clean solution.

I generally have big blocks of custom code depending on the figure --  not sure there's a general solution there. When saving PNG or PDF, I currently just duplicate the savefig() line, adjusting extension as needed. But this decorator just adds functionality to whatever custom function you have, so I think it's a pretty clean solution.

Cool, I am going to play with it for one of my projects. If it works out I will go ahead and add to lcdblib.


Update log4cats-slf4j to 1.2.1

Updates org.typelevel:log4cats-slf4j from 1.2.0 to 1.2.1.
GitHub Release Notes - Changelog - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.typelevel", artifactId = "log4cats-slf4j" } ]

labels: library-update, semver-patch

Superseded by #641.


New management files don't pass extract_management.R

The new management files RHH_2017 and RHH_2018 don't work with the script extract_management.R. @Illustratien could please try to find out? I didn't get it easily.

Because they have repetitive rows named Managements.
I removed it.


Make RGB led report active flight mode, not GPS lock status

Last night we were doing a lot of LPE + lidar + flow testing and found that when the pilot initially started in position control (switch position, but actually in alt control) while the light was blue and then took off and the light turned green (local position was published), the mode would not switch to position control from alt control until the pos-alt control switch was toggled. In the past, the led would flash red indicating an unallowed transition and keep attempting until it succeeded.

Which implies you liked the previous approach better?

The existing setup is ok if there were some feedback to the pilot that he was not in position control. The problem is the light is green and the pilot thinks position control is drifting, but he is actually still in alt mode.

Still needed?

Hey, this issue has been closed because the label status/STALE is set and there were no updates for 30 days. Feel free to reopen this issue if you deem it appropriate.
(This is an automated comment from GitMate.io.)


[Localization] Tests: Track and remove temporary files on fixture tea…

…r down
Let's replace use llvm::sys::fs::RemoveFileOnSignal with manual
tracking of created temporary files. Signals are unavailable on
Windows so all of the temporary files are going to be left in place
otherwise.

@swift-ci please smoke test


[utils/build-parser-lib] Disable LLVM_ENABLE_TERMINFO for building the parser library

We don't need it and it causes the parser library to reference non-public symbols.

@swift-ci smoke test and merge


Unsupported URL Type "workspace:": workspace:*

Command
other
Is this a regression?

[ ] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
No response
Description
npm i -D @angular-builders/jest
fails with
npm ERR! code EUNSUPPORTEDPROTOCOL
npm ERR! Unsupported URL Type "workspace:": workspace:*

Minimal Reproduction
npm i -D @angular-builders/jest
Exception or Error
No response
Your Environment
Angular CLI: 18.0.2
Node: 20.14.0
Package Manager: npm 8.5.5
OS: win32 x64

Angular: 18.0.1
... animations, common, compiler, compiler-cli, core, forms
... localize, platform-browser, platform-browser-dynamic, router

Package                            Version
------------------------------------------------------------
@angular-devkit/architect          0.1800.2
@angular-devkit/build-angular      18.0.2
@angular-devkit/core               18.0.2
@angular-devkit/schematics         18.0.2
@angular/cdk                       17.3.9
@angular/cli                       18.0.2
@angular/material                  17.3.9
@angular/material-moment-adapter   17.3.9
@schematics/angular                18.0.2
rxjs                               6.6.7
typescript                         5.4.5
zone.js                            0.14.5

Anything else relevant?
 Please open an issue for that package here: https://github.com/just-jeb/angular-builders


Camera entities unavailable in HA

Firstly thank you to fwestenberg for creating this integration.
Has any one seen this happen in the Home Assistant Log every hour everything but the "Front House motion" entity becomes unavailable and then some time later they all turn back on again.
Is there a special log somewhere I can check.
Thanks

Same here... Every hour each entity goes unavailable for 1 minute, and then comes back online for exactly 1 hour before going unavailable for 1 minute.

Ok great I’m not the only one then
that makes me feel a bit better.