Feature request: support for Yaml column renaming

A user already expressed interest in this feature for renaming columns coming from apis as hashes.
This would be an easy way for a user to reconfigure field names already listed to them, instead of them having to figure out the names upfront in python.
This is an overlap with the previously discussed name mapping via the schema for long column names for databases with short names support such as Postgres.

As a novice dlt user, this is a feature I was expecting and how I thought it might be implemented.
I would strongly consider adding this for table names as well. Many services have unique identifiers for constructs which may have their actual name as renamable objects. Airtable, Google Sheets, etc.
I think there are other ideas for implementation. Any would be good.

Pass in a python dictionary
Separate yaml file specifically for hard coded name adjustments

@dapomeranz @adrianbr
we'll go for Python for now. renaming directly in yaml requires a full lineage data to be kept - in essence dlt would need to generate unique id for every named entity in schema (tables and columns) based on the names in source (ie. API endpoints and filed names in json). if we are able to map each single data item into an entity in schema via separate automatic id, the names are free to be changed. until now they function as ids.
we have plenty of ticket requesting the behavior above. this will be quite a big implementation step
what I plan for now

renaming of the table will be released on Monday (resource.table_name ="xxx")
renaming of columns: what I'm missing is some really neat interface to do that. ie
resource.rename_columns(list of mappings)

do you think it makes sense to do (2) it in Python? any ideas for better interface?

If it is eventually going to be possible in the schema, then maybe it makes sense to delay this effort in favor of waiting. I can't speak very well to prioritites. I do think this is an important feature but maybe if it was easier to chain a dbt transformation immediately after running a pipeline then this feature isn't as necessary.
If we do want to implement it now, I imagine a parameter for the pipeline could just be a dictionary for table names. Then in processing the pipeline:
if table name exists in dictionary, use the value from that dictionary instead of the table name
It is a simple implementation but should be effective for the problem at hand.


Check Box to Save specific Suggestion instead of button

Check boxes can be used instead of the buttons for "Specific Suggestion" and "Measurable Suggestion".

Current "Keep Specific... Measurable... Attainable... Suggestion" will work for user interactions.


test: test hapi tails

Fixes #382
Since child spans are now allowed to track tail latencies, this PR adds a test to ensure that users of the Hapi tails API (up to v16, but also including the more general recommended mechanism in the v17 release notes under "Misc") can observe these child spans.
Note that child spans still cannot begin after a root span has ended.

Note that child spans still cannot begin after a root span has ended.

Does this still fix #382 ?

@ofrobots The Hapi API for request tails doesn't give us enough control to define a new root span under which tail work will be associated. It's also a removed API in v17+ so I would consider it not high priority to support additional instrumentation specific to Hapi tails beyond just this test.


Switch TreeViewItem.IsExpanded to a StyledProperty

What does the pull request do?
Switches the TreeViewItem.IsExpanded property to a StyledProperty. This follows the Expander control changes.
What is the current behavior?
IsExpanded is a DirectProperty and cannot be styled (see discussion https://github.com/AvaloniaUI/Avalonia/discussions/9719) cc @cristinathoughtpennies
What is the updated/expected behavior with this PR?
IsExpanded is now a full StyledProperty and usable in the styling system.
How was the solution implemented (if it's not obvious)?
See code.
Checklist

~[ ] Added unit tests (if possible)?~
~[ ] Added XML documentation to any related classes?~
~[ ] Consider submitting a PR to https://github.com/AvaloniaUI/Documentation with user documentation~

Breaking changes
Yes, StyledProperty instead of DirectProperty but functionality is largely only expanded for app developers.
Obsoletions / Deprecations
None
Fixed issues
Part of #9944

You can test this PR using the following package version. 11.0.999-cibuild0029297-beta. (feed url: https://pkgs.dev.azure.com/AvaloniaUI/AvaloniaUI/_packaging/avalonia-all/nuget/v3/index.json) [PRBUILDID]

You can test this PR using the following package version. 11.0.999-cibuild0029331-beta. (feed url: https://pkgs.dev.azure.com/AvaloniaUI/AvaloniaUI/_packaging/avalonia-all/nuget/v3/index.json) [PRBUILDID]

You can test this PR using the following package version. 11.0.999-cibuild0029355-beta. (feed url: https://pkgs.dev.azure.com/AvaloniaUI/AvaloniaUI/_packaging/avalonia-all/nuget/v3/index.json) [PRBUILDID]

You can test this PR using the following package version. 11.0.999-cibuild0029363-beta. (feed url: https://pkgs.dev.azure.com/AvaloniaUI/AvaloniaUI/_packaging/avalonia-all/nuget/v3/index.json) [PRBUILDID]

Good point, I'll update after that PR is merged.


UnoCss prettier class ordering plugin

is there an equivalent plugin to this one here but for unocss?

pnpm i prettier-plugin-tailwindcss -D

da06c6562ddc15447a4eaa7419fdf7cc73ce8249 this commit added a utility to sort the rules, if anyone is interested, feel free to create a eslint/prettier plugin for that

Duplicated of https://github.com/unocss/unocss/issues/1825


Fix list item indexing in race condition

What this PR does?
The todo_widget was using the using a counter to store the key value of
every task. This would cause assiging multiple tasks the same key value
in a race condition. To avoid this we make "sender_id" a part of the key
along with the counter.
Also the key now not being a integer value, we can't use it to find the
index of the task using it. Thus, a function is made that will find the
index of task whose key is sent by the user to strike.
Screenshot:

Merged, thanks @majordwarf!


fix: use head hash as parent hash

fixes a bug where the supplied parent hash to the payload builder was incorrectly set to head.parentHash and not head.hash

Client software MUST begin a payload build process building on top of forkchoiceState.headBlockHash

Codecov Report

Merging #2427 (a0e9ef8) into main (c7341b5) will decrease coverage by 55.33%.
The diff coverage is 0.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@             Coverage Diff             @@
##             main    #2427       +/-   ##
===========================================
- Coverage   71.64%   16.31%   -55.33%
===========================================
  Files         486      486
  Lines       59811    59811
===========================================
- Hits        42851     9758    -33093
- Misses      16960    50053    +33093

Flag
Coverage Δ

integration-tests
16.31% <0.00%> (-2.07%)
:arrow_down:

unit-tests
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

crates/consensus/beacon/src/engine/mod.rs
3.81% <0.00%> (-88.33%)
:arrow_down:

... and 354 files with indirect coverage changes


Adding custom error class

As the title.
Close #79

pre-commit.ci autofix

Codecov Report

Merging #80 (e88f107) into main (8e8607c) will decrease coverage by 0.56%.
Report is 6 commits behind head on main.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##             main      #80      +/-   ##
==========================================
- Coverage   92.13%   91.57%   -0.56%
==========================================
  Files           2        2
  Lines          89       95       +6
==========================================
+ Hits           82       87       +5
- Misses          7        8       +1

Files Changed
Coverage Δ

pytest_pyvista/pytest_pyvista.py
91.39% <75.00%> (-0.56%)
:arrow_down:

I think I'm done with this!


Add a UI

Add a simple UI that focuses on package search. Yarn does this very well.

https://github.com/aspnet/JavaScriptServices

Look into https://github.com/Microsoft/TypeScript-React-Starter

Prototype UI is complete.


What to do if uCxAtClientCmdGetRspParamLine fails inside uCxAtClientCmdGetRspParamLine?

Right now we set pClient->executingCmd = false; when uCxAtClientCmdGetRspParamLine fails. But uCxAtClientCmdGetRspParamLine can fail for many reasons. We must make sure that the AT status has been read before setting pClient->executingCmd = false;

This line has been removed


AMDGPU: gfx906: usage of omp_get_device_num() gives warning

Compilation of a code with omp_get_device_num() gives warning about implicit declaration of the function, while compiled on gfx906 only (e.g. https://github.com/ROCm-Developer-Tools/aomp/tree/master/test/smoke/devices).
Although this is only a warning, the test can be compiled and executed.
devices.c:19:25: warning: implicit declaration of function 'omp_get_device_num' is invalid in C99 [-Wimplicit-function-declaration]

This turns out to be the header file issue. which copy of omp.h being found in include search path affected the warning …


gardener/gardener and all modules that vendor gardener/gardener require<EMAIL_ADDRESS>

How to categorize this issue?

/area open-source
/kind bug
What happened:
For some reason gardener/gardener and all modules that vendor gardener/gardener require quite outdated client-go<EMAIL_ADDRESS>https://github.com/gardener/gardener/blob/9c6789844109351f4512871d8b1de25f9d91bd3c/go.mod#L50
https://github.com/gardener/gardener-extension-provider-aws/blob/09a896d305c843138e953d033e8e6e970abfb00e/go.mod#L26
This is quite inconvenient because it leads the module vendoring gardener/gardener to use replace statements to overwrite the correct/expected version of client-go. Such replace statements are not interpreted well by external tools like dependabot.
What you expected to happen:
gardener/gardener to require<EMAIL_ADDRESS>and not<EMAIL_ADDRESS>How to reproduce it (as minimally and precisely as possible):
See above.
Anything else we need to know?:
The issue is a potential blocker for https://github.com/gardener/gardener/issues/6328.
Environment:

Gardener version: v1.57.0
Kubernetes version (use kubectl version):
Cloud provider or hardware configuration:
Others:

I tried  go mod graph | grep<EMAIL_ADDRESS>It gives the following result -
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
So old client-go version is needed by etcd-druid and DWD, I checked later in etcd-druid here old client-go dependency is required because an old version of the gardener(v1.36) is imported and it imports<EMAIL_ADDRESS>which uses client-go version 11.0.1-0.20190409021438-1a26190bd76a+incompatible.
So same case with DWD.
So upgrading the Gardner version in these repos should resolve this issue.

Any update? We are also suffering from the same issue.

@shahar-h This issue should be resolved after my PR in etcd-druid is merged and we upgrade to the new release of etcd-druid. The maintainers of etcd-druid have currently put my PR on hold to resolve this issue first.

Should be resolved with merge of etcd-druid v0.15.3 into g/g -> https://github.com/gardener/gardener/pull/7333
@ialidzhikov @AleksandarSavchev please confirm and close the issue, thanks.

@shreyas-s-rao The part for etcd-druid is resolved. However we are waiting for a new release of DWD to resolve this issue.

Any update?

@shahar-h Basically the problem is more complex than we expected because there is a dependency cycle that carries<EMAIL_ADDRESS>and before the cycle gets broken it's nearly impossible to remove it. And there is an expected split of etcd-druid that will remove the cycle but the split has no EDA

/assign

We can drop<EMAIL_ADDRESS>in two steps as follows -

In gardener, update go mod to exclude<EMAIL_ADDRESS>Vendor above version of gardener in etcd-druid and DWD and exclude<EMAIL_ADDRESS>in respective project go.mod.
Vendor the new version of etcd-druid and DWD in gardener and once both are vendored drop the exclude<EMAIL_ADDRESS>from gardener go.mod.
Vendor new version of gardener in etcd-druid and DWDand drop<EMAIL_ADDRESS>respective projectgo.mod`.

And that's it we get rid of<EMAIL_ADDRESS>dependency.
Working POC branches with the above solution -

gardener -> https://github.com/acumino/gardener/tree/fix/mod
etcd-druid -> https://github.com/acumino/etcd-druid/tree/fix/mod
DWD -> https://github.com/acumino/dependency-watchdog/tree/fix/mod

/cc @ialidzhikov @rfranzke  @Kostov6
If the above solution looks good to you I can start working on it.

Looks promising. What about the following order of steps

Exclude<EMAIL_ADDRESS>in dependency-watchdog and etcd-druid and cut releases there.
Adopt the dependency-watchdog and etcd-druid releases in gardener -> with this<EMAIL_ADDRESS>should be gone in gardener. Release a new gardener version.
dependency-watchdog and etcd-druid can then adopt the released gardener version and drop their exclude statements.

?
From #6807 (comment), I don't understand the 2nd step - why dependency-watchdog and etcd-druid have the exclude the faulty version of k8s.io/client-go when they already vendor a github.com/gardener/gardener which brings the correct version k8s.io/client-go. My assumption is that the correct k8s.io/client-goversion will be picked and we don't need any exclusion.
I hope my suggested steps work, I haven't tried out. But they seems less effort - less steps and less vendoring here and there. And we won't need to exclude anything in github.com/gardener/gardener.

I will give it a try and if things work, will follow the suggested approach.

Steps to follow -

Exclude<EMAIL_ADDRESS>in dependency-watchdog and etcd-druid and cut releases there.
Adopt the dependency-watchdog and etcd-druid releases in gardener -> with this<EMAIL_ADDRESS>will be gone in gardener. Release a new gardener version(Etcd-druid is must here).
Vendor new gardener inetcd-druid  and DWD and drop the exclude<EMAIL_ADDRESS>
Thanks to @ialidzhikov, this can be done in three steps instead of four.

Steps to follow -

Exclude<EMAIL_ADDRESS>in dependency-watchdog and etcd-druid and cut releases there.
Adopt the dependency-watchdog and etcd-druid releases in gardener -> with this<EMAIL_ADDRESS>will be gone in gardener. Release a new gardener version(Etcd-druid is must here).
Vendor new gardener inetcd-druid  and DWD and drop the exclude<EMAIL_ADDRESS>
Thanks to @ialidzhikov, this can be done in three steps instead of four.

All the steps are completed.
/close


The new roadmap to v1.0.0

Foreword
This year is special to me, since I am finally able to provide more of my time to open source. First of all due to the commitment https://github.com/db-migrate/node-db-migrate/issues/605 of sponsoring time and money from my own company (which currently means effectively plan in whole days/weeks of time that I do not spend on paid projects and invest those into open source instead). This is important, since nothing is free in this world and this is my new way, to give something back to the open source community, since a lot changed for me and I do not have as much free time as I wish I would have.
Further I added opencollective and sponsoring slots, from which funds also solely progress and work on db-migrate will be paid. So this way also people that are not able to contribute to the project in terms of providing PRs and their workpower, can contribute to the success and further development by providing resources otherwise.
That being said, i am very grateful for all the contributors of the past and of the future of any kind. All support matters!
The roadmap to v1.0.0
That being said, the new roadmap which we try to achieve over the next few month consists out of the following:

[ ] Adding a concurrency manager
[x] Adding a state manager
[x] Adding internal schema representation
[x] Make db-migrate learn about the schema
[x] Introduce the new v2 migration schema
[x] Rework db-migrates structure and components
[x] Deprecate node <8
[x] Add chainable steps for new migration schemas
[ ] Add support for multiple driver instances
[x] Add new methods to the drivers for the state table creation (this time without any future need to adjust again)
[ ] Make sql migrations first class, but separate them into plugins (no more javascript loaders, wont be in the standard features anymore, and will be needed to installed once as a plugin)
[x] Adding new templates and move them to plugins
[ ] Clearly define the new concept for seeders (in work, but will integrate more seamlessly than the current approach)
[x] Adding default columns to table creation, such as an indexed seeder key column to reference and control seeded content and as well support easier support for reproducible system in integration testing
[ ] Add separate static seeds for very simple needs, without any version control (still making use of the default columns)
[ ] Enhance the documentation in general
[ ] Add the new features to the documentation
[x] Remove transitioner from the project

Work In Progress
The good news here are, a lot of this is done or in progress (master branch, not released yet). For example the structure has been reworked completely already. Yet I am thinking to also replace the CLI, but in favor of a shorter release time I might drop that into the next step. What is in its current state already ready is the new migration schema with a lot of new features. More to that later. The next step will be the templates and the default columns for table creation. After that I either go on the seeders or the concurrency manager, which highly depends on how the dependencies of this task resolve to each other (it turns out, most if not all tasks are somehow related to each other and require each other).
Also this is a very intensive rework and a lot of work put into thoughts and new processes, I am looking to keep the breaking changes to a minimum. In fact, I will never drop the version 1 migration schema. So expect that most should continue working, except that we're completely dropping support for old node versions, to move the db-migrate code base to a more modern state. This should be fine for the majority, for everyone else that does not integrate the db-migrate api, this change shouldn't affect to badly either. But it is finally time to move on and drop the support at this stage (also noting that node v6 is almost out of the maintenance cycle anyways.
The new migration schema
So the biggest thing in this release will be that I will finally clean up with the missing seeders and get the strings straight on this point and of course the migration schema. Which is leveling up the game a lot.
So to give you a preview of what is coming (and already working, if you're feeling fancy checkout the master branch and pg driver which already carry support). Here is what it is about:
The migration schema v2
The migration schema v2 is a very strict new standard, in which ultimately only reproducible components are allowed to be executed. It adds support for atomicity without transactions, it is to a certain degree self healing and it massively decreases the complexity of creating migrations. It allows an as seamless process of migrating up, as rolling back. It clearly defines guidelines for zero downtime compatible migrations and some more compelling features which will incorporate the DDL as well as the DML.
To give you a more alive example, lets take a series of actually working v2 migrations.
So assume this two migrations:
Migration 1
'use strict';

exports.migrate = async (db, opt) => {
  const type = opt.dbm.dataType;
  await db.createTable('test', {
    id: {
      type: 'UUID',
      primaryKey: true
    },
    test: {
      type: type.INTEGER
    },
    test2: {
      type: type.INTEGER,
      notNull: true
    }
  });
  return db.createTable('test2', {
    id: {
      type: 'UUID'
    },
    test_id: {
      type: 'UUID',
      foreignKey: {
        name: 'namename',
        table: 'test',
        mapping: 'id'
      }
    },
    test: {
      type: type.INTEGER
    }
  });
};

exports._meta = {
  version: 2
};

Migration 2
'use strict';

exports.migrate = async db => {
  await db.dropTable('test2');
  await db.removeColumn('test', 'test2');
};

exports._meta = {
  version: 2
};

Explanation
So what you can see here is, migration 1 creates two tables and migration 2 deletes one of the table completely and and removes a column from the other one. The first thing you will notice is that the interface looks different. First of all there is no up and down anymore. Only a migrate and a required _meta export are provided. The latter will later also allow to introduce some more features like specific driver instances per migration later on. The second change in the interface is, that callbacks are dead. Instead the migrate function now gets passed the driver and an options object. The options object contains the objects you used to retrieve through the setup function and might also hold some other toolings in the future.
So all in all the migrations look and feel very simple now and you might ask, where is my definition of rolling back? The answer is, there is none and you don't need any. The reason for this is the same reason why the new standard is so strict and the same reason why the v1 standard will keep alive forever. Certainly everything can be covered in the v2 schema, but in the meantime it will always be possible to execute operations that are not yet executable on the v2 schema. But that aside. Why you don't need the down operation anymore?
db-migrate got a few new components exclusively available to the v2 schema (with some exceptions such as manual schema feeding which will be available for inter compatibility to the v1 as well). These components are for example a component that learns about the manipulation of the DDL you're applying currently and another that stores the state of the new manipulated schema and information about the alteration that has been made. When you now execute an action in the new schema you're not accessing the driver directly anymore.  This allows db-migrate not only to intercept but also to correct actions. For example the new migration v2 schema also comes with failureStrategies. Currently the only available and also the defacto standard is rollback the current migration on failure automatically. Should the migration being interrupted by for example a failure of the database in the process of migrating, then db-migrate will be able to recognize this on the next run and rollback the executed actions before continuing by default.
The rollback is being recreated from the stored information. That also means, when you decide to delete a table for example, db-migrate knows exactly how your table looked before it was deleted and can completely automatically revert to the before state. An exception to this, is of course the data in the table. This is why a dropTable option is not safe by the zero downtime guidelines and a new deprecateX function for tables and columns will be added to the driver. Those will support a full zero downtime compatible cycle. Which will consist out of the following step:

mark the entity to be deleted on the first round.
Now you will make your first release in which you will implement support for the new schema coming up.

rename the entity with an unique name
Now you will release the new version which uses the new schema instead and your old version already respects this in which way ever necessary.

after a cycle of by default 4 releases (including the 2 before already) db-migrate will remove the table

These deprecation functions will also get later on support for signaling from the application directly, to shorten the process and they require information about a new release. For this there will be _meta export signal to define a certain migration as a start for a new release. This also might be improved in the future, since there are plans to also implement new migration patterns, to organize migrations in features and versions directly (for example a long requested feature like semver versioning might become reality, but this is yet to be defined).
Some final words about the standard itself. There will be a definition of the standard, which might be extended with features, as long as it is not altered. So already defined components might only change in a new version of the schema, which shall be avoided when possible. But new components might be added to make it more feature rich.
Some more notes
As always I want to express, I welcome any help. Be it documentation, implementation or helping out others with their issues. I want to thank all the people that helped in the process and all the contributors that added value to the project and I am looking forward to make db-migrate a migration framework that is not staying in your way, but helping you to do the right things and get the things done. Especially important to me is to search for a way to transfer knowledge, especially on the zero downtime section. I don't know yet, how I best accomplish this, but I consider this being even more important than everything else since I might provide a powerful tool, but what does it help you if you don't know how to handle it. Whoever has ideas or wants to help out on that section. I greatly appreciate your input!
Post v1.0.0
So to answer the question, what comes post v1.0.0. I can't answer yet, but there will always be room for improvement and i want to encourage you the community, to take active part in creating meaningful value for the project.
Notable references

#508
#324
#401
#518
#538
#539
#537
#514
#464
#338
#233
#215
#230

Issues to be checked further

#664

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

/cc @BorntraegerMarc

To add some afternotes, v1 migrations will also benefit from concurrency control, should the driver beneath support it, but they won't be able to automatically cooperate with v2 schemas. For that manual feeding of the schema will be added, so that it is still possible, to run rawSql and apply the alterations made manually.

some significant progress lately.

templates are there now instead of any templating language, we will just decide the plugins how to render the template. a hook for cli options will be added as well. Internal templates will just use es6 templating and we move each template to its dedicated file.
v2 schema is maturing and a lot of bugs are fixed, it is being used in production right now in some of my biggest projects and clients. still in dog feeding phase, but the worst bugs have been identified and fixed by now
state manager is fully working by now, just missing the concurrency manager

next up is

concurrency manager
seeder concept (and maybe implementation already)

@wzrdtales would it make sense to release a release candidate? Of course only if you think API is not gonna change :) I would be glad to test a RC in my biggest projects as well.

there is a beta tag released already :)

there is a beta tag released already :)

yeah, saw that :) I was specifically referring to a release candidate because I would like to wait with adoption until we're sure that the API is not gonna change anymore. I can invest some time to test pre-releases but don't have time unfortunately to re-implement changes, when they occur...

I will make sure to hit you up as a test candidate as soon as I freeze the standard (actually lot of it is alreay freezed).
which drivers are you using btw.? currently I implemented the support for the new functions only on pg and cockroachdb.

which drivers are you using btw.?

I'm using MongoDB

Also decided to remove the transitioner entirely. It was a utility to help migrate away from the very old async libs and globals from the first incubation before I rerolled the package completely. Will be available in the older versions should anyone need it to still migrate away pre 0.9.x migrations.
Note to myself: add to breaking change list.

tunnel ssh will move into the plugins, again
Note to myself: add to breaking change list.

@wzrdtales Any news on concurrency manager? It could be mighty useful for some of the things we are looking to do.

Where are the docs for the beta versions? I was unable to find them, sorry...

Would it be possible to set a different, non-latest tag to the beta releases? The current latest tagged release is for 1.0.0-beta.14, which means that anyone running npm install db-migrate installs a version of the library that is still in beta (rather than latest stable 0.11.11)
From the npm-dist-tag docs: https://docs.npmjs.com/cli/v6/commands/npm-dist-tag

Publishing a package sets the latest tag to the published version unless the --tag option is used. For example, npm publish --tag=beta.
By default, npm install  (without any @ or @ specifier) installs the latest tag.

It also produces (what I assume is) unwanted behavior from the npm outdated command
> npm outdated
Package                    Current   Wanted         Latest  Location
...
db-migrate                 0.11.11  0.11.11  1.0.0-beta.14  <redacted>
...

ahh i hate npm for being inconsistent... It was indeed not wanted, I already wondered why suddenly more people are starting to use the beta.
I will see to push a "new" old version out, however, the latest version is pretty stable, it is in use in huge production systems (however many of them maintained by me where I am dogfeeding the new migration schemas and by that battle testing them before the release) and there are really not much issues, especially if we leave out the new functionality.

Totally agree on wishing that npm treated the latest tag differently...
Would be very much appreciated if you could publish a 0.11.12 and added a different --tag option to future beta releases
And thank you 🙏  for all your work on testing v1.0.0!

v1 is for long time in use in our projects where we battle test it, there are still some issues with the v2 migrations although it is very stable by now, if you rely on v1 but use v1-beta it will be rock solid though. We have been scaling for the last period a lot company wise, so that ate a lot of time, we still are involved in db-migrate and continue to support it. It might be really time to consider dropping out v1 without the final support of the v2 schema, I would need to check if there was any breaking change pending to avoid another major version jump.

Also worth mentioning, we are the biggest investors as a company behind db-migrate right now, but we support more than just db-migrate as a open source project, so it is not like we could expand the efforts we already make (also not everything is visible directly to everyone, especially considering the v2 efforts and driver related things like the stuff on the cockroachdb driver). So obviously it would help the project to get more backers and funding, and individual contributors to accelerate the pace.


Cleaning up the main library file

Switching from if !condition to the more accepted idiom unless condition, which happens to be ever so slightly faster
Making the variable names a little more specific
Removing unneeded variable assignment, or switching to mass assignment
Changing a few multi-liners to single line expressions (hopefully to have those expressions turned into functions)
Adding a version specification to the redcarpet dependency, because otherwise it breaks everything.

Digging all of this.


Integration Test - Release - 2019/02/28 14:49:08

Auto create for integration test. AnotherAuthorUpdatedArticleFlow

#backlog

@adunndevster - This issue has in the backlog for 80 days. In 10 days, it will be automatically closed.
For more information about the process by which we handle documentation feedback, see "How we respond to Azure technical documentation feedback" in our public contributor guide.

The feedback you logged for Integration Test - Release - 2019/02/28 14:49:08 has been in our backlog for 90 days. While we try to respond individually to as many feedback items as possible, we are not able to respond to all of them. Since this item has been in the backlog for 90 days, we’re closing it.
For more information about the process by which we handle documentation feedback, see "How we respond to Azure technical documentation feedback" in our public contributor guide.


When using debug training data, the program stops and the data cannot be read

Bug description

How to reproduce the bug
No response
Error messages and logs
# Error messages and logs here please

Environment

Current environment
#- Lightning Component (e.g. Trainer, LightningModule, LightningApp, LightningWork, LightningFlow):
#- PyTorch Lightning Version (e.g., 1.5.0):
#- Lightning App Version (e.g., 0.5.2):
#- PyTorch Version (e.g., 2.0):
#- Python version (e.g., 3.9):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):
#- Running environment of LightningApp (e.g. local, cloud):

More info
No response

Hi
Could you please provide a bit more information about what is not working? It would be very helpful if you could fill out the section "How to reproduce the bug" and the verison of Lightning you tried. Without this information we won't be able to help  unfortunately, and we would have to close the issue.
And sorry for the late response, this fell through the cracks.


As a user, I want to show my profile

A profile page needs to be created where users can show:

Their sociables
Their goals
In summary, their contributions
With new functionalities, it then can also show achievements, scores

What do you think?

There is a profile off-canvas in the sociable detail page. Here there can be more statistics shown. Goals are not relevant and sociables are meant to be all the same


Action gives up once it meets a failed PR

We have multiple dependabot PRs on our repo, something like this:

#1013, status checks have passed
#1014, status checks have failed
#1015, status checks have passed

With autoapprove/merge enabled, I would have expected this action to merge both #1013 and #1015. Instead, it has merged #1015 and then failed with the error "All statuses are not success". On subsequent runs, with #1013 and #1014 still unmerged, it continues to give that error.
I think the issue here is in line 80 of index.ts. If it finds an unsuccessful status here it uses break to cancel the entire for loop, so no more PRs are processed. I think it should instead use continue like line 66, where it finds and unsuccessful check.
I'm happy to raise a PR to change this, but wanted to check first because I'm not sure if this difference is intentional.

Thanks Anand, I've created #71 :)


community: Fix LanceDB similarity search bug

Description: LanceDB didn't allow querying the database using similarity score thresholds because the metrics value was missing. This PR simply fixes that bug.
Issue: not applicable
Dependencies: none
Twitter handle: not available

To enable similarity_search_with_score, this fix is required, since we have to set the metric to cosine, to get scores between 0 and 1, as langchain expects it. Default is l2.
This is independent of the create_index functionality. Whatever distance metric we set when creating index, does not influence the metric when doing the search. The distance metric is only used to create the IVF-PQ. Moreover, with LanceDB we should run without index (aka exhaustive search with 100% recall) until up to 100k vectors or even more.
Thanks for this fix.

Checking LanceDB I can see:
`export enum MetricType {
/**

Euclidean distance
*/
L2 = "l2",

/**

Cosine distance
*/
Cosine = "cosine",

/**

Dot product
*/
Dot = "dot",
}`


Add option to allow only gitea users to access public endpoints

Feature Description

I want an option to restrict access to public endpoints like public repos and user profile .

As I'm self hosting my gitea instance, I want only users with an account to access any page the gitea and avoid leaking some code which is supposed to be shared only with some specific users.

Below in the screenshot below I able access any repos once I have the its url

Screenshots

Yes exactly  😳,

Why not make REQUIRE_SIGNIN_VIEW default to true?


Link to user support forum in README broken

Noticed that https://forums.docker.com/c/kitematic (listed in the README) doesn't lead anywhere useful.

Thanks!

#dibs

https://forums.docker.com/c/kitematic links to the correct location, there's just very few posts yet :)

This was broken when I tried, looks fixed now... Cheers


React integration doc and/or article

This should focus on some basic tasks, such as creating your own using Fine Uploader "core" or "basic", and using Fine Uploader UI w/ template. There are some gotchas here, so it will be helpful for other users to point these out. This will make use of common React tools, such as Webpack and Babel.
Demos for the website may be appropriate as well.

Another option (though much more work) is to develop a fine-uploader-react library that duplicates the UI side of Fine Uploader as React components. If the components are small and focused, this would allow easy creation of a UI in React and make it easy to customize and create your own UI as well without building it from scratch. Of course, top-level wrapper components for S3, Azure, and Traditional endpoint builds will be needed too.

Any changes that need to be made to fine-uploader (likely small ones) can be covered by individual cases, to be opened as the need becomes clear. The React "wrapper" will be developed and planned in a separate repo: https://github.com/FineUploader/react-fine-uploader.


Fix for Tournament Bets

This makes sure that kelly criterion suggested bet is not used in Tournament mode if balance is under 20000.
#14

Oh, I realize now I'm not sure what my PR's logic does if tournament <20000 ...

If you're looking at your fork of mine, it's currently 2 behind my main...but I've also made some other changes in the meantime that I haven't committed yet.
I'll be pushing those in a little bit here, then you should sync with the new commit.


[BUG] Spark Cosmos connector does not close its threads

Hi,
I have noticed that the cosmos connector does not close the threads that it creates. When the connector is put in a streaming application, those threads are accumulated over time. Eventually, the JVM cannot create any new threads and it crashes with the following exception
java.io.IOException: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: unable to create new native thread
I have tested the connector in stream mode and batch mode; the behaviour is the same.
Here is how you can reproduce the problem for both scenarios (batch & stream).
The following information is the same for both scenarios:
Environment: Databricks 9.1
Library: azure-cosmos-spark_3-1_2-12 version 4.12.2
For the batch example:
Databricks job:
{
    "settings": {
        "name": "Cosmos Connector Experiment - Batch Mode",
        "email_notifications": {
            "no_alert_for_skipped_runs": false
        },
        "timeout_seconds": 0,
        "max_concurrent_runs": 1,
        "tasks": [
            {
                "task_key": "Cosmos_Connector_Experiment",
                "notebook_task": {
                    "notebook_path": "/point/to/notebook/batchModeNotebook.scala",
                    "source": "WORKSPACE"
                },
                "job_cluster_key": "Shared_job_cluster",
                "libraries": [
                    {
                        "maven": {
                            "coordinates": "com.azure.cosmos.spark:azure-cosmos-spark_3-1_2-12:4.12.2"
                        }
                    }
                ],
                "timeout_seconds": 0,
                "email_notifications": {}
            }
        ],
        "job_clusters": [
            {
                "job_cluster_key": "Shared_job_cluster",
                "new_cluster": {
                    "cluster_name": "",
                    "spark_version": "9.1.x-scala2.12",
                    "azure_attributes": {
                        "first_on_demand": 1,
                        "availability": "ON_DEMAND_AZURE",
                        "spot_bid_max_price": -1
                    },
                    "node_type_id": "Standard_DS3_v2",
                    "enable_elastic_disk": true,
                    "data_security_mode": "LEGACY_SINGLE_USER_STANDARD",
                    "runtime_engine": "STANDARD",
                    "num_workers": 1
                }
            }
        ],
        "format": "MULTI_TASK"
    },
}

Notebook code:
// Databricks notebook source
import org.apache.spark.sql.functions._
import org.apache.spark.sql.{Dataset, Row}

object DeltaToCosmos {

  def run(): Unit = {
    val cosmosConfig = Map(
      "spark.cosmos.accountEndpoint" -> "https://<fill>.documents.azure.com:443/",
      "spark.cosmos.accountKey" -> "<fill>",
      "spark.cosmos.database" -> "<fill>",
      "spark.cosmos.container" -> "<fill>",
      "spark.cosmos.applicationName" -> "Experiment",
      "spark.cosmos.write.strategy" -> "ItemAppend",
      "spark.cosmos.write.maxRetryCount" -> "10",
      "spark.cosmos.write.bulk.enabled" -> "true")

    spark
      .readStream
      .format("delta")
      .load("<fill>")
      .select("col1", "col2", "col3", "col4", "col5")
      .withColumn("id", concat_ws("|", lit("copy"), col("col1"), col("col2"), col("col3"), col("col4"), col("col5")))
      .writeStream
      .outputMode("append")
      .queryName("experiment")
      .option("checkpointLocation", "<fill>")
      .foreachBatch((batch: Dataset[Row], batchId: Long) =>
        batch
          .write
          .format("cosmos.oltp")
          .options(cosmosConfig)
          .mode(SaveMode.Append)
          .save()
      )
      .start()

  }

}

DeltaToCosmos.run()

For the stream example:
Databricks job:
{
    "settings": {
        "name": "Cosmos Connector Experiment - Stream Mode",
        "email_notifications": {
            "no_alert_for_skipped_runs": false
        },
        "timeout_seconds": 0,
        "max_concurrent_runs": 1,
        "tasks": [
            {
                "task_key": "Cosmos_Connector_Experiment",
                "notebook_task": {
                    "notebook_path": "/point/to/notebook/streamModeNotebook.scala",
                    "source": "WORKSPACE"
                },
                "job_cluster_key": "Shared_job_cluster",
                "libraries": [
                    {
                        "maven": {
                            "coordinates": "com.azure.cosmos.spark:azure-cosmos-spark_3-1_2-12:4.12.2"
                        }
                    }
                ],
                "timeout_seconds": 0,
                "email_notifications": {}
            }
        ],
        "job_clusters": [
            {
                "job_cluster_key": "Shared_job_cluster",
                "new_cluster": {
                    "cluster_name": "",
                    "spark_version": "9.1.x-scala2.12",
                    "azure_attributes": {
                        "first_on_demand": 1,
                        "availability": "ON_DEMAND_AZURE",
                        "spot_bid_max_price": -1
                    },
                    "node_type_id": "Standard_DS3_v2",
                    "enable_elastic_disk": true,
                    "data_security_mode": "LEGACY_SINGLE_USER_STANDARD",
                    "runtime_engine": "STANDARD",
                    "num_workers": 1
                }
            }
        ],
        "format": "MULTI_TASK"
    }
}

Notebook code:
// Databricks notebook source
import org.apache.spark.sql.functions._

object DeltaToCosmos {

  def run(): Unit = {
    val cosmosConfig = Map(
      "spark.cosmos.accountEndpoint" -> "https://<fill>.documents.azure.com:443/",
      "spark.cosmos.accountKey" -> "<fill>",
      "spark.cosmos.database" -> "<fill>",
      "spark.cosmos.container" -> "<fill>",
      "spark.cosmos.applicationName" -> "Experiment",
      "spark.cosmos.write.strategy" -> "ItemAppend",
      "spark.cosmos.write.maxRetryCount" -> "10",
      "spark.cosmos.write.bulk.enabled" -> "true")

    spark
      .readStream
      .format("delta")
      .load("<fill>")
      .select("col1", "col2", "col3", "col4", "col5")
      .withColumn("id", concat_ws("|", col("col1"), col("col2"), col("col3"), col("col4"), col("col5")))
      .writeStream
      .format("cosmos.oltp")
      .queryName("Experiment")
      .option("checkpointLocation", "<fill>")
      .options(cosmosConfig)
      .outputMode("append")
      .start()

  }

}

DeltaToCosmos.run()

As you can see, both examples are quite simple. They read data from a delta lake and they push that data into a cosmos container. You will need to fill in any sensitive information (such as endpoints & keys) to run them.
At first glance, the streams work fine. They work as they should, they read data from the delta lake and store them into the cosmos container.
Looking at the logs of the job, you will see the aforementioned exception occur periodically, i.e. once per day.
The metrics of the jobs show a good picture of what is happening. The following graph shows the total processes of the worker over time:

You can see that threads are accumulating over time. Eventually, the worker reaches the upper limit for max processes and crashes. A new worker instance is spawned and the cycle continues. The timestamp of the exception matches the timestamp of the worker crash. The end of the graph shows a dangerous outcome of the problem: it is possible that the worker is unable to restart and it stays "stuck" indefinitely. This is why the graph reaches a plateau. At this point, the worker VM needs to be restarted manually.
So far, I have explained that threads accumulate over time but I have not explained why the cosmos connector is responsible for that. I wrote a small piece of code that prints the total number of threads and the name of each thread. I injected that code into the worker VMs. Out of tens of thousands of threads, the majority of them had the prefix spark-cosmos-db-.... In case it helps, here's the snippet:
import scala.collection.JavaConverters._
val threads = Thread.getAllStackTraces.keySet().asScala.toList
val builder = new StringBuilder
threads.map(t => t.getName).foreach(tuple => builder.append(tuple))
println(s"there are currently ${threads.size} threads ${builder.toString}")

As an extra sanity check, I removed the connector from the streaming applications. Instead, I used the delta lake connector. As expected, the total threads remain stable over time.

This bug may not be noticeable for short-living applications but it is a big problem for streaming applications that run 24/7.

Thanks - great bug description. I will take a look and get back with investigation result.

@FabianMeiswinkel I am looking forward to your results.
Let me know if you need more information, we have both demos running 24/7 in our Databricks environment.

Thanks - what would indeed help would be the output of the following to Spark UI views (if it is easier you could also share them via mail fabianm[AT]microsoft.com)
Thread dump (Spark UI/Executors/Thread Dump - for one of the impacted workers)
Heap histogram (Spark UI/Executors/Heap Histogram - for one of the impacted workers)
The most likely area where threads could "leak" is that there are many Cosmos Client instances (which use connection and buffer pooling for network operations). We use a cache for Cosmos Client instances - where the cache key is the account name of the cosmos account plus a few config entries. The only config entry that I know could accidentally result in many clients being cached would be if the "spark.cosmos.applicationName" config entry has unique values for every streaming batch iteration. But form the code you shared above that doesn't seem to be the case. So, looking at the thread dump and histogram would help to further narrow down where the problem is. I am in parallel trying to repro right now.

@achistef - I think I was able to reproduce the issue you described. Can you confirm that you also see a growing number of threads with the thread name "cosmos-daemon-BulkExecutor-*" in the Thread Dump Spark UI view? (Spark UI/Executors/xxx (some worker)/Thread Dump)?
I am going to start working on a fix for that.

@FabianMeiswinkel I can confirm that this is what I also see. There are a lot of threads named cosmos-daemon-BulkExecutor. I was wrong about the name of the threads in my description, pardon me.
I have just sent you the email you requested with two text files, one for the thread dump and one for the histogram.

@achistef - thank you! I am working on a fix. Worst case hotfix will be available at the end of this month.

Hotfix (in version 4.13.1 - https://repo1.maven.org/maven2/com/azure/cosmos/spark/azure-cosmos-spark_3-1_2-12/4.13.1/azure-cosmos-spark_3-1_2-12-4.13.1.jar) has been released to Maven

@FabianMeiswinkel Thank you so much, you've done an amazing job that it is very helpful to us

@achistef - Thanks! You did most of it by putting together the excellent problem description. We will also adjust our long-running release-validation tests. We didn't clearly identify the leak there because the number of micro-batches executed was too low - we are changing the test - so chances are in the future we will identify this kind of leak earlier. So, thanks for the feedback again and please let me know if you hit any other issues in the future.


Improve loading times of opossum files with no output.json

When opeing a large file with thousands of pre-selected attributions for the first time (the file does not contain an output.json, but instead this one is created in this process), the user has to wait for a long time. The performance can be poor such that the user might have to wait for several minutes.
The goal of this ticket is to improve here. The bottle neck is the creation of the output.json file.
A generator could mitigate the issue by pre-populating the output.json.

This ticket is related to #1481 and addresses a special case.

Checked performance:

initial loading with no output.json present (inside the .opossum file) is now comparable to opening in the case of already generated outpu.json files


Tooling and standardization for sporks.json

Context: https://github.com/onflow/flow/pull/809#issuecomment-1032943859
Once we establish some standard for the sporks.json format, we should have some tool to generate this file in order to avoid manual errors.
Also, I think it would be a good idea to create a new file per spork to retain the history of sporks instead of appending to the same file every time. This makes it easier to track down the details of a specific spork rather than looking through a large file. It's also likely that the sporks.json format will continue to change over time, and so it would be better to have separate files per spork rather than try to keep multiple formats in a single file.

Agreed, a standard needs to be establish and some tooling needs putting in place to ensure the validity and formatting of the json.
Right now the sporks.json isn't valid JSON code as it contains a trailing curly bracket. #1186
This renders it unusable by any JSON interpreter.


Replace Supervisord web app.

The stock one displays statuses of services but has log links that do not work - doesn't provide an extension point for additional Galaxy-specific functionality.

WIP @ https://github.com/jmchilton/gravity-webapp.


Exponential mechanism

Sensitivity calculations are stored in the compute_sensitivity function of aggregators. There also needs to be some utility function, which could be represented as a computation graph and passed to the runtime to execute on private data. A potential implementation-
In the validator:
The utility function (embedded as a graph), any constants in the utility function (embedded as a release), the candidate_node_id and the output_node_id are contained as options in the ExponentialMechanism component, which is returned when the exponential mechanism is expanded. Adversarial utility functions are overwritten when the node is expanded. A hardcoded utility function is stored in the aggregator.
In the runtime:
The exponential mechanism iterates through the candidate set:

it sets the candidate_node_id's release to the candidate
it evaluates the graph using the same function for running the analysis graph
it retrieves the utility from the output_node_id

Still open to suggestions on this.

sync with whitepaper proofs

Integrated into whitenoise-core


Notify user when press back button for exit

Is your feature request related to a problem? Please describe.
If the user accidentally presses the back button on the home page, the app will minimize. The user will also be notified if they accidentally press the back button.
Describe the solution you'd like
User must press the back button twice continuously to exit the app.
Additional context
A toast message will also appear if the user only taps the back button once.
Are you working on this? (Yes/No)
Yes

@amitamrutiya2210 this looks like a legit issue to be worked on. Assigning this issue to you.

@amitamrutiya2210  hey..are you working on this issue?

Hey @Interstellar07 i already submitted PR regarding this issue


practicality of read-only config.xml API endpoint

To be discussed, see https://forum.opnsense.org/index.php?topic=9707.0

+1

I'm not against an endpoint for config retrieval, although I believe it should be part of a rewrite for diag_backup.php, since it's overlapping functionality. It's not very high on my list of things to work on to be honest.
Building a plugin that only serves a download config isn't much work, for consistency reasons probably won't be offered by us (certainly not in core, without replacing the old backup solution).

I was thinking such a temporary spot could maybe reside in the firmware API?

It could, but eventually the backup api should probably have a different endpoint to better control access (people able to upgrade might not be granted access to everything that's stored in the box).
Another short term solution could be to add the download action to diag_backup.php which makes it the responsibility of whoever is going to replace this code in the future (which isn't a big issue given the functionality should be in the new api as well).

The issue with diag_backup.php or any other static page is that it's not API-key enabled. Otherwise that'd be working already...

sorry, my mistake, moving the download to a new spot at a different namespace and using it in the legacy page could also be an option (kind of a hybrid approach), but the risk stays that eventually nobody is willing to invest into the real elephant in the room (moving the functionality), creating an indefinite hybrid component....
Adding a plugin with only a download action is easy to build and doesn't taint our support status (could stay at community support level) and can use it's own access control (lower risk of leaking sensitive data).

Now that you mention it... great idea to go forward indeed. :) That satisfies requirements from all sides and does not create roadblocks. Shall I move this to plugins then with a concrete plan?

sure, if anyone is planning to work on it, I would even offer help if needed (just don't want to own the plugin).

Without encryption support, it would be an API endpoint with around less than 10 lines of code to download the current configuration.

@fabianfrz is that a "I'll take it"? :)
moving over to https://github.com/opnsense/plugins/issues/865


build: Change dockerfile to use intels fortran compiler

Changed Dockerfile and and compile.sh to use Intel's Fortran compiler through oneAPI. Building the docker image takes a bit of time. Everything is very slow on M1 Mac when using the platform flag in docker-compose. This is expected, but testing on other platforms should be done to see if this is a local problem or connected to the use of Fortran compiler.
Note that the tests will not pass as long as dat/Asg_20VA001.dat and info/CompEXP.inf (and possibly the other files inside info) is present inside the api folder. The files can be found in the gpa-libhg repo.
I am not yet sure whether or not these files are used for testing or by the generated library as well. Though I suspect that the Fortran functions calls on these files => They are necessary to run calculations. At this point in time both the dat and info folder is added to the gitignore, so we should consider adding them back in or possibly adding them to /usr/local/lib/, but I suspect the Fortran code would not find them there.

Tested on x86 linux. Build time is approx 2min, and tests take about 5 seconds to run. I think this is expected and acceptable.


Small issue on typeshelper

The typeshelper.py in the package downloaded by pip install datatrove[all] does not match the repo's typeshelper. I have to edit the source code. Can the next version make sure to include all languages in Languages?

Hi. The version on pypi is the latest stable version. For the latest dev version (the one here on github) you should clone the repository and run pip install -e .[all]


Scala 3 compatibility ?

Would the kernel work with scala3 dependencies ? If not, would it be a matter of just adding a new target for almond or are there major blockers ?
This would be important for the company I work for and we would probably be willing to spend some dev hours to make it work if there was a feasible path.

I don't think it works yet… Once the next release is out with Scala 2.13.4 support, you may be able to add Scala 3 dependencies, using an explicit suffix, like
import $ivy.`org.typelevel:cats-effect_3.0.0-M1:3.0.0-M4`

Almond heavily relies on Ammonite. For a proper Scala 3 support, we'll need Scala 3 support in Ammonite first. Work on this is well underway (see https://github.com/lihaoyi/Ammonite/pull/1135 or this contributors thread), I'm confident we'll have preliminary Scala 3 support in an Ammonite release later this month.
I'll probably try to use that in Almond right after. Scala 3 support in Ammonite should happen either via 2.13 / 3 compatibility mode (both Scala 2.13 and 3 artifacts in the same class path), or via heavier isolation between the internals of Ammonite and user space (relying on class loader isolation, allowing to expose a Scala version to users while using another one internally). Almond should use the same mechanism that Ammonite ends up using.
So hopefully, I should have the time to add preliminary Scala 3 support myself, based on what I'm doing in Ammonite. But I'd really welcome some testing / feedback of it, once it's there.

I don't think it works yet… Once the next release is out with Scala 2.13.4 support, you may be able to add Scala 3 dependencies, using an explicit suffix, like
import $ivy.`org.typelevel:cats-effect_3.0.0-M1:3.0.0-M4`

Almond heavily relies on Ammonite. For a proper Scala 3 support, we'll need Scala 3 support in Ammonite first. Work on this is well underway (see https://github.com/lihaoyi/Ammonite/pull/1135 or this contributors thread), I'm confident we'll have preliminary Scala 3 support in an Ammonite release later this month.
I'll probably try to use that in Almond right after. Scala 3 support in Ammonite should happen either via 2.13 / 3 compatibility mode (both Scala 2.13 and 3 artifacts in the same class path), or via heavier isolation between the internals of Ammonite and user space (relying on class loader isolation, allowing to expose a Scala version to users while using another one internally). Almond should use the same mechanism that Ammonite ends up using.
So hopefully, I should have the time to add preliminary Scala 3 support myself, based on what I'm doing in Ammonite. But I'd really welcome some testing / feedback of it, once it's there.

I strongly suspect I lack the skill to contribute to making it happen, but I'd be happy to invest some time on testing it. Thanks for your efforts.

I am with @Quafadas. I think almond is a super nice project and i would love to help move it to Scala 3.
Would be nice if maintainers (@alexarchambault ?) could give us (or me, if Quafadas is not interested anymore) a thumbs up or down to let us know whether this would be a welcomed feature of if this is already being worked on.

Hi there — any updates on this? I'm very interested as well.

@lucag I think we have to take matters into our own hands hehe

Here is what I did in Ubuntu 20.04 (actually, in WSL2)
# jupyterlab
sudo apt install nodejs python3 python3-pip
pip install --user jupyterlab pandas matplotlib
# almond per se.
git clone git@github.com:almond-sh/almond.git
cd almond
git submodule init
git submodule update
./mill -i jupyter 3.0.1

After a while, there will be messages like this:
...
[I 2021-10-12 21:16:00.022 ServerApp] jupyterlab | extension was successfully linked.
[I 2021-10-12 21:16:00.036 LabApp] JupyterLab extension loaded from /home/rouquette/.local/lib/python3.8/site-packages/jupyterlab
[I 2021-10-12 21:16:00.037 LabApp] JupyterLab application directory is /home/rouquette/.local/share/jupyter/lab
[I 2021-10-12 21:16:00.039 ServerApp] jupyterlab | extension was successfully loaded.
[I 2021-10-12 21:16:00.039 ServerApp] Serving notebooks from local directory: /opt/local/almond.sh/almond/notebooks
[I 2021-10-12 21:16:00.039 ServerApp] Jupyter Server 1.11.1 is running at:
[I 2021-10-12 21:16:00.039 ServerApp] http://localhost:8888/lab?token=38ca7528c18ce2d78faf5b9afe7a04adba8f1ef5313ce948
[I 2021-10-12 21:16:00.039 ServerApp]  or http://<IP_ADDRESS>:8888/lab?token=38ca7528c18ce2d78faf5b9afe7a04adba8f1ef5313ce948
[I 2021-10-12 21:16:00.039 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2021-10-12 21:16:00.147 ServerApp]

    To access the server, open this file in a browser:
        file:///home/rouquette/.local/share/jupyter/runtime/jpserver-5740-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/lab?token=38ca7528c18ce2d78faf5b9afe7a04adba8f1ef5313ce948
     or http://<IP_ADDRESS>:8888/lab?token=38ca7528c18ce2d78faf5b9afe7a04adba8f1ef5313ce948

I realize that it would be better to build/publish docker images; this should at least help confirm that scala3 support is available.

@NicolasRouquette
I'm working on a Docker Image using almond and Ubuntu 20.04, so try it if you need it.
Docker Image
https://hub.docker.com/layers/poad/docker-jupyter/scala/images/sha256-3f1508ed460919e842dd1632142f3da196a38ba84574cdd7928079ab960ff5bd?context=explore
Source
https://github.com/poad/docker-jupyter/tree/main/scala

Almond supports Scala 3 fine as far as I can tell. Is this issue specific to the current docker image?


Liked tweets stays hidden even when set to do nothing

I have not been able to see any liked tweet by people I follow.
I have changed the option of few days ago already so it shouldn't be a refresh issue I suppose.
Maybe a conflict with another option set to "Hide" ?
If it's an expected conflict and not a bug, might I suggest an UI indication of it's existence ?

I can't reproduce this on the main timeline, but is this happening when you click a tweet to view replies to it, then the tweet itself is hidden?
Just found the cause of that and it will be fixed in the next release.

Switching to Home timeline seems to help my case, I just know I used to see liked tweets in the "Latest tweets" timeline.
I guess using the Home timeline with tweak new twitter is now more bearable so I'll do that for now !


Initial implementation of launcher JVM auto-detection

AKA "Toolchains for Daemon"
Adds a new experimental gradle property for configuring the minimum version of Java to use to run the Gradle Daemon. Adds one integration test to verify functionality
Questions:

How is this affected by Java Serialization? Do we use Java serialization to communicate between the launcher and daemon processes?

If so, does java serialization work between Java versions? If not, can we avoid Java serialization?

What are the performance implications of running auto-detection in the launcher?

It is probably slow. We probably want to cache the result.
What if someone messes with our cached value?

Not sure why the test fails on CI. Its working locally.

Not sure why the test fails on CI. Its working locally.

Looks like the -Porg.gradle.java.installations.paths=/opt/java/openjdk getting set in the command line isn't turning into a system property in the launcher, since it isn't listed as a build option by LayoutToPropertiesConverter

@bot-gradle test qfl

I've triggered the following builds for you:

QuickFeedbackLinux build

@bot-gradle test qfl

I've triggered the following builds for you:

QuickFeedbackLinux build

@bot-gradle test qfl

I've triggered the following builds for you:

QuickFeedbackLinux build

@Bot-gradle clear

@bot-gradle help

Currently, the following commands are supported:
@bot-gradle test <BuildTrigger1> <BuildTrigger2> ... <BuildTriggerN>

A trigger is a special build for this PR on TeamCity, common triggers are:

SanityCheck/CompileAll/QuickFeedbackLinux/QuickFeedback/PullRequestFeedback/ReadyForNightly/ReadyForRelease
Shortcuts: SC/CA/QFL/QF/PRF/RFN/RFR
Specific builds:

BD: BuildDistributions/BuildDocs so that you can preview the generated docs/distribution.
PT: PerformanceTest, all performance tests for Ready For Nightly stage.
APT: AllPerformanceTest, all performance tests, including slow performance tests.
AST: AllSmokeTestsPullRequestFeedback
AFT: AllFunctionalTestsPullRequestFeedback
ASB: AllSpecificBuildsPullRequestFeedback
ACC: AllConfigCacheTestsPullRequestFeedback
ACT: AllCrossVersionTestsReadyForNightly
AFTN: AllFunctionalTestsReadyForNightly
ACTR: AllCrossVersionTestsReadyForRelease
AFTR: AllFunctionalTestsReadyForRelease

@bot-gradle test and merge

queues this PR for testing and merges if all tests pass by:

Creating a merge commit from your PR branch HEAD and the target branch
Running a ReadyForNightly build against the merge commit
When it passes, fast-forward the target branch to this merge commit (i.e. merge the PR)

The merge commit is called a pre-tested commit, which means that it fully tests the integration
of your branch HEAD and latest master, instead of only testing your branch HEAD.
@bot-gradle cherrypick to <branch>

Cherrypicks the current PR to another branch.
A new PR will be created and a build will triggered automatically if there is no conflict.

@bot-gradle cancel

cancel a running pre-tested commit build or remove it from queue

@bot-gradle clean

clear the conversation history

@bot-gradle help

display this message

To run a command, simply submit a comment.  For detailed instructions see here.

@bot-gradle clean

@bot-gradle test qfl

I've triggered the following builds for you:

QuickFeedbackLinux build


Export other data besides the workouts

Thank you very much for your application! Could you please expand the application by adding functionality to export other data besides the workouts. Such additional data worth exporting is weight measurements, etc. Thank you again!

I agree with you. In Zepp and Mi-Fit application, there is no possibility to have data of swimming pool activity. We have access to summary data but not timing of every lap... I will try to find this, I hope someone could find the good variable to extract

@maximmasiutin Unfortunately, I no longer use a Mi Band (and Zepp), which somewhat limits my motivation to continue expanding the scope of this application. After some digging, I found an old script which synchronizes weight measurements to Garmin Connect, here is the relevant endpoint if you'd like to experiment with a data exporter:
from datetime import datetime

from httpx import AsyncClient

async def get_weight_records(last_weigh_in_date: datetime, token: str, user_id: str) -> list[dict]:
    async with AsyncClient() as client:
        resp = await client.get(
            url=f"https://api-mifit.huami.com/users/{user_id}/members/-1/weightRecords",
            headers={
                "apptoken": token,
                "appPlatform": "web",
                "appname": "com.xiaomi.hm.health",
            },
            params={"fromTime": int(last_weigh_in_date.timestamp()) + 1},
        )
        resp.raise_for_status()

    return resp.json()

I hope it helps!

thanks you but I want to extract the lap or distance from a swimpool activity because I  havent  any GPS indication.
At the end of the activity, the summary indicate the number of rotation  but not the time of each rotation. I think, these data exist as the heart rate or other things, but I dont find the good variables.
Do you think is it possible to extract the whole data of each workout  ? Or to find the distance (or lap) parameter ?