Stack overflow for deep expressions in bincode serialization/deserialization

With rust code containing deep expressions, hax fails with a stack overflow. For example:
fn main() -> i32 {
        1 + 1 + 1 + 1 + 1 + 1 + 1 +
        1 + 1 + 1 + 1 + 1 + 1 + 1 +
        1 + 1 + 1 + 1 + 1 + 1 + 1 +
        1 + 1 + 1 + 1 + 1 + 1 + 1
}

The stack trace suggests that this has to do with serialization/deserialization with bincode that was introduced in #743. However this problem cannot be reproduced with the merge commit of #743 itself (but it loops for a long time). The version of bincode hasn't changed since then so there might be an interaction with another later change that triggers the stack overflow.

cargo hax json works fine for me. So the frontend seems fine. Only when extracting F* it fails for me (actually it just doesn't finish before I kill it).

@franziskuskiefer You were right, the problem is in the engine. It happens in phase specialize where in some cases recursive calls are done twice over the same thing resulting in exponential complexity.
I don't know why this started making Rust stack overflows instead of just taking a long time.

It turns out this fix doesn't improve the original problem. The example of deep expressions was only an attempt to minimize that had a different problem.
My current understanding is that when we have a rather big (or deep) module hierarchy, the items in HaxMeta contain a lot of data, and the data for a submodule seems to be duplicated in every parent module. There is no problem for encoding this with bincode, but decoding uses recursion and gives a stack overflow.
I tested a few serialization libraries and most don't support recursive types. Among those that do it seemed that bincode was one of the most efficient (I found none that seemed to behave better with deep recursive data).

I think we should bisect here, to understand exactly what went wrong and when

The result of bisecting is that the first "bad" commit is 748c488

Removing is_local from the DefId struct (on the latest main) does fix the problem. It would be really surprising that just this boolean field would make bincode crash. We tried to compare what else is different a json version of the HaxMeta object (that we save with bincode). There are some differences but it is hard to analyze as the file is very big. Minimizing the example is also very hard.

Is the recursive call on the side of hax or inside bincode? If it's in hax you can use stacker to dynamically allocate more stack at each recursive call. This is what charon and rustc use:
// This is the amount of bytes that need to be left on the stack before increasing the size.
// It must be at least as large as the stack required by any code that does not call
// `ensure_sufficient_stack`.
const RED_ZONE: usize = 100 * 1024; // 100k

// Only the first stack that is pushed, grows exponentially (2^n * STACK_PER_RECURSION) from then
// on. This flag has performance relevant characteristics. Don't set it too high.
const STACK_PER_RECURSION: usize = 1024 * 1024; // 1MB

/// Grows the stack on demand to prevent stack overflow. Call this in strategic locations
/// to "break up" recursive calls. E.g. almost any call to `visit_expr` or equivalent can benefit
/// from this.
///
/// Should not be sprinkled around carelessly, as it causes a little bit of overhead.
#[inline]
pub fn ensure_sufficient_stack<R>(f: impl FnOnce() -> R) -> R {
    stacker::maybe_grow(RED_ZONE, STACK_PER_RECURSION, f)
}

If it's in bincode then you should report it upstream, that sounds like something they should be handling gracefully.

Yeah, we wanted to use stacker as well, thanks for the tip! :)
But there are a number of odd things around this bug, we need to understand exactly what's going on there!
And I'd like to address #1001 in the same time :)


Bucket_logging: Change log object name format.

At present we are creating log objects and giving them name based on log_prefix and log_bucket.
There are some constraint to that and one is that a user can set same log bucket and log prefix for two different source buckets. That will cause logs from two different source buckets to go to same log objects.
In addition to that, source bucket name in log file name will help to get it easily without processing log file.
This source bucket name can be used to get more logging configuration, which can be helpful in further processing.

Note: As we are not providing log_prefix in log object name, it should be retrieved and processed from system_store by bg service at the time of uploading objects on log bucket.

will resend it again


Missing Large (Nested Virtualization) size mapping

Missing Large (Nested Virtualization) size mapping from azure-docs/articles/lab-services/classroom-labs/administrator-guide.md /cc @EMaher

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 191dec24-ed36-2aea-3384-dfeb69c6d1c8
Version Independent ID: c3cc622e-c92e-feb0-2835-551ddd601cf3
Content: Azure Lab Services - Administrator guide
Content Source: articles/lab-services/classroom-labs/administrator-guide.md
Service: lab-services
GitHub Login: @spelluru
Microsoft Alias: spelluru

@francedot , Thanks for brining this to our attention. We will investigate it further and update you shortly.

@francedot - Thanks again for bringing this to our attention. We have updated the Doc and it should merge once approved, and We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Add metrics for Shelly 3EM neutral

Thanks for the exporter. I almost gave up (and wrote my own) but then I discovered this exporter and it did exactly what I needed, except missing the neutral current metrics. But even they were already in the Shelly API part so adding them was easy. One small ask: would you add a link to the archived prometheus exporter project README; it would have helped me discover this current version.
I figured it's better to not mix it with the normal current emeter because it's also separate in the API and it's more logical to get normal current readings just by selecting the metric, as the use case for neutral current is a bit different.

Thanks for the exporter. I almost gave up (and wrote my own) but then I discovered this exporter and it did exactly what I needed, except missing the neutral current metrics. But even they were already in the Shelly API part so adding them was easy. One small ask: would you add a link to the archived prometheus exporter project README; it would have helped me discover this current version.
I figured it's better to not mix it with the normal current emeter because it's also separate in the API and it's more logical to get normal current readings just by selecting the metric, as the use case for neutral current is a bit different.

Thanks for the PR. FYI: I'm currently vacationing and can't review until the end of Feb. What README do you think I should adjust?

No worries, I am already running version I built myself so take your time.
https://gitlab.com/cimnine/shelly-prometheus-exporter is what I found from Google and based on README I realize it's an earlier version of the same codebase but superseded by this.


parcellite: fix GTK theme

The gtk.theme option is not picked up due to hardcoded XDG_DATA_DIRS.
Since icons do show up without this environment variable in the service unit I don't think it is necessary (anymore). According to the basic test we hope XDG_DATA_DIRS is set anyway.
cc @gleber

Yeah, seems these could safely removed. As I mentioned in the line comment I think you can remove PATH as well. If you do then you could rename the commit to

parcellite: remove obsolete backward compatibility fixes

Nice, I kept it to err on the safe side, but this is cleaner.
I was going to merge this myself to spare you a pointless review, but now I'm uncertain because the Travis build failed. It seems that's not an issue to address in this PR though.

Yeah, the CI error is unrelated. I'm not certain what it is about but asked in the #nixos IRC channel a while ago, hopefully somebody knows.

I rebased the commit to master in fdd65e5fada5883f4a8a07abc9aa6160e15a22e7. Note, I wrapped the commit message to make it a bit less wide.


fix(masterbots): google signIn

Summary by Sourcery
Fix the Google sign-in process by implementing a signIn callback that checks for existing users and creates new users if necessary, ensuring proper user authentication and database integration.
Bug Fixes:

Fix Google sign-in functionality by adding a signIn callback to handle user authentication and creation in the database.

Summary by CodeRabbit

New Features

Enhanced authentication system to support dynamic user creation when signing in with Google.
Added a provider field to the token structure for better identification of authentication sources.

Improvements

Improved user management capabilities by checking for existing users during the sign-in process.

 Andler Development Support - September 2024


add environment variables parameter

I tried to set environment variables at the start of $command, but then the string fails the validation that checks that a full path is being used.

but if using a full path is not a hard requirement (it isn't), how about relaxing the string validation instead of adding a new parameter?

The check I will be running requires sudo as well. A full path for use with sudo is a hard(er) requirement, right?

It should only validate the full path if needs_sudo I suppose.

Closing this PR. Issues with the interplay between sudo/environment variables would make this too complex. Instead, a wrapper script will be written for this specific case I have.


Test failures

Checklist:

[ ] I have read and followed the contribution guidelines.
[ ] I have read and followed the how to open a pull request guide.
[ ] My pull request targets the main branch of freeCodeCamp.
[ ] I have tested these changes either locally on my machine, or GitPod.

Closes #XXXXX

Hi @liricooli @nitayH @brittf19
We understand that you are working on a fork of this project. Unless you are working on improvements, we would appreciate it if you avoided creating PRs (even accidentally) to this repository. It creates unwanted notification noise, and CI runs.
Here is our code of conduct: https://www.freecodecamp.org/news/code-of-conduct. We enforce this strictly.
While what you're working on may not be spam yet, the notification noise annoys our moderators and community.
Please be mindful of this :)
If you want to work on a "copy" of this repo, GitHub offers way see: https://docs.github.com/en/repositories/creating-and-managing-repositories/duplicating-a-repository
Thanks.


Turn on moderation for new users for docker-user

Spam is becoming an issue on docker-user - this actively damages the value of the list and the community
https://groups.google.com/forum/#!topic/docker-user/axfrk9fa8oA
Not sure how often core team read docker-user so filing an issue to turn on new member moderation - https://support.google.com/groups/answer/2627595?hl=en
This would require regular commitment from the community manger/team to regularly check the queue.

It's common in other projects to give community members in other timezone rights to moderate posts and I've seen this work well.  I'd be happy to do so in the UK timezone.

cc @spf13

Thanks for bringing this to our attention. We've been working on a better solution for our users that will not only help to keep spam out, but will also enable community members to take more ownership of the channel. In the meantime we should turn on new member moderation and clean this mess up.

I generally really don't comment with just +1 comment, but I'll do an exception here. Many users (including me) have already asked on the ML that such spams be moderated, and the spammers be kicked, to no avail. Thanks a lot for cleaning this quickly.
As one of the last mails said, simple solution:
http://www.xaprb.com/blog/2009/09/13/stopping-spam-on-google-groups/
Thanks in advance

It keeps coming. Please... Moreover almost always from the same ones...

@spf13 can you audit the list and ban the current spammers.
https://groups.google.com/d/msg/docker-user/YcKgXKuwKiw/5q9t2TVPS1wJ

+1 to moderation

I actually don't have access, but I've already alerted the people that do and asked for access myself so I can take care of it. I anticipate it will happen soon.

+1 to moderation

On the subject of the docker-user google group, how about that intro? My own personal opinion, but the thing is overly huge, and doesn't even belong there. It reads like a sales pitch for docker, but that's what the docker web site is for (docker.com). It also hasn't been updated in ages: the first line still references http://github.com/dotcloud/docker.
Can we just get rid of it, or scope it so that it's something relevant to the mailing list, such as what is on-topic, what should be a github issue, etc?

I'm now an admin. I've enabled moderation of new members and will work on the other suggestions here over the next few days.

@spf13 Beware that there seems to be a lot of already registered users that are actually only spammers. I don't know if you monitor docker-user, but there was another spam 5 minutes ago with the title "Best ETL TESTING Training", and that's indeed not the first mail from that<EMAIL_ADDRESS>

Thanks @Batmat. I've banned the ones I saw. I've also been trying to create a new "moderator" role that we will give out to a handful of community members so they can police this as well. Unfortunately Google Groups has a bug that seems to prevent me from creating any roles. It says it's saved, but then doesn't actually create it. I've tried a few times now.

@spf13<EMAIL_ADDRESS>to ban, don't know if you saw it.


An idea and/or request #2.

For another future update (and maybe for UI changes), something nice as an alternative to the switch '-w --date', would be to also add a sub version of it (or maybe combine it with the current '-w --date' switch), by putting a number based on the first post as a 1, and from there counting until last post ever.
But the previous idea might be a bit over-complicated, and an easier to develop idea would be to instead add the post-number of each specific post to the folder.
Example:
User = https://kemono.party/patreon/user/71224239
Post 1 of user = https://kemono.party/patreon/user/71224239/post/66779297
Post 2 of user = https://kemono.party/patreon/user/71224239/post/67020419
Post 3 of user = https://kemono.party/patreon/user/71224239/post/66441109
From here, we can utilize the number at the end of each link onto the folders
Folder of post 1 = 66779297 [post name goes here] [date could go here]
Folder of post 2 = 67020419 [post name goes here] [date could go here]
Folder of post 3 = 66441109 [post name goes here] [date could go here]
This way, we can in a single click arrange an entire artist's earliest to oldest post by name, without any need of alternate searching-by-words/date
(this is especially useful for artists that have hundreds of posts to search trough).
(PD: Also, thanks on the advice of using 7zip to erase the bugged folders from issue #20, it helped and Winrar also works to do the fix... AND on the quick response & fix on the previous issues)

It doesn't seem to difficult to implement, I'll probably have something ready around next week.

Small update from me. I know close to cero in programming, but after a bit of tinkering, I found this really nice work-around for the issue of mine using a freeware software called "Rename Master"  (http://www.j oejoesoft.com/cms/s howpage.php?cid=108) (remove spaces after 'j' and 's') (as always, please make sure whatever is one downloads doesn't contains malware, use your antivirus, and be careful from wherever you download. I did mine, and consider it safe.)
After opening the software:

Go middle-left, click view, activate 'files/folders' to view folders also.
On the left panel, click on 'swap', write ' Kemono' WITH EMPTY SPACES.
On top left, click 'Rename'.

Boom. Done. From using this current switch as is now, into Rename Master, all the folder names will begging with the date (year/month/day is the best form of date for path files, TY OP for picking it) essentially performing the same utility as my idea for adding the post number at the start of each folder name.
I make this update because I'm not sure how much longer Kemono will stay up online, as it will for certain end like coomer party getting taken down, soon or later.
As always, thank you very much OP, the day I finally get myself organized enough to make a Ko-fi account, I will be giving you one for certain, Cheers.

Awesome solution.  I haven't had any time as of recently to work on this project but I may have a fix to what you suggested in <IP_ADDRESS>.a branch but I haven't had the chance to verify that works completely. It does seem Kemono does have some safeguards or some plans to prevent shutdown such as the kemono.su mirror link but I do hope that it stays up. There isn't anything else like it from what I can see.

I have a similar issue for some names folders, that KMPD - <IP_ADDRESS>-pre create a folder name that end with a space when unzipping (not all), it happen in some particular example post like:
Example:
https://kemono.su/fanbox/user/16456081/post/3851092
(Post that have a folder name that end with a space)

Thanks for bringing this up It has been fixed and you can download it from here: #25.

Fixed in releases <IP_ADDRESS>.1 and <IP_ADDRESS>


Create SunKit.yml

Since @retroactive keyword has been added on latest commit, it is necessary to run the build through an Xcode version 16 or higher. A new Xcode version will be forced if new keywords/function will not be anymore compatible with Xcode 16.1.

@seldon1000  I think we shall keep @retroactive keyword in order to not have any warnings.  It's ok to stick with Xcode 16.1 to run test and build.

The @retroactive keyword was added to Date's extension of Strideable to silence a warning in Xcode 16+ releases. Given that it is unlikely that warning will cause any trouble, we could consider reverting that change instead of merging this PR. This way, we could theoretically keep retro-compatibility with previous versions of Xcode.
Otherwise, if you think it's appropriate to move forward with this PR, I'll proceed approving your changes.


fix: lateral view alias解析错误

LATERAL VIEW ... AS aparse成SQLStatement，再toString后变成LATERAL VIEW ... a， 丢失了AS。
Spark中，LATERAL VIEW的语法如下：
LATERAL VIEW [ OUTER ] generator_function ( expression [ , ... ] ) [ table_alias ] AS column_alias [ , ... ]

Druid大概是把column_alias解析成了table_alias。

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

ci构建失败
Error:  src/main/java/com/alibaba/druid/sql/parser/SQLSelectParser.java:[30,8] (imports) UnusedImports: Unused import - java.util.ArrayList.
请在本地 运行构建命令 mvn clean install 进行验证，确保验证通过。

删除UnusedImports后，在本地运行mvn clean install， 构建成功

Codecov Report
Attention: Patch coverage is 60.00000% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 58.68%. Comparing base (920c53a) to head (5f40626).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##             master    #5817   +/-   ##
=========================================
  Coverage     58.67%   58.68%
- Complexity    27920    27926    +6
=========================================
  Files          1341     1341
  Lines        128289   128288    -1
  Branches      24209    24209
=========================================
+ Hits          75278    75287    +9
+ Misses        43623    43614    -9
+ Partials       9388     9387    -1

Files
Coverage Δ

.../com/alibaba/druid/sql/parser/SQLSelectParser.java
59.40% <50.00%> (+0.20%)
:arrow_up:

...alibaba/druid/sql/visitor/SQLASTOutputVisitor.java
68.74% <66.66%> (-0.02%)
:arrow_down:

... and 7 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 920c53a...5f40626. Read the comment docs.


feat!: add client observability configuration codegen

Issue #

awslabs/aws-sdk-swift#696
Description of changes

Note: dependent on DirectedCodegen implemented in #681 and awslabs/aws-sdk-swift#1405
Note: coupled with awslabs/aws-sdk-swift#1425
Adds client observability configuration codegen. Notably, config.logger is replaced with
config.telemetryProvider, which provides a default loggerProvider.
Changes:

Breaking change: replace config logger with telemetryProvider
Breaking change: codegen client configuration telemetryProvider
make top-level default telemetry impls public
add telemetry runtime plugin

Scope

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)

Breaking change included

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Closed in favor of #689


Add KIT phantom dataset

I've uploaded the data to OSF but not sure how to proceed. Would appreciate any help!
What does this implement/fix?
Add new dataset - KIT phantom data.

[x] mne/datasets/phantom_kit/*.py to add some dataset functions
[x] mne/datasets/config.py to add hashes and URLs
[x] mne/datasets/utils.py to add it to our private download-all-data function
[x] mne/datasets/__init__.pyi to add an import of the new data function
[x] mne/utils/config.py to add a new config var for the path
[x] .circleci/config.yml to cache the data
[x] tools/circleci_download.sh to get CircleCI to download the data when the tutorial changes
[x] doc/api/datasets.rst to document the new data_path function
[x] doc/overview/datasets_index.rst to briefly describe the dataset
[x] doc/changes/latest.inc to mention your addition (ideally this PR would also include a tutorial
[ ] tutorials/inverse/95_phantom_kit.py that shows that source localization works with this dataset

It looks like a good start, you'll also need to add the dataset information (URL, hash) to the configuration file: https://github.com/mne-tools/mne-python/blob/main/mne/datasets/config.py

Looking back at a recent dataset addition d757026 I've edited your top comment to add checkboxes for the files that you'll want to make equivalent changes in (including config.py that @mscheltienne mentioned). There are a lot but looking at what's there hopefully it's easy enough to follow the patterns.
If you'd like help with this part don't hesitate to ask -- I can push a quick commit to do everything except adding the new tutorials/inverse/95_phantom_kit.py. Hopefully you can create that, look at existing tutorials/inverse/*phantom*.py files for examples!

Thanks a lot for the help! I've updated most of the files, except for "doc/overview/datasets_index.rst" which I couldn't locate.
Happy to try creating the tutorial - but I may need to come back and do this a bit later!

Looks like that file has moved to doc/documentation/datasets.rst

Thanks - now updated.
Some of the checks are failing and I'm not sure why - could you help me with these?

Thank you @larsoner !
I have fixed the login for CircleCI - please let me know if this still causes issues.
I'm still happy to attempt the analysis, but it might just take a while as I have quite a lot going on at the moment. To test things locally, how do I get it to run the version of code I'm working on rather than the mne version already installed on my computer?
Alternatively, if you are happy to do the analysis, that'd also be great! I think there are 49 dipoles activated in sequence within each "train", with each activation being a sinusoidal signal at 11Hz (see MISC 017). Please let me know if you require more information.
Do we need to get the "ground truth" dipole locations in order to verify the accuracy of source localisation?

It would improve the tutorial to know the true locations, yeah

@JD-Zhu based on the symmetry I'm assuming this is correct:

https://output.circle-artifacts.com/output/job/4233b1c2-26a5-49f0-aecc-d996027fd63e/artifacts/0/html/auto_tutorials/inverse/95_phantom_KIT.html#sphx-glr-auto-tutorials-inverse-95-phantom-kit-py

But it would be good to get the dipole locations if possible. And we should really fix the dev_head_t if possible as well (see comment above).

This is awesome! Thanks @larsoner
The dipole locations are available in this paper (see Table 3):
https://pubmed.ncbi.nlm.nih.gov/25985908/

The dipole locations are available in this paper (see Table 3):

Awesome, I'll look soon!

Sorry, errant close!

Okay I think this is good enough:
Average loc error: 1.7 mm
Average ori error: 9.6°

Just pushed one tiny commit to separate the fit_dipole output from the print(<dipole results info>) stuff so it's easier to see the average errors in the output.
Ready for review/merge from my end @drammock !

Thanks @JD-Zhu !


Add no-chunk option in series API

Signed-off-by: yeya24<EMAIL_ADDRESS>

[x] I added CHANGELOG entry for this change.
[ ] Change is not relevant to the end user.

Changes
Fixes #1822 #1774
Add a no-chunk bool in series API to return labels instead of the whole series. Only in querier /api/v1/series API, this option will set to true, otherwise it set to false.

Verification

I have already added test cases for all stores.
For the response time using no-chunk, I tested in my local env.
Using Thanos v0.9.0
curl -sw '%{time_total} %{size_download}\n' -o /dev/null 'http://localhost:10904/api/v1/series?' -d 'match[]={job="prometheus"}'
0.297961 81680

Using this PR
curl -sw '%{time_total} %{size_download}\n' -o /dev/null 'http://localhost:10908/api/v1/series?' -d 'match[]={job="prometheus"}'
0.024901 81680

@bwplotka  Thanks for your detailed review. I have addressed all of your comments.


Fail summarization and retry when there are pending ops

Description
This change introduces a new summary validation - If there are pending (unacked) ops after a summary is generated, the summarization attempt will fail. Summarization will be attempted 5 times when this error is hit.
To ensure that this does not cause summarization failures to corrupt documents, there is a fallback to succeed summarization in the final attempt if the feature flag Fluid.Summarizer.SkipFailingIncorrectSummary is enabled.
The above is achieved as following:

A new summarize option finalAttempt is added and passed to ContainerRuntime::submitSummary. It uses this to determine whether to pass summarization even when there are pending ops.
RunningSummarizer::trySummarizeWithRetries does 4 attempts in a loop followed by a final attempt where finalAttempt option is set to true.

Why are pending ops a problem?
If there are pending ops during summarization, it can lead to incorrect summaries and eventual consistency issues.
For example, if a summary is generated at seq#100 and it has pending ops, the summary will contain the contents of these ops. which will be sequenced at >100. If we compare a client that loads from this summary at seq#100 to a client that starts at 0 and processes op till seq#100, their data will be different breaking eventual consistency.
Additionally, the pending ops may never be sequenced by the server due to any number of reasons but the summary will contain these changes which makes the sumamry incorrect.
Reviewer guidance
Currently, in tests, its almost impossible to know when a summarize attempt fails without some nasty hacking. The only thing summarizer does when an attempt fails is to log a telemetry and we cannot rely on that. I am planning to add events when a summarize attempt completes and that can be used to write tests. Sending this PR out to get feedback while I work on that.
Todo

[ ] Add test to validate this functionality works as expected.

AB#4904

i'm a little worried about a change like this. i understand that local changes in a summary are very, very bad, but i also worry it happens unintentionally, like due to auto-upgrade like code. In that case won't this just block all summarization? blocked summarization also leads to many other issues? Do we, or can we just wait for ack of pending ops, rather than fail?

i'm a little worried about a change like this. i understand that local changes in a summary are very, very bad, but i also worry it happens unintentionally, like due to auto-upgrade like code. In that case won't this just block all summarization? blocked summarization also leads to many other issues? Do we, or can we just wait for ack of pending ops, rather than fail?

I have a PR to do just that - https://github.com/microsoft/FluidFramework/pull/16465. But this doesn't guarantee anything because ops could be sent during summarization process.

The scenario i worry about is some code that is like "create new things if it doesn't exist", for containers where it doesn't exist it will never exist upfront until a summary succeeds, but this change will prevent summarizing in that case as well. I know the "create new things if it doesn't exist" isn't a good pattern, and in some cases we have better alternative, but ideally the system still works in spite of bad patterns, rather than bad patterns griding the system to a halt.

I am aware of scenarios like this which could summarization to a halt. That's why there are 2 safety nets in place:

Summarization will be retried 5 times when an error like this happens with delay in between to let ops be processed. This will add some overhead but it is not a common scenario and we will measure how much disruption this causes before we enable it by default (it is under a feature flag and not enabled by default).
There is a feature flag that can be turned on to let summarization succeed in the 5th retry attempt even if there are pending ops. Customers who can't (or won't) fix these bad patterns can use this to make progress with summarization.

i think the PR to wait need to go in first. I also don't think we or our partners track summary health well, which means by the time we realize something is wrong, and we need to turn on the config, we're probably already in a bad state? Do you have a plan to address that?

i think the PR to wait need to go in first.

The PR to wait is more of an optmization where we prevent doing summarization if already know there are pending ops. Even if that one doesn't go in, it will be handled by this PR. Having said that, the plan is to have both in place before any of this is enabled.

I also don't think we or our partners track summary health well, which means by the time we realize something is wrong, and we need to turn on the config, we're probably already in a bad state? Do you have a plan to address that?

The plan is to make partners aware that this is happening and let them have things in place before its enabled. We are already working with them to stop local changes in summarizer (which is slow progress) and this ties in with that effort. IMO, the feature flag should be enabled to begin with so we can get data while not having documents stuck because of this. If things look stable enough, the feature flag can be disabled. I believe this is what we will go with.


Added merge_with class method for merging two multiple test suite results and tests

Added merge_with class method for merging two multiple test suite results and unit tests.

Coverage increased (+0.04%) to 91.021% when pulling a5da820d5f350b6e9a5880f00e1725645af2c924 on expan_data_restructure into 1aa47f507fa461811435f2005e97aa8809b373a2 on master.

Coverage increased (+0.04%) to 91.021% when pulling a5da820d5f350b6e9a5880f00e1725645af2c924 on expan_data_restructure into 1aa47f507fa461811435f2005e97aa8809b373a2 on master.

Coverage increased (+0.04%) to 91.021% when pulling a5da820d5f350b6e9a5880f00e1725645af2c924 on expan_data_restructure into 1aa47f507fa461811435f2005e97aa8809b373a2 on master.


sv_maxusrcmdprocessticks  incorrect value

tell me pls about ur value in the sv_maxusrcmdprocessticks according to my knowledge, this value can be 17 ( bypassing send_move )
so the best solution would be to change this value to 17?

afaik you need to do stuff to bypass it no? this is also the value valvo use so...

afaik you need to do stuff to bypass it no? this is also the value valvo use so...

yep but i think its better to do it now than later

Author

I mean yeah i suppose i think it would prob be better to have it as a constexpr/define somewhere however though and have a comment saying about how you can bypass or w/e considering this is just a base


RFC logo

REANA has been without a proper logo for too long.
We have been musing about various options on paper, basically expressing "re"-usable "ana"-lyses concept, keeping closeness to CAP, and introducing some leading illustrative glyph or picture that could resemble computational workflow concept (=triangled graph) or the reuse concept (=recycle arrow) and the like.  The leading glyph/illustration could be later used for favicon.
Here is the representation prepared by @jbenito3:

Are we happy with the idea and with the prepared logo "as is"?  Any minor suggestions regarding  appearance? Any major suggestions with conceptually different proposals?

I really like the result! I am all in for going with this version 😄

I like it too, I agree with @hjhsalo regarding the kerning.
Two nitpicky fixes would be:
to center the top red square with the intersecting lines, there is a little less on the left side:

and to align the lines of e and a (not possible without chaning the a -> making it incosistent with CAP, but looks a bit off right now):

Thank you all for your feedback. Just one comment regarding spacing between letters: it was done on purpose that re is connected (like workflows are connected through steps) to ana. ana was kept as is in CAP logo as a reference to the service. So the metaphor in there is: the "connected" re sort of represents the connected workflows (steps) towards our CAP analyses.
Separating them with the same spacing everywhere will give yet another classical textual logo (personallly I tried it and I didnt like it...).

More options following some of your ideas:

FWIW from the first batch I like the option number one, from the second batch the option number two.
(Perhaps making the leading image ten per cent smaller and slightly more separated from the text, so that it would stand out without blending with the main logo text...  Otherwise we risk people seeing words like "areana" for the first batch and "oreana" for the second batch?)
@reanahub/developers Any other proposals, ideas, observations? Let's aim at closing this issue later this week...

I also vote for number two of the second batch

I also like better option number two from the second batch

Oh wow, I like how 'ana' has it's own font 😁
I vote for no. 4 in the second batch.

I am not 100% sold on the colours (but I don't really have better ideas 😅)

Also maybe 'r' and 'e' can have a bit of space in between?

Ok, the third batch with somebody else's ideas:

Despite them showing great creativity... I feel that they all look too familiar and evoke the wrong images - the third set are ying/yang, the second set are recycling. So the only one that looks new/unique and therefore distinctive for a new service is the first one, the branching workflows...

It is true that they do look familiar. I don't find it too bad though. I like first batch, 1st one and Ying/Yang

It can be considered a good thing if the logo looks just a little bit familiar, since it may make people feel comfortable with the product via gentle subliminal messages :wink:  Obviously, if the similarity is too obvious or too crude, then it wouldn't work so well.
E.g. from the third batch the second logo may evoke also two swimming koi carps around each other, or two gravitating galaxies, as it were... so the resemblance to yin/yang is more on the gentle side and the logo might work. While the first logo may look perhaps too close to a Pepsi, and the sixth logo too close to a tennis ball, so these logos might work less well.

I like some general/common aspects of all the logo: in particular nice colors and size/proportions, good job!
I also like a lot the "recycle" idea which is well espressed by the recycle logo (in one of the sets) and the font for "re".
The "analysis" idea unfortunately does not seem surfacing to me. For that I would consider:

replacing the dot in the center of the recycle icon with a glass flask (e.g.
 ) or a magnifying lense (which reminds of Invenio, too)
using for "ana" a more "scientific looking"(?) font (not sure what to suggest, perhaps Century)

Great point Flavio - whilst a pure recycle symbol looks too "familiar", a modified one with a science symbol in the middle would indeed match Tibor's "just a little bit familiar" criterion :)   So it could be a flask, or an atom, or anything else scientific looking - what's the most familiar science symbol ?!?

I think that first batch first version still delivers the clearest message, expressing how REANA allows physicists to produce/reproduce scientific outputs, using the abstract idea of a graph (even if it is not all what REANA can do, but it is its essence).
However, the recycle batch refers to what REANA does in a very generic or "familiar" way as mentioned before. Besides the fact that this could be fixed by Flavio's idea, I think that it refers to the result, whereas the graph (or first batch first version) refers to the process, which in my opinion is more transcendental, a workflow/graph helps to create better structured analysis making reproducibility easy.
To sum up, it is more a "philosophy/way of working" versus a "result/what you get" difference. For example, you could use REANA to make your analysis reproducible and make all the analysis in one step, everything in one go. It would be reproducible, REANA will use the right data|code|environment, but it will be more difficult to understand for a new collaborator.

We seem to like the textual part of the logo, so I guess this one is settled. I propose to simply deploy the textual part of the logo today while the graphics part of the logo is being discussed and various ideas graphically polished. This will enable deployment, the textual logo looks already nice, and using it will give us time to explore the other options in calm next week.
(Besides we could even consider sticking to the textual version only :wink: like Google and others)

@tiborsimko I did a pull request with your suggestion. In my eyes I miss the logo, maybe I grew too attached to the simple workflow logo, and also now we are even closer to nextflow...see:

I like it without logo, in fact. More zen.

Closing the RFC as the textual logo is being used.  In case we would like to revive leading graphics, we can open a new one.


Question about WebNLG (v3.0) Test Data Size Discrepancy

Thank you for sharing your work. I have a question regarding the use of the WebNLG (v3.0) dataset in your work. In your paper, it is mentioned that the test data contains 1,165 text-triplet pairs. However, when I checked the official WebNLG repository in the file semantic-parsing-test-data-without-refs-en.txt, I found 2,155 text entries.
Could you please clarify the reason for this difference? Did you apply any specific filtering or processing to the dataset that resulted in the smaller test set? Thank you for your time and help.

Thank you for raising the issue and sorry for the late reply.
The 2155-entry file as you mentioned, if I remember correctly, is the split without references. So I was not able to evaluate with that set. The one with references should have a size  of 1165. I can double check it later.

Thank you for your reply @bzhangj13zzz , but I still don't understand your explanation. I checked both files (semantic-parsing-test-data-with-refs-en.xml and semantic-parsing-test-data-without-refs-en.xml), and I found 2155 entries in both files. Could you please confirm if there might be other factors contributing to this discrepancy? Or could you verify whether there are different versions of the test set? Thank you for your assistance!


Do results indicate area-weighted averages?

I have a question about running a basic recipe as attached, which masks out the sea and generates annual averages of the two variables. Are these averages area-weighted, perhaps during preprocessing? That is, is the fact that the cells differ in area (depending on latitude) taken into consideration?
land_reference.txt

hi @hahsan1 any of the mask_* preprocessors do not apply any weighting, they simply apply an fx mask on the input data and return the original data but with a boolean mask attached to it (where True represents the masked off points that fall in the area that needs to be masked off - either land, seas, or ice). The fx masks are specific to the model and are available as standard CMOR-compliant variables. Can you maybe elaborate a bit more on the weighting that you're interested in please?

yes, that's the one! You can mask before so you keep only the desired areas then run area_statistics - that uses a method called compute_area_weights that computes the area weights then asses that as grid weights during the area statistics computation :+1:

So I tried running with area_statistics instead and get this error in the log file: iris.exceptions.CoordinateCollapseError: Cannot collapse a dimension which does not describe any data.
Any idea what could be causing this? I have attached the recipe and the log files.
land_reference.txt
log.txt
main_log_debug.txt

you need to reverse the order of your preprocessors: first do land/sea masking, then area statistics. Area statistics collapses the data on [longitude, latitude] and returns time- or time-plev data, so the masking can not be performed after that since the horizontal dimensions have been collapsed

I tried reordering it like this but still get the same error
preprocessors:
  preproc_nolev:
    regrid:
      target_grid: 1x1
      scheme: linear
    mask_landsea:
      mask_out: sea
    area_statistics:
      operator: mean

@hahsan1 the error happens in the emissions_mip/initial_analysis-giss.py diagnostic: have a look at the input data of that diagnostic and what the diagnostic needs to do with it please, I don't have access to your diagnostic and wouldn't know what happens there :smiley: Just looking the diag's log trace I see it crashes at:
  File "/pic/projects/GCAM/Emissions-MIP/ESMValTool/esmvaltool/diag_scripts/emissions_mip/initial_analysis-giss.py", line 60, in <listcomp>
var_list = [ESMVariable(var_dict).get_area_statistic('mean') for var_dict in dict_list]

which suggests to me you are trying to calculate an area statistic, inside your diagnostic, using preprocessed data files that have had area statistic computed already in the preprocessor - so either you don't do it in the diagnostic and do it in the preprocessor, or the other way round, definitely not twice: in the preprocessor and the diagnostic (because the diagnostic will ingest data that has already had lat and lon collapsed from the area statistic performed in the preprocessor). But as I said, I don't know what's going on in the diagnostic, have a look :+1:

Thanks for pointing that out! I see this chunk in the script you mentioned:
    # Iterate over the variable dictionary and process each varible one-by-one.
    for esm_var, dict_list in var_groups.items():
        # Get list of ESMVariable objects.
        var_list = [ESMVariable(var_dict).get_area_statistic('mean') for var_dict in dict_list]
        common_emip_funcs.plot_timeseries(var_list, plt_config)

So it looks like we already compute the mean of the area-weighted cells for each variable.

indeed, but diag_scripts/emissions_mip/initial_analysis-giss.py is not a diagnostic in the main branch, so I assumed that is either the diagnostic you are currently developing, or it's someone else's development branch, if you could tell me what PR/branch that is then I can take a better look

You can find the diagnostic file here (this is our fork of ESMValTool)

cheers! OK so that diag is actually computing area statistics inside it, so you'd have to decide where to best do it - the diag is using the ESMValCore's area statistic functionality, so maybe just not use it in the preprocessor? Incidentally note that your fork is about 500+ commits behind the main, so I'd suggest you refresh it with the latest main and use the latest ESMValCore - there are a number of changes wrt fx/cell measures that impact masking and area/volume statistics :beer: Also, please feel free to close this issue whenever you think it's been solved :+1:

Thanks for looking into this! We will look into merging the new commits.


New --only-managed / --only-unmanaged flags

Q
A

🐛 Bug fix?
no

🚀 New feature?
yes

⚠ Deprecations?
no

❌ BC Break
no

🔗 Related issues
none

❓ Documentation
yes

Description
This PR adds 2 new flags that will output a different summary at the end of the scan.
The --only-managed flag should sum up what's under control only in your states:
Found missing resources:
  - vol-xxxx (aws_ebs_volume)
  From tfstate://terraform.tfstate
    - i-xxxxx (aws_instance.bar)
Found changed resources:
  From tfstate://terraform.tfstate
    - i-xxxxxx (aws_instance.foo):
        ~ instance_type: "t2.nano" => "t2.micro" (computed)
Found 4 resource(s)
 - 50% coverage
 - 2 resource(s) managed by Terraform
     - 1/2 resource(s) out of sync with Terraform state
 - 2 resource(s) found in a Terraform state but missing on the cloud provider

The --only-unmanaged flag should sum up what's NOT under control in your cloud provider:
Found resources not covered by IaC:
  aws_iam_policy_attachment:
    ....
  aws_iam_role:
    ....
  aws_iam_role_policy:
    ....
  aws_network_acl_rule:
    ....
Found 18 resource(s)
 - 11% coverage
 - 2 resource(s) managed by Terraform
 - 16 resource(s) not managed by Terraform

WARNING: I wonder if we should enforce to have only one of the three flags at a time --only-managed --only-unmanaged and --deep, once we agree on what I've coded.

Codecov Report

Merging #1359 (cfe9b82) into main (2116470) will increase coverage by 7.08%.
The diff coverage is 61.11%.

@@            Coverage Diff             @@
##             main    #1359      +/-   ##
==========================================
+ Coverage   75.14%   82.23%   +7.08%
==========================================
  Files         482      410      -72
  Lines       18122    15000    -3122
==========================================
- Hits        13618    12335    -1283
+ Misses       4214     2356    -1858
- Partials      290      309      +19

Impacted Files
Coverage Δ

pkg/analyser/analysis.go
92.89% <0.00%> (-3.45%)
:arrow_down:

pkg/driftctl.go
74.21% <ø> (ø)

pkg/analyser/analyzer.go
89.69% <50.00%> (-4.13%)
:arrow_down:

pkg/cmd/scan/output/console.go
95.25% <66.66%> (-2.08%)
:arrow_down:

pkg/cmd/scan.go
74.88% <83.33%> (-17.57%)
:arrow_down:

pkg/remote/scanner.go
86.25% <100.00%> (ø)

pkg/terraform/plugin_client.go
0.00% <0.00%> (-100.00%)
:arrow_down:

pkg/remote/aws/client/s3_client_factory.go
0.00% <0.00%> (-100.00%)
:arrow_down:

pkg/remote/aws/init.go
0.00% <0.00%> (-95.74%)
:arrow_down:

pkg/resource/azurerm/azurerm_image.go
12.50% <0.00%> (-87.50%)
:arrow_down:

... and 172 more

WARNING: I wonder if we should enforce to have only one of the three flags at a time --only-managed --only-unmanaged and --deep, once we agree on what I've coded.

I still haven't read the PR yet, but here are my first thoughts
I think that it's pretty clear that --only-managed and --only-unmanaged should be mutually exclusives.
But I don't see why we are talking about deep mode here ?
This is something else and it has nothing to do with manage or unmanaged notion.
For sure in a perfect world we do not want to enable deep mode and detail fetching for unmanaged resources, but it's another performance topic that is more complicated that we thought initally.

For the deep mode it's more about the fact that the --only-managed, from @sjourdan product point of view, should output drifted attributes. Thus, it does not make sense to do driftctl scan --deep --only-managed.
But I'm good having a simple --only-managed where deleted/managed resources are shown, and the deep mode will enable us to show drifted attributes.

Thus, it does not make sense to do driftctl scan --deep --only-managed.

I strongly disagree with that assumption.
People should be able to see missing resource without drift, exactly like they want to have only unmanaged.
We should keep those small notions separated, but if we want to address that kind of feature, a --profile flag may be more appropriate, and a profile may contains multiples switch to be able to run driftctl in a certain configuration: in our case both --only-unmanaged and --deep.
With --only-managed we are targeting a specific switch in the product (by the naming of the flag itself), we are not expecting it to do some magic under the hoods and activate other flags like deep mode, it will add a lot of mess IMO.
WDYT @sjourdan ?

Thanks everyone for raising the points.
Let me summarise the current state of discussion:

What's the use-case for --only-managed? Do users expect support for deleted resources only, or for changed resources also? That can be confirmed through more user testing if we want to double check user data already collected (current suggestion is based on feedback). --only-managed is supposed to target users who want to know if anything changed in any resource is managed by Terraform using many states, with the lowest access rights, outside of deployment pipelines, using a lightweight binary.
 AFAIK nobody reported such a need and I currently don't see a lot of value in just displaying deleted resources for managed resources.
Regardless of user expectations, what do we think is a good default for users in this situation: when --only-unmanaged does not need any more flag to be useful, we may offer a false sense of satisfaction for users thinking --only-managed is enough, when actually --only-managed --deep is needed to be really useful in this context. That said, both answers are acceptable!
A future --profile flag with combined configuration (one named profile expands to --quiet --output="json://console" --only-managed --deep). I love that idea.

Did I get the various threads right?

Regardless of user expectations, what do we think is a good default for users in this situation: when --only-unmanaged does not need any more flag to be useful, we may offer a false sense of satisfaction for users thinking --only-managed is enough, when actually --only-managed --deep is needed to be really useful in this context. That said, both answers are acceptable!

The problem is that enforcing --deep when using --only-managed can cause rate limit issues.
At that point the flag --only-managed will not prevent use to enumerate the whole remote cloud account. It's only filtering results in the analysis step.
By enabling --deep flag under the hood I'm afraid that it will bring a lot of frustration for users that may expect us only to read only resources from remote that are only defined in IaC (like tfplan). They may get some rate limits issues without really understanding why.
But we can document that and start gathering user feedback, but we have to keep in mind that the actual --only-managed flag is not really the same behavior than a Terraform plan, and may not work at all for users with a lot of mess in their remote (because of rate limits). In a more complete version we may want to avoid enumerating everything when --only-managed is used, but this will need more work and maybe starting gathering user feedback before is safer.
Anyway, if we decide to go with that, we need to switch the deep mode flag on in options parsing level when the --only-managed is passed instead than adding new conditions on existing if deepModeEnabled statements.

At that point the flag --only-managed will not prevent use to enumerate the whole remote cloud account. It's only filtering results in the analysis step.

This is indeed what's next in our todo list for this specific flag. The first iteration was on the output of it. And you're absolutely right about that we need to only read details of resources in our states and not everything.

Anyway, if we decide to go with that, we need to switch the deep mode flag on in options parsing level when the --only-managed is passed instead than adding new conditions on existing if deepModeEnabled statements.

Good catch, didn't think it that way (probably because I didn't think it was possible) but if we all agree to go down this road, I'm going to change it real quick.

I'm a bit confused, isn't this directly related to https://github.com/snyk/driftctl/discussions/272 ? Looks like this PR doesn't reflect the discussion in question. Or may be I missed another discussion somewhere else?

rate limit issues

Correct! Internal issue CFG-1599 was created to address this asap.

user feedback, documentation

Agreed! Limitations need to be clear, roadmap to remediate too.


[WFCORE-4600] Resolve problems with ByteBufferSlicePool usage that are affecting memory footprint and causing leaks

Jira: https://issues.jboss.org/browse/WFCORE-4600

Core - Full Integration Build 8900 outcome was UNKNOWN using a merge of 0ed5e2e680ab20eeddb33887be89f24125eedcf2
Summary: Canceled (Tests passed: 1405, ignored: 64; exit code 143 (Step: Build & test full (Maven)) (new)) Build time: 00:24:32


Leaking the API key?

Hey @whoiskatrin, nice project, congrats!
I was curious about its implementation and noticed that the openai API key is publicly visible. That's because the props from getServerSideProps are injected into the page. This might lead to unexpected billing costs If someone decides to abuse it.

Have you considered using an API route for the translation part where you could use the API key without exposing it to the public??

yes, finally fixed it, not an issue any more


#bodhi Undefined telemetry event CP_IMPRESSION

A telemetry event with id  CP_IMPRESSION is being generated from the Sunbird Staging portal. However, the API call to https://staging.open-sunbird.org/private/service/v1/content/data/v1/telemetry  is failing with the error unable to find event definition for eid :cp_impression"

Can you share the full event?


Map doesn't load for /register

The Map doesn't load for https://firefoxsprint.mozilla.community/register/ as there is a country (West Bank) that we do not have in our mapping list for the map icons.
Proposed solution:

Handle that error correctly
Show placeholder map for West Bank

@couci as we don't have a map icon for West Bank, I would show a placeholder (though that can get quite political). I'd propose the following: http://mapglyphs.com/middle-east-map
With the new design this won't be an issue anymore as we won't have those anymore.

I agree let's move forward with the suggestion added since this is temporary


Use py-tree-sitter-type-provider for TLA+ state parsing once it's able to output .py sources

See:

TODO.md in this repo
https://github.com/wenkokke/py-tree-sitter-type-provider/issues/93

@SnoozeThis https://github.com/wenkokke/py-tree-sitter-type-provider/issues/93

(https://snoozeth.is/W8sl7T9rPAs) I will wait until https://github.com/wenkokke/py-tree-sitter-type-provider/issues/93 is closed and then add a comment.


Just 1 port for all backend and frontend ?

here  is my idea

that run on 1 port to forward to cloudflare tunnel in just 1 port and no need for a subdomain
( that means need to code custom server for next for support unix port )

Close with reason python has not support new unix socket on window
info here
AF_UNIX comes to Windows blog


[release-4.8]Bug 1979012: Disable thick provisioning support from UI

Change feature guard annotations in CSV to diable thick provisioning from UI

Partial revert of: #1142
/assign @umangachapagain

Can we keep these changes only in release-4.8?
I have done the same in https://github.com/openshift/ocs-operator/pull/1257
I have already checked the same with Eran, we don't want to disable this for 4.9 as of now.

/hold

/hold cancel

/retest

/cherry-pick release-4.9


Fix #1. Workaround for angular not cleaning up routing events when module is destroyed.

See #1, https://github.com/CanopyTax/single-spa/issues/115, https://github.com/angular/angular/issues/19079, and http://plnkr.co/edit/43a4sWEsdm5yfpVkJ5Zs?p=preview

@daniele-zurico could you review this, since you were the one who originally found this workaround?

I think that is missed also the import of the Router? Or maybe I missed something

ah yes, you're right about both of those things. I was lazy in creating this, should have been a bit more thorough

Just pushed up the fixes and some documentation. Also tested it with your microservice-test repository and it worked!

It's perfect now!

Released as v1.1.0 to npm


[Docs]: Sample GitHub Actions for deploying to GitHub Pages might be wrong

Request Type
Small Fix (wording, errors, typos, etc.)
Description
As I followed Deployment > GitHub Pages, the sample actions did not worked correctly.
I'm guessing BASE_PATH is not working as expected so that uploading artifact fails.
Sorry but I did not dig into implementation of cli yet.
- name: build
  env:
    BASE_PATH: '/${{ github.event.repository.name }}' # -> guessing this part is not working so that upload artifact part fails
    ## Add and uncomment any environment variables here
    ## EVIDENCE_SOURCE__my_source__username: ${{ secrets.EVIDENCE_SOURCE__MY_SOURCE__USERNAME }}
    ## EVIDENCE_SOURCE__my_source__private_key: ${{ secrets.EVIDENCE_SOURCE__MY_SOURCE__PRIVATE_KEY }}
  run: |
    npm run sources
    npm run build

- name: Upload Artifacts
uses: actions/upload-pages-artifact@v3
with:
    path: 'build/${{ github.event.repository.name }}' # -> fails for "directory not found"

changing to below fixed the issue for me
 - name: build
   env:
-    BASE_PATH: '/${{ github.event.repository.name }}'
     ## Add and uncomment any environment variables here
     ## EVIDENCE_SOURCE__my_source__username: ${{ secrets.EVIDENCE_SOURCE__MY_SOURCE__USERNAME }}
     ## EVIDENCE_SOURCE__my_source__private_key: ${{ secrets.EVIDENCE_SOURCE__MY_SOURCE__PRIVATE_KEY }}
   run: |
     npm run sources
     npm run build

 - name: Upload Artifacts
 uses: actions/upload-pages-artifact@v3
 with:
-    path: 'build/${{ github.event.repository.name }}'
+    path: './build

Anyway this project is incredible and we are already planning to migrate our internal tools.
Are you interested in contributing this improvement to the docs?
I'm just making a suggestion

To use basePath as expected you should also update the build command in your package.json.
This seems to have caught a few people out, so I will update those docs


Install Failed on AWS AMI

Bcrpyt 0.5.0 is failing npm install. I've tried npm cache clean and npm update to no avail.
https://gist.github.com/f761b87aae1cc1a0ff31
Thanks,
Matt

What version of python are you using?

Looks like this was actually a problem with 'n'. I was able to get around the problem using: https://github.com/visionmedia/n/issues/48

Ahh, good to know. Thanks!


Shows with DRM not working (FairPlay)

Just tried it with showid 60825 fo e.g. Script nicely output all 10 eposides in csv. I tried the generated urls with yt-dlp and youtube-dl and found, that the url is essential if it works or not:

Links containing akamaized.net/.../playlist.m3u8 works great
Links containing akamaized.net/.../hls_fairPlay_ts/.../master.m3u8 doesnt work (seams drm protected, see output below).

Is there any workaround?
Thanks
Sample output of second url:
[generic] master: Downloading webpage
WARNING: [generic] Falling back on generic information extractor.
[generic] master: Downloading m3u8 information
[info] master: Downloading 1 format(s): 9301+audio-aacl-125-English
[hlsnative] Downloading m3u8 manifest
ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format

Interesting, didn't know that DMAX has shows with DRM. It's Apple's FairPlay DRM. This is nothing I can fix, sorry.

Maybe this (youtube-dl fork with benefits)  will help, i have't use this, cause a have no Case for this :)

This project already generates yt-dlp commands. It also can't download DRM'd content sadly.

Ok, that's sad.
But cool, that you also use yt-dlp - i don't know that :)
I was thinking that project has only youtube-dl support and with a quick GoogleSeach yesterday i found this yt-dlp with --allow-u parameter. But i not try it out.
So finally as a workaroud -> Screenrecording with OBS... :'D :(

FYI: Updated the program and it uses an endpoint without DRM (well it's one parameter...). https://github.com/Brawl345/Get-DMAX-Links/releases/tag/2024-06-21


Generator fix pwa-kit.config template path

Description

Fixes the path to obtain the pwa-kit.config template in the pwa-kit-create-app/scripts/create-mobify-app.js script. The template includes the configuration to not include the locale in the URL as part of generating a project with a single-locale.
The previous solution assumed that the create-mobify-app.js script was always run from the root of the monorepo.
Types of Changes

[X] Bug fix (non-breaking change that fixes an issue)
[ ] New feature (non-breaking change that adds functionality)
[ ] Documentation update
[ ] Breaking change (could cause existing functionality to not work as expected)
[ ] Other changes (non-breaking changes that does not fit any of the above)

Breaking changes include:

Removing a public function or component or prop
Adding a required argument to a function
Changing the data type of a function parameter or return value
Adding a new peer dependency to package.json

Changes

Use require instead of path.join to read the pwa-kit.config template.

How to Test-Drive This PR

Checkout this branch generator-single-locale-fix-path.
Run npm ci.
Generate a project locally from the root of the monorepo by running the create-mobify-app script:

GENERATOR_PRESET=test-project node packages/pwa-kit-create-app/scripts/create-mobify-app-dev.js --outputDir /tmp/project1

Generate a project locally from packages/pwa-kit-create-app by running the create-mobify-app script:

GENERATOR_PRESET=test-project node ./scripts/create-mobify-app-dev.js --outputDir /tmp/project2

Confirm the two resulted generated projects have this URL configuration for the locale type in the pwa-kit.config.js file:

{
  "url": {
    "locale": "none"
  }
}

Checklists

General

[ ] Changes are covered by test cases
[ ] CHANGELOG.md updated with a short description of changes (not required for documentation updates)

Accessibility Compliance
You must check off all items in one of the follow two lists:

[ ] There are no changes to UI

or...

[ ] Changes were tested with a Screen Reader (iOS VoiceOver or Android Talkback) and had no issues
[ ] Changes comply with WCAG 2.0 guidelines levels A and AA
[ ] Changes to common UI patterns and interactions comply with WAI-ARIA best practices

Localization

[ ] Changes include a UI text update in the Retail React App (which requires translation)

+1 LGTM
Thanks for making this change @adamraya


mobile - admin front : go back should not make you return to the login screen unless you disconnect

duplicate with #452


Control - Switches jump back to the previous status

Hey…
I have a cFos Power Brain,2.0,1.20.2,W00-03HK (11 kw Solar version) and Home Assistant 2023.7.2 installed.
The integration works fine with all sensors (measurements).
but the control panel with the switches doesn’t work for me. If I switch off as an example “Wallbox Charging Rules Enabled” the status changes for a second and then it jumps back to “on”.
Is there anything I need to configure in the wallbox or HA?
and by the way… how to use pin/rfid via HA… is there any kind of documentation?
Beside this, thanks for your work and this nice implementation
jörg

Do you have a car connected to the cfos? I think without a car connected you cannot enable/disable charging rules etc.  These options are then also not accessible in the cfos web-ui.
Regarding the RFID function: please check the enter_RFID homeasistant service. The parameters are documented and should be rather self explanatory. Just call the service to enter a RFID.

Hey Markus,
you are right with a car connected it works. Many thanks.

Could we take the connected state into account to represent the switch state?
Would be nice if the switch could stay off / deactivated in case no car is connected.


[Tenancy] "vmdkops_admin ls" need a new column "tenant"

With multi-tenency, it is possible that two volumes with the same name are created on the same datastore(Those two volumes are created by two different tenents). In in the output of "vmdkops_admin ls" command, we will see two entries with the same volume name.  See the following example (two "vol1" are shown):
[root@localhost:~] /usr/lib/vmware/vmdkops/bin/vmdkops_admin.py ls
Volume    Datastore   Created By VM  Created                   Attached To VM (name/uuid)  Policy  Capacity  Used  Disk Format  Filesystem Type  Access      Attach As
--------  ----------  -------------  ------------------------  --------------------------  ------  --------  ----  -----------  ---------------  ----------  ----------------------
vol1      datastore1  photon-6       Sat Sep 10 01:11:07 2016  detached                    N/A     100MB     13MB  thin         ext4             read-write  independent_persistent
testvol   datastore1  photon-6       Sat Sep 10 04:21:02 2016  detached                    N/A     100MB     13MB  thin         ext4             read-write  independent_persistent
Vol100MB  datastore1  photon-6       Sat Sep 10 08:36:29 2016  detached                    N/A     100MB     13MB  thin         ext4             read-only   persistent
vol1      datastore1  photon-6       Sat Sep 10 09:36:51 2016  detached                    N/A     500MB     23MB  thin         ext4             read-write  independent_persistent

We need to add a new column named "tenant" to specify which tenant the volume belongs to.

Duplicate to #964


Revert "[flutter_tools] Update background isolates when performing hot reload/restart"

Reverts flutter/flutter#52149
It looks like this is causing some parts of commands_test to time out in the device lab.

TBR. Landing on red to make the tree green.

LGTM


Update _docker-compose to latest version

Hey there -- I really appreciate the contribution.
Cheers,
Robby


Unknown format error in driver log when sendTo() function calling

Adapter settings:

I use this script in js-driver for example:
sendTo('places.0', {
        user:       "Andrew",
        latitude:   55.754119,
        longitude:  37.620229,
        elevation:  100,
        timestamp:<PHONE_NUMBER>
});

When run script, adapter places.0 stopped with error in log:

PLease check 1.0.0

PLease check 1.0.0

Working!


support esp32 dlna render

hi, I have check your library, it could scan dlna server.
but I have wonder know it could for dlna render? so that it cloud found by HA

Hi, pls have a look at https://github.com/yellobyte/ESP32-Webradio-PlusDLNA/blob/main/Software/src/main.cpp.
You will find an example of how to access a media server and read/play its audio files. Regards Th

@yellobyte Hi! excuse me. Can you make an example similar to the ADF example DLNA? Play music with your phone and computer
https://github.com/espressif/esp-adf/tree/master/examples/advanced_examples/dlna

Hi! Actually the library has no DLNA render functionality at all. It's aim is to provide basic SOAP functions (Simple Object Access Protocol), enabling you to find/browse a DLNA server and download media files. In order to play a media file you need media render hardware and then feed it with media content, which is outside the scope of this lib.
As I mentioned already in my comment from 14.April , please have a look at https://github.com/yellobyte/ESP32-Webradio-PlusDLNA which does basically what you want and gives you an idea how to do it.
Regards Thomas

@yellobyte   Someone ported the ADF framework to the Arduino platform
【我成功将ESP-ADF移植到了Arduino上，并封装成ArduinoADFSimplify库，ESP32音频开发不再有门槛-哔哩哔哩】 https://b23.tv/h1f7luY


"kn-admin" use case collection [discussion]

As we have kn admin as our first 'official' kn plugin, it's time to collect features which people would like to see as part of a "kn admin" which is all about to configure a Kubernetes based Knative installation (in contrast to non-Kubernetes based implementations like Cloud Run).
This plugin is special as the persona it is adresses is not necessarily the end-user of Knative (like a developer) but an admin setting up Knative. Of course, if used in a standalone development environment (like minikube) that functionality might be helpful for everyone.
As soon we have kind of an initial set which we would like to have configured, we can revisit the command structure for this plugin.
Currently the following initial features / commands are available:

kn admin domain set - to set Knative route domain
kn admin registry add - add registry with credentials to enable Service deployment from it

What other use case can you imagine and which would make sense to add it to this plugin ? Please forward this issue to anybody who is interested in having features added for such a plugin.
// cc: @mattmoor @vaikas @evankanderson (TOC) @zhanggbj (kn-admin maintainer)

I like the idea a lot! Can we have this run in two "modes", where one mode is mutating configmaps directly and the other mutates the KnativeServing CR if you're running in an environment that's installed by an operator?
You could even try to guess which one to use based on the availability of a KnativeServing CR in the cluster but of course that should be overridable via flags.

Can we have this run in two "modes", where one mode is mutating configmaps directly and the other mutates the KnativeServing CR if you're running in an environment that's installed by an operator?

Definitely. And as you say, it also can auto-detect the installation mode but allow it to be overwritten via an option.

One of the first things that comes to mind would be all autoscaling options as described in https://knative.dev/docs/serving/configuring-autoscaling/ . And of course it would be cool if kn admin would be able to detect the serving version installed so that it could act differently depending on the Knative serving version running on the clustee.

@rhuss @mattmoor
Thanks for opening this! I have a proposal doc for the kn admin with 12 user stories, including domain & registry (which are merged) and auto-scaling as you mentioned.
I think working with operator mode is also a good idea and we can have more discussion in the proposal doc.

Proposal Doc for kn admin
https://docs.google.com/document/d/1oD5P4WQZs39GIr_4ezcURgRCuUx8dXrcCi3N7GKmp5c/edit#

A quick overview for use case

CC @duglin @maximilien


Using superscript and subscript simultaneously

To replicate, try either:

\alpha_{2}^{4}
\alpha^{4}_{2}

You will observe that they don't render as they should. The subscripts and superscripts should both apply directly to \alpha, not to each other; the two expressions should render identically.

The resulting MathML should use the msubsup element.

According to the source, it's currently hard-coded to only consider using Node::SubSup in the context of Token::Integral, which is why these are the only two tests using <msubsup>.
(r"\int_0^1 dx",  r#"<msubsup><mo>∫</mo><mn>0</mn><mn>1</mn></msubsup><mi>d</mi><mi>x</mi>"#),
(r"\int^1_0 dx",  r#"<msubsup><mo>∫</mo><mn>0</mn><mn>1</mn></msubsup><mi>d</mi><mi>x</mi>"#),


⚠ Bofrost.hr has degraded performance

In 77b70f9, Bofrost.hr (https://www.bofrost.hr) experienced degraded performance:

HTTP code: 200
Response time: 2716 ms

Resolved: Bofrost.hr performance has improved in 14a69e4 after 11 minutes.


Bug: Open File with which Program Keeps Popping Up

Which API Provider are you using?
Anthropic
Which Model are you using?
Claude 3.5 Sonnet
What happened?
In Windows, whenever you try to save an edit to a file, the popup asking for which program to use to open a certain file type opens up. It's constantly happening every time you get to the point where you save an edit to a file. It doesn't happen in the regular Cline extension only this forked version of it, Roo Cline. Is everyone else using Linux or am I the only person who exists, because there is no way someone else hasn't noticed this which makes the extension almost unusable.
Steps to reproduce

Ask to do anything which requires editing a file
Once it's done and it gets to the save button it shows the popup in Windows asking which program to use to open the file
It happens with or without the auto write features enabled

Relevant API REQUEST output
No response
Additional context
No response

Which version are you on? That should be fixed in 2.1.13 and later.

As someone exclusively running on Windows, I haven't seen this behavior related to editing files since I started using Roo in v2.1.14.
That being said, back when the sound effects were first introduced I did run into this every time Roo played a sound - but only because I did not have a default handler for .wav files.
@deathtothenig what file type is the popup trying to open? The quickest and simplest fix for your specific case is to:

Select Visual Studio Code (or w/e it's named in the popup)
Then click on the Always button (which sets VSCode as the default handler for that file type)

Which version are you on? That should be fixed in 2.1.13 and later.

I think I was using 2.1.6. Now I'm using 2.2 and it still does it. I disabled all auto-approve to get a better idea of when it does it and it happens literally every time the save button appears after Cline wants to edit a file: or the approve button appears after it says Cline wants to read a file:. It happens every time. I'm using VS Cod 1.95.3. It's not usable like this.

Which version are you on? That should be fixed in 2.1.13 and later.

I think I was using 2.1.6. Now I'm using 2.2 and it still does it. I disabled all auto-approve to get a better idea of when it does it and it happens literally every time the save button appears after Cline wants to edit a file: or the approve button appears after it says Cline wants to read a file:. It happens every time. I'm using VS Cod 1.95.3. It's not usable like this.

I'm trying to figure out how to get it working. So far this change below has stopped the, "What program to open this file type with," when sound effects is enabled. But it still doesn't play any sounds like the celebration.wav etc. when it should.
In sound.ts, change:
From:
		const player = require("play-sound")()
		player.play(filepath, function (err: any) {
			if (err) {
				throw new Error("Failed to play sound effect")
			}
		})

To:
		// Extract the audio type from the filepath
		const audioType = filepath.split('/').pop()?.split('.')[0]
		if (!audioType) {
			throw new Error("Invalid audio file path")
		}

		// Get the active Cline instance
		const vscode = require('vscode')
		const ClineProvider = vscode.extensions.getExtension('rooveterinaryinc.roo-cline')?.exports?.ClineProvider
		const activeInstance = ClineProvider?.getVisibleInstance()

		if (activeInstance?.view?.webview) {
			// Use the existing webview message system to play audio
			activeInstance.postMessageToWebview({
				type: "playSound",
				audioType: audioType
			})
		}

Turns out there are a bunch of issues with play-sound on Windows. Maybe https://www.npmjs.com/package/sound-play is better?

We released a new version with the sound-play library. Should be fixed now - can you reopen if not? Thank you!


[oh-my-zsh] plugin 'zsh-syntax-highlighting' not found

I don't know exactly what I did wrongly. But everytime I try to open my terminal, this message is showed:

Here it is my .zshrc:
# If you come from bash you might have to change your $PATH.
# export PATH=$HOME/bin:/usr/local/bin:$PATH

# Path to your oh-my-zsh installation.
export ZSH="$HOME/.oh-my-zsh"

# Set name of the theme to load --- if set to "random", it will
# load a random theme each time oh-my-zsh is loaded, in which case,
# to know which specific one was loaded, run: echo $RANDOM_THEME
# See https://github.com/ohmyzsh/ohmyzsh/wiki/Themes
ZSH_THEME="robbyrussell"

# Set list of themes to pick from when loading at random
# Setting this variable when ZSH_THEME=random will cause zsh to load
# a theme from this variable instead of looking in $ZSH/themes/
# If set to an empty array, this variable will have no effect.
# ZSH_THEME_RANDOM_CANDIDATES=( "robbyrussell" "agnoster" )

# Uncomment the following line to use case-sensitive completion.
# CASE_SENSITIVE="true"

# Uncomment the following line to use hyphen-insensitive completion.
# Case-sensitive completion must be off. _ and - will be interchangeable.
# HYPHEN_INSENSITIVE="true"

# Uncomment one of the following lines to change the auto-update behavior
# zstyle ':omz:update' mode disabled  # disable automatic updates
# zstyle ':omz:update' mode auto      # update automatically without asking
# zstyle ':omz:update' mode reminder  # just remind me to update when it's time

# Uncomment the following line to change how often to auto-update (in days).
# zstyle ':omz:update' frequency 13

# Uncomment the following line if pasting URLs and other text is messed up.
# DISABLE_MAGIC_FUNCTIONS="true"

# Uncomment the following line to disable colors in ls.
# DISABLE_LS_COLORS="true"

# Uncomment the following line to disable auto-setting terminal title.
# DISABLE_AUTO_TITLE="true"

# Uncomment the following line to enable command auto-correction.
# ENABLE_CORRECTION="true"

# Uncomment the following line to display red dots whilst waiting for completion.
# You can also set it to another string to have that shown instead of the default red dots.
# e.g. COMPLETION_WAITING_DOTS="%F{yellow}waiting...%f"
# Caution: this setting can cause issues with multiline prompts in zsh < 5.7.1 (see #5765)
# COMPLETION_WAITING_DOTS="true"

# Uncomment the following line if you want to disable marking untracked files
# under VCS as dirty. This makes repository status check for large repositories
# much, much faster.
# DISABLE_UNTRACKED_FILES_DIRTY="true"

# Uncomment the following line if you want to change the command execution time
# stamp shown in the history command output.
# You can set one of the optional three formats:
# "mm/dd/yyyy"|"dd.mm.yyyy"|"yyyy-mm-dd"
# or set a custom format using the strftime function format specifications,
# see 'man strftime' for details.
# HIST_STAMPS="mm/dd/yyyy"

# Would you like to use another custom folder than $ZSH/custom?
# ZSH_CUSTOM=/path/to/new-custom-folder

# Which plugins would you like to load?
# Standard plugins can be found in $ZSH/plugins/
# Custom plugins may be added to $ZSH_CUSTOM/plugins/
# Example format: plugins=(rails git textmate ruby lighthouse)
# Add wisely, as too many plugins slow down shell startup.
plugins=(git zsh-syntax-highlighting zsh-autosuggestions)

source $ZSH/oh-my-zsh.sh

# User configuration

# export MANPATH="/usr/local/man:$MANPATH"

# You may need to manually set your language environment
# export LANG=en_US.UTF-8

# Preferred editor for local and remote sessions
# if [[ -n $SSH_CONNECTION ]]; then
#   export EDITOR='vim'
# else
#   export EDITOR='mvim'
# fi

# Compilation flags
# export ARCHFLAGS="-arch x86_64"

# Set personal aliases, overriding those provided by oh-my-zsh libs,
# plugins, and themes. Aliases can be placed here, though oh-my-zsh
# users are encouraged to define aliases within the ZSH_CUSTOM folder.
# For a full list of active aliases, run `alias`.
#
# Example aliases
# alias zshconfig="mate ~/.zshrc"
# alias ohmyzsh="mate ~/.oh-my-zsh"

neofetch
source /usr/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh
source /home/marinho/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh

The folder for zsh-syntax-highlighting is located on the right directory as shows:
➜  zsh-syntax-highlighting git:(master) pwd
/home/marinho/zsh-syntax-highlighting
➜  zsh-syntax-highlighting git:(master)

This is a question for your distro or plugin manager: we don't debug downstream installers.  Also, as a general rule, don't expect anyone to bisect your dotfiles for you.  Good luck.

What do you recommend for me on this kind of problem, @danielshahaf? How can I debug the installer since I followed exactly what the installation instructions said?

I was checking the configurations and seems that I didn't follow exactly what the installation instructions said. For future references, always clone your plugins inside the folder ~/.oh-my-zsh/custom/plugins/ and you should be able to set your plugins on plugins=( [plugins...] your-new-plugin) inside your .zshrc without any additional configuration.
Closing this issue. Thanks, @danielshahaf!

You're welcome.  And what instructions are those, are they in INSTALL.md in this repository, or downstream?  If the former, we can fix/clarify as needed.
Gabriel Marinho wrote on Sun, 14 Apr 2024 16:01 +00:00:

I was checking the configurations and seems that I didn't follow
exactly what the installation instructions said. For future references,
always clone your plugins inside the folder
~/.oh-my-zsh/custom/plugins/ and you should be able to set your
plugins on plugins=( [plugins...] your-new-plugin) inside your .zshrc
without any additional configuration.
Closing this issue. Thanks, @danielshahaf!
--
Reply to this email directly or view it on GitHub:
https://github.com/zsh-users/zsh-syntax-highlighting/issues/950#issuecomment-2054106669
You are receiving this because you were mentioned.
Message ID: @.***>


Add docs updated

Summary
This PR contains
User Documentation Conversion

Reconstruing categories
Replacing build tool: Hugo -> Sphinx
Replacing md files to rst files
Aligning themes & template to OTX

Developer Documentation Conversion

Reconstruing categories
Aligning themes & template to OTX

One Unified Datumaro Documentation

Combining user documentation and developer documentation

How to test
$ cd datumaro/docs
$ make html

Checklist

[ ] I submit my changes into the develop branch
[ ] I have added description of my changes into CHANGELOG
[ ] I have updated the documentation accordingly
[ ] I have added tests to cover my changes
[ ] I have linked related issues

License

[ ] I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.
[ ] I have updated the license header for each file (see an example below)

# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT

Codecov Report
Patch coverage has no change and project coverage change: -0.01 :warning:

Comparison is base (070d716) 78.49% compared to head (2f713e0) 78.49%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #863      +/-   ##
===========================================
- Coverage    78.49%   78.49%   -0.01%
===========================================
  Files          206      206
  Lines        24939    24939
  Branches      5007     5007
===========================================
- Hits         19577    19575       -2
- Misses        4236     4237       +1
- Partials      1126     1127       +1

Flag
Coverage Δ

macos-11_Python-3.8
?

ubuntu-20.04_Python-3.8
78.49% <ø> (ø)

windows-2019_Python-3.8
?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

I briefly saw the changes of this draft and I know this is just a draft. However, why do we have to change all markdown files to *.rst? Although we move to Sphinx, we can use them as well: https://www.sphinx-doc.org/en/master/usage/markdown.html.

I briefly saw the changes of this draft and I know this is just a draft. However, why do we have to change all markdown files to *.rst? Although we move to Sphinx, we can use them as well: https://www.sphinx-doc.org/en/master/usage/markdown.html.

I briefly saw the changes of this draft and I know this is just a draft. However, why do we have to change all markdown files to *.rst? Although we move to Sphinx, we can use them as well: https://www.sphinx-doc.org/en/master/usage/markdown.html.

I got a feedback by doc team that Sphinx uses the RST markup language by default, and Intel recommend to use RST, not markdown.

I briefly saw the changes of this draft and I know this is just a draft. However, why do we have to change all markdown files to *.rst? Although we move to Sphinx, we can use them as well: https://www.sphinx-doc.org/en/master/usage/markdown.html.

I got a feedback by doc team that Sphinx uses the RST markup language by default, and Intel recommend to use RST, not markdown.

I read what you sent me but in the last page I can see "shift to rst for the main portion of documents, use markdown as a secondary.". I believe this direction does not mean that we have to move to rst now and never use md anymore.
I have no objection to moving to rst, but it cannot be allowed in this way. You made 228 file changes and 13,438 lines changes in this single PR. This is the number that can never be reviewed.
I found three documentation updates in the latest develop that you already missed in this PR:

YoloLooseFormat
https://github.com/openvinotoolkit/datumaro/blob/070d716e746c9ff44b9bc5d2d8b947736954696d/site/content/en/docs/formats/yolo.md?plain=1#L114-L162
YoloUltralyticsFormat
https://github.com/openvinotoolkit/datumaro/blob/develop/site/content/en/docs/formats/yolo_ultralytics.md
Searcher
https://github.com/openvinotoolkit/datumaro/blob/develop/site/content/en/docs/user-manual/command-reference/searcher.md

My opinion is that you should gradually migrate from md to rst and the amount of PR you put up at once should be reasonable for reviews.

My opinion is that you should gradually migrate from md to rst and the amount of PR you put up at once should be reasonable for reviews.

Additional work has not yet been done on the documents added after the time the doc was first created, and work on the detailed pages has not yet been completed. I made this PR first so that Yunchu could start the build doc work in CI at the same time.
While working on it, I realized belatedly that the amount of work was enormous. Are you want to turn files except index back into .md files again? @wonjuleee  What do you think of it?

My opinion is that you should gradually migrate from md to rst and the amount of PR you put up at once should be reasonable for reviews.

Additional work has not yet been done on the documents added after the time the doc was first created, and work on the detailed pages has not yet been completed. I made this PR first so that Yunchu could start the build doc work in CI at the same time.
While working on it, I realized belatedly that the amount of work was enormous. Are you want to turn files except index back into .md files again? @wonjuleee What do you think of it?

In my opinion, it looks better to let doc files staying with .md for now.


Make default heading style configurable

Markdown Builder currently uses underline (Setext) headings:
Heading
======

And only switches to hashtag (Atx) style headings from level 3+
# Heading

This can be set on individual Heading elements, but not with the builder pattern and it cannot be changed globally via MarkdownSerializationOptions.
Adding this to the builder is probably not such a good idea. Should the need arise to change this then it will surely be required to apply globally, for the whole markdown document.

PR merged, so this issue can be closed.


Add missing test for "multiple dataloader + percent_check fix"

What does this PR do?
This adds a missing test which got lost when #1920 was taken over by #2213 .
Hope it's ok with you @williamFalcon ?

this is rebased, and the tests that fail come from master.

docs shall be fixed in #2227

Also some doc updates here:
limit_train_batches: How much of training dataset to check (floats = percent, int = num_batches)
https://github.com/PyTorchLightning/pytorch-lightning/blob/1635ba1bb38e69d985b9a96e1e8f6b6dbdeff268/pytorch_lightning/trainer/trainer.py#L226-L228

@awaelchli @rohitgr7 I have merged the #1920 here to preserve the contribution of both, mind check and fix the tests, pls :rabbit:


MM-13622 MM-13620 Adding client4 methods, reducers, actions, and selectors for bot accounts.

Summary
Added for bot accounts:

Client4 methods
Reducers
Actions
Selectors

Ticket Link
https://mattermost.atlassian.net/browse/MM-13620
https://mattermost.atlassian.net/browse/MM-13622
Checklist

[x] Ran make check-style to check for style errors (required for all pull requests)
[x] Ran make test to ensure unit tests passed
[x] Added or updated unit tests (required for all new features)
[x] All new/modified APIs include changes to the drivers

Test Information
This PR was tested on: ArchLinux

@hmhealey Ready for re-review


License for datasets

This thread can be for a discussion of the license options for the underlying datasets listed on Harvard's Open Data website.
Would suggest 3 popular options, with the default and recommended choice being CC0.

+1 to CC0. It'd be great to have dataset licenses listed as a URL for consistent structured analysis. Below are some open options:
Worldwide Public Domain Dedications

Creative Commons Zero Public Domain Dedication (CC0), e.g. "license":"https://creativecommons.org/publicdomain/zero/1.0/"
Open Data Commons Public Domain Dedication and Licence (PDDL), e.g. "license":"http://opendatacommons.org/licenses/pddl/1.0/"

Open Licenses

Open Data Commons Attribution License (ODC-By), e.g. "license":"http://opendatacommons.org/licenses/by/1.0/"
Open Data Commons Open Database License (ODbL), e.g. "license":"http://opendatacommons.org/licenses/odbl/1.0/"
Creative Commons Attribution (CC BY), e.g. "license":"https://creativecommons.org/licenses/by/4.0/"
Creative Commons Attribution-ShareAlike (CC BY-SA), e.g. "license":"https://creativecommons.org/licenses/by-sa/4.0/"
GNU Free Documentation License, e.g. "license":"http://www.gnu.org/licenses/fdl-1.3.en.html"

Thanks @rebeccawilliams !


Ability to clone dependency repository via https or ssh

The scope of this is being changed. For now I am going to enforce that all urls are https. There doesn't seem to be a consistent way to translate between https and ssh protocols. For those users who are hell bent on using ssh (I'm in that camp) you have options
After cloning all dependencies using https retarget the remote to ssh
OR clone all dependencies manually ahead of time.
neither of these options are ideal. I've flirted with the idea of specifying both the https and ssh checkout urls, and then allowing the user to specify which protocol to use when checking out.
I may still go this route in the future, but I want to see how bad it is to just force https.


TS writer: add chunkSize param with 1MB default

Public-Facing Changes
The TypeScript Mcap0Writer now accepts a chunkSize option rather than creating a new chunk every 10 messages.
Description
Fixes https://github.com/foxglove/mcap/issues/252

1MB is in line with defaults for Go and Python writers currently (C++ defaults to 768k)


[Task]: [Choreo Sample - GitHub New Issue to Google Sheets Row] Listener Replacement with Async API Triggers

Connector Name
other (None Listed Above)
Task Description
We came to an agreement to update the samples with Async API triggers as a solution to the issue [3]. The burden of manually registering the callback URL at the API side is given to the user in Async API triggers. The user doesn't need to provide a callback URL during the listener configuration in Async API triggers. For example refer [1] [2]. The user can obtain the Choreo callback URL after deployment and use it to manually register the listener in the API side. In the future, if we really have a callback URL configuration requirement within the listener, we agreed to handle it within the listener init function itself. But this approach is not recommended and we will go forward with the Async API trigger approach. All the samples will be updated as soon as the Async API triggers are ready for use. We need to update the sample GitHub New Issue to Google Sheets Row.
[1] https://github.com/ballerina-platform/asyncapi-triggers/blob/dcbc81bf52394a6cdcad7f881227fbd1e2240006/asyncapi/github/data_types.bal#L19
[2] https://github.com/ballerina-platform/asyncapi-triggers/blob/dcbc81bf52394a6cdcad7f881227fbd1e2240006/asyncapi/github/listener.bal#L25
[3] https://github.com/wso2-enterprise/choreo/issues/10217
The execution plan is summarized in the following sheet
https://docs.google.com/spreadsheets/d/1CVmNi8MBHdILbNPlFmmdQmSLuZAai3w7YYP_Vz1dLvQ/edit?usp=sharing
Replace listener with async api trigger of the sample GitHub New Issue to Google Sheets Row.

Closing this issue since it's already fixed by https://github.com/wso2-enterprise/choreo-samples/pull/409


Bump sentry-sdk from 1.4.3 to 1.5.4

Bumps sentry-sdk from 1.4.3 to 1.5.4.

Release notes
Sourced from sentry-sdk's releases.

1.5.4

Add Python 3.10 to test suite (#1309)
Capture only 5xx HTTP errors in Falcon Integration (#1314)
Attempt custom urlconf resolve in got_request_exception as well (#1317)

1.5.3

Pick up custom urlconf set by Django middlewares from request if any (#1308)

1.5.2

Record event_processor client reports #1281
Add a Quart integration #1248
Sanic v21.12 support #1292
Support Celery abstract tasks #1287

Work in this release contributed by @johnzeringue, @pgjones and @ahopkins. Thank you for your contribution!
1.5.1

Fix django legacy url resolver regex substitution due to upstream CVE-2021-44420 fix #1272
Record lost sample_rate events only if tracing is enabled #1268
Fix gevent version parsing for non-numeric parts #1243
Record span and breadcrumb when Django opens db connection #1250

1.5.0

Also record client outcomes for before send #1211
Add support for implicitly sized envelope items #1229
Fix integration with Apache Beam 2.32, 2.33 #1233
Remove Python 2.7 support for AWS Lambda layers in craft config #1241
Refactor Sanic integration for v21.9 support #1212
AWS Lambda Python 3.9 runtime support #1239
Fix "shutdown_timeout" typing #1256

Work in this release contributed by @galuszkak, @kianmeng, @ahopkins, @razumeiko, @tomscytale, and @seedofjoy. Thank you for your contribution!

Changelog
Sourced from sentry-sdk's changelog.

1.5.4

Add Python 3.10 to test suite (#1309)
Capture only 5xx HTTP errors in Falcon Integration (#1314)
Attempt custom urlconf resolve in got_request_exception as well (#1317)

1.5.3

Pick up custom urlconf set by Django middlewares from request if any (#1308)

1.5.2

Record event_processor client reports #1281
Add a Quart integration #1248
Sanic v21.12 support #1292
Support Celery abstract tasks #1287

Work in this release contributed by @johnzeringue, @pgjones and @ahopkins. Thank you for your contribution!
1.5.1

Fix django legacy url resolver regex substitution due to upstream CVE-2021-44420 fix #1272
Record lost sample_rate events only if tracing is enabled #1268
Fix gevent version parsing for non-numeric parts #1243
Record span and breadcrumb when Django opens db connection #1250

1.5.0

Also record client outcomes for before send #1211
Add support for implicitly sized envelope items #1229
Fix integration with Apache Beam 2.32, 2.33 #1233
Remove Python 2.7 support for AWS Lambda layers in craft config #1241
Refactor Sanic integration for v21.9 support #1212
AWS Lambda Python 3.9 runtime support #1239
Fix "shutdown_timeout" typing #1256

Work in this release contributed by @galuszkak, @kianmeng, @ahopkins, @razumeiko, @tomscytale, and @seedofjoy. Thank you for your contribution!

Commits

817c6df release: 1.5.4
f3c44bd meta: Fix changelog typo (#1321)
b9bef62 meta: Changelog for 1.5.4 (#1320)
4dc2deb fix(django): Attempt custom urlconf resolve in got_request_exception as well ...
639c941 build(tests): Python 3.10 support (#1309)
bebd815 fix(python): Capture only 5xx HTTP errors in Falcon Integration (#1314)
6d89f4f Merge branch 'release/1.5.3'
95a8e50 release: 1.5.3
ca382ac meta: Changelog for 1.5.3 (#1313)
20f0a76 feat(django): Pick custom urlconf up from request if any (#1308)
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Codecov Report

Merging #397 (ff5e155) into master (b49254c) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master      #397   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           10        10
  Lines         1298      1298
=========================================
  Hits          1298      1298


Use GitPython library

to select only tracked files
works with .gitignore

This worked great for me. Thanks!

I've checked the above commit. This creates a dependency on Git repos. It throws an exception as git.exc.InvalidGitRepositoryError, when the code is run on a project without an initiated repo.
I guess it would be good to have the ability to run code on a non-git repo as well (the current behaviour). It might be worthwhile to make the code detect if it's a git repo or not automatically, or have a flag to override it.