Feature Request: Links to headers

In Obsidian you'd be able to click on a
[[AWS#Monitoring]]

and the AWS.md file would open under any header that had Monitoring as title.
Right now, the behaviour of memo is that it creates a file AWS#Monitoring.md

Hi,
I'm aware of this feature and wonder how useful it is for your knowledge base?
As far as I know, Obsidian headers' transclusion implemented only halfway, and links synchronization is not supported yet. Am I right that if the user changes header all other links pointing to it become obsolete until manually fixed?
Maybe you have some other improvements to this feature in mind after using it in your knowledge base?
I think showing in UI how many other files reference to certain header might be useful, so it will be easier to notice that this header is transcluded n times.
Also, I would be happy to collect more ideas on how to make headers transclusion compatible with links auto-sync.

I don't tend to use transclusion much to be honest (as defined by https://en.wikipedia.org/wiki/Wikipedia:Transclusion):
I'm not interested in seeing same content in both notes. I just want to link a concept to its proper page.
In this case, its location is not a standalone page, but it's a paragraph.
When I start a new page on a topic, I tend to create an outline by using headers.
When I develop a note, sometimes it gets pretty big since there are many headers: sometimes I split those big notes into multiple pages/notes and sometimes is not necessary, so having a link to those headers is quite useful.
Having an way to link to headers would be useful, imho.
Happy to clarify if needed.

Another use case:
I have a flag in some notes, where I link to one header in the Status.md page like:
status: [[Status#on-hold]]
status: [[Status#in-progress]]
status: [[Status#urgent]]

It's a hack to fix not being able to enumerate / select one choice from a set of values.
I could have those as separate pages, but it would create more clutter.

@svsool What are your thoughts on what I've said?

When I start a new page on a topic, I tend to create an outline by using headers.

Regarding outline - I do the same, but I rarely use headers for referencing in other files, I extract the whole concept to another file under that concept folder and link directly to it.

When I develop a note, sometimes it gets pretty big since there are many headers: sometimes I split those big notes into multiple pages/notes and sometimes is not necessary, so having a link to those headers is quite useful.

Given that I start to think that "extract to note" command might be useful.
Also, another command that allows sending selected paragraph to some existing note. Sometimes I refactor my dailies and extracting certain chunks to individual files and that might speed up this process a little bit.

Another use case:
I have a flag in some notes, where I link to one header in the Status.md page like:
status: [[Status#on-hold]]
status: [[Status#in-progress]]
status: [[Status#urgent]]

It's a hack to fix not being able to enumerate / select one choice from a set of values.
I could have those as separate pages, but it would create more clutter.

Do you have something else in Status.md besides headers? Would links uncreated to filesystem work like: status-on-hold, status-in-progress, status-urgent for you? Does Obsidian provide backlinks to headers?
How do you find those that are on hold or in progress? Using backlinks panel or via search?
About headers linking and how Obsidian implements it: I don't think that encouraging headers redundancy by referencing them as a whole is a scalable solution. Headers on their own tend to change less often than the rest of the content, but it does not make them a better id. Given that redundancy changing header in the original file wouldn't change it in the referencing files.
I think adding this feature is a good chunk of work because another level of analysis (namely markdown AST - for headers only might be not necessary and simply regex will work) is needed along with the cache. Given time I'd think how to make any markdown text content referable and not only headers because headers are just one level of transclusion.

I extract the whole concept to another file under that concept folder and link directly to it.
Make sense. I personally tend not to use folders (but I'm willing to try it out).
On another side, I find it helpful to have some autocompletion, so having knowing that I've got to pick something from status, I start typing Status# and then pick the relevant one from the selection, without need to remember the right name.

How do you find those that are on hold or in progress? Using backlinks panel or via search?
Search. Backlinks don't work well when you are linking the same file literally in every notes. Which might make pointless to actually have it as link vs having it as a tag.

I'm going to try out having separate files for status: so that will lower my urgency in this feature request.
This system is a WIP anyway so :+1:

I think there is a misunderstanding.
Linking to a header is NOT a transclusion, it is merely a link, which, if clicked on, takes you to the header in the same note or in another note. It is the nearest thing to Roam's block reference, though with less granularity.
You are right when you say links are not updated (yet) when the user changes the header title.
A transclusion is an embedded header with its text block/paragraph below. It is like an embedded image.
The syntax is ![[note-name#header-name]]. Notice the exclamation mark.
I use both links to headers and transclusions extensively in Obsidian. Transclusions allow me to combine different notes or part of notes in order to construct different stories. Very useful for writing.
I know this issue is closed, so this comment can maybe be transferred to a relevant issue.

Links of this sort (linking any content inside the file) very quickly require transclusion, and I would even imply having it. So what Obsidian implemented with headers is a special case. The problem is how to make them consistent and in sync across the knowledge base given just text files. Currently, I wouldn't like to introduce half working concept with the possibility of out of sync links to headers as in Obsidian. How it works produces more questions than the answers.

Obsidian has both linking to headers and transclusions. The link to a header and its block of text/paragraph below can easily be converted to a transclusion by putting an ! in front of the link.

I understand and yet Obsidian allows to break links as soon as file content changes link it or transclusion which I don't want in Memo at this point.

I understand and yet Obsidian allows to break links as soon as file content changes ……
Only if the header name changes is the link broken. Other file content changes do not affect links. Even changing the file name does not affect links as they are automatically updated.

Another thing is that for a transclusion you don't necessarily link to a header only, you can transclude an entire file into another one.

I do agree that header changes not being updated in links is a shortcoming.

Another thing is that for a transclusion you don't necessarily link to a header only, you can transclude an entire file into another one.

You can transclude an entire file in Memo too using ![[file.md]] and files of files and so on, but not a part of the file as Obsidian does via headers due to what I mentioned. Links become fragile and out of sync, also it brings content redundancy. Ideally header should not be duplicated in the link, and reference via id and editor should hint what kind of header is referenced.

Hi, sorry to disturb you again. I am intrigued by your Memo extension for VS Code, but since I have only ever used VSC for CSS files, I have no idea how to make markdown work.
In the screenshot you can see all the markdown extensions I have installed. When I open an existing file with another complete file embedded, it does not show that embedded file.
Another thing is I can open individual pre-existing .md file, but if I open a whole folder with those, they are shown with a prefix ._filename.
Obviously, I am doing something wrong. Could you set me right? If you are too busy I understand.

Another thing is I can open individual pre-existing .md file, but if I open a whole folder with those, they are shown with a prefix ._filename.

I need more context, hard to say anything. I recommend you to open Obsidian help folder or Memo’s help folder https://github.com/svsool/vscode-memo/tree/master/help and see if it works as you expect. There are examples with embedded files in both vaults.
Have you tried to open Obsidian help folder with Memo, does it work for you, including file embedding?

When I open an existing file with another complete file embedded, it does not show that embedded file.

Try to disable all extensions except Memo. Does it work? If yes, then try to disable one by one until you find the culprit - extension that interferes with Memo.
I hope this helps!

On 2. Oct 2020, at 13.20, DutchPete<EMAIL_ADDRESS>wrote:
Hi, sorry to disturb you again. I am intrigued by your Memo extension for VS Code, but since I have only ever used VSC for CSS files, I have no idea how to make markdown work.
In the screenshot you can see all the markdown extensions I have installed. When I open an existing file with another complete file embedded, it does not show that embedded file.
Another thing is I can open individual pre-existing .md file, but if I open a whole folder with those, they are shown with a prefix ._filename.
Obviously, I am doing something wrong. Could you set me right? If you are too busy I understand.
https://user-images.githubusercontent.com/10794265/94913449-f620cf80-04b1-11eb-8243-c3e37ecfa523.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/svsool/vscode-memo/issues/19#issuecomment-702649919, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALCQ3PF6IBBZKXCRHHZXW3SIWSPJANCNFSM4O4WOD2A.

Thanks for your help. I deleted the post because I figured out a couple of things.

I don't know why VS Code creates those ._filename files, one for each file in the folder I open, but below the last such strange file are the regular .md files.
The embedding does work, I tested it, but only for entire .md files, which I understood after reading the comments on the actual extension's page.

I find your extension nice and useful. I would have 2 wishes:

make header sections embeddable too, like in Obsidian. Why is it not possible to have automatic links synchronization on header rename? After all, you already have made possible to do it with file names.
support multi-level embeds. So, note 1 is embedded in note 2, and when note 2 is embedded in note 3, note 1 is also rendered in note 3.

support multi-level embeds. So, note 1 is embedded in note 2, and when note 2 is embedded in note 3, note 1 is also rendered in note 3.

Have you tried? It's already supported. Cyclic embedding detected as well.

make header sections embeddable too, like in Obsidian. Why is it not possible to have automatic links synchronization on header rename? After all, you already have made possible to do it with file names.

It's not impossible, just harder. Filepath = id in filesystem, so there is a good basis to sync links. Text within file content != id, so there should be whole system introduced for linking text and syncing links.

Have you tried? It's already supported. Cyclic embedding detected as well.

Sorry, no, I had not tried. In any case, for me multi-level embeds with entire files is not workable. It is workable when headers with their blocks are embeddable.

It's not impossible, just harder.

Does that mean you won't implement it?

Sorry, no, I had not tried. In any case, for me multi-level embeds with entire files is not workable. It is workable when headers with their blocks are embeddable.

That :point_up: contradicts with the following:

So, note 1 is embedded in note 2, and when note 2 is embedded in note 3, note 1 is also rendered in note 3.

Does that mean you won't implement it?

It won't be implemented in Memo until I or any other contributor to Memo or a similar tool like Obsidian / Foam understand how to do it well. Here is the long thread on that, and headings, as said, are just a special case of full transclusion.

That ☝ contradicts with the following

Yes it does and no it does not.
Yes it does, if we are talking about entire notes.
No, it does not because that was just an example of what I meant by multi-level embeddings. Since I knew Memo can only handle entire notes, there was no need to take header examples from 3 different notes to clarify my point.
On the Obs page you linked to, the discussion is about backlinks and showing more of their context like in an embed (i.e. transclusion). There is no discussion about automatic syncing when the original is changed.
What you and I are discussing is your reluctance at this stage for Memo to support embeds (i.e. transclusions) of a part of a note, i.e. a section/block/paragraph that has a header, because if the name of the header is changed the link is broken and the transclusion is no longer rendered - lack of syncing, as you call it.
You want to wait till there is a solution for that, fair enough, that is a choice you as the dev have to make. I think it is a pity because having no option to transclude a header and its section is still worse than having that option with a risk of a transclusion being broken. And let's face it, a broken link still carries info of where it linked to so it can be fixed. So, having the option with the status as it is, would still be an improvement by 1 notch of Memo.
Having said that, I do not want to persuade you to support header transclusions at this stage. I just give you my personal view. I am not a coder so cannot provide a solution.

I think it is a pity because having no option to transclude a header and its section is still worse than having that option with a risk of a transclusion being broken.

I think the opposite. We can agree to disagree. Thank you for the opinion on this topic.

I think the opposite. We can agree to disagree. Thank you for the opinion on this topic.

Fair enough. I have uninstalled Memo. Thank you for the discussion.

I wonder if now with "find all references to header" feature arriving to vscode, it'll be easier to keep links updated on header name change - I guess that's the key part of this feature request


Non-root pipeline images: build public certs directly into our worker binary

See https://github.com/pachyderm/pachyderm/issues/3394 for context
Currently the pachyderm worker binary copies a collection of included public SSL certs into /etc/ssl in its host container so that it can talk to public object store providers (e.g. during egress or during merges). However, when pipelines use docker images that don't run as root by default, the worker binary doesn't have the necessary permissions to do that copy.
Instead, the worker binary could have a collection of public certs built directly into the binary, and use those for communicating with object store. Then it wouldn't need to copy certs into /etc and would just work out of the box.
A potential downside to doing this is that the certs we build in are likely to get stale. Eventually, as more people are using old versions of pachyderm, the certs built into their worker binaries will eventually expire and then their pipelines will stop running. We'll have to re-build old versions of the worker binary with new certs, and users will have to somehow use the re-built worker binaries (and we'll have to implement a way for users to use the re-built worker binaries without fully upgrading. Fortunately WORKER_IMAGE is already an env var in pachd, so they'll probably just have to edit that and restart pachd. Not very ergonomic, but at least it's possible now). Or we'll have to give pachd some kind of way to automatically increment the worker binaries it's using, so that their pipelines never stop in the first place

IIUC this issue of stale certs isn't really made any worse if the certs are baked into the binary, is it? Right now the certs are baked into the worker docker image, which, while it could theoretically be updated, is effectively static (at least as much so as the binary it contains). Am i missing something here?

Nope, that's true (stale certs aren't any more of a problem if we build them into the binary than they are now)—hadn't thought that through all the way


Set pointer-events to none in the HomepageHero

Set pointer-events to none for the HomepageHero canvas so that text can be selected.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Not allowing to select the hero text was intentional: https://github.com/jsr-io/jsr/pull/115. Is there a specific issue you are running into?

Not allowing to select the hero text was intentional: #115. Is there a specific issue you are running into?

Did not realize this was intentional. I was trying to copy and paste the text to send to someone and noticed I could not. Seemed like a usability issue to me and opened a PR. If the JSR team feels users shouldn't be able to select text on the page, then I don't think Its my place to say otherwise.
Let me know if you want me to close this, otherwise, feel free to decline. Thanks for your time.

@AdamAnSubtractM Thanks for the contribution! I agree; this is an issue for usability.
However, this approach prevents some interactions with the background. I believe the better way to achieve the desired outcome would be to simply add the pointer-events-auto class to the <div> that contains the text in question. That will allow its text to be selectable, while keeping everything around it as-is.

@AdamAnSubtractM Thanks for the contribution! I agree; this is an issue for usability.
However, this approach prevents some interactions with the background. I believe the better way to achieve the desired outcome would be to simply add the pointer-events-auto class to the <div> that contains the text in question. That will allow its text to be selectable, while keeping everything around it as-is.

Thanks @josh-collinsworth . I'll update the PR with your suggestions.

@AdamAnSubtractM Thanks for the contribution! I agree; this is an issue for usability.
However, this approach prevents some interactions with the background. I believe the better way to achieve the desired outcome would be to simply add the pointer-events-auto class to the <div> that contains the text in question. That will allow its text to be selectable, while keeping everything around it as-is.

@josh-collinsworth  - PR updated, thanks.


New package: CitableParserBuilder v0.1.0

Registering package: CitableParserBuilder
Repository: https://github.com/neelsmith/CitableParserBuilder.jl
Created by: @neelsmith
Version: v0.1.0
Commit: b66ba3f49e3a3a8919b54a0112f0a05d3427082e
Git reference: main
Release notes:

Initial release.

@neelsmith just like other of your packages (#29532, #29531) also this one lacks a license file

Some more information on the license check is available here, by the way: https://discourse.julialang.org/t/fyi-the-general-registry-now-checks-for-an-osi-approved-software-license-when-registering-new-packages-and-new-versions/57018
Let me know if you have any questions!


Get around the "deleted function" error

Workaround for "deleted function" compilation error
Fixes #594
Task List

[x] Defined the tests that specify a complete and functioning change (It may help to create a design specification & test specification)
[x] Implemented the source code change that satisfies the tests
[x] Documented the feature by providing worked example
[x] Updated the README or other documentation
[x] Completed the pre-Request checklist below

Change Description
When compiling with the Intel compiler, we get an error: the default constructor of "Nextsim::DynamicsKernel<6, 8>" cannot be referenced -- it is a deleted function. The solution proposed here is not to use the = default keyword but to manually define a constructor that sets the key parameters to sensible default values.
It is not clear why the other compilers don't flag this.
The compiler warns that warning #2196: routine is both "inline" and "noinline" for a few of the tests (and only tests), but I don't know how to fix that one.

Test Description
The model compiles and (almost) all tests run.

Documentation Impact
N/A

Other Details
N/A

Pre-Request Checklist

[x] The requirements of this pull request are fully captured in an issue or design specification and are linked and summarised in the description of this PR
[x] No new warnings are generated
[x] The documentation has been updated (or an issue has been created to track the corresponding change)
[x] Methods and Tests are commented such that they can be understood without having to obtain additional context
[x] This PR/Issue is labelled as a bug/feature/enhancement/breaking change
[x] File dates have been updated to reflect modification date
[x] This change conforms to the conventions described in the README

Please complete your pull request description and delete the instructional text before submitting.
Congratulations and thank you for making your contribution to neXtSIM_DG!

There are still two problems with this PR:

The TOPAZ ocean test doesn't pass
The Intel compiler generates several warnings.

The TOPAZ failure is:
===============================================================================
/cluster/home/einola/src/nextsimdg/physics/test/TOPAZOcn_test.cpp:28:
TEST SUITE: TOPAZOcean
TEST CASE:  TOPAZOcean test

/cluster/home/einola/src/nextsimdg/physics/test/TOPAZOcn_test.cpp:76: FATAL ERROR: REQUIRE( sst(45, 35) == -(0 + targetFrac) ) is NOT correct!
  values: REQUIRE( -0.035045 == -0.035045 )

The warning is warning #2196: routine is both "inline" and "noinline", and this is issued for:

core/test/FileCallbackCloser_test.cpp(58)
core/test/ModelArray_test.cpp(302)
core/test/MonthlyCubicBSpline_test.cpp(57)
core/test/Iterator_test.cpp(75)
core/test/Time_test.cpp(241)
physics/test/IceMinima_test.cpp(37)
physics/test/SpecificHumidity_test.cpp(39)
core/test/ModelArrayRef_test.cpp(202)
dynamics/tests/DGModelArray_test.cpp(287,171)
dynamics/tests/ParametricMesh_test.cpp(142)
core/test/RectGrid_test.cpp(208)
core/test/CommandLineParser_test.cpp(44)
core/test/ConfiguredModule_test.cpp(129)
core/test/Configurator_test.cpp(234)
core/test/ConfigOutput_test.cpp(167)
physics/test/ThermoIce0_test.cpp(164)
core/test/ParaGrid_test.cpp(495)
physics/test/ConstantOceanBoundary_test.cpp(36)
core/test/ParaGrid_test.cpp(495)
physics/test/BasicIceOceanFlux_test.cpp(148)
core/test/ParaGrid_test.cpp(495)
physics/test/SlabOcean_test.cpp(180)
physics/test/IceGrowth_test.cpp(631)
physics/test/BenchmarkBoundaries_test.cpp(83)
physics/test/UniformOcean_test.cpp(93)
physics/test/FiniteElementFluxes_test.cpp(304)
core/test/PrognosticData_test.cpp(111)
physics/test/ERA5Atm_test.cpp(118)
core/test/ModelComponent_test.cpp(196)
physics/test/TOPAZOcn_test.cpp(107)
physics/test/ThermoWintonTemperature_test.cpp(360)

Unfortunately, neither the integration test, nor the dynamics benchmark run. In both cases, the model seg faults and dumps the core.

This was not the solution


Fix PHP 5.6 errors

In PHP 5.6 I get the following error:
Strict standards: Static function WebPConvert\Convert\Converters\BaseTraits\OptionsTrait::getConverterId() should not be abstract in /var/www/webp-express-tests/we6/wp-content/plugins/webp-express/vendor/rosell-dk/webp-convert/src/Convert/Converters/BaseTraits/OptionsTrait.php on line 43

I have added a PHP 5.6 test in Travis.
The 5.6 build fails with :
Fatal error: Call to undefined method WebPConvert\Tests\Convert\TestConverters\SuccessGuaranteedConverter::setLogger() in /home/travis/build/rosell-dk/webp-convert/src/Convert/Converters/AbstractConverter.php on line 119

I commented out all tests that uses SuccessGuaranteedConverter.
Now I get:
Fatal error: Class WebPConvert\Convert\Converters\Cwebp contains 4 abstract methods and must therefore be declared abstract or implement the remaining methods (WebPConvert\Convert\Converters\Cwebp::logLn, WebPConvert\Convert\Converters\Cwebp::ln, WebPConvert\Convert\Converters\Cwebp::getOptions, ...) in /home/travis/build/rosell-dk/webp-convert/src/Convert/Converters/Cwebp.php on line 461

So something isn't working with traits. But I only experience these problems in the testing framework. I can convert with cwebp on Apache on PHP 5.6 without errors or warnings.

Removing the abstract methods in the traits solves it.
These abstract methods are not really needed. I added them because Scrutinizer told me so - like this:
"It seems like getMimeTypeOfSource() must be provided by classes using this trait. How about adding it as abstract method to this trait?"
But it appears that these abstract declarations overrides the actual implementations in PHP 5.6.

The "setLogger()" error is strange, because setLogger isn't even declared abstract anywhere. However, removing all abstract methods in the traits makes the error go away.

Removing this line in OptionsTrait:
"abstract protected static function getConverterId();"
Solved the setLogger error.
And yes, one should not declare static functions abstract

Ha! - That turned out to be the only line that needed to be removed. The abstract declarations in the traits all works fine in now, in PHP 5.6


Scalpel/MLDBM::Sync "can't write lock ... function not implemented"

Hi,
Yet another issue with Scalpel and the MLDBM perl module.

[2016-10-27T22:24Z] can't write lock /lustre/data/t-tcga/esebesty/bcbio_run/uuid_batchS00837/work/mutect/6/SCLC_S00837-6_31784802_65347240-somaticIndels-scalpel-work/main/normal/variants.1.db.lock: Function not implemented at /lustre/data/t-tcga/esebesty/bcbio/data/anaconda/share/scalpel-0.5.1-3/MLDBM/Sync.pm line 240,  line 13016658.
[2016-10-27T22:25Z] Command failure: findVariants (/lustre/data/t-tcga/esebesty/bcbio/data/anaconda/share/scalpel-0.5.1-3/FindVariants.pl --bam /lustre/data/t-tcga/esebesty/bcbio_run/uuid_batchS00837/work/bamprep/SCLC_S00837_N/6/SCLC_S00837_N-sort-6_31784802_65347240-prep.bam --bed /lustre/data/t-tcga/esebesty/bcbio_run/uuid_batchS00837/work/mutect/6/SCLC_S00837-6_31784802_65347240-somaticIndels-regions-nolcr-nolcr.bed --ref /lustre/data/t-tcga/esebesty/bcbio/data/genomes/Hsapiens/GRCh37/seq/GRCh37.fa --kmer 25 --mincov 5 --covthr 3 --lowcov 1 --covratio 0.01 --radius 100 --window 400 --maxregcov 10000 --step 100 --mapscore 1 --mismatches 3 --pathlimit 10000 --dir /lustre/data/t-tcga/esebesty/bcbio_run/uuid_batchS00837/work/mutect/6/SCLC_S00837-6_31784802_65347240-somaticIndels-scalpel-work/main/normal --sample ALL --numprocs 0 --coords null --format vcf --cov2file --intarget)...
[2016-10-27T22:25Z] child exited with value 38

This seems to be caused by the MLDBM::Sync module,
where

flock($self->{lock_fh}, ($read_lock ? $LOCK_SH : $LOCK_EX))
|| die("can't ". ($read_lock ? "read" : "write") ." lock $file: $!");

gives the can't write ... error. flock is coming from the Fcntl perl module.
Is it possible that our lustre filesystem is causing the error? Any experience with this?

Endre;
Sorry about the issue. In general, I'm not really sure how best to deal with the issues you've been running into with scalpel scaling. I'm not especially knowledgeable about Perl debugging within these modules. This one here does look like the filesystem might have been unhappy with giving up a lock, which can happen under load. Is this a consistent issue or does it happen intermittently?
As an attempt at a workaround, I enforce use of a temporary directory for writing this intermediate. If you update to the latest development and set your temporary directory as something local to the machine (http://bcbio-nextgen.readthedocs.io/en/latest/contents/configuration.html#temporary-directory) instead of Lustre, hopefully this will work around the problem. Hope this helps.

Everything running ok now. Fcntl.pm (so Scalpel) needs the lustre filesystem to be mounted with the flock option.

Hi, razZ0r,
I have encountered the same problem as you. Did you solve it by reinstall the perl or just reinstall scalpel? I don't have root permission.

I did not reinstall anything. As I wrote in the previous comment, you need the lustre filesystem mounted with the flock option if you are running scalpel on this type of shared filesystem.


Adding probes to opensearch, opensearch-dashboards

Initial implementations of probes for opensearch and
opensearch-dashboards.
Default probes are reasonable to use, though for opensearch readiness it will require
credentials.
all probes have custom option where the end user can pass in whatever
check they prefer.
Signed-off-by: Harrison Goscenski<EMAIL_ADDRESS>Description
Adds liveness, readiness, and startup probes
Issues Resolved
#44
Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@peterzhuamazon I have updated this with some basic probes, and the option for entire custom probes. The readiness probe is something that works well for how I am utilizing the chart, but I believe leaving it disabled by default is the correct choice as there is underlying security configuration required to allow for the probe to work.

@peterzhuamazon I have updated this with some basic probes, and the option for entire custom probes. The readiness probe is something that works well for how I am utilizing the chart, but I believe leaving it disabled by default is the correct choice as there is underlying security configuration required to allow for the probe to work.

Thanks @hgoscenski-imanage this is a rather large PR I would wait for reviews from @DandyDeveloper @TheAlgo.
In the meantime do you have any test results to share?
Thanks.

@peterzhuamazon it is definitely a sizable PR. I combined it with a few of my other PR's in order to get a version that runs with the configuration I needed and it functions as I would expect.
I configured the master/client nodes to use the simple readiness probe, which remained in the Unready state until the master came up and the securityconfig was initialized (in our deployment we reinitialize the securityconfig on each helm depoyment).
Once it was initialized and the monitoring user was available, the client/masters went to ready, the data nodes were configured with the liveness probe to wait for green to ensure only one node is taken down at a time (in combination with the PDB) ensuring stability. I tested this over the course of a few deploys to verify it behaved in that manner.
The other change I made to our deployment is to ensure that each statefulset (client, master, data) are set to use RollingUpdate instead of parallel to ensure they are handled one at a time and that any misconfigured readiness probe does not take down the entire cluster.
I am unsure what to attach to prove this however, especially since on its own this PR does not actually run given the other PR's contain needed changes.

@DandyDeveloper I apologize for the unnecessary changes there, it looks like my formatter was a tad bit overzealous. I will revert them all.

@hgoscenski-imanage Overall the PR looks neat and will be very helpful to the end users. We move one step closer to a robust chart for OepnSearch stack. Thanks a ton 😋

@TheAlgo I'm a bit confused by having conditional probes. Why wouldn't a user want liveness, startup and readiness probes by default?
@hgoscenski-imanage I'm replied to you on the topic of the readiness. In theory I think this is a great idea, but I'm questioning the real world benefits to this.
Additional thing to note: If any nodes are ready, it won't matter if another node is not ready because the k8s service will route the document to a ready node, then that node will just pass the documents over to the node responsible (assume this is a not ready node) over the transport SSL which uses a headless service (I think?). So... It effectively circumvents the readiness probe anyway.

@TheAlgo I'm a bit confused by having conditional probes. Why wouldn't a user want liveness, startup and readiness probes by default?

@DandyDeveloper I agree with you completely, we can keep the default as enabled but there should be an option to turn it off . As @hgoscenski-imanage mentioned here
I think being able to disable the probes is a good thing for the users to have, and by default I think not enabling the readiness is good, especially as it is dependent on their security config. If they don't have a robust way of updating their security config their readiness probe would always be failing, which is a bad experience.

@TheAlgo

especially as it is dependent on their security config

Hence my general apprehension on actually adding this readiness. In addition to my other observations for @hgoscenski-imanage .
We may have to simplify the readiness, unless I'm wrong (which I'd like input on if I'm misunderstanding).

We may have to simplify the readiness, unless I'm wrong (which I'd like input on if I'm misunderstanding).
I vote on simplifying the readiness a bit as well @DandyDeveloper

@hgoscenski-imanage Got some merge conflicts.
I haven't had the opportunity to review your response yet RE: Readiness. Sorry.

@hgoscenski-imanage Apologies but will it be possible for you to raise this PR in our new repo for Helm Charts. Thanks

@hgoscenski-imanage Apologies but will it be possible for you to raise this PR in our new repo for Helm Charts. Thanks

@hgoscenski-imanage Hi would you mind move it to the helm-charts repo? Thanks.

@hgoscenski-imanage Apologies but will it be possible for you to raise this PR in our new repo for Helm Charts. Thanks

@hgoscenski-imanage Hi would you mind move it to the helm-charts repo? Thanks.

My apologies for the delay, I have opened https://github.com/opensearch-project/helm-charts/pull/43 for this.


pip install on openSuse problematic

15:07:41 [INFO    ] Executing state pip.installed for pip
15:07:41 [INFO    ] Executing command '/usr/bin/pip2 --version' in directory '/root'
15:07:41 [INFO    ] Executing command '/usr/bin/pip2 freeze' in directory '/root'
15:07:42 [INFO    ] Executing command "/usr/bin/pip2 install --index-url='https://pypi-jenkins.saltstack.com/jenkins/develop' --extra-index-url='https://pypi.python.org/simple' --upgrade 'pip'" in directory '/root'
15:07:42 [ERROR   ] Command "/usr/bin/pip2 install --index-url='https://pypi-jenkins.saltstack.com/jenkins/develop' --extra-index-url='https://pypi.python.org/simple' --upgrade 'pip'" failed with return code: 1
15:07:42 [ERROR   ] stdout: DEPRECATION: Failed to find 'pip' at https://pypi-jenkins.saltstack.com/jenkins/develop/pip/. It is suggested to upgrade your index to support normalized names as the name in /simple/{name}.
15:07:42 Cannot fetch index base URL https://pypi-jenkins.saltstack.com/jenkins/develop/
15:07:42 Could not find any downloads that satisfy the requirement pip in /usr/lib/python2.7/site-packages
15:07:42 Collecting pip
15:07:42   No distributions at all found for pip in /usr/lib/python2.7/site-packages

And locally on the machine:
li977-230:~ # pip2 install --upgrade --index-url='https://pypi-jenkins.saltstack.com/jenkins/develop' --extra-index-url='https://pypi.python.org/simple' pip
DEPRECATION: Failed to find 'pip' at https://pypi-jenkins.saltstack.com/jenkins/develop/pip/. It is suggested to upgrade your index to support normalized names as the name in /simple/{name}.
Cannot fetch index base URL https://pypi-jenkins.saltstack.com/jenkins/develop/
Could not find any downloads that satisfy the requirement pip in /usr/lib/python2.7/site-packages
Collecting pip
  No distributions at all found for pip in /usr/lib/python2.7/site-packages
li977-230:~ # pip2 install --upgrade --index-url='https://pypi.python.org/simple' pip
Could not find any downloads that satisfy the requirement pip in /usr/lib/python2.7/site-packages
Collecting pip
  No distributions at all found for pip in /usr/lib/python2.7/site-packages
li977-230:~ # pip2 install --upgrade pip
Could not find any downloads that satisfy the requirement pip in /usr/lib/python2.7/site-packages
Collecting pip
  No distributions at all found for pip in /usr/lib/python2.7/site-packages
li977-230:~ #

Make anything out of the above @aboe76?

Ah Ha!
Starting new HTTPS connection (1): pypi.python.org
There was an error checking the latest version of pip
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pip/utils/outdated.py", line 126, in pip_version_check
    headers={"Accept": "application/json"},
  File "/usr/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 473, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.7/site-packages/pip/download.py", line 365, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 461, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/pip/_vendor/cachecontrol/adapter.py", line 43, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/usr/lib/python2.7/site-packages/pip/_vendor/requests/adapters.py", line 431, in send
    raise SSLError(e, request=request)
SSLError: [Errno 0] _ssl.c:358: error:00000000:lib(0):func(0):reason(0)
Getting page https://pypi-jenkins.saltstack.com/jenkins/develop/pip/
Starting new HTTPS connection (1): pypi-jenkins.saltstack.com
Could not fetch URL https://pypi-jenkins.saltstack.com/jenkins/develop/pip/: connection error: [Errno 0] _ssl.c:358: error:00000000:lib(0):func(0):reason(0)
Will skip URL https://pypi-jenkins.saltstack.com/jenkins/develop/pip/ when looking for download links for pip in /usr/lib/python2.7/site-packages
DEPRECATION: Failed to find 'pip' at https://pypi-jenkins.saltstack.com/jenkins/develop/pip/. It is suggested to upgrade your index to support normalized names as the name in /simple/{name}.
Getting page https://pypi-jenkins.saltstack.com/jenkins/develop/
Starting new HTTPS connection (2): pypi-jenkins.saltstack.com
Could not fetch URL https://pypi-jenkins.saltstack.com/jenkins/develop/pip/: connection error: [Errno 0] _ssl.c:358: error:00000000:lib(0):func(0):reason(0)
Will skip URL https://pypi-jenkins.saltstack.com/jenkins/develop/pip/ when looking for download links for pip in /usr/lib/python2.7/site-packages
DEPRECATION: Failed to find 'pip' at https://pypi-jenkins.saltstack.com/jenkins/develop/pip/. It is suggested to upgrade your index to support normalized names as the name in /simple/{name}.
Getting page https://pypi-jenkins.saltstack.com/jenkins/develop/
Starting new HTTPS connection (2): pypi-jenkins.saltstack.com
Could not fetch URL https://pypi-jenkins.saltstack.com/jenkins/develop/: connection error: [Errno 0] _ssl.c:358: error:00000000:lib(0):func(0):reason(0)
Will skip URL https://pypi-jenkins.saltstack.com/jenkins/develop/ when looking for download links for pip in /usr/lib/python2.7/site-packages
Cannot fetch index base URL https://pypi-jenkins.saltstack.com/jenkins/develop/
URLs to search for versions for pip in /usr/lib/python2.7/site-packages:
* https://pypi-jenkins.saltstack.com/jenkins/develop/pip/
* https://pypi.python.org/simple/pip/
Getting page https://pypi-jenkins.saltstack.com/jenkins/develop/pip/
Starting new HTTPS connection (3): pypi-jenkins.saltstack.com
Could not fetch URL https://pypi-jenkins.saltstack.com/jenkins/develop/pip/: connection error: [Errno 0] _ssl.c:358: error:00000000:lib(0):func(0):reason(0)
Will skip URL https://pypi-jenkins.saltstack.com/jenkins/develop/pip/ when looking for download links for pip in /usr/lib/python2.7/site-packages
Getting page https://pypi.python.org/simple/pip/
Starting new HTTPS connection (1): pypi.python.org
Could not fetch URL https://pypi.python.org/simple/pip/: connection error: [Errno 0] _ssl.c:358: error:00000000:lib(0):func(0):reason(0)
Will skip URL https://pypi.python.org/simple/pip/ when looking for download links for pip in /usr/lib/python2.7/site-packages
Could not find any downloads that satisfy the requirement pip in /usr/lib/python2.7/site-packages
Collecting pip
  Cleaning up...
  No distributions at all found for pip in /usr/lib/python2.7/site-packages
  Exception information:
  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 232, in main
      status = self.run(options, args)
    File "/usr/lib/python2.7/site-packages/pip/commands/install.py", line 339, in run
      requirement_set.prepare_files(finder)
    File "/usr/lib/python2.7/site-packages/pip/req/req_set.py", line 330, in prepare_files
      raise not_found
  DistributionNotFound: No distributions at all found for pip in /usr/lib/python2.7/site-packages

Not the best fix but...
curl -sO http://cacerts.digicert.com/DigiCertHighAssuranceEVRootCA.crt
openssl x509 -inform DES -in DigiCertHighAssuranceEVRootCA.crt -out DigiCertHighAssuranceEVRootCA.pem -text
export PIP_CERT=`pwd`/DigiCertHighAssuranceEVRootCA.pem

heh, stupid!
I need to install ca-certificates-mozilla....


Boost regression

It seems that, following the latest environment changes, Boost cannot be found by CMake anymore on Windows Server 2019:
https://github.com/apache/arrow/pull/6991/checks?check_run_id=601773054#step:7:105
-- Could NOT find Boost (missing: Boost_INCLUDE_DIR regex system filesystem) (Required is at least version "1.58")

Area for Triage:
C/C++
Question, Bug, or Feature?:
Bug
Virtual environments affected

[ ] macOS 10.15
[ ] Ubuntu 16.04 LTS
[ ] Ubuntu 18.04 LTS
[ ] Windows Server 2016 R2
[x] Windows Server 2019

Hello, @pitrou
We have deprecated BOOST_ROOT environment variable - #319.  How to fix - https://github.com/actions/virtual-environments/issues/661 and https://github.com/actions/virtual-environments/issues/687

@al-cheb This seems to have worked, thank you.


Problems saving configuration (v3.23 - v3.27)

I have two identical gamepads (mac addresses are different), in the system they are defined as:
Only one of the two is connected:
Bluetooth Gamepad (c974e020-90ae-11e9-8005-444553540000)
Two are connected at the same time:
Bluetooth Gamepad (c974e020-90ae-11e9-8006-444553540000)
Bluetooth Gamepad (c974e020-90ae-11e9-8007-444553540000)
About the problem:

I configure two simultaneously connected gamepads (in turn)
Save configuration file
I disconnect one gamepad (the identifier of the remaining one changes)
I configure it
If I click "save configuration" now, the configure "only one is connected" will be saved, bat the configure of "two simultaneously connected" will be deleted
(when configure at the beginning of "one", and then "two", the settings of "one" are deleted)

Just in case, the log and saved configurations with:
XOutput.log
settings - One connected.json.txt
settings - Two connected.json.txt

Settings is really messed up, I will rewrite it in the next version.

Fixed in v3.29


Need to add the instruction on how to delete host on this page

In this page only shows how to scale out AVS that is to add new host into cluster. However, there should also be an tutorial for how to scale in AVS. To achieve this, user needs to click the three dots of the existing host and click delete. What's more importance is that Microsoft needs to mention when deleting an existing host, what does user needs to eb prepared? Does it cause any downtime? Or the host will go into maintenance mode and move the VMs that's reside on the host to another host without down time and how long does it takes? Does it depends on the VM amount, resources? Does Microsoft suggests users to delete the host that has no management VM like HCX manager or NSX-T manager or it doesn't matter?
Thank you for taking this as consideration!

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: b0d8e000-7a71-4b85-6699-2db08b305633
Version Independent ID: a5deb09e-157c-dc1d-9050-462ea25188d3
Content: Tutorial - Scale clusters in a private cloud - Azure VMware Solution
Content Source: articles/azure-vmware/tutorial-scale-private-cloud.md
Service: azure-vmware
GitHub Login: @jjaygbay1
Microsoft Alias: jacobjaygbay

@Alicia-Chou
Thanks for your feedback! We will investigate and update as appropriate.

Hi @Alicia-Chou
Thanks for your contribution and suggestion. Please add your ideas in below link, so our production team can review it and update the same.
Ideas · Community (azure.com)

I cannot open this link, can you please confirm?


Add additional tests for multi token transfers

As a follow up to https://github.com/cosmos/ibc-go/pull/6401, add tests to cover cases where multiple packets are sent over.
Assertions on end state (i.e how many packets were sent, balances after multi token sent) would be nice to have to ensure failure of any possible early returns in packet processing.

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate contributors tagged/assigned

Can we close this issue or is there more work to be done?


feat: update data without limitation of min 50 occurences

This PR is here to expose and suggest a fix to inaccurate computed data when third party runs a script from dynamic subdomain.
As an example we took Kameleoon third party analytic script. We can see that before our change (and on 2024-06 data), it has an averageExecutionTime of ~245ms on a total of 988 websites which is pretty good but not ranked on README because of its low occurences (below 1000).
What's wrong here is that this third party uses a dynamic subdomain to serve its main script on websites (e.g 9e9soula8o.kameleoon.eu). Some of these subdomain scripts are kept in final result but analyzing raw observed-domains we found a lot that are ignored (~450).
So we've decided to remove this filter of 50 occurrences of an observed-domain to be included in averageExecutionTime computation.
After updating data without this filter, we found that third party using dynamic subdomains are less favored in ranking because of much more accurate averageExecutionTime closer to its concurrents. As an example, for Kameleoon, we found that its totalOccurences grows-up to 1438 (+450) meaning it now appears in README ranking (n°62) but with a much more accurate averageExecutionTime value of ~599ms (+354ms).
As a concurrent comparison we can list:

Product
averageExecutionTime With filter
averageExecutionTime Without filter
Diff

Kameleoon
245ms
600ms
+355ms

Optimizely
857ms
861ms
+4ms

VWO
638ms
638ms
+0ms

ABTasty
492ms
495ms
+3ms

We know that this filter has been set to remove from computation scripts with low occurrences, but in some cases (e.g. subdomains) it provides inaccurate public data and ranking which may influence product choice in the wrong way.
We're pleased to get feedbacks and discuss our demonstration to maybe find a more relevant solution that could prevent this kind of issue without misrepresenting all third-parties.

TODO:

add links on averageExecutionTime Without filter column in table
generate new readme with graph (@Kporal)


OSError: [Errno 6] Device not configured: '/dev/tty'

Problem
Upgrading to VSCode extension v3.0.0 lead to the following bug when I try to format code
OSError: [Errno 6] Device not configured: '/dev/tty'
pip freeze | grep sqlfluff
sqlfluff==2.3.5
sqlfluff-templater-dbt==2.3.5

I suspect version 3.0.0 is not backward compatible with a SQLFLUFF<3.0
Current solution
Downgrade extension to v2.4.4

Same here, thanks for the current solution, roll back to v2.4.4 does work.

@Jacques-Peeters @ericdmontu does adding the "--force" argument to the format arguments in settings.json fix this?
    "sqlfluff.format.arguments": [
        "--force"
    ]

@keraion Adding the force argument worked for me.

Closing as it did work 👍


Update UG with sections on responsive Ui and viewing tips

Codecov Report

Merging #152 (87492e5) into master (de2b99b) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #152   +/-   ##
=========================================
  Coverage     64.82%   64.82%
  Complexity      477      477
=========================================
  Files            83       83
  Lines          1669     1669
  Branches        181      181
=========================================
  Hits           1082     1082
  Misses          524      524
  Partials         63       63

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

For your AnnotatedUiNarrow, might just wna bring user's attention to the lightbulb, meaning to say probably just annotate the lightbulb and leave the rest to the other fully annotated UI.
Otherwise, LGTM, gotta see how it looks like overall

I've changed the picture as requested.


NAT Gateway New IPs - Security Group Fix

What?
When we added new Gateways and changed the Route tables, the new IPs weren't added to some existing SG rules - this is now causing some issues in Staging.
How?
By adding the EIP Associations to a Data Resource, we can merge it into the allowed list of IPs.

Planning the cluster-infrastructure TF:
Changes to Outputs:
  + public_licensify_gateway_ips                       = [
      + "<IP_ADDRESS>",
      + "<IP_ADDRESS>",
      + "<IP_ADDRESS>",
    ]

Then once that's applied, we can go back and run the govuk-publishing-infrastructure TF to update the SGs.


The shield in slate.md says "Owned needed"

The shield before the sentence "We would also like to do the following but currently lack owners" currently says "Owned needed". Should point to a new shield without the typo.
Or better yet, use regular markdown formatting instead of a link to an image.

Fixed in https://github.com/rust-lang/rust-project-goals/pull/13

"What is the secret to open source", they ask me?
"Good timing", I tell them.


[WATSONX] Enable tool functionality for the model llama3.1

Added the ability to use tools with models:

meta-llama/llama-3-405b-instruct
meta-llama/llama-3-1-70b-instruct

Thanks!


Cmdlet included in sfb 2019

This cmdlet is also present in SfB 2019
https://github.com/MicrosoftDocs/office-docs-powershell/issues/4098

Docs Build status updates of commit cdffda6:
:clock10: Full build: the average full build time is 11 min(s) 46 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit cdffda6:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

skype/skype-ps/skype/Get-CsIPPhonePolicy.md
:white_check_mark:Succeeded
View (skype-ps)

skype/skype-ps/skype/Grant-CsIPPhonePolicy.md
:white_check_mark:Succeeded
View (skype-ps)

skype/skype-ps/skype/Set-CsIPPhonePolicy.md
:white_check_mark:Succeeded
View (skype-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


PyTorch version: 2.1.2+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.3 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: version 3.29.0
Libc version: glibc-2.35

Python version: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-3.10.0-1160.105.1.el7.x86_64-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA A100 80GB PCIe
GPU 1: NVIDIA A100 80GB PCIe
GPU 2: NVIDIA A100 80GB PCIe
GPU 3: NVIDIA A100 80GB PCIe

Nvidia driver version: 550.54.15
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Address sizes:       48 bits physical, 48 bits virtual
Byte Order:          Little Endian
CPU(s):              64
On-line CPU(s) list: 0-63
Vendor ID:           AuthenticAMD
Model name:          AMD EPYC 7763 64-Core Processor
CPU family:          25
Model:               1
Thread(s) per core:  1
Core(s) per socket:  64
Socket(s):           1
Stepping:            1
Frequency boost:     enabled
CPU max MHz:         2450.0000
CPU min MHz:         1500.0000
BogoMIPS:            4900.49
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb cat_l3 cdp_l3 invpcid_single hw_pstate sme ssbd rsb_ctxsw ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload vgif umip pku ospke vaes vpclmulqdq overflow_recov succor smca
Virtualization:      AMD-V
L1d cache:           2 MiB (64 instances)
L1i cache:           2 MiB (64 instances)
L2 cache:            32 MiB (64 instances)
L3 cache:            256 MiB (8 instances)
NUMA node(s):        1
NUMA node0 CPU(s):   0-63

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.1.2
[pip3] triton==2.1.0
[conda] Could not collectROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.4.0
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0	GPU1	GPU2	GPU3	CPU Affinity	NUMA Affinity	GPU NUMA ID
GPU0	 X 	PIX	SYS	SYS	0-63	0		N/A
GPU1	PIX	 X 	SYS	SYS	0-63	0		N/A
GPU2	SYS	SYS	 X 	PIX	0-63	0		N/A
GPU3	SYS	SYS	PIX	 X 	0-63	0		N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

🐛 Describe the bug
Build: stockvllm/vllm-openai:v0.4.0.post1 container image.
I'm receiving bad output from Command-R with total prompt+generation sequence length >8192 tokens, while in theory the model supports 128k via rope scaling.   Vanilla transformers responds appropriately.  Any suggestions?

Tot Seq Length
Env
HF
vLLM

<=8k
HF
ok
ok

>8k
HF
ok
bad

HF output >8k:
Your favorite word is "chicken"! This is a fun and playful word, often bringing to mind images (8212 total tokens)

vLLM output >8k:
Your ignore ignore\n\n\n\n ignore\n\nconst\n � � � �  (8212 total tokens)

Example code:
from vllm import LLM, SamplingParams
from transformers import AutoTokenizer, AutoModelForCausalLM
import sys
import os
import gc
import torch

os.environ['TOKENIZERS_PARALLELISM']="false"

model_id = "CohereForAI/c4ai-command-r-v01"
tokenizer = AutoTokenizer.from_pretrained(model_id)

target_seq_len=8192+20  # push us a bit past 8k
max_new_tokens=20
base_prompt_len=22 # Template+"my favorite word..."+"What is my favorite word"

num_extra_tokens=target_seq_len - base_prompt_len - max_new_tokens

# Pad with nonsense "Blah " tokens to the desired length
messages=[
    {"role": "user", "content": "My favorite word is \"Chicken\".\n" +
                                "Blah "*num_extra_tokens +
                                "\nWhat is my favorite word?\n"
    }
    ]

input_ids  = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")

print(f"Tokens (incl. template): {input_ids.shape[1]}")

#############################
## HF
tokenizer = AutoTokenizer.from_pretrained(model_id)
hf_model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto", torch_dtype=torch.float16)

gen_tokens = hf_model.generate( input_ids, max_new_tokens=max_new_tokens, do_sample=True, temperature=0.3)

gen_text = tokenizer.decode(gen_tokens[0, input_ids.shape[1]:])

tot_tokens=gen_tokens.shape[1]
print(f"HF output:{gen_text} ({tot_tokens} total tokens)")

del hf_model ; gc.collect() ; torch.cuda.empty_cache()

#############################
## vLLM

vllm_model = LLM(model=model_id, tokenizer=model_id, tensor_parallel_size=4, max_model_len=target_seq_len)

samplingparams = SamplingParams(max_tokens=max_new_tokens, temperature=0.3, skip_special_tokens=False)

output = vllm_model.generate(prompt_token_ids=[ input_ids[0].tolist() ], sampling_params=samplingparams)

tot_tokens=target_seq_len-max_new_tokens+len(output[0].outputs[0].token_ids)
print(f"vLLM output: {output[0].outputs[0]} ({tot_tokens} total tokens)")

Output:
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.

No chat template is defined for this tokenizer - using the default template for the CohereTokenizerFast class. If the default is not appropriate for your model, please set `tokenizer.chat_template` to an appropriate template. See https://huggingface.co/docs/transformers/main/chat_templating for more information.

Tokens (incl. template): 8192
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [02:05<00:00,  8.34s/it]
/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py:2756: UserWarning: Specified kernel cache directory could not be created! This disables kernel caching. Specified directory is /tmp/xdg-cache/torch/kernels. This warning will appear only once per process. (Triggered internally at ../aten/src/ATen/native/cuda/jit_utils.cpp:1442.)
  next_tokens.tile(eos_token_id_tensor.shape[0], 1).ne(eos_token_id_tensor.unsqueeze(1)).prod(dim=0)
This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (8192). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.
HF output:Your favorite word is "chicken"! This is a fun and playful word, often bringing to mind images (8212 total tokens)
2024-04-06 22:19:02,193	INFO worker.py:1752 -- Started a local Ray instance.
INFO 04-06 22:19:03 llm_engine.py:81] Initializing an LLM engine (v0.4.0.post1) with config: model='CohereForAI/c4ai-command-r-v01', speculative_config=None, tokenizer='CohereForAI/c4ai-command-r-v01', tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=8212, download_dir=None, load_format=auto, tensor_parallel_size=4, disable_custom_all_reduce=True, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, seed=0)
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
INFO 04-06 22:19:11 selector.py:16] Using FlashAttention backend.
(RayWorkerVllm pid=772) INFO 04-06 22:19:13 selector.py:16] Using FlashAttention backend.
INFO 04-06 22:19:13 pynccl_utils.py:45] vLLM is using nccl==2.18.1
(RayWorkerVllm pid=772) INFO 04-06 22:19:13 pynccl_utils.py:45] vLLM is using nccl==2.18.1
INFO 04-06 22:19:17 weight_utils.py:177] Using model weights format ['*.safetensors']
(RayWorkerVllm pid=772) INFO 04-06 22:19:18 weight_utils.py:177] Using model weights format ['*.safetensors']
(RayWorkerVllm pid=920) INFO 04-06 22:19:13 selector.py:16] Using FlashAttention backend. [repeated 2x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/ray-logging.html#log-deduplication for more options.)
(RayWorkerVllm pid=920) INFO 04-06 22:19:13 pynccl_utils.py:45] vLLM is using nccl==2.18.1 [repeated 2x across cluster]
INFO 04-06 22:20:12 model_runner.py:104] Loading model weights took 16.2916 GB
(RayWorkerVllm pid=772) INFO 04-06 22:20:13 model_runner.py:104] Loading model weights took 16.2916 GB
(RayWorkerVllm pid=920) INFO 04-06 22:19:18 weight_utils.py:177] Using model weights format ['*.safetensors'] [repeated 2x across cluster]
INFO 04-06 22:20:17 ray_gpu_executor.py:244] # GPU blocks: 10620, # CPU blocks: 819
INFO 04-06 22:20:20 model_runner.py:791] Capturing the model for CUDA graphs. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI.
INFO 04-06 22:20:20 model_runner.py:795] CUDA graphs can take additional 1~3 GiB memory per GPU. If you are running out of memory, consider decreasing `gpu_memory_utilization` or enforcing eager mode. You can also reduce the `max_num_seqs` as needed to decrease memory usage.
(RayWorkerVllm pid=772) INFO 04-06 22:20:20 model_runner.py:791] Capturing the model for CUDA graphs. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI.
(RayWorkerVllm pid=772) INFO 04-06 22:20:20 model_runner.py:795] CUDA graphs can take additional 1~3 GiB memory per GPU. If you are running out of memory, consider decreasing `gpu_memory_utilization` or enforcing eager mode. You can also reduce the `max_num_seqs` as needed to decrease memory usage.
(RayWorkerVllm pid=920) INFO 04-06 22:20:13 model_runner.py:104] Loading model weights took 16.2916 GB [repeated 2x across cluster]
INFO 04-06 22:20:31 model_runner.py:867] Graph capturing finished in 10 secs.
Processed prompts:   0%|                                                                                | 0/1 [00:00<?, ?it/s](RayWorkerVllm pid=772) INFO 04-06 22:20:31 model_runner.py:867] Graph capturing finished in 10 secs.
(RayWorkerVllm pid=920) INFO 04-06 22:20:20 model_runner.py:791] Capturing the model for CUDA graphs. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI. [repeated 2x across cluster]
(RayWorkerVllm pid=920) INFO 04-06 22:20:20 model_runner.py:795] CUDA graphs can take additional 1~3 GiB memory per GPU. If you are running out of memory, consider decreasing `gpu_memory_utilization` or enforcing eager mode. You can also reduce the `max_num_seqs` as needed to decrease memory usage. [repeated 2x across cluster]
Processed prompts: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:02<00:00,  2.76s/it]
vLLM output: CompletionOutput(index=0, text='Your ignore ignore\n\n\n\n ignore\n\nconst\n � � � � ', token_ids=[17700, 24434, 24434, 206, 206, 206, 206, 24434, 206, 206, 4399, 206, 199861, 199861, 199861, 199861, 228, 199861, 199861, 199861], cumulative_logprob=-16.78925259313553, logprobs=None, finish_reason=length, stop_reason=None) (8212 total tokens)
(RayWorkerVllm pid=920) INFO 04-06 22:20:31 model_runner.py:867] Graph capturing finished in 10 secs. [repeated 2x across cluster]

Hi @agt, could you try to change max_position_embeddings equal to model_max_length in config.json file?

Thanks @esmeetu , no change with max_position_embeddings==model_max_length==131072.
It appears the model config doesn't enable rope scaling, while in the Transformers implementation it may happen implicitly.
A crude patch to models/commandr.py setting rope_scaling seems to yield acceptable output for longer sequence lengths up to 32k, but I'm not sure if that's a reasonable direction to take, or how that factor ought to be calculated/set, or whether higher scaling factors (e.g. *16=128k seq_len) ought to work.
class CohereModel(nn.Module):
    def __init__(
        self,
        config: CohereConfig,
        linear_method: Optional[LinearMethodBase] = None,
    ):
        # Force vLLM to adopt rope scaling
        config.rope_scaling = { "type" : "dynamic", "factor": 4.0 }
        super().__init__()

@saurabhdash2512 , would you be able to weigh in?

@agt But changing to max_position_embeddings==model_max_length==131072 is good for me.
Also I made a PR for this. You can try to apply #3919 patch.

thanks @esmeetu - my edits to model's config.json didn't take effect (will investigate), so I wasn't actually changing max_position_embeddings as I thought.    (As a side effect, enabling RoPE scaling also expands max_position_embeddings, which is probably why I saw some success there.)
I can confirm that with your patch I'm able to run the larger context.  Closing & will note on your PR.

This issue also seems to happen on ROCm with any sized prompt when using the GPTQ version.
Built fresh with docker yesterday post-PR and it outputs only nonsense.
This is using an AMD Instinct MI100 (gfx908) x4 cluster


Need "Clear" button for left column.

I think we need "Clear" button for left column, where we store http-requests:

"Clear" button all requests (like we can see in Network).
little "Clear" button for each request which a user doesn't want to store (maybe the cross like in right column?).

Functionality added.


Accessibility roles 'radio' & 'checkbox' not announced by screenreader on ios since 0.73.0

Description
since version v0.73.0 the roles for radio & checkbox are not announced by Screen readers on ios
this was working in v0.72.10 but stopped working in v0.73.0
Steps to reproduce

install the app with yarn ios
navigate with a screenreader or Accessibility inspector to the text Welcome to React Native and go down from there
listen to the output - it does not announce roles for radio & checkbox - button is still announced fine

React Native Version
0.73.5
Affected Platforms
Runtime - iOS
Output of npx react-native info
System:
  OS: macOS 13.6.1
  CPU: (10) arm64 Apple M1 Max
  Memory: 1.12 GB / 32.00 GB
  Shell:
    version: "5.9"
    path: /bin/zsh
Binaries:
  Node:
    version: 20.10.0
    path: ~/.nvm/versions/node/v20.10.0/bin/node
  Yarn:
    version: 1.22.21
    path: ~/.nvm/versions/node/v20.10.0/bin/yarn
  npm:
    version: 10.2.3
    path: ~/.nvm/versions/node/v20.10.0/bin/npm
  Watchman:
    version: 2<IP_ADDRESS>
    path: /opt/homebrew/bin/watchman
Managers:
  CocoaPods: Not Found
SDKs:
  iOS SDK:
    Platforms:
      - DriverKit 23.2
      - iOS 17.2
      - macOS 14.2
      - tvOS 17.2
      - visionOS 1.0
      - watchOS 10.2
  Android SDK: Not Found
IDEs:
  Android Studio: 2022.3 AI-223.8836.35.2231.10811636
  Xcode:
    version: 15.2/15C500b
    path: /usr/bin/xcodebuild
Languages:
  Java:
    version: 11.0.15
    path: /Users/lutz.kwauka/.sdkman/candidates/java/current/bin/javac
  Ruby:
    version: 2.6.10
    path: /Users/lutz.kwauka/.rbenv/shims/ruby
npmPackages:
  "@react-native-community/cli": Not Found
  react:
    installed: 18.2.0
    wanted: 18.2.0
  react-native:
    installed: 0.73.5
    wanted: 0.73.5
  react-native-macos: Not Found
npmGlobalPackages:
  "*react-native*": Not Found
Android:
  hermesEnabled: true
  newArchEnabled: false
iOS:
  hermesEnabled: true
  newArchEnabled: false

Stacktrace or Logs
no crash happened

Reproducer
https://github.com/lutzk/accessibility-reproducer/
Screenshots and Videos
v0.72.10 working:
https://github.com/facebook/react-native/assets/793755/cec0dd90-6b7b-40d1-9fb5-9cd1d00e3f97
v0.73.5 not working:
https://github.com/facebook/react-native/assets/793755/c86794d2-f4fa-43df-835e-93315a63aa46

Bug added with https://github.com/facebook/react-native/commit/2d07d5f160efdd25f5b3dbfa65c13884df9f3117
Accidentally miscopied some mappings from previous JS shim.

Hmm, actually, I am seeing these were mapping to UIAccessibilityTraitNone before that change as well.
Let me check if they are read someone else. The blamed commit is potentially responsible for changes though.

Okay, these roles are specially handled here: https://github.com/facebook/react-native/blob/ec928d7a669fa2624bcf7da520041f140dd0fb03/packages/react-native/React/Views/RCTView.m#L343
There was another change around localization of these strings, but RCTLocalizedString does work a bit differently in OSS and internal (and the translation part has not been wired to external).
@lutzk would you be willing to take a look around this area, to see what might be happening in current OSS build?

Thank you for looking into it!
i can have a look into it but i am not experienced with objC so probably not of much help

Also FYI @Saadnajmi who needed to touch this recently for macOS.

This seems to apply to 'tab' role as well, but other roles like 'button' and 'header' works correctly.

@falselobster fyi as well

My suspicion is that the if-checks added in 2d07d5f are too strict and they exclude valid cases:
UIAccessibilityTraits accessibilityRoleTraits = json ? [RCTConvert UIAccessibilityTraits:json] : UIAccessibilityTraitNone;
if (view.reactAccessibilityElement.accessibilityRoleTraits != accessibilityRoleTraits) {
    view.accessibilityRoleTraits = accessibilityRoleTraits;
    view.reactAccessibilityElement.accessibilityRole = json ? [RCTConvert NSString:json] : nil;
    [...]

UIAccessibilityTraits roleTraits = json ? [RCTConvert UIAccessibilityTraits:json] : UIAccessibilityTraitNone;
  if (view.reactAccessibilityElement.roleTraits != roleTraits) {
      view.roleTraits = roleTraits;
      view.reactAccessibilityElement.role = json ? [RCTConvert NSString:json] : nil;
      [...]

Views such as radio have no matching trait in UIAccessibilityTraits, so the if-block won't be executed and neither role nor accessibilityRole will be set on view.reactAccessibilityElement.
As a result, they have no role and hence none is accounced by the SR.
could that be the case or am I missing something, @NickGerleman ?

Here is how I fixed it in case that someone finds it helpful:
diff --git a/React/Views/RCTViewManager.m b/React/Views/RCTViewManager.m
index 9b4775ceec3c220c521cd8428af1da528bd688de..0cf1787d1a961ab385e97dc8d376e92835a10a19 100644
--- a/React/Views/RCTViewManager.m
+++ b/React/Views/RCTViewManager.m
@@ -231,7 +231,8 @@ - (RCTShadowView *)shadowView
 {
   UIAccessibilityTraits accessibilityRoleTraits =
       json ? [RCTConvert UIAccessibilityTraits:json] : UIAccessibilityTraitNone;
-  if (view.reactAccessibilityElement.accessibilityRoleTraits != accessibilityRoleTraits) {
+  NSString *roleString = json ? [RCTConvert NSString:json] : nil;
+  if (view.reactAccessibilityElement.accessibilityRoleTraits != accessibilityRoleTraits || view.reactAccessibilityElement.accessibilityRole != roleString) {
     view.accessibilityRoleTraits = accessibilityRoleTraits;
     view.reactAccessibilityElement.accessibilityRole = json ? [RCTConvert NSString:json] : nil;
     [self updateAccessibilityTraitsForRole:view withDefaultView:defaultView];
@@ -241,7 +242,8 @@ - (RCTShadowView *)shadowView
 RCT_CUSTOM_VIEW_PROPERTY(role, UIAccessibilityTraits, RCTView)
 {
   UIAccessibilityTraits roleTraits = json ? [RCTConvert UIAccessibilityTraits:json] : UIAccessibilityTraitNone;
-  if (view.reactAccessibilityElement.roleTraits != roleTraits) {
+  NSString *roleString = json ? [RCTConvert NSString:json] : nil;
+  if (view.reactAccessibilityElement.roleTraits != roleTraits || view.reactAccessibilityElement.role != roleString) {
     view.roleTraits = roleTraits;
     view.reactAccessibilityElement.role = json ? [RCTConvert NSString:json] : nil;
     [self updateAccessibilityTraitsForRole:view withDefaultView:defaultView];

Additionally, we needed to add key-value pairs to React/I18n/strings/.lproj/Localizable.strings so that roles are read out in the user's target language and not in English. Keys were determined by debugging the RCTLocalizedStringFromKey function, for example:
"rDy1kaz-"="alert"; // "alert"="alert" would not be picked up by RN


feat: support Commit gossiping on P2P level

This PR introduces Commit gossiping logic. Only the latest commit is interesting for full nodes (as older ones are available in DA).
Work will be continued in future PRs.
Resolves #479.

Codecov Report

Merging #486 (959462c) into main (705f608) will decrease coverage by 0.00%.
The diff coverage is 43.24%.

@@            Coverage Diff             @@
##             main     #486      +/-   ##
==========================================
- Coverage   56.10%   56.09%   -0.01%
==========================================
  Files          50       50
  Lines        9237     9272      +35
==========================================
+ Hits         5182     5201      +19
- Misses       3296     3311      +15
- Partials      759      760       +1

Impacted Files
Coverage Δ

node/node.go
61.11% <11.76%> (-4.22%)
:arrow_down:

p2p/client.go
62.91% <66.66%> (-0.04%)
:arrow_down:

block/manager.go
62.57% <100.00%> (+0.21%)
:arrow_up:

rpc/json/ws.go
61.90% <0.00%> (+7.93%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us.

In celestia-node we gossip commit along with headers. Is the separation between those a requirement in Optimint?

In celestia-node we gossip commit along with headers. Is the separation between those a requirement in Optimint?

No, it's just how I implemented it now. I just thought this way it will be a bit easier and cleaner (no need to have a separate structure just to serialize/deserialize both, etc). Also, theoretically this way full node may pause receiving commits if it's catching up.

The structure already exists fwiw: https://github.com/tendermint/tendermint/blob/d68d25dcd5d6a66aa4f9a67bd79e9f09cee84458/types/light.go#L132-L137


[Bug] [CheckPoint] checkpoint ineffective

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

SeaTunnel Version
2.3.5
SeaTunnel Config
env {
  execution.parallelism = 2
  job.mode = "BATCH"
  checkpoint.interval = 2000
  read_limit.rows_per_second=1
  job.name = cell-Job
}

source {

    InfluxDB {
        result_table_name = "test"
        url = "http://localhost:8086"
        sql = "select number from test1"
        database = "mydb"
        username = "admin"
        password = "123456"
        upper_bound = 10
        lower_bound = 1
        partition_num = 10
        split_column = "number"
        query_timeout_sec = 60
        connect_timeout_ms = 150000
        chunk_size = 1
        chunk_read = true
        schema {
                fields {
                         number = INT
                }
        }
    }

}

sink {
  Console {
          source_table_name = "test"
        }

}

Running Command
./start-seatunnel-flink-15-connector-v2.sh --config ../jobs/job.txt

Error Exception
**BATCH    checkpoint ineffective**

Zeta or Flink or Spark Version
No response
Java or Scala Version
No response
Screenshots
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

using streaming mode

using streaming mode
this source support batch mode only


Allow configuration of the directory where Terraform runs

Feature Request
When creating Terraform files to use for deployment, I'd like the ability to place these files into a sub-directory for better organization rather than at the root of my project. I'd like there to be an option to specify which directory to run Terraform commands against when using terraform-pipeline
Planned Implementation
Terraform commands all allow for passing an argument to specify a directory to run against. If no directory is specified, the default behavior is to use the current directory. The plan command for example, can be used like so:
terraform plan [options] [dir]
There are several classes in the terraform-pipeline project currently that deal with running specific Terraform commands, such as the TerraformEnvironmentStage class. Some of these classes take parameters to allow for specific configuration.
My plan would be to add an optional constructor argument to these classes to allow for the configuration of this run directory. See the following example:

// No directory specified as constructor argument.
// No directory will be passed as an argument to the
// underlying terraform command and the current directory
// will be used by default
def test1 = new TerraformEnvironmentStage(environment:'test1')

// Directory specified as constructor argument.
// This directory will be passed as an argument to the
// underlying terraform command
def test2 = new TerraformEnvironmentStage(environment:'test2', directory:'./terraform/')

It might be a little repetitive to add the directory to each Stage.  Additionally, every Stage is expected to behave in the same way, so passing in the directory to each Stage opens the opportunity for one environment to be different from another (because one EnvironmentStage may receive a different directory from another EnvironmentStage).
The TerraformCommands were intended to be extensible, to handle whatever options that they would make available on the commandline.  Backend config is one such option that's already in use.
So:

It seems like the existing TerraformCommands need to be updated to accept the directory option.
This feels like, from a "Users" perspective, we could simplify it's use by implementing this as a plugin?

For example:
TerrraformDirectoryPlugin.init(directory: './terraform')

^-- the above could then systematically alter all the Terraform commands:
TerraformInitCommand.withDirectory(directory)
TerraformPlanCommand.withDirectory(directory)
TerraformApplyCommand.withDirectory(directory)

^-- Since those generate the actual CLI, it seems like an appropriate place for that code to live.  Then TerraformEnvironmentStage doesn't even have to know about this change - it's just delegating work to the TerraformCommands, and they do whatever they do to make things work.
Thoughts?

To be a little more concrete, we can look at the AwssumePlugin.  It too, needs to systematically change the TerraformCommands.
./src/AwssumePlugin.groovy
So a new TerraformDirectoryPlugin might look like:
class TerraformDirectoryPlugin implements TerraformPlanCommandPlugin, TerraformApplyCommandPlugin {
    private String directory

    public static void init(Map options =[:]) {
        TerraformDirectoryPlugin plugin = new TerraformDirectoryPlugin(options)

        TerraformPlanCommand.addPlugin(plugin)
        TerraformApplyCommand.addPlugin(plugin)
    }

    @Override
    public void apply(TerraformPlanCommand command) {
        command.withDirectory(directory)
    }

    @Override
    public void apply(TerraformApplyCommand command) {
        command.withDirectory(directory)
    }
}

I'm unclear whether or not TerraformInit would also need to be modified - but that detail can be figured out during implementation.  The idea is the same, either way.
Also, as explained in the previous post, withDirectory() doesn't currently exist, and would need to be implemented as part of this work.

That seems better than having to list it as a parameter for each stage. :+1:

It might be worth considering altering the example I provided above too.  I believe every plugin has a simple init() method with no arguments.  So this will be the first plugin that changes that.  I'm not sure if I like that???
So perhaps, from the "Users" perspective, it should look more like this?
TerraformDirectoryPlugin.withDirectory('./terraform').init()

That keeps the .init() nice and consistent, and leaves open the opportunity to make a generic Plugin interface that every possible plugin could implement:
interface TerraformPipelinePlugin {
    public void init()
}


Update to be compatible with gravitorch

Codecov Report
Patch coverage: 100.00% and no project coverage change.

Comparison is base (7fc613d) 100.00% compared to head (35bc17f) 100.00%.

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #49   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           29        29
  Lines          456       456
  Branches        23        23
=========================================
  Hits           456       456

Impacted Files
Coverage Δ

src/gtvision/datasources/mnist.py
100.00% <ø> (ø)

src/gtvision/creators/dataflow/dataloader.py
100.00% <100.00%> (ø)

src/gtvision/creators/dataloader/__init__.py
100.00% <100.00%> (ø)

src/gtvision/creators/dataloader/factory.py
100.00% <100.00%> (ø)

src/gtvision/datasources/vanilla.py
100.00% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Error loading DWG in .NET/WebAssembly

Hi,
We're evaluating your product to read DWG in .NET/WebAssembly but, have found a stopper. The same should happen with Blazor as well. Under Windows it works as expected.
You can reproduce it by following these steps:

Clone https://github.com/MarcosCobena/WasmApp1/tree/bug-encoding
Build
Serve the output folder (bin/Debug/netstandard2.0/) with a local web server (dotnet serve, Fenix, etc.)
Browse the local URL (http://<IP_ADDRESS>:9000, for instance)

In the Developer Console (you can open it through F12 in most of the browsers), you will see the following stacktrace. However, I would expect the output "Success!"
Unhandled Exception: dotnet.js:2271:16
Aspose.CAD.CadExceptions.ImageLoadException: Image loading failed: Cannot process loading further due to incorrect file format structure. Error details: Encoding 1252 data could not be found. Make sure you have correct international codeset assembly installed and enabled. ---> System.Exception: Cannot process loading further due to incorrect file format structure. Error details: Encoding 1252 data could not be found. Make sure you have correct international codeset assembly installed and enabled. dotnet.js:2271:16
  at �   .ReadDwg (System.IO.Stream �, �    �, �    �) <0x7d35150 + 0x00096> in <filename unknown>:0 dotnet.js:2271:16
  at �   .� (System.IO.Stream �, Aspose.CAD.LoadOptions �) <0x7d0a0c8 + 0x000e8> in <filename unknown>:0 dotnet.js:2271:16
  at �   .Load (Aspose.CAD.StreamContainer �, Aspose.CAD.LoadOptions �) <0x7cff550 + 0x0008c> in <filename unknown>:0 dotnet.js:2271:16
  at Aspose.CAD.Image.LoadImage (Aspose.CAD.StreamContainer streamContainer, Aspose.CAD.LoadOptions loadOptions) <0x2a70c60 + 0x000aa> in <filename unknown>:0 dotnet.js:2271:16
   --- End of inner exception stack trace --- dotnet.js:2271:16
  at Aspose.CAD.Image.LoadImage (Aspose.CAD.StreamContainer streamContainer, Aspose.CAD.LoadOptions loadOptions) <0x2a70c60 + 0x0015a> in <filename unknown>:0 dotnet.js:2271:16
  at Aspose.CAD.Image.Load (System.IO.Stream stream, Aspose.CAD.LoadOptions loadOptions) <0x2a60f00 + 0x00018> in <filename unknown>:0 dotnet.js:2271:16
  at WasmApp1.Program.Main () [0x0001a] in C:\Users\Marcos\Repositorios\WasmApp1\WasmApp1\Program.cs:21 dotnet.js:2271:16

I initially opened this issue at mono; however, I have thought it may be any obfuscation process being made to the assembly which is making such contain Unicode chars: you can see such with tools like ILSpy. I understand this should be addressed from mono side but, wanted to share it here in case any workaround is feasible.
Thanks in advance,
-Marcos

This particular issue has already been solved at https://github.com/mono/mono/issues/19822


Create the error type for openwhisk client error

@vincent did you remove the YAML lib version update? because of my comments here: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/456
If so, where is the YAML version update being done at?

@mrutkows
https://github.com/apache/incubator-openwhisk-wskdeploy/commit/68d74d52f489aff69630f422ac2d3ccc21cf635f
Priti has already submitted the PR and merged.


fixed Registration EUREKA on docker compose running

fixed when running on docker compose each author service and article service register on eureka
added environment on Dockerfile and microservices

I already add maintainer!


Improvements

Component.getActionHandlers() allows to define action handlers without overriding the Component constructor
Some fixes in Node schema compilation

Codecov Report

Merging #1214 into develop will decrease coverage by 0.01%.
The diff coverage is 71.42%.

@@             Coverage Diff             @@
##           develop    #1214      +/-   ##
===========================================
- Coverage    63.46%   63.44%   -0.02%
===========================================
  Files          345      345
  Lines        12826    12826
===========================================
- Hits          8140     8138       -2
- Misses        4686     4688       +2

Impacted Files
Coverage Δ

ui/Component.js
73.33% <100%> (+0.08%)
:arrow_up:

model/Node.js
88.18% <60%> (-1.66%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80a90dd...0971a13. Read the comment docs.


Add Synapse Stats Exporter

Overview
In order to collect and monitor the number of users and rooms in the Synapse instance, this PR install the synapse-stats-exporter so the metrics can be extracted by Prometheus.
Also, it refactors how the charm handle the admin_access_token.
Before the refactor, the admin access token was handled only by the charm and used by Mjolnir.
But since the Stats Exporter needs it too, it was complex to let this in the charm instead of letting the Synapse module handle it.
So what happens now?

start event is emitted meaning that (hopefully) the workload was started (not just the container)

If the admin access token already exists (as a secret or as peer data), start the Stats Exporter and return

If the admin access token doesn't exist, create it via create_admin_user and save it in the charm_state

Mjolnir and Stats Exporter will get the admin_access_token via charm_state

Also, Mjolnir will create the user using the create_user available in the synapse module instead of calling actions.register_user (that didn't make much sense)

This is how the modules depend on each other now:

Rationale
Collect Synapse statistics.
Juju Events Changes
N/A
Module Changes
Pebble module has a new layer.
Library Changes
N/A
Checklist

[x] The charm style guide was applied
[x] The contributing guide was applied
[x] The changes are compliant with ISD054 - Manging Charm Complexity
[x] The documentation is generated using src-docs
[ ] The documentation for charmhub is updated.
[x] The PR is tagged with appropriate label (urgent, trivial, complex)

Documentation should be updated after the exporter being available.

I'm closing this PR in favor of splitting the changes into smaller ones. This is already too messy.


chore(deps): lock file maintenance

Supersedes #2225 - has some changes to it though:

cucumber did not have node types pinned, so the build failed
nestjs instrumentation was missing a dependency that was required for tests to run properly

Nice work.

Some other versions of @types/node that I see.
├─┬<EMAIL_ADDRESS>-> ./plugins/node/opentelemetry-instrumentation-fastify
│ └──<EMAIL_ADDRESS>├─┬<EMAIL_ADDRESS>-> ./propagators/propagator-aws-xray-lambda
│ └──<EMAIL_ADDRESS>├─┬<EMAIL_ADDRESS>-> ./propagators/propagator-aws-xray
│ └──<EMAIL_ADDRESS>deduped

Those are older than the "blessed" version "18.18.14". Perhaps that's fine then.

nestjs instrumentation was missing a dependency that was required for tests to run properly

Hrm, perhaps I broke that in this PR ~7w ago then: https://github.com/open-telemetry/opentelemetry-js-contrib/pull/2464/files#diff-5bd19fcb42c8adf472fa8ddceb76b947ab770bbb899a1cacc40dbf4aaa138531L2

Something is up wth the runner for TAV for PR / tav / Run test-all-versions (22) (pull_request). It has been almost 2h running so far.


Disable recursive reference transaction hooks

This PR disables recursive reference transaction hooks in the MerkleDB v2 case.  It is unclear why they are needed in the MDB v1 case; this warrants further investigation.

Do we have the correct mdb version at the point when the hooks are written? Thinking about other initialization conditions where we may have an incomplete information about the repository.

Yes. In both the implicit and explicit paths, the hooks are set up last.

How much time does this save approximately? Should be worthy documenting that somewhere, either in the code or in the PR comment.


Documentation Link Throws 404

The line in the posting (shown below) includes a broken link:
"Read complete documentation about WebTerminal."

Thank you Jeff for pointing this out! Should be fixed now.


Articles are not being filtered correctly on length

We do not include articles that are less than 95 words, however it seems that the filter is not operating on the same text that is stored in the DB, resulting in some very small texts being saved:

Some examples from the test DB:

Turns out the reason for this is due to the splitting done at the Article content differs from that done when filtering.

Fixed with https://github.com/zeeguu/api/pull/149


Fixed bug where EncryptedBlobClient would not encrypt data before upl…

…oad on new methods

/azp run java - storage - ci


Upgrade to Bootstrap 3.x

Do you have any plans to upgrade Bootstrap?

You are right it is time for a new maintenance release.

New release 2.0.3 with latest bootstrap version is out now.

thanks a lot! :)


Dev/hand eye calibration

Added hand-eye calibration sample using cpp, as well as updated README and CmakeLists

Get rid of UtilizeHandEyeCalibration folder. The UtilizeHandEyeCalibration.cpp file can go directly to the HandEyeCalibration folder. Modify CMakeLists if needed to make sure it configures the project and taht the project builds.

You need to run clang-formating. I think that is why it doesn't build on Azure. If you are using Visual Studio Code, it is Ctrl+K Ctrl+F, for Visual Studio it is CTRL+R, CTRL+F

It's not just formatting which breaks CI, CMakeLists.txt is broken too. There's an extra ) in HandEyeCalibration/UtilizeEyeInHandCalibration).
I recommend always ensuring that a branch builds locally before making a PR.

You should clean up the git history here:

7e207a7 says "Formatted and renamed variable names to camelCase", but it also changes Zivid version, includes a new sample in CMakeLists.txt etc.
d415c0f8 says "formatting", but is actually a merge commit.
Commit messages should be in the imperative mood (here's a good article on commit messages)
There should be no merge commits in a PR

I suggest you fix the CI errors, then squash all your commits into one and re-write the commit message. It will be much more efficient for reviewers when the PR compiles and the git history is easy to read.

CI failed because of
"Could not find a configuration file for package "Zivid" that is compatible
with requested version "1.6.0"."
Should I change this back to 1.5.0 for the CI checks and then back to 1.6.0 or how do I do this?
Squashing the commits tomorrow morning.

CI chekcs failed due to
Could not find a configuration file for package "Zivid" that is compatible
with requested version "1.6.0".
Changing it back to 1.5.0 and re-run it

@chrisasc  Does this sample build on your machine now?

Yes, it build on my computer!

Figured out the build issue, @chrisasc is using msvc, which has extensions for both of the things that failed on my computer: https://godbolt.org/z/1mPYfN

Tested the sample now on Ubuntu 18.04, it works well.


6371 migrate sharding tags

Issue
https://github.com/gravitee-io/gravitee-api-management/pull/245
Description
next step for migration of sharding tags
Additional context

Gravitee.io Automatic Deployment
📚 View the storybook of this branch here

Gravitee.io Automatic Deployment
Congrats, all the CI steps were 🟢 . The Bot was able to deploy the build of this PR, so you can now try it directly here! 🚀
Notes: The deployed app is linked to the management API of the Element Zero team's environment.

Gravitee.io Automatic Deployment
📚 View the storybook of this branch here

Gravitee.io Automatic Deployment
Congrats, all the CI steps were 🟢 . The Bot was able to deploy the build of this PR, so you can now try it directly here! 🚀
Notes: The deployed app is linked to the management API of the Element Zero team's environment.


Remove Email Connector from ballerina-lang

Purpose
Remove Email Connector from repository, ballerina-lang to migrate it as an independent repository.
Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

There are so many unrelated commits.


Add support for YouTube Live Streaming chat

Is your feature request related to a problem? Please describe.
Matterbridge doesn't support Youtube Live Streaming chat
Describe the solution you'd like
Have Youtube Live Streaming as a supported messaging channel.

This is for syncing stream chats between different streaming services.

Thanks @cwchristerw! Great idea. I'm just going to rename for clarity.
I see that you have some Go experience -- is this something you'd be able to take a first run at? If so, happy to point you in the right direction :)
For posterity: API Docs https://developers.google.com/youtube/v3/live/docs/liveChatMessages

I could try to do something but usually I dont create code my self but fix things that I know ;D

Totally your call, but... I believe in you! It's mostly copying strategies from elsewhere that have been pretty robustly modelled in other bridges :)

I will start creating this now

It seems that this is really hard because I would need help to understand different things specific for go language and this library. :( So I'm desparate and there for stopped working. I have learned PHP procedular and OOP but they look so much different than Go.

 Maybe try browsing this: https://github.com/youtube/api-samples/tree/master/go
There aren't as many golang examples in there compared to other languages in that repo, but you can probably compare what's there and learn a lot :)
If you really don't feel ready to dive in, you might be able to try setting this issue up like a first-timer-only issue -- just set this issue up (or a new one) with as much information as possible for someone interested to learn Go, and then we can tag it as a first-timers-only issue and see who shows.
You might want to read up a bit on the philosophy of first-timers-only, so that you make the issue as welcoming as possible for whoever shows up from places like this:
https://twitter.com/first_tmrs_only
Public Lab is an organization that has tons of great templates and support on creating first-timers-only issues:

https://publiclab.org/wiki/software-outreach (down for maintenance for 2h right now)
example of how the get everyone creating FTO issues: https://github.com/publiclab/image-sequencer/issues/842
template they use for creating FTO issues: https://github.com/publiclab/plots2/blob/master/.github/first-timers-issue-template.md

Anyhow, hopefully that's not too much information :) Just trying to support you in helping yourself, and just because you're not feeling like you have the skills to do the work yet, that doesn't mean you can't support this project (and your own wishes!) by helping encourage someone else to drop in and make the contribution 🎉

Is anyone working on this?

@10by10pixel not that I know of.
Pull requests are welcome.

I'd also love to see this

Me too. This'd let me have my PeerTube audience and YouTube audience all talking in the same room.

i wish there was support for this
would be great to bridge my irc and discord channels to my youtube chat


Display package jsdoc

Minor typos in display package:

mantained => maintained
overriden => overridden

Codecov Report

Merging #7089 (f0bcca4) into dev (cef1bd9) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##               dev     #7089   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           17        17
  Lines          699       699
=========================================
  Hits           699       699

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cef1bd9...f0bcca4. Read the comment docs.