Support multi-sample VCF files as input

This PR addresses Issue #35 for supporting multi-sample VCF files.
The following changes were made:

removed the check for single_samples in vcf_check.py
changed create_vembrane_fields.py to automatically parse samples in vcf. This makes use of the "for_each_sample()" function implemented in vembrane. All fields specified in the parameters that affect the FORMAT fields are done for each sample in teh vcf and included into the reports with the VCF-internal sample name incorporated into the column name.
Changed documentation accordingly and fixed minor formatting and spelling errors.

I investigated this issue and it seems to be related with vembrane handling missing entries in the FORMAT field.
This is discussed in https://github.com/vembrane/vembrane/issues/171
Hence multi-sample VCF files only work if they do not have missing entries OR if the respective VCF header for the FORMAT field is changed from "Number=R" to "Number=." definitions of FORMAT fields.
We will fix this with an update of vembrane, so this PR can be merged.


Add an option for a 2-bar intro

We already analyse the tune structure to determine the various repeatable sections.  It should be possible to use this analysis to generate a 2-bar intro, appropriate for the given structure and eventually offer this to the user (who wants to play along).
Different approaches will be needed for a) no repeats b) simple repeats and c) variant repeats and each of these will have to be modified to take account of whether or not the tune already has an intro.

Fixed by merging the intro branch with final commit a3a4a1ac2d3bb0921d06a7cddca3abe915c29be9


Add Support for windows .manifest files through the add_files interface.

Windows .manifest files should be supported through the add_files interface, similar to how .rc files are handeled.

I think you can use add_ldflags directly. add_ldflags("/manifest", "/ManifestFile:\"xxx.manifest\"")

Can you provide a demo project with manifest file?

https://github.com/Force67/TiltedOnline/tree/launcherino/Code/launcher, currently i'm the manifest through https://github.com/Force67/TiltedOnline/blob/launcherino/Code/launcher/launcher.rc#L30

I have supported it. https://github.com/xmake-io/xmake/blob/dev/tests/projects/c++/manifest/xmake.lua
add_rules("mode.debug", "mode.release")
target("test")
    set_kind("binary")
    add_files("src/*.cpp")
    add_files("src/*.manifest")


chore(setup): Add gov-uk frontend and restify dependencies

Description of PR...
Changes

Item 1
Item 2

PR Template Checklist

[ ] Grammar
[ ] Spelling
[ ] Clarity

Fixes #85, Fixes #22, Fixes username/repo#123
Connects #123

Yup, got the commit syntax correct. I could argue that it would be better as chore(dependencies), but that would be overly picky.
PR template looks good


#26: Create unit test for LazilyInstantiate

Contribution toward #26

@spencergibb The new file uses tabs now. I did a force push to update my branch--let me know if that works.


Not possible to adjust the size of tiles (v0.11.0)

Environment
Docker
Version
0.11.0
Describe the problem
After updating to the latest version (docker), I cannot adjust the size of any tile from the Dashboard.
Also, the option to configure a general size for all tiles at once has disappeared, or it's located in another place?
Because of this, the icons are now in the default size that Homarr v.0.11.0 set, looking as shown below:

Also, when I try to configure the size of any tile, it does nothing after saving changes.
Also, is the width value type, compared to height, correct? > width (%) and height (px)?

Additional info
No response
Please tick the boxes

[X] You've read the docs
[X] You've checked for duplicate issues
[X] You've tried to debug yourself

Environment
Docker
Version
0.11.0
Describe the problem
After updating to the latest version (docker), I cannot adjust the size of any tile from the Dashboard. Also, the option to configure a general size for all tiles at once has disappeared, or it's located in another place?
Because of this, the icons are now in the default size that Homarr v.0.11.0 set, looking as shown below:
Also, when I try to configure the size of any tile, it does nothing after saving changes. Also, is the width value type, compared to height, correct? > width (%) and height (px)?
Additional info
No response
Please tick the boxes

[x]  You've read the docs
[x]  You've checked for duplicate issues
[x]  You've tried to debug yourself

Have you tried resizing in edit mode ?

@manuel-rw maybe we should make it so the title of Apps can't overflow on another line and instead truncate it ? https://mantine.dev/core/text/#truncate
I've also thought of an option to hide the title of a tile and only show it when hovering it with the mouse if the tile is set the the minimal size (1x1) what do you think about that change @SmartPhoneLover ?

Yes, truncation would be very good. We should implement this for 0.11.1

https://github.com/ajnart/homarr/issues/549
this is what I mentioned above @SmartPhoneLover

Have you tried resizing in edit mode ?

Yes, I already tried in Edit Mode. This is what happends when I try to adjust values and click on save changes:
https://user-images.githubusercontent.com/42898749/213166913-6a8c8d66-c31c-4c1a-adb7-89bb87e1527d.mp4
Note: The dynamic resizing (resize with mouse) works well, but I would like more control over it.

It is intended behavior that the PiHole app will be moved, since there is not enough space.
About the modal, this seems to be a bug.

@manuel-rw maybe we should make it so the title of Apps can't overflow on another line and instead truncate it ? https://mantine.dev/core/text/#truncate
I've also thought of an option to hide the title of a tile and only show it when hovering it with the mouse if the tile is set the the minimal size (1x1) what do you think about that change @SmartPhoneLover ?

Yeah, I like the option to be able to hide the titles of the tiles. If possible, offer the option to hide all titles at once from general settings, maybe.

It is intended behavior that the PiHole app will be moved, since there is not enough space. About the modal, this seems to be a bug.

Yes, I'm already aware of the behaviour when the Pihole tile moved from its original location.
So, the other related problem: confirmed to be a bug then? (@ajnart )

Yes, confirmed to be a bug.

I just did some investigation on this.
It's most likely, that the issue is being caused here:
https://github.com/ajnart/homarr/blob/727b4c4eca6a8565bfab01172e6d6f90234167be/src/components/Dashboard/Modals/ChangePosition/ChangeAppPositionModal.tsx#L25
Usually, that should never happen. Have you tried reloading the site?

Confirmed and fixed. Bugfix will be rolled out with 0.11.2

Fixed in #604


Dead space in diagram edit pane

According to the width of the main window, a vertical dead space is present on the right hand side of the diagram edit area:

Closing this issue as it now forms part of the reinstatement of the zoom feature issue #95


Show correct name of signal when editing an existing one

Use popup_dialog() instead of popup_centered() to show edit dialog/
With popup_centered it is not possible to change the displayed
signal name. When this is not set the previous shown name is show
for the current dialog.
This is no problem when creating a new conenction as popop_dialog
is used there and this would update the title.
Fixes #42074

Thanks!

Cherry-picked for 3.2.4.


Add debug information when SSL peers are being verified

It's useful when debugging the SSL configuration of agents to have an easy way to debug what's the SSL peer being verified and the trusted CA certificates being used.
This patch adds a debug message indicating the subject of the peer being verified, the subject of each CA certificate being used to verify signatures with and the result of the vefification for each pair.

Can one of the admins verify this patch?

Hi @nbarrientos, thanks for your contribution! The Puppet::SSL::Validator code is deprecated, and is not called by puppet itself as of recent 6.x releases. However, the deprecated code will be invoked if you're running puppet apply with a custom plugin (terminus, report processor, etc), and it calls the old Puppet::Network::HttpPool.http_instance method. The newer HTTP/SSL code uses Puppet::SSL::Verifier, so it would be good to make this change in both places:

We've had issues with non US-ASCII subject DNs in the past. Could you change the debug message to use peer_cert.subject.to_utf8 and authz_cert.subject.to_utf8?
Could you add a similar debug statement to the Verifier callback which is invoked once per cert in the server's cert chain in order from the root CA to the server's cert. So in https://github.com/puppetlabs/puppet/blob/master/lib/puppet/ssl/verifier.rb#L103  do something like:def call(preverify_ok, store_context)
  Puppet.debug("Verified #{store_context.current_cert.subject.to_utf8}, result #{preverify_ok ? OpenSSL::X509::V_OK : store_context.error}")

Note the old code unnecessarily compares the cert against all possible certs in the CA bundle, which is kind of meaningless when puppetserver's cert is issued from an intermediate CA. For example, trying to verify puppetserver's cert using the self-signed root CA is always going to report that verification failed (as it should). The other thing is that verification can fail for reasons other than OpenSSL::X509::Certificate.verify returning false. So you really need to the store_context.error to understand why it failed. I think it would be better to move the debug statement from has_authz_peer_cert to https://github.com/puppetlabs/puppet/pull/8187/files#diff-9391709bc53a43d9eef8c8fc0fd3046aR68. That way the behavior is the same for the old and new code paths.
The travis check failed because the subject line in the commit message needs to start with (maint)
The appveyor check failed due to an entropy issue on the worker. I'll rekick that.

Waiting for CLA signature by @nbarrientos
@nbarrientos - We require a Contributor License Agreement (CLA) for people who contribute to Puppet, but we have an easy click-through license with instructions, which is available at https://cla.puppet.com/
Note: if your contribution is trivial and you think it may be exempt from the CLA, please post a short reply to this comment with details. http://docs.puppet.com/community/trivial_patch_exemption.html

Hi @joshcooper
Thanks for your comments. I have to confess that our production Puppet version is 5 over here and I was just kind of blindly forward-porting the patch hoping my future me to be able to reduce the size of our stack of local patches :D. Not sure I'll have the time to address everything but I'll do my best. Perhaps in my spare time during the week-end.

Not yet, sorry :(

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@nbarrientos The code affected by this patch no longer exists in 7.x or main, so I'm going to close this. If you'd like to make the equivalent change in 7.x, take a look at my comment in https://github.com/puppetlabs/puppet/pull/8187#issuecomment-638994697


Wrap local classes in namespace for internal linkage

When the SDK is built using static libraries, classes that are local to a .cpp file with generic names like DateParser, can conflict with other classes and their constructors in an upstream project or package. This has caused crashes in our project because since our DateParser also uses a virtual destructor, it ends up getting called from the AWS code. Wrapping these date parser classes in anonymous namespace will ensure they are internally linked, according to the ISO C++ standard.
Example stack trace from the crash log:
#0  0x00005555582993ef in std::_Rb_tree<Str, std::pair<Str const, Str>, std::_Select1st<std::pair<Str const, Str> >, std::less<Str>, std::allocator<std::pair<Str const, Str> > >::_S_right (__x=0x48e0758948e87d89)
    at /usr/include/c++/9/bits/stl_tree.h:798
#1  0x0000555558294c0f in std::_Rb_tree<Str, std::pair<Str const, Str>, std::_Select1st<std::pair<Str const, Str> >, std::less<Str>, std::allocator<std::pair<Str const, Str> > >::_M_erase (this=0x7fffee1fca40, __x=0x48e0758948e87d89)
    at /usr/include/c++/9/bits/stl_tree.h:1913
#2  0x0000555558294c21 in std::_Rb_tree<Str, std::pair<Str const, Str>, std::_Select1st<std::pair<Str const, Str> >, std::less<Str>, std::allocator<std::pair<Str const, Str> > >::_M_erase (this=0x7fffee1fca40, __x=
    0x55555a006574 <ScopedPointer<BundlesSetup>::~ScopedPointer()+48>) at /usr/include/c++/9/bits/stl_tree.h:1913
#3  0x0000555558294c21 in std::_Rb_tree<Str, std::pair<Str const, Str>, std::_Select1st<std::pair<Str const, Str> >, std::less<Str>, std::allocator<std::pair<Str const, Str> > >::_M_erase (this=0x7fffee1fca40, __x=0x7fffee1fcb80)
    at /usr/include/c++/9/bits/stl_tree.h:1913
#4  0x0000555558294c21 in std::_Rb_tree<Str, std::pair<Str const, Str>, std::_Select1st<std::pair<Str const, Str> >, std::less<Str>, std::allocator<std::pair<Str const, Str> > >::_M_erase (this=0x7fffee1fca40, __x=0x7fffee1fca70)
    at /usr/include/c++/9/bits/stl_tree.h:1913
#5  0x000055555828f48e in std::_Rb_tree<Str, std::pair<Str const, Str>, std::_Select1st<std::pair<Str const, Str> >, std::less<Str>, std::allocator<std::pair<Str const, Str> > >::~_Rb_tree (this=0x7fffee1fca40,
    __in_chrg=<optimized out>) at /usr/include/c++/9/bits/stl_tree.h:995
#6  0x0000555558283142 in std::map<Str, Str, std::less<Str>, std::allocator<std::pair<Str const, Str> > >::~map (this=0x7fffee1fca40, __in_chrg=<optimized out>) at /usr/include/c++/9/bits/stl_map.h:300
#7  0x00005555595125e0 in DateParserGuesser::~DateParserGuesser (this=0x7fffee1fca10, __in_chrg=<optimized out>) at src/util/DateParser.h:197
#8  0x000055555950b7ac in DateParser::~DateParser (this=0x7fffee1fc8b0, __in_chrg=<optimized out>) at src/util/DateParser.cpp:324
#9  0x000055555ae17c8e in RFC822DateParser::~RFC822DateParser (this=0x7fffee1fc8b0, __in_chrg=<optimized out>) at /home/hyan/code/some/main/contrib/aws-sdk-cpp-1.8.8/aws-cpp-sdk-core/source/utils/DateTimeCommon.cpp:506
#10 0x000055555ae160a2 in Aws::Utils::DateTime::ConvertTimestampStringToTimePoint (this=0x7fffee1fcb70, timestamp=0x7fffedc8ed20 "Wed, 30 Sep 2020 00:07:00 GMT", format=Aws::Utils::DateFormat::AutoDetect)
    at /home/hyan/code/some/main/contrib/aws-sdk-cpp-1.8.8/aws-cpp-sdk-core/source/utils/DateTimeCommon.cpp:1328
#11 0x000055555ae14c15 in Aws::Utils::DateTime::DateTime (this=0x7fffee1fcb70, timestamp=0x7fffedc8ed20 "Wed, 30 Sep 2020 00:07:00 GMT", format=Aws::Utils::DateFormat::AutoDetect)
    at /home/hyan/code/some/main/contrib/aws-sdk-cpp-1.8.8/aws-cpp-sdk-core/source/utils/DateTimeCommon.cpp:1033
#12 0x000055555ad98a02 in GetServerTimeFromError (error=...) at /home/hyan/code/some/main/contrib/aws-sdk-cpp-1.8.8/aws-cpp-sdk-core/source/client/AWSClient.cpp:153

Note: the key part is here:
#8  0x000055555950b7ac in DateParser::~DateParser (this=0x7fffee1fc8b0, __in_chrg=<optimized out>) at src/util/DateParser.cpp:324
#9  0x000055555ae17c8e in RFC822DateParser::~RFC822DateParser (this=0x7fffee1fc8b0, __in_chrg=<optimized out>) at /home/hyan/code/some/main/contrib/aws-sdk-cpp-1.8.8/aws-cpp-sdk-core/source/utils/DateTimeCommon.cpp:506

I cannot show our code here, but the DateParser of our code has a constructor which takes a bool, but in C++ any pointer is treated as a boolean value on whether or not its nullptr. Either way, wrapping these classes in an anonymous namespace fixes the problem , and I believe it's the intention not to expose these classes externally anyways.
Check all that applies:

[x] Did a review by yourself.
[x] Added proper tests to cover this PR. (If tests are not applicable, explain.) Not applicable, this is a leakage of symbols when using the sdk as static libs
[x] Checked if this PR is a breaking (APIs have been changed) change.
[x] Checked if this PR will not introduce cross-platform inconsistent behavior.
[x] Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

[x] Linux
[x] Windows
[ ] Android
[x] MacOS
[ ] IOS
[ ] Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@KaibaLopez

@KaibaLopez any update on this ? This is a fairly simple change since functions / classes in the global namespace should be wrapped in an anonymous namespace to ensure internal linkage. This doesn't change any behavior.

@singku Friendly bump :)

@KaibaLopez friendly bump

Hi @jweinst1 ,
Sorry yea, we have reviewed/approved the PR and it should be merged in our next release
Again sorry for the slow response and thanks for the contribution!

@KaibaLopez  Any update on when this might be merged ?

Hi, Sorry for the lack of updates, but this was already merged since commit 2ec34e4 .
Thanks for the contribution!


Remove v1 metrics

Scope & Purpose
Remove v1 metrics from devel

[ ] :hankey: Bugfix
[ ] :pizza: New feature
[ ] :fire: Performance improvement
[x] :hammer: Refactoring/simplification

Checklist

[ ] Tests

[ ] Regression tests
[ ] C++ Unit tests
[ ] integration tests
[ ] resilience tests

[ ] :book: CHANGELOG entry made
[ ] :books: documentation written (release notes, API changes, ...)
[ ] Backports

[ ] Backport for 3.9: (Please link PR)
[ ] Backport for 3.8: (Please link PR)
[ ] Backport for 3.7: (Please link PR)

Related Information
(Please reference tickets / specification / other PRs etc)

[ ] Docs PR:
[ ] Enterprise PR:
[ ] GitHub issue / Jira ticket:
[ ] Design document:

We cannot delete v1 metrics. The API is used widely and we introduced it to not break existing dashboards

We cannot delete v1 metrics. The API is used widely and we introduced it to not break existing dashboards

But we deprecated v1 in 3.8? When we can remove it?
This changes for 3.10+

Deprecation does not automate a break of existing client-side methods. At least we should have approval of PM for a visible change like this. Could you please open an APM ticket?

@kvahed FYI deprecation of v1 endpoints, is apporved by @joerg84.


Esteemer processes message history

Esteemer currently doesn't process message history.
Proposal:
If the current candidate message template matches the message sent last month, set the message recency count to 1 (month). Apply this factor to the score.
If a message template was selected and delivered via email the previous month, Esteemer should score it significantly lower than other templates, so that it is only selected if no other message templates are available.
The downvoting should happen without regard to differences in measures, but if it is for the same measure, it should be downvoted more.
Don't do this now, but down the road we need to consider exceptions to this downvoting, for cases where continued improvement is motivating and receiving the same message about the same measure is a good thing.

Point 1

How are we recording message history into the input messages? Is that being done by the consortium when they generate input_messages or do we need infrastructure to implement this?

Point 2

If a message template was selected and delivered via email the previous month, Esteemer should score it significantly lower than other templates, so that it is only selected if no other message templates are available.

May be best accomplished with an MPM adjustment to the weight coefficient for message recency - setting the coefficient to -5, -10, -30, any sufficiently large negative integer. We can manipulate Message Recency with the following math so it starts large when the message is most recent, and decreases asymptotically so the 'downvote' effect decreases as time gets bigger. Bear with me here.

Recall history portion of the algorithm looks something like:
data component + (X_t)(Message Recency) + (X_m)(Measure Recency) + (X_n)(Number Received<?>)
We can manipulate Message recency (below notated as X) like this:
Message Recency term = X_t (e^(-X)) (X+1)^(-1))

Why is this so complicated now? Because without the second (x+1) term in a denominator when X=0 we would have the largest 'downvoting'.

Examples
Now we can evaluate this for the interval X = `Message Recency' = [0, ..., 4] to see how this impacts the overall rank:

Message recency = 0

Rank = ... + (-10)([e^(-0)] (0+1)^(-1)) + ...
Rank = ... + 0 + ...

Message recency = 1

Rank = ... + (-10)([e^(-1)] (1+1)^(-1)) + ...
Rank = ... + -1.84 + ...

Message recency = 2

Rank = ... + (-10)([e^(-0)] ()+1)^(-1)) + ...
Rank = ... + -0.45 + ...

Message recency = 3

Rank = ... + (-10)([e^(-0)] ()+1)^(-1)) + ...
Rank = ... + -0.11 + ...

Message recency = 4

Rank = ... + (-10)([e^(-0)] ()+1)^(-1)) + ...
Rank = ... + -0.01 + ...

Mull it over, I know it's tough to look at, but the implications are good! As message recency decreases, so does the overall rank, and with no recency the term is entirely ignored. We would only be evaluating this term once per candidate as well, so it's computationally efficient. We would likely want to use testing to determine a reasonable X_t value, depending on the median expected rank that a selected candidate will evaluate to. Maybe a change of -0.1 will move the needle, or maybe we need X_t to be -100 to make sure the needle is moving appropriately in the overall rank.
Point 3

The downvoting should happen without regard to differences in measures, but if it is for the same measure, it should be downvoted more.

We can use the same transform above on measure recency to get a similar effect, where more recent feedback on the same measure causes stronger decreases in the overall ranking. Because we have the measure and message recency terms separated, they run independently - it will downvote similar message templates, and do independent downvoting of repeated measures as well. Therefore, message and measure repetition will cause 'double' downvoting.

Objectives:

Pull in requisite data for calculations:

the current_feedback_month (month for which feedback is being generated), assign as time 0
Acceptable Candidates' measure(s)
Acceptable Candidates' message template name(s)

Make empty dicts for storing message and measure recency values

Compare row 'month' to t0, convert to integers representing distance from t0

Ex: t0 = Dec, Nov therefore t(-1), Oct = t(-2), etc

Calculate message recency

if acceptable_candidate['message_template_name'] matches a message_template_name value in matrix, calculate how many months it has been since that match
ex, if month is december now and candidate template is 'not top performer', calc months since last 'not top performer' noted in history

Calculate measure recency

if acceptable_candidate['measure'] matches measure in matrix, calculate months since that match

Per team meetings, will be resolving this issue, opening new issue to describe remaining work for getting history processing online.


Add client-side support for target_info endpoint

nice! mostly just nits, and _SuperstaqClient.target_info needs its own test
(also assume you've checked/will check this locally against the server endpoint?)

Yes, did check locally! But need to double check CSS (modified locally) before merging.


Add interim member status to working mode

Addresses https://github.com/ampproject/meta-ac/pull/95/.
/Cc @sumodas

:+1:

+1

👍

+1


Tooltip position to the mouse

It would be useful to beable to position the tooltip to the mouse insted of the element. That way, we can use in on multi-line element. Actualy, if an element is multi-lined, the tooltip display in the middle of the 2 part... not userful...

Don't think we want to make this follow the mouse at this point - thanks for the request though!

I'd like to complement asakurayoh's bug report with a screenshot from the Bootstrap site:
 (Taken on a Mac, Firefox 10.)
The tooltip placement in this case is quite confusing.

In a case like this, one workaround would be to add white-space: nowrap to those links. Not a very good workaround, though.


Map doesn't work on physical pixel 4a android 12

Will add more details later as I explore the bug in more detail but, for now make do with this:

applies to commit b80af4270fcfc504068c1f41270c46d16eab065a on JJ branch

I installed a Pixel 4a emulator running Android 12.0 with API 31, and the map did not crash.
From the bug report, a GeoLocation is being created with  new GeoLocation(Location location), but the Location object being passed to it is null.
Therefore locationManager.getCurrentLocation( ... ) on line 156 is returning a null location. I have no idea how MapController or LocationManager works, but the tests are inconsistent because the same setup emulated on my desktop seems to run fine. If you are looking into this bug, I hope this (somehow) helps.


Add support for Google Analytics 4

Hi!
It would be nice to have support for Google Analytics 4 ( gtag ).
/// _config.yml
google-analytics: G-xxxxxxxx

Also interested on this...

Hi! You can now customize this with v0.2.0. The Google Analytics code is now an include so you can overwrite it in your site: https://github.com/pages-themes/cayman#customizing-google-analytics-code

Does it work?
I guess you have to add gtag.


Bullet points remain unfaded when using [click]s

Describe the bug
A minor issue: when using [click]s in notes with (nested) bullet points, the bullet points of faded list elements that have one or more [click]s remain unfaded.

To reproduce
Steps to reproduce the behavior:

Run my presentation
git clone https://github.com/gy-mate/kalauz
cd kalauz/presentation
git checkout ef1967e5
npx slidev

Open http://localhost:3030/presenter/18?clicks=1

Desktop

OS: macOS 14.5
Browser: Safari 17.5 (same in the latest Chrome)
Slidev version: 0.49.10 (global)

@KermanX Thank you for your work!
I'd like to add that I would expect all bullet points of faded list elements to be also faded. Like the second bullet point in the picture of the issue description. Now (version 0.49.12) it looks like this:

And if you move to http://localhost:3030/presenter/18?clicks=6 I would also expect the circled bullet points to be faded:

What do you think about this?

Also, sometimes the list element that should be highlighted is not highlighted at all. If you go to http://localhost:3030/presenter/18?clicks=2 the list element "tul ritkak a merovonati bejarasok" should be highlighted but it isn't:

@gy-mate The [click] markers should be put before the content like this:
<!--
Content before the first click

[click] This will be highlighted after the first click

Also highlighted after the first click.

[click] This will be highlighted after the second click

[click:3] Last click (skip two clicks)
-->

Then the current behavior will be reasonable - The bullet points unfade when there is sone content in this <li> unfade.

@KermanX I see, thank you for the explanation! I've opened a PR to include these in the docs.


Custom loss function for sampling loss area

Hello,
I have an end-to-end network (like FCN), which has the input shape:
(batch_size, channel, img_col, img_row)
and the output shape:
(batch_size, img_col, img_row, num_class)
the labels are one-hot-encoded and have the shape:
(batch_size, img_col, img_row, num_class)
When computing the loss, I don´t want to compute the loss for the whole output but just compute on the randomly sampling part of the whole region. The custom loss function is like:
def custom_loss_fun(y_true, y_pred)

      from theano.tensor.shared_randomstreams import RandomStreams

      # reshape to (batch_size*img_col*img_row, n_class)
      y_pred = y_pred.dimshuffle(3, 0, 1, 2).flatten(2).dimshuffle(1, 0)
      y_true = y_true.dimshuffle(3, 0, 1, 2).flatten(2).dimshuffle(1, 0)

      # sampling
      mask_size = y_true.shape[0]
      rs = RandomStreams(seed=234)
      # make mask for sampling
      mask = rs.binomial(n=1, p=0.8, size=mask_size)
      # get the indices
      label_idx = (mask > 0).nonzero()

      y_pred = y_pred[label_idx] # size has changed
      y_true = y_true[label_idx] # size has changed

      return keras.objectives.categorical_crossentropy(y_true, y_pred)

The error is as follows:
Note: batch_size = 2, batch_size * img_col * img_row = 921600
Traceback (most recent call last):
  File "C:\lib\site-packages\theano\compile\function_module.py", line 859, in __call__
    outputs = self.fn()
ValueError: Input dimension mis-match. (input[0].shape[0] = 921600, input[1].shape[0] = 2)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
.......
  File "C:\lib\site-packages\theano\compile\function_module.py", line 859, in __call__
    outputs = self.fn()
ValueError: Input dimension mis-match. (input[0].shape[0] = 921600, input[1].shape[0] = 2)
Apply node that caused the error: Elemwise{mul,no_inplace}(mean, reshape_2_sample_weights)
Toposort index: 1709
Inputs types: [TensorType(float32, vector), TensorType(float32, vector)]
Inputs shapes: [(921600,), (2,)]
Inputs strides: [(4,), (4,)]
Inputs values: ['not shown', array([ 1.,  1.], dtype=float32)]
Outputs clients: [[Elemwise{true_div,no_inplace}(Elemwise{mul,no_inplace}.0, DimShuffle{x}.0)]]

Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
 .....
  File "C:/.....py", line 436, in train_model
    model.compile(loss=self.sampled_loss_CE, optimizer='adam', metrics=[keras.metrics.categorical_accuracy])
  File "C:\lib\site-packages\keras\engine\training.py", line 625, in compile
    sample_weight, mask)
  File "C:\lib\site-packages\keras\engine\training.py", line 344, in weighted
    score_array *= weights

HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

any ideas?

You probably have to reshape your loss tensor back to the original shape so it can be propagated back. Currently,  your metric is being called on y_true, y_pred that are of different shape than what the model.loss uses and expects downstream.

@pechersky Thank you for the reply. Sorry I had an error in the comments, the size of y_pred, y_true in the end has changed. Could you please elaborate?

Since the size of the tensors y_true, y_pred you are calculating your objective on has changed, the resulting objective tensor is now that new size. However, the model expects the objective tensor to be of the same shape as the original batch. This comes out in your code when the objective tensor (score_array) is being scaled by the weights you are using (it seems you are passing in sample_weight).
To fix it, I suggest figuring out how to "undo" the flatten and dimshuffle operations, so that you get back an objective tensor where the first dimension is your batch_size. This will allow your score_array to be scaled properly, and then meaned.

Did it work?

@pechersky thank you for your help, it seems to be working now, I'll check the result when the training is done.

@pechersky actually I've got another problem, say the values of the label y_true are ranging from 0 to N, now I want to ignore all the 0s when calculating the loss. Then for every single image in one batch, the number of pixels will be different, then I guess it wouldn't be possible in Keras? @fchollet

You want to supply a sample_weight when you fit, that has 1 where you
want to keep the pixel, and 0 where you don't. Since your inputs seem to be
2D per sample, you'll have to have make sure the weights are the right
dimension. If you want to weigh rows instead, as in have weights be one
dimension smaller, then compile the model with
sample_weight_mode='temporal'.
On Nov 13, 2016 1:55 PM, "GC"<EMAIL_ADDRESS>wrote:

@pechersky https://github.com/pechersky actually I've got another
problem, say the values of the label y_true are ranging from 0 to N, now
I want to ignore all the 0s when calculating the loss. Then for every
single image in one batch, the number of pixels will be different, then I
guess it wouldn't be possible in Keras? @fchollet
https://github.com/fchollet
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/issues/4345#issuecomment-260204430,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFGDhnusVx5Zva_4yOACDZsJACCBe6jnks5q910egaJpZM4KuqPL
.

@pechersky I think in this case maybe the class_weight would make more sense? But I still find it not so easy to do. The thing is, I have N+1 classes in the labels but now I'm only interested in N classes and the model has already been defined to classify N classes, so my intuition to do this is to mask the original labels, delete the undesired class label values and put the rest in loss calculation.


Node v16 Released  🎉

:tada: :tada: :tada: :tada:  https://nodejs.medium.com/node-js-16-available-now-7f5099a97e70
PLEASE don't wait a year to implement this in a new base image!

Would love this. Any update?

It's been nearly4 months since the v 16 release. Will AWS Lambda support v 16 in the near future? @carlzogh @krk

v14 dose not plan to support Apple M1.
So work in v14 is little annoying.
We need v16 ASAP.
https://github.com/nodejs/build/issues/2474#issuecomment-839683917

Seems AWS only does LTS builds so according to the schedule https://nodejs.org/en/about/releases/ it'll be around November.

Seems AWS only does LTS builds so according to the schedule https://nodejs.org/en/about/releases/ it'll be around November.

Hopefully we'll also get proper module support also.

Is there a way to create a custom runtime to use the latest version of node? If so, can anyone point me in the right direction?

@paulcanning you have to use a docker image function, and the cold start on those is pretty bad.

any warriors? :)

bump

Also here's the current runtimes for lambda for reference:
https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html

@Robert-Rendell looks like you might be new to Github. there's a great "Edit" function for replies that wont spam subscribers to this thread.

@Robert-Rendell looks like you might be new to Github. there's a great "Edit" function for replies that wont spam subscribers to this issue.

Yes I'm an escapee from bitbucket, thank you for the tip!

+1

I think they are waiting for re:Invent where they will present this as big feature :)

Github Actions as of today will start moving to Node 16 as the default for actions not specifying a version. Meanwhile AWS doesn't even provide a roadmap or any signals about this... Way to go from the market leader 🙄
For anyone stuck waiting, to avoid any issues, you should specify a node version, when setting up node in your .yml file.
- uses: actions/setup-node@v2
  with:
    node-version: '14'

AWS: Node.js 14 is in maintenance-mode. Users deserve to know when you're going to support the long-term support version of their runtime will be available.

@o5 do you happen to know if they mentioned anything at Re:Invent?

If node 14 support timeline is any indicator, we wont see v16 til Jan-Feb
https://aws.amazon.com/blogs/compute/node-js-14-x-runtime-now-available-in-aws-lambda/

@jessecollier hopefully, although having to wait the best part of a year to get an LTS version of Node is a literal joke.

If node 14 support timeline is any indicator, we wont see v16 til Jan-Feb https://aws.amazon.com/blogs/compute/node-js-14-x-runtime-now-available-in-aws-lambda/

If node 12 support timeline as an indicator then we should have v16 support by saturday this week. :crossed_fingers:

@carlzogh Any updates on this please? Probably the worst thing is leaving people in the dark about this, if we know it's in 3 months, at least we can plan accordingly in our organizations.

1

wtf is the deal, this needs to have been addressed months ago

The difference between the urgency that orgs like GitHub (and many other services) and AWS have with staying up to date with latest versions... AWS deserves to die

if anyone on AWS team is happening to read this, Google is beating you
https://cloud.google.com/functions/docs/concepts/nodejs-runtime

Well that's embarrassing ☝

sigh.. anything worth developing would be faster and easier using Tornado instead

sigh.. anything worth developing would be faster and easier using Tornado instead

Can you guys stop spamming the thread? Tons of useless emails sent for no reason ffs

If you can like her comment on twitter maybe it will help us : https://twitter.com/sarutule/status/1479018085789179905?s=21

I had to remove my last comment, the link to the tweet I sent was link to a person not in charge of Lamba but in AWS
sorry for the false hope.
She received also few unkind message.
I agree it’s annoying we all wait for this, but please AWS, or any people working there do not deserve base messages.
the only way to make things improve is by trying to understand each other’s :)

Not node 16.... But modules and top level await now supported in 14
https://github.com/aws/aws-cdk/issues/13274#issuecomment-1007262232

Just for context:
https://github.blog/changelog/2021-12-10-github-actions-github-hosted-runners-now-run-node-js-16-by-default/

This is a blocker for our organization. We've migrated from Node 14 to 16 for M1 support. Our lambda projects are blocked without a Node 16 runtime.

@ahawkins you'll have to use docker and upload an image for the lamba functions. fwiw this will drastically increase cold start times.

@shellscape Thanks for suggesting this workaround.

This is a blocker for our organization. We've migrated from Node 14 to 16 for M1 support. Our lambda projects are blocked without a Node 16 runtime.

We are in the same situation

It's really tough to understand why this is taking so long.

@mwilson71 bureaucracy mostly. but then think about the number of services that Amazon offers, and how much of that probably tangentially runs off of lambda. then consider all of the internal business processes and serverless functions that Amazon must use to operate the retail business. combine all of that and it makes sense why something like this would take over 6 months. not saying it's right, or couldn't be prevented, but it does make some assemblance of sense.

@shellscape I don't see how that could be true ... AWS doesn't need to change from Node X to Node 16 where they might be using Node X. This is just about making an additional runtime environment available. Shouldn't impact their existing services. I think we're just seeing that they don't have enough resources dedicated to this, and it's not as important to them than other things, which is more disheartening.

@jthomerson if I were running an org that big, I'd want to test a new version image against everything. Just because it's inconvenient to us all doesn't mean there's not a reason. If you're vendor locked, be patient. No amount of hand ringing or issue comments is going to make this move any faster.

This is such a joke

@jthomerson if I were running an org that big, I'd want to test a new version image against everything. Just because it's inconvenient to us all doesn't mean there's not a reason. If you're vendor locked, be patient. No amount of hand ringing or issue comments is going to make this move any faster.

are you in fact Jeff Bezos in disguise doing damage control

If I had to guess, we have to wait until Amazon Linux 2022 becomes GA, before new runtimes for Lambda will be released.

@krk @carlzogh
Sorry, if I'm pinging the wrong people, but you are the only contributors apart from @amazon-auto.
Could we get some clarity from AWS as in when to expect this? I am sure you understand we all need to make decisions when designing solutions for our customers. It would be great if we could get some information, please 🙏.

PLEASE don't wait a year to implement this in a new base image!

And they did!!!

An estimate would be nice at least, this is very frustrating..

C'mon Amazon we need a response! It's been a year!

This is a mission critical failure. Any response from the @aws-lambda-runtimes team?

This issue is by far the most frustrating thing about AWS so far... :/

@Negan1911 @jernejc @TRCSamurai @decentralgabe
You're probably new at GitHub, so please look at my screenshot.  MY EYESS!!!  I'm using dark mode

Thank you everyone for your feedback regarding AWS Lambda support for Node 16. Work is in progress, and we are planning to ship a Node 16 managed runtime and base container image within the next 90 days.
We also apologize for the confusion caused by locking this thread. We are addressing the gap in our process that caused it to happen. The feedback and engagement from this community is very valuable to us, and we will keep iterating on improving how we engage here.
If you have any suggestions for specific features or enhancements you would like to see included in our Node 16 support, please let us know on this thread.

Thank you everyone for your feedback regarding AWS Lambda support for Node 16. Work is in progress, and we are planning to ship a Node 16 managed runtime and base container image within the next 90 days.
We also apologize for the confusion caused by locking this thread. We are addressing the gap in our process that caused it to happen. The feedback and engagement from this community is very valuable to us, and we will keep iterating on improving how we engage here.
If you have any suggestions for specific features or enhancements you would like to see included in our Node 16 support, please let us know on this thread.

Would the v3 SDK been integrated into the image? Thanks

Would the v3 SDK been integrated into the image? Thanks

That's an open question we're working through right now. If you have strong opinions, we'd be interested to hear them! For example, is it useful to ship the SDKv2 and SDKv3 side-by-side, so you can upgrade functions to Node 16 without switching SDK? Or is it OK to ship the SDKv3 only, and require an upgrade to Node 16 to also mean upgrading to SDKv3?  (Obviously this only applies to functions using the bundled SDK.)

@jtuliani It’d be better if no SDKs (v2 and v3) are included in v16+ as they increase cold starts.
However, to have a more user-friendly approach, please consider to:

let a user to choose an image without an SDK
and by default offer an image with SDK v3 or v2

My preference would be to include the v3 SDK as long as it doesn't noticeably affect cold starts.

I'd vote to include both versions side by side, or at the very least just v3

Would the v3 SDK been integrated into the image? Thanks

That's an open question we're working through right now. If you have strong opinions, we'd be interested to hear them! For example, is it useful to ship the SDKv2 and SDKv3 side-by-side, so you can upgrade functions to Node 16 without switching SDK? Or is it OK to ship the SDKv3 only, and require an upgrade to Node 16 to also mean upgrading to SDKv3? (Obviously this only applies to functions using the bundled SDK.)

It depends on the affect on cold start. If not much then no brainer - both or V3.. otherwise better to be none of them.

I think it's best not to have surprises: I'm expecting an image without pre-existing packages. Plus, this means that it's easier to increment the version in case the SDK has security vulnerabilities

I think it's best not to have surprises: I'm expecting an image without pre-existing packages. Plus, this means that it's easier to increment the version in case the SDK has security vulnerabilities

Thank you everyone for your feedback regarding AWS Lambda support for Node 16. Work is in progress, and we are planning to ship a Node 16 managed runtime and base container image within the next 90 days.
We also apologize for the confusion caused by locking this thread. We are addressing the gap in our process that caused it to happen. The feedback and engagement from this community is very valuable to us, and we will keep iterating on improving how we engage here.
If you have any suggestions for specific features or enhancements you would like to see included in our Node 16 support, please let us know on this thread.

@jtuliani thank you for an official update, but could we please get an actual answer as to why this is taking so long? And will we have to wait as long when the next node version hits LTS for another new image?

@jtuliani thanks for the apology. I feel the community has been kept in the dark for a while (9 months) regarding the release of this image. This led to confusion in the community that only increased when NodeJS 14 went into maintenance mode.
Your comment underlines that closing the thread was a mistake and that you're going to find out why, and that is already great.
I'm more interested into the bigger issue though. What's your analysis of the (high) delay between Node 16's release and the  availability of a new base image?

I think it's best not to have surprises: I'm expecting an image without pre-existing packages. Plus, this means that it's easier to increment the version in case the SDK has security vulnerabilities

Fully agree. Additionally, users who are using ES modules and a bundler with tree-shaking at build time already, do not need (or even do not want) the SDK in the base image imo. Maybe a solution to tackle the different worlds could be to provide different lambda layers for the SDKs. Just my2c.

I would also be in favour of not including the SDK in the image, allowing folks to bundle the version they want to use and keeping the image lean for those who don't need the SDK.

I would advocate for having v2 in Node <= 14 and v3 in Node >= 16. I assume that the sdk is contained in a layer so there seems to be no impact on the cold start by having it (see e.g. https://medium.com/consulner/performance-of-aws-lambda-with-and-without-layers-9bffbb5434f3). However, not having the sdk in the image means that we have to bundle it with the code, which means a bigger package, which in turn means bigger cold start time. Am I wrong on that?

This article by Yan Cui should help to understand factual implications of an SDK and bundling only necessary parts of it on cold starts.
The reason why the layer didn’t affect cold starts noticeably in the comment above is (quoting the author):

In my case the layer contained only some static content (jar files in particular)

so in the tests of the layer above the function doesn’t actually require/import anything from the layer as unused static files were included, thus not affecting the cold starts

so in the tests of the layer above the function doesn’t actually require/import anything from the layer as unused static files were included, thus not affecting the cold starts

So with this line of reasoning, the SDK will act as unused static files for people not using it (because it won't be imported), thus not affecting the cold start for them.
For the others, only what is used will be imported in both cases (bundled or provided), but with a provided SDK the package will be smaller, reducing the deployment time.

@EHadoux please consider to familiarize with tree-shaking and therefore the fact that you cannot do tree-shaking on the provided SDK, thus importing more than needed affecting actually both cold starts and performance.
With sdk v2 the problem is even bigger than with v3 as it’s modularity is worse.
You should forget about comparisons with static files and consider cases when an SDK is used.
But coming back to even static files, it’s theoretically impossible to claim that bigger (size-wise) containers would load exactly the same as smaller ones.

I came here looking for a way to run a lambda function in docker with a node 16 base image. I'm not entirely sure if that's what these images are for of if this is the wrong place to post this feel free to remove this comment.
Otherwise if anyone needs a solution in the meantime until an official base image is create, this is works for me:
# Define custom function directory
ARG FUNCTION_DIR="/function"

########################### Base ###########################
FROM node:16.13.2-buster-slim as base
ARG FUNCTION_DIR

# Make sure the directory exists and set it as the default for all future images
RUN mkdir -p ${FUNCTION_DIR}
WORKDIR ${FUNCTION_DIR}

########################### Lambda runtime ##############################
# This step is required for the docker container to be executed on lambda.
# We do this as a seperate build stage to reduce the size of the output
# docker image and also to seperate the logic specific to lambda
FROM base as lambda-installer

# Install aws-lambda-cpp build dependencies
RUN apt-get update && \
    apt-get install -y \
            autoconf \
            automake \
            cmake \
            g++ \
            libcurl4-openssl-dev \
            libtool \
            make \
            python3 \
            unzip

# Builds and installs aws-lambda-ric
RUN npm install aws-lambda-ric

# Uncomment the below if you want to install sharp
# RUN npm config set ignore-scripts true
# RUN npm install --platform=linux --arch=x64 sharp --unsafe-perm

########################### Installer ##############################
FROM base as installer

# Copies the files needed for installing
COPY ./package.json ./

# Install
RUN npm install --production

########################### Final Lambda App ###########################
FROM base

# Copy in the lambda node modules
COPY --from=lambda-installer ${FUNCTION_DIR}/node_modules ${FUNCTION_DIR}/node_modules

# Copy in the installer node modules
COPY --from=installer ${FUNCTION_DIR}/node_modules ${FUNCTION_DIR}/node_modules

# Copy the built package
COPY ./dist ./dist/

# If this directory does not exist, lambda shows an annoying warning
RUN mkdir -p /opt/extensions

ENTRYPOINT [ "/usr/local/bin/npx", "aws-lambda-ric" ]
CMD [ "dist/index.handler" ]

@EHadoux please consider to familiarize with tree-shaking and therefore the fact that you cannot do tree-shaking (and minification as well as optimization of the SDK for the specific node.js version of the container) on the provided SDK, thus importing more than needed affecting actually both cold starts and performance.
With sdk v2 the problem is even bigger than with v3 as it’s modularity is worse.

Not to mention always HORRIBLY out of date.
At the time of this writing, lambda, at least in US-EAST-1, are running version 2.1001.0. The current version according to https://github.com/aws/aws-sdk-js/blob/master/CHANGELOG.md is 2.1069.0
The other reason is that what you install, lint, and run tests against (locally or through build pipeline), are almost NEVER the version that's installed in the lambdas. So unless you include your own SDK, you always run the risk of either A. It not working or B. crashes altogether.
I find including the SDK to be completely a waste for anything production worthy.
I'd really like the lambda system to have global AWS repository of layers that you could include but don't have too.  AWS-SDK would be one of them. I do find it useful to have to run feature tests in lambdas, where I write the code directly in their editor.

The whole purpose of v3 was to allow consumers to only import parts which you want to reduce the bundle size, so including all of the v3 modules would be defeating the purpose of it.

@jtuliani semi-related but it would be nice if the AWS docs mentioned exact version of node that is running, and not only "12.x" or "14.x"; or document the policy -- how often are those minor versions updated? Are they reasonably up-to-date with the most recent node release in a given line?
Compare AWS docs vs GitHub actions runners specs

@michaelhays Thanks, link fixed!

Thank you @jthomerson! I noticed increased latency of function creation but no noticeable difference in startup latency or function execution time. Having said that, it was not a very scientific measurement.

Nice initiative @tjanczu https://github.com/tjanczuk!
However, those depending on AWS lambda official node support through other
vendors like Vercel are left with no option, it's still important and
crucial for AWS to officially support this.
Imagine Vercel still being stuck with Node v14...
Disclaimer: I do not work at Vercel, I'm just one of their users.
On Tue, Feb 15, 2022, 8:15 PM Tomasz Janczuk @.***>
wrote:

Thank you @jthomerson https://github.com/jthomerson! I noticed
increased latency of function creation but no noticeable difference in
startup latency or function execution time. Having said that, it was not a
very scientific measurement.
—
Reply to this email directly, view it on GitHub
https://github.com/aws/aws-lambda-base-images/issues/14#issuecomment-1040689200,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHP73ECXIL2ZPJSJ53RADDU3KQ6ZANCNFSM43IVOWLQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Thanks @tjanczuk for the huge work!
I have asked vercel on Twitter if they can let us use it instead of official one.
It can be a suitable alternative, unlock most of us, and remove pressure of AWS
https://twitter.com/martindonadieu/status/1493727339288670208?s=20&t=mDwKVHjjdoPTeQ2vQLuPQQ

I also would prefer to not have any packages installed by default as this will add unnecessary bloat to the runtime. Using modern tooling with tree shaking and bundling should in most cases be the optimal solution. I would however see that an official separate layer could be provided for certain use cases (maybe this could be a part of the build process for v2 and v3 to create these?).

Thanks @riderx. We're already looking into this, and great work @tjanczuk. Our main concern right now is compatibility, and maintaining confidence that the bootstrap code works consistently across versions. We would likely need thorough testing of each version before we could deploy it.

Vercel users, go upvote my comment here to let Vercel know that adding support for fusebit/everynode would be very helpful.

🎉 🎉 🎉 🎉 https://nodejs.medium.com/node-js-16-available-now-7f5099a97e70
PLEASE don't wait a year to implement this in a new base image!

Narrator: It was a year before it was implemented in a new base image.

🎉 🎉 🎉 🎉 https://nodejs.medium.com/node-js-16-available-now-7f5099a97e70
PLEASE don't wait a year to implement this in a new base image!

Narrator: It was a year before it was implemented in a new base image.

and node 18 LTS is in less than a year, at this pace maybe start planning the base image / runtime for node 18...
but node 16 will be great too!!! following LTS release pace would be perfect.

I really don't understand why there is no progress on this, this is a shame from AWS

@thematan your comment is unnecessary and rude. It has no place here.
While I will publicly say that I think AWS is really dropping the ball here, that's AWS as an organization. When you call someone "losers" you're talking about the people. Remember that there's a lot of really good people at AWS who are working their hardest. But they have to work within the priorities given to them by their management. What we need to be discussing here is prioritization, based on customer needs. This fits with one of AWS' core values - customer-obsession. What we don't need to be doing is bad-mouthing the people who are working hard there, just trying to feed their families like all of us. I'd appreciate if you would consider removing your comment.

Hi think @jthomerson said it all. Let's end the string of written complaints here and now if not bringing anything new to the table. And if you do want to complain nevertheless, find the comment you most agree with and upvote it.

AWS are losers

To AWS developers: I think you're all doing a great job.
To AWS managers: It's time to listen to your developers and customers and prioritize this issue.

Thank you everyone for your feedback regarding AWS Lambda support for Node 16. Work is in progress, and we are planning to ship a Node 16 managed runtime and base container image within the next 90 days.
We also apologize for the confusion caused by locking this thread. We are addressing the gap in our process that caused it to happen. The feedback and engagement from this community is very valuable to us, and we will keep iterating on improving how we engage here.
If you have any suggestions for specific features or enhancements you would like to see included in our Node 16 support, please let us know on this thread.

this is the only reply that matters. day 24 of 90. 66 more days until the deadline.
may the 4th be with AWS devs.

@fitzgerald1337 where did you see that reply?

If you have any suggestions for specific features or enhancements you would like to see included in our Node 16 support, please let us know on this thread.

@jtuliani I'd love to see the team responsible for these images publicly commit to a release scheduling that follows Node LTS' own schedule. AWS is currently behind GCP, CloudFlare, Vercel, and Supabase (in beta) with regard to LTS Node version support. These are the kinds of gaps in community engagement and trust building that lead independent and smaller shops to competitors, and eventually pull away larger customers. Such a commitment would enshrine reliability and trust for the community around an extremely popular platform.

Already 6+ months after LTS :-D

/reminder in 183 days

+1
following
👍🏻

PLEASE don't wait a year to implement this in a new base image!

getting closer 😔

Node 16 transitioned to LTS 5 months ago and AWS still doesn't support it? Surprising and kinda lame really. At least put up a blog post discussing why this is so late. It's not like new Node versions are a surprise. They're released on a very predictable cadence.
No bueno AWS.

following +1

locked down much
https://github.com/opensearch-project/OpenSearch/issues/1400#issuecomment-948921005

for the stockholm haters just making it clear that I mistakenly posted this from my pro account, so hey 👋 I'm vincent, and not worth to elaborate when the reason for the trolling are obvious

I have literally no idea what you are drivelling on about, but I'm guessing you're going to wake up with a big hangover and a lot of embarrassment in the morning...
So this isn't wasted post Connor I just want to add my thanks to @tjanczuk for doing what the world's largest tech company couldn't.

following +1

There's a "subscribe" button @rawpixel-vincent -- been there for ohhhh 5 years now?

@digitaltoast I appreciate it, thank you. Any time. Every node ;) https://github.com/fusebit/everynode

@digitaltoast I appreciate it, thank you. Any time.

huh?

Every node ;) https://github.com/fusebit/everynode

Hmm that actually might be useful...

following. I've been waiting for a v16 runtime for a while now.

@Prophet32j and everyone who wants to follow a thread/discussion on Github. Use the button in the screenshot to do so instead of commenting.

Thank you @tjanczuk .
For others using SAM Templates for local development, here is a fairly streamlined example of using the latest Node Version from fusebit.  Note that the Runtime is rendered inconsequential by the Layer list.
It takes a second the first time for the layer to be pulled down and the docker image built.  Then you're good.
AWSTemplateFormatVersion: 2010-09-09
Transform: 'AWS::Serverless-2016-10-31'
Description: Fusebit for the win

Globals:
  Function:
    Runtime: nodejs14.x
    MemorySize: 128
    Timeout: 15

Resources:
  ServerlessHttpApi:
    Type: AWS::Serverless::HttpApi
    Properties:
      Description: Amazon API Gateway HTTP API + Lambda Function.
      ProtocolType: HTTP
      CorsConfiguration:
        AllowOrigins:
          - '*'
        AllowMethods:
          - GET

  DefaultLambdaFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: handlers/testing.handler
      Layers:
        - "arn:aws:lambda:us-east-1:072686360478:layer:node-17_8_0:1"
      Events:
        Other:
          Type: HttpApi
          Properties:
            Path: /{proxy+}
            Method: get
            ApiId: !Ref ServerlessHttpApi

@jtuliani

If you have any suggestions for specific features or enhancements you would like to see included in our Node 16 support, please let us know on this thread.

Your https://github.com/aws/aws-lambda-nodejs-runtime-interface-client needs to be upgraded with the Node 16 release.  I strongly encourage you to keep it up to date https://github.com/aws/aws-lambda-nodejs-runtime-interface-client/issues/44#issuecomment-1032928940 and merge https://github.com/aws/aws-lambda-nodejs-runtime-interface-client/pull/8 to avoid complex and long build steps.  In addition, I would like to see compatibility stated for AWS Linux 2022.
It is vital to maintain parity between aws lambda zip and aws lambda docker deploys.

@tjanczuk Don't do Promise instance check using result instanceof Promise. Global Promise can be mutated by third party libraries like Bluebird and Instrumentation support libraries like async-listener.  Official Lambda runtime had issue with their first Node.js 8 lambda runtime release. See https://github.com/aws/aws-xray-sdk-node/issues/27#issuecomment-380081229 for historical records.

@jtuliani

That's an open question we're working through right now. If you have strong opinions, we'd be interested to hear them! For example, is it useful to ship the SDKv2 and SDKv3 side-by-side, so you can upgrade functions to Node 16 without switching SDK? Or is it OK to ship the SDKv3 only, and require an upgrade to Node 16 to also mean upgrading to SDKv3? (Obviously this only applies to functions using the bundled SDK.)

IMHO It would be great if both SDK are available in newer Lambda runtime (not meant for Container Image)
Keeping SDK V2 will give users backwards compatibility - Most programs will run in newer runtime without additional migration. Previously Node.js had multiple breaking changes (e.g. Stream implementation changes in v0.8 ~ v0.12), but nowadays most core modules are stable or frozen so same deployment artifact will be okay in newer Node.js runtime.
but if Node.js 16 Lambda runtime removes AWS SDK V2, it forces users to change their applications - It forces developers to include AWS SDK packages to their artifact. and No one likes unexpected change. I believed that every AWS Lambda runtime has "bundled" packages. but now i realize it can be broken - Also I remember when AWS Lambda changed base OS to Amazon Linux 2. A lot of built-in system packages were removed in Amazon Linux 2 and Migrating existing softwares were super annoying - I had to rebuild build environment, rebuild native dependencies (e.g. ImageMagicK, OpenSSL dependent softwares), rebuild native addons, and repackage everything - It was totally nightmare.
Introducing AWS SDK v3 should be worthless because AWS SDK v3 has more modular and bundler friendly that AWS SDK v2. but It would be worth if function package is heavy (e.g. deployment size exceeds 250MB with AWS SDK V3).
For performance-wise issues, I don't think that would make negative performance impact. because I think most "AWS-provided" official Lambda runtime would be cached in their compute infrastructure. Also Node.js runtime does not load dependences unless you importing it.
and for Runtime Container Image - Just give choices to developers.

@mooyoul Thanks for the pointer, we will fix this.

I would just turn all results into a promise. I've done this in several libraries.
try{
  tmp = await Promise.resolve(handler(body, context, done))
} catch(e) {
  reject(e)
}

That way I don't have to do special processing for a function that may or may not return a promise, it just naturally "unwraps".

@jtuliani

That's an open question we're working through right now. If you have strong opinions, we'd be interested to hear them! For example, is it useful to ship the SDKv2 and SDKv3 side-by-side, so you can upgrade functions to Node 16 without switching SDK? Or is it OK to ship the SDKv3 only, and require an upgrade to Node 16 to also mean upgrading to SDKv3? (Obviously this only applies to functions using the bundled SDK.)

I would prefer if the Node Lambda image is as small as possible, that means it should not include SDK V2 (or V3 for that matter). I understand the argument "it forces users to change their applications", but for a Node-Lambda version change, that is to be expected, and also should be the case. I rather remove aws-sdk from my exclude list in the bundle script or add it to the dependencies list, than have it included in the runtime.
As a solution for those who do not want to change their build output, AWS could potentially provide an official and public layer containing the SDK v2, and maybe also provide a layer for each V3 package.
The above is my preferred outcome, but if you decide to include the SDK in the runtime rather than providing layers, I would prefer that both versions would be included.

I'd prefer a combination of those two approaches:

Lambda by default builds with no layers, as light as possible.'
The console (and CLI) allows the addition of the most commonly added layers with a couple mouse clicks. Example: a checkbox to add AWS SDKv2 as a layer.
Stretch goal: a select dropdown for SDKv3, that allows you to add layers for the clients you need in a fast but fine-grained way.

@halvardssm

I would prefer if the Node Lambda image is as small as possible, that means it should not include SDK V2 (or V3 for that matter). I understand the argument "it forces users to change their applications", but for a Node-Lambda version change, that is to be expected, and also should be the case. I rather remove aws-sdk from my exclude list in the bundle script or add it to the dependencies list, than have it included in the runtime.

Why would you EVER put it in an exclude list. That's a recipe for problems. If you are building against the current runtime version (2.1001.0) you are missing out on 112 releases (2.1113.0) including bug fixes. If you are building against the current, you are testing your code against a newer version but running it on a way older version. Unless its TEST code, you should always use the sdk for which it has been tested against. I would trim other code than use a library you are not running tests against.

Is there another thread or discussion you all can jump into to hash this out? I'm subscribed here for updates from AWS on the original post.

@shellscape
It does pertain to the original post, as it was asked what did we want in the image of Node 16. I, along with several people, hope they remove the javascript SDK from the node image. It may not fit your limited scope of what pertains to the post, but it does to many other people.

Cool, cool. Well then.

If you are building against the current runtime version (2.1001.0) you are missing out on 112 releases (2.1113.0) including bug fixes

Exactly why I would like to not have it included in the runtime, and rather have layers that can be versioned and updated alongside the SDK.
@dl748 funnily enough, this is sometimes the only solution if your function is too large as v2 does not allow for tree shaking.

Hello, why did this get closed? I am subscribing in here for the runtime image version update, but I did not see that actually happened, right?

Cool, cool. Well then.

This issue is not resolved. Why have you closed it?

I dont want to lose the context in this issue, @shellscape do you mind re-opening it? Thanks!

Folks, AWS isn't interested in resolving this in any semblance of a timely manner. We're 8 days shy of a year since I created the issue. Too many people who are commenting on this issue don't understand the concept of "off topic," and I've grown tired of notifications from replies by people like @dl748, who's reply you can thank for prompting me to close this. I'm not going to reopen this, and I won't see further replies. You're welcome to open your own issues for tracking this.

@shellscape couldn't you just unsubscribe from more updates?

No, no, he had to throw his giant rant onto us

I get it: I see that the OP was told that he didn't understand what the topic was. While it may have grown beyond the original scope, the OP gets to decide what the scope is.
While closing this is petty, it's within the OP's rights. Especially when this request is getting so long in the tooth.
Maybe one of us should open a new post and link it back here? Just be warned: you'll get a lot of comments, be ready for them! :D - haha Just as I was writing this a few people did!

@shellscape https://github.com/aws/aws-lambda-base-images/issues/14#issuecomment-1029958936

If you have any suggestions for specific features or enhancements you would like to see included in our Node 16 support, please let us know on this thread.

yeah, i totally get why OP would want to close this shit show of a thread; i totally get why the people following it want it to stay open; i think ultimately what it comes down to is that AWS devs/reps (@jtuliani) should have an official thread and shouldn't assume that OP's would automatically serve as an official one.
please, AWS, get a better dev advocate(s) on this.

What I suggest is that everyone who would need NodeJS 16 support on Lambda to regularly reach out to AWS support to remind them we need this feature. I think this is a best way than going through this GH issue.
Personally I don't really mind what is included in the image, I just want an AWS supported way of running NodeJS 16 in Lambda, and I want it ASAP, because we migrated our other systems to NodeJS 16, and we are not fond of juggling with several versions of Node.

What I suggest is that everyone who would need NodeJS 16 support on Lambda to regularly reach out to AWS support to remind them we need this feature. I think this is a best way than going through this GH issue.

Agree, I think everyone should reach out to the AWS TAM and reference this issue as source. So that we have both parts running (internal TAM and public nudge as well as transparency).

Thank you everyone for your feedback regarding AWS Lambda support for Node 16. Work is in progress, and we are planning to ship a Node 16 managed runtime and base container image within the next 90 days.
We also apologize for the confusion caused by locking this thread. We are addressing the gap in our process that caused it to happen. The feedback and engagement from this community is very valuable to us, and we will keep iterating on improving how we engage here.
If you have any suggestions for specific features or enhancements you would like to see included in our Node 16 support, please let us know on this thread.

this is the only reply that matters. day 24 of 90. 66 more days until the deadline.
may the 4th be with AWS devs.

Day 50 of 90. 40 to go. This will likely end to be more than a year from the release.

Day 70 of 90. 20 to go (?)

Thank you everyone for your feedback regarding AWS Lambda support for Node 16. Work is in progress, and we are planning to ship a Node 16 managed runtime and base container image within the next 90 days.
We also apologize for the confusion caused by locking this thread. We are addressing the gap in our process that caused it to happen. The feedback and engagement from this community is very valuable to us, and we will keep iterating on improving how we engage here.
If you have any suggestions for specific features or enhancements you would like to see included in our Node 16 support, please let us know on this thread.

I have a suggestion :

It can help a lot of people to plan a migration to the next NodeJS LTS in AWS Lambda if AWS could provide a kind of "implementation roadmap" to announce a date (or at least a month or a quarter) when the next LTS version will be available in AWS Lambda, exactly like NodeJS roadmap does for nodeJS releases, but no need to be so precise (a month or at least a quarter may be sufficient). So customers will be able to know that AWS Team is aware of the nodeJS LTS upgrade and is working on it.

This is the reply I got from AWS technical support:

The team has shared that, Node 16 runtime support in Lambda is being worked upon, and they have plans to release it in MAY-2022. You can keep an eye on the following links for more information and updates on the same :
...
Additionally, the team is aware of the feedback we have been receiving from customers, that they would like to see new runtimes released more promptly. Please be assured that the service team will take the feedback very seriously and work towards improving the Lambda service for better customer experience. Hence, the team has shared that they are reviewing the runtime release processes and resourcing, so we can better meet these expectations in the future.

This is the reply I got from the AWS technical support:

The team has shared that, Node 16 runtime support in Lambda is being worked upon, and they have plans to release it in MAY-2022. You can keep an eye on the following links for more information and updates on the same :
...
Additionally, the team is aware of the feedback we have been receiving from customers, that they would like to see new runtimes released more promptly. Please be assured that the service team will take the feedback very seriously and work towards improving the Lambda service for better customer experience. Hence, the team has shared that they are reviewing the runtime release processes and resourcing, so we can better meet these expectations in the future.

Vary good, now we have a more precise release period, now we now nodeJS v16 is planned to be release in AWS Lambda in May 2022. Thanks for sharing this information.

Log an issue for node v18, #47

🎉 🎉 🎉 🎉 https://nodejs.medium.com/node-js-16-available-now-7f5099a97e70
PLEASE don't wait a year to implement this in a new base image!

They waited a year, and more !

Is there a way to create a custom runtime to use the latest version of node? If so, can anyone point me in the right direction?

It seems so: https://github.com/fusebit/everynode

🎉 🎉 🎉 🎉 https://nodejs.medium.com/node-js-16-available-now-7f5099a97e70
PLEASE don't wait a year to implement this in a new base image!

lol they did!!

@klhex any idea how to use everynode in Cloudformation, specifically with a AWS::Serverless::Function resource?

@klhex I thought the same thing, and tried implementing it, but I get the following error when trying to run the Lambda

Unknown application error occurred
Runtime.ErrorLoadingHandler

No other information given to help me debug. I made sure my other Layer (node modules) had provided as a runtime compatability, and tried various things with the Handler setting, but nothing seems to work.

@klhex I thought the same thing, and tried implementing it, but I get the following error when trying to run the Lambda

Unknown application error occurred
Runtime.ErrorLoadingHandler

No other information given to help me debug. I made sure my other Layer (node modules) had provided as a runtime compatability, and tried various things with the Handler setting, but nothing seems to work.

I see, that's a pity. (I haven't tried it out at all yet, since we are only allowed to use AWS runtimes and therefore have to wait until AWS releases a new nodejs runtimes...)
But I saw that you raised a ticket for this issue. I hope that someone will be able to help you out.

@klhex sadly im not hopeful, as I dropped in to their Discord to ask and it's dead.
Fingers crossed for the offical AWS runtime.

@klhex any idea how to use everynode in Cloudformation, specifically with a AWS::Serverless::Function resource?

I imagine it wouldn't be much different from how you do it in terraform:
resource "aws_lambda_function" "test" {
  #...

  runtime     = "provided"
  layers      = ["arn:aws:lambda:eu-north-1:072686360478:layer:node-17_0_0:1"]

  environment {
    variables = {
      AWS_NODEJS_CONNECTION_REUSE_ENABLED = "1"
    }
  }
}

@BeeeQueue I tried adding it to my Cloudformation configs, and got nothing but unhelpful errors (see above)

Thank you everyone for the continued engagement and feedback on this issue. The AWS Lambda Node 16 runtime implementation is in progress, and we are currently on track to launch in mid-May. Our Node 16 runtime will continue to use AWS SDKv2, and we plan to transition to AWS SDKv3 with Node 18.

@jtuliani when are you looking to release Node 18 support? Node 16 is already yesterdays news.

@jtuliani thank you for the update!  Will you also be updating https://github.com/aws/aws-lambda-nodejs-runtime-interface-client/issues/44#issuecomment-1032928940 ?  I open-sourced prebuilt packages for amazonlinux:2022, but I need parity between your default aws-lambda-ric and the docker aws-lambda-ric, especially for ESM support.

The AWS Lambda Node 16 runtime implementation is in progress, and we are currently on track to launch in mid-May

@jtuliani Could you specify the year, please?

Thank you everyone for your feedback regarding AWS Lambda support for Node 16. Work is in progress, and we are planning to ship a Node 16 managed runtime and base container image within the next 90 days.
We also apologize for the confusion caused by locking this thread. We are addressing the gap in our process that caused it to happen. The feedback and engagement from this community is very valuable to us, and we will keep iterating on improving how we engage here.
If you have any suggestions for specific features or enhancements you would like to see included in our Node 16 support, please let us know on this thread.

this is the only reply that matters. day 24 of 90. 66 more days until the deadline.
may the 4th be with AWS devs.

aaaaaaaaaaaand deadline missed

Update: There are multiple parts to a runtime launch. There's the managed runtime itself, the container base image, plus support in the console, SAM CLI, AWS CLI, CloudFormation, various AWS SDKs, etc. In previous releases we have coordinated all of these to land on launch day. For the Node 16 release, we're experimenting with a different approach, with the various components landing separately. A release candidate of the 'nodejs16.x' runtime identifier is already deployed (you can try it today if you deploy via the Lambda APIs directly), with the other components in progress. Once everything is complete, we'll make the GA announcement.
@fromtheexchange That's a great question regarding the alignment of RICs. This is something we currently plan to address in Node 18.
@The-Code-Monkey Let's discuss Node 18 in #47, so we keep that information in one place.

@jtuliani thanks for the update, I will have a play with it

Confirmed that this works on Netlify functions. Set AWS_LAMBDA_JS_RUNTIME to nodejs16.x and it works: https://upbeat-snyder-05e8fe.netlify.app/api

It also works on Vercel with the new Build Output API

Thanks @forty for flagging that error message, I'll get that looked at.
Regarding the exact ETA: there are a lot of moving pieces to a runtime release (console, docs, SDK, SAM, CDK, CLI, CFN, etc). There are also multiple teams involved. Plans for any one of these teams can change at short notice, for example due to a live site issue, a deployment roll-back, or surprises such as Log4J2. So, I'm sure you'll understand why I'm holding off from providing an exact schedule. However, we are getting closer :)

Thanks for a swift response @jtuliani - that's totally understandable 👍 Fingers crossed for a smooth release process 🤞 Serverless Framework users are also very eager to try the new runtime - https://github.com/serverless/serverless/pull/11049 🙌

Node 16 support in Lambda is now GA: https://aws.amazon.com/about-aws/whats-new/2022/05/aws-lambda-adds-support-node-js-16/

@jtuliani wrote:

Node 16 support in Lambda is now GA: https://aws.amazon.com/about-aws/whats-new/2022/05/aws-lambda-adds-support-node-js-16/

In this announcement, the following sentence caught my attention:

Node.js 16 is the latest long-term support (LTS) release of Node.js and will be supported for security and bug fixes until April 2024.

Given that Node 16 reaches EOL in September, 2023, how is AWS planning to make security and bug fixes after that date?

What are the time scales for this ?
On Mon, 25 Apr 2022 at 3:52 pm, Jonathan Tuliani @.***>
wrote:

Thank you everyone for the continued engagement and feedback on this
issue. The AWS Lambda Node 16 runtime implementation is in progress, and we
are currently on track to launch in mid-May. Our Node 16 runtime will
continue to use AWS SDKv2, and we plan to transition to AWS SDKv3 with Node
18.
—
Reply to this email directly, view it on GitHub
https://github.com/aws/aws-lambda-base-images/issues/14#issuecomment-1108679026,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACJFNXWD3LRRRVHC5SDEM3VG2WSJANCNFSM43IVOWLQ
.
You are receiving this because you commented.Message ID:
@.***>

See #47 for Node 18 support in Lambda


Consuming Qualys scan results

Can you please extend prenus to consume Qualys scan exports.

Currently in beta testing


feat(node): bump from node 12 to node 16 default

node 12 is out of support I think (?), and node v16 is known to work with react-native, I hope it's mostly backwards compatible and this is okay?
Either way, it was needed as an override for the semantic-release package which just dropped support for 12 and works with 16 now - https://github.com/react-native-share/react-native-share/pull/1218
Not sure if this is a breaking change or not, as I'm unsure of backwards compatibility status

I hope it's mostly backwards compatible and this is okay?

Ideally moving to 14 would be preferrable, but as this is the orb code, 16 should also work. Potentially using lts here is more maintainable in the long run, but also exposes to version bumps out of control, so the version you picked is probably the correct one 👍

With 18 already out 14 seemed too low (as long as it is moving...) but I did want a fixed version for control yes. So I settled on 16 a but uneasily as well yes

@cortinico I found references to 12 in 5 other places after seeing it still failing over on https://github.com/react-native-share/react-native-share/pull/1218
I added commits for the same, should squash-merged if acceptable

I added commits for the same, should squash-merged if acceptable

Code looks fine to me. The CI is failing though, I believe due to an expired CircleCI token.


homebrew formula breaks homebrew cleanup

Describe the bug
Having lazydocker installed breaks Homebrew's cleanup process.
To Reproduce
Steps to reproduce the behavior:

Install lazydocker
Run brew doctor

Expected behaviour
I can run brew upgrade after using Homebrew to install lazydocker.
Screenshots
n/a
Desktop (please complete the following information):

OS:
macOS
Lazydocker Version
0.12
The last commit id if you built project from sources (run : git rev-parse HEAD)
I installed with Homebrew

Additional context
More details at https://github.com/Homebrew/discussions/discussions/1413#discussioncomment-699179

with the latest release, brew doctor works fine for me. I'll close this off but let me know if you still have the issue and I'll reopen


Real device model

Where does the real device schema lives, is it part of Device simulation project only?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2aca7d13-4905-1ec1-7f93-3152897aa3ef
Version Independent ID: 3de08582-739f-5cf7-e791-d47b9b638c35
Content: Device schema in remote monitoring solution - Azure | Microsoft Docs
Content Source: articles/iot-suite/iot-suite-remote-monitoring-device-schema.md
Service: iot-suite
GitHub Login: @dominicbetts
Microsoft Alias: dobett

@pankajisgarg
Thanks for the feedback! We are currently investigating and will update you shortly.

Hi @pankajisgarg,
The remote monitoring preconfigured solution supports two types of device: Physical and Simulated. A Physical device is a real device sending real telemetry that connects to the solution, a Simulated device only exists in the device simulation microservice and generates simulated telemetry to help you understand how the solution works.  I'm assuming that by "real" device you mean Physical.
A Physical device sends its schema information to the solution when it first connects - if you look at the sample code in Connect your device to the remote monitoring preconfigured solution (C) or Connect your device to the remote monitoring preconfigured solution (Node.js) you can see that they send a set of reported properties at start-up that specify information such as the message schema and supported methods. The solution dynamically configures itself based on this information.
Dominic

#please-close


Outdated Vite example

I am trying to setup vite @itk-wasm/mesh-io to load and render vtk meshes in vtk.js.  I was following this guide: https://wasm.itk.org/en/latest/typescript/distribution/vite.html. However the example seems a bit outdated. Could you provide an updated vite example?
The specific issue I am seeing in vite:

https://github.com/InsightSoftwareConsortium/ITK-Wasm/blob/2bbb8bbc94aa8deebeeb2c4d8988aaa939f945aa/examples/vite/vite.config.js#L10-L24

Hi @aghouas ,
We use vite in the tests for this package, which can be used as a reference:
https://github.com/InsightSoftwareConsortium/ITK-Wasm/blob/main/packages/mesh-io/typescript/vite.config.js

Thanks @thewtex !
I managed to get @itk-wasm/mesh-io to load and run properly.
Now I am faced with another issues.
At a high-level this is how I am trying to load and render a mesh stored in a .vtk file:

Load .vtk file from a remote source using fetch. ✅
Read file as an ITK Mesh using the readMesh function provided by@itk-wasm/mesh-io ✅
Convert Mesh to an itk PolyData object using the meshToPolyData function from @itk-wasm/mesh-to-poly-data. ❌
Convert itk PolyData to vtk Poly data using convertItkToVtkPolyData from @kitware/vtk.js/Common/DataModel/ITKHelper
Render mesh using vtk.js

When I check the console, this is all I see:

I know the error occurs in meshToPolyData, but I don know why.
Have you seen this error before or do you have any tips on how to debug?
For reference, here is the react code:
import { useRef, useEffect } from "react";

import<EMAIL_ADDRESS>
import vtkFullScreenRenderWindow from<EMAIL_ADDRESS>
import vtkActor from<EMAIL_ADDRESS>import vtkMapper from<EMAIL_ADDRESS>import type vtkRenderWindow from<EMAIL_ADDRESS>import type vtkRenderer from<EMAIL_ADDRESS>import vtkITKHelper from<EMAIL_ADDRESS>import { readMesh } from "@itk-wasm/mesh-io";
import { meshToPolyData } from "@itk-wasm/mesh-to-poly-data";

interface VtkJsContext {
  fullScreenRenderer: vtkFullScreenRenderWindow;
  renderWindow: vtkRenderWindow;
  renderer: vtkRenderer;
  actor?: vtkActor;
  mapper?: vtkMapper;
}
const fileName = "brain.vtk";
export function VtkViewer() {
  const vtkContainerRef = useRef<HTMLDivElement | null>(null);
  const context = useRef<VtkJsContext | null>(null);

  useEffect(() => {
    if (!context.current) {
      const fullScreenRenderer = vtkFullScreenRenderWindow.newInstance({
        container: vtkContainerRef.current ?? undefined,
      });

      const renderer = fullScreenRenderer.getRenderer();
      const renderWindow = fullScreenRenderer.getRenderWindow();

      renderer.resetCamera();

      context.current = {
        fullScreenRenderer,
        renderWindow,
        renderer,
      };
    }

    return () => {
      if (context.current) {
        const { fullScreenRenderer, actor, mapper } = context.current;
        actor?.delete();
        mapper?.delete();
        // coneSource.delete();
        fullScreenRenderer.delete();
        context.current = null;
      }
    };
  }, [vtkContainerRef]);

  useEffect(() => {
    async function renderMesh(context: VtkJsContext) {
      const { renderer, renderWindow } = context;
      const vtkFile = await fetch(fileName)
        .then((resp) => {
          return resp.blob();
        })
        .then((blob) => {
          return new File([blob], fileName);
        });

      console.log("file", vtkFile);
      const { mesh, webWorker } = await readMesh(vtkFile);

      console.log("Mesh", mesh);
      const { polyData: itkPolyData } = await meshToPolyData(mesh, {
        webWorker: webWorker,
      });

      webWorker.terminate();

      console.log("Poly Data", itkPolyData);
      const polyData = vtkITKHelper.convertItkToVtkPolyData(itkPolyData);

      const mapper = vtkMapper.newInstance();
      mapper.setInputData(polyData);
      const actor = vtkActor.newInstance();
      actor.setMapper(mapper);

      renderer.addActor(actor);

      renderer.resetCamera();
      renderWindow.render();
    }

    if (context.current) {
      renderMesh(context.current);
    }
  }, [context]);

  return (

  );
}

export default VtkViewer;

Hello @aghouas ,
Great work! :sparkles:
To investigate the source of the error, one option is to create a native Debug build per:
https://github.com/InsightSoftwareConsortium/ITK-Wasm/pull/1201
Then build a native debug binary from:
https://github.com/InsightSoftwareConsortium/ITKMeshToPolyData/tree/master/wasm
and investigate in gdb or another debugger.

Hi, I've also encountered the same issue.
In my case,

stl files would work
obj files will throw below error:

vtk/vtp files will throw below error:

All those files are converted with Paraview.

Hi @sigmama
A patch to address the third is here: https://github.com/InsightSoftwareConsortium/ITKMeshToPolyData/pull/71

Hi @thewtex, actually I've a little bit confused by the examples typically for vite. With the latest change, how would I config the itkConfig.js or use other approaches to make my app to get the worker files locally instead of CDN? I've checked both the document site and the vite example in the code base and seems that both are outdated.


PayPal - Plugin prevents styles of additional plug-ins

PHP Version
8.3
Shopware Version
<IP_ADDRESS>
Affected area / extension
Platform(Default)
Expected behaviour
Plugin styles should be loaded.
Actual behaviour
If a theme is assigned to the sales channel and the PayPal plugin is activated, all styles of additional plugins are ignored.
How to reproduce

Install Theme + PayPal Plugin for example:
https://store.shopware.com/de/rhweb75271565266m/furniture-one-pro-responsive-premium-theme.html
https://store.shopware.com/de/swag588949310744f/paypal-fuer-shopware-6.html

Install any plugin from shopware store with custom styling for example:
https://store.shopware.com/de/zenit41392822759m/banner-fuer-hinweise-nachrichten-angebote-sw6.html

Styles from additional plugin is ignored.

If the PayPal - Plugin is uninstalled everything works fine.

[public] Shopware automated response: This issue is linked to the internal issue: NEXT-38085
[created from NEXT-38085, comment 491078]

Hey there, I have installed the mentioned theme and the two plugins, all work well together. I don't see any issues at first hand. Can you tell me, where exactly the incompatibilities are? If you are talking about "styles", I assume SCSS, but the banner from the last plugin is showing perfectly fine and seems well styled. Or are you talking about templates?
As you can see from my screenshot, PayPal, Theme + Banner working together 🤷♂

It’s unfortunate that you’re unable to reproduce the error. However, several customers have reported the same issue after updating to the latest version of Shopware (see the ticket), and this problem has also been escalated to Shopware.
The issue persists on my system (screenshot attached). Neither running storefront:build nor theme:compile resolves the problem. Any additional Plugin can not compile the styles or styles will not even be loaded. When PayPal is deactivated, everything works fine.
Unfortunately, I’m unable to explain why the error isn’t occurring on your end. I experienced the error on several customer stores as well. I would greatly appreciate it if you could take a closer look at this matter, because this error causes many shop owners trouble and provides a lot of support for us all.

Interesting, thanks for doing so. Would you mind sharing what file it is? Or is it very specific to your theme?

It was some kind of more complicated SCSS in the theme which caused a problem when merging the styles of the PayPal - Plugin. But it's solved now! Theme update is available now in the Shopware Store.  :)


Breaks long headings

For headings that exceed the line length limit, the tool currently splits them into multiple lines.
But this is incorrect behaviour, since now the heading is truncated during rendering (it does not continue after a newline)
before formatting:
source:

rendered:

after formatting:
source:

rendered:

Thanks for reporting!

This is fixed on master!


Unable to do npm workflow

Matts-iMac:ramme matt$ npm install
npm WARN package.json<EMAIL_ADDRESS>os is also the name of a node core module.
npm WARN unmet dependency /Users/matt/Projects/ramme/node_modules/electron-packager/node_modules/asar/node_modules/mksnapshot/node_modules/fs-extra/node_modules/rimraf requires<EMAIL_ADDRESS>but will load
npm WARN unmet dependency undefined,
npm WARN unmet dependency which is version undefined

Any ideas?

Have you tried to to do a npm cache clean and delete node_modules folder and rerun npm install?
Which version of node are you using?

<EMAIL_ADDRESS>sudo nam cache clean did it
Thanks!


Allow installation on Laravel 9

It would be nice to have this, @cycoslave / @wickedsoft :)

Would love to be able to use this project in Laravel 9, @cycoslave / @wickedsoft. Any chance that you can merge it?

I will take a look this weekend and have it merged for laravel 9 :)

Hi @cycoslave! Any news on getting the commit merged? Would really appreciate it. 👍🏻


Page Header Title Icon is missing css className 'c4p--page-header__title-icon' when icon is used as a function

What package(s) are you using?

[X] Carbon for IBM Products (or Carbon for IBM Cloud & Cognitive)
[ ] Legacy/CDAI
[ ] Legacy/Security

Detailed description
When using the Page Header components from @carbon/ibm-products
https://v11-carbon-for-ibm-products.netlify.app/?path=/story/ibm-products-components-page-header-pageheader--fully-loaded-and-some
For me, the following code will add this class 'c4p--page-header__title-icon' into title SVG
title: {    icon: User,    text: 'Title Text', },
However, the following code does not add the expected class
title: {     icon: (): ReactElement => <User size={24}/>,     text: 'Title Text', },

What version of the @carbon/ibm-products
2.0 rc

Steps to reproduce the issue

Create a page header component
Supply title as an object where icon is a function rather then object
Notice the padding are off, investigate to see 'c4p--page-header__title-icon' is missing from title svg

Additional information

@akhandsi For this, I believe the syntax would be:
icon: (props) => <User size={24} {...props} />


Feature/support field to field filters

This supports field-to-field filters and fixes a bug with redshift symmetric aggregates


The first two lab of getting started is same

https://cilium.io/labs/categories/getting-started/
and the content of the lab seems different from the description

Thanks for report, fixed with #570


[DOC-3514] Add warning about reusing an incompatible store directory

[DOC-3514] Add warning about reusing an incompatible store directory

[DOC-3514] Add warning about reusing an incompatible store directory

There are existing checks in the code to bail out during an attempt to open a store with an incompatible version. I am curious what led to add an explicit warning. Is there a case where an attempt went undetected?

[DOC-3514] Add warning about reusing an incompatible store directory

There are existing checks in the code to bail out during an attempt to open a store with an incompatible version. I am curious what led to add an explicit warning. Is there a case where an attempt went undetected?

Well, what did you have in mind when you filed the issue? :)


feat: add kubectx command

Adds ability to change context from within neovim with a kubectx command, maybe we should have a way to do it in the ui as well, not sure about that.


Uniform intrinsic filename

Hi,
I notice that intrinsic filename is in a chaos status.
V extension uses riscv_vector.h and b extension uses rvintrin.h.
Maybe we can separate each intrinsic into an independent file, and users can include what they really need instead of all intrinsics.
(e.g. to use v, b and p extension, they should include riscv_v_intrinsic.h, riscv_b_intrinsic.h and riscv_p_intrinsic.h respectively)
If users want to use all intrinsics, they can include riscv_intrinsic.h.
Any idea?

The B extension rvintrin.h file can't be upstreamed to FSF GCC due to lack of copyright assignments.  It should be ignored.
The B extension plan was to have a single header file for all of the intrinsics, but given the size of the V intrinsic header file, that seems unwise.  We should have separate files for each extension, and maybe a file that combines them all.  Your suggestion sounds reasonable to me.

Good idea.
IMO, V intrinsic header is too big, so I prefer to have separate header for each extension.