midi-trigger: init at 0.0.4

Description of changes
MIDI-Trigger: LV2 plugin which generates MIDI notes by detected audio signal peaks.
Tested on NixOS using Ardour 6.
Also it’s tested that it successfully builds on MacOS in the CI:
https://github.com/unclechu/MIDI-Trigger/actions/runs/2693437043
Applied nixfmt to pkgs/applications/audio/midi-trigger/default.nix.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Testing report (not sure if it’s relevant since there are no packages depending on it):
$ nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"
this path will be fetched (0.03 MiB download, 0.13 MiB unpacked):
  /nix/store/0g1hfy64vsqk1k7v5avjnxi8cql8rdwm-nixpkgs-review-2.6.4
copying path '/nix/store/0g1hfy64vsqk1k7v5avjnxi8cql8rdwm-nixpkgs-review-2.6.4' from 'https://cache.nixos.org'...
$ git -c fetch.prune=false fetch --no-tags --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 7 (delta 0), reused 0 (delta 0), pack-reused 2
Unpacking objects: 100% (7/7), 145.78 KiB | 1.03 MiB/s, done.
From https://github.com/NixOS/nixpkgs
   5e39406e007..f2c952b6883  master     -> refs/nixpkgs-review/0
$ git worktree add /home/wenzel/.cache/nixpkgs-review/rev-cc3f24b4545892bd67d4ccb48d7f33c54936a297/nixpkgs f2c952b68837c2437b88039ff01149bb0954e6e0
Preparing worktree (detached HEAD f2c952b6883)
Updating files: 100% (31112/31112), done.
HEAD is now at f2c952b6883 pythonModules.polars: mark as broken
$ nix-env --option system x86_64-linux -f /home/wenzel/.cache/nixpkgs-review/rev-cc3f24b4545892bd67d4ccb48d7f33c54936a297/nixpkgs -qaP --xml --out-path --show-trace
$ git merge --no-commit --no-ff cc3f24b4545892bd67d4ccb48d7f33c54936a297
Automatic merge went well; stopped before committing as requested
$ nix-env --option system x86_64-linux -f /home/wenzel/.cache/nixpkgs-review/rev-cc3f24b4545892bd67d4ccb48d7f33c54936a297/nixpkgs -qaP --xml --out-path --show-trace --meta
1 package added:
midi-trigger (init at 0.0.4)

$ nix --experimental-features nix-command build --no-link --keep-going --option build-use-sandbox relaxed -f /home/wenzel/.cache/nixpkgs-review/rev-cc3f24b4545892bd67d4ccb48d7f33c54936a297/build.nix
1 package built:
midi-trigger

$ nix-shell /home/wenzel/.cache/nixpkgs-review/rev-cc3f24b4545892bd67d4ccb48d7f33c54936a297/shell.nix
these 4 paths will be fetched (0.55 MiB download, 2.67 MiB unpacked):
  /nix/store/01bkry44wndk7m5d1bi3gyfmc3izs9lz-bash-interactive-5.1-p16-dev
  /nix/store/2aj5gchjipzd8y2bnl91038ypf599gxg-bash-interactive-5.1-p16-doc
  /nix/store/9g6xbz9la8mnhiy8h0hbnizsp1qihy1p-bash-interactive-5.1-p16-man
  /nix/store/hbfmxc331mma5p6ss5rfxn9fyqx77fn0-bash-interactive-5.1-p16-info
copying path '/nix/store/hbfmxc331mma5p6ss5rfxn9fyqx77fn0-bash-interactive-5.1-p16-info' from 'https://cache.nixos.org'...
copying path '/nix/store/2aj5gchjipzd8y2bnl91038ypf599gxg-bash-interactive-5.1-p16-doc' from 'https://cache.nixos.org'...
copying path '/nix/store/01bkry44wndk7m5d1bi3gyfmc3izs9lz-bash-interactive-5.1-p16-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/9g6xbz9la8mnhiy8h0hbnizsp1qihy1p-bash-interactive-5.1-p16-man' from 'https://cache.nixos.org'...

How big is this queue? :thinking:

Also, seeing as this is your code, maybe look into the warnings produced at build time? I see some pretty suspicious FORTIFY_SOURCE messages there.

@K900 I’m not sure when I’ll come back to look deeper for it. I didn’t touch the project for years, just added Nix config basically and made sure it works.


minify generated javascript file with grunt uglify

Description
Added grunt clean and uglify tasks to minift generated outputs
Purpose
To reduce generated file sizes

I signed it!


Package Installation Support For Python Play Ground

Assign this to me, I will resolve the issue

Yes @Rock0007 you can start working on it
Thank You
Instructions

Yes you can start working on it by taking latest pull from main branch
Don't Forget to join onboarding group for knowledge flow and direct conversation and support from other developers link  Curious Developers Community - Hacktoberfest
Please attach the screenshots and video of the development work once it is done
Contributions more than 3 will be awarded a certificate of appreciation from curious ecosystem please mail to<EMAIL_ADDRESS>for claiming certificate and please attach your issue number
Respect Co Developers And Please Reachout to admins if any issues


updates Marketo action to use streaming

adds a MarketoTransaction class to handle the stateful data needed for the duration of the request.
also adds a Queue class to manage sending 10 requests at a time.

@wilg I'm not able to make you a reviewer, but I think this is in your court at this point.  Keep in mind that 1) we've not tested outside of Heroku, so our handling of a queue that takes longer than 30 seconds to resolve is theoretical (although we believe it works).  2) We've tried to handle both the specific errors we know about (e.g. record level) and larger errors we have not yet experienced (e.g. API connection failures).  But again, some of this code is as of yet unexercised.  Let us know if it would be useful to have @pascal4mile walk you through any of this.  Cheers!

(apologies for the terse commit messages; was having to commit and push a lot in order to test changes on heroku, so i got lazy : )

@pascal4mile Looking forward to reviewing this. Can you please fix the tests and update the pull request title and description with an overview of your changes? Thanks!

ok @wilg I think this is ready for your eyes. be gentle! no i can take it.

Poke @wilg (in case you missed this last week). We have a couple customers waiting on it.

@wilg or @phillipperalez can one of you take a look at this? thanks!

/cc @ankit-bhatia-looker

@wilg @phillipperalez i've updated the action to use winston for logging and removed the commented logging code.

Awesome! @phillipperalez can you merge?


Fixed bug preventing overriding inherited parameter with None

See Issue #97.
Test case:
import param

class A(param.Parameterized):

    a = param.Number(default=10)

    def __init__(self, **params):
        super(A, self).__init__(**params)
        print self.a

class B(A):

    a = param.Number(default=None)

    def __init__(self, **params):
        super(B, self).__init__(**params)
        print self.a

a=A()
b=B()

Output:
10
10
10

Fixed output:
10
None
None

Coverage remained the same at 68.8% when pulling 353358c8d65256edb010aee3cc82985ccdc22c93 on philippjfr:master into 3210731c4761bcbae97fdea9aa1b27d8bf90a624 on ioam:master.

Coverage remained the same at 68.8% when pulling 353358c8d65256edb010aee3cc82985ccdc22c93 on philippjfr:master into 3210731c4761bcbae97fdea9aa1b27d8bf90a624 on ioam:master.

This is the wrong fix so don't merge it. Will either find a way around this or implement it properly.

Coverage increased (+0.04%) to 69.77% when pulling 6f04d3f022cdd358f517aa1ac661d27344a0bab7 on philippjfr:master into 74e06f20aa7caf22e33bda50ffe8a9f6cd3062b5 on ioam:master.

Coverage increased (+0.04%) to 69.77% when pulling 6f04d3f022cdd358f517aa1ac661d27344a0bab7 on philippjfr:master into 74e06f20aa7caf22e33bda50ffe8a9f6cd3062b5 on ioam:master.

Coverage increased (+0.04%) to 69.77% when pulling 6f04d3f022cdd358f517aa1ac661d27344a0bab7 on philippjfr:master into 74e06f20aa7caf22e33bda50ffe8a9f6cd3062b5 on ioam:master.

Coverage increased (+0.04%) to 69.77% when pulling 6f04d3f022cdd358f517aa1ac661d27344a0bab7 on philippjfr:master into 74e06f20aa7caf22e33bda50ffe8a9f6cd3062b5 on ioam:master.

Coverage increased (+0.04%) to 69.77% when pulling 6f04d3f022cdd358f517aa1ac661d27344a0bab7 on philippjfr:master into 74e06f20aa7caf22e33bda50ffe8a9f6cd3062b5 on ioam:master.

Coverage increased (+0.04%) to 69.77% when pulling 6f04d3f022cdd358f517aa1ac661d27344a0bab7 on philippjfr:master into 74e06f20aa7caf22e33bda50ffe8a9f6cd3062b5 on ioam:master.

It doesn't, my fork of param was still linked against this PR so when I pushed a change it automatically updated it. I've since closed it.


Padding with insets

This adds a new constructor to the Padding widget to take advantage fo the new Insets type in kurbo.
Question: should this just be called Padding::new? Should we get rid of Padding::uniform, which can be well handled by a single impl Into<Insets> argument?
edit: this is based off of #236, which should go in first

This isn't a new problem of this PR, but it's pretty easy for padding to pass along negative BoxConstraints, especially with the subtraction happening in here. Not sure if this is where we want to take a stand, just wanted to give a heads up.

Agreed. This can happen in Flex when the non-flex widgets exceed the constraint as well.


Wrong at and it for html tag with "-"

Contributing guidelines

[X] I have read CONTRIBUTING.md
[X] I have read CODE_OF_CONDUCT.md
[X] I have updated 'mini.nvim' to latest version

Module(s)
ai, surround
Description
For vue, svelte or other template language, textobj "at" or "it"  only work for common tag without "-". For example div

But for "el-dropdown-menu", it dones select  tag "el-dropdown-menu", but select the outer tag "template", because "template" has no "-"

Neovim version
0.9.0
Steps to reproduce
use "vat" or "vit" on html tag with "-"
Expected behavior

Actual behavior

I have not install nvim-treesitter.
The nvim default textobj "at" or "it"  was correct. but "mini.ai" or "mini.surrount" was wrong

Thanks for the issue!
Yes, tag implementation in 'mini.ai' relies on an appropriate Lua patterns to match a tag region. I would strongly suggest using tree-sitter to get more proper information about tag structure. Something like how it this mentioned in this comment.
However, you can still customize tag textobject to include punctuation in tag name by slightly modifying built-in one like this:
require('mini.ai').setup({
  custom_textobjects = {
    t = { '<([%p%w]-)%f[^<%w][^<>]->.-</%1>', '^<.->().*()</[^/]->$' }
  }
})

Thanks.
The treesitter has so many bugs for template language filetypes. so I cannot use it.
For  custom_textobject of example, it was incorrrect.
For example

I want to select vat for template.
The correct is

Those two options (using tree-sitter and customizing built-in pattern) are the only ones available right now, sorry.


fix: progress indicator not becoming visible in fullscreen mode

:tada: This PR is included in version 4.0.0-alpha.23 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0-alpha.80 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0-alpha.15 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.0.0-alpha.6 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 4.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Connection Problems with Keep-Alive

Hey there,
it seems to me that the HTTP keep-alive feature in aleph servers might be broken. I created a minimal project to illustrate the problem:
https://github.com/xsc/aleph-stream-testcase
I tested against 0.4.1 and 0.4.2-alpha4, by setting up a server that returns either an InputStream or a byte array as :body, then used Apache Bench with the -k option to reuse connections during the test run, e.g.:
$ ab -n 500 -c 4 -k http://localhost:9877/bytes
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)

Test aborted after 10 failures

Total of 12 requests completed

Note that a run without -k works perfectly fine:
$ ab -v1 -n 500 -c 4 http://localhost:9877/bytes
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests

Server Software:        Aleph/0.4.1
Server Hostname:        localhost
Server Port:            9877
...

For the InputStream version there are some Exceptions (e.g. error in manifold.utils/future-with java.io.IOException: Stream closed and error in message propagation java.io.IOException: Broken pipe). I can provide complete stacktraces but they should be produced by the test project I provided.
If you need any more information, let me know!
Yannick

$ lein version
Leiningen 2.6.1 on Java 1.8.0_45 Java HotSpot(TM) 64-Bit Server VM

I'm able to reproduce your failure, but this, for instance, works fine:
 httperf --num-conns=16 --rate=16 --num-calls=10000 --port=8=9877 --uri=/stream

Also, Aleph has successfully run similar tests on the TechEmpower benchmarks without issue.  I noticed that the throughput of the tests improved significantly when I added this to the project.clj:
:jvm-opts ["-server" "-Xmx1g"]

I also noticed that when I returned a shorter byte array via /bytes, ApacheBench works fine.  I'm tempted at this point to blame ApacheBench for handling large responses via keep-alive improperly.  Thoughts?

We experienced problems (i.e. connections being closed before the data was fully transmitted) with aleph behind proxies like CloudFront or HAProxy which prompted me to investigate the matter, especially the keep-alive behaviour between the two.
But, seeing as ab is seemingly drunk, those might just have been instances of #191, so I'll close the issue for now and see what I can gather over the course of next week.
Thanks for the quick response!


Notifications

At that point I feel it's time to get some feedback.
Here it is a slightly improved notification system.
It's still missing quite a lot of stuff, but it works (I hope :stuck_out_tongue: ).
Important bit:
https://github.com/elkarte/Elkarte/blob/b3422b62723d064bc4d4662e53dc516ee7db68f9/sources/subs/Notifications.class.php
the main Notifications class, the one that should coordinate the work.
Entry point is the add method, that accepts as parameter a Notifications_Task object, nothing more than a "bag" with some custom methods to retrieve some stuff we may need later.
Central point is the send method. Here each task (i.e. mention or new reply notification, etc.) is processed independently:

1st the mentionType object (those in sources/subs/MentionType) is instantiated (line 57)
2nd the members to be notifies are determined (I'm a bit unsure yet how that happens, but it's a WIP, for the moment it's at line 62)
3rd it works out what kind of notification/s each member wants (none, simple notification (the mentions area), notification + email, etc., line 63)
4th for each frequency, it 1) gets the "message" (body and subject) from the mentionType object (line 69), 2) sends/inserts each of them (line 77)

To have an idea of how MentionType classes look like, I think that the buddies one is the best:
https://github.com/elkarte/Elkarte/blob/b3422b62723d064bc4d4662e53dc516ee7db68f9/sources/subs/MentionType/BuddyMention.php
just because it's the easiest, all the others had to hook into some action to do their work mixing the actual notification part with some other work.
Special mention is for the Mentioning class (and the name of the file is wrong...) that takes care of doing some validation and actions on the mentions themselves (i.e. log_mentions that means the mentions are in the menu).
It's not yet ready, it still misses all the user interface to configure the settings, but at least it "works" (mostly I guess LOL).

That's funny, I'll restart it tomorrow and see.

Tried a restart ... seems its the web tests that are failing http://i.imgur.com/WYI6bGM.png and PHPUnit_Framework_Exception: Fatal error: Call to undefined method Errors::display_db_error() in /var/www/sources/database/Db-mysql.class.php on line 78  hummm

hmmm...
it may be related to https://twitter.com/traviscistatus/status/592902357144498176
Let's see after the rollback.
Otherwise I don't understand why only one out of 6...

Could be .. but only 5.4 runs the selenium webtests and the code coverage reports, thats why it takes so much longer than the other test beds.

It likely is.
For curiosity I tested an old already merged PR and it breaks as well:
https://travis-ci.org/elkarte/Elkarte/jobs/59643301
I guess we have to wait a bit more to have back the green light.

Guess we can wait till they rollback the updates this PM and see what that brings.

I figured the update PR may break this ones ability to merge ... should be a simple conflict I hope
After its fixed I think its time to merge this one in as well.

Rebase'ed and pushed. :hamster:

Should this be labeled as a block, since it's pretty much all I'm still waiting on in #1968 (the non-standard DSNs can be handled when the rest of the function is in place) which is labelled as a block.

Well, yes, we can label this as a blocker, but it's already merged in the development branch, so already in 1.1.

Oh, well if it's done, then I guess I have some work to do. Wish I'd caught it sooner. :)

I'm still having trouble getting notifications going for testing, even without making any changes. I've got them turned on in the forum features, and I've got the preferences for each type turned on in the database ({$user, 1, $type} in elkarte_notifications_pref) but not mentions, buddy, nor quotes are sending any notifications. I'm guessing I missed something.

I'm going to guess, and this really is nothing more,     that the event "triggers" were not registered in the db.  I think they are in the install or the update file, can't remember.
But I seem to remember some values need to be set during install and they are not set / reset / fixed when you hit save from the ACP (at the current time).

I'll check that out. Thanks.

Did you already try to turn them off and on again?
Do notifications popup in the mentions area or not even there?

No, I saw no notifications anywhere

I have the terrible feeling I forgot to add a require_once... :astonished:
In Notifications.class.php, add a:
		require_once(SUBSDIR . '/Mail.subs.php');

just before:
		$notification_frequencies = filterNotificationMethods($this->_notification_frequencies, $class::getType());

Let me know if that makes any difference (in my case it did).

Doesn't seem to be helping me, but it's entirely possible I didn't so something.
I've got "Enable site Notifications" turned on (and off, and on) and "Enable User Notification of This Event" for all event types.  Then I make a post with one test user, log in with another test user, and quote it, @ it, and like it.  The first user's notifications stay empty. I'm using two separate browsers to make sure that there's no session-clobbering or anything like that.

Is it a fresh install or did you overwrite the files of an existing dev branch? (Or other alternatives?)
No errors in the log?

Brand new install on a brand new VM. The only things in the error log are the aforementioned "Unidentified Index" notices from getTimeLastMention()

Ouch, I didn't notice the updates.
That undefined are likely because nothing is inserted (and it's a bug because it should not return an undefined index anyway => tracked #2422), now I have to try a new install and see what's wrong.
This evening (in ~7 hours).

Sorry, I've lost a lot of time on another thing, will take care of this in the morning... :sob:

No worries. I'm not in any terrible rush. :)
On March 8, 2016 6:48:57 PM EST, "emanuele45 -<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:

Sorry, I've lost a lot of time on another thing, will take care of this
in the morning... :sob:

Reply to this email directly or view it on GitHub:
https://github.com/elkarte/Elkarte/pull/2049#issuecomment-194028190

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Okay, I was able to debug it.
Could you check Subs.php, in the obExit function, there should be this piece:

	// Clear out the stat cache.
	trackStats();

	// If we have mail to send, send it.
	if (!empty($context['flush_mail']))
		// @todo this relies on 'flush_mail' being only set in AddMailQueue itself... :\
		AddMailQueue(true);

	Notifications::getInstance()->send();

	$do_header = $header === null ? !$header_done : $header;

move the Notifications::getInstance()->send(); up before the AddMailQueue if block like this:

	// Clear out the stat cache.
	trackStats();

	Notifications::getInstance()->send();

	// If we have mail to send, send it.
	if (!empty($context['flush_mail']))
		// @todo this relies on 'flush_mail' being only set in AddMailQueue itself... :\
		AddMailQueue(true);

	$do_header = $header === null ? !$header_done : $header;

No dice, but I'm wondering if we might be at cross purposes here. I can't
get notifications/mentions to work at all - not just for the email disable
notice, but not for quotes, @mentions, nothing. I've been trying to do that
since I can't be sure if the disable notice is broken until I can make the
rest of the notifications work like they're supposed to. It may well be
that I missed some step somewhere, or some combination of settings isn't
just quite right.
On Wed, Mar 9, 2016 at 3:02 PM, emanuele45 -<EMAIL_ADDRESS><
<EMAIL_ADDRESS>wrote:

Okay, I was able to debug it.
Could you check Subs.php, in the obExit function, there should be this
piece:
// Clear out the stat cache.
trackStats();

// If we have mail to send, send it.
if (!empty($context['flush_mail']))
    // @todo this relies on 'flush_mail' being only set in AddMailQueue itself... :\
    AddMailQueue(true);

Notifications::getInstance()->send();

$do_header = $header === null ? !$header_done : $header;

move the Notifications::getInstance()->send(); up before the AddMailQueue
if block like this:
// Clear out the stat cache.
trackStats();

Notifications::getInstance()->send();

// If we have mail to send, send it.
if (!empty($context['flush_mail']))
    // @todo this relies on 'flush_mail' being only set in AddMailQueue itself... :\
    AddMailQueue(true);

$do_header = $header === null ? !$header_done : $header;

—
Reply to this email directly or view it on GitHub
https://github.com/elkarte/Elkarte/pull/2049#issuecomment-194480672.

--
"The nitrogen in our DNA, the calcium in our teeth, the iron in our blood,
the carbon in our apple pies were made in the interiors of collapsing
stars. We are made of starstuff." - Carl Sagan

True enough, the @ and quote mentions are not "really" enabled at install (I noticed just now and applied the fix, will push later on), though, following this procedure:

go to admin > config > features and options > notifications
remove the tick from the "Enable user notifications of this event." corresponding to @ and quote mentions
save
add again the tick
save

the mentions start to work as expected for me, and with the change suggested above the emails are sent as well (well, I checked they land in the mail_queue table, but considering all the other emails should be delivered those should go as well... unless there is a bug in the sending of email).
Tomorrow I'll set up an open test site, so we can try it out live.

Awesome, that made quoting work, so now I've got a baseline. I've got
server updates and maintenance tying me up most of the next few days, but
I'll see what I can squeeze in.
On Wed, Mar 9, 2016 at 4:36 PM, emanuele45 -<EMAIL_ADDRESS><
<EMAIL_ADDRESS>wrote:

True enough, the @ and quote mentions are not "really" enabled at install
(I noticed just now and applied the fix, will push later on), though,
following this procedure:

go to admin > config > features and options > notifications
remove the tick from the "Enable user notifications of this event."
corresponding to @ and quote mentions
save
add again the tick
save

the mentions start to work as expected for me, and with the change
suggested above the emails are sent as well (well, I checked they land in
the mail_queue table, but considering all the other emails should be
delivered those should go as well... unless there is a bug in the sending
of email).
Tomorrow I'll set up an open test site, so we can try it out live.
—
Reply to this email directly or view it on GitHub
https://github.com/elkarte/Elkarte/pull/2049#issuecomment-194516529.

--
"The nitrogen in our DNA, the calcium in our teeth, the iron in our blood,
the carbon in our apple pies were made in the interiors of collapsing
stars. We are made of starstuff." - Carl Sagan

YAY!


[STORM-855] Add tuple batching

added new parameters to TopologyBuilder

batch_size (int): set the same batch size for all UDF declared output streams
batch_sizes (Map<String,Number>): sets the batch sizer for individual streams (including system streams)

added Batch type (as alternative to Tuple)

extended Kryo (de)serialization for Batch
integrated Tuple and Batch (de)serialization

add tuple batching

added internal tuple buffers (one for each output stream)
added Spout/Bolt output batching
added Bolt input debatching
added batching of "acks"

Could you fix typo in title, to let asfgit matches PR and JIRA issue properly? Thanks!

I guess I need to close an re-open this PR to get the linkage to JIRA...

asfbot recognizes your change and links. :)

Great!
Just a heads up: This is my first PR for Storm and I just started to learn Clojure. Please review very carefully. Looking forward to your feedback.

@mjsax
I'm not familiar with clojure, too. Many committers will take a look, so don't worry. :)
Before taking a look, I think you're encouraged to do benchmark and attach results, since it modifies critical path, especially latency vs throughput.

FYI, you can use https://github.com/yahoo/storm-perf-test for benchmarking if you don't have your own.
You need to modify pom.xml to let it points recent version of Storm and build.

Sure. Is there any specific approach I should take? How to add/report those result? I did a simple test already, using ExclamationTopology example. Setting batch size to 100 in Spout roughly doubled throughput and increased latency from 1 to 3 seconds. But this is of course not representative.

No, there's no specific approach.
You can refer @d2r's approach, https://github.com/apache/storm/pull/521#issuecomment-106038982
I think we would be interesting to compare these - not applying this patch, applying this patch & disable batch, applying this patch & some kind of batch sizes.

Thanks for your guidance! I will have a look at #521 and add some results. May take a few days...

Sure, please take your time!

Hi, I did a few performance tests and unfortunately, the impact of my changes is quite high. Not using batching in my branch (compared to master) reduces throughput by 40%. :(
I dug into the problem and it seems that there are two critical point in the code. First, I introduced function emit-msg (in executor.clj). This function uses a Java HashMap (output-batch-buffer) and it branches. Both have a large negative impact. Can it be that the access via HashMap-get() is slow in Clojure? What about branching? To me it seems that there performance impact is ridiculously high (even if I put into account that this code is called a few 100,000 times a second). Especially branch prediction should avoid the branching overhead at all. Because batching was disabled in the test, the else branch is taken every time...
Furthermore, I changed the serialization by overloading KryoTuple(Batch)Serializer.serialize(...) (I renamed this class). It seems that this overloading makes the call to .serialize(Tuple) much more expensive. I was thinking about changing the code, such that even if batching is disabled, I just use batches of size 1 to eliminate the overload. Of course, if comes with the drawback, that a single tuple consumers more space as I need to write the batch size in front of each batch.
Does this observation make sense? Or do you think I oversee something?

What happens when Batch.capacity - 1 tuples are emitted and then there's a long pause in the input?

Type hinting improves by 10% :) But 30% is still a huge gap.
About batches that don't fill up. We need to introduce a timeout (and a flushing thread or other flushing strategy). Otherwise, a batch might starve. I just wanted to get the basic design right before taking care of this problem.

Keep an eye out for more missing types. Those are often a source of unexpectedly bad performance.
I'm hesitant to add another thread, especially since we're operating on the critical path. We need to figure out some other way to flush.

@mjsax, I'm not sure it's not related. Here is the output from my benchmarks:
Pre-batching (apache master)
status	time            time-diff ms	transferred	throughput (MB/s)
WAITING<PHONE_NUMBER>132	0	        0	        0.0
WAITING<PHONE_NUMBER>132	30000	        99560           0.031649271647135414
WAITING<PHONE_NUMBER>131	29999	        2073460         0.6591571546037212
RUNNING<PHONE_NUMBER>131	30000	        2043680         0.6496683756510416
RUNNING<PHONE_NUMBER>132	30001	        2000600         0.6359524140536461
RUNNING<PHONE_NUMBER>133	30001	        1921980         0.6109606221947549
RUNNING<PHONE_NUMBER>131	29998	        1846040         0.5868794369819967
RUNNING<PHONE_NUMBER>132	30001	        1847640         0.5873293603429365
RUNNING<PHONE_NUMBER>133	30001	        1688440         0.5367227301733171
RUNNING<PHONE_NUMBER>131	29998	        1733740         0.5511778482986105
RUNNING<PHONE_NUMBER>131	30000	        1751700         0.5568504333496094
RUNNING<PHONE_NUMBER>133	30002	        1748280         0.5557261962158252
RUNNING<PHONE_NUMBER>132	29999	        1706500         0.54249982364321
RUNNING<PHONE_NUMBER>133	30001	        1787720         0.568281940243919
RUNNING<PHONE_NUMBER>131	29998	        1717900         0.5461421121922451
RUNNING<PHONE_NUMBER>132	30001	        1779180         0.5655672378466291
RUNNING<PHONE_NUMBER>132	30000	        1639820         0.5212847391764323
RUNNING<PHONE_NUMBER>131	29999	        1743720         0.5543321374058823
RUNNING<PHONE_NUMBER>134	30003	        1681140         0.5343665767700574
RUNNING<PHONE_NUMBER>133	29999	        1716760         0.5457614985278155
RUNNING<PHONE_NUMBER>131	29998	        1700800         0.5407058061683279
RUNNING<PHONE_NUMBER>132	30001	        1754100         0.5575947863098574
RUNNING<PHONE_NUMBER>134	30002	        1669220         0.53059537445225
RUNNING<PHONE_NUMBER>132	29998	        1757240         0.5586487951735845
RUNNING<PHONE_NUMBER>131	29999	        1781520         0.5663488343491658
RUNNING<PHONE_NUMBER>131	30000	        1772920         0.5635960896809896
RUNNING<PHONE_NUMBER>134	30003	        1656020         0.5263819422907969
RUNNING<PHONE_NUMBER>135	30001	        1557820         0.49520113449017844
RUNNING<PHONE_NUMBER>134	29999	        1750760         0.5565701677360599
RUNNING<PHONE_NUMBER>131	29997	        1773740         0.5639131519760049
RUNNING<PHONE_NUMBER>132	30001	        1656360         0.5265251127371275
RUNNING<PHONE_NUMBER>131	29999	        1770140         0.5627311091847593
RUNNING<PHONE_NUMBER>131	30000	        1767480         0.5618667602539062

With batching, no typehints.
These were gathered with the command:
storm jar storm_perf_test-1.0.0-SNAPSHOT-jar-with-dependencies.jar com.yahoo.storm.perftest.Main --ack --bolt 4 --name test -l 1 -n 1 --workers 4 --spout 3 --testTimeSec 900 -c topology.max.spout.pending=1092 --messageSize 10

I just pushed some changes (I added a new commit, so you can better see what I changed):

added type hints
split tuple and batch serialization in separate classes
assemble different "emit function" in Clojure for single tuple and batch case (to add more type hints)

I get the following result running on a 4 node cluster with parameters: storm jar storm_perf_test-1.0.0-SNAPSHOT-jar-with-dependencies.jar com.yahoo.storm.perftest.Main --bolt 3 --name test -l 1 -n 1 --messageSize 4 --workers 4 --spout 1 --testTimeSec 300
Master Branch:
status  topologies      totalSlots      slotsUsed       totalExecutors  executorsWithMetrics    time    time-diff ms    transferred     throughput (MB/s)
WAITING 1       48      0       4       0<PHONE_NUMBER>638   0       0       0.0
WAITING 1       48      4       4       4<PHONE_NUMBER>638   30000   6122840 0.7785593668619791
WAITING 1       48      4       4       4<PHONE_NUMBER>638   30000   11565400        1.4706166585286458
RUNNING 1       48      4       4       4<PHONE_NUMBER>638   30000   11394040        1.4488271077473958
RUNNING 1       48      4       4       4<PHONE_NUMBER>638   30000   11718240        1.49005126953125
RUNNING 1       48      4       4       4<PHONE_NUMBER>638   30000   11615920        1.4770406087239583
RUNNING 1       48      4       4       4<PHONE_NUMBER>638   30000   11557380        1.4695968627929688
RUNNING 1       48      4       4       4<PHONE_NUMBER>638   30000   11581080        1.4726104736328125
RUNNING 1       48      4       4       4<PHONE_NUMBER>638   30000   11492600        1.4613596598307292
RUNNING 1       48      4       4       4<PHONE_NUMBER>638   30000   11413760        1.4513346354166667
RUNNING 1       48      4       4       4<PHONE_NUMBER>638   30000   11300580        1.4369430541992188
RUNNING 1       48      4       4       4<PHONE_NUMBER>638   30000   11368760        1.4456125895182292
RUNNING 1       48      4       4       4<PHONE_NUMBER>638   30000   11509820        1.463549296061198

Batching branch with batching disabled:
status  topologies      totalSlots      slotsUsed       totalExecutors  executorsWithMetrics    time    time-diff ms    transferred     throughput (MB/s)
WAITING 1       48      0       4       0<PHONE_NUMBER>767   0       0       0.0
WAITING 1       48      4       4       4<PHONE_NUMBER>767   30000   7095940 0.9022954305013021
WAITING 1       48      4       4       4<PHONE_NUMBER>767   30000   11136640        1.4160970052083333
RUNNING 1       48      4       4       4<PHONE_NUMBER>767   30000   11159220        1.4189682006835938
RUNNING 1       48      4       4       4<PHONE_NUMBER>767   30000   7757660 0.9864374796549479
RUNNING 1       48      4       4       4<PHONE_NUMBER>767   30000   11375580        1.4464797973632812
RUNNING 1       48      4       4       4<PHONE_NUMBER>767   30000   11669980        1.4839146931966145
RUNNING 1       48      4       4       4<PHONE_NUMBER>767   30000   11344380        1.4425125122070312
RUNNING 1       48      4       4       4<PHONE_NUMBER>767   30000   11521460        1.4650293986002605
RUNNING 1       48      4       4       4<PHONE_NUMBER>767   30000   11401040        1.4497172037760417
RUNNING 1       48      4       4       4<PHONE_NUMBER>767   30000   11493700        1.461499532063802
RUNNING 1       48      4       4       4<PHONE_NUMBER>767   30000   11452680        1.4562835693359375
RUNNING 1       48      4       4       4<PHONE_NUMBER>767   30000   11148300        1.4175796508789062

Batching branch with batch size of 100 tuples:
status  topologies      totalSlots      slotsUsed       totalExecutors  executorsWithMetrics    time    time-diff ms    transferred     throughput (MB/s)
WAITING 1       48      1       4       0<PHONE_NUMBER>710   0       0       0.0
WAITING 1       48      4       4       4<PHONE_NUMBER>710   30000   11686000        1.4859517415364583
WAITING 1       48      4       4       4<PHONE_NUMBER>710   30000   18026640        2.292205810546875
RUNNING 1       48      4       4       4<PHONE_NUMBER>710   30000   17936300        2.2807184855143228
RUNNING 1       48      4       4       4<PHONE_NUMBER>710   30000   18969300        2.4120712280273438
RUNNING 1       48      4       4       4<PHONE_NUMBER>710   30000   18581620        2.3627751668294272
RUNNING 1       48      4       4       4<PHONE_NUMBER>711   30001   18963120        2.4112050268897285
RUNNING 1       48      4       4       4<PHONE_NUMBER>710   29999   18607200        2.3661067022546587
RUNNING 1       48      4       4       4<PHONE_NUMBER>710   30000   19333620        2.4583969116210938
RUNNING 1       48      4       4       4<PHONE_NUMBER>710   30000   18629100        2.3688125610351562
RUNNING 1       48      4       4       4<PHONE_NUMBER>711   30001   18847820        2.3965443624209923
RUNNING 1       48      4       4       4<PHONE_NUMBER>710   29999   18021400        2.291615897287722
RUNNING 1       48      4       4       4<PHONE_NUMBER>710   30000   18143360        2.3070475260416665

The negative impact is gone and batching increases output rate by about 50%. Need to do more tests. Also need to investigate the performance impact of input debachting. Furthermore, need to test with acking enabled.
Some more question:

What about assert-can-serialize? Is it performance critical? Did not test it, but it seems that a generic approach for tuple and batch should be good enough.
I also do not understand the following code (line 648-622 in executor.clj in batching branch). I used batching, but did not modify this code. Nevertheless it works (I guess this part is not executed?). Can you explain?

(task/send-unanchored task-data
                                      ACKER-INIT-STREAM-ID
                                      [root-id (bit-xor-vals out-ids) task-id]
                                      overflow-buffer)

What about batching acks? Would it make sense? I don't understand the acking code path good enough right now to judge. As acking is quite expensive, it might be a good idea.

Excellent work. This looks very promising. I'll address your questions tomorrow.

I just checked some older benchmark result doing batching in user land, ie, on top of Storm (=> Aeolus). For this case, a batch size of 100 increased the spout output rate by a factor of 6 (instead of 1.5 as the benchmark above shows). The benchmark should yield more than 70M tuples per 30 seconds... (and not about 19M).
Of course, batching is done a little different now. In Aeolus, a fat-tuple is used as batch. Thus, the system sees only a single batch-tuple. Now, the system sees all tuples, but emitting is delayed until the batch is full (this still saved the overhead of going through the disruptor for each tuple). However, we generate a tuple-ID for each tuple in the batch, instead of a single ID per batch. Not sure how expensive this is. Because acking was not enabled, it should not be too expensive, because the IDs have not to be "registered" at the ackers (right?).
As a further optimization, it might be a good idea not to batch whole tuples, but only Values and tuple-id. The worker-context, task-id, and outstream-id is the same for all tuples within a batch. I will try this out, and push a new version the next days if it works.

Here are some additional benchmark results with larger --messageSize (ie, 100 and 250). Those benchmarks are run in a 12 node cluster (with nimbus.thrift.max_buffer_size: 33554432 and worker.childopts: "-Xmx2g") as follows:
storm jar target/storm_perf_test-1.0.0-SNAPSHOT-jar-with-dependencies.jar com.yahoo.storm.perftest.Main --name test -l 1 -n 1 --messageSize 100 --workers 24 --spout 1 --bolt 10 --testTimeSec 300
As you can observe, the output rate is slightly reduced for larger tuple size in all cases (what is expected due to larger serialization costs). That batching case with batch size 100 and--messageSize 250 does not improve output rate as significant as before, because it hits the underlaying 1Gbit Ethernet limit. In the beginning, it starts promising. After about 2 minutes performance does down. Because tuples cannot be transfered over the network fast enough, they get buffered in main memory which reaches it's limit (assumption) and thus Storm slows down the spout. Not sure, why performances goes down in each report. Any ideas?
--messageSize 100
master branch
status  topologies      totalSlots      slotsUsed       totalExecutors  executorsWithMetrics    time    time-diff ms    transferred     throughput (MB/s)
WAITING 1       144     0       11      0<PHONE_NUMBER>523   0       0       0.0
WAITING 1       144     11      11      11<PHONE_NUMBER>523   30000   6336040 20.14172871907552
WAITING 1       144     11      11      11<PHONE_NUMBER>523   30000   10554200        33.55089823404948
RUNNING 1       144     11      11      11<PHONE_NUMBER>523   30000   10005840        31.807708740234375
RUNNING 1       144     11      11      11<PHONE_NUMBER>523   30000   10514120        33.4234873453776
RUNNING 1       144     11      11      11<PHONE_NUMBER>523   30000   10430700        33.158302307128906
RUNNING 1       144     11      11      11<PHONE_NUMBER>523   30000   7608960 24.188232421875
RUNNING 1       144     11      11      11<PHONE_NUMBER>523   30000   10279460        32.67752329508463
RUNNING 1       144     11      11      11<PHONE_NUMBER>524   30001   10496260        33.36559974774929
RUNNING 1       144     11      11      11<PHONE_NUMBER>523   29999   10382860        33.00732328691556
RUNNING 1       144     11      11      11<PHONE_NUMBER>523   30000   10138280        32.22872416178385
RUNNING 1       144     11      11      11<PHONE_NUMBER>523   30000   10072940        32.02101389567057
RUNNING 1       144     11      11      11<PHONE_NUMBER>523   30000   10095820        32.09374745686849

batching branch (no batching)
status  topologies      totalSlots      slotsUsed       totalExecutors  executorsWithMetrics    time    time-diff ms    transferred     throughput (MB/s)
WAITING 1       144     0       11      0<PHONE_NUMBER>917   0       0       0.0
WAITING 1       144     11      11      11<PHONE_NUMBER>917   30000   4467900 14.203071594238281
WAITING 1       144     11      11      11<PHONE_NUMBER>917   30000   10616160        33.74786376953125
RUNNING 1       144     11      11      11<PHONE_NUMBER>917   30000   10473700        33.294995625813804
RUNNING 1       144     11      11      11<PHONE_NUMBER>917   30000   10556860        33.55935414632162
RUNNING 1       144     11      11      11<PHONE_NUMBER>917   30000   10580760        33.63533020019531
RUNNING 1       144     11      11      11<PHONE_NUMBER>917   30000   10367580        32.95764923095703
RUNNING 1       144     11      11      11<PHONE_NUMBER>917   30000   10646760        33.84513854980469
RUNNING 1       144     11      11      11<PHONE_NUMBER>917   30000   10750300        34.17428334554037
RUNNING 1       144     11      11      11<PHONE_NUMBER>917   30000   10607220        33.719444274902344
RUNNING 1       144     11      11      11<PHONE_NUMBER>917   30000   10456920        33.24165344238281
RUNNING 1       144     11      11      11<PHONE_NUMBER>917   30000   10108000        32.132466634114586
RUNNING 1       144     11      11      11<PHONE_NUMBER>917   30000   10576120        33.6205800374349

batching branch (batch size: 100)
status  topologies      totalSlots      slotsUsed       totalExecutors  executorsWithMetrics    time    time-diff ms    transferred     throughput (MB/s)
WAITING 1       144     0       11      0<PHONE_NUMBER>049   0       0       0.0
WAITING 1       144     11      11      11<PHONE_NUMBER>049   30000   11346480        36.069488525390625
WAITING 1       144     11      11      11<PHONE_NUMBER>050   30001   17333500        55.0998758822297
RUNNING 1       144     11      11      11<PHONE_NUMBER>049   29999   17815260        56.635074176137906
RUNNING 1       144     11      11      11<PHONE_NUMBER>049   30000   17993660        57.200304667154946
RUNNING 1       144     11      11      11<PHONE_NUMBER>049   30000   17720880        56.333160400390625
RUNNING 1       144     11      11      11<PHONE_NUMBER>050   30001   17957200        57.08249869861108
RUNNING 1       144     11      11      11<PHONE_NUMBER>049   29999   18286500        58.13315572840058
RUNNING 1       144     11      11      11<PHONE_NUMBER>050   30001   18027820        57.306986149778076
RUNNING 1       144     11      11      11<PHONE_NUMBER>049   29999   12470520        39.64403692199896
RUNNING 1       144     11      11      11<PHONE_NUMBER>049   30000   17256760        54.8577626546224
RUNNING 1       144     11      11      11<PHONE_NUMBER>049   30000   17288300        54.95802561442057
RUNNING 1       144     11      11      11<PHONE_NUMBER>049   30000   17077300        54.28727467854818

--messageSize 250
master branch
status  topologies      totalSlots      slotsUsed       totalExecutors  executorsWithMetrics    time    time-diff ms    transferred     throughput (MB/s)
WAITING 1       144     0       11      0<PHONE_NUMBER>498   0       0       0.0
WAITING 1       144     11      11      11<PHONE_NUMBER>498   30000   6047420 48.06057612101237
WAITING 1       144     11      11      11<PHONE_NUMBER>498   30000   9909100 78.7504514058431
RUNNING 1       144     11      11      11<PHONE_NUMBER>498   30000   9846400 78.25215657552083
RUNNING 1       144     11      11      11<PHONE_NUMBER>498   30000   7100940 56.43320083618164
RUNNING 1       144     11      11      11<PHONE_NUMBER>498   30000   9870500 78.4436861673991
RUNNING 1       144     11      11      11<PHONE_NUMBER>498   30000   9856460 78.33210627237956
RUNNING 1       144     11      11      11<PHONE_NUMBER>498   30000   9662740 76.79255803426106
RUNNING 1       144     11      11      11<PHONE_NUMBER>498   30000   10060860        79.9565315246582
RUNNING 1       144     11      11      11<PHONE_NUMBER>499   30001   10049660        79.86485975980162
RUNNING 1       144     11      11      11<PHONE_NUMBER>498   29999   9937680 78.98021751278428
RUNNING 1       144     11      11      11<PHONE_NUMBER>498   30000   10061600        79.96241251627605
RUNNING 1       144     11      11      11<PHONE_NUMBER>498   30000   9804960 77.92282104492188

batching branch (no batching)
status  topologies      totalSlots      slotsUsed       totalExecutors  executorsWithMetrics    time    time-diff ms    transferred     throughput (MB/s)
WAITING 1       144     0       11      0<PHONE_NUMBER>880   0       0       0.0
WAITING 1       144     11      11      11<PHONE_NUMBER>880   30000   4983420 39.60466384887695
WAITING 1       144     11      11      11<PHONE_NUMBER>880   30000   9760600 77.57027943929036
RUNNING 1       144     11      11      11<PHONE_NUMBER>881   30001   9344540 74.26125626338171
RUNNING 1       144     11      11      11<PHONE_NUMBER>880   29999   9323920 74.10232867951068
RUNNING 1       144     11      11      11<PHONE_NUMBER>880   30000   9354460 74.3425687154134
RUNNING 1       144     11      11      11<PHONE_NUMBER>880   30000   9601440 76.30538940429688
RUNNING 1       144     11      11      11<PHONE_NUMBER>880   30000   9476900 75.3156344095866
RUNNING 1       144     11      11      11<PHONE_NUMBER>880   30000   9581560 76.14739735921223
RUNNING 1       144     11      11      11<PHONE_NUMBER>881   30001   9494500 75.4529915429414
RUNNING 1       144     11      11      11<PHONE_NUMBER>880   29999   9260020 73.59448017774095
RUNNING 1       144     11      11      11<PHONE_NUMBER>880   30000   9054000 71.95472717285156
RUNNING 1       144     11      11      11<PHONE_NUMBER>880   30000   9278340 73.73762130737305
RUNNING 1       144     11      11      11<PHONE_NUMBER>880   30000   9153180 72.74293899536133

batching branch (batch size: 100)
status  topologies      totalSlots      slotsUsed       totalExecutors  executorsWithMetrics    time    time-diff ms    transferred     throughput (MB/s)
WAITING 1       144     0       11      0<PHONE_NUMBER>492   0       0       0.0
WAITING 1       144     11      11      11<PHONE_NUMBER>492   30000   10616760        84.37442779541016
WAITING 1       144     11      11      11<PHONE_NUMBER>492   30000   15851220<PHONE_NUMBER>1420898
RUNNING 1       144     11      11      11<PHONE_NUMBER>492   30000   12884860<PHONE_NUMBER>0408528
RUNNING 1       144     11      11      11<PHONE_NUMBER>492   30000   8743860 69.48995590209961
RUNNING 1       144     11      11      11<PHONE_NUMBER>492   30000   6445660 51.225503285725914
RUNNING 1       144     11      11      11<PHONE_NUMBER>492   30000   5974480 47.48090108235677
RUNNING 1       144     11      11      11<PHONE_NUMBER>493   30001   5198640 41.3137016119645
RUNNING 1       144     11      11      11<PHONE_NUMBER>492   29999   5209800 41.405150618464624
RUNNING 1       144     11      11      11<PHONE_NUMBER>492   30000   4585960 36.445935567220054
RUNNING 1       144     11      11      11<PHONE_NUMBER>492   30000   3870140 30.75710932413737
RUNNING 1       144     11      11      11<PHONE_NUMBER>492   30000   3751040 29.810587565104168
RUNNING 1       144     11      11      11<PHONE_NUMBER>493   30001   3450600 27.421990901898322

acking does seem to be having some issues.  The ack tuples don't seem to be batched, so the serialization code gets confused.

@mjsax did you get a chance to look at the ack tuples issue. This is going to be great perf improvement would like to see this merged in.

I did not have time yet. However, I am still on it... I did already some more work for this PR that is not pushed yet (for example to support different output batches for different output streams and some "clean up" of my changes). I think I have an overall cleaner design now of my code. As a next step, I actually want to investigate why this PR only gives factor two throughput improvement (because Aeolus give factor five). But if you prefer, I can also work on the ack problem first to get a fully running PR for now. (I just disabled acking in my current tests ;))

@mjsax Thanks for the quick reply. It would be great to ack issue fixed than others can also run  some tests.

Ok. Hope to get it done over the weekend...

I just realized, that some commits from other people got added to this PR. This confuses me. Can you guide me through the process Storm development is following here? I am not used to that...

@mjsax the issue is with how you upmerge.  If you just do a git merge or a git pull github can become confused because it thinks you are still based off of the original commit, and will include the commits from the upmerge.
Alternatively you could do a git rebase and rebase all of your changes on a new version, instead of merging in the new version.  This will clean it up, but before you do this please make sure you have a backup of your changes, and it does a destructive write that can, and I have had this personally happen to me, delete all of your code.  When you are pushing to the github repo you will have to include a -f because git by default does not like destructive writes and you have to force it to do them.

I see. It's a github issue... Usually I rebase before updating a PR. This time I did not... Thanks for the quick response.

I just pushed a new version (rebased to current master and squahed commits etc). The PR is in much cleaner state now. Looking forward to your feedback.
Batching does now work with acks and each output stream can have a different batch size. The ack streams (__ack_init, __ack_ack, and __ack_fail) are treated as regular streams from a batching point of view. Metric and Eventlogger should work too (but I did not test is).
If you want to test this, be aware that hybrid batching is not supported yet (for performance reasons -- at least for the non-acking case). Thus, right now you set batch size via and int only if acking is disabled or via HashMap and the map must contain an entry for each output stream (including ack streams) and the batch size must be at least 1. (After we decided how to proceed -- see next paragraph --, this can be cleaned up.)
The problem with hybrid batching is the serialization path in executor.clj at mk-transfer-fn and start-batch-transfer->worker-handler!. I wrote the code for hybrid serialization already but disabled it, ie, put it into comments. Because, I am not able to set different serializers for different output stream, only a hybrid serializer could be used. However, the runtime binding to the correct method for TupleImpl or Batch reduced the throughput (see numbers below). Not sure if/how this could be resolved. On the other hand, a batch size of one does not have a big performance penalty -- maybe it would be worth to enable batching all the time (ie, even for the non-batching case, just use a batch size of one) to avoid the hybrid setup.
I also did some network monitoring using nmon. It shows that batching does reduce the actually transfered number of byte over the network. The perf tool does not measure but compute the number of transfered bytes (what is not quite accurate). Right now, I don't have the numbers, but if you wish I could rerun those experiments and post here.
I collected the following number for non-acking and acking:
NO ACKING
--name test -l 1 -n 1 --messageSize 100 --workers 24 --spout 1 --bolt 10 --testTimeSec 40
no batching:
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	0	11	0<PHONE_NUMBER>669	0	0	0.0
WAITING	1	96	11	11	11<PHONE_NUMBER>669	30000	6688960	21.263631184895832
WAITING	1	96	11	11	11<PHONE_NUMBER>669	30000	7518460	23.900540669759113
RUNNING	1	96	11	11	11<PHONE_NUMBER>669	30000	10428980	33.15283457438151
RUNNING	1	96	11	11	11<PHONE_NUMBER>669	30000	10395200	33.045450846354164

batch size = 1 (to measure overhead; about 5%):
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	0	11	0<PHONE_NUMBER>193	0	0	0.0
WAITING	1	96	11	11	11<PHONE_NUMBER>193	30000	3089120	9.820048014322916
WAITING	1	96	11	11	11<PHONE_NUMBER>193	30000	9134740	29.038556416829426
RUNNING	1	96	11	11	11<PHONE_NUMBER>193	30000	9502680	30.208206176757812
RUNNING	1	96	11	11	11<PHONE_NUMBER>193	30000	9672300	30.747413635253906

batch size = 100 (throughput improvement by about 85%)
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	0	11	0<PHONE_NUMBER>994	0	0	0.0
WAITING	1	96	11	11	11<PHONE_NUMBER>994	30000	8345560	26.529820760091145
WAITING	1	96	11	11	11<PHONE_NUMBER>994	30000	19876460	63.18556467692057
RUNNING	1	96	11	11	11<PHONE_NUMBER>994	30000	18229880	57.95122782389323
RUNNING	1	96	11	11	11<PHONE_NUMBER>994	30000	18294660	58.15715789794922

ACKING
--name test -l 1 -n 1 --messageSize 100 --workers 24 --spout 1 --bolt 10 --testTimeSec 40 --ack --ackers 10
no batching:
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	0	21	0<PHONE_NUMBER>024	0	0	0.0
WAITING	1	96	21	21	21<PHONE_NUMBER>024	30000	864800	2.7491251627604165
WAITING	1	96	21	21	21<PHONE_NUMBER>024	30000	1768760	5.6227366129557295
RUNNING	1	96	21	21	21<PHONE_NUMBER>024	30000	1910340	6.072807312011719
RUNNING	1	96	21	21	21<PHONE_NUMBER>025	30001	1888740	6.003942629809475

Complete Latency (from WebUI): 6.256
all batch sizes = 1 (to measure overhead; acking dominates; no overhead measurable):
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	2	21	0<PHONE_NUMBER>224	0	0	0.0
WAITING	1	96	21	21	21<PHONE_NUMBER>225	30001	803060	2.552773895980811
WAITING	1	96	21	21	21<PHONE_NUMBER>225	30000	2001520	6.362660725911458
RUNNING	1	96	21	21	21<PHONE_NUMBER>224	29999	1789860	5.690001373255411
RUNNING	1	96	21	21	21<PHONE_NUMBER>224	30000	1925420	6.120745340983073

Complete Latency (from WebUI): 9.686 (no impact -- 3ms difference is just too small to be a reliable number)
default batch size = 100 (almost no throughput improvement; about 15%; acking dominates as acks are not batched)
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	0	21	0<PHONE_NUMBER>423	0	0	0.0
WAITING	1	96	21	21	21<PHONE_NUMBER>423	30000	866980	2.7560551961263022
WAITING	1	96	21	21	21<PHONE_NUMBER>423	30000	2076100	6.599744160970052
RUNNING	1	96	21	21	21<PHONE_NUMBER>424	30001	2100200	6.676133459939356
RUNNING	1	96	21	21	21<PHONE_NUMBER>424	30000	2191620	6.966972351074219

Complete Latency (from WebUI): 11.721 (compared to 6ms, almost doubles, but this is still tiny)
all batch sizes = 100 (acks are batched too; throughput improvement increases to more than 50%)
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	0	21	0<PHONE_NUMBER>845	0	0	0.0
WAITING	1	96	21	21	21<PHONE_NUMBER>846	30001	1728100	5.493298843977336
WAITING	1	96	21	21	21<PHONE_NUMBER>845	29999	2814100	8.946081182035496
RUNNING	1	96	21	21	21<PHONE_NUMBER>845	30000	2084440	6.6262563069661455
RUNNING	1	96	21	21	21<PHONE_NUMBER>845	30000	2949700	9.376843770345053

Complete Latency (from WebUI): 65.225 (latency increased by factor of 10 due to ack batching)
===== Hybrid =====
I repeated the same experiment using hybrid serializer (KryoTupleBatchSerializer): It is a similar, however with reduced throughput by 25% for the non-acking case. In the acking case, the hybrid approach has no influence as acking dominates.
NO ACKING (hybrid)
--name test -l 1 -n 1 --messageSize 100 --workers 24 --spout 1 --bolt 10 --testTimeSec 40
no batching:
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	0	11	0<PHONE_NUMBER>016	0	0	0.0
WAITING	1	96	11	11	11<PHONE_NUMBER>016	30000	5118760	16.27209981282552
WAITING	1	96	11	11	11<PHONE_NUMBER>017	30001	8131680	25.84905291568406
RUNNING	1	96	11	11	11<PHONE_NUMBER>016	29999	7960300	25.305955734820067
RUNNING	1	96	11	11	11<PHONE_NUMBER>016	30000	7711800	24.515151977539062

batch size = 1 (to measure overhead):
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	0	11	0<PHONE_NUMBER>726	0	0	0.0
WAITING	1	96	11	11	11<PHONE_NUMBER>726	30000	4799100	15.255928039550781
WAITING	1	96	11	11	11<PHONE_NUMBER>726	30000	7695580	24.463589986165363
RUNNING	1	96	11	11	11<PHONE_NUMBER>727	30001	7828800	24.886255419090197
RUNNING	1	96	11	11	11<PHONE_NUMBER>726	29999	5613780	17.84632089054661

batch size = 100:
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	0	11	0<PHONE_NUMBER>666	0	0	0.0
WAITING	1	96	11	11	11<PHONE_NUMBER>666	30000	9114920	28.975550333658855
WAITING	1	96	11	11	11<PHONE_NUMBER>666	30000	16459180	52.32232411702474
RUNNING	1	96	11	11	11<PHONE_NUMBER>666	30000	16347100	51.96603139241537
RUNNING	1	96	11	11	11<PHONE_NUMBER>666	30000	16594400	52.752176920572914

ACKING  (hybrid)
--name test -l 1 -n 1 --messageSize 100 --workers 24 --spout 1 --bolt 10 --testTimeSec 40 --ack --ackers 10
no batching:
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	0	21	0<PHONE_NUMBER>486	0	0	0.0
WAITING	1	96	21	21	21<PHONE_NUMBER>486	30000	934640	2.9711405436197915
WAITING	1	96	21	21	21<PHONE_NUMBER>486	30000	1774140	5.639839172363281
RUNNING	1	96	21	21	21<PHONE_NUMBER>486	30000	1875420	5.961799621582031
RUNNING	1	96	21	21	21<PHONE_NUMBER>486	30000	1832160	5.82427978515625

Complete Latency (from WebUI): 43.945
all batch sizes = 1 (to measure overhead):
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	3	21	0<PHONE_NUMBER>807	0	0	0.0
WAITING	1	96	21	21	21<PHONE_NUMBER>807	30000	1192380	3.7904739379882812
WAITING	1	96	21	21	21<PHONE_NUMBER>807	30000	1915040	6.087748209635417
RUNNING	1	96	21	21	21<PHONE_NUMBER>808	30001	1387960	4.412058945365883
RUNNING	1	96	21	21	21<PHONE_NUMBER>807	29999	2106580	6.696860700206934

Complete Latency (from WebUI): 9.132
default batch size = 100:
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	5	21	0<PHONE_NUMBER>612	0	0	0.0
WAITING	1	96	21	21	21<PHONE_NUMBER>612	30000	1449080	4.6065012613932295
WAITING	1	96	21	21	21<PHONE_NUMBER>613	30001	2369020	7.5306607414843985
RUNNING	1	96	21	21	21<PHONE_NUMBER>612	29999	2323580	7.386708117321359
RUNNING	1	96	21	21	21<PHONE_NUMBER>612	30000	2414040	7.6740264892578125

Complete Latency (from WebUI): 11.063
all batch sizes = 100:
status	topologies	totalSlots	slotsUsed	totalExecutors	executorsWithMetrics	time	time-diff ms	transferred	throughput (MB/s)
WAITING	1	96	0	21	0<PHONE_NUMBER>562	0	0	0.0
WAITING	1	96	21	21	21<PHONE_NUMBER>562	30000	1278060	4.062843322753906
WAITING	1	96	21	21	21<PHONE_NUMBER>562	30000	2170020	6.898307800292969
RUNNING	1	96	21	21	21<PHONE_NUMBER>563	30001	2209060	7.022178545383122
RUNNING	1	96	21	21	21<PHONE_NUMBER>562	29999	2155180	6.851361089477723

Complete Latency (from WebUI): 122.958

One question: There are a lot of changes in storm-core/src/jvm/backtype/storm/generated/* resulting from rebuild those files with genthrift.sh. However, it seems to me that only the changes to ComponentCommon.java (and I guess to storm-core/src/py/storm/ttypes.py) are relevant. For all other classes it this package it seems they include variable renaming only. The files in which only the generation date was changed are not included already. Can I safely revert those other files, too? Or might I break something?

Hi, what is the next step?

I still need to run some tests.  I am way behind on my open source commitments.  I will try really hard this week to play around with this and let you know the results.

Thx. Take your time; there is actually no rush. I was just curious :)

Closing this (also closing the JIRA) as Bobby's work (https://github.com/apache/storm/pull/765) got merged and I don't have time to work on this right now.


Visualisation on video is added

hi @Anujg935 and @akshitagupta15june  visualisation on video is added please look into it.

good work merged your PR


AWS permissions in docs wrong for s3:AbortMultipartUpload

I'm filing this issue in part to remind myself to submit a PR for this later today: the s3:AbortMultipartUpload permission is not in the right place in docs/aws-config.md. Aborting a multipart upload is an object- and not bucket-level permission.
I'm not sure there's a circumstance where ark would submit a multipart upload, but in the interest of correctness this should be changed so that AbortMultipartUpload appears in this policy statement in the docs (docs/aws-config.md):
{
    "Effect": "Allow",
    "Action": [
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:PutObject",
        "s3:AbortMultipartUpload"
    ],
    "Resource": [
        "arn:aws:s3:::${BUCKET}/*"
    ]
}

I have AWS's policy UI checker thing to thank for noticing this.

Thanks @hopkinsth! Looking forward to the PR to fix this.


[BUG] Improve the review section

Description
Improve to a much more pleasing sliding review section
Screenshots

Any additional information?
No response
What browser are you seeing the problem on?
No response
Checklist

[X] I have checked the existing issues.
[X] I have read the Code Of Conduct
[X] The changes don't break the code
[X] Please attach the screenshots and video of the development work once it is done

Assign me this task i will do it asap !


traits: Introduce std feature

This makes it possible to build traits without std. For this a new
trait BasicFloat was introduced, implementing some basic functionality
that works with core. Most notably this is lacking functions like
cos, sin, etc.
Float is not available without std.
Refs #216.

There's a CI failure reported across all versions.  I think this probably needs some direct no-std testing in ci/test_full.sh too.
Did you consider just gating #[cfg(feature = "std")] on Float methods as needed?  I know that makes a lot more places that have to be marked, but I think it's easier to explain.  I don't like the method duplication, and I don't want users to be confused about when to use Float or BasicFloat.
If you really want to have two separate traits, maybe it would be better to have Float: BasicFloat so they are directly related.

maybe it would be better to have Float: BasicFloat

Hmm, but that's a breaking change for implementors. :/

Hmm, but that's a breaking change for implementors. :/

Yes, I considered that, but discarded it for this reason.
I like your suggestion of gating the methods, this would have the advantage that there will be no redundant reimplementation of the methods currently in BasicFloat if std is available.

I did what you suggested, which coincidentally also fixed the tests.
I added quite a few default implementations to Float. They only matter for no_std at the moment.

@cuviper I think I addressed all your comments.

@homu r+

:pushpin: Commit ba73ba2 has been approved by cuviper

:hourglass: Testing commit ba73ba2 with merge 8b5d4ac...

:sunny: Test successful - status

This is a breaking change and the release should be yanked. Feature-gating things that were previously not feature-gated is a breaking change.


Add an implicit Converter for Long

fixes #254

Hmm...  the build seems to have failed, but I think it's a build config issue.  Let me know if I've borked something.


[3.3.2] Software Design

Issue Details
Description
This Main Task is for the team to refine our Software Design Diagrams that we have created previously, based on the changes made after some feedback.
Goals
Have a Completed Architecture Diagram and 2 Detailed UML Design Diagrams in out M3 Report.
Success or Completion Criteria

[x] #181
[x] #161

Allocated Time
6 Days
Start Date: 07/11/23, 0900
End Date: 14/11/23, 2359

Issue Comments
Description
All Sub-Issues have been completed!
Changes to success or completion criteria? y/n (if y, describe)
No Changes, Criteria Completed!
Start Date: 07/11/23, 0900
End Date: 14/11/23, 2359


Update camranger.rb

Remove version key from app location

Ok @vitorgalvao should be ready now but I did it through the web interface so don't think I can squash without downloading the patch branch locally which I'm not able to do right now.

No problem, I’ll pull and squash, then.

Thank you for the contribution. It needed some fixes, so they were made in 3f3def7a15ff002e48707d469d5f3cf83e2c9db9. Your contribution is still included (and still credited to you), with the appropriate modifications. Please feel free to ask about any of the changes.


Add key binding to answer "Y" to pry requests in the REPL.

Pry prompts the user for permission, which forces the user to type "Y" directly in the REPL. It would be more convenient to add a key binding (suggest cmd+alt+Y) to automatically respond to the pry request so we can avoid changing focus back and forth between the REPL.

Done.


Update mixer-rule-additional-telemetry.yaml

Fix Additional telemetry for reviews' response size

I signed it!

thx!


Publish Godot VRM for Godot 4

Describe the proposed feature and how it helps to overcome a problem or limitation
I want to be able to download the godot vrm addon from the asset library in 4.
Other information about your context or usecase.
Developing content changes.
If known, describe how your proposal will work, with code, pseudo-code, mock-ups, or diagrams
Submit godot vrm to the godot 4 asset library @lyuma
Anything else
Started because of discussion with https://twitter.com/AlfredBaudisch/status/1586999148171198465?s=20&t=nUHLXskgrxnTbPFN9WzZxQ about mtoon outlines

Added https://github.com/V-Sekai/V-Sekai.github.io/commit/ee74ba5ae6b98b48701fac8c157d91f335d58561


Add proxy support

Also reorganize tests a bit to make them shorter and use different MockServer instances.
The code is ready, but I'll need to debug tests on travis

Tests pass now, using a proxy.py proxy on travis to simplify setup.
@kmike can you review this one too? :)

One thing I'm not sure about is how to better expose it in the autologin-middleware. We can either pass HTTP_PROXY and HTTPS_PROXY directly, as we do with SPLASH_URL now, or prefix them with AUTOLOGIN_.


Turn NME doc page into an NME doc chapter

@PatrickRyanMS commented on Tue Oct 27 2020
The NME doc page covers a lot of ground and does not go deep enough on some aspects of the tool. We should split the page up into several pages that handle one of the modes from node material such as:

creating a node material, loading, sharing, snippet
using NME
standard material
pbr material
procedural textures
post process
particles

In this way we could take space to dive deeper into how to use the nodes and explain if there are expectations around input/output ports on nodes. PBR and its subnodes could use more in-depth explanation about how to use the individual nodes and what each one expects.

Addressed in this PR:
https://github.com/BabylonJS/Documentation/pull/401


make stringly typed types more consistent with aliri_braid

I kindof just want to add this, then make a new release.
I like what it does, unsure if the cost is worth it though.
I don't want the crate to become too heavy...

I think I'll do this, and publish new version

should consider moving the types to a new crate, but not right now

bors r+

oops, missing changelog for this....


Allow FileSystemLoader to follow symlinks

Since python 2.6 os.walk has the 'followlinks' parameter.
Could this be made an option or even a default when using python 2.6 or higher?

When loading templates the FileSystemLoader follows symlinks. So that its list_templates() method don't follow symlinks, is  definitely a bug. However just calling os.walk(..., followlinks=True) when using Python >=2.6, won't fix that bug for people using jinja2 with older versions of Python.


Event Signup Cleanup

A NumberOfVolunteersRequired property was left on the event model after I removed the concept of Event level volunteer signups.
There are also a couple of extra lines of JS code that could be removed relating to the concept of event signups.

I have code to address this. Having some issues running the site locally though to test I haven't broken and functionality. As soon as I can sort that I will put in a PR


Another quantms MSstats error

Description of the bug

nf-core/quantms execution completed unsuccessfully!

The exit status of the task that caused the workflow execution to fail was: 1.

The full error message was:

Error executing process > 'NFCORE_QUANTMS:QUANTMS:TMT:MSSTATSTMT (PDC000114.sdrf_openms_design_msstats_in.csv)'

Caused by:
  Process `NFCORE_QUANTMS:QUANTMS:TMT:MSSTATSTMT (PDC000114.sdrf_openms_design_msstats_in.csv)` terminated with an error exit status (1)

Command executed:

  msstats_tmt.R \
      PDC000114.sdrf_openms_design_msstats_in.csv \
      "pairwise" \
      "" \
      true \
      true \
      true \
      sum \
      msstats \
      true \
      true \
      true \
      PDC000114.sdrf_openms_design_msstats_in \
      0.05 \
       \
      2>&1 | tee msstats_tmt.log

  cat <<-END_VERSIONS > versions.yml
  "NFCORE_QUANTMS:QUANTMS:TMT:MSSTATSTMT":
      r-base: $(echo $(R --version 2>&1) | sed 's/^.*R version //; s/ .*$//')
      bioconductor-msstatstmt: $(Rscript -e "library(MSstatsTMT); cat(as.character(packageVersion('MSstatsTMT')))")
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Work dir:
  /hps/nobackup/juan/pride/reanalysis/differential-expression/tmt/PDC000114/work/95/ed92d865febad9cc5ee8e5ba542e96

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

Command used and terminal output
No response
Relevant files
No response
System information
No response

The actual error in MSstats is:
INFO  [2023-09-09 13:06:46] ** Protein-level summarization done by MSstats.
There were 50 or more warnings (use warnings() to see the first 50)
Drew the Profile plot for 10910 proteins.
  |                                                                      |   0%Error in data.frame(groupline, PSM = unique(single_protein$PSM)[1], PeptideSequence = unique(single_protein$PeptideSequence)[1]) :
  arguments imply differing number of rows: 0, 1
Calls: dataProcessPlotsTMT -> .plotProfileTMT -> data.frame
Execution halted
pst_prd@hl-codon-102-04


Support for k,K

Hi,
Any chance you can add support for k and K?
H	Hour in day (0-23)	Number	0
k	Hour in day (1-24)	Number	24
K	Hour in am/pm (0-11)	Number	0
h	Hour in am/pm (1-12)	Number	12

moment doesn't support this.


7.4: Deadlock when resolving dependencies using dependency verification and GradleBuild task

Current Behavior
The Gradle execution deadlocks during dependency resolution in a GradleBuild task execution if dependency verification is used.
Steps to Reproduce
Unfortunately the project is still closed source, but I think the thread dump can be of help here.
blocked.txt
Your Environment
Gradle 7.4
JDK 1.8

I can now confirm that this was also broken in 7.3.3.

What's the last working version for you?

Let me try and get back to you with that. This was the first Gradle release I really tried it out, so I'll have to step all the way back to 6.2 if necessary.

Maybe related to the same race condition, I randomly but infrequently get
Execution failed for task ':sub:task'.
> Dependency verification failed for configuration ':sub:task'
  One artifact failed verification: core-0.1.39-SNAPSHOT.jar (project:core:0.1.39-SNAPSHOT) from repository MavenLocal

when I'm trying to provoke the issue. This was also on 7.3.3.

I've been able to reproduce the deadlocks in 7.0.2, 7.1.1, 7.2 7.3.3 and 7.4, and the ones from 6.x had other issues that I can't seem to get around. I haven't bothered with the other tiny releases in 7.x, such as 7.0 and 7.0.1.

I can confirm that this is still an issue in Gradle 8.0.2. I've attached a thread dump.
blocked.gradle-8.0.2.txt


[bug] featuredWalletIds option is broken, starting from v4.2.2

Link to minimal reproducible example
let options = {     defaultChain: aurora,     featuredWalletIds: [         '76260019aec5a3c44dd2421bf78e80f71a6c090d932c413a287193ed79450694', //AuroraPass     ]};
Summary
Try to use the options as above, you will not see Aurora Pass wallet added to the main wallets page.

In 4.2.1 and lower versions - it is working and is diplayed as expected.

List of related npm package versions
4.2.2, 4.2.3

Same issue here.

Related issue with customization:

https://github.com/WalletConnect/web3modal/discussions/2657

trying to feature -> Coinbase
featuredWalletIds: [
  'fd20dc426fb37566d803205b19bbc1d4096b248ac04548e3cfb6b3a38bd033aa'
]}

please update to the latest version so solve featuredWalletIds & excludeWalletIds issues.


Remove 4 HHS Domains

🗣 Description
This pull request removes four HHS domains so that they no longer appear on BOD 18-01 reports.
💭 Motivation and context
All of these domains have been removed from HHS ownership and are available for sale/registration.
🧪 Testing
All linters pass.
✅ Pre-approval checklist

[x] This PR has an informative and human-readable title.
[x] Changes are limited to a single goal - eschew scope creep!
[x] All relevant type-of-change labels have been added.
[x] I have read the CONTRIBUTING document.
[x] These code changes follow cisagov code standards.
[x] All new and existing tests pass.

@cfx47 - Please fill out the PR template (at least the "Description" and "Motivation" sections) as opposed to just putting freeform text at the top. Also please note that I have removed the checkboxes that do not apply to this PR; you can do that as well with future PRs.

@cfx47 (and @climber-girl, for visibility) - I modified the PR description for you.  Please take note and do it this way in future PRs.  Can you spread the word among your cohort as well?


Fix ElasticsearchDataSource.php data source

I don't think the data source should be the client since it's quite pointless to provide to filter condition

@Creeperface01 Could you rebase and point the branch to next branch, please?

Also, could you bump elasticsearch dependency as composer offer new version.
elasticsearch/elasticsearch v7.17.1 v8.6.0 PHP Client for Elasticsearch

@Creeperface01 Thanks for contribution 👍


Prisma Query Engine GraphQL error: 'PANIC: Decimal is not a f64'

Hi Prisma Team! My Prisma Client just crashed when the number size is bigfloat. This is the report:
Defined some properties like below, Able to save some number like '1000000000000000000000000000.0000' but while retrieving getting parse error and client is getting crashed with 'PANIC: Decimal is not a f64'
{
amount_incl_tax Float?
amount_excl_tax Float?
}
Versions

Name
Version

Node
v14.8.0

OS
linux-musl

Prisma
2.5.0

Logs
2BPrisma%25252BClient%25252Bcall%2525253A%2525250Aprisma-client%25252Bprisma.executeRaw%25252528UPDATE%25252Bpublic.%25252522CntrContract%25252522%25252B%25252BSET%25252Bis_latest%2525253DFALSE%2525252C%25252Bis_effective%2525253DFALSE%25252BWHERE%25252Bref%2525253D%25252527C_20-21_00014%25252527%25252529%2525250Aprisma-client%25252BPrisma%25252BClient%25252Bcall%2525253A%2525250Aprisma-client%25252Bprisma.executeRaw%25252528UPDATE%25252Bpublic.%25252522CntrContract%25252522%25252B%25252BSET%25252Bstatus%2525253D%25252527Approved%25252527%2525252C%25252Bis_latest%2525253DTRUE%2525252C%25252Bis_effective%2525253Dis_active%25252BWHERE%25252Bid%2525253D%25252527ckf5c3e4d53120sodh3ewtwpd%25252527%25252529%2525250Aprisma-client%25252BPrisma%25252BClient%25252Bcall%2525253A%2525250Aprisma-client%25252Bprisma.executeRaw%25252528UPDATE%25252Bpublic.%25252522CntrContract%25252522%25252B%25252BSET%25252Bis_latest%2525253DFALSE%2525252C%25252Bis_effective%2525253DFALSE%25252BWHERE%25252Bref%2525253D%25252527C_20-21_00015%25252527%25252529%2525250Aprisma-client%25252BPrisma%25252BClient%25252Bcall%2525253A%2525250Aprisma-client%25252Bprisma.executeRaw%25252528UPDATE%25252Bpublic.%25252522CntrContract%25252522%25252B%25252BSET%25252Bstatus%2525253D%25252527Approved%25252527%2525252C%25252Bis_latest%2525253DTRUE%2525252C%25252Bis_effective%2525253Dis_active%25252BWHERE%25252Bid%2525253D%25252527ckf5f0ypk100590sod35q7e9tu%25252527%25252529%2525250Aprisma-client%25252BPrisma%25252BClient%25252Bcall%2525253A%2525250Aprisma-client%25252Bprisma.executeRaw%25252528UPDATE%25252Bpublic.%25252522CntrContract%25252522%25252B%25252BSET%25252Bis_latest%2525253DFALSE%2525252C%25252Bis_effective%2525253DFALSE%25252BWHERE%25252Bref%2525253D%25252527C_20-21_00016%25252527%25252529%2525250Aprisma-client%25252BPrisma%25252BClient%25252Bcall%2525253A%2525250Aprisma-client%25252Bprisma.executeRaw%25252528UPDATE%25252Bpublic.%25252522CntrContract%25252522%25252B%25252BSET%25252Bstatus%2525253D%25252527Approved%25252527%2525252C%25252Bis_latest%2525253DTRUE%2525252C%25252Bis_effective%2525253Dis_active%25252BWHERE%25252Bid%2525253D%25252527ckf6o7ia4146980sod9muyu8ce%25252527%25252529%2525250A%25252560%25252560%25252560%252526title%25253DPANIC%2525253A%25252BDecimal%25252Bis%25252Bnot%25252Ba%25252Bf64.%252526template%25253Dbug_report.md%25250A%25250AIf%252Byou%252Bwant%252Bthe%252BPrisma%252Bteam%252Bto%252Blook%252Binto%252Bit%25252C%252Bplease%252Bopen%252Bthe%252Blink%252Babove%252B%2525F0%25259F%252599%25258F%25250A%25250A%252B%252B%252B%252Bat%252BNodeEngine.handleRequestError%252B%252528%25252Fapp%25252Fnode_modules%25252F%252540prisma%25252Fclient%25252Fruntime%25252Findex.js%25253A1%25253A123522%252529%25250A%252B%252B%252B%252Bat%252B%25252Fapp%25252Fnode_modules%25252F%252540prisma%25252Fclient%25252Fruntime%25252Findex.js%25253A1%25253A138781%25250A%252B%252B%252B%252Bat%252BrunMicrotasks%252B%252528%25253Canonymous%25253E%252529%25250A%252B%252B%252B%252Bat%252BprocessTicksAndRejections%252B%252528internal%25252Fprocess%25252Ftask_queues.js%25253A93%25253A5%252529%25250AplusX%252BExecution%252Bpermissions%252Bof%252

Hello! Can you please provide a minimal reproduction for the above crash? Unfortunately the automatically collected information in the issue does not give us much to go with here - we will need a bit more information. What were you doing? Can you share the code and schema that was active?

Hello! Can you please provide a minimal reproduction for the above crash? Unfortunately the automatically collected information in the issue does not give us much to go with here - we will need a bit more information. What were you doing? Can you share the code and schema that was active?

Please find the updated details, let us know if more details are required.

@ru4sam326 Sorry for the late reply here
I am unable to reproduce this in latest version 2.13.1
I remember we refactored the logic for that part a bit so this might be already fixed. Please try again with the latest version.

@ru4sam326 Sorry for the late reply here
I am unable to reproduce this in latest version 2.13.1
I remember we refactored the logic for that part a bit so this might be already fixed. Please try again with the latest version.

I assume this is fixed now. If that is not the case, please let us know @ru4sam326 and we will reopen this.

I assume this is fixed now. If that is not the case, please let us know @ru4sam326 and we will reopen this.


[WIP] Support RPC v0.4.0

Closes #1098
Introduced changes

added estimate_message_fee method in FullNodeClient
added missing field contract_address_salt in DeployTransaction

[ ] This PR contains breaking changes

Closes #1097


pre-populate mesh size

As a fisherman I would like the app to automatically pre-populate mesh size (in the FISH-1 form) https://github.com/StAResComp/sifids/issues/6, https://github.com/StAResComp/sifids/issues/8

Must


FlexSerial_Transmit.ino hangs T4.1

No compiling issues, it simply hangs. The example sketch was unedited and no hardware was attached to pin 2 on my T4.1

Thanks,
Should be fixed now.
Try syncing up and see

Any luck?  I just pushed up more changes here that might help.

Sorry, I lost hope for this and found a work around. I think I wanted to use this lib to interface with an FRSky pitot device, but after having trouble with it, I decided to rip off the pressure sensor and interface with it directly - no need for serial comms anymore. Thanks though!


Service ID + request params undefined

Request frames indicate the payload data should contain a service ID and request parameters. The encoding of these two things are undefined. I suspect in fact these two things are part of the application protocol running on top of Reactive Socket and they should not even be mentioned in the Reactive Socket protocol.

Correct. Although, I think there is a default Schema.md file that talks about a minimal set of payload for requests.

In issue #22 we decided that service ID and parameters would be part of the metadata, but looks like the spec did not get updated with that decision. The encoding of both data and metadata are specified in the setup frame via MIME types.
I think at this point Reactive Socket is more of an application protocol as it defines more of application semantics (requests, leases, etc.) than transport semantics, so personally I don't see a problem specifying these as they anyways be required by any serious application (running more than one endpoint on a server)

The specification should then provide enough information that an implementer can accept a service ID and parameters (in what form? a map of string -> string?) from a caller and know how to encode the metadata for specifically supported MIME types. As well as how that will work when needing a service ID and parameters to be mixed in with other random application metadata.

The protocol does not address serviceID/parameters data. It is completely up to the application, and the application determines what the client/server agrees upon using the SETUP frame.


Nvidia Video SDK install on ArchLinux places headers outside cuda install

nvidia-sdk on arch linux's AUR installs the headers to /usr/include/nvidia-sdk. However the cuda is installed to /opt/cuda.
Is it possible we could get another variable for the nvidia video SDK considering it is a separate install?

Yes. I will be able to work on this in a few days. If you would like this to be done sooner feel free to make a pull request and I will happily review it.

I'm not in any rush, I just copied the headers for now.

@FallingSnow
Let me know if the master branch is working for you now. https://github.com/ViliamVadocz/nvidia-video-codec-sdk/commit/a01c07cdbb16e7571beb85c98cd63792d9dec01b added /usr/include/nvidia-sdk to the candidate paths and the build script is no longer assumes the same installation location as CUDA.
If you report back that it is working, I will publish the new version as 0.1.2.

I had to use this on windows as that was the default install path. $env:CUDA_ROOT="C:/Program Files/NVIDIA GPU Computing Toolkit/Cuda/v12.2"
Let me test the linux change you just made.

Linux changes fixed the issue. Thank you!
Would you like me to open another issue about the Windows install location?

Would you like me to open another issue about the Windows install location?

You may. I think I need more information about what errors you are getting because right now I am somewhat confused. Specifically, you said you set CUDA_PATH which makes me think this is a cudarc issue.

I'm unsure if it's a cudarc issue but the build couldn't find the headers on a fresh cuda install. I figured this was because build.rs looks at C:/Program Files/NVIDIA GPU Computing Toolkit instead of C:/Program Files/NVIDIA GPU Computing Toolkit/Cuda/v12.2.

@FallingSnow Yes that's correct, but that is a cudarc issue, since this library does not actually look for CUDA anymore (cudarc, which is a dependency, needs it though).

Ah, after rereading build.rs I see what you're saying. You only look for Video SDK headers. Yeah, this must be a cudarc issue then. Thanks for pointing that out.


External state management using FASTER

The present (default) implementation of Trill has all memory management done by .Net and in-memory. We are in the process of making implementations of Trill operators that use the FASTER key-value store engine for state management instead. Doing so will allow the option of having operator state spanning both memory and disk, and would allow for potentially faster checkpointing and recovery.
Should the FASTER version of operators prove to have similar or better performance than their previous versions, we will remove the previous versions in favor of the new ones to reduce code duplication.
Repo link for FASTER:
https://github.com/microsoft/faster

@cybertyche @peterfreiling Any updates about TRILL+FASTER, please?

Hi, @cybertyche @peterfreiling I would like to ask if you are working on a Faster Trill implementation? Thank you.

@jurby we currently do not have work planned for this integration.


tls: remove legacy socket BIOs and runtime guard

Signed-off-by: Florin Coras<EMAIL_ADDRESS>Fixes #16013
Risk Level: Low
Testing: n/a
Docs Changes: n/a
Release Notes: removed envoy.reloadable_features.tls_use_io_handle_bio runtime guard

Merged main so conflict should be solved now. As for the test, this should not change default logic, so could it be a flake?

Merged main so conflict should be solved now. As for the test, this should not change default logic, so could it be a flake?

Yeah, could be a flake. Let's see what happens with this CI run.


Consolidate and disarm the defaults in the DB dump scripts

Rather than have two scripts, we now have one which takes parameters for the host, port, user, and database and runs pg_dump from that.
Also, this script now has defaults that dump the prod copy that we've been calling spi-prod-db-1-dump so if it's run without the correct env variables, it can't ever take prod down while we resolve the other issue.

One thing to note here @finestructure: I'm not sure if the updated script runs with fish, I know it works differently with variables.

It’s shebanged to bash so that’s not an issue!


Upgrade mypy to 0.501.

Zulip's current mypy has a caching bug that causes mypy to randomly fail in the dev environment for many zulip developers. The new mypy (released last Wednesday) fixes this!
To upgrade:

Replace the contents of requirements/mypy.txt with

six==1.10.0
mypy==0.501
typed-ast==1.0.1

Run tools/provision
Run tools/mypy

You should get a bunch of errors for things that mypy didn't use to check for and now does, like not importing Dict and List.

Fix the errors!

http://zulip.readthedocs.io/en/latest/mypy.html has some information about Zulip's use of mypy, but all the mypy changes here should be pretty straightforward if you've been developing for Zulip for a bit. Also don't worry if you're only able to fix 90% of the errors; someone can probably fix the rest!

@zulipbot claim
Let's see if i could fix the errors!

Merged #3908 which resolved this.  Thanks @rishig and @jajodiaraghav!!


[WIP] Python examples

Following the discussion in #48, I've started on the examples.
I've taken the approach of:

Self-contained, fully explicit, examples that you'd be able to copy-paste into a console. This does mean there's some duplication of coordinates, and index arrays.
Minimal metadata in the attributes to clarify which ones are essential, and to clarify the roles of the different entries.

Some things I haven't quite figured out yet:

How do you validate whether a file is up to spec? I've currently resorted to seeing whether I could open the files in the dev version of QGIS (3.13) -- earlier versions don't support 1D topology yet. That's only one implementation of the spec, of course...
On that topic: the 1D mesh works fine in QGIS 3.13. However, I might be doing something wrong with the 2D, it doesn't seem recognize the edges at all (and the data on put on them).
Does gridded support 1D topology? (The current example doesn't seem to work)
gridded always seems to write latitude and longitude in the attributes. My use case would be almost exclusively projected cartesian coordinates -- so maybe a gridded issue.
I haven't started with the 3D layered mesh yet: the spec isn't entirely clear to me here (because I can't really validate the file I've written), and I'm generally unable to find existing 3D layered files to use for inspiration.
The examples currently exist with a markdown box with a Python directive, which doesn't test if they actually work. Using Python reStructuredText, it's pretty easy to include a few .py files via literalinclude. I don't believe Markdown supports that.
Ideally, examples are tested -- we could get setup some CI? It's running Python + conda anyway.
The math markdown extension breaks the build, both locally for me and on CI. It's used only to make the tables look a bit better, I believe?

Thanks for kicking this off. A few comments on your notes:

How do you validate whether a file is up to spec? I've currently resorted to seeing whether I could open the files in the dev version of QGIS (3.13) -- earlier versions don't support 1D topology yet. That's only one implementation of the spec, of course...

It would be nice to have a validator -- the early version(s) of py_ugrid (now part of gridded) kinda-sorta acted like a validator, but it was not really designed for that. And ideally, you would get nice messages for invalid files. I do want to refactor gridded's loading code, so that it would be a two step process: 1) identify all the "parts", and then 2) actually load the variables. This would let step (1) act as a validator, at least to some extent.
but ideally, a validator is more than (and less that) simply a tool that can load the files

On that topic: the 1D mesh works fine in QGIS 3.13. However, I might be doing something wrong with the 2D, it doesn't seem recognize the edges at all (and the data on put on them).

I'm curious what QGIS is supposed to do with an unstructured mesh anyway -- it doesn't fit the GIS data models -- though maybe make a TIN?

Does gridded support 1D topology? (The current example doesn't seem to work)

No, not at all. I'm not sure if it should -- 1D "meshes" are a pretty different concept. And if it should, how? Though the included UGRID object could, I suppose.

gridded always seems to write latitude and longitude in the attributes. My use case would be almost exclusively projected cartesian coordinates -- so maybe a gridded issue.

yes, that is a gridded issue. that one we should probably address. I'm pretty sure that CF only recently allowed projected coordinates wihtout lat-lon as the "primary" one. And we were shooting for CF compliance. But that would be a nice feature -- Issues (and especially PRs) excepted.

I haven't started with the 3D layered mesh yet: the spec isn't entirely clear to me here (because I can't really validate the file I've written), and I'm generally unable to find existing 3D layered files to use for inspiration.

Maybe results from an FVCOM model?

The examples currently exist with a markdown box with a Python directive, which doesn't test if they actually work. Using Python reStructuredText, it's pretty easy to include a few .py files via literalinclude. I don't believe Markdown supports that.

no idea, here, sorry. Though I'd be haooy if someone wanted to port to Sphinx -- not sure how others feel about that though. Does this not work?
https://developer.lightbend.com/docs/paradox/current/directives/includes.html

Ideally, examples are tested -- we could get setup some CI? It's running Python + conda anyway.

Sure, I'd support that.

The math markdown extension breaks the build, both locally for me and on CI. It's used only to make the tables look a bit better, I believe?

no clue here from me, sorry

There's been some nice developments in QGIS regarding unstructured meshes:
https://github.com/lutraconsulting/qgis-crayfish-plugin
https://github.com/lutraconsulting/MDAL
The C++ part of the Crayfish plugin has been moved to QGIS (since 3.x).
I guess with regards to validation: yeah, the proof of the pudding is in the eating (the loading). I guess specifications become "reified" as different implementations of the spec work out the differences -- I think I'll raise an issue in the MDAL repo. A (Python-based) validator would be useful, but wouldn't be used in a C++ binary like e.g. MDAL anyway.
Not having 1D topology in gridded makes sense to me.
I'll open an issue about reStructuredText versus Markdown.


How to read MP3 properties with NAudio

Hi, sorry for the question as an issue, but the docs aren't helpful and neither was google.
http://naudio.codeplex.com/wikipage?title=Create an ID3v2 Tag shows how to create an ID3v2 tag, which is all fine, but given an Mp3 file, how do I get the artist or title or etc. Do I really have to do ID3v2.Read and decode the raw bytes myself?
I'm new to NAudio so I might be missing something simple, essentially what I want to do is
Mp3FileReader reader = new Mp3FileReader();
 reader.Title;
 reader.Artist

or
Id3v2Tag.ReadTag(stream).Title;

NAudio's support for ID3 tags is very basic. There are other NuGet packages (e.g. TagLibSharp that can do a better job

Yeah, I ended up going with TagLibSharp, I was hoping to avoid more third party dependencies though. Thanks for the response.


lint hacking

Lint cleanup

Looks good, agree w/ @Racer159 about the periods, another inconsequential note would be whether Zarf is capitalized in comments.

Just pushed a fix to add the periods and capitalize Zarf.


NSLocalizedString isn't replaced after injection

I was wondering if this is possible to refresh all NSLocalizedStrings after injection? Now, when I inject the code and re-render the view in "injected" method, instead of proper translated labels I have key strings.
What do you recon?

Not sure why that would be or how it could be automated. Any ideas?

I bet the NSLocalizedString  macro looks inside the current bundle for the key value lookup, then when injected it looks up inside the injection bundle for the key value, which fails


Adding todos

todo API endpoint exists for the client now and the page can be accessed

there are no tests yet


setting cni network plugins dynamically

fix #110
if we want to use https://github.com/kubernetes/kubernetes/pull/43474 in dockershim in frakti, kubernetes vendor must be updated. @feiskyer

This change is

@feiskyer Will we wait after kubernetes 1.6 final release to merge this, or we are OK since we already released 0.1 for k8s?

@resouer Yep, we could continue fixing our bugs.
@heartlock There is also a check in NewAlternativeRuntimeService(), please fix togather with vendor.

ready for review @resouer @feiskyer

test fail related with https://github.com/hyperhq/hyperd/pull/559, I'll take a look at this.

@heartlock There is also a check in NewAlternativeRuntimeService(), please fix togather with vendor.

@heartlock NewAlternativeRuntimeService seems not updated.

check make in dockershim, already updated in vendor. @feiskyer

updated it @feiskyer

@heartlock LGTM. Thanks.
/lgtm


Fix Issue with PGP Keys

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Adam Fraser seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.