Skip processing handler if this is not a valid shortcode

I was getting some very strange content corruption when I enabled my shortcode plugin that uses your fantastic little shortcode library. This was caused by my having some square bracketed content that did not match up with any of my defined shortcode handlers:
##### html([title][, alt][, classes])

!! In Markdown this method is implicitly called when using the `![]` syntax.

The `html` action will output a valid HTML tag for the media based on the current display mode.

This [title], [, alt] and [, classes] text is used in my documentation to display optional method params.  However, it was corrupting that title to something like:

html([t[title] alt][, classes])

I debugged this in the Shortcode library and discovered that you were first collecting all the potential shortcodes in the entire page (quite a lot for me), and then iterating over them.  However, when you retrieved the handler for every shortcode, it would continue to try to process the shortcode even if the handler was null.  This ultimately leads to all kinds of corruption of the end content.
Simply checking to see if this handler is null, and then continuing without doing any more processing ensures there is no corruption, and also speeds up the page parsing considerably.
FYI - I created a core shortcode plugin for Grav CMS, and a subsequent shortcode plugin that adds some UI specific shortcodes in case you are interested.

https://getgrav.org
https://github.com/getgrav/grav-plugin-shortcode-core
https://github.com/getgrav/grav-plugin-shortcode-ui

Weird it didn't pass travis checks, looks valid to me :)
Anyway, something else is going on. Even with valid shortcodes that work in other pages, it's not working in this one page. so will investigate further.

Ok, have narrowed down the corruption problem with valid shortcodes to this in my content before any of the shortcodes:
`![](image.jpg)`

I'm guessing the [] in the markdown URL is causing the issue.  Not had chance to dig in deeper to see where that is tripping things up.

@rhukster Hi, thanks for reporting the issue! I'd really like to help you, but first let me clarify some points you made here:

first of all, shortcode name can be only a non-empty string with alphanumeric characters and dash -, so there will be no matched shortcodes in ![](image.jpg) and only one in html([title][, alt][, classes]) (the [title]), the rest of the bracketed parts (both [, alt] and [, classes]) you mentioned have comma as the name so it won't be even matched,
secondly, the PR you submitted skips processing shortcodes if no handler is present right in the heart of Processor::processIteration() method, but if you look closely there is a method Processor::processHandler() called just under that line which does exactly that (look at the ternary operator at the end). It does not "skip" processing the shortcode but if there is no handler it just returns its raw string match. So if there is no handler for [title], the result of processing it will be [title]. There is one exception though - if you set up the default handler using HandlerContainer::setDefault() then it will be processed with to the callable you passed there,
to add more to the previous point, this is because shortcode without handler may contain other shortcodes inside its content that do have handlers and such case needs to be processed correctly (you can of course configure Processor to disable recursion, but the point remains valid),
as for performance penalty for parsing all shortcodes it is recommended to cache the processing result and reprocess only when saving edited version of the article,
while writing this answer I found a bug in RegularParser that allowed arbitrary shortcode names (no validation for the rules mentioned above), it was fixed in master in https://github.com/thunderer/Shortcode/commit/a22cd63eb8b9642465b87c0df386e57131c73114 . RegexParser is not affected, though I really recommend using RegularParser.

In order to fully debug this, I'd need the text that is causing problems for you and which parser you used (I saw RegexParser in the plugin code, is that it?). I'd check first for what is returned from the parser and if that is correct I'd then check if the replacements are applied correctly. Could you please prepare a gist and tell me what did you get and what did you expect? Thanks in advance.

I didn't take into account all the various nested situations that may arise, so my fix was probably a bit hasty.  Also I switched to the RegularParser and all my corruption problems went away.
I definitely feel like my markdown image code sample: ![](image.jpg) was triggering an opening empty shortcode. Surely an empty bracket pair [] is not a valid shortcode?  maybe the RegexParser is treating it as such?  Anyway i'll close this PR as it seems it is not valid.  Cheers!

@rhukster I'm glad that everything is correct now. I've tagged release 0.5.2 with the fix mentioned above so you can go back to using thunderer/shortcode in composer.json.
I'd really want to take a look at the problem you had with RegexParser though. I added invalid test cases extracted from the data you provided and since tests passed I'm pretty sure that none of the parsers included in this library recognize shortcodes in mentioned invalid sequences ([] and [, alt]). Also from the whole ParserTest class you see that I made sure that no invalid string can get inside. :) Could you please send me a gist with the problematic text and sample Shortcode implementation?
All in all I'm very happy that you use Shortcode in GravCMS! If you have any ideas how to improve the library or any other problems with using it, let me know! I'm now working in my spare time on the better documentation, there is also events subsystem and bundled handlers waiting in line, you can see the code in the other branches in this repository.

I actually have a test setup for you.  I think you will find it interesting.. give me a few to package it up.

Ok, so please clone https://github.com/rhukster/shortcode-test into your webroot somewhere and composer install.
I have created a simple test to replicate the issues as closely as possible.
Basically I am loading a markdown file, using Parsedown to process it and get the resulting HTML.  Then i'm running it through Shortcode with a couple of simple shortcode handlers.

As is, it's using the RegularParser and a small version of the file.  All looks good, although it's very slow taking 224ms even on this small chunk of content.

If you change this to the RegexParser, the corruption occurs, although things are much faster, taking only 3.2ms:

Now switching the content file to examle-full.md on line 16, and rerunning, it is of course still corrupted, but processing is still only 38ms, slower than I would prefer but acceptable as Grav does of course cache the contents of these files.

Now changing back to the RegularParser, the page fails to load because it takes > 30 seconds to process.   If I turn off the time limit, it was still spinning after 15 minutes!

Grav is a flat-file CMS where speed is paramount.  A typical Grav site renders every page within 20ms or so after cache, and certainly < 100ms without cache.  So we really need the speed of the RegexParser without the corruption issues :)
Hopefully this test repo will give you a good largish-real world example to do some performance optimizations.
Cheers!

Oh one last thing.. I noticed this in PHP 5.6 and 5.7, where as PHP 5.5 doesn't seem to cause the problems with the RegexParser

Arrggg.. sorry for the rambling but i think i found the culprit. I had this text:
## Where to put your media files

Usually you’ll use a media file within a page...

Looks harmless but I noticed that in PHP 5.5 the apostrophe showed as a special character in my test setup.  But looked like a regular quote in PHP 5.6 and 7.0.  So this was pasted in as an extended UTF-8 character.
This appears to be throwing the regex out of whack (I tried WordpressParser and RegexParser with same results).
Changing this to a regular ' stops the corruption in all versions of PHP.

After adding an mb_internal_encoding("UTF-8"); in the test index.php i can get the same results under PHP5.5 -> 7.0 as PHP 5.6 encoding was set to UTF-8 by default.

@rhukster Thanks for the test repository, it's great! I already made some changes and shaved off several milliseconds. :) I'm happy that you resolved the issue with your Markdown code. Internal encoding is a great clue because I use mb_*() functions a lot, possibly forcing the encoding to UTF-8 inside parsers and processor will solve such problems. This could also solve the issue #25 where @michaloo had to change multibyte functions to regular ones because he suspected that it may be corrupting his output. I'll also add some tests with various UTF-8 sequences to be sure that everything is okay.
BTW Could you tell me what your default mb_internal_encoding() result is? Something different than UTF-8? What is the environment you're working in (OS, HTTP server, PHP version, FPM or mod_php)?

I know i didn't make it very clear, but let me recount the situation:
PHP 5.5 - Default internal encoding is ISO-8859-1, the UTF-8 character displayed strangely as a garbled font character, but the shortcodes rendered OK.
PHP 5.6+ - Default internal encoding is already UTF-8, the UTF-8 character displays correctly , but the shorcodes are corrupted.
I forced the mb_internal_encoding to UTF-8 in the test so PHP 5.5 would act the same as 5.6+
This is also set this way in Grav for consistency.
Regarding platform.  I'm running on Mac OS X 10.11.3.  PHP 5.5, 5.6 and 7.0.  I am able to run Apache, Nginx, Caddy server in parallel (different ports), with mod_php, and php-fpm, all exhibit the same behavior now irrespective of the PHP version or web server.

BTW, in summary there are two issues currently outstanding:

Regex and WordPress parsers are both effected by UTF-8 characters causing corruption in page content when used in a PHP environment with UTF-8 encoding.

RegularParser can easily crash PHP when used on real-world content.  Your testing only provides tiny snippets of shortcodes and does not show how crippling slow and processor-intensive this parser is.  It should definitely NOT be used for real-world situations until sorted.  Regex and WordPress parsers are the only ones that you can safely use beyond a few lines of text.

Should I open up issues for each of these?

@rhukster thanks for the clarification. I think I understand what is going on.
I spent some time tinkering on the "small" example from your repo and Blackfire reported 54% improvement in RegularParser, you can see the changes on the new branch. I could make a PR right now, but I want to analyze it a bit more as I have a couple of ideas to try.
Unfortunately "full" example is still unable to complete, but I know where the problem is: there are many unclosed shortcodes like [media-name]. RegularParser was built first and foremost for parsing correctness (that is impossible with any regex-based parser) so for each opening tag it searches till the end of the text for the closing tag, matching every nested shortcode on the way. If it reaches the end without finding closing tag, it discards everything, marks that shortcode as "just closed" and starts over from the next opening tag. If those shortcodes would be "self-closed" like [media-name /] then they would be consumed by parser in no time. I could "dodge" that problem by automatically merging all found shortcodes on the second level without reparsing the rest of the text, but then it would create exact same problem for the third nesting level and so on. Just like in programming language there is no way of telling that code will have a syntax error without parsing it up to the place with that error. If I'll find any better approach, you'll be first to know.
I didn't have enough time to check the corruption problem in RegexParser yet, but I'll work on it as soon as possible. Opening separate issues for these problems is a good idea, especially given the excellent input you've provided here. Discussing isolated issues is much better than commenting on everything on closed PR. :)

ok will do.. btw i have some ideas that would improve flexibility too.. i'll open up an issue to discuss that too.

Probably already saw but I added some ideas to the ideas issue: https://github.com/thunderer/Shortcode/issues/16
I think at this point we can communicate on those 3 open issue, and this one can remained closed, and RIP :)


Add optional ulimit for erchef

Add a chef-server.rb setting opscode_erchef['memory_maxbytes'] which,
when set to a non-nil value, adds -d $BYTES to the chpst command
which sets the process environment for erchef. This setting limits the
data memory segment for the erchef process to the specified value, which
caps the amount of RAM erchef may use.
This setting is intended to be used for standalone topologies, such as
when Chef Automate and Chef Server are colocated on the same machine. In
these topologies, heavy search load can cause erchef to consume large
amounts of memory which then triggers memory related failures in other
components, such as Elasticsearch. The memory_maxbytes setting allows
the administrator to make erchef fail instead of the other components in
these overload situations, allowing the system to recover more rapidly.
This setting is inteded to act as an emergency "circuit breaker" for
excessive load situations and should be set at the highest value at
which it can be effective, i.e., the value at which the host system's
memory would be completely utilized. Future load testing efforts are
required to provide guidance around recommended values.
Signed-off-by: Daniel DeLeo<EMAIL_ADDRESS>

I'm OK with this since it is an opt-in feature; but I'm worried about how heavy of a hammer this is and how unpredictably it seems like it may cause us to fail.
I can imagine this thing would be hard to test in an automated fashion, but have we tried doing something like artificially creating memory pressure by setting a low max value and then watching it fail.  If so, how did it fail?

I created a small OWCA instance (4GB RAM), created 600 node objects and then ran a load test of 100 simulated CCRs on a fairly tight interval (don't recall the exact params, but it was around 1m average interval), with all simulated runs doing a search for *:* on the nodes index. When enough of the simulated runs did their searches simultaneously, erchef hit a 1GB RAM limit I had set. The erlang VM's behavior in this case is to just exit without running any further erlang code. There are some mailing list discussions about changing this or adding some other option but there doesn't seem to be much enthusiasm for that. Anyway, runit was able to restart erchef in less than 10s in my configuration each time. In none of the cases was erchef unable to restart. I would suspect that some node saves were inconsistent, e.g., data wasn't correctly saved to both postgres and search, but I didn't test for that.
In any event, though this is a big hammer, it's a smaller hammer than the OOM killer wields, and is intended to be tuned such that you only hit this ulimit just before the OOM killer would kick in and kill a process (like Elasticsearch) that has significantly worse MTTR (in my testing, I also saw Elasticsearch get a malloc failure, which has the same result because the JVM just exits in that case, so the outcome was the same in terms of visible behavior).

@markan sadly I did not keep that in my investigation notes

@markan got it off my demo box:
2017-06-01_23:04:51.34889 binary_alloc: Cannot reallocate 1143523 bytes of memory (of type "binary").

guess it changes depending on what specifically erchef is doing when it gets the malloc failure:
2017-06-01_23:13:12.72098 eheap_alloc: Cannot allocate 6664208 bytes of memory (of type "old_heap").


Move to a flag deviation

Pull Request Functional Test Report for #1768 / 45fa608beadbaa7d050e43137fc88807f613eb08
No tests identified for validation.
Help

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 47.739%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1341

Relevant Lines:
2809

💛  - Coveralls

Why is this being moved back to a flag?


New version: MosekTools v0.7.5

Registering package: MosekTools
Repository: https://github.com/JuliaOpt/MosekTools.jl
Created by: @blegat
Version: v0.7.5
Commit: 44ce6845cac40e468b8422b7d6679bd7839d18ee
Reviewed by: @blegat
Reference: https://github.com/JuliaOpt/MosekTools.jl/commit/44ce6845cac40e468b8422b7d6679bd7839d18ee#commitcomment-35667704

Your new version pull request does not meet all of the guidelines for automatic merging.
Specifically, your pull request does not meet the following guidelines:

Does not meet sequential version number guideline

If you want to prevent this pull request from being auto-merged, simply leave a comment.
(If you want to post a comment without blocking auto-merging, you must include the text [noblock] in your comment.)

It is important to note that if your pull request does not meet the guidelines for automatic merging, this does not mean that your pull request will never be merged. It just means that your pull request will require manual review by a human.

These guidelines are intended not as requirements for packages but as very conservative guidelines, which, if your new package or new version of a package meets them, it may be automatically merged.

On a separate note, I see that you are registering a release with a version number of the form v0.X.Y.
Does your package have a stable public API? If so, then it's time for you to register version v1.0.0 of your package. (This is not a requirement. It's just a recommendation.)
If your package does not yet have a stable public API, then of course you are not yet ready to release version v1.0.0.

[noblock]

Does not meet sequential version number guideline

This is expected since the v0.7.x versions are for Mosek v8 while the latest releases are for Mosek v9.

Yea, this should be allowed.


five_mem_node example panics

Describe the bug
Running the five_mem_node produces the following panic:
$ cargo run --example five_mem_node
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/examples/five_mem_node`
 INFO 2019-04-05T13:12:51Z: raft::raft: peer_1 became follower at term 0
 INFO 2019-04-05T13:12:51Z: raft::raft: peer_1 newRaft [peers: [(1, Progress { matched: 1, next_idx: 2, state: Probe, paused: false, pending_snapshot: 0, recent_active: false, ins: Inflights { start: 0, count: 0, buffer: [] } })], term: 0, commit: 1, applied: 1, last_index: 1, last_term: 0, pending_membership_change: None]
 INFO 2019-04-05T13:12:52Z: raft::raft: peer_1 is starting a new election at term 0
 INFO 2019-04-05T13:12:52Z: raft::raft: peer_1 became candidate at term 1
 INFO 2019-04-05T13:12:52Z: raft::raft: 1 received MsgRequestVote from 1 at term 1
 INFO 2019-04-05T13:12:52Z: raft::raft: peer_1 became leader at term 1
 INFO 2019-04-05T13:12:52Z: raft::raft:  became follower at term 0
 INFO 2019-04-05T13:12:52Z: raft::raft:  newRaft [peers: [], term: 0, commit: 0, applied: 0, last_index: 0, last_term: 0, pending_membership_change: None]
 INFO 2019-04-05T13:12:52Z: raft::raft:  [term: 0] received a MsgHeartbeat message with higher term from 1 [term: 1]
 INFO 2019-04-05T13:12:52Z: raft::raft:  became follower at term 1
 INFO 2019-04-05T13:12:52Z: raft::raft:  [commit: 0, lastindex: 0, lastterm: 0] fast-forwarded commit to snapshot [index: 1, term: 0]
thread '<unnamed>' panicked at ' to_commit 1 is out of range [last_index 0]', src/raft_log.rs:248:13
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:59
             at src/libstd/panicking.rs:200
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:215
   4: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:478
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:385
   6: std::panicking::begin_panic_fmt
             at src/libstd/panicking.rs:340
   7: <raft::raft_log::RaftLog<T>>::commit_to
             at ./<::std::macros::panic macros>:8
   8: <raft::raft::Raft<T>>::restore_raft
             at ./src/raft.rs:2011
   9: <raft::raft::Raft<T>>::restore
             at ./src/raft.rs:2065
  10: <raft::raft::Raft<T>>::handle_snapshot
             at ./src/raft.rs:1975
  11: <raft::raft::Raft<T>>::step_follower
             at ./src/raft.rs:1849
  12: <raft::raft::Raft<T>>::step
             at ./src/raft.rs:1207
  13: <raft::raw_node::RawNode<T>>::step
             at ./src/raw_node.rs:340
  14: five_mem_node::Node::step
             at examples/five_mem_node/main.rs:186
  15: five_mem_node::main::{{closure}}
             at examples/five_mem_node/main.rs:69
 WARN 2019-04-05T13:12:52Z: five_mem_node: send raft message to 2 fail, let Raft retry it
 WARN 2019-04-05T13:12:53Z: five_mem_node: send raft message to 2 fail, let Raft retry it
 WARN 2019-04-05T13:12:53Z: five_mem_node: send raft message to 2 fail, let Raft retry it
 WARN 2019-04-05T13:12:53Z: five_mem_node: send raft message to 2 fail, let Raft retry it
 WARN 2019-04-05T13:12:53Z: five_mem_node: send raft message to 2 fail, let Raft retry it

To Reproduce
Steps to reproduce the behavior:

Go to the project root
Run cargo run --example five_mem_node
The panic occurs

Expected behavior
The example does not panic.
System information

CPU architecture: i7-6700HQ (amd64)
Distribution and kernel version: custom built linux kernel
SELinux on?: no
Any other system details we should know?: using rustup nightly

oh
PTAL @Hoverbear

Oh, I think this is related to @hicqu 's changes recently around the initial dummy entry?

Fixed by #214


Order tickets act very weird

1- http://nimb.ws/KVx3JZ (weird amount of cash/share + erroneous order cost calculation)
2- http://nimb.ws/7qswgo (buying 1 share of donald trump is not supposed to have such a huge impact, is it?)

see this discussion on reddit regarding point 2:
https://www.reddit.com/r/Augur/comments/4924wo/something_is_wrong_with_the_way_shareprices_and/

Can you paste the original images here?  Neither is available any longer.  Re: point 2, hopefully this was clarified in the Reddit discussion.  We're moving to an order book system anyway, so this will be a moot point shortly.


SCMOD-11932: Added package locks

JIRA: https://portal.digitalsafe.net/browse/SCMOD-11932

A developer build has not yet been created for this branch. Click here to go ahead and create the build...

CI Build Link:
http://sou-jenkins2.swinfra.net:8080/job/CAFapi/job/CAFapi~opensuse-java11-images~SCMOD-11932~CI

@dermot-hardy Tested in the opensuse-tomcat-jre11-image and I can see this in the build log:
DOCKER>
DOCKER> ---> bc16cb326a4f
DOCKER> Step 23/36 : RUN zypper -n refresh &&     zypper -n update
DOCKER>
DOCKER> ---> Running in 6c3603613d41
DOCKER> Repository 'Non-OSS Repository' is up to date.
DOCKER> Repository 'Main Repository' is up to date.
DOCKER> Repository 'Main Update Repository' is up to date.
DOCKER> Repository 'Update Repository (Non-Oss)' is up to date.
DOCKER> All repositories have been refreshed.
DOCKER> Loading repository data...
DOCKER> Reading installed packages...
DOCKER> The following item is locked and will not be changed by any action:
 Installed:
  java-11-openjdk

Nothing to do.```


write() to a filename w/o image extension triggers a mimetype error

ex.
image.write( "tempfile" )
[Error: Unsupported MIME type: application/octet-stream]

How would you expect it to behave? Write .png, which is lossless, by default?
Without an extension it doesn't know which output image format to write to.

@arminrosu is right. You need to state what kind of image you write to write: PNG, JPEG, BMP.
Right now, the file extension is used to determine what kind you want to write. The method could be extended to allow an explicit MIME type, like this:
image.write( "tempfile", Jimp.MIME_PNG);

Would you want that?

My first thought was that without an extension, it would write using the source image's mime-type.
Either @arminrosu idea of extending the write method to image.write( "tempfile", Jimp.MIME_PNG);  or adding an optional mime() method would work.
lenna.resize(256, 256)
   .quality(60)
   .greyscale()
   .mime(Jimp.MIME_PNG)                 // set optional mime-type
   .write("lenna-small");

Thinking about it a bit more, maybe the behaviors should be:

Write to file based on mime-type set with mime()
if mime-type is not set, write to file based on file extension
if mime-type and file extension is not set, default to source file mime-type

hmm...

explicit
implicit
somewhat ambiguous

ya, maybe #3 is not needed, your call. The ability to optionally set the output mime solve for most cases.
By the way, thumbs up on Jimp and thanks for putting it out there.

Sorry for not getting back sooner. I'm planning on doing a release later in the week. I'll add this issue to the list of issues to be addressed in the release after that.

the work around is to just add ".png" to the file path.

It is a shame to have to add an extension so that the program understands the format that one wants to use.
When storing files on the server, we do not necessarily use extensions because we can do a processing before delivering the file to the client.

how to get the output var name from Jimp
Jimp.read(array).then(function (lenna) {
lenna.resize(256, 256)            // resize
.quality(60)                 // set JPEG quality
.greyscale()                 // set greyscale
.write("lenna-small"); // save
})


[WIP] Samsung TV integration using UPNP

Breaking Change:
nothing
Description:
This is a new integration for Samsung TV devices using UPNP.
The URL needed for the configuration could be read with upnp-client --pprint search --service_type urn:samsung.com:service:MainTVAgent2:1.
My device has the following actions with current implementation state:

[ ] AddSchedule, not needed for Home Assistant (automations?)
[ ] ChangeSchedule, not needed for Home Assistant (automations?)
[ ] CheckPIN, not needed for Home Assistant
[ ] DeleteChannelList, not needed for Home Assistant
[ ] DeleteChannelListPIN, not needed for Home Assistant
[ ] DeleteRecordedItem, not needed for Home Assistant (automations?)
[ ] DeleteSchedule, not needed for Home Assistant (automations?)
[ ] EditChannelNumber, not needed for Home Assistant
[ ] EditSourceName, not needed for Home Assistant
[ ] EnforceAKE, not supported by my device
[ ] GetAllProgramInformationURL
[ ] GetAvailableActions, not needed for Home Assistant
[ ] GetAVOffStatus, not supported by my device
[ ] GetBannerInformation, not supported by my device
[ ] GetBDVideoID, not supported by my device
[ ] GetChannelListURL
[ ] GetChannelLockInformation
[ ] GetCurrentBrowserMode
[ ] GetCurrentBrowserURL
[x] GetCurrentContentRecognition, read media_title and media_channel
[x] GetCurrentExternalSource, also available with GetSourceList
[ ] GetCurrentMainTVChannel
[ ] GetCurrentProgramInformationURL
[ ] GetDetailChannelInformation
[ ] GetDetailProgramInformation
[ ] GetDTVInformation
[ ] GetHTSAllSpeakerDistance, not supported by my device
[ ] GetHTSAllSpeakerLevel, not supported by my device
[ ] GetHTSSoundEffect, not supported by my device
[ ] GetHTSSpeakerConfig, not supported by my device
[ ] GetMBRDeviceList
[ ] GetMBRDongleStatus
[ ] GetRecordChannel
[ ] GetRegionalVariantList
[ ] GetScheduleListURL
[x] GetSourceList, get all available and connected sources
[ ] ModifyChannelName
[ ] ModifyFavoriteChannel
[ ] PlayRecordedItem
[ ] ReorderSatelliteChannel, not supported by my device
[ ] RunBrowser
[ ] SendBrowserCommand
[ ] SendMBRIRKey
[ ] SetAntennaMode
[ ] SetAVOff, not supported by my device
[ ] SetChannelLock, not needed for Home Assistant
[ ] SetHTSAllSpeakerDistance, not supported by my device
[ ] SetHTSAllSpeakerLevel, not supported by my device
[ ] SetHTSSoundEffect, not supported by my device
[ ] SetMainTVChannel
[ ] SetMainTVChannelPIN
[x] SetMainTVSource, change source
[ ] SetRecordDuration
[ ] SetRegionalVariant
[ ] StartCloneView
[ ] StartExtSourceView
[ ] StartInstantRecording
[ ] StartSecondTVView
[ ] StopBrowser
[ ] StopRecord
[ ] StopView

Related issue (if applicable): fixes #
Pull request with documentation for home-assistant.io (if applicable): home-assistant/home-assistant.io#<home-assistant.io PR number goes here>
Example entry for configuration.yaml (if applicable):
media_player:
  - platform: samsungtv_upnp
    name: Samsung TV
    url: http://<IP_ADDRESS>:7676/smp_10_

Checklist:

[ ] The code change is tested and works locally.
[ ] Local tests pass with tox. Your PR cannot be merged unless tests pass
[ ] There is no commented out code in this PR.
[ ] I have followed the development checklist

If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.io

If the code communicates with devices, web services, or third-party tools:

[ ] The manifest file has all fields filled out correctly. Update and include derived files by running python3 -m script.hassfest.
[ ] New or updated dependencies have been added to requirements_all.txt by running python3 -m script.gen_requirements_all.
[ ] Untested files have been added to .coveragerc.

If the code does not interact with devices:

[ ] Tests have been added to verify that the new code works.

The main problem I see: the interface doesn't support all the features of the current implementations, so this won't replace it. There are other UPNP services which can fill the gap, but this raises the implementation costs.

@tommyjlong I've added the source IDs. Please try again.

Added the custom components to my Home-Assistant setup for my Samsung Frame (2018) and this to my configuration:
* platform: samsungtv_upnp
  name: Samsung Frame
  url: http://<IP_ADDRESS>:9197/dmr

But media player stays in idle

Where did you get the URL? Did you use this command? upnp-client --pprint search --service_type urn:samsung.com:service:MainTVAgent2:1

Added the component to my custom_components and added to my configuration.yaml

platform: samsungtv_upnp
name: Samsung Frame
url: http://<IP_ADDRESS>:9197/dmr

Media player device stays in idle.

Added the component to my custom_components and added to my configuration.yaml
* platform: samsungtv_upnp
  name: Samsung Frame
  url: http://<IP_ADDRESS>:9197/dmr

Media player device stays in idle.

@Imperial-Guard Where did you get the URL? Did you use this command? upnp-client --pprint search --service_type urn:samsung.com:service:MainTVAgent2:1

Added the custom components to my Home-Assistant setup for my Samsung Frame (2018) and this to my configuration:
* platform: samsungtv_upnp
  name: Samsung Frame
  url: http://<IP_ADDRESS>:9197/dmr

But media player stays in idle

Where did you get the URL? Did you use this command? upnp-client --pprint search --service_type urn:samsung.com:service:MainTVAgent2:1

No I used a scanner in my iPhone called "uPnP Mate"

@Imperial-Guard Do you see some log messages? This is absolutely work-in-progress. If you're experienced enough to debug this would help much. If not please be patient and test again from time to time and report when something changes.

@Imperial-Guard Do you see some log messages? This is absolutely work-in-progress. If you're experienced enough to debug this would help much. If not please be patient and test again from time to time and report when something changes.

I will enable dubbing, I am not a very well experienced programmer but helping with testing and far as debugging what I can do I will able to help.
I did a check with the changing the ip of the url and received a error and when I turned my tv off the component wend to unavailable.
So the port seems to be right and there could be difference within the Samsung TVs reacting on uPnP.

I will close this PR because it contains protocol specific info. You can re-open when the logic has been extracted into a PyPI lib.


Sftp Saving issue

uunable to save a file in sftp location
but open a file from sftp, saving file that is from sftp is working

It's works fine, share some more info
It may be possible that your server stopped or it doesn't allow modification or due to any security reason


Kalomaze's Quadratic Sampling

Quadratic Sampling

I don't have any capacity to make decisions here, but i approve.

I think an exponent of 2.3 works better than 2. Change my mind.

I know I know, but this one is good, a lot of people have tested it. I do need to make a change after I'm done with work today though so OOP is sub-optimal for quad sampling application. So more to come there.

Okay, updates are made!

This one is huge boss! 💪

Guess i drop both both tabby and exl2 until is implemented, just like the frog... off to the shadow realms with ye.

So there. With a secret cubic sampling option for those brave enough to seek it out.


[Hyperengage Actions] - Added Documentation for New Destination

Proposed changes
Added documentation for the Hyperengage Actions Destination.
Merge timing

ASAP

hey @tcgilbert - can you take a look at this PR when you get a chance?

hi @zhadier39 - I'm not able to directly edit this branch due to protections on your branch. I've duplicated the contents of this PR in #5606 and we'll use that one to merge the docs into the site!


Expose which timers are idle and loss detection

The documentation for quinn_proto::generic::Connection indicates that certain timers have special meaning.  I currently use an ugly hack (hard-coding numbers) to determine which timer is which.  Applications should be able to tell if a timer is the idle or loss-detection timer.

What's your use case? These are intended to be implementation details.

Performance.  I want to avoid polling for events that cannot possibly happen.

Such as?

I only need to poll for outgoing packets if the LossDetection timer expired.  I only need to poll for application-facing events if the idle timer expired.

Those aren't guarantees we want to make on the public API. For example, we might in the future propagate the detection of lost application datagrams to the application, which would constitute an application-facing event on loss detection timeout. I see there's some outdated comments from before we made timers opaque; I'll update those.

Can we reduce the number of timers down to 1, with quinn_proto handling the bookkeeping internally?

We could, and I wouldn't mind the simplification of the API. I'm not sure there's even a hypothetical advantage to the way it works presently; the thinking was that you might have one great big pool of timers with a single layer of multiplexing, but that's not how it worked out.

@Ralith I agree.  That’s what quiche does.  Ideally, we would have just one timer per endpoint.  The current design strongly encourages having a background task per connection, which is annoying.


Introduce Bloom Filter as non-experimental/core postings format

Description
Today, BloomFilteringPostingsFormat in Lucene is marked experimental.
Based on our analysis of the the data structure in OpenSearch for the Primary Key field using the nyc_taxis workload [See Issue], we have found that it proves very useful for indexing performance, and also certain term queries/Get Document calls on the PK.
We want to introduce this as an opt-in feature in OpenSearch for customers so that they can take advantage of the performance improvements, and wanted inputs from the community on the following:

Why is the BloomFilteringPostingsFormat in experimental status?
Is it possible to contribute and mark it as a core feature with support for backward compatibility in Lucene?

We've done few changes in OpenSearch to support  an off-heap implementation and introduce certain knobs which may prove useful for Lucene customers and would like to see if they can be contributed to Lucene as well [opensearch-project/OpenSearch/pull/11027].

supporting back compat is a one-way door and a big deal. Back compat has a heavy price and is responsible for lots of bugs (e.g. Lucene 9.9.1 release). It can't be done on a whim and IMO we already support way "too much" here by far. Too much commercial influence on the project...
I also don't think this bloom format is ready. You argue it should have lucene's backwards compatibility support at the same time as pointing to thousands of lines unmerged PR with recommended changes to it! That's not stability.

@rmuir : We understand supporting back compatibility is one way door, but bloom filter has been in experimental mode for years now. So, we would like to understand from Lucene community - has it been abandoned?

I also don't think this bloom format is ready.

The intend here is to help make bloom ready by contributing to lucene if the community is aligned and intent to release it. It can be controlled under feature flag/ experiment mode until it is production ready.
@mikemccand would like to hear your thoughts as well.

I agree with @rmuir -- promising backwards compatibility (API or index format) is a huge burden on Lucene developers, and it's hard enough with the default Codec today.
Given that the bloom postings format is still very much in flux, let's wait on removing the experimental tag.  E.g. we are also pursuing another experimental codec (inspired by Tantivy) that also seems to speed up the primary-key lookup use case.
Note that OpenSearch devs could also choose to offer this backwards compatibility to its users.  The promise need not be implemented only in Lucene.
Thank you for sharing those benchmark results.  That is indeed quite a sizable impact on indexing throughput / long-pole latencies, especially as you greatly increase the bloom filter size to lower the false-positive rate.  It looks like that test was 100% updateDocument calls with 25% of the updates being updates not appends?
+1 to pursue the linked improvements (off-heap option) -- the linked PR looks interesting -- maybe open a PR here for that?  Or is that change somehow OpenSearch specific?

Thanks @mikemccand for the feedback. We can pursue the route to offer the backward compatibility in OpenSearch directly if there are no other takers among Lucene users.
💯 for contributing the off heap implementation to Lucene to prevent memory overhead due to large bloom filter size.

Thanks @mikemccand @rmuir for your feedback.

It looks like that test was 100% updateDocument calls with 25% of the updates being updates not appends?

That is correct. We used Lucene updateDocument calls for the entier test, with  25% of the updates being updates not appends.

+1 to pursue the linked improvements (off-heap option) -- the https://github.com/opensearch-project/OpenSearch/pull/11027 looks interesting -- maybe open a PR here for that? Or is that change somehow OpenSearch specific?

Sure, I can work on a PR to contribute the changes to enable the support with Lucene constructs. I think we should have a way to achieve this in Lucene using a wrapper on existing bit-set implementation and/or the underlying array within the bitset.

I'd like to stick to the current policy that only the default codec has backward compatibility support. One idea would be to fold bloom filters into the default postings format like we did for the Pulsing codec (inlining postings in the terms dictionary when docFreq=1). One reason why I'm a bit on the fence about it is that this format still needs to materialize the bloom filter in heap at write time, while I'd like to keep heap requirements of the default codec as low as possible.
That said, +1 to improve the bloom filter postings format to load the bloom filter off-heap at read time, this would be an interesting improvement.


✗ Android license status unknown Couldn't solve this

Logs

 I tried to solve the problem of android license while running flutter doctor and for that, I went through many github issues, StackOverflow discussions, and youtube videos so I am creating this new issue so that I could fix my problem and be ready for #30DaysOfFlutter.Thank you! Also, this is my first ever issue so I mayn't be clear on what I want to convey so please ask if you want  to know more about the problem ;)
$ flutter doctor --android-licenses
Error occurred during initialization of boot layer
java.lang.module.FindException: Module java.se.ee not found

$ java -version
java version "15.0.1" 2020-10-20
Java(TM) SE Runtime Environment (build 15.0.1+9-18)
Java HotSpot(TM) 64-Bit Server VM (build 15.0.1+9-18, mixed mode, sharing)

$ $JAVA_HOME
bash: /usr/lib/jvm/jdk-15.0.1: Is a directory

$ flutter doctor -v
[✓] Flutter (Channel stable, 1.22.6, on Linux, locale en_US.UTF-8)
    • Flutter version 1.22.6 at /home/bibekdhkl/snap/flutter/common/flutter
    • Framework revision 9b2d32b605 (10 days ago), 2021-01-22 14:36:39 -0800
    • Engine revision 2f0af37152
    • Dart version 2.10.5

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /home/bibekdhkl/Android/Sdk
    • Platform android-30, build-tools 30.0.3
    • Java binary at: /usr/bin/java
    • Java version Java(TM) SE Runtime Environment (build 15.0.1+9-18)
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/linux#android-setup for
      more details.

[!] Android Studio (not installed)
    • Android Studio not found; download from
      https://developer.android.com/studio/index.html
      (or visit https://flutter.dev/docs/get-started/install/linux#android-setup
      for detailed instructions).

[✓] Connected device (1 available)
    • Android SDK built for x86 (mobile) • emulator-5554 • android-x86 • Android
      10 (API 29) (emulator)

P.S. I am using Ubuntu 20.04

Hello! The same problem is described in this issue and on stackoverflow. You need java 8,9 or 10, because java.se.ee was removed in java 11. Try to accept licences with old java and switch back to java 15 afterwards.

@Bibekdhkl
please run flutter doctor --android-licenses
if the issue persists provide your flutter doctor -v

@iapicca
I have attached every such command's output on the description so do please check them out above

I have attached every such command's output on the description so do please check them out above

sorry I haven't noticed the expansion

$ java -version
java version "15.0.1" 2020-10-20
Java(TM) SE Runtime Environment (build 15.0.1+9-18)
Java HotSpot(TM) 64-Bit Server VM (build 15.0.1+9-18, mixed mode, sharing)

now I see the problem, you are using a wrong java version
please install OpenJDK8 and try again
the easy way is through android studio running sudo snap install android-studio --classic
alternatively you can follow this guide
I recommend to close this issue as it's not a bug with flutter,
but an error in the installation process and better suited for stack overflow

ohh okay thank you @iapicca
And yeah I finally fixed the issue by installing an older version of JAVA JDK i.e. 14.0.2 as I read somewhere the license acceptance is not possible in this new JAVA 15.0.1

@Bibekdhkl
Flutter does not support Java 14
you might have some negative surprise down the line
I recommend you to use jdk8

I don't have much idea about that but for me, it worked after I switched to javac 14.0.2 from java 15.0.1.

@iapicca
Hi!
I also meet this problem on windows.
This is my java version:
java version "1.8.0_281"
Java(TM) SE Runtime Environment (build 1.8.0_281-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.281-b09, mixed mode)

And flutter doctor output:
[!] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
    • Android SDK at C:\software\android\SDK
    • Platform android-R, build-tools 29.0.3
    • ANDROID_HOME = C:\software\android\SDK
    • ANDROID_SDK_ROOT = C:\software\android\SDK
    • Java binary at: C:\Program Files\Java\jdk1.8.0_261\bin\java
    • Java version Java(TM) SE Runtime Environment (build 1.8.0_261-b12)
    X Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/windows#android-setup for more details.

When i use flutter doctor --android-licenses, it told me:
Error: Unknown argument --licenses

Do you know how to solve this?

@lizhongzhen11
Cam you post the logs from
flutter doctor --android-licenses - v

@iapicca
This is the logs:
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Error: Unknown argument --licenses
Usage:
  sdkmanager [--uninstall] [<common args>] \
    [--package_file <package-file>] [<packages>...]
  sdkmanager --update [<common args>]
  sdkmanager --list [<common args>]

In its first form, installs, or uninstalls, or updates packages.
    <package> is a sdk-style path (e.g. "build-tools;23.0.0" or
             "platforms;android-23").
    <package-file> is a text file where each line is a sdk-style path
                   of a package to install or uninstall.
    Multiple --package_file arguments may be specified in combination
     with explicit paths.
In its second form (with --update), currently installed packages are
    updated to the latest version.
In its third form, all installed and available packages are printed out.

Common Arguments:
    --sdk_root=<sdkRootPath>: Use the specified SDK root instead of the SDK containing this tool
    --channel=<channelId>: Include packages in channels up to <channelId>.
                           Common channels are:
                           0 (Stable), 1 (Beta), 2 (Dev), and 3 (Canary).

    --include_obsolete: With --list, show obsolete packages in the
                        package listing. With --update, update obsolete
                        packages as well as non-obsolete.
    --no_https: Force all connections to use http rather than https.
    --proxy=<http | socks>: Connect via a proxy of the given type.
    --proxy_host=<IP or DNS address>: IP or DNS address of the proxy to use.
    --proxy_port=<port #>: Proxy port to connect to.

* If the env var REPO_OS_OVERRIDE is set to "windows",
  "macosx", or "linux", packages will be downloaded for that OS.

@iapicca
This is the logs:
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Error: Unknown argument --licenses

@lizhongzhen11
Are you sure you run exactly
flutter doctor --android-licenses - v
It seems that you try to either put a an extra  space or run --licenses
Please try again and post the logs including your command

Oh! I solve this problem!
It's the problem with sdkmanager, i don't use Android-studio. I use vscode to configure flutter. I find the solution in https://www.cnblogs.com/yinniora/p/14250226.html (chinese)
In general, the sdkmanager lack --licenses command.

Switched to java-8-openjdk and it allows me to accept the licenses... but no matter how many times I run through it, soon as I'm done I recheck and none of the licences have been accepted
through the process I've also managed to break the emulator, lots of fun going on here (5.11.1-1-manjaro)


Performance issue when publishing to certain topics

Hi all,
I've discovered a weird quirk with certain subjects and performance.  If a subject matches the regex ".*sub.*", then the performance in jnats drops by at least a factor of 10.  I'm not sure why this is.  You can verify this rather easily by taking the autobench code and changing the getSubject() method to return some string matching that regex.
Funny enough, I don't think this occurs with TLS on, but not 100% sure.  I'm also guessing the problem exists in jnats, because the go program nats-bench doesn't seem to have the same issue and the issue occurs for even just publishing messages.
Versions:
jnats 2.6.6
nats-server 2.1.8
Examples:
Normal Benchmarks:

Task :NatsAutoBench.main()
Connecting to NATS server at nats://localhost:4321
Running warmup
Current memory usage is 226.27 mb / 256.00 mb / 3.97 gb free/total/max
Executing tests .........................
PubOnly 0                 1,000            389,165 msg/s       0.00 b/s
PubOnly 8                 1,000            330,163 msg/s      2.52 mb/s
PubOnly 32                1,000            126,361 msg/s      3.86 mb/s
PubOnly 256               1,000             20,482 msg/s      5.00 mb/s
PubOnly 512               1,000             10,598 msg/s      5.18 mb/s
PubSub 0                  1,000            362,673 msg/s       0.00 b/s
PubSub 8                  1,000            310,964 msg/s      2.37 mb/s
PubSub 32                 1,000            104,960 msg/s      3.20 mb/s
PubSub 256                1,000             27,154 msg/s      6.63 mb/s
PubSub 512                1,000             14,420 msg/s      7.04 mb/s
PubDispatch 0             1,000            263,414 msg/s       0.00 b/s
PubDispatch 8             1,000            394,866 msg/s      3.01 mb/s
PubDispatch 32            1,000            132,041 msg/s      4.03 mb/s
PubDispatch 256           1,000             27,492 msg/s      6.71 mb/s
PubDispatch 512           1,000             15,946 msg/s      7.79 mb/s
ReqReply 0                    2                655 msg/s       0.00 b/s
ReqReply 8                    2                876 msg/s      6.84 kb/s
ReqReply 32                   1                721 msg/s     22.56 kb/s
ReqReply 256                  1                689 msg/s    172.50 kb/s
ReqReply 512                  1                648 msg/s    324.19 kb/s
Latency 0         5    374 / 472.00 / 500    +/- 251.22  (microseconds)
Latency 8         5    386 / 429.20 / 484    +/- 239.74  (microseconds)
Latency 32        5    400 / 449.90 / 501    +/- 250.38  (microseconds)
Latency 256       5    453 / 538.40 / 584    +/- 294.85  (microseconds)
Latency 512       5    397 / 428.50 / 609    +/- 276.90  (microseconds)
Final memory usage is 5.89 mb / 8.00 mb / 3.97 gb free/total/max

Benchmarks with subject "this.is.a.bad.subject.for.some.reason"

Task :NatsAutoBench.main()
Connecting to NATS server at nats://localhost:4321
Running warmup
Current memory usage is 218.12 mb / 256.00 mb / 3.97 gb free/total/max
Executing tests .........................
PubOnly 0                 1,000             14,072 msg/s       0.00 b/s
PubOnly 8                 1,000             14,998 msg/s    117.18 kb/s
PubOnly 32                1,000             13,479 msg/s    421.25 kb/s
PubOnly 256               1,000              7,356 msg/s      1.80 mb/s
PubOnly 512               1,000              5,334 msg/s      2.60 mb/s
PubSub 0                  1,000             15,486 msg/s       0.00 b/s
PubSub 8                  1,000             13,736 msg/s    107.32 kb/s
PubSub 32                 1,000             14,538 msg/s    454.34 kb/s
PubSub 256                1,000             10,440 msg/s      2.55 mb/s
PubSub 512                1,000              7,388 msg/s      3.61 mb/s
PubDispatch 0             1,000             14,760 msg/s       0.00 b/s
PubDispatch 8             1,000             13,696 msg/s    107.00 kb/s
PubDispatch 32            1,000             14,690 msg/s    459.08 kb/s
PubDispatch 256           1,000             10,734 msg/s      2.62 mb/s
PubDispatch 512           1,000              8,056 msg/s      3.93 mb/s
ReqReply 0                    2                626 msg/s       0.00 b/s
ReqReply 8                    2                810 msg/s      6.33 kb/s
ReqReply 32                   1                775 msg/s     24.23 kb/s
ReqReply 256                  1                600 msg/s    150.07 kb/s
ReqReply 512                  1                540 msg/s    270.36 kb/s
Latency 0         5    438 / 548.20 / 745    +/- 314.75  (microseconds)
Latency 8         5    438 / 609.10 / 663    +/- 321.49  (microseconds)
Latency 32        5    495 / 530.40 / 718    +/- 326.90  (microseconds)
Latency 256       5    395 / 602.10 / 799    +/- 349.26  (microseconds)
Latency 512       5    497 / 620.10 / 707    +/- 343.75  (microseconds)
Final memory usage is 5.89 mb / 8.00 mb / 3.97 gb free/total/max

I am curious if this happens with other clients?

The only other one I tried was nats-bench, which did not exhibit this problem as near as I could tell.

I ran the PubOnly portion of this against the latest server today and also did not see this issue. There have been many changes to that server (mostly for Jetstream) in the last 5 months. There have also been a few changes for performance in the client, specifically some flushing when publishing. I will run a full benchmark tonight, but I'm encouraged that I did not see this.


Support for multiple repositories

This PR adds support for multiple repositories in a single call of the role.
By default, this changes nothing. Role configs that simply set the borg_repository variable to a repository will continue to work.
If the borg_repository variable is set, while also having create_repo set, this will combine both to create a repository at borgbase.com, while also including the originally set repository.
If multiple repositories should be set, they will simply need to be given to the role as a list.
ie.:
    borg_repository:
      - ssh://borg@backupserver.domain:22/./{{ inventory_hostname }}
      - abcdefg@abcdefg.repo.borgbase.com:repo

I was not able to test these changes via the molecule test suite as I am using Fedora 33, and Docker is simply not available there currently, it seems. (podman is the default and the official docker repositories arent available for Fedora 33).
I tested this via my borgbase.com account and my own borg backup server.
As such, I also wasnt able to look into adding to the test cases, sadly. Ill have to see if I can install a different distribution to go back to this if you would like me to.

Looks fine. Just added some minor comments.
Thanks for taking the time to contribute! 🙏

I do not see any comments. Where did you add them?

Should be here: https://github.com/borgbase/ansible-role-borgbackup/pull/41/files

I might be blind, but I'm not seeing any comments there, sorry. :D

Weird, but I've seen stranger things 😜

In regards to the default: Ive run into problems later on when I append a string to an undefined variable in line 63 of tasks/main.yml. If I append to an undefined variable it always comes out as a string.
This is a problem if no repository has been set, as its entirely optional if you use the repository creation for borgbase.com.
It might be a good idea to add a test that either create_repo is set to true or borg_repository has been set.
It would also be possible to solve this issue in a different way if you really want the default gone.
We could set a bb_created_repo variable when the repository has been created by the role and simply append it in the template. I didnt choose this, because it makes the template longer, which I try to avoid in general unless necessary.
I also have an idea for another different solution to this problem. that is closer to my first one, while removing the need for the default. I will commit this sometime today once Ive worked out a few issues.
As for the list test: I have used the part without mapping for a while to test for lists in other projects at work.  Havent had any problems so far with it. The mapping part simply adds to it to see if its a dict.

I have made the change and its now done without needing the default value of borg_repository.
Btw: please have a look at issue #13 . I had a few questions in regards to your expectations for the automatic init of repos.

Thanks! Merged.


add node_modules ignore to prettier

Change description

adds the node_modules folder to prettierignore

Type of change

[ ] Bug fix (fixes an issue)
[x] New feature (adds functionality)

Checklists
Development

[x] Lint rules pass locally
[x] Application changes have been tested thoroughly
[x] Automated tests covering modified code pass

Security

[x] Security impact of change has been considered
[x] Code follows company security practices and guidelines

Code review

[x] Pull request has a descriptive title and context useful to a reviewer. Screenshots or screencasts are attached as necessary
[ ] "Ready for review" label attached and reviewers assigned
[ ] Changes have been reviewed by at least one other contributor
[x] Pull request linked to task tracker where applicable

LGTM


tool Config does not accept decimels

The config for tool stats does not seem to accept decimels for the dmg value. Setting it from 3.0 to 2.5 have it be 6 in game, when it should be 6.5

Which value is this for? If it's for swords then it's intended behavior as in vanilla it only accepts int value.


Added security.md

When submitting a connector, please make sure that you follow the requirements below, otherwise your PR might be rejected. We want to make you have a well-built connector, a smooth certification experience, and your users are happy :)
If this is your first time submitting to GitHub and you need some help, please sign up for this session.

[ x] I attest that the connector doesn't exist on the Power Platform today. I've verified by checking the pull requests in GitHub and by searching for the connector on the platform or in the documentation.
[ x] I attest that the connector works and I verified by deploying and testing all the operations.
[ x] I attest that I have added detailed descriptions for all operations and parameters in the swagger file.
[ x] I attest that I have added response schemas to my actions, unless the response schema is dynamic.
[ x] I validated the swagger file, apiDefinition.swagger.json, by running paconn validate command.
[ x] If this is a certified connector, I confirm that apiProperties.json has a valid brand color and doesn't use an invalid brand color, #007ee5 or #ffffff. If this is an independent publisher connector, I confirm that I am not submitting a connector icon.

If you are an Independent Publisher, you must also attest to the following to ensure a smooth publishing process:

[ ] I have named this PR after the pattern of "Connector Name (Independent Publisher)" ex: HubSpot Marketing (Independent Publisher)
[ ] Within this PR markdown file, I have pasted screenshots that show: 3 unique operations (actions/triggers) working within a Flow. This can be in one flow or part of multiple flows. For each one of those flows, I have pasted in screenshots of the Flow succeeding.
[ ] Within this PR markdown file, I have pasted in a screenshot from the Test operations section within the Custom Connector UI.
[ ] If the connector uses OAuth, I have provided detailed steps on how to create an app in the readme.md.

Hello @pkisigning,
Congratulations, your Pull Request is approved and merged.
Please update your submission in ISV Studio by using the latest commit ID.
I'd like to point out that security.md file will not be a part of public documentation. The only file included in public documentation is intro.md file, which can be updated during the submission in ISV Studio.
Let us know if you have any questions.


fix(fmt): write single param on multiline if params_first

Motivation
Closes #7721
params_first  for fns with single params doe not write param on its own line, as per docs :
"params_first" - Break the function header into multiple lines, with each parameter on its own line. The function name stays on the first line.

Solution

write single param multiline if setting to params first

Thank you for fixing this bug. However, I personally never noticed it as an issue and actually found it to be a feature. When dealing with a single parameter, I think it improves readability to have both the function name and the input parameter on the same line.
Is there a way to bring this behavior back? i.e. to only allow params_first and multiline formatting when the number of parameters > 1?

I agree with @smol-ninja, the diff on our codebase can be seen here https://github.com/ethereum-optimism/optimism/pull/11620/commits/2f9da08b3d70c4f21a00859a0440166ab6bce54b
I tried to use a params_first flag or the flag in the docs to undo this: https://github.com/foundry-rs/foundry/blob/2442e7a5fc165d7d0b022aa8b9f09dcdf675157b/crates/fmt/README.md?plain=1#L116


instant giveaway audit fixes

Description
Brief description of the issue or feature implemented in this PR.
Link to Jira Issue or relevant doc
Dev
Any explanation for the devs that will review your implementation/code.
Qa
Any guidance or important information for the team that will be testing your solution.

Note: you can add any additional information you think is important for giving context to your PR.

Checklist and Markdown

[ ] Remember you could add any type of formatting to enhance your PR.
[ ] Like this checklist
[ ] And with this markdown format

A different PR against master was merged, this one was used to track the differences during the audit


Speed test for the class LibraryRankedSpectrumInfo

Test which measures how long it takes for the class LibraryRankedSpectrumInfo to match predicted ions to library spectra. This way we can make sure that my changes in #1050 aren't going to slow down Skyline.
Here is what the output on my machine looks like when I run it:
[16:54]   1.0   TestLibraryRankedSpectrumInfoSpeed             (en)
Time to get LibraryRankedSpectrumInfo for Short Peptide 300 times: 00:00:00.0817566
7 matched peaks. Ranked peaks: { 147.1,246.1,359.3,488.4 }
Time to get LibraryRankedSpectrumInfo for Long Peptide 300 times: 00:00:00.2393599
9 matched peaks. Ranked peaks: { 1352.3,1423.8,1475 }
Time to get LibraryRankedSpectrumInfo for Short Peptide with 1 Neutral Loss 300 times: 00:00:00.0897600
4 matched peaks. Ranked peaks: { 322.0948,609.278,707.1737,708.1829 }
Time to get LibraryRankedSpectrumInfo for Long peptide with 1 Neutral Loss 300 times: 00:00:00.5255929
7 matched peaks. Ranked peaks: { 1203.3427,1250.8986,1274.571,1298.2545,1329.6705,1341.2415 }
Time to get LibraryRankedSpectrumInfo for Short peptide with 2 neutral losses 300 times: 00:00:00.2818595
8 matched peaks. Ranked peaks: { 343.2071,403.681,443.1311,610.0934,709.152 }
Time to get LibraryRankedSpectrumInfo for Long peptide with 2 neutral losses 300 times: 00:00:01.5727919
12 matched peaks. Ranked peaks: { 682.7864,1031.3154,1069.0573,1090.3293,1251.3538,1358.1454,1431.825,1489.1063,1497.3608 }
  0 failures, 2.66/3.72/57.1 MB, 11/361 handles, 2 sec.

Currently, the times that I get in #1050 are almost exactly the same.

The spectrum density matters also. How many spectrum peaks are being matched?

The spectrum density matters also. How many spectrum peaks are being matched?

The number of matched peaks was part of the output and varied between 4 and 12 depending on the peptide. I was only passing in "3" for "minPeaks" to the LibraryRankedSpectrumInfo constructor. I did not realize the value that Skyline usually uses is "TransitionGroup.MAX_MATCHED_MSMS_PEAKS" (which is equal to 100). I'll update the test.

I am talking about the number of spectrum peaks, which should be in the hundreds. I think you should look at some real libraries of spectra.

I am talking about the number of spectrum peaks, which should be in the hundreds. I think you should look at some real libraries of spectra.

These are pretty typical looking spectra from NIST yeast and PeptideAtlas phosph spectral libraries. We could certainly add other spectra to the test file if we could think of anything that might have different performance characteristics. (I also still have the big Skyline document with 72000 peptides in it that I selected these 6 peptides from if we need it).


encoding/protojson: opt out of whitespace randomization

Commit https://github.com/protocolbuffers/protobuf-go/commit/582ab3de426ef0758666e018b422dd20390f7f26 added randomization to the multi-line json output.
I understand there are good reasons for doing this. I hope you will understand I also have good reasons for not doing this. Among others, generating a protobuf with fake data, re-rendering it on every test run is one of the best ways to detect drift in the configuration, accidental or purposeful.
I can handle the output changing across multiple versions of the protobuf library. It's much easier for me to sync the library version than it is to sync Go versions. If there are breaking changes that generate a diff, I can manage them all at once, instead of
Is there any way to opt out? A UnsafeDisableRandomization flag, or something. Right now I am probably going to make a second pass with a marshaler that does not generate random whitespace.

The reason why we were unable to make improvements to both text and JSON was because of the many hard-coded tests that expected the output to be byte-for-byte identical. We're not going to repeat the mistake again with the new prototext and protojson implementations.
I understand that stable output has benefits, but unless there is a canonical definition (which there isn't) for how to serialize protobufs in text and JSON, we're not going to pretend like one exists.
If you want a frozen output, then use github.com/golang/jsonpb. The output will be stable, but will also continue to have buggy behavior in some cases and it won't received any improvements. If you want a decent amount of stability, then pass the output through a formatter. For example, just parsing the JSON with encoding/json and formatting it again should provide some degree of stability.

This is what I'm doing, for the moment
	marshalerOnce.Do(func() {
		marshaler = &protojson.MarshalOptions{
			EmitUnpopulated: true,
			Indent:          "",
			Multiline:       true,
			UseProtoNames:   true,
		}
	})
	data, err := marshaler.Marshal(cfg)
	if err != nil {
		return err
	}
	data = append(data, '\n')
	var rm json.RawMessage = data
	data2, err := json.MarshalIndent(rm, "", " ")
	if err != nil {
		return err
	}
	_, err = w.Write(data2)
	return err

That seems reasonable?

Closing as we're not going to be changing the randomization. Analysis of all tests at Google (we have a lot) for text and JSON usages indicates that 89.5% do not care about the exact output. Of the remaining, 10.4% were brittle tests that really should have been written to perform a semantic comparison on the structured messages (rather than a byte-for-byte comparison on the serialized output). The remaining 0.1% were cases like generated configurations and/or code where a degree of stability is actually needed. For that 0.1%, the workaround is to pass the output to a JSON formatter or text formatter to produce more stable output.
If randomization pushes the 10.4% to properly perform semantic comparisons, then I'll be really happy. Since a workaround exists for the 0.1% with a legitimate need for some degree of stability, I don't think a opt-out is justified.

I don't think a opt-out is justified.

It's slightly amusing that your own tests disable randomization 😅
https://github.com/protocolbuffers/protobuf-go/blob/c2a26e757ed57746693fca53de0b6136a8e46b74/encoding/protojson/encode_test.go#L32-L33

Those frustrated by the randomization should be focusing on the development of a stable output format: #1121

I don't think a opt-out is justified.

It's slightly amusing that your own tests disable randomization, and yet you don't want to expose that to users 😅

The protobuf library is subject to the contract of the standardization. However, certain things (e.g. ordering of fields in a message) are undefined in the contract, but this library is still an authority on what its own self should be producing. Knowing how we have chosen those implementation details, we can generate golden tests dependent upon our own implementation details. We remain, after all, the authority on what we expect our code to generate, even if the contract is silent on certain details. We’re allowed to make stronger assumptions about the output than the standard provides. After all, if we change our output, we can also change our tests at the same time, and then the deltas of golden examples even documents how our output has changed.
Meanwhile, users of this package are not in that same situation. If this package changed the ordering of fields, it would break anyone else using golden test, even though the two variations are semantically identical. Their tests would break, even though the output itself remains valid and within the standard. So their golden tests fail, even though their code has not broken. It’s generally bad form to break people’s tests for non-functional issues.
Trying to keep Hyrum’s Law at bay is part of us looking forward to a potentially future formal clarification of deterministic output. We should hope that even if that standard requires us to make drastic changes, that no one using this package now should break tests one we implement it.


[ISSUE #6594] skip verification when admin user

Make sure set the target branch to develop
What is the purpose of the change

fix #6594
Brief changelog
XX
Verifying this change
XXXX
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

[x] Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[x] Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[x] Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test  to make sure integration-test pass.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

Codecov Report

Merging #6613 (a969fff) into develop (3fe81bf) will decrease coverage by 0.02%.
The diff coverage is 75.00%.

@@              Coverage Diff              @@
##             develop    #6613      +/-   ##
=============================================
- Coverage      43.10%   43.08%   -0.02%
+ Complexity      8996     8988       -8
=============================================
  Files           1107     1107
  Lines          78287    78287
  Branches       10202    10202
=============================================
- Hits           33745    33733      -12
- Misses         40314    40331      +17
+ Partials        4228     4223       -5

Impacted Files
Coverage Δ

...che/rocketmq/acl/plain/PlainPermissionChecker.java
79.16% <75.00%> (+8.33%)
:arrow_up:

... and 16 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


asPaginator keeps returning the same results

const twitter = new Twitter(argv['twitter-bearer-token']).readOnly.v2;

const user = (await twitter.userByUsername('gajus')).data;

const followerPaginator = await twitter.followers(user.id, {
  asPaginator: true,
  max_results: 1_000,
});

for await (const follower of followerPaginator) {
  console.log('follower', follower);
}

Returns
follower { id: '1080856485528244226', name: 'ben', username: '_BenHQ' }
follower { id: '288845518', name: 'aleida cray', username: 'aleidaipcray' }
follower { id: '14738889', name: 'DrunkenDuckJapan', username: 'Mr1Duck' }
follower { id: '1080856485528244226', name: 'ben', username: '_BenHQ' }
follower { id: '288845518', name: 'aleida cray', username: 'aleidaipcray' }
follower { id: '14738889', name: 'DrunkenDuckJapan', username: 'Mr1Duck' }
follower { id: '1080856485528244226', name: 'ben', username: '_BenHQ' }
follower { id: '288845518', name: 'aleida cray', username: 'aleidaipcray' }
follower { id: '14738889', name: 'DrunkenDuckJapan', username: 'Mr1Duck' }
follower { id: '1080856485528244226', name: 'ben', username: '_BenHQ' }
follower { id: '288845518', name: 'aleida cray', username: 'aleidaipcray' }
follower { id: '14738889', name: 'DrunkenDuckJapan', username: 'Mr1Duck' }
follower { id: '1080856485528244226', name: 'ben', username: '_BenHQ' }
follower { id: '288845518', name: 'aleida cray', username: 'aleidaipcray' }
follower { id: '14738889', name: 'DrunkenDuckJapan', username: 'Mr1Duck' }
[..]

Expected:
follower { id: '1080856485528244226', name: 'ben', username: '_BenHQ' }
follower { id: '288845518', name: 'aleida cray', username: 'aleidaipcray' }
follower { id: '14738889', name: 'DrunkenDuckJapan', username: 'Mr1Duck' }

Also, is this thread still accurate?
https://github.com/PLhery/node-twitter-api-v2/issues/81
Reading documentation, I thought that paginator automatically handles rate limit.

got TIMEOUT for me, help please

Hello, sorry for the long delay.
I've identified the encountered issue.
It happens with paginators having exactly one page, only in Twitter v2 API and only when you use the for await-of iterator.
For your second question, it does when using for await-of or .fetchLast. It automatically stops fetching (without any error thrown), but it does not prevent you to call .next.

Hi again,
After running an npm update, do you still experiment issues with paginators?
It should be fixed since 1.7.2 :)


python310Packages.polyline: modernize

switch to pytestCheckHook
add format and changelog
disable on unsupported Python releases
add pythonImportsCheck

Description of changes
Fix build (https://hydra.nixos.org/build/220019226)
ZHF: #230712

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 232777 run on x86_64-linux 1

2 packages failed to build:

python310Packages.mapbox
python310Packages.mapbox.dist

4 packages built:

python310Packages.polyline
python310Packages.polyline.dist
python311Packages.polyline
python311Packages.polyline.dist


Use name validations from our API instead of the k8s core

LGTM

@pmorie what's the status of this PR? If you still want to merge it, I'm ok with that as-is. I've approved my review and LGTM-ed as a result.


Re-introduce extended checks for psa unlock/wipe key slot

As part of #4516, commit https://github.com/ARMmbed/mbedtls/pull/4516/commits/cc7074128a437cbca66dcc0f710c764d0021d0cd remove calls to MBEDTLS_PARAM_FAILED() in psa_unlock_key_slot() and psa_wipe_key_slot() that we want to replace in some way, see for example https://github.com/ARMmbed/mbedtls/pull/4516#issuecomment-851317735
These will probably be re-introduced by the means of some kind of callback when MBEDTLS_TEST_HOOKS is defined, somewhat similarly to what's done for testing mutexes (see tests/src/threading_helpers.c).

@ronald-cron-arm @gilles-peskine-arm Could one of you double-check that the description makes sense and that the size estimate looks realistic? Thanks!

I think S is a realistic size estimate: psa_unlock_key_slot already returns an error code, and all we need to do is make sure that we don't further corrupt an already corrupt slot on the recovery path.
Related: testing


Reconnecting blocked！！！

When the account failed to connect ,it will blocked at reconnecting!!!

tracking in https://github.com/Auties00/Cobalt/issues/306
also try 3.4.5


Add cobertura.xml files for codecov

As per https://github.com/Drops-of-Diamond/diamond_drops/issues/11.

@jamesray1 I don't think this PR fixes the build issue, you can check out the job on Travis (here)
I don't think we need to add these files to the repository. The files you added are automatically generated by Travis CI when the code is being built.
The changes we need to make is to fix the failing build is in the Travis CI yml file.
  for file in target/debug/examplerust-*[^\.d]; do mkdir -p "target/cov/$(basename $file)"; ./kcov-build/usr/local/bin/kcov --exclude-pattern=/.cargo,/usr/lib --verify "target/cov/$(basename $file)" "$file"; done &&

The error is in
target/debug/examplerust-*[^\.d];
It should be diamond-drops-node I think instead of examplerust.
I feel it should be
`
target/debug/build/diamond-drops-node-*[^.d];
`

Ah yes, looks like you're right! The lines should be used for each crate, i.e.:
for file in *[_A-Za-z0-9]/target/debug/diamond_drops*[_A-Za-z0-9]-*[^\.d]; do mkdir -p "target/cov/$(basename $file)"; ./kcov-build/usr/local/bin/kcov --exclude-pattern=/.cargo,/usr/lib --verify "target/cov/$(basename $file)" "$file"; done &&

@jamesray1 You still need to remove the 271 files added.

Yep, I'm making another PR.

To find your GitHub-provided no-reply email, navigate to your email settings page under "Keep my email address private."

To find the email you used to configure Git on your computer, run git config user.email on the command line.

https://help.github.com/articles/creating-a-commit-with-multiple-authors/

@jamesray1 diamond_drops is setup as a library not an executable binary, the current kcov structure assumes it an executable which is leading to failing build.
The email is<EMAIL_ADDRESS>

Rebased to #80, however that didn't fix the issue. @samparsky, feel free to take the initiative on this.


High visibility icons

I made edits of the original muted/unmuted status icons for better visibility. The microphone is larger and is striked out with a thick red line when muted.

Thank you for the PR, the icon looks great.
Although I think it should have a thin dark outline so it's more visible on white backgrounds, as you can see in the screenshot below it's barely visible

Sure, I could definitely give the whole thing an outline, but it might not look great in the system tray at 16x16px unless I make a pixel-perfect version. If that turns out to be necessary, you may want to consider using different resources for the OSD microphone icon and the system tray.
I was under the impression that the dark icons in the Resources folder were used in certain configurations, but I haven't found where they're used. Maybe we could expose some toggle in the UI between the light and dark icons.
Either way, I'll soon get around to making some changes to the base icons and play with different scales.

Yeah, I think I'll need to use separate icons for the overlay, also might as well add the option to choose between light and dark in that case.
currently, the black icon is only used for the tray and only when windows 10/11 is set to light mode.

Sounds good. I already made a vector version of the base microphone, so I can crank out the different sizes in a bit. What is the size of the overlay version? Been searching through the scripts, but I can't find the spec.

Sounds good, Saif. I'll get working on those later tonight. Thanks for the info!

Thank you, the icons look great, I will merge this now.
I'm gonna work on the overlay though, I'm looking into using Gdip to render the icon because it looks really awful right now

yeah, overlay is all grainy/blurred, looks low res or something.

I can confirm Gdip does indeed fix the issue

I can confirm Gdip does indeed fix the issue

Cool. I was about to experiment with a transparent background to see if that helps, but this is looking very nice.

@Reenuip sorry for the delay (I was busy with other stuff).
I have pushed the new overlay, you can test it out in the pre-release version if you want, I still need to add the new overlay options to the config UI though


Search for competitive optimisation libraries

Search for competitive optimisation libraries Mantella should be compared with in runtime and/or quality of found solutions.

Just a little list of first found libraries:

DAKOTA Page
NOMAD Page
LocalSolver Page
Ipopt Page
minbleic (in AlgLib) Page
NLopt Page


Development Discussion

Some possible extensions to the package:

Ability to assign weights to individual objects. Use those weights to make maps.
Add colors and magnitudes for streams (ugali; isochrones) and background (galfast)? Do we want to be running a matched filter or do we assume that streamsim is the output of the matched filter?
Add (angular) power spectrum density fluctuations. Currently thinking in terms of angular power spectrum Cls.
Integration with observational selection functions

@MatthieuPE has been addressing 2 on #8


[Issue] Running azd pipeline config on Aspire apphost project fails to recognize repo root

Output from azd version
azd version 1.5.0 (commit 012ae734904e0c376ce5074605a6d0d3f05789ee)
Describe the bug
When running azd pipeline config in the aspire apphost project directory it fails with:
Error: no CI/CD provider configuration found. Expecting either github and/or azdo folder in the project root directory.

The project however is in a git repository and there is a .github\workflows\build.yaml file present.  azd seems to look at project root versus repo root
To Reproduce

dotnet new aspire-starter
(add to a github repo)
azd init for Aspire
add a workflow file at the repo root (.github\workflows\foo.yaml)
run azd pipeline config

Expected behavior
I expect the pipeline config to work.  It does not :-).

Yeah, I think the whole convention is that it looks in the project root. @ellismg should we consider probing or is the guidance here to specify the relative path in azure.yaml?

Need to consider GHA modifications for Aspire applications and azd synth

azd seems to look at project root versus repo root

Yes, that's per design. You can have multiple azd projects within one only repo (for example, one repo full of azd samples).
In the future, once we implement https://github.com/Azure/azure-dev/issues/2752, azd won't require to have .github/workflows or .azdo folders next to the azure.yaml. But that is still TBD.


Limit molecule tests to a specific docker image

Issue Type

Feature request

Molecule and Ansible details
$ ansible --version
ansible <IP_ADDRESS>
  config file =
  configured module search path = Default w/o overrides
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

$ molecule --version
molecule, version 1.24.0

Molecule installation method: virtualenv + pip
Ansible installation method: virtualenv + pip

e.g.
virtualenv --python $(type -P python2.7) .venv
source .venv/bin/activate
pip install -U pip molecule ansible docker

Desired Behaviour
There's no way to execute molecule test with a specific docker image when multiple docker images are defined.  Take for example the following YAML.
---
dependency:
  name: galaxy
driver:
  name: docker
docker:
  containers:
    - name: ubuntu-16.04
      image: ubuntu
      image_version: 16.04
      ansible_groups:
        - ubuntu
    - name: debian-jessie
      image: debian
      image_version: jessie
      ansible_groups:
        - debian
verifier:
  name: testinfra
molecule:
  ignore_paths:
    - .git
    - .venv

I would like to run my tests with, for example, only Debian Jessie.  It would be nice to do something along the lines of:
molecule test --driver docker --container 'debian-jessie'

Actual Behaviour (Bug report only)
No combinations of options from molecule test --help give me the ability to limit which docker containers to launch.
$ molecule test --help
Usage: molecule test [OPTIONS]

  Runs a series of commands (defined in config) against instances for a full
  test/verify run.

Options:
  --driver TEXT                   Specificy a driver.
  --platform TEXT                 Specify a platform.
  --provider TEXT                 Specify a provider.
  --destroy [passing|always|never]
                                  Destroy behavior.
  --sudo / --no-sudo              Enable or disable running tests with sudo.
                                  Default is disabled.
  --help                          Show this message and exit.

@samrocketman What would the use case for this be?  I'm not so sure I see much value in adding another comand line option to test.
The developer workflow is generally to manually modify molecule.yml in the desired way while developing a role.  Once the role is done with development, molecule.yml should represent what they wish to test.
So basically I recommend manually changing things during development phase.  Otherwise, if you wish to change the container in CI I recommend this approach.

One use case is parallelizing Travis CI builds with matrix builds. (or any CI system)
Also, there doesn't have to be an additional option.  I'm okay with repurposing an existing option like --platform which does nothing in the docker driver.

The drivers unfortunately do not have consistency.  Molecule v2 corrects this.  If you wish to create a matrix for travis, I suggest using a combination of tox and ENV vars in molecule.yml.

Okay, thanks for letting me know.  Another workaround is to symbolic link.

Any reason you don't simply test all OSes together as a multi-node multi-distro test?  What would the benefit of looping though each distro on it's own?

It's not looping through each.  It runs them in parallel.  So let's say you're testing 4 different OSes.  You can launch each OS on a separate machine and test them all at once.  The intent there is to speed up the build by running independent parts in parallel on separate machines.

Found this issue while trying to do something similar. In my case I have a container config with some volume_mounts that do not work with my local macOS docker setup, but do with my Linux/Jenkins setup. I was searching for a way to selectively run one of these container configs that doesn't include or has an alternative volume_mounts locally. Then the other when running under CI.

@samrocketman I am used to testing ansible roles with kitchenci. The first utility I looked upon the molecule was the matrix with multiple OSes. @retr0h Travis runs either in parallel or serial. Multiple OS/distros are useful mostly driver testing, sandboxing etc. Just like DinD either in parallel or serial executions. Travis provides multiple host support though. However, it is not container-style more like VM style. A container is faster than VMs. I did not try yet, however, distro-based scenarios in shared style would be a workaround, like trusty-scenario, bionic-scenario etc . molecule looks like a scenario-based, something developers would expect, not sysadmins expect. I think the approach should be software is encapsulated within the container (distro), integration testing style, contrary to the developers' approach, a kind of platform-based approach.


Wrong ViewModel Implementation in the point 14-Connect with the data

In onCreate() below the RecyclerView code block, get a ViewModel from the ViewModelProvider.
mWordViewModel = new ViewModelProvider(this).get(WordViewModel.class);
the code should be
mWordViewModel = ViewModelProviders.of(this).get(WordViewModel.class);

@abdullah-alialdin Thank you for raising this issue. It has solved my problem. I was doing again and again same thing and was using same code provided in codelab which was not working and tried to find the correct code but I failed and finally I came to your reported issue and your code worked for me and my believed that Google developers also make mistakes.

@iamrohitkiransawai n 2.2.0 version viewModelProviders is marked ad deprecated so imho the google's code is correct :).


Query list of supported distros

We use distro in the Spack package manager to help us identify which Linux distro we are building software for. One feature we would like to add is something like distro.supported_distros, which would return the same list as contained in the docs. See https://github.com/spack/spack/issues/14717 for more explanation as to why this feature would be useful.

@adamjstewart The thing of it is, that list is not exhaustive. And in fact, it's not even comprehensive. Calling it the "supported" list would be a misrepresentation, because distro supports many more Linux distributions than just the ones listed there.
As the docs say, those are (some of) the distros where the identification is stable — if it comes back with one of those values, you can rely on the OS being the indicated one. But there are many, many possible outputs beyond just the strings listed in that table.
(Both manjaro and mandrake are recognized, but not listed. Nor is Intel's Clear Linux, just as a random pick of one I noticed was missing. So, on a Clear system distro.id() will return... well, I have no idea, because there's no special-case code for it in the module. But it will return something that's not on that list, most likely.)
distro supports ANY Linux installation on a best-effort level, and even plenty of non-Linux installations. (There are even several BSD flavors on the stable-ID list.) The list of possible outputs from distro.id() is not finite, and (with its current design) never can be.

Agree with @ferdnyc, as much as this would be a useful feature we'd be over-promising that OS distros don't break in the future. The only way to ensure this wouldn't happen would be testing every new version of a distro which isn't sustainable for this team as-is. I'm proposing we close this one?

Sounds good, thanks for the clarification!


Make fabfile more user-friendly

Put host and user into the file itself, use default ssh key.

Also maybe add comments on in not obvious places.

Plus make the script ask for sudo password once.

Plus move all prompts to the start of the scripts.

Plus add "update dependencies" option.

Plus try to simplify letsencrypt setup with piping.


Generic Boxes

Currently, boxes are parameterized only by proposition, with a value type of Long. It seems that a lot of the code is intertwined with this assumption, but there's no particular reason why box values should be limited to Long.
One example is using String as the value type for a box in order to define access restrictions to a blockchain resource.

Could you provide some example? If it is about "access restrictions", so I assume restricting access to the block itself, then probably String field should be a part of a proposition?

As we are developing smart contract templates, we have chosen to store a contract shared amongst members as a box with a value type that can hold the state data for that contract (sealed by a 1-of-n proposition). It's different than the above, but it is a case where the default value of Long seems unnecessarily cumbersome.
I understand that one could add additional data to the boxes, but given especially the way that boxes are handled in wallet, it seems a bit silly to do so just to end up with an extraneous Long value.
So far, we've ended up building a set of genetic type structures that sit parallel to a lot of the existing pieces (e.g. Box, Minimal State, BoxTransaction, etc) all the way up to the NodeViewHolder and Application level, largely duplicating existing logic because of how tightly coupled they all are.

As far as a String used for access restriction, technically transactions are access-limited by access to boxes necessary for their execution. Therefore, if one wanted to create a "permission" box, one could allow elevated user permissions (i.e. the submission of particular transactions) with the presentation of a box containing a particular string value (e.g. "Admin"). This is somewhat irrelevant for the point of the discussion now, but it was where I was going with that example.

Any further thoughts on this @kushti ?

@mpk2  Sorry for the long reply Matt! I imagine, a contract(a box) has some number of tokens associated with it?  Then value is needed. State could be made a part of Proposition, if transaction applicability depends on it (proposition could be e.g. a tuple of <state, set of possible transitions> then).
I never thought about token-less applications actually. I'll revise whether it it possible to do painless generalization regarding value field (another code review is coming, final before next Release Candidate version).

Thanks! Really enjoy the work that has been put into this project. We're building out our sections as it goes along.
I think that since we're running contract templates ourselves, the contract state is sufficient to determine valid transitions itself (it's not a sandboxed Turing-complete sort of setup). Instead, the contract can locally check the validity of an executed internal transaction. What you've just written seems generally applicable, though!
We've already decided to generalise some of the structures for our project. Would it be worth submitting a pull request if you decide to revise?

Thanks for the great words. would be extremely happy to get pull requests.
So a contract is enough to validate transactions associated with it(no anything external is needed)?  and no tokens associated with it?

To be more specific, contract templates as we're currently developing them have a limited lifespan. There are 3 types of transactions for this lifespan: ContractCreation , ContractMethodExecution (perhaps better named ContractMethodInvocation), and ContractCompletion. Each of these transactions involve state validation of some sort (e.g. validating that the party is authorized to perform such a transaction). But the state transition of the contract itself as a result of these transactions (in particular for ContractMethodExecution) is simply a function of parameters and is not user-definable other than by specifying different parameters. It's then easy enough to verify that the new state agrees with the execution.
The purpose here is for a contract to keep track of and modify the state of affairs as they pertain to a group of accounts. Depending on the final state of affairs at the end of lifespan, balances of cryptocurrencies do change. However, you wouldn't call these tokens as they are not maintained or modified during contract execution. It would be more similar to the idea that contracts in Ethereum could return ether to a user depending on their internal state.
The system can probably be envisioned in Ethereum as something like some master contract that maintains user token balances and spawns sub-contracts.
It's probably a strange use-case, but it currently matches our needs.

In case I wasn't clear (I sometimes write a little too much) each type of contract transaction does rely at least partially on minimal state to validate. For ContractMethodExecution, the contract method is invoked, and the new contract state is the result. Each party can simply execute locally to update their version of the contract held in the minimal state.

That sounds like a proposition is a state of a (finite?) state machine . so Proposition is the machine description. Is that a good model?

That's probably not a bad way to think about it. I separated this out from the proposition itself.
As it's currently implemented, the signature on the tx is checked against a list of valid parties that can interact with the contract, so the proposition right now on the contract box is a 1-of-n version of the PublicKey25519Proposition. The method and params are then attempted to be executed against the state of the contract, which returns a Try[ContractBox] (more or less), depending on the checks made by the contract.

I think the only reason I didn't include it in the Proposition is that it also needs to be part of the generation for the new box (i.e. new contract state)

See #25

@kushti Any thoughts on the PR?

@zihehuang Sory, we were a bit overbusy last week, will regard pull request in few days


ticketaddress should be validated as multisig when 'using a pool' is checked

One of the stake pool operators reported that they had a user who reversed the fields.

Paymetheus' UI for this is currently extremely minimal, it just calls the ticket purchasing RPC.  Should we consider adding the error there as well, or is this something that should only have client-side validation?

Oh right, yes it should be added there as well.


Set input type for SpecifiedPersonForm email/phone

This makes little difference on desktop browsers, but causes mobile software keyboards to customise their layout.