devilutionx: 1.4.0 -> 1.4.1

Description of changes
Removed SDL_AUDIOLIB patch as it is included in the 1.4.1 release (look for 33aab7e4b2eb4aa45b7549e86e4cd1831aaf7cc3), updated libzt dependency to match
https://github.com/diasurgical/devilutionX/commit/6bf474a6ba
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[x] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 208235 run on x86_64-linux 1

1 package built:

devilutionx


Realsense context when working with processes in Python

Required Info

Camera Model
D415

Firmware Version
<IP_ADDRESS>

Operating System & Version
Win 10 / Ubuntu 18.04

Platform
PC

SDK Version
2.30.01216

Language
Python

Issue Description
A process that initializes a Realsense pipeline periodically freezes after a single cycle when sharing context.
Here is a minimal recreating code:
import pyrealsense2 as rs
from multiprocessing import Process

ctx = rs.context()

class RS:

    def __init__(self):
        print("init")
        pipe = rs.pipeline(ctx)
        print(pipe)

class Proc(Process):

    def __init__(self):
        super(Proc, self).__init__()

    def run(self):
        for i in range(10):
            print(i)
            RS()
            print("after")

vs = Proc()
vs.start()

If the context isn't shared, the code crashes after 337 cycles:
import pyrealsense2 as rs
from multiprocessing import Process

class RS:

    def __init__(self):
        print("init")
        ctx = rs.context()
        pipe = rs.pipeline(ctx)
        print(pipe)

class Proc(Process):

    def __init__(self):
        super(Proc, self).__init__()

    def run(self):
        for i in range(400):
            print(i)
            RS()
            print("after")

vs = Proc()
vs.start()

When working with a thread instead of a process this doesn't happen.

Initializing the context inside the init (only once) while keeping it global solved the problem:
import pyrealsense2 as rs
from multiprocessing import Process

ctx = None

class RS:

    def __init__(self):
        global ctx
        if ctx is None:
            ctx = rs.context()
        print("init")
        pipe = rs.pipeline(ctx)
        print(pipe)

class Proc(Process):

    def __init__(self):
        super(Proc, self).__init__()

    def run(self):
        for i in range(700):
            print(i)
            RS()
            print("after")

if __name__ == '__main__':

    vs = Proc()
    vs.start()


uiSchema for definitions

Currently it seems there is no way to define ui options via uiSchema for the definitions. So where using definitions we can remove the repetition in schema, we are still forced to repeat ui options for each field in uiSchema.
Quick sample https://jsfiddle.net/9rwrn19d/1/

This is indeed a current limitation. Note that meanwhile you can leverage good old JavaScript to avoid too much verbosity: https://jsfiddle.net/yx09e5gp/1/
But I agree it'd be better to natively support uiSchema for definitions.

@n1k0 How would you solve uiSchema for a sample like References in playground?

Merging this into #701.


initial patch fixing high priority bottleneck

This patch need to be checked for side effect (also need to clean up the actual function) but created the pull request for the test. @amaltaro, Alan, could you review whether there will be side effect for not checking the create job condition

Sorry Seangchan, it seems I missed this patch.
A possible effect could be that this WQ thread would try to create the wmbs entries (subscriptions, files available) for the same work already pulled in the previous cycle. I didn't check the moment we move the work Status to Acquired though.
I agree this needs to be well tested

Seangchan, I ran more than 100 workflows with this patch in and I couldn't spot any issues in the workflow outcome. The only thing - that I believe to be new - is a few errors in the logs while pulling work, e.g.:
CouchError: CouchError - reason: None, data: {'descending': 'true', 'include_docs': 'true', 'resources': '{"T2_FI_HIP": 166, "T2_UK_SGrid_RALPP": 475, "T2_FR_GRIF_LLR": 331, "T2_KR_KNU": 98, "T2_DE_DESY": 1552, "T1_UK_RAL": 357, "T2_US_Caltech": 969, "T2_FR_CCIN2P3": 295, "T2_EE_Estonia": 818, "T2_UK_London_IC": 642, "T2_IT_Bari": 426, "T2_US_Nebraska": 951, "T2_US_UCSD": 1100, "T2_ES_CIEMAT": 306, "T2_RU_IHEP": 200, "T1_RU_JINR": 644, "T2_CH_CERN": 2531, "T2_BR_SPRACE": 212, "T1_IT_CNAF": 2000, "T2_CH_CSCS": 595, "T2_UA_KIPT": 80, "T2_US_Purdue": 1206, "T2_BE_UCL": 198, "T2_BE_IIHE": 474, "T2_UK_SGrid_Bristol": 206, "T2_FR_IPHC": 308, "T1_ES_PIC": 338, "T1_US_FNAL": 3150, "T2_IT_Legnaro": 604, "T2_IT_Rome": 263, "T2_IT_Pisa": 558, "T2_US_Vanderbilt": 420, "T2_GR_Ioannina": 50, "T2_IN_TIFR": 190, "T2_CN_Beijing": 165, "T1_FR_CCIN2P3": 700, "T2_US_Florida": 820, "T1_DE_KIT": 584, "T2_PT_NCG_Lisbon": 167, "T2_US_Wisconsin": 1660, "T2_HU_Budapest": 162, "T2_US_MIT": 1279, "T2_DE_RWTH": 802, "T2_PL_Swierk": 578}', 'teams': '["testbed-vocms0230"]'} result: None

but this is pulling work from cmsweb, not from LQ. So I think we're safe to merge it.

Seangchan, have you created the wmagent PR of this patch?

Just found it here #7174  without any description or link to this one. Bad job! :P


cache URL to cacheId()

Adding a simple layer to cache the mapping of SERVER_NAME+REQUEST_URI to
cacheId should yield quite a performance bump. This would allow most requests
to bypass the entire architecture, including checking filemtimes, settings, etc.

The TTL should probably be low so that source file edits won't be ignored too
long. Even very short TTLs should yield big gains under load.

This would be easiest built into the 2.2 refactoring, but hacking it into 2.1.4
would probably be worth it.

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 5 Oct 2010 at 6:58

Original comment by<EMAIL_ADDRESS>on 3 Feb 2011 at 10:55

Added labels: Priority-Medium
Removed labels: Priority-High


Attempt to make check for -A x64 flag more robust

As the version of the compilers used will change from time to time I think this might be a slightly more robust check to use.

This certainly works in the three cases we know about: our CI and the two cases encountered in the conda-forge deployment. Since they of the strings differed in only having "Win64" at the end, perhaps this is a suffix. But "NMake Makefiles" is completely different, and might be hiding whether the bit size is fixed or not.


Minor export fix & rename mistyped file

Renamed credetials.ts into credentials.ts
Fixed the way credentials are exported, since the previous way was yielding { credentials: { username: 'hi', password: 'there' } } instead of expected { username: 'hi', password: 'there' }, causing an exception in instaAutomation.ts
ignoring package-lock.json in git

This is awesome! 😍
Thanks for the contribution!


Set agent_factory in Vumi Bridge transport.

Currently the agent_factory is not set which means the Vumi Bridge transport is unable to make HTTP requests.

Ready for review.

:+1:


Adapt to starkent.js V5

Starknet js V5 will introduce a breaking change with BigInt instead of BN.
We'll need to update that when it's ready 🔥

done


add standard for the programming language to be used

Thank you @merkata, I have made some changes to cover:

Encouraging exploration and experimentation in other programming languages
Tying the recommended programming language to the ecosystem which is more general than what was originally proposed and
Added a clarifying note that some people might can even specialise down to just 1 language if they are focussing only on charms

@jdkandersson thanks for addressing the feedback - looking at the doc with the latest changes, it is clear and concise, very well done. I don't have anything to add at this point, really like it.

Here's some proposed wording, based on what we discussed yesterday:
## Programming Languages and Frameworks

We are a team primarily focused on developing charms, and we should follow the recommended tools and best practices of the ecosystem we work within (and help to improve them where appropriate). As a result, the primary programming language we work in is Python, and the primary framework we use to develop charms with is the [Operator Framework](https://github.com/canonical/operator).

We will also be working on older charms which may be written in older frameworks such as the [Reactive Framework](https://charmsreactive.readthedocs.io/en/latest/). In some cases we'll work on converting those to the Operator Framework but for small bug fixes or changes this may not be the case.

As a team we'll also be exposed to a much lesser extent to other programming languages as a result of working with specific applications that we deploy on Juju, or via small changes to Juju itself, which is written in Go. We also encourage exploration and experimentation in a range of programming languages depending on the interest of the individual as part of taking advantage of Canonical's annual training budget.

Let me know if you'd prefer it in a different format, or have any other thoughts on it.

Here's some proposed wording, based on what we discussed yesterday:
## Programming Languages and Frameworks

We are a team primarily focused on developing charms, and we should follow the
recommended tools and best practices of the ecosystem we work within (and help
to improve them where appropriate). As a result, the primary programming
language we work in is Python, and the primary framework we use to develop
charms with is the [Operator
Framework](https://github.com/canonical/operator).

We will also be working on older charms which may be written in older
frameworks such as the [Reactive
Framework](https://charmsreactive.readthedocs.io/en/latest/). In some cases
we'll work on converting those to the Operator Framework but for small bug
fixes or changes this may not be the case.

As a team we'll also be exposed to a much lesser extent to other programming
languages as a result of working with specific applications that we deploy on
Juju, or via small changes to Juju itself, which is written in Go. We also
encourage exploration and experimentation in a range of programming languages
depending on the interest of the individual as part of using Canonical's
annual training budget.

Let me know if you'd prefer it in a different format, or have any other thoughts on it.

Thank you Tom, I have included this and left in the first paragraph explaining the reason for needing the standard.


(IAC-593) Updates to handle OpenSSL Support for Viya 4 Monitoring

Changes
Update to support the upcoming release of sassoftware/viya4-monitoring-kubernetes. The monitoring/logging installation script now looks for the CERT_GENERATOR environment variable during the installation script to determine what tool it should use to generate certificates. The value for CERT_GENERATOR can either be "openssl" or "cert-manager"
Tests

Scenario
K8s Version
Cadence
V4_CFG_TLS_GENERATOR
V4M_VERSION
V4M_CERT & V4M_KEY
Logging and Monitoring Stack Accessible/Stabilized
Deployment Stabilized

1
1.23.5
Fast:2020
openssl
master (31b8307)
not set
yes
yes

2
1.23.5
Fast:2020
openssl
master (31b8307)
provided
yes
yes

3
1.23.5
Fast:2020
cert-manager
master (a4a6fec)
not set
yes
yes

4
1.23.5
Fast:2020
cert-manager
master (a4a6fec)
provided
yes
yes

See internal ticket for more details.

This PR has been approved, forgot to unmark this as draft.


Murmur3fix

CODEC-264: Fixed sign extensions.
Fix for both 32 and 128 bit hashes.
Additional test cases added including comments re: source of tests.

Coverage decreased (-0.07%) to 92.71% when pulling 141701313bc78fcc57c27cd1a96df7e10c688a84 on Claudenw:murmur3fix into f0e122da19684394123c85577405d6dc7cddbb98 on apache:master.

I got the original implementation from Apache Hive assuming it was correct.  See: https://github.com/apache/hive/blob/master/storage-api/src/java/org/apache/hive/common/util/Murmur3.java

while we are on the tabs or spaces topic I wonder if Apache commons needs to have a standard formatter for all projects. Similar to this: https://uima.apache.org/codeConventions.html
Where devs can just import it and we can enforce it with a Maven plugin.

I have one that was given to me for commons-collections
On Wed, Nov 6, 2019 at 6:21 PM Melloware<EMAIL_ADDRESS>wrote:

while we are on the tabs or spaces topic I wonder if Apache commons needs
to have a standard formatter for all projects. Similar to this:
https://uima.apache.org/codeConventions.html
Where devs can just import it and we can enforce it with a Maven plugin.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/commons-codec/pull/27?email_source=notifications&email_token=AASTVHUZ57HVONFMXI6A42TQSMDI3A5CNFSM4JIKUON2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDHQEPY#issuecomment-550437439,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AASTVHU55WROTGPT7L2YK2LQSMDI3ANCNFSM4JIKUONQ
.

--
I like: Like Like - The likeliest place on the web
http://like-like.xenei.com
LinkedIn: http://www.linkedin.com/in/claudewarren

Is there any way you could attach the formatter to this ticket or point me to where I can get it?

The one I got apparently is from Commons RNG but here it is:
On Thu, Nov 7, 2019 at 12:38 AM Melloware<EMAIL_ADDRESS>wrote:

Is there any way you could attach the formatter to this ticket or point me
to where I can get it?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/commons-codec/pull/27?email_source=notifications&email_token=AASTVHQ5OL5HZB3PWWRHRULQSNPOXA5CNFSM4JIKUON2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDIPSEI#issuecomment-550566161,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AASTVHVWTVLUXCNYIZWZC3TQSNPOXANCNFSM4JIKUONQ
.

--
I like: Like Like - The likeliest place on the web
http://like-like.xenei.com
LinkedIn: http://www.linkedin.com/in/claudewarren

Doesn't look like the attachment came through to GitHub

Each component in Apache Commons has it's own formatting rules :-(
We can start by storing canned configs in each component, for example:
https://gitbox.apache.org/repos/asf?p=commons-dbcp.git;a=blob;f=src/conf/eclipse/formatter.xml;h=e181b4b0e8c03b7d7480cab63d04d94caf383f86;hb=HEAD
Gary
On Thu, Nov 7, 2019 at 2:44 PM Melloware<EMAIL_ADDRESS>wrote:

Doesn't look like the attachment came through to GitHub
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/apache/commons-codec/pull/27?email_source=notifications&email_token=AAJB6NZFLZQ64YRZ3UQXTQTQSRVYTA5CNFSM4JIKUON2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDNSS4Q#issuecomment-551233906,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJB6N2HHIM325LZ5ERJTILQSRVYTANCNFSM4JIKUONQ
.

I like the idea the file should be stored with the project.  And I have used a Maven plugin that you can run the formatter as part of the build or a 1 time job to format the whole project.

As the author of the Eclipse formatting configuration in question I would state that I provided it for demo purposes to make formatting a bit easier on a PR. It is by no means a fire and forget configuration. I never use it on an entire file for commons code as it makes no allowances for changing preferences, for example on indentation for method parameters on new lines. Eclipse also mangles HTML in javadoc comments, particularly list elements . So it would not help with a big formatting inconsistency in the javadoc that is often seen across most shared codebases.
I use automated eclipse formatting on other projects with copious amounts of @formatter:off tags to prevent formatting code that has been laid out as easy to read. This litters a project with specific tags and I would be wary of doing this for a repository with a lot of contributors with different work set-ups for IDEs, etc.
I like automated tools to do mindless jobs. But code is formatted with the express purpose of being easy to read (compilers don't care) so at least leave the option to allow human formatting to overrule the formatter.
I prefer the option of using a very strict set of checkstyle rules. You can then maintain consistency without having to use an auto-formatter which may rearrange your carefully laid out array declarations, etc. A case in point is that checkstyle for this project should be set up to refuse tabs in source code which arrived in MurmurHash3 and its tests. This would be missed in a PR review as Github converts tabs to whitespace when showing the diff view.
I think a start point would be updating checkstyle to use all the latest rules. Just updating the checkstyle version is not enough, you need to pull in the new rules too.

@aherbert those are excellent points.  I think I was looking for something because @garydgregory and I want to format all the files in Commons Beanutils2 before its first release.  So we at least start out all of the files in a good state without having to hand fix 200 files.

Going forward, I think we should:

Store IDE formatting configs in Git
Encourage use of @formatter:off/on

Gary
On Fri, Nov 8, 2019 at 7:04 AM Melloware<EMAIL_ADDRESS>wrote:

@aherbert https://github.com/aherbert those are excellent points. I
think I was looking for something because @garydgregory
https://github.com/garydgregory and I want to format all the files in
Commons Beanutils2 before its first release. So we at least start out all
of the files in a good state without having to hand fix 200 files.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/commons-codec/pull/27?email_source=notifications&email_token=AAJB6NZOL654WAGJXFW5E7TQSVIUFA5CNFSM4JIKUON2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDQ6EDQ#issuecomment-551674382,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJB6N3A3NBSHTUWIZEJBOLQSVIUFANCNFSM4JIKUONQ
.

In the projects I work on at work we place the formatting configs for
various systems in a directory called "devtools" (historical reasons).
That directory includes a README.md that describes the shortcomings of the
various formatting config (e.g. Eclipse version does not do X, netbeans
version does not do Y, eclipse requires you enable Z directly from
properties, etc).  This works well for us.
Claude
On Fri, Nov 8, 2019 at 12:51 PM Gary Gregory<EMAIL_ADDRESS>wrote:

Going forward, I think we should:

Store IDE formatting configs in Git
Encourage use of @formatter:off/on

Gary
On Fri, Nov 8, 2019 at 7:04 AM Melloware<EMAIL_ADDRESS>wrote:

@aherbert https://github.com/aherbert those are excellent points. I
think I was looking for something because @garydgregory
https://github.com/garydgregory and I want to format all the files in
Commons Beanutils2 before its first release. So we at least start out all
of the files in a good state without having to hand fix 200 files.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/apache/commons-codec/pull/27?email_source=notifications&email_token=AAJB6NZOL654WAGJXFW5E7TQSVIUFA5CNFSM4JIKUON2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDQ6EDQ#issuecomment-551674382
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AAJB6N3A3NBSHTUWIZEJBOLQSVIUFANCNFSM4JIKUONQ
.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/commons-codec/pull/27?email_source=notifications&email_token=AASTVHTGW33XFKK2V2EDJ5TQSVODPA5CNFSM4JIKUON2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDRODYQ#issuecomment-551739874,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AASTVHT4YOZY3UJSDBESRETQSVODPANCNFSM4JIKUONQ
.

--
I like: Like Like - The likeliest place on the web
http://like-like.xenei.com
LinkedIn: http://www.linkedin.com/in/claudewarren

The latest attempt leaves the earlier code in place but deprecates it with a note about the sign extension error.  There is no change to the existing code (despite the change detected by git) except for:
1 the deprecated annotation.
2. the deprecated javadoc tag which says to use the newer version.
3. addition to the 128 bit  method javadocs that indicate the existence of the sign extension error.
New methods are named
hash32_x86and hash128_x64
Addition of UBYTE_MASK, UBYTE_LONG_MASK, and UINT_MASK.

Would someone please merge this?  The two errors  are in a file I did not touch and are in the base build.

There is a lot of new code in here.
IIUC the only method that was broken was the hash32 variant when the input had a number of bytes that was not a factor of 4. This was due to a missing mask to perform sign extension when processing the final 3 bytes. So actually for 1/4 cases of random byte lengths and 1/2 cases for strings it works anyway.
I think the 64-bit and 128-bit versions are not broken as masks are used in the final processing of the last bytes to convert byte to integer then cast to long:
@Test
public void testShift() {
    for (int i = 0; i < 256; i++) {
        byte bi = (byte) i;
        for (int j = 0; j < 8; j++) {
            int shift = j * 8;
            // Current method
            long shift1 = (long) (bi & 0xff) << shift;
            // Your new version
            long shift2 = (bi & 0xffL) << shift;
            Assert.assertEquals(shift1, shift2);
        }
    }
}

Why deprecate the 128-bit version and provide a whole new implementation that you have copied from elsewhere. Do the two methods produce the same results? Can you call both of them with a few thousand arrays of random bytes with different lengths.
If so then this is just bringing in a new implementation with a different name (hash128 -> hash128_x64).
All of the convenience methods for hash32 are not broken. An int and a long already have a factor of 4 for the bytes. The new methods are just name changes and exactly the same code. So you have a lot of duplication and no need for deprecation other than a name change.
I think the only methods to be deprecated are:
public static int hash32(final byte[] data, final int offset, final int length, final int seed)
(plus the 4 methods that call it)

IncrementalHash32.end()
(this can be deprecated in favour of `.end32x86()`)

You have added a hash for a CharSequence. Although this would be useful it is not in the scope of this PR which is fixing the broken 32-bit hash implementation.
Am I missing something here?

The 128 bit was broken, the 32 bit was not, the 64 bit is not covered in the original murmurhash implementation.
I brought across the implementation from Yonik Seeley and the tests that go with it.
I duplicated the existing 128 bit tests calling the new 128 hash and and renamed them with "_x86" in the middle (e.g. test128_Double became test128_x64_Double) to verify that all the old tests passed the new hash.
The "testCorrectValues_x64" (which I now realize is misnamed) verifies  that the 32_x86 and 128_x64 return the expected results when compared with the results of the original "C" implementation.
If you modify the testCorrectValues_x86 code to call the original hash32 and hash128 version you will see that the 128 version fails because of the sign extension that occurs early in the process.  (This is what I showed in the original bug) The effective change is at lines 1047 and 1048.
I provided new names for the standard hash methods as there are x86 and x64 versions of the hashes and they do not produce the same values.  The change of Hash32 to Hash32_x86 makes it clear which hash it is.  The change from Hash128 to Hash128_x64 does the same.
While the old Hash128 is incorrectly implemented it has been released and is in the wild, so we have to expect that some uses of the hash128 function can not change to hash128_x64.  To this end the hash128 functions are deprecated (but will probably not be removed) and new implementations are expected to use the hash128_x64 version.
The Hash32 methods are deprecated to refer to the new names.  Given that the code returns the same values one of the implementations could be removed and just call the other.

In the case of hash32 I would deprecate those that are known to be broken but not the ones that are not (i.e. the convenience methods). Then provide the new hash32x86 method and delegate calls to a single implementation to eliminate code duplication.
For the hash128x64 method this seems to be the easy solution:
// Current
public static long[] hash128(final byte[] data, final int offset, final int length, final int seed) {
    return hash128x64(data, offset, length, seed, seed);
}

// New
public static long[] hash128x64(final byte[] data, final int offset, final int length, final int seed) {
    return hash128x64(data, offset, length, seed & 0xffffffffL, seed & 0xffffffffL);
}

// Single implementation
private static long[] hash128x64(final byte[] data, final int offset, final int length,
        final long h1, final long h2) {
    ...

Would that work?

I think you have it backwards.  hash32 is not broken.
hash32 -> hash32_x64 to indicate the implementation in the name (deprecate hash32 infavor of the new name). (code can be coalesced around one implementation with hash32() delegating to hash32_x86() calls.
hash128 is broken.  However there are probably users that can not change from the broken implementation.  I think it is the Cassandra project has the same issue and has elected not to change their hash as it breaks all the existing data sets.  So hash128 deprecated with a note indicating the error in the code so that users can decide if they need to retain the broken one or not.
I will make the changes for Hash32() to call hash30_x86 methods.

There must be a difference between hash32 and hash32_x86 because when I change hash32() to
 public static int hash32(final byte[] data, final int offset, final int length, final int seed) {
     return hash32_x86( data, offset, length, seed );
}

The incremental hash test no longer passes.  it fails with
java.lang.AssertionError: Block size 1 expected:<2094498084> but was:<2002902189>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.apache.commons.codec.digest.MurmurHash3Test.testIncremental(MurmurHash3Test.java:176)
.
.
.

The hash32 method is the one with the sign extension error in the finalisation of any remaining bytes. It does not mask the byte with 0xff before the left shift.

and the 128 has a sign extension issue at the start where the original does not use a UINT_MASK., so both implementations have the error.

No need for a replacement class for IncrementalHash32. Only the end() method of the class should be deprecated. You can just add hash32x86() to match the static method name for the same computation.
Can you remove all the underscores from the method names please. That is not Java standard.

Your suggestion not replace Incrementalhash32 means that it will not be possible to use incrementalhash32 to generate hashes that match the old Hash32.  I think we need to be very clear about which methods work with which implementations and to my mind naming of the method is the way to do this.
We also need to preserve backward compatibility until such time as we make a major version change.

In your PR it looks like the start and add methods of IncrementalHash32 and IncrementalHash32_x86 are exactly the same. So just deprecate end in favour of a new method hash32x86. You can still use the class to generate the old flawed hash but now have a new finalisation method to compute the correct hash.
No need to duplicate the entire class just to change one method.

Are there any other comments on this pull request?  Any open issues?

All the deprecated hash32 method can just call hash32x86.
I think you should drop the new implementation of hash128. As I previously suggested you can create a private version that is fixed and call it with the two h1 and h2 values. The old broken method will not mask them to unsigned. This makes it very clear that the two implementations are basically the same, just the initialisation of the hashes from the seed is broken.
I had a quick test of the methods against Google Guava com.google.common.hash.Hashing.murmur3_32 and murmur3_128. They have chosen to return the values as 16 bytes that are little endian ordered. The output from the the new hash32x86 method matches. Unfortunately (for Google) their murmur3_128 algorithm has the sign extension error when setting the h1 and h2 from the seed. So their implementation matches the broken hash128 method. I did not do a formal check of the algorithm against MurmurHash3.cpp but checked it by eye. I think this link to the original source should be in the class header. This is the reference for the new implementations, even if adapted from Yonik Seeley.
The original MurmurHash3 code has an oddity in that the input bytes and returned output byte hash are read/written using the native byte order of the platform. This is little-endian for x86 and x86-64 intel architecture but nothing mandates that this is done. All the computations are done on 64-bit integers which are logically big endian. Perhaps we should state somewhere that the 128-bit hash is returned as two Java longs. To produce the same output as MurmurHash3.cpp would require endianness conversion as appropriate:
long[] hash128 = MurmurHash3.hash128x64("hello world");
byte[] bytes = ByteBuffer.allocate(16)
                         .order(ByteOrder.LITTLE_ENDIAN)
                         .putLong(hash128[0])
                         .putLong(hash128[1])
                         .array();

There is little in the Javadoc to explain why things are deprecated. All of the helper hash32 methods that act on primitives that are deprecated are not broken. This is just for a name change. The broken method is the one that processes byte[]. For hash128 the only method that is broken is one that processes byte[] and does not use the default seed. Sign extension of the default seed does not happen as it is positive. The javadoc should explain that the method is broken when a negative seed is used.
If you insist on a new implementation for hash32x86 then it may be more maintainable to copy that to the old hash32 method and unfix the code to have the bug. Otherwise I would leave hash32 as is and copy it to hash32x86 and fix the sign extension bug. Then add explicit comments in them both above the // tail section that this is/is not using the UBYTE_MASK to avoid sign extension in the tail processing.
I also still see a System.out.print in the test method doString. The whitespace formatting from testCorrectValues_x64 down is all wrong.
IncrementalHash32x86 has no comments on any public methods or the class. It uses raw constants for the final mix step when it can call fmix32. Same goes for hash32x86.
I would replace inlined rotations with Integer.rotateLeft for clarity.
The aim is to reduce code duplication and make this class more maintainable given that it is supporting the same algorithm with and without bugs. I see no point in having two implementations with variations in their style and code. The code in the two methods should be the same except for the bug. It should be very clear to a future maintainer that the code in hash32 and hash32x86 and IncrementalHash32x86 is the same, save for the bug in the finalisation of hash32 and the end function of IncrementalHash32. It should also be clear that hash128 and hash128x64 are the same except for the initialisation of the hash from the seed.

I've been considering the new method hash32x86(CharSequence data, ...
I do not think that we should enforce the UTF-8 encoding on the char sequence. A char is a 16-bit unsigned integer. A simpler approach would be to encode the chars as unsigned 16-bit integers. This can process blocks of data just as is done for the byte[] but in blocks of size 2 and not 4. Removing all the if statements and just blindly encoding everything would be simpler.
How this would effect most cases of ASCII text where the top 8 bits will be empty I do not know. If a user would like to hash using a character set then they should convert to bytes and encode that.
IMO the character set is not relevant. String.equals(Object) treats two Strings as equal if they have the same character in each position. The character set does not matter. It is a binary data comparison. I think hashing should be a binary data operation. It does not care what the data represents (i.e. the character set). The way Guava handles adding a char to a hasher is exactly this. It just puts in 2 * 8-bit bytes for each char. The AbstractHasher has these methods:
  @Override
  public Hasher putUnencodedChars(CharSequence charSequence) {
    for (int i = 0, len = charSequence.length(); i < len; i++) {
      putChar(charSequence.charAt(i));
    }
    return this;
  }

  @Override
  public Hasher putString(CharSequence charSequence, Charset charset) {
    return putBytes(charSequence.toString().getBytes(charset));
  }

So you either add chars as unencoded or you specify a character set which uses a byte conversion.
This makes me think we should change hashXXX(CharSequence, ...) to be documented as processing uncoded 16-bit chars and not add new hashXXX(String [, ...]) methods which are vague. The originals will have to stay but deprecated with an appropriate message. The choice of charsets can be left to the user to convert and use the appropriate byte[] methods.
If you prefer we can drop new methods with String and CharSequence and discuss on the mailing list. This PR can just fix the byte[] implementations and clarify naming conventions for the methods.

With a bit more time to look at the new CharSequence encoding method I found it to be broken. This test demonstrates how it does not handle bad code points correctly:
@Test
public void testHash32x86_UTF8_encoding_InvalidCodePoints() {
    // Since RFC 3629 (November 2003), the high and low surrogate halves used by
    // UTF-16 (U+D800 through U+DFFF) and code points not encodable by UTF-16
    // (those after U+10FFFF) are not legal Unicode values, and their UTF-8
    // encoding must be treated as an invalid byte sequence.
    String invalid = String.valueOf('\uD801');
    int seed = 0;

    // Correct answer
    byte[] bytes = invalid.getBytes(StandardCharsets.UTF_8);
    int h1 = MurmurHash3.hash32x86(bytes, 0, bytes.length, seed);

    // This is NOT EQUAL
    int h2 = MurmurHash3.hash32x86((CharSequence) invalid, 0, invalid.length(), seed);
    Assert.assertNotEquals("Did not expected the encoding to be correct", h1, h2);

    // This is what the encoding should do. Represent bad code points as unknown.
    int h3 = MurmurHash3.hash32x86((CharSequence) "?", 0, 1, seed);
    Assert.assertEquals("The invalid code point should be encoded as '?'", h1, h3);
}

I think we should leave charset encoding out of this.
I'm also confused as to why the default seed has been set at that value. This should be changed for the new methods.
IMO we should have the following new methods and class:
public static int hash32x86(byte[] data)
public static int hash32x86(byte[] data, int offset, int length)
public static int hash32x86(byte[] data, int seed)
public static int hash32x86(byte[] data, int offset, int length, int seed)

public static int hash128x64(byte[] data)
public static int hash128x64(byte[] data, int offset, int length)
public static int hash128x64(byte[] data, int seed)
public static int hash128x64(byte[] data, int offset, int length, int seed)

IncrementalHash32x86

The default seed for the new methods should be zero. This is what is done in Google Guava and the python library mmh3.
This removes adding new methods for primitives since these are not broken. They are just strangely configured helper methods. These should be documented correctly with their equivalent code and scheduled for removal e.g.
/**
 * Generates 32 bit hash from two longs with default seed value.
 * This is a helper method that will produce the same result as:
 *
 * <pre>
 * int seed = 104729;
 * int hash = hash32x86(ByteBuffer.allocate(16)
 *                                .putLong(l0)
 *                                .putLong(l1)
 *                                .array(), seed);
 * </pre>
 *
 * @param l0 long to hash
 * @param l1 long to hash
 * @return 32 bit hash
 * @deprecated This method to be removed in 2.0
 */
@Deprecated
public static int hash32(final long l0, final long l1) {

The hash64 methods should not be documented as "MSB 8 bytes of Murmur3 128-bit variant". They do not return the same value as and the 128-bit method since they neglect the second hash to be mixed into the first. This has would have an effect as it should be initialised with the default seed (which is not zero). The methods are just plain wrong and they also do not use little endian format for the equivalent calls via a ByteBuffer to the byte[] hash method.
The methods bloat the class. If removed in 2.0 and someone wants them back then this can be addressed then.
I can do documentation changes for all methods to be deprecated in master if you want and then you can rebase the PR to bring in the fixed hash implementations.
Thanks for your patience.

I do not think that we should enforce the UTF-8 encoding on the char sequence. A char is a 16-bit unsigned integer. A simpler approach would be to encode the chars as unsigned 16-bit integers.

It looks like this is originally mine, so I'll chime in... the purpose of this method is not a generic string hash, but an optimization for when one wants murmur3 over UTF-8 specifically.  This level of specification is needed for systems that need well defined hashes to enable matching in different languages (typically for things like routing in distributed systems.)

Hi @yonik, The code for the UTF-8 hash is nice. But I do not think it should be part of this PR. This PR is specifically trying to fix the current broken implementation of MurmurHash3 for byte[] due to some sign extension bugs. It should be simple but it turning into an epic.
I found a bug in your code where it does not support the correct interpretation of illegal characters which should be represented as an invalid byte sequence.
See the example above trying to encode String.valueOf('\uD801');
I think the UTF-8 encoding should be part of another PR and ideally fix the support for invalid characters.

@Claudenw I have created a separate ticket for the sign extension bug in hash32 [CODEC-267].
I fully documented what the class is currently doing. I rewrote the test class entirely to ensure that all the documentation for what the various methods are doing is tested. This tests hashing with random bytes from the full range of [0-255]. The reference data is from Python mmh3 which is based on the MurmurHash3 c++ code.
I created tests to show the sign extension error.
I then took your fixes for the hash32 bug and hash128 sign extension bug and implemented them in master. The changes log has been attributed to you.
Please can you verify that master passes any additional tests you have written for the PR for the methods with bugs and the new methods, hopefully, without bugs.
The new API has a minimal footprint with 4 new methods and a new IncrementHash32x86 class. I don't think all the other methods are warranted. See [CODEC-268] for a wish to deprecate most of them.

Alex,
Thank you for all your assistance and in getting a final fix in place.  It
passes all the tests I had.  Thank you.
Claude
On Thu, Nov 21, 2019 at 4:13 PM Alex Herbert<EMAIL_ADDRESS>wrote:

@Claudenw https://github.com/Claudenw I have created a separate ticket
for the sign extension bug in hash32 [CODEC-267].
I fully documented what the class is currently doing. I rewrote the test
class entirely to ensure that all the documentation for what the various
methods are doing is tested. This tests hashing with random bytes from the
full range of [0-255]. The reference data is from Python mmh3 which is
based on the MurmurHash3 c++ code.
I created tests to show the sign extension error.
I then took your fixes for the hash32 bug and hash128 sign extension bug
and implemented them in master. The changes log has been attributed to you.
Please can you verify that master passes any additional tests you have
written for the PR for the methods with bugs and the new methods,
hopefully, without bugs.
The new API has a minimal footprint with 4 new methods and a new
IncrementHash32x86 class. I don't think all the other methods are
warranted. See [CODEC-268] for a wish to deprecate most of them.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/commons-codec/pull/27?email_source=notifications&email_token=AASTVHSEMVVSIAB5SCF3XHLQU2XT5A5CNFSM4JIKUON2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE2YT6A#issuecomment-557156856,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AASTVHSUYLX5SKT2HLZDKUDQU2XT5ANCNFSM4JIKUONQ
.

--
I like: Like Like - The likeliest place on the web
http://like-like.xenei.com
LinkedIn: http://www.linkedin.com/in/claudewarren

Thanks for checking. Shall we close this PR?

Does commons-codec publish snapshots?  I don't find them on
https://repository.apache.org/content/repositories/snapshots/ nor any
mention of them on the commons codec page
https://commons.apache.org/proper/commons-codec/
Claude
On Sat, Nov 23, 2019 at 8:38 AM Claude Warren<EMAIL_ADDRESS>wrote:

Alex,
Thank you for all your assistance and in getting a final fix in place.  It
passes all the tests I had.  Thank you.
Claude
On Thu, Nov 21, 2019 at 4:13 PM Alex Herbert<EMAIL_ADDRESS>wrote:

@Claudenw https://github.com/Claudenw I have created a separate ticket
for the sign extension bug in hash32 [CODEC-267].
I fully documented what the class is currently doing. I rewrote the test
class entirely to ensure that all the documentation for what the various
methods are doing is tested. This tests hashing with random bytes from the
full range of [0-255]. The reference data is from Python mmh3 which is
based on the MurmurHash3 c++ code.
I created tests to show the sign extension error.
I then took your fixes for the hash32 bug and hash128 sign extension bug
and implemented them in master. The changes log has been attributed to you.
Please can you verify that master passes any additional tests you have
written for the PR for the methods with bugs and the new methods,
hopefully, without bugs.
The new API has a minimal footprint with 4 new methods and a new
IncrementHash32x86 class. I don't think all the other methods are
warranted. See [CODEC-268] for a wish to deprecate most of them.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/commons-codec/pull/27?email_source=notifications&email_token=AASTVHSEMVVSIAB5SCF3XHLQU2XT5A5CNFSM4JIKUON2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE2YT6A#issuecomment-557156856,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AASTVHSUYLX5SKT2HLZDKUDQU2XT5ANCNFSM4JIKUONQ
.

--
I like: Like Like - The likeliest place on the web
http://like-like.xenei.com
LinkedIn: http://www.linkedin.com/in/claudewarren

--
I like: Like Like - The likeliest place on the web
http://like-like.xenei.com
LinkedIn: http://www.linkedin.com/in/claudewarren

Currently codec is not building snapshots. I will raise this on the dev mailing list.

Nit: I think that in MurmurHash3Test line 740 it should use answers2 not
answers:
for (int i = 0; i < answers2.length; i++) {
not
for (int i = 0; i < answers.length; i++) {
On Thu, Nov 21, 2019 at 4:13 PM Alex Herbert<EMAIL_ADDRESS>wrote:

@Claudenw https://github.com/Claudenw I have created a separate ticket
for the sign extension bug in hash32 [CODEC-267].
I fully documented what the class is currently doing. I rewrote the test
class entirely to ensure that all the documentation for what the various
methods are doing is tested. This tests hashing with random bytes from the
full range of [0-255]. The reference data is from Python mmh3 which is
based on the MurmurHash3 c++ code.
I created tests to show the sign extension error.
I then took your fixes for the hash32 bug and hash128 sign extension bug
and implemented them in master. The changes log has been attributed to you.
Please can you verify that master passes any additional tests you have
written for the PR for the methods with bugs and the new methods,
hopefully, without bugs.
The new API has a minimal footprint with 4 new methods and a new
IncrementHash32x86 class. I don't think all the other methods are
warranted. See [CODEC-268] for a wish to deprecate most of them.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/commons-codec/pull/27?email_source=notifications&email_token=AASTVHSEMVVSIAB5SCF3XHLQU2XT5A5CNFSM4JIKUON2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE2YT6A#issuecomment-557156856,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AASTVHSUYLX5SKT2HLZDKUDQU2XT5ANCNFSM4JIKUONQ
.

--
I like: Like Like - The likeliest place on the web
http://like-like.xenei.com
LinkedIn: http://www.linkedin.com/in/claudewarren

Yes. I have separated them into tests for positive and negative seed.

Pull request objective fulfilled by other changes to master branch


Is this plugin actively maintained ?

Hey All
I wanted to use this plugin as part of our production setup.
Before starting the testing,  just wanted to confirm if the plugin is actively maintained as the last commit is done almost 3 years ago.

@rams3sh You're right. The plugin is not maintained anymore. It was part of the PoC several years ago.

Hey
Sure. Thanks for the quick response. Closing the issue.

No worries.
If someone wants to take the initiative and start maintaining the serverless plugin, - no blockers here as the project is supported by the open-source community in full.


More intelligent Safari handling

We can add logic to do properly handle Safari capabilities.

Add latest supported platform based on browser version requested
Add latest supported browser based on platform requested

Update, this second point *should already "just work" since we default to "latest" if it isn't otherwise specified.

The basic use case is if a user specifies "browserName": "Safari" they'll get an error since the default platform is Windows 10. The quick fix is to specify the latest version of Mac
The problem is that this isn't enough, because if the user also specifies "browserVersion": "8", the latest version of Mac will not work. So we'll need some kind of lookup table, and it will need to be done in a way that minimizes maintenance requirements.

Note that the logic for this needs to be handled from the start() method, not from setters or constructors.


Tag separate deployments to track all resource costs in individual deployments under same account

Hi,
Is there a current functionality to tag separate deployments of this tool/POC in order to track all/each resource costs associated with one deployment individually, under the same account?
I tried to use the CloudFormation default tagging options by activating them in my account. Not sure how to connect them to my existing deployment. I think I have to add it in the Cloud Formation template and make a new deployment with that change, as per the AWS best practices? Could this be a feature addition to the base repo? A tagging question under the step 5, 'npm run config' step?
One work around would be to have different deployments in different regions and filter the cost with that, from what I understand? Though not really scalable.
Thank you

Hi @nkay28 ,
CDK makes it easy to add Tags in bulk, see here: https://docs.aws.amazon.com/cdk/v2/guide/tagging.html#tagging-use
If the scope is the stack for example it should add the tag to itself and all its children.
You could try to add here (I have not tested this change)
https://github.com/aws-samples/aws-genai-llm-chatbot/blob/main/lib/aws-genai-llm-chatbot-stack.ts#L35
Tags.of(this).add('Project', props.config.prefix);
Note some resources would not be covered here. For example Bedrock on demand usage is not a Cloudformation resource and would not have a tag.
A possible option to isolate these different instances is to use multiple accounts:
https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/organizing-your-aws-environment.html

I was able to separate out the CF deployment in billing using the prefix option at deployment.
Thanks for the suggestions. I will try them out as I get into further weeds.


fwupd: 1.1.2 -> 1.1.3

https://github.com/hughsie/fwupd/blob/1.1.3/NEWS
Upgrade to 1.1.2 broke things but has known fix that should be landing soon--
this bump needs same fix so test appropriately. :)

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Fits CONTRIBUTING.md.

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: fwupd
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: fwupd
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Attempted: fwupd
Partial log (click to expand)
shrinking RPATHs of ELF executables and libraries in /nix/store/8yvqrjjsf0nyxdkwx8lgbpgxk5466nmi-fwupd-1.1.3-installedTests
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/8yvqrjjsf0nyxdkwx8lgbpgxk5466nmi-fwupd-1.1.3-installedTests
checking for references to /build in /nix/store/8yvqrjjsf0nyxdkwx8lgbpgxk5466nmi-fwupd-1.1.3-installedTests...
Wrapping program /nix/store/5y1yq7i7nmmk7x8iw294asf7hbds72b1-fwupd-1.1.3/bin/fwupdmgr
Wrapping program /nix/store/5y1yq7i7nmmk7x8iw294asf7hbds72b1-fwupd-1.1.3/bin/dfu-tool
Wrapping program /nix/store/5y1yq7i7nmmk7x8iw294asf7hbds72b1-fwupd-1.1.3/libexec/fwupd/fwupd
Wrapping program /nix/store/5y1yq7i7nmmk7x8iw294asf7hbds72b1-fwupd-1.1.3/libexec/fwupd/fwupdate
Wrapping program /nix/store/5y1yq7i7nmmk7x8iw294asf7hbds72b1-fwupd-1.1.3/libexec/fwupd/fwupdtool
/nix/store/5y1yq7i7nmmk7x8iw294asf7hbds72b1-fwupd-1.1.3

cc #48427  (the PR mentioned above).

Failure on aarch64-linux (full log)
Attempted: libxmlb
The following builds were skipped because they don't evaluate on aarch64-linux: fwupd
Partial log (click to expand)
Build dir: /build/source/build
Build type: native build
WARNING: Unknown command line options: "auto_features"
This will become a hard error in a future Meson release.

meson.build:1:0: ERROR:  Meson version is 0.46.1 but project requires >=0.47.0.

A full log can be found at /build/source/build/meson-logs/meson-log.txt
builder for '/nix/store/b4617nnc7062152r5aqfni6pnpb1h9sc-libxmlb-0.1.4.drv' failed with exit code 1
error: build of '/nix/store/b4617nnc7062152r5aqfni6pnpb1h9sc-libxmlb-0.1.4.drv' failed

Failure on x86_64-linux (full log)
Attempted: fwupd, libxmlb
Partial log (click to expand)

meson.build:1:0: ERROR:  Meson version is 0.46.1 but project requires >=0.47.0.

A full log can be found at /build/source/build/meson-logs/meson-log.txt
builder for '/nix/store/pbqjgm0rqczsz41lxlf6c3xaczqg4m8h-libxmlb-0.1.4.drv' failed with exit code 1
copying path '/nix/store/3wzy2k9ib6dfy78axhfqhc61qzgbvd7h-glib-networking-2.56.0' from 'https://nix-cache.s3.amazonaws.com'...
copying path '/nix/store/006fjpp880j2s334fgl00vq64w5bblkj-libproxy-0.4.15-dev' from 'https://nix-cache.s3.amazonaws.com'...
copying path '/nix/store/nicvm41jdz2mrfdmif7dznh2mmvk4a3y-glib-networking-2.56.0-dev' from 'https://nix-cache.s3.amazonaws.com'...
cannot build derivation '/nix/store/f690zqmk60v92bgz42alz6z46rgbj40x-fwupd-1.2.0.drv': 1 dependencies couldn't be built
error: build of '/nix/store/f690zqmk60v92bgz42alz6z46rgbj40x-fwupd-1.2.0.drv', '/nix/store/pbqjgm0rqczsz41lxlf6c3xaczqg4m8h-libxmlb-0.1.4.drv' failed

Failure on x86_64-darwin (full log)
Attempted: libxmlb
The following builds were skipped because they don't evaluate on x86_64-darwin: fwupd
Partial log (click to expand)
Build dir: /private/tmp/nix-build-libxmlb-0.1.4.drv-0/source/build
Build type: native build
WARNING: Unknown command line options: "auto_features"
This will become a hard error in a future Meson release.

meson.build:1:0: ERROR:  Meson version is 0.46.1 but project requires >=0.47.0.

A full log can be found at /private/tmp/nix-build-libxmlb-0.1.4.drv-0/source/build/meson-logs/meson-log.txt
builder for '/nix/store/y05nrrhj6nb80h6zp7lf07nchbgnalmi-libxmlb-0.1.4.drv' failed with exit code 1
error: build of '/nix/store/y05nrrhj6nb80h6zp7lf07nchbgnalmi-libxmlb-0.1.4.drv' failed

@GrahamcOfBorg build fwupd
@GrahamcOfBorg test fwupd
(not sure if tests pass currently, but let's see...)

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: fwupd
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: fwupd
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: tests.fwupd
Partial log (click to expand)
Cannot nix-instantiate `tests.fwupd' because:
error: while evaluating 'recursiveUpdate' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/attrsets.nix:415:26, called from /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/attrsets.nix:148:28:
while evaluating 'recursiveUpdateUntil' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/attrsets.nix:384:37, called from /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/attrsets.nix:416:5:
while evaluating 'zipAttrsWith' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/attrsets.nix:347:21, called from /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/attrsets.nix:394:8:
while evaluating 'zipAttrsWithNames' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/attrsets.nix:332:33, called from /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/lib/attrsets.nix:347:27:
while evaluating the attribute 'fwupd' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/nixos/tests/all-tests.nix:77:3:
while evaluating 'handleTestOn' at /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/nixos/tests/all-tests.nix:19:33, called from /var/lib/gc-of-borg/nix-test-rs-20/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/grahamc-aarch64-community-20/nixos/tests/all-tests.nix:77:11:
access to path '/nix/store/6xd0ci2zzb3c0b11ki33vhk8wq1maljr-grahamc-aarch64-community-20' is forbidden in restricted mode

libxmlb 1.1.5 is out, do we want to include it as well?

Yup.

Failure on x86_64-linux (full log)
Attempted: fwupd
Partial log (click to expand)

meson.build:1:0: ERROR:  Meson version is 0.46.1 but project requires >=0.47.0.

A full log can be found at /build/source/build/meson-logs/meson-log.txt
builder for '/nix/store/35mqrbxnlah72s9jn7h8i7sgl1insn3f-libxmlb-0.1.4.drv' failed with exit code 1
copying path '/nix/store/3c5yqwrirqr0vijz88bji9iz0lpph5r7-glib-networking-2.56.0' from 'https://cache.nixos.org'...
copying path '/nix/store/w995pgjd7pn498x9vx3rb0z08261nhad-libproxy-0.4.15-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/s9av84m8jdvfv73d3xszx5riza52a9d3-glib-networking-2.56.0-dev' from 'https://cache.nixos.org'...
cannot build derivation '/nix/store/bjck5v5aw5d1mc7w4hz7psxz52lnqgsg-fwupd-1.2.0.drv': 1 dependencies couldn't be built
error: build of '/nix/store/bjck5v5aw5d1mc7w4hz7psxz52lnqgsg-fwupd-1.2.0.drv' failed

We need to wait for https://github.com/NixOS/nixpkgs/pull/50876

No attempt on x86_64-linux (full log)
The following builds were skipped because they don't evaluate on x86_64-linux: tests.fwupd
Partial log (click to expand)
cannot link '/nix/store/.links/0c0vwmjcswdzm234zjffdlp45c9hk9yn8i4xcysnmb4id34h64bv' to '/nix/store/2yvljc4wa6jxcvwicp4acajfx1pswpfv-gleber-bastion/pkgs/os-specific/linux/spl/default.nix': No space left on device
error: while evaluating 'recursiveUpdate' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:415:26, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:148:28:
while evaluating 'recursiveUpdateUntil' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:384:37, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:416:5:
while evaluating 'zipAttrsWith' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:347:21, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:394:8:
while evaluating 'zipAttrsWithNames' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:332:33, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:347:27:
while evaluating anonymous function at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/lists.nix:113:41, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/lib/attrsets.nix:347:46:
while evaluating the attribute 'fwupd' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/nixos/tests/all-tests.nix:77:3:
while evaluating 'handleTestOn' at /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/nixos/tests/all-tests.nix:19:33, called from /var/lib/gc-of-borg/nix-root/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/gleber-bastion/nixos/tests/all-tests.nix:77:11:
access to path '/nix/store/2yvljc4wa6jxcvwicp4acajfx1pswpfv-gleber-bastion' is forbidden in restricted mode

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: fwupd
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Well FWIW bumping to 0.1.5 (and adding a python3 dependency, apparently) and rebasing onto staging -- produces a bunch of build errors that didn't seem obvious.  My internet is being wonky or I'd be linking a branch/commit as well as logs, but anyway just a heads-up I suppose.
Oh hey managed to get the logs to gist anyway: https://gist.github.com/dtzWill/6412a806c9993ef01e1b4be4e20c938d

Success on x86_64-linux (full log)
Attempted: fwupd
Partial log (click to expand)
shrinking RPATHs of ELF executables and libraries in /nix/store/ma4k76jk1ldl7c36narhvcv3xpvzxk8z-fwupd-1.1.2-installedTests
strip is /nix/store/rpbg8gmqxhz8g61p1plz5d2srs84pvmv-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/ma4k76jk1ldl7c36narhvcv3xpvzxk8z-fwupd-1.1.2-installedTests
checking for references to /build in /nix/store/ma4k76jk1ldl7c36narhvcv3xpvzxk8z-fwupd-1.1.2-installedTests...
Wrapping program /nix/store/f65863mkcdjvr1pgkwfsmk0ajgm6jgh3-fwupd-1.1.2/bin/fwupdmgr
Wrapping program /nix/store/f65863mkcdjvr1pgkwfsmk0ajgm6jgh3-fwupd-1.1.2/bin/dfu-tool
Wrapping program /nix/store/f65863mkcdjvr1pgkwfsmk0ajgm6jgh3-fwupd-1.1.2/libexec/fwupd/fwupd
Wrapping program /nix/store/f65863mkcdjvr1pgkwfsmk0ajgm6jgh3-fwupd-1.1.2/libexec/fwupd/fwupdate
Wrapping program /nix/store/f65863mkcdjvr1pgkwfsmk0ajgm6jgh3-fwupd-1.1.2/libexec/fwupd/fwupdtool
/nix/store/f65863mkcdjvr1pgkwfsmk0ajgm6jgh3-fwupd-1.1.2

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: fwupd
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Needs docbook_xsl docbook_xml_dtd_43 in nativeBuildInputs.

Failure on x86_64-darwin (full log)
Attempted: libxmlb
The following builds were skipped because they don't evaluate on x86_64-darwin: fwupd
Partial log (click to expand)
Build dir: /private/tmp/nix-build-libxmlb-0.1.5.drv-0/source/build
Build type: native build
WARNING: Unknown command line options: "auto_features"
This will become a hard error in a future Meson release.

meson.build:1:0: ERROR:  Meson version is 0.46.1 but project requires >=0.47.0.

A full log can be found at /private/tmp/nix-build-libxmlb-0.1.5.drv-0/source/build/meson-logs/meson-log.txt
builder for '/nix/store/q5zm9nrss9qplmf3a4wvrryz83w819wv-libxmlb-0.1.5.drv' failed with exit code 1
error: build of '/nix/store/q5zm9nrss9qplmf3a4wvrryz83w819wv-libxmlb-0.1.5.drv' failed

(still needs meson bump that's on staging, as earlier mentioned)

Failure on x86_64-darwin (full log)
Attempted: libxmlb
The following builds were skipped because they don't evaluate on x86_64-darwin: fwupd
Partial log (click to expand)
Build dir: /private/tmp/nix-build-libxmlb-0.1.5.drv-0/source/build
Build type: native build
WARNING: Unknown command line options: "auto_features"
This will become a hard error in a future Meson release.

meson.build:1:0: ERROR:  Meson version is 0.46.1 but project requires >=0.47.0.

A full log can be found at /private/tmp/nix-build-libxmlb-0.1.5.drv-0/source/build/meson-logs/meson-log.txt
builder for '/nix/store/q5zm9nrss9qplmf3a4wvrryz83w819wv-libxmlb-0.1.5.drv' failed with exit code 1
error: build of '/nix/store/q5zm9nrss9qplmf3a4wvrryz83w819wv-libxmlb-0.1.5.drv' failed

Failure on x86_64-linux (full log)
Attempted: fwupd, libxmlb
Partial log (click to expand)
A full log can be found at /build/source/build/meson-logs/meson-log.txt
builder for '/nix/store/yfa1daxgjag8bfi0sb9fv61ajiy49rch-libxmlb-0.1.5.drv' failed with exit code 1
building '/nix/store/sapbdikllbidvdpf92gya995qb4i3ih2-fwupd-1.2.1.tar.xz.drv'...

trying https://people.freedesktop.org/~hughsient/releases/fwupd-1.2.1.tar.xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1287k  100 1287k    0     0   959k      0  0:00:01  0:00:01 --:--:--  959k
cannot build derivation '/nix/store/2qr1p6ygh3zl42idjzbjnzmdsrvm5gp6-fwupd-1.2.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/2qr1p6ygh3zl42idjzbjnzmdsrvm5gp6-fwupd-1.2.1.drv', '/nix/store/yfa1daxgjag8bfi0sb9fv61ajiy49rch-libxmlb-0.1.5.drv' failed

Failure on aarch64-linux (full log)
Attempted: libxmlb
The following builds were skipped because they don't evaluate on aarch64-linux: fwupd
Partial log (click to expand)
Build dir: /build/source/build
Build type: native build
WARNING: Unknown command line options: "auto_features"
This will become a hard error in a future Meson release.

meson.build:1:0: ERROR:  Meson version is 0.46.1 but project requires >=0.47.0.

A full log can be found at /build/source/build/meson-logs/meson-log.txt
builder for '/nix/store/4mib19izh87fs8i9yp6iyirbr53brkz4-libxmlb-0.1.5.drv' failed with exit code 1
error: build of '/nix/store/4mib19izh87fs8i9yp6iyirbr53brkz4-libxmlb-0.1.5.drv' failed

Failure on aarch64-linux (full log)
Attempted: libxmlb
The following builds were skipped because they don't evaluate on aarch64-linux: fwupd
Partial log (click to expand)
Build dir: /build/source/build
Build type: native build
WARNING: Unknown command line options: "auto_features"
This will become a hard error in a future Meson release.

meson.build:1:0: ERROR:  Meson version is 0.46.1 but project requires >=0.47.0.

A full log can be found at /build/source/build/meson-logs/meson-log.txt
builder for '/nix/store/4mib19izh87fs8i9yp6iyirbr53brkz4-libxmlb-0.1.5.drv' failed with exit code 1
error: build of '/nix/store/4mib19izh87fs8i9yp6iyirbr53brkz4-libxmlb-0.1.5.drv' failed

Failure on x86_64-darwin (full log)
Attempted: libxmlb
The following builds were skipped because they don't evaluate on x86_64-darwin: fwupd
Partial log (click to expand)
Build dir: /private/tmp/nix-build-libxmlb-0.1.5.drv-0/source/build
Build type: native build
WARNING: Unknown command line options: "auto_features"
This will become a hard error in a future Meson release.

meson.build:1:0: ERROR:  Meson version is 0.46.1 but project requires >=0.47.0.

A full log can be found at /private/tmp/nix-build-libxmlb-0.1.5.drv-0/source/build/meson-logs/meson-log.txt
builder for '/nix/store/q5zm9nrss9qplmf3a4wvrryz83w819wv-libxmlb-0.1.5.drv' failed with exit code 1
error: build of '/nix/store/q5zm9nrss9qplmf3a4wvrryz83w819wv-libxmlb-0.1.5.drv' failed

Failure on x86_64-linux (full log)
Attempted: fwupd, libxmlb
Partial log (click to expand)
Build type: native build
WARNING: Unknown command line options: "auto_features"
This will become a hard error in a future Meson release.

meson.build:1:0: ERROR:  Meson version is 0.46.1 but project requires >=0.47.0.

A full log can be found at /build/source/build/meson-logs/meson-log.txt
builder for '/nix/store/yfa1daxgjag8bfi0sb9fv61ajiy49rch-libxmlb-0.1.5.drv' failed with exit code 1
cannot build derivation '/nix/store/ylm8qvaxy2fwm6iy4ygpgbw7mrc1carz-fwupd-1.2.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/yfa1daxgjag8bfi0sb9fv61ajiy49rch-libxmlb-0.1.5.drv', '/nix/store/ylm8qvaxy2fwm6iy4ygpgbw7mrc1carz-fwupd-1.2.1.drv' failed

@GrahamcOfBorg build fwupd

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: fwupd
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: fwupd
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Attempted: fwupd
Partial log (click to expand)
shrinking RPATHs of ELF executables and libraries in /nix/store/3j6lnhncma5a5zd2irz5pzcrmjzarlpn-fwupd-1.2.1-installedTests
strip is /nix/store/6s8dkfj3irwv52cap2hd9k7gaddklfs1-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/3j6lnhncma5a5zd2irz5pzcrmjzarlpn-fwupd-1.2.1-installedTests
checking for references to /build/ in /nix/store/3j6lnhncma5a5zd2irz5pzcrmjzarlpn-fwupd-1.2.1-installedTests...
Wrapping program /nix/store/vxcp502g1r6vjjh0kz0hdp81s2qfgk5k-fwupd-1.2.1/bin/dfu-tool
Wrapping program /nix/store/vxcp502g1r6vjjh0kz0hdp81s2qfgk5k-fwupd-1.2.1/bin/fwupdmgr
Wrapping program /nix/store/vxcp502g1r6vjjh0kz0hdp81s2qfgk5k-fwupd-1.2.1/libexec/fwupd/fwupdate
Wrapping program /nix/store/vxcp502g1r6vjjh0kz0hdp81s2qfgk5k-fwupd-1.2.1/libexec/fwupd/fwupd
Wrapping program /nix/store/vxcp502g1r6vjjh0kz0hdp81s2qfgk5k-fwupd-1.2.1/libexec/fwupd/fwupdtool
/nix/store/vxcp502g1r6vjjh0kz0hdp81s2qfgk5k-fwupd-1.2.1

I guess we should squash the commits.

@jtojnar @dtzWill
I've resolved the conflicts, corrected the commit msg, etc.
I hope you don't mind :sweat_smile:

@GrahamcOfBorg test fwupd

Failed locally for me with
Installing test firmware...
Decompressing…         [***************************************]
failed to add public keys to gpgme-v1.12.0 keyring: failed to load /etc/pki/fwupd/GPG-KEY-Linux-Foundation-Metadata: No such file or directory
FAIL: fwupd/fwupdmgr.test (Child process exited with code 1)
error: Attempt to end element "line" with "nest" tag at /nix/store/vpm0bbgi88jj52sq9kfnpif5jrmqxcr9-nixos-test-driver/lib/perl5/site_perl/Logger.pm line 53.
Attempt to end element "line" with "nest" tag at /nix/store/vpm0bbgi88jj52sq9kfnpif5jrmqxcr9-nixos-test-driver/lib/perl5/site_perl/Logger.pm line 53.
cleaning up
killing machine (pid 597)
Attempt to end element "line" with "logfile" tag at /nix/store/vpm0bbgi88jj52sq9kfnpif5jrmqxcr9-nixos-test-driver/lib/perl5/site_perl/Logger.pm line 27.
END failed--call queue aborted, <__ANONIO__> line 98.
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
builder for '/nix/store/3ac86f46d96zyjpsvqa5nyq0adsij4jl-vm-test-run-fwupd.drv' failed with exit code 22
error: build of '/nix/store/3ac86f46d96zyjpsvqa5nyq0adsij4jl-vm-test-run-fwupd.drv' failed

Test succeeds locally, and I double checked filesInstalledToEtc.

Something to keep in mind for next update: https://github.com/hughsie/fwupd/pull/865

I think everything looks good. Lets go for it :+1:


dovecot: 2.3.2 -> <IP_ADDRESS>

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/dovecot/versions.
meta.description for dovecot is: '"Open source IMAP and POP3 email server written with security primarily in mind"'.

Checks done (click to expand)

built on NixOS
Warning: no invocation of /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/bin/dsync had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/bin/doveconf had a zero exit code or showed the expected version
Warning: no invocation of /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/bin/doveadm had a zero exit code or showed the expected version
/nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/bin/dovecot passed the binary check.
1 of 4 passed binary check by having a zero exit code.
0 of 4 passed binary check by having the new version present in output.
found <IP_ADDRESS> with grep in /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>
directory tree listing: https://gist.github.com/50cf4ba76c0b69c8b19d86e00828f0b8
du listing: https://gist.github.com/42a5eb0f745a588bb83008758425cb5f

Test this build with Cachix (click to expand)

One time setup in nixpkgs Git checkout:
cachix use r-ryantm
git remote add r-ryantm https://github.com/r-ryantm/nixpkgs.git

Test this build:
git fetch r-ryantm && git checkout 05e44640423587d9954392334ddcd04eca9f796e && nix-shell --pure -I nixpkgs=. -p dovecot

cc @viric @peti @rickynils @fpletz for testing.

Success on x86_64-linux (full log)
Attempted: dovecot
Partial log (click to expand)
shrinking /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/lib/dovecot/lib20_expire_plugin.so
shrinking /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/sbin/dovecot
gzipping man pages under /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/share/man/
strip is /nix/store/4qvrxzxa535y8304mk195x50b6p9607d-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/lib  /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/libexec  /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/bin  /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/sbin
patching script interpreter paths in /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>
/nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/libexec/dovecot/decode2text.sh: interpreter directive changed from "/bin/sh" to "/nix/store/8zkg9ac4s4alzyf4a8kfrig1j73z66dw-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>...
moving /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/sbin/* to /nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>/bin
/nix/store/xy0wkg563qbwf9j473giihhyx81b46z1-dovecot-<IP_ADDRESS>

Success on x86_64-darwin (full log)
Attempted: dovecot
Partial log (click to expand)
make[2]: Leaving directory '/private/tmp/nix-build-dovecot-<IP_ADDRESS>.drv-0/dovecot-<IP_ADDRESS>/doc'
make[1]: Leaving directory '/private/tmp/nix-build-dovecot-<IP_ADDRESS>.drv-0/dovecot-<IP_ADDRESS>/doc'
post-installation fixup
gzipping man pages under /nix/store/8mvx2w4l6alfjw0hmclwj14agzcbx6x3-dovecot-<IP_ADDRESS>/share/man/
strip is /nix/store/7ddbq63v97nk8gkbf7gcsfmby37h6gbl-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/8mvx2w4l6alfjw0hmclwj14agzcbx6x3-dovecot-<IP_ADDRESS>/lib  /nix/store/8mvx2w4l6alfjw0hmclwj14agzcbx6x3-dovecot-<IP_ADDRESS>/libexec  /nix/store/8mvx2w4l6alfjw0hmclwj14agzcbx6x3-dovecot-<IP_ADDRESS>/bin  /nix/store/8mvx2w4l6alfjw0hmclwj14agzcbx6x3-dovecot-<IP_ADDRESS>/sbin
patching script interpreter paths in /nix/store/8mvx2w4l6alfjw0hmclwj14agzcbx6x3-dovecot-<IP_ADDRESS>
/nix/store/8mvx2w4l6alfjw0hmclwj14agzcbx6x3-dovecot-<IP_ADDRESS>/libexec/dovecot/decode2text.sh: interpreter directive changed from "/bin/sh" to "/nix/store/q2wqq1k20v8kc3vckapqf5nws30brnni-bash-4.4-p23/bin/sh"
moving /nix/store/8mvx2w4l6alfjw0hmclwj14agzcbx6x3-dovecot-<IP_ADDRESS>/sbin/* to /nix/store/8mvx2w4l6alfjw0hmclwj14agzcbx6x3-dovecot-<IP_ADDRESS>/bin
/nix/store/8mvx2w4l6alfjw0hmclwj14agzcbx6x3-dovecot-<IP_ADDRESS>

Success on aarch64-linux (full log)
Attempted: dovecot
Partial log (click to expand)
shrinking /nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>/bin/doveconf
shrinking /nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>/bin/doveadm
gzipping man pages under /nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>/share/man/
strip is /nix/store/0pjsgkxz0rp5baycq5sp2s72lrr5q9sg-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>/lib  /nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>/libexec  /nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>/bin  /nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>/sbin
patching script interpreter paths in /nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>
/nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>/libexec/dovecot/decode2text.sh: interpreter directive changed from "/bin/sh" to "/nix/store/p0vy17dp9jk2mvqsxsqnb14s3797lay7-bash-4.4-p23/bin/sh"
checking for references to /build in /nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>...
moving /nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>/sbin/* to /nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>/bin
/nix/store/vq888jrpnkg5r7av61sm387hggkrv4cj-dovecot-<IP_ADDRESS>

@GrahamcOfBorg test dovecot

Success on x86_64-linux (full log)
Attempted: tests.dovecot
Partial log (click to expand)
machine: exit status 1
syncing
machine: running command: sync
machine: exit status 0
test script finished in 16.97s
cleaning up
killing machine (pid 593)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/arvmxgwdzl4bm2y780fsv2x9rllig2vr-vm-test-run-dovecot

Success on aarch64-linux (full log)
Attempted: tests.dovecot
Partial log (click to expand)
machine: exit status 1
syncing
machine: running command: sync
machine: exit status 0
test script finished in 29.11s
cleaning up
killing machine (pid 627)
vde_switch: EOF on stdin, cleaning up and exiting
vde_switch: Could not remove ctl dir '/build/vde1.ctl': Directory not empty
/nix/store/p4xwixx33crnbx5v1rhlq61dgqfq6k7q-vm-test-run-dovecot


No error: hash-map object expected but Map object provided

I get it that everything is an object - but "Objects as maps" is a special type that even got its own syntax.
Below function fn expects such an "object as map", but when I call the function with a Map as parameter there is no error. The wrong kind of object-map does produce an error.
Try it.
// @flow
'use strict';

const fn = (services: {[string]: Function}): void => {
    //...
};

// Okay - Error.
const m1 = {'foo': false};
fn(m1);

// NO ERROR???
const m2 = new Map();
fn(m2);

// NO ERROR???
const m3: Map<number, string> = new Map();
fn(m3);

You can work around this by changing your function annotation to this:
// @flow

type $PlainObj<T> = T & { +constructor?: empty }
const fn = (services: $PlainObj<{[string]: Function}>): void => {};

fn({foo:fn}); // passes
fn({foo:false}) // error
fn(new Map()); // error

Hmm, this is presumably caused by the Object typings having the line [key:any]: any;. Hard to see how that can changed to disallow this but allow the existing ergonomics of working with plain objects.

Now errors.


EtcdBackup.management.cattle.io "------" not found

Rancher 2.2.1
4 node baremetall
Error after restore snapshot
EtcdBackup.management.cattle.io "c-25d48/c-25d48-rs-x5mhc" not found

Cluster is now in update status all the time.
Edit and Save cluster not work

@aleksey005 is it 2.2.1 fresh install, or an upgraded setup? If upgraded, what version of Rancher have you upgraded from?
Can you also provide a Rancher log taken after you hit "Edit and Save" resulting in "Cluster spec conversion error"

@alena1108  upgrade from version 2.1.7 (all new release i upgrade correctly)
I am surprised and most likely I looked badly in the documentation, but did not find how to see the logs. Please tell me how to do it?

This file does not exist, because I have already deleted it from the S3 cloud, but the rancher still tries to download it. How to make him change his mind?
#docker logs --tail 100 kind_kirch
...
2019/04/03 07:23:59 [INFO] Provisioning cluster [c-25d48]
2019/04/03 07:23:59 [INFO] Restoring cluster [c-25d48] from backup
2019/04/03 07:23:59 [ERROR] ClusterController c-25d48 [cluster-provisioner-controller] failed with : etcdBackup.management.cattle.io "c-25d48/c-25d48-rs-x5mhc" not found
2019/04/03 07:24:02 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:45094: i/o timeout
2019/04/03 07:24:17 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:24:27 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:46112: i/o timeout
2019/04/03 07:24:43 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:24:53 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:47062: i/o timeout
2019/04/03 07:25:09 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:25:19 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:47996: i/o timeout
2019/04/03 07:25:35 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:25:45 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:48948: i/o timeout
2019/04/03 07:26:00 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:26:10 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:49884: i/o timeout
2019-04-03 07:26:24.906457 I | mvcc: store.index: compact 12996022
2019-04-03 07:26:25.026314 I | mvcc: finished scheduled compaction at 12996022 (took 112.37078ms)
2019/04/03 07:26:26 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:26:36 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:50828: i/o timeout
2019/04/03 07:26:52 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:27:02 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:51778: i/o timeout
2019/04/03 07:27:14 [ERROR] API error response 422 for PUT /v3/clusters/c-25d48?_replace=true. Cause: json: cannot unmarshal string into Go struct field RotateCertificates.services of type []string
2019/04/03 07:27:17 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:27:27 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:52722: i/o timeout
2019/04/03 07:27:43 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:27:53 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:53674: i/o timeout
2019/04/03 07:28:09 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:28:19 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:54620: i/o timeout
2019/04/03 07:28:34 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:28:44 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:55560: i/o timeout
2019/04/03 07:29:00 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:29:10 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:56506: i/o timeout
2019/04/03 07:29:26 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:29:36 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:57448: i/o timeout
2019/04/03 07:29:52 [INFO] Handling backend connection request [c-25d48:m-37b0224f2574]
2019/04/03 07:29:57 [ERROR] API error response 422 for PUT /v3/clusters/c-25d48?_replace=true. Cause: json: cannot unmarshal string into Go struct field RotateCertificates.services of type []string
2019/04/03 07:30:02 [INFO] error in remotedialer server [400]: read tcp <IP_ADDRESS>:443-><IP_ADDRESS>:58390: i/o timeout

Can anyone suggest that this is an intractable problem and I should reinstall Rancher? Now I can’t manage the cluster at all. It is still in an update state.

@aleksey005

This file does not exist, because I have already deleted it from the S3 cloud, but the rancher still tries to download it

Rancher operates with an assumption that the snapshot created by rancher, should be managed by rancher only. To restore, can you try:

remove corresponding snapshot record from rancher (either via UI or api)
try restoring to the snapshot that does exist on s3

@alena1108 There are no local snapshots. I deleted them, as well as those that were on S3 (10 GB in total), because I thought that they were not substantial. In fact, I came to the conclusion that the goal of deployment should be quickly repeated and, accordingly, the cluster should be developed for each microservice of its own. Rancher does not fail and continues to work, well, what I did for all management services with a web interface is domain names. I'll try rancher to update in this state.

After update 2.2.2 - no result

After clik Save - Error

@aleksey005 can you share the output of "View as yaml" for your cluster?

@alena1108 If you need access, I can provide it on request.
addon_job_timeout: 30
authentication:
  strategy: "x509"
bastion_host:
  ssh_agent_auth: false
ignore_docker_version: true
#
#   # Currently only nginx ingress provider is supported.
#   # To disable ingress controller, set `provider: none`
#   # To enable ingress on specific nodes, use the node_selector, eg:
#      provider: nginx
#      node_selector:
#        app: ingress
#
ingress:
  provider: "nginx"
kubernetes_version: "v1.13.5-rancher1-2"
monitoring:
  provider: "metrics-server"
#
#   # If you are using calico on AWS
#
#      network:
#        plugin: calico
#        calico_network_provider:
#          cloud_provider: aws
#
#   # To specify flannel interface
#
#      network:
#        plugin: flannel
#        flannel_network_provider:
#          iface: eth1
#
#   # To specify flannel interface for canal plugin
#
#      network:
#        plugin: canal
#        canal_network_provider:
#          iface: eth1
#
network:
  options:
    flannel_backend_type: "host-gw"
  plugin: "flannel"
restore:
  restore: false
  snapshot_name: "c-25d48:c-25d48-rs-zb8pv"
rotate_certificates:
  ca_certificates: false
#
#      services:
#        kube-api:
#          service_cluster_ip_range: <IP_ADDRESS>/16
#        kube-controller:
#          cluster_cidr: <IP_ADDRESS>/16
#          service_cluster_ip_range: <IP_ADDRESS>/16
#        kubelet:
#          cluster_domain: cluster.local
#          cluster_dns_server: <IP_ADDRESS>
#
services:
  etcd:
    backup_config:
      enabled: true
      interval_hours: 12
      retention: 6
      s3_backup_config:
        access_key: "-----------"
        bucket_name: "-----------"
        endpoint: "-----------"
        region: "-----------"
        secret_key: "-----------"
    creation: "12h"
    extra_args:
      election-timeout: "5000"
      heartbeat-interval: "500"
    retention: "72h"
    snapshot: false
  kube-api:
    always_pull_images: false
    pod_security_policy: false
    service_node_port_range: "30000-32767"
  kubelet:
    fail_swap_on: false
ssh_agent_auth: false
#
#   # Rancher Config
#
docker_root_dir: "/var/lib/docker"
enable_cluster_alerting: true
enable_cluster_monitoring: true
enable_network_policy: false
local_cluster_auth_endpoint:
  enabled: false
name: "-----------"

@aleksey005

The configuration is not correct, probably due to certificate rotation error.

Just to confirm that you didn't call rotateCertificate? Also all the cluster updates that you've done, have you done it using Rancher UI fields, or View as Yaml ->edit -> Save?
To restore the setup, try removing these lines from the yaml file, and click save:
rotate_certificates:
  ca_certificates: false

@alena1108
I did press the rotateCertificate button (10 times if not more) when an error occurred. I performed all actions with the cluster through Rancher UI, for almost half a year everything was fine. I started making requests through the API when the cluster broke.
rotate_certificates:    ca_certificates: false
I set the value to "true", but did not delete this line.
I had to reinstall the cluster again (especially since it began to work slowly, and now after reinstallation everything works quickly). If you need a snapshot file of the old cluster for improved diagnostics of what suddenly stopped working there, can I send it to you?
Now everything is working fine.

@aleksey005

If you need a snapshot file of the old cluster for improved diagnostics of what suddenly stopped working there, can I send it to you?

Sure. Also it would help to get the exact steps to reproduce if the issue happens again.

@aleksey005 closing the issue for now; could you reopen once you hit it again with the steps to reproduce?

@alena1108 ok. i send access e-mail for snapshots.


TST: Make events and metrics suites modules.

These directories were not being picked up by the test runner.

Coverage increased (+1.0%) to 87.716% when pulling 22d3673a1e673ab70738a4948a6528599ae0d191 on run-tests-modules-that-were-being-ignored into 07e0e5625c679adbbd3d8f485b48938164af8dbe on master.


[FALSE-POSITIVE] https://gowc.us Incorrectly marked as Phishing on Phishtank or OpenPhish?

Domains or links
Please list any domains and links listed here which you believe are a false positive.
More Information
How did you discover your web site or domain was listed here?

Website was hacked
Incorrectly marked as Phishing on Phishtank or OpenPhish?

Have you requested removal from other sources?
Please include all relevant links to your existing removals / whitelistings.
Additional context
Add any other context about the problem here.
:exclamation:
We understand being listed on a Phishing Database like this can be frustrating and embarrassing for many web site owners. The first step is to remain calm. The second step is to rest assured one of our maintainers will address your issue as soon as possible. Please make sure you have provided as much information as possible to help speed up the process.
Send a Pull Request for faster removal
Users who understand github and creating Pull Requests can assist us with faster removals by sending a PR to mitchellkrogza/phishing repository, on the falsepositive.list file

https://github.com/mitchellkrogza/phishing/blob/main/falsepositive.list
Please include the same above information to help speed up the whitelisting process.

Don't see any domains here ~Closing
/close


[core] Measure JVM Behavior

When comparing the various databases it's critical to determine how the YCSB JVM is behaving regarding thread count, memory usage, garbage collection, etc. Java provides some useful tools out of the box so I'd like to do the following:

Add a core param like "measurejvm" that enables the JVM tracking
Track:

max/min thread count in the Status thread
max/min memory used count in the Status thread
System load avg in the Status thread (from the JVM's perspective at least, not going to call any system utilities)
gc count in Status thread to track GCs per execution
final gc count for each pool (depends on the JVM)
Threads and mem just before each client cleanup (to see resource usage before closing down clients in case users don't have the Status thread running)
Hopefully track full GCs using the GC MXBeans

So far it looks pretty straightforward. We can slap a GC notification consumer on the GC beans then poll the JVM as needed for the other stats.

It sounds like an investment in extra monitoring utils, all of which available externally in different forms on different platforms. Not sure it would be worth including the extra code in this project. If there's a clean external module (platform independent, maintained by other people, well established) you could use that would be nice.

It would be really nice to get some system stats while running the benchmarks, but that would be completely platform dependent like you said. It would still be good to look at doing that, see if there's a module that would call the proper utilities based on the system it was installed on.
For the JVM metrics though, Java includes all of the calls you need for high-level details (memory, threads, GCs) and I think those are a good enough start for YCSB for now. Anything else you'd like to see measured?

anything else going to happen here beyond #719?

@manolama @busbey  looked at the merge, not clear: why is a secondary reporting thread required? why not use the StatusThread?

Looks like this was addressed via #719
@nitsanw if you are interested in addressing changes due to #719 can you open a new issue?


Firebase Auth suddenly stopped resulting in an error.

Hi there. I am using this package and it failed on the server all of a sudden.
Resulting in an error

Error creating resource: [message] fopen(https://oauth2.googleapis.com/token): failed to open stream: Connection timed out\n

       $auth = (new Factory)
            ->withHttpClientConfig([
                'debug' => true
        ])
        ->withServiceAccount(Storage::get('JSON_FILE_PATH'))
        ->createAuth();

        try {
            //dd('+91-'.substr($phone, 0, 2).'-'.substr($phone, 2, 9));
            $user = $auth->getUserByPhoneNumber('+91-'.substr($phone, 0, 2).'-'.substr($phone, 2, 9));
            dd($user);

        }
        catch(UserNotFound $e){
            //dd("hello");
            return ['status'=>false,'message'=>$e->getMessage()];
        }

Kindly help. Thank you.

Done. Did it by updating the curl extension of php and restarting the server


1-tuple of union should be the same as union of 1-tuples

Search Terms
Union of singletons vs singleton of union, one-element array, single-value array, singleton.
Intersection of 1-tuples should be the same as 1-tuple of the intersection.
Suggestion
As a 1-tuple, or single-element array, contains only one wrapped value, the exact same type operations that are applied to them also apply to their elements. That is, saying "a 1-tuple containing either A or B" is the same as saying "either a 1-tuple containing A, or a 1-tuple containing B". This applies both to type unions and type intersections.
This all should be true in TypeScript:
[A|B|C] === [A|B] | [C] === [A] | [B] | [C]
[A|B] & [C] === [(A|B)&C] === ([A]|[B])&[C]
// where A === B means both A is assignable to B and B is assignable to A

[A|B|C] ⊂ Array<A|B|C>
[A|B|C] ⊂ A[] | B[] | C[]
// where A ⊂ B means A is assignable to B, but B is not assignable to A

Use Cases
type value = string | number;
type array = string[] | number[];

function(a: value | array)
{
  const arr = Array.isArray(a) ? a : [a]; // this should work
}

Examples
See Use Cases, I couldn't come up with another example that would be different enough to be worth mentioning.
Checklist
My suggestion meets these guidelines:

[x] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[x] This wouldn't change the runtime behavior of existing JavaScript code
[x] This could be implemented without emitting different JS based on the types of the expressions
[x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
[x] This feature would agree with the rest of TypeScript's Design Goals.

This is at best only true under a full unification algorithm. As-is, this would be a substantial breaking change since:
const arr1: [string] | [number] = ["0" as any];
const arr2: [string | number] = ["0" as any];

const k1 = arr1.filter(e => e === "0");
const k2 = arr2.filter(e => e === "0");

You're right, didn't think of that. Still I think the code in Use Cases should work, I'm just not sure how to achieve that reasonably. Could we, for example, infer the type of 1-tuple literals to be either union-of-tuple or tuple-of-union depending on the context?
declare let value: string | number;
let a = [value]; // [string | number] as before
let b: [string | number] = [value]; // works
let c: [string] | [number] = [value]; // also works

Or is the idea dumb on a fundamental level and should I abandon it and rather summon as any every time I'd use something like that?

The code in "Use Cases" already works?
Anyway I think the "right fix" is a sort of combinatorial assignability reasoning that considers all possible expansion forms of a source type and sees if they have a corresponding valid assignment target. IOW in theory [string | number, string | number] should be assignable to [string, string] | [number, number] | [string, number] | [number, string].

Okay, I did too much posting and not enough testing :)
The code I had problem with was this:
function fn(patterns: string | string[] | RegExp | RegExp[]): void
{
    // I have to uncomment the assertion or else it throws
    if (!Array.isArray(patterns)) patterns = [patterns] // as [string]|[RegExp];
}

But it seems that I'd better close this issue and open a more specific one about that precise problem.

The inferred type of [patterns] says this is allowed:
function fn(patterns: string | string[] | RegExp | RegExp[]): void
{
    if (!Array.isArray(patterns)) {
        // k: Array<string | RegExp>
        const k = [patterns];
        // therefore these are both valid
        k.push("");
        k.push(/expr/);

        // Writes a heterogeneous array where a homogeneous one is expected
        patterns = k;
    }
}

Somewhat awkwardly, this is legal 🙃
        patterns = typeof patterns === "string" ? [patterns] : [patterns];

This is probably fixed by https://github.com/Microsoft/TypeScript/pull/30779 since a union of tuples should be a discriminated union.