Enable postgres checksums

PR to postgres repo: https://github.com/neondatabase/postgres/pull/122
Resolve neondatabase/cloud#536

Also I want to notice that checksums can be useful for ... storage manager architecture. Having checksums we may implement in-place updates for image layers. In this case checksums can  be used to enforce page consistency after failure. Still not quite clear where to get previous (correct) page image if page is corrupted. But in-place updates can significantly reduce write amplification.

This branch has conflicts that must be resolved

@ololobus , did you plan to resolve them?
alternatively, @lubennikovaav can pick it up
please discuss with her

This branch has conflicts that must be resolved

@ ololobus , did you plan to resolve them?

@stepashka, I rebased it already, but new conflicts arrived. No problem with rebasing again, of course.
The only reason why I still haven't merged it yet is that @hlinnaka hasn't liked this approach, if I understood him correctly.
In this PR we do all checksum checks in the redo process: for incoming WAL records and pages. It means that we don't have a feedback channel for pageserver to figure out that some WAL record or page is damaged. IIRC, @hlinnaka's idea was that we anyway trust pageserver <-> redo Postgres communication channel, so we can check checksum on the pageserver side / in the rust code.
So I wanted to move CRC checks into pageserver code, but haven't started it yet. Or we want to merge it as is and change later? As far I can see, pageserver starts redo process per each tenant, so one damaged page won't affect others and we will be able to investigate.
Another option is that I can remove these checks from redo process, but only keep stamping CRC in pageserver and redo process, so checks only happen on the compute side.

So I wanted to move CRC checks into pageserver code, but haven't started it yet.

Please notice, that we are already calculating checksums at page server side (for FPI), so this code is already implemented in Rust. So it seems to be not difficult to add this checksum calculation to get_page_at_lsn...

I've removed checksum checking from redo loop. Now we do:

stamp checksum on page in the WAL redo loop
stamp checksum on page when we grab it from FPI during WAL ingest in pageserver
compute verifies checksum of every page after GetPage@LSN (that's the vanilla path)
verify checksum of WAL records received from safekeeper (done already in the WalStreamDecoder::poll_decode)

The only missing part is checksum checking in the pageserver, when it reads layer files from local disk / S3 back into memory. There are two issues for doing that https://github.com/neondatabase/neon/issues/987 and https://github.com/neondatabase/neon/issues/1083
So if no objections (cc @hlinnaka @kelvich @knizhnik), I'd merge this compute-related part of checksums now and we can add pageserver files checksum later.
IIUC, when we merge this, old tenants should continue to work as control file in their basebackup doesn't have data checksums turned on. Yet, we do a fresh initdb per each redo process startup. So for old tenants their WAL redo process will stamp checksum, but compute just won't check it

So if no objections (cc @hlinnaka @kelvich @knizhnik), I'd merge this compute-related part of checksums now and we can add pageserver files checksum later.

I do not have objections, but I wan to notice that enabling checksums actually implies wal_log_hints. Yes, we still have wal_log_hints=on  by default,but we have discussed that it dramatically increased WAL size (~3 times) and it is safe to disable it.
But with enabled checksums there is no chance to turn off wal logging of hint bits. The only positive  effect is that there is no need to discuss whether itis safe to disable wal_log_hints or not.

Just tested locally that enabling checksums doesn't affect existing tenants. They still have disabled checksums and work as before.
So I'm going to merge this PR, we can iterate on it and add missing parts as mentioned above.

So does this patch change the number of FPI's from compute nodes?

Enabled data checksums effective set wal_log_hints=on

Can we change this behavior in postgres? IIUC wal_log_hints=on would dramatically affect performance and storage size. Is it correct that in our scenario we are safe without FPIs when CRC is enabled?

First of all, right now we have wal_log_hints=on by default.
It really cause increase of WAL size about 3 times in some scenarios.  I have tried to add special WAL records for updating hint bits, but looks like it cause negative effect in some cases (because hint bits are set for each record, so using FPI vacuum can set hint bits for all records on the page and it may be more efficient than generating separate WAL record for each heap record).
Postgres implies wal_log_hints in case of enabled checksums because it needs to restore page checksum in case of failure (as faras there is not warranty that page checksum and hints bits are updated atomically). In Neon we can guarantee some atomicity, so looks like we really do not need to force FPI in this case.

First of all, right now we have wal_log_hints=on by default.

In the cloud we have wal_log_hints=off for quite a while:
https://github.com/neondatabase/cloud/blob/f168b3f64f756e1baae46ea541530e41e10f228a/goapp/internal/model/cluster.go#L466-L467

I replaced XLogHintBitIsNeeded definition:
-#define XLogHintBitIsNeeded() (DataChecksumsEnabled() || wal_log_hints)
+#define XLogHintBitIsNeeded() (wal_log_hints)

and set wal_log_hints=off by default.
Regression tests are passed.
So looks like we can enable checksums without necessity to perform FPI for hints.

I replaced XLogHintBitIsNeeded definition:
-#define XLogHintBitIsNeeded() (DataChecksumsEnabled() || wal_log_hints)
+#define XLogHintBitIsNeeded() (wal_log_hints)

Yes, I did the same and all tests passed in CI. On first run got a failure in test_wal_restore https://app.circleci.com/pipelines/github/neondatabase/neon/6451/workflows/53113797-18ca-471f-b802-2c7bc1ba59e2/jobs/64971/tests#failed-test-0, but this doesn't look related and test recovered on re-run

I've modified page checksum code in pageserver in two ways:

Refactored pg_checksum_page to work in a similar way as C Postgres implementation, i.e. if checksum is already present on page it doesn't affect the computation. I did a clone only of that 32-bit chunk where checksum is set, so I assume that performance shouldn't be affected. Also added some unit tests to check that it behaves as expected
Added a per-tenant config option data_checksums to don't affect old tenants. It's false by default, so we don't check checksums for old tenants, where they are not present. It's set to true for all new tenants explicitly, though

@knizhnik @lubennikovaav please, have a look on the two latest commits:

https://github.com/neondatabase/neon/pull/1185/commits/5a0703de1ace4b778bdb52b956ef3a3968550616
https://github.com/neondatabase/neon/pull/1185/commits/2127df13ad749b8a3cfdf86f8d891d574d6a91dd

@LizardWizzard you can have a look on the last one, where per-tenant config is modified :)

As far as I remember the main @hlinnaka concern against this patch was that enabling postgres checksums doesn't protect all data: both delta and image layers contains more data except page images themselves. For example delta layer contains B-Tree. WAL records themselves are protected by CRC, but Value and ZenithWalRecord (which are serialized) - not. So looks like we really protect only half of data.

So looks like we really protect only half of data.

With this patch we protect two important paths for data pages and WAL records:

network failures between pageserver and compute
and on-disk data corruption on the pageserver side

Having a single checksum per layer file doesn't seem to be a good idea, as only one page could be corrupted. So we need a more fine-grained checksum and remaining parts of the data could be protected separately.

It is not true. pg_checksum is applied to the whole page, including checksum field

I mean that I changed your pg_checksum_page rust version to do the same. It sets zeroes to the checksum field and calculates the checksum. So it's not needed to do it twice in the page_set_checksum. I also added a test, which checks that having a checksum on the pages doesn't affect the calculation, which passes fine. Do I miss something?

So looks like we really protect only half of data.

With this patch we protect two important paths for data pages and WAL records:
* network failures between pageserver and compute

* and on-disk data corruption on the pageserver side

It is not completely protecting data from disk failure. If corruption affect B-Tree page, then Postgres page-level cheksums will not help to detect such error.

Having a single checksum per layer file doesn't seem to be a good idea, as only one page could be corrupted. So we need a more fine-grained checksum and remaining parts of the data could be protected separately.

File level checksums are really unusable, but not because to be able to calculate it we need to load all file in memory. So we need checksums for individual B-Tree pages.
May be we need our own checksum mechanism. But it is not clear where to store this checksum. as far as postgres page header already contains field for checksum, it seems to be natural to also use it and implement postgres-compatible checksum calculation algorithm. But I am not sure.

Added safety notes, unsafe where needed, criterion benchmark. Confirmed @knizhnik result that having an if in a loop slows it down by ~3 times. With a bit of refactoring put it down to ~700ns again.

As discussed moving to draft.


v2.10.1 backmerge

This PR:

backmerges v2.10.1 release

Remember NOT to squash this PR

Travis automatic deployment:
https://pr1268--safereact.review.gnosisdev.com/app

@fernandomg it has to be done in a separate branch and then merged into dev


!fraids sends an unbounded number of messages

You can post a raid to the faction any number of times, and then !fraids will spam

No more use in fixing this


Now retrieve link attributes and display them into dialog box :)

Hello,
We now retrieve link data values and display them into the dialog box.
I thing code need some refactoring maybe next step...
:)
Nico

Hi Nico, thanks for doing this. However, when implementing your new changes to retrieve link attributes, I'm getting some error's.
Uncaught SyntaxError: Unexpected token )
                    $linkTarget.find('option').each(function(i, obj)){
Chrome Dev Tools places the cursor between the two closing brackets after obj on that line. I couldn't work out what's going on there.


Bulk bugs

Where I going to Errors tab where is exception:
Error: SQLSTATE[42702]: Ambiguous column: 7 ERROR: column reference "id" is ambiguous LINE 1: select "id", "error_id", "session_id", "path_id", "updated_a... ^ (SQL: select "id", "error_id", "session_id", "path_id", "updated_at" from "tracker_log" left join "tracker_errors" on "tracker_log"."error_id" = "tracker_errors"."id" where "tracker_log"."updated_at" >= 2017-02-07 07:13:31 and "tracker_log"."updated_at" <= 2017-02-08 07:13:31 and "error_id" is not null order by "created_at" desc, "tracker_errors"."code" asc limit 10 offset 0)

Also please change length of "source" column in tracker_referers  from varchar(255) to type TEXT, because referer from facebook adv throws error
String data, right truncated: 7 ERROR: value too long for type character varying(255) (SQL: insert into "tracker_referers" ("url", "host", "domain_id", "medium", "source", "search_terms_hash", "updated_at", "created_at") values (https://l.facebook.com/l.phpl.facebook.comu=https%3A%2F%2Fxxxxxxxx%2F%3Futm_source%3Dtarget_fb%26utm_campaign%3Dtraffic%26utm_medium%3Dcpc%26utm_term%3DKZ-18plus-RUlang&h=ATOffpYwCBg0bKb-5Iinn_XLztIj0cGe3DzRuqUt8vZVK5W74RQCIkN3Gcfff6XZ4nVJvpx47img05YVJ62aiprh-KdK4JNppZ0Cxj-d4Kg, 17, social, Facebook, xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx, 2017-02-08 08:50:29, 2017-02-08 08:50:29, ?) returning "id")

lang/en/tracker.php
'no_days'                  => ':count Day|Days',
should be changed to
'no_days'                  => ':count Day|:count Days',
no_years too

tracker_queries, change column query to text
Error: SQLSTATE[22001]: String data, right truncated: 7 ERROR: value too long for type character varying(255) (SQL: insert into "tracker_queries" ("query", "updated_at", "created_at") values (urlwww_google_essa=t|rct=j|q=|esrc=s|source=web|cd=3|ved=0CDQQFjAF|url=http://robo.cash/2017-02-13 02:34:27utm_source=google-adwords&utm_medium=search&utm_campaign=Test-invest-Latv&utm_term=Procent&utm_content=%D0%BF%D0%BE%D0%BB%D0%BE%D0%B6%D0%B8%D1%82%D1%8C%20%D0%B4%D0%B5%D0%BD%D1%8C%D0%B3%D0%B8%20%D0%BF%D0%BE%D0%B4%20%D0%BF%D1%80%D0%BE%D1%86%D0%B5%D0%BD%D1%82%D1%8B|ei=ZLegWOSsLanVTuGfAQ|usg=AFQjCNF_bvZQAbqCPjn-fw5j1BlY0u7I2w, 2017-02-13 02:34:27, ?) returning "id")

tracker_referers change url to text
Error: SQLSTATE[22001]: String data, right truncated: 7 ERROR: value too long for type character varying(255) (SQL: insert into "tracker_referers" ("url", "host", "domain_id", "medium", "source", "search_terms_hash", "updated_at", "created_at") values (http://www.google.lv/urlwww.google.lvsa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CEEQFjAB&url=http%3A%2F%2Frobo.cash%2F%3Futm_source%3Dgoogle-adwords%26utm_medium%3Dsearch%26utm_campaign%3DTest-invest-Latv%26utm_term%3DKuda%26utm_content%3D%25D0%25BA%25D1%2583%25D0%25B4%25D0%25B0%2520%25D0%25B2%25D0%25BB%25D0%25BE%25D0%25B6%25D0%25B8%25D1%2582%25D1%258C%2520%25D0%25B4%25D0%25B5%25D0%25BD%25D1%258C%25D0%25B3%25D0%25B8%2520%25D0%25BF%25D0%25BE%25D0%25B4%2520%25D0%25BF%25D1%2580%25D0%25BE%25D1%2586%25D0%25B5%25D0%25BD%25D1%2582%25D1%258B&ei=I7egWJqdCau3lgbEEA&usg=AFQjCNFu79iRg7k2IMrpwyPviGHamjhtow, 13, search, Google, da39a3ee5e6b4b0d3255bfef95601890afd80709, 2017-02-12 19:27:32, 2017-02-12 19:27:32, ?) returning "id")

tracker_agents name to text
Error: SQLSTATE[22001]: String data, right truncated: 7 ERROR: value too long for type character varying(255) (SQL: insert into "tracker_agents" ("name", "browser", "browser_version", "updated_at", "created_at") values (Mozilla/5.0 (iPhone; CPU iPhone OS 9_3_4 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Mobile/13G35 [FBAN/FBIOS;FBAV/<IP_ADDRESS>.70;FBBV/40764466;FBRV/0;FBDV/iPhone7,2;FBMD/iPhone;FBSN/iPhone OS;FBSV/9.3.4;FBSS/2;FBCR/Carrier;FBID/phone;FBLC/ru_RU;FBOP/5], Facebook, 66.0.0, 2017-02-11 00:39:37, 2017-02-11 00:39:37) returning "id")

ERROR: duplicate key value violates unique constraint "tracker_agents_name_unique"
DETAIL: Key (name)=(Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0) already exists.

Also please change length of "source" column in tracker_referers  from varchar(255) to type TEXT, because referer from facebook adv throws error

No updates


proposal: Add GetClientCAs to tls.Config

tls.Config provides a GetCertificate function for providing TLS certificates dynamically. I suggest to add a GetClientCAs function to provide the same for the ClientCAs field.
Rationale: On a server the ClientCAs field is used for client certificate authentication but to my knowledge it isn't possible to extend the list of client certificates at runtime without interruption of existing connections (restart service or listener) since x509.CertPool is a struct and not safe for use by multiple go routines. A GetClientCAs function would also mirror the GetCertificate function.
I have written a reverse proxy http://github.com/eBay/fabio for which I've added the dynamic reloading of TLS certificates without restart and would like to provide the same functionality for the client cert authentication.
I'm willing to write the change if this is something that could be accepted. Target would be Go 1.8 obviously.

CC @agl

Ping @agl.

FWIW, I've implemented the patch yesterday to see how difficult it would be and it looks like it is simple. In essence, I've added a GetClientCAs func() *x509.CertPool function to tls.Config and updated clone(), the handshake, the test and api/next.txt. ~20 lines with comments. The question remains whether this is the right approach.

You need tests too, which I imagine will be more than 20 lines.

True. So far I've only updated the clone() test. But I could factor the new decision logic out into a separate function and write a simple test for that instead of writing a full integration test. Not sure what the preferred approach is.

You'll probably want something closer to an integration test than a single unit test. For things like this, it's too easy for a unit test to be misleading.

OK, so far I was not successful finding a test for the ClientCAs field and its current behavior in the server handshake which I could mimic/enhance but I'll keep digging. Otherwise, I'll create one.

@bradfitz Is there a way to run a specific test with go tool dist test ? Right now I'm running go tool dist test -v go_test:crypto/tls to run the entire crypto/tls suite but I'd like to run only my TestClientCAs test if possible.
Also, how can I provide the -update parameter as suggested in crypto/tls/handshake_test.go? Working around this by setting the default to true for now.

The same way as normal Go tests:
$ go test -v -run=YourNewTest crypto/tls

Works - of course ... I was using go from another go1.7 directory which then failed :/
Thx for the quick reply.

OK, so I have a patch with the changes and a full integration test which is an addition to the existing tests. This involved adding a new option to generate_cert.go to create a cert which contains x509.ExtKeyUsageClientAuth and replacing the test certificates in handshake_server_test.go since the new test needs to verify the client certificate and the old test certificates do not support that. Since I'm not a crypto expert a review would be nice :)
The tests check three cases:

Only ClientCAs
Only GetClientCAs()
Prefer GetClientCAs() over ClientCAs

I've generated the updated testdata/* files with the homebrew version OpenSSL 1.0.2h  3 May 2016 of openssl on OSX but I don't imagine that this is the reference implementation that the crytpo/tls test refer to.
@bradfitz @agl How do we proceed?

Submitted a CL with the code: https://go-review.googlesource.com/#/c/28773/
This is my first on this project so pls bear with me.

Changed the signature of GetClientCAs() to GetClientCAs(clientHello *ClientHelloInfo) in the CL to match GetCertificate() and to provide a mechanism to return different CA pools for different server names.

@bradfitz @agl ping

Pung. Left comments.

Fixed except for one consistency thing. See comments

This is very related to #15707 and #15699
Perhaps instead of adding a dozen hooks GetFoo hooks, we should have one for clients and one for servers which returns a new *tls.Config as a function of the ServerHello or ClientGoo, and use the returned config's fields before the base config's fields.
That is, instead of your:
type Config struct {
    ClientCAs *x509.CertPool
    GetClientCAs func(*ClientHelloInfo) *x509.CertPool
}

We could do:
type Config struct {
    ClientCAs *x509.CertPool

    // GetClientConfig returns server TLS configuration specialized for
    // the incoming user. The current supported fields are: ClientCAs.
    // If a supported field is its zero value, the base *tls.Config value is used
    // instead.
    GetClientConfig func(*ClientHelloInfo) *Config
}

/cc @dpiddy @FiloSottile for thoughts.

Yes, please. That is what I was thinking in https://github.com/golang/go/issues/15699#issuecomment-229772406. I had it also return error to mirror tls.Config.GetCertificate but maybe it is not required.

And as I said there, too, I think to make it truly useful we'd need some more info added to ClientHelloInfo.

@dpiddy, whoops, I totally missed that. An error might be nice. If you're doing a database/network lookup to find the config for that incoming connection and fail, it might be better to close the connection than accept it.

I think to make it truly useful we'd need some more info added to ClientHelloInfo.

@dpiddy, like what?

Again as I said in https://github.com/golang/go/issues/15699#issuecomment-229772406 my immediate need was for ALPN info so I can conditionally choose whether to serve, say, http/2 to a client.

Yes, that is better. Do we want to resolve it recursively since the returned *Config could have another GetConfig() function? I guess not but maybe there is a use-case.
Also, since we need to maintain GetCertificate() I suggest this fallback order:
GetConfig() -> GetCertificate() -> Config

As for extending ClientHelloInfo:
type ClientHelloInfo struct {
	...

	// ALPNProtocols ...
	ALPNProtocols []string

	// what else ?
}

I can update the patch later to see what this would look like.

On the same note: What if the returned *Config has a GetCertificate() function set? Ignore or call?

@magiconair as @bradfitz said I think it would be good to make clear that only certain items on the Config returned by GetConfig will be respected. Then the recursion and ordering issues can be avoided.
Maybe having ALPN info on the ClientHello isn't absolutely necessary for what I'm trying to do. I could offer h2 or not and the client can accept it or not if it wants. Hopefully I'm thinking that through correctly.
Also, @bradfitz, my last comment went out too quickly and ended up sounding short. That wasn't what I intended, sorry about that. Thanks for helping move this along!

@bradfitz @danp I've added an implementation for the suggested GetClientConfig approach in https://golang.org/cl/30752. Since it takes a different approach I've created a separate change. See big comment on approach for patch set 3.

@agl @bradfitz Shall I submit the patch I've added in the comment of https://golang.org/cl/30790 for replacing hs.c.config, c.config and config in crypto/tls/handshake_server.go with just c.config as a separate CL? This is inconsistent in the current code and I found it a bit confusing.

If you left a comment on that CL and @agl hasn't submitted it yet, he'll probably just address it before he submits.

This (the CL 30790 approach) is exactly what we've been experimenting with at Cloudflare in our fork, and seems to work.
Agreed with @danp that we'll need a few more fields in the CHI, but it should be another issue/proposal/CL.


Added a brief explanation on how to add Topics

Also added a line break to the first paragraph making it easy to read.

I appreciate your effort but I don't think we need to guide users to add the topic to the repo. We can't spoonfeed users and they should figure out this thing on their own

You are right 🤔


Ability to handle raw svg strings with #image

Description
a way of feeding raw images into typst to be rendered.
so to make things compatible (no breaks) you could introduce a type to the image function where the default is "path"
#image("./File.jpg")
but you could also say explicitly:
#image("./File.jpg",from:"path")
or you could use raw svg
#image("<svg ....>",from:"svg")
Use Case
I can use svg generated from plugins and display it.

Duplicate of #1084.


Reconnect sticks to the last known connection point and does not use additional connection points

Is it possible to connect to more than one node? We'd like to run a cluster of disque nodes, and currently it looks like we can only pass a single URI to the DisqueClient.

You can pass multiple connection points encapsulated within the DisqueURI.
There is only one connection at a time, the hosts are used as HA failover
hosts. If a connection goes down, another host is used for reconnect. I'm
open for discussion how to balance load amongst different hosts from the
client perspective. Use the scheme
disque://[password@]host[:port][,host2[:port2]][,hostN[:port2N]]
to specify multiple hosts.
Robert Gründler<EMAIL_ADDRESS>schrieb am Fr., 21. Aug. 2015 um
13:30:

Is it possible to connect to more than one node? We'd like to run a
cluster of disque nodes, and currently it looks like we can only pass a
single URI to the DisqueClient.
—
Reply to this email directly or view it on GitHub
https://github.com/mp911de/spinach/issues/7.

Alright, we'll give that a try. Thanks a lot for the fast response.

I've tried the following setup:

2 Nodes running in a vagrant box on different ports
The client has been setup with the URI disque://<IP_ADDRESS>:7711,<IP_ADDRESS>:7712
The 2 nodes run clustered (disque -p 7711 cluster meet <IP_ADDRESS> 7712)

When start the application, the client seems to connect to the first node running on 7711. Now when i stop that node, i'm seeing this in the logs:
2015-08-21 16:18:09.239 DEBUG c.l.redis.protocol.ConnectionWatchdog    : Reconnecting, last destination was /<IP_ADDRESS>:7711
2015-08-21 16:18:09.239 DEBUG c.l.redis.protocol.ConnectionWatchdog    : Cannot connect: java.net.ConnectException: Connection refused: /<IP_ADDRESS>:7711
2015-08-21 16:18:09.239 DEBUG c.l.redis.protocol.ConnectionWatchdog    : scheduleReconnect()

The client seems to try to reconnect to the same node over and over, without ever trying the 2nd node.
Is there anything we might configure additionally to the disque uri?

You've done everything right. I'll label the issue as bug then.
Robert Gründler<EMAIL_ADDRESS>schrieb am Fr., 21. Aug. 2015 um
16:23:

I've tried the following setup:

2 Nodes running in a vagrant box on different ports
The client has been setup with the URI disque://<IP_ADDRESS>:7711,
<IP_ADDRESS>:7712
The 2 nodes run clustered (disque -p 7711 cluster meet <IP_ADDRESS> 7712
)

When start the application, the client seems to connect to the first node
running on 7711. Now when i stop that node, i'm seeing this in the logs:
2015-08-21 16:18:09.239 DEBUG c.l.redis.protocol.ConnectionWatchdog    : Reconnecting, last destination was /<IP_ADDRESS>:7711
2015-08-21 16:18:09.239 DEBUG c.l.redis.protocol.ConnectionWatchdog    : Cannot connect: java.net.ConnectException: Connection refused: /<IP_ADDRESS>:7711
2015-08-21 16:18:09.239 DEBUG c.l.redis.protocol.ConnectionWatchdog    : scheduleReconnect()
The client seems to try to reconnect to the same node over and over,
without ever trying the 2nd node.
Is there anything we might configure additionally to the disque uri?
—
Reply to this email directly or view it on GitHub
https://github.com/mp911de/spinach/issues/7#issuecomment-133441147.

Thanks. Let me know if i can provide more details to help debugging.

Fixed, uploaded spinach-0.2-SNAPSHOT to https://oss.sonatype.org/content/repositories/snapshots/. Please retest.

Great, that fixed it. Failover works now, thanks a lot!

Is there another release planned in the near future, or should we stick to the SNAPSHOT repo?

It depends on how you define near future. I need to implement https://github.com/mp911de/spinach/issues/5 to publish a 0.2 release. I'm targeting Sep 7 for releasing 0.2 unless some blocker bugs cross the way. Feedback appreciated.

Great, thanks for the info.

Would creating a minor release be much of a hassle for you? We're working on an apache-camel component which uses spinach, and it would be great to have failover support.
Another question: Would you accept a PR implementing JMX support for spinach?

Ok, I'll release a 0.1.1 later this day.
What do you want to expose by JMX? I'm thinking a while about a statistics publisher to expose latencies and so forth. You can catch me on IRC (freenode) or Gitter (https://gitter.im/mp911de/lettuce) for a chat

@pulse00 Just released https://github.com/mp911de/spinach/releases/tag/0.1.1 Should be available on mvn central in a couple of hours

Superb, thanks!


Compare and unify JSON serialization types

Both in the validation package and in the core package, there are type definitions for the JSON serialisation format.  They should be reconciled.
We might want to define a separate package for these types, as they are used in validation and in core.  And if we move the LionWeb diff from validation to a separate package, it will be used by this package as well.

I think that we need a separate package just for types and (specific, low-level) functions around the serialization format as well.


[BUG] [Zone Decommission] MetadataDiff is 0 when publishing cluster state for status update

Describe the bug
While updating the decommission status, the cluster state update is not applied to non leader nodes, and the local state of the nodes doesn't receive the status update. The reason being, while executing the cluster state update for decommission status update, the same instance of metadata was being update causing the relative diff to be 0 for non leader nodes and hence, the leader though had the appropriate decommission status but rest of the nodes had a stale decommission state.
To Reproduce
Steps to reproduce the behavior:

Execute attribute decommission - curl -X PUT "localhost:9201/_cluster/decommission/awareness/zone/zone-2?pretty
Check the local cluster state of any node - curl -X GET "localhost:9202/_cluster/state?local&pretty"
Check the cluster level cluster state - curl -X GET "localhost:9202/_cluster/state?pretty"
The cluster state from (3) would have the latest updated the status but cluster state from (2) has status as INIT

Expected behavior
The latest cluster state and decommission status to be published to all nodes.

Closing as PR is merged


Make stars and forks buttons useful

I don't know if my suggestion is technically possible, but it would be interesting. Currently the icons displaying the number of stars and the number of forks of a repository are of little use. It would be helpful if I could click on this icon to simply add a star to the repository and if I could click on the other icon to fork the repository.
When the repository is not starred and not forked:

And when starred and forked, the icons become a little blue:

It would also be useful to see if we have already added a star to this repository or if we already have a fork of this repository.

We hide the same Star button in on the dashboard with the reasoning that you probably want to learn more about a projects instead of starring after one sentence
https://github.com/sindresorhus/refined-github/blob/2e5c19a4d8fa7db94e775beeac9b8bd610a2fa2b/source/features/clean-dashboard.css#L85-L89

Thanks for the suggestion. Unfortunately, we're going to pass on this.


reduce hardcoded dependencies

used pipdeptree to reduce number of hardcoded dependencies in Pipfile.
The installed dependencies are almost unchanged (just backports-abc is
gone which is probably not needed).

This should make things easier for us to maintain going forward.
Once you've checked that this looks fine, please make a new release.

Just made a release 19.4.3


add integration springCloud test

Make sure that:

[x] You have read the contribution guidelines.
[x] You submit test cases (unit or integration tests) that back your changes.
[x] Your local test passed ./mvnw clean install -Dmaven.javadoc.skip=true.

Codecov Report

Merging #5139 (580f2b9) into master (d214baf) will decrease coverage by 1.36%.
Report is 17 commits behind head on master.
The diff coverage is 14.28%.

@@             Coverage Diff              @@
##             master    #5139      +/-   ##
============================================
- Coverage     61.79%   60.43%   -1.36%
+ Complexity     8495     8402      -93
============================================
  Files          1227     1242      +15
  Lines         36967    37341     +374
  Branches       3515     3519       +4
============================================
- Hits          22843    22568     -275
- Misses        12166    12862     +696
+ Partials       1958     1911      -47

Files Changed
Coverage Δ

...org/apache/shenyu/k8s/common/IngressConstants.java
0.00% <ø> (ø)

...g/apache/shenyu/k8s/parser/DubboIngressParser.java
0.00% <0.00%> (ø)

...g/apache/shenyu/k8s/parser/MotanIngressParser.java
0.00% <0.00%> (ø)

...rg/apache/shenyu/k8s/parser/SpringCloudParser.java
0.00% <0.00%> (ø)

.../org/apache/shenyu/k8s/parser/WebSocketParser.java
0.00% <0.00%> (ø)

...e/shenyu/k8s/repository/ShenyuCacheRepository.java
0.00% <0.00%> (ø)

.../apache/shenyu/k8s/cache/IngressSelectorCache.java
71.42% <50.00%> (-3.58%)
:arrow_down:

.../apache/shenyu/k8s/parser/DivideIngressParser.java
42.21% <50.00%> (ø)

...pache/shenyu/k8s/reconciler/IngressReconciler.java
34.13% <52.17%> (+0.12%)
:arrow_up:

...apache/shenyu/k8s/common/IngressConfiguration.java
100.00% <100.00%> (ø)

... and 72 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


feat: create amaro loader

I'm currently trying to figure out how to export amaro as a loader.
The main issue is that I would like transformSync to get the configuration from a .swcrc file.
@kdy1 I dont think it will automatically search for it, or is there another way?

This will behave exactly as the one in core, and it requires --experimental-strip-types to be enabled, so it uses node.js .ts resolution. This only overrides the swc transform function, and I would like to make it customizable

PTAL @nodejs/typescript @nodejs/loaders

There are cases in node core where the cjs loader will return undefined source code https://github.com/nodejs/node/blob/3cbeed88d9de9231f420ba53ad0e4d4a7af9c637/lib/internal/modules/esm/get_format.js#L131-L136

There are cases in node core where source will be undefined https://github.com/nodejs/node/blob/3cbeed88d9de9231f420ba53ad0e4d4a7af9c637/lib/internal/modules/esm/get_format.js#L131-L136

IIUC that's only true for when format is nullish; the comment explains when defaultResolve returns a nullish format – and it doesn't matter for us, because Amaro will only deal with TS files

There are cases in node core where source will be undefined https://github.com/nodejs/node/blob/3cbeed88d9de9231f420ba53ad0e4d4a7af9c637/lib/internal/modules/esm/get_format.js#L131-L136

IIUC that's only true for when format is nullish; the comment explains when defaultResolve returns a nullish format – and it doesn't matter for us, because Amaro will only deal with TS files

Apparently it happens so I add to check here https://github.com/nodejs/node/blob/3cbeed88d9de9231f420ba53ad0e4d4a7af9c637/lib/internal/modules/helpers.js#L335


Add bootstrap support to system-classnames

Great work on system-classnames! I use bootstrap utils quite often, and I figured it would be a good fit for system-classnames.

Codecov Report

Merging #369 into master will decrease coverage by 0.76%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #369      +/-   ##
==========================================
- Coverage     100%   99.23%   -0.77%
==========================================
  Files           1        4       +3
  Lines         182      261      +79
==========================================
+ Hits          182      259      +77
- Misses          0        2       +2

Impacted Files
Coverage Δ

<EMAIL_ADDRESS>100% <100%> (ø)

<EMAIL_ADDRESS>100% <100%> (ø)

src/index.js
100% <0%> (ø)
:arrow_up:

packages/system-classnames/src/index.js
93.33% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cc2a9ca...a149b8a. Read the comment docs.

Sorry for the delay and thanks for this!

@jxnblk thank you, congratulations on the v4 release! 😄


Component Highlight Matching Issue

The next token obtained by the html/languageService scanner for the s-c-feature is 2.
https://github.com/microsoft/vscode-html-languageservice/blob/1030c1eef51438cb674482338bf4ddfbb1fc4289/src/parser/htmlScanner.ts#L189C13-L189C13)s://github.com/microsoft/vscode-htm-a  uageservice/blob/1030c1eef51438cb674482338bf4ddfbb1fc4289/src/parser/htmlScanner.ts#L189C13-L189C13

Please check if this issue has been resolved in version 2.x. If not, feel free to create a new one.


1.0 container / server process not listening on IPv6

Not sure if this is a material change to the image, or something in the game server itself with 1.0. It appears the server process no longer listens internally on IPv6 regardless of the SERVERIP environment variable. This behavior continues even after the current 1.0 game hotfix and workaround removal from the image.
Exec'd into running container & ran apt update && apt install net-tools to provide access to netstat -
root@satisfactory:/config# netstat -an
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 <IP_ADDRESS>:7777            <IP_ADDRESS>:*               LISTEN
tcp        0      0 <IP_ADDRESS>:33712        <IP_ADDRESS>:443      ESTABLISHED
tcp        0      0 <IP_ADDRESS>:47608        <IP_ADDRESS>:443       ESTABLISHED
tcp6       0      0 :::44429                :::*                    LISTEN
tcp6       0      0 2607:XXXX:XXXX:XXXX::44786 2620:2d:4002:1::103:80  TIME_WAIT
tcp6       0      0 2607:XXXX:XXXX:XXXX::44792 2620:2d:4002:1::103:80  TIME_WAIT
tcp6       0      0 2607:XXXX:XXXX:XXXX::44792 2620:2d:4002:1::102:80  TIME_WAIT
udp        0      0 <IP_ADDRESS>:7777            <IP_ADDRESS>:*
udp6       0      0 :::7778                 :::*
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node   Path
unix  3      [ ]         STREAM     CONNECTED     116566441
unix  3      [ ]         STREAM     CONNECTED     116566442
unix  3      [ ]         STREAM     CONNECTED     116561616
unix  3      [ ]         STREAM     CONNECTED     116561617

Container attached to docker default bridge network does have IPv6 connectivity and IPv6 global transit. IE: can ping6 ipv6.google.com if inetutils-ping is added to running container.
Current docker-compose.yml
services:
  satisfactory:
    container_name: satisfactory
    hostname: satisfactory
    image: ghcr.io/wolveix/satisfactory-server:latest
    restart: unless-stopped
    network_mode: bridge
    ports:
      - 7777:7777/tcp
      - 7777:7777/udp
    volumes:
      - ./config:/config
      - /etc/localtime:/etc/localtime:ro
    environment:
      - TZ=UTC
      - PGID=999
      - PUID=999
      - MAXPLAYERS=16
      - STEAMBETA=false
      - SERVERIP=
      # - ROOTLESS=true
    deploy:
      resources:
        limits:
          memory: 6G
        reservations:
          memory: 4G

Hey @goofball222! Sorry to hear that, could you try ghcr.io/wolveix/satisfactory-server:dev-301 and let me know if that addresses the issue for you, please?

Does not appear to be any change in state with that image:
root@satisfactory:/config# netstat -an
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 <IP_ADDRESS>:7777            <IP_ADDRESS>:*               LISTEN
tcp        0      0 <IP_ADDRESS>:59010        <IP_ADDRESS>:443      ESTABLISHED
tcp        0      0 <IP_ADDRESS>:38006        <IP_ADDRESS>:80       TIME_WAIT
tcp        0      0 <IP_ADDRESS>:38062        <IP_ADDRESS>:443       ESTABLISHED
tcp6       0      0 :::35469                :::*                    LISTEN
tcp6       0      0 2607:XXXX:XXXX:XXXX::55068 2620:2d:4000:1::101:80  TIME_WAIT
tcp6       0      0 2607:XXXX:XXXX:XXXX::55080 2620:2d:4000:1::101:80  TIME_WAIT
udp        0      0 <IP_ADDRESS>:7777            <IP_ADDRESS>:*
udp6       0      0 :::7778                 :::*
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node   Path
unix  3      [ ]         STREAM     CONNECTED     116911191
unix  3      [ ]         STREAM     CONNECTED     116910130
unix  3      [ ]         STREAM     CONNECTED     116911190
unix  3      [ ]         STREAM     CONNECTED     116910131
root@satisfactory:/config# ps auxwww
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
steam          1  0.0  0.0   2892   760 ?        Ss   18:05   0:00 /bin/sh /config/gamefiles/FactoryServer.sh -Port=7777 -ini:Engine:[HTTPServer.Listeners]:DefaultBindAddress=any -ini:Engine:[Core.Log]:LogNet=Error -ini:Engine:[Core.Log]:LogNetTraffic=Warning -ini:Engine:[/Script/FactoryGame.FGSaveSession]:mNumRotatingAutosaves=5 -ini:Engine:[/Script/Engine.GarbageCollectionSettings]:gc.MaxObjectsInEditor=2162688 -ini:Engine:[/Script/OnlineSubsystemUtils.IpNetDriver]:LanServerMaxTickRate=30 -ini:Engine:[/Script/OnlineSubsystemUtils.IpNetDriver]:NetServerMaxTickRate=30 -ini:Engine:[/Script/OnlineSubsystemUtils.IpNetDriver]:ConnectionTimeout=30 -ini:Engine:[/Script/OnlineSubsystemUtils.IpNetDriver]:InitialConnectTimeout=30 -ini:Engine:[ConsoleVariables]:wp.Runtime.EnableServerStreaming=1 -ini:Game:[/Script/Engine.GameSession]:ConnectionTimeout=30 -ini:Game:[/Script/Engine.GameSession]:InitialConnectTimeout=30 -ini:Game:[/Script/Engine.GameSession]:MaxPlayers=16 -ini:GameUserSettings:[/Script/Engine.GameSession]:MaxPlayers=16
steam        109  5.3  2.2 2953340 749940 ?      Sl   18:06   0:25 /config/gamefiles/Engine/Binaries/Linux/FactoryServer-Linux-Shipping FactoryGame -Port=7777 -ini:Engine:[HTTPServer.Listeners]:DefaultBindAddress=any -ini:Engine:[Core.Log]:LogNet=Error -ini:Engine:[Core.Log]:LogNetTraffic=Warning -ini:Engine:[/Script/FactoryGame.FGSaveSession]:mNumRotatingAutosaves=5 -ini:Engine:[/Script/Engine.GarbageCollectionSettings]:gc.MaxObjectsInEditor=2162688 -ini:Engine:[/Script/OnlineSubsystemUtils.IpNetDriver]:LanServerMaxTickRate=30 -ini:Engine:[/Script/OnlineSubsystemUtils.IpNetDriver]:NetServerMaxTickRate=30 -ini:Engine:[/Script/OnlineSubsystemUtils.IpNetDriver]:ConnectionTimeout=30 -ini:Engine:[/Script/OnlineSubsystemUtils.IpNetDriver]:InitialConnectTimeout=30 -ini:Engine:[ConsoleVariables]:wp.Runtime.EnableServerStreaming=1 -ini:Game:[/Script/Engine.GameSession]:ConnectionTimeout=30 -ini:Game:[/Script/Engine.GameSession]:InitialConnectTimeout=30 -ini:Game:[/Script/Engine.GameSession]:MaxPlayers=16 -ini:GameUserSettings:[/Script/Engine.GameSession]:MaxPlayers=16
root         423  1.0  0.0   7636  2940 pts/0    Ss   18:14   0:00 bash
root         431  0.0  0.0  10072  1368 pts/0    R+   18:14   0:00 ps auxwww

docker inspect satisfactory shows:
            "Labels": {
                <snip>
                "org.opencontainers.image.ref.name": "ubuntu",
                "org.opencontainers.image.version": "22.04",
                "version": "vDEV-301"
            }

so it's definitely running on the updated image.

Hm, that's strange. So, we removed multihome entirely from this image, which is why SERVERIP had no effect. We did this as Coffee Stain said it was no longer needed, as it should bind to all available interfaces.
dev-301 added it back to see whether it would make a difference, though clearly not. It seems others are heaving issues with IPv6 in the Satisfactory Discord server, so I suspect it's a bug with the new server unfortunately.

This is similar to the issue I encountered before. The Satisfactory server requires a UDP port for the game port and a TCP port for the API port. Actually, the server still listens on IPv6, but only on port UDP6/7778. However, the API port is not bound to an IPv6 address. Therefore, even if you connect to the game port in IPv6 format within the game, you will still be prompted with an API connection error.
For Docker containers, this issue is unlikely to cause any impact. If systemd is used to manage the server, it can listen to requests on :::7777 and forward them to <IP_ADDRESS>:7777.
(base) ovoovo@ubuntu:~$ netstat -tunlp | grep Factory
(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)
tcp        0      0 <IP_ADDRESS>:7777            <IP_ADDRESS>:*               LISTEN      3269100/FactoryServ
tcp6       0      0 :::36429                :::*                    LISTEN      3269100/FactoryServ
udp        0      0 <IP_ADDRESS>:7777            <IP_ADDRESS>:*                           3269100/FactoryServ
udp6       0      0 :::7778                 :::*                                3269100/FactoryServ

I think this is causing connection issues that aren't present in the native windows build of the server

Hey guys, was just setting up a Satisfactory server where ONLY IPv6 can be exposed via DNS (Student accomodations):
I've got it to work by adding the -multihome=:: parameter to the command (-field in TrueNAS SCALE).
The DNS contains the direct IPv6 of the docker host, with Port 7777 forwarded on TCP and UDP.
The DNS Entry contains both the local IPv4 of the host, aswell as the public IPv6 of the host.
I will be making a PR soon with the parameter brought back and setting this be default (as the wiki says that using this ensures listening on both v4 and v6 (v4 is automatically translated to [ffff::ffff:<IP_ADDRESS>] for example)

@Chaos02 thanks for sharing. When I spoke with CoffeeStain directly, they advised that multihome wasn't necessary since the release of 1.0. And yes, multihome respects IP norms as you've rightfully highlighted, where <IP_ADDRESS> is v4 and :: is dual stack.
I look forward to reviewing your PR!


Prioritise what we need to know from UR

What do you need?
Research with a range of users to check out the new Content Guide is valuable - before launch.
We especially need to do validation of the new IA.

Blocked by resourcing, for now.

Drafted here https://docs.google.com/a/digital.gov.au/document/d/1BPVpyOYBoTV77JwOTbhGlgxBZ5Ztqgq4AHGqik90h0Q/edit?usp=drive_web

Ok, I'm calling this done. Will move on to #111 - can revise this if required later (and probably will be).


Patches #246

Temporary fix for #246.
defaults GRPC_DNS_RESOLVER to native until a fix for https://github.com/grpc/grpc/issues/24018 is found.

I think I was running Ubuntu 18.04, will see if I can double check

Yup, this solves it for me


specular_disabled render mode in shaders does not make the SPECULAR output zero

Tested versions
Reproducible in 4.3.rc1
System information
Godot v4.3.rc1 - Windows 10.0.22631 - GLES3 (Compatibility) - NVIDIA GeForce GTX 1660 SUPER (NVIDIA; <IP_ADDRESS>70) - Intel(R) Core(TM) i7-10700F CPU @ 2.90GHz (16 Threads)
Issue description

Setting the render mode specular_disabled does eliminate specular contribution from lights, but not reflected light. The SPECULAR output is non-zero in this case. If this is intended, the namings are very confusing and should be either changed or documented better.
Steps to reproduce
Open attached project, and look at the two balls side-by-side. Note that one is blacker than the other.
ball1 has the black shader, and ball2 has the true_black shader. Both shaders have the render mode specular_disabled, but only true_black explicitly sets SPECULAR output to zero. ball still receives reflected light from the WorldEnvironment in the scene.
Minimal reproduction project (MRP)
bug-report.zip

This is done by design. The specular render modes only affect direct light specular modes, not reflected light.
Set Metallic > Specular to 0 to disable specular contribution entirely (all values below 0.02 do that).

godotengine/godot-docs#9691 updates the manual.

That PR definitely helps. Thanks!

Reopening, as the associated documentation PR hasn't been merged yet.


HDFS-17150. EC: Fix the bug of failed lease recovery.

If the client crashes without writing the minimum number of internal blocks required by the EC policy, the lease recovery process for the corresponding unclosed file may continue to fail. Taking RS(6,3) policy as an example, the timeline is as follows:

The client writes some data to only 5 datanodes;
Client crashes;
NN fails over;
Now the result of uc.getNumExpectedLocations() completely depends on block report, and there are 5 datanodes reporting internal blocks;
When the lease expires hard limit, NN issues a block recovery command;
The datanode checks the command and finds that the number of internal blocks is insufficient, resulting in an exception and recovery failure;
https://github.com/apache/hadoop/blob/b6edcb9a84ceac340c79cd692637b3e11c997cc5/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockRecoveryWorker.java#L534-L540
The lease expires hard limit again, and NN issues a block recovery command again, but the recovery fails again......

When the number of internal blocks written by the client is less than 6, the block group is actually unrecoverable. We should equate this situation to the case where the number of replicas is 0 when processing replica files, i.e., directly remove the last block group and close the file.

@zhangshuyan0 Thanks a lot for reporting this bug. This phenomenon also happens on our clusters. Leave some questions, hope to receive your reply~ Thanks.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 29s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
32m  0s

trunk passed

+1 :green_heart:
compile
0m 53s

trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
compile
0m 48s

trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

+1 :green_heart:
checkstyle
0m 44s

trunk passed

+1 :green_heart:
mvnsite
0m 55s

trunk passed

+1 :green_heart:
javadoc
0m 52s

trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
1m 10s

trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

+1 :green_heart:
spotbugs
1m 58s

trunk passed

+1 :green_heart:
shadedclient
23m 55s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 46s

the patch passed

+1 :green_heart:
compile
0m 45s

the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javac
0m 45s

the patch passed

+1 :green_heart:
compile
0m 38s

the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

+1 :green_heart:
javac
0m 38s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
0m 32s

the patch passed

+1 :green_heart:
mvnsite
0m 42s

the patch passed

+1 :green_heart:
javadoc
0m 38s

the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
1m  6s

the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

+1 :green_heart:
spotbugs
1m 52s

the patch passed

+1 :green_heart:
shadedclient
24m 48s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
190m 50s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 39s

The patch does not generate ASF License warnings.

288m 11s

Reason
Tests

Failed junit tests
hadoop.hdfs.TestFileChecksum

hadoop.hdfs.server.namenode.ha.TestObserverNode

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5937/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5937

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 068005fabb65 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 4e7cd881a92b7deb7d8d188b8c1dc85ca6e8ee5f

Default Java
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5937/1/testReport/

Max. process+thread count
4019 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5937/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 30s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
34m 54s

trunk passed

+1 :green_heart:
compile
0m 52s

trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
compile
0m 50s

trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

+1 :green_heart:
checkstyle
0m 47s

trunk passed

+1 :green_heart:
mvnsite
0m 52s

trunk passed

+1 :green_heart:
javadoc
0m 48s

trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
1m  8s

trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

+1 :green_heart:
spotbugs
1m 57s

trunk passed

+1 :green_heart:
shadedclient
27m  6s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 49s

the patch passed

+1 :green_heart:
compile
0m 52s

the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javac
0m 52s

the patch passed

+1 :green_heart:
compile
0m 45s

the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

+1 :green_heart:
javac
0m 45s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 38s
/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 110 unchanged - 0 fixed = 111 total (was 110)

+1 :green_heart:
mvnsite
0m 47s

the patch passed

+1 :green_heart:
javadoc
0m 41s

the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
1m 16s

the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

+1 :green_heart:
spotbugs
2m 17s

the patch passed

+1 :green_heart:
shadedclient
27m 59s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
194m 35s

hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 39s

The patch does not generate ASF License warnings.

302m 11s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5937/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5937

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux e3965eccf0d3 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 35eb68154529a0568e80d3ca94540225b1dda911

Default Java
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5937/2/testReport/

Max. process+thread count
3238 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5937/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@Hexiaoqiao @haiyang1987 Some comments have been added. Please take a check when you have free time.

LGTM. +1.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 28s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+0 :ok:
detsecrets
0m  1s

detect-secrets was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
32m 56s

trunk passed

+1 :green_heart:
compile
0m 54s

trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
compile
0m 50s

trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

+1 :green_heart:
checkstyle
0m 44s

trunk passed

+1 :green_heart:
mvnsite
0m 55s

trunk passed

+1 :green_heart:
javadoc
0m 51s

trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
1m 12s

trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

+1 :green_heart:
spotbugs
2m  0s

trunk passed

+1 :green_heart:
shadedclient
22m 11s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 47s

the patch passed

+1 :green_heart:
compile
0m 48s

the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javac
0m 48s

the patch passed

+1 :green_heart:
compile
0m 43s

the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

+1 :green_heart:
javac
0m 43s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

-0 :warning:
checkstyle
0m 34s
/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 110 unchanged - 0 fixed = 111 total (was 110)

+1 :green_heart:
mvnsite
0m 45s

the patch passed

+1 :green_heart:
javadoc
0m 40s

the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04

+1 :green_heart:
javadoc
1m  4s

the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

+1 :green_heart:
spotbugs
1m 55s

the patch passed

+1 :green_heart:
shadedclient
22m 18s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

-1 :x:
unit
183m 51s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

+1 :green_heart:
asflicense
0m 39s

The patch does not generate ASF License warnings.

278m  3s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.blockmanagement.TestBlockTokenWithShortCircuitRead

hadoop.hdfs.TestFileChecksum

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5937/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/5937

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets

uname
Linux 41e15ee1ded0 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / b0ce7e38eb84e7ffc0f14493ce3dbae8c8f7393c

Default Java
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5937/3/testReport/

Max. process+thread count
3372 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5937/3/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

LGTM. +1.

Committed to trunk. Thanks @zhangshuyan0 for your works. And @haiyang1987 , @hfutatzhanghb for your reviews!


Add validation for MachinePool/NodePool taints

Relates https://issues.redhat.com/browse/OCPBUGS-10423

lgtm

@enxebre I am extending this in https://github.com/openshift/rosa/pull/1125 since we had other fixes needed wrt taints, this one can be closed

/hold

Closing in favor of PR from @andreadecorte's comment


npx create-react-app my-app No command 'npx' found, did you mean:

Is this a bug report?
(write your answer here)

Did you try recovering your dependencies?

(Write your answer here.)
Which terms did you search for in User Guide?

(Write your answer here if relevant.)
Environment

(paste the output of the command here)
Steps to Reproduce

(Write your steps here:)

Expected Behavior

(Write what you thought would happen.)
Actual Behavior

(Write what happened. Please add screenshots!)
Reproducible Demo

(Paste the link to an example project and exact instructions to reproduce the issue.)

You should have npx if you're running npm 5.2.0 or later.
If you don't have it:
npm i -g create-react-app
create-react-app myapp

Thanks for mentioning. I did the same and it worked.

Easy to resolve, just add
alias npx='/usr/local/Cellar/node/10.11.0/bin/npx'

In your .bash_profile and refresh your profile with this command
source ~/.bash_profile

I hope this will resolve your issue. On my side, it is already working :)


100% Coverage For MockSupport_c - Part 3 of 3

Add crashOnFailure_c() interface and test crashing on failure with mock_c().

Hey... can't merge this pull request anymore :(

sorry had to do a small manual merge first 'cause i hadnt put it on top of the others

Changes Unknown when pulling 1ceff3cb5f0b6f7f020e53b04ccec5f87b98c592 on arstrube:mockc_3 into ** on cpputest:master**.


Can I make connection to Impala from this driver

Our users use both hive and impala.
From this thrift client, is it possible to create connection to impala? If yes do you have any examples/usage/test cases around it?

I didnt realize its so much easy with this driver to make impala connections with same code written for hive, I was able to make connection just by changing the port number (from 10000 for hive to 21051 for impala). I am also able to run some queries locally, will test it out more next week.
Meanwhile if you have any additional details on this please let me know. Thanks

hi @ntallapa12 unfortunately, I didn’t test the hive-driver on impala. It seems to me the api is similar, but impala’s one is more extended. Probably, it is a good thing to add it to the story. But I cannot promise to implement it in the near future. I keep this issue updating. Also, I appreciate any help, so if you find out something let me know. Thanks


Support for moded beacon blocks

Can you create a config to add moded beacon blocks with different potencies?
(I'm assuming that beacon power is calculated by the total potency of all the blocks its made of per layer instead, of checking if the whole thing is the same block and then if it is, using the potency of that block. so you could mix blocks of the same potency and the beacon would still recognize that)
For example a full netherite beacon gives 3 potency, but the netherite stair from Blokus only acts as a normal beacon block. The stair still costs a full netherite block per stair, yet it wont provide any additional potency. Same can be said for the Aeternium block from BetterEnd, it has the same amount of netherite as a netherite block.
Basically, I want my cool beacon to be powerful too. cuz rn it has the same power as a full iron one.

Currently, the way it behaves, is it works on minimum potency. So if you have a beacon made of low potency blocks (diamond) and replace one or two blocks with a high potency block (netherite) it will still be low potency. You would need to replace all blocks with high potency for it to be considered a high potency beacon. The simplest solution for mod support would be for me to add block tags for the additional tiers of potency. I'm just unsure how to describe them currently. Names such as low_potency and high_potency may be misleading as there would be a lower potency still, of blocks not in those two tags but still in minecraft:beacon_base_blocks.

Implemented in https://github.com/ChloeDawn/BeaconOverhaul/releases/tag/1.4.0+1.17 https://www.curseforge.com/minecraft/mc-mods/beaconoverhaul/files/3350334


Bump Connect to 2022.04.2

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Dex Server doesn't return system:cluster-admins when integrated with OpenShift OAuth

Describe the bug
OpenShift GitOps Operator v1.3 and integrated with Openshift OAuth using the Dex server.  Dex integration with OpenShift OAuth is working fine, however I find an issue with the Dex Server response where it responds with corresponding user groups back to ArgoCD after authentication.
Dex is not providing a "system:cluster-admins" group in its response to ArgoCD along with other groups.
For example: I am a cluster-admin on a cluster and part of other groups as well, however, in Dex response, I don't find "system:cluster-admins"
Example response from Dex to Gitops Server,
time="2021-11-03T15:30:14Z" level=info message="received unary call ............ groups: ["system:authenticated", system:authenticated:oauth", "container_users", "container_prod_users"]
To Reproduce
Steps to reproduce the behavior:

Add user to cluster admin
oc adm policy add-cluster-role-to-user cluster-admin <>

Go to OpenShift Cluster (4.7.18) GUI console, Open the Administrator perspective of the web console and navigate to Operators → OperatorHub in the menu on the left.
Search for OpenShift GitOps, click the Red Hat OpenShift GitOps tile, and then click the Install button.

After the operator is installed, using the dropdown Actions on top right of the page, choose Edit Subscription
On the YAML code, under the spec level, enable the DEX feature to enable external authentication and click Save
spec:
config:
env:

name: DISABLE_DEX
value: 'false'

Change the project to openshift-gitops, Go to the menu Operators>Installed Operators and click on Red Hat OpenShift GitOps, select tab Argo CD, On the ArgoCD instance list, click on the three dots at the very left of the openshift-gitops and select Edit ArgoCD,
On the YAML code, under the spec level, update the DEX and RBAC section to match the following.
...
spec:
dex:
openShiftOAuth: true
rbac:
defaultPolicy: 'role:readonly'
policy: |
g, system:cluster-admins, role:admin
scopes: '[groups]'
...

Login to ArgoCD UI (user with "cluster-admin" ), and in Open Shift GUI, Go to openshift-gitops project --> pods --> argocd-server-XXXX --> logs

See error
time="2021-11-03T15:30:14Z" level=info message="received unary call ............ groups: ["system:authenticated", system:authenticated:oauth", "container_users", "container_prod_users"]

***system:cluster-admins is not part of Dex response
Expected behavior
A clear and concise description of what you expected to happen.
Dex Server response should include "system:cluster-admins" as well along with other groups (such as "system:authenticated", "system:authenticated:oauth") when user part of "system:cluster-admins"
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.

Hi @sumantripuraneni , Sorry for responding late on this. It somehow got missed. Do you still see this issue ? Can you share the oc command output to confirm that your user is part of system:cluster-admin group ?
If this really turns out to be true, we should work with github.com/dex-idp/dex to get this fixed.

Any update on this @sumantripuraneni

@sumantripuraneni I can speak to this as we ran into this ourselves (not with this Operator, but Argo in general). The group you need is actually cluster-admin not system:cluster-admin if memory serves.

Fresh openshift install here using ROSA. The argocd-rbac-cm reads

but Argo receives

Specifying just cluster-admins instead of system:cluster-admins seems to solve the issue for me:
  rbac:
    policy: g, cluster-admins, role:admin
    scopes: '[groups]'

This issue is fixed in the latest version of the Operator.
Both system:ckuster-admins and cluster-admins is part of the default RBAC now.


Include RestRequestOption interface and OnResult implementation

add RestRequestOption in restlib
add unit tests on rest request result
update arrai transformation

Closes #105 #159

Failed on compare Sysl and arr.ai transforms output, still need to work on the sysl transformation.


Remove default channel resolving from guild input.

Please describe the changes this PR makes and why it should be merged:
Removes guild as a possible parameter from resolveChannelID as default channels aren't in the API anymore (or won't be) idk.
Semantic versioning classification:

[ ] This PR changes the library's interface (methods or parameters added)

[x] This PR includes breaking changes (methods removed or renamed, parameters moved or removed)

[ ] This PR only includes non-code changes, like changes to documentation, README, etc.

Just FYI, this probably won't get merged because the data manager is removed in #1618.

Oh, resolver not manager.  mb

[cries in spanish]

sorry this has already been done in another commit where I removed loads of other resolve crap!


Add an e2e matmul transpose test

Currently fails to compile (error message copied into test).

Can you try to add vectorization passes and see if bf16 ci pass after that? @erwei-xilinx reported peano + bf16 can only work in ci with vectorization passes.

Can you try to add vectorization passes and see if bf16 ci pass after that? @erwei-xilinx reported peano + bf16 can only work in ci with vectorization passes.

No I meant the pack-peel pipeline which is missing vectorization passes. The transpose pipeline is still using pad-pack which already has vectorization.

Can you try to add vectorization passes and see if bf16 ci pass after that? @erwei-xilinx reported peano + bf16 can only work in ci with vectorization passes.

No I meant the pack-peel pipeline which is missing vectorization passes. The transpose pipeline is still using pad-pack which already has vectorization.

A good point! @newling We should also add matmul transpose test for pack-peel pipeline. If that works with vectorization, we don't have to waste time fixing failure for pad-pack.

Hey @yzhang93 and @erwei-xilinx PR https://github.com/nod-ai/iree-amd-aie/pull/319 adds vectorization to the pack peel pipeline, but the pipeline needs work to make vectorization work. So tests are not fully enabled in https://github.com/nod-ai/iree-amd-aie/pull/319

@newling the error comes from AIR failing to analyze a data access pattern with vectorized code. If linalg.generic can remain at AIR input then the test should pass. I wonder how difficult do you think it is to move the vectorization passes to after, or in the middle of, AIR passes?

@newling the error comes from AIR failing to analyze a data access pattern with vectorized code. If linalg.generic can remain at AIR input then the test should pass. I wonder how difficult do you think it is to move the vectorization passes to after, or in the middle of, AIR passes?

I think the vectorization pass needs to happen before bufferization, but I'm not sure.

@erwei-xilinx @yzhang93 @MaheshRavishankar
I tried permuting the order in which vectorization and bufferization happen from:
appendVectorizationToPipeline(funcPassManager);
addAMDAIEBufferizePasses(funcPassManager);                                                                                ```

to

addAMDAIEBufferizePasses(funcPassManager);
appendVectorizationToPipeline(funcPassManager);

but the vectorization pass fails. I think it crashes in `linalg::tileLinalgOp(rewriter, genericOp, opts);` (assertion error about incorrect number of results).

So hopefully you can find another solution, because running vectorization later doesn't seem like it will be a quick-fix. Hopefully the L1 memory can be created to be the correct size at construction..

@erwei-xilinx @yzhang93 @MaheshRavishankar
I tried permuting the order in which vectorization and bufferization happen from:

Thanks for trying that, @newling . I put together this PR to AIR which attempts to analyze the data access pattern of post-vectorization kernel code: https://github.com/Xilinx/mlir-air/pull/565 This will actually get the CI to pass for pack-peel bf16 GEMMs, but currently causing pad-pack to fail... Will keep looking into that issue tomorrow.

Thanks Erwei. Yes I think ideally we'd have transpose working on both pipelines.


Vehicle.commands is throwing away the first command sent

The first command set in the mission is being discarded (probably by DroneKit). The following code shows how to create  a number of missions, starting with takeoff.
#add a takeoff command
altitude=100 #target altitude
pitch=45 #take off pitch. Need to check if degrees or radians, and what is a reasonable valued.
cmd = Command(  0, 0, 0, mavutil.mavlink.MAV_FRAME_GLOBAL_RELATIVE_ALT,
                mavutil.mavlink.MAV_CMD_NAV_TAKEOFF, 0, 0,
                pitch, 0, 0, 0, 0, 0, altitude)
cmds.add(cmd)

lat = -35.363768
lon = 149.154275
altitude = 110
cmd = Command(0,0,0, mavutil.mavlink.MAV_FRAME_GLOBAL_RELATIVE_ALT, mavutil.mavlink.MAV_CMD_NAV_WAYPOINT,
    0, 0, 0, 0, 0, 0,
    lat, lon, altitude)
cmds.add(cmd)

lat = -35.3666207
lon =<PHONE_NUMBER>
altitude = 120
cmd = Command(0,0,0, mavutil.mavlink.MAV_FRAME_GLOBAL_RELATIVE_ALT, mavutil.mavlink.MAV_CMD_NAV_WAYPOINT,
    0, 0, 0, 0, 0, 0,
    lat, lon, altitude)
cmds.add(cmd)

lat = -35.363769
lon = 149.154276
altitude = 130
cmd = Command(0,0,0, mavutil.mavlink.MAV_FRAME_GLOBAL_RELATIVE_ALT, mavutil.mavlink.MAV_CMD_NAV_LOITER_UNLIM,
    0, 0, 0, 0, 0, 0,
    lat, lon, altitude)

cmds.add(cmd)
v.flush()

The output below shows the missions as sent, and the missions that are then re-retrieved from the vehicle. The first mission item is simply discarded (the first mission item is now the home)
AUTO> Create mission
Got MAVLink msg: MISSION_ACK {target_system : 255, target_component : 0, type : 1}
Sent waypoint 0 : MISSION_ITEM {target_system : 1, target_component : 0, seq : 0, frame : 3, command : 22, current : 0, autocontinue : 0, param1 : 45, param2 : 0, param3 : 0, param4 : 0, x : 0, y : 0, z : 100}
Sent waypoint 1 : MISSION_ITEM {target_system : 1, target_component : 0, seq : 1, frame : 3, command : 16, current : 0, autocontinue : 0, param1 : 0, param2 : 0, param3 : 0, param4 : 0, x : -35.363768, y : 149.154275, z : 110}
Sent waypoint 2 : MISSION_ITEM {target_system : 1, target_component : 0, seq : 2, frame : 3, command : 16, current : 0, autocontinue : 0, param1 : 0, param2 : 0, param3 : 0, param4 : 0, x : -35.3666207, y : 149.16628, z : 120}
Sent waypoint 3 : MISSION_ITEM {target_system : 1, target_component : 0, seq : 3, frame : 3, command : 17, current : 0, autocontinue : 0, param1 : 0, param2 : 0, param3 : 0, param4 : 0, x : -35.363769, y : 149.154276, z : 130}
Sent all 4 waypoints
Got MAVLink msg: MISSION_ACK {target_system : 255, target_component : 0, type : 0}
APM: flight plan received

CHECK: Getting commands
Requesting 4 waypoints t=Wed May 20 17:58:44 2015 now=Wed May 20 17:58:44 2015
commands
Command MISSION_ITEM {target_system : 255, target_component : 0, seq : 0, frame : 0, command : 16, current : 0, autocontinue : 1, param1 : 0.0, param2 : 0.0, param3 : 0.0, param4 : 0.0, x : -35.3632621765, y :<PHONE_NUMBER>27, z :<PHONE_NUMBER>86}
Command MISSION_ITEM {target_system : 255, target_component : 0, seq : 1, frame : 3, command : 16, current : 0, autocontinue : 1, param1 : 0.0, param2 : 0.0, param3 : 0.0, param4 : 0.0, x : -35.3637695312, y :<PHONE_NUMBER>16, z : 110.0}
Command MISSION_ITEM {target_system : 255, target_component : 0, seq : 2, frame : 3, command : 16, current : 0, autocontinue : 1, param1 : 0.0, param2 : 0.0, param3 : 0.0, param4 : 0.0, x : -35.3666191101, y :<PHONE_NUMBER>24, z : 120.0}
Command MISSION_ITEM {target_system : 255, target_component : 0, seq : 3, frame : 3, command : 17, current : 0, autocontinue : 1, param1 : 0.0, param2 : 0.0, param3 : 1.0, param4 : 0.0, x : -35.3637695312, y :<PHONE_NUMBER>16, z : 130.0}
cmds count: 4
cmds next: 4

This is a serious bug! I suspect that someone thought that since copter ignores the takeoff command in AUTO it could be removed, but testing shows that it removes any command irrespective of type. Also Plane doesn't ignore the command.
In the short term someone could work around this by adding the takeoff command twice.

Thanks for the report @hamishwillee I'll look into this.
I think this also brings up another topic we haven't properly discussed "Unit Tests", if we have a setup for unit tests we can probably tests this behavior and apply a fix/test, but lets create another issue for that

Hi @mrpollo
Some simple test code showing the problem (visual inspection only) can be found  here.
Yes, we do need to talk about Unit tests, and that would have found this problem. I don't think some of this will be easy though because even simple testing like checking arming can be dependent on Mode and other factors. It would be really nice to have a compatibility suite too.
Hope you can fix this soon, as examples with workarounds are ugly!
Cheers
H

Hey @hamishwillee, I adapted your test case to make a unit test! https://github.com/diydrones/dronekit-python/commit/1a114948ac0fc1099cbdd13cb6b1b3cbae558997
So if I understand the issue (which still exists) properly:

Clearing a waypoint list should result in one waypoint, "home"
Adding waypoints, then uploading, then downloading, should result in an identical list

And what appears to be happening:

Clearing a waylist list without flushing collides with...
Adding way points, then uploading, which...
Results in a list with one item too few: the first "added" item is overwritten by the "home" waypoint

Specifically, the v.flush() line I added to this test makes it behave properly: https://github.com/diydrones/dronekit-python/blob/1a114948ac0fc1099cbdd13cb6b1b3cbae558997/tests/sitl/test_132.py#L32 If the line is removed, we observe the behavior as seen in your test case.
Does this sound right? With a concrete test I can delve into why this is happening.

@tcr3dr You are correct in every detail - thanks very much. Clearly what I was doing was a mis-use of the API - I will improve the documentation.
That said, with other commands you can normally batch them up and then call flush to make sure they are sent - so sending a mode change and arm request before calling flush works. Others will make this mistake and it would be good if our API was robust enough to help avoid this problem. Not a high priority though, so I will change label above.
As an aside/FYI the use of invisible waypoint "0" as the home point is apparently a hack that will be undone at some point.

@tcr3dr @mrpollo I've run into another issue from this behaviour, which you can find out about in https://github.com/dronekit/dronekit-python/issues/230#issuecomment-125521588.
IMO the change that needs to happen is that DK-Python makes the 1st mission item (Home location) invisible to user through this API - and only ever hands it a CommandSequence from the first "real" waypoint. Under the hood the API should make sure that any commands are always added above the home location.
We would of course need another way to get/set the home location - I suggest an attribute.

@hamishwillee Let's fix this properly for the minor branch then. Vehicle.commands pairs with Vehicle.home or similar. I'll start developing this on my existing PR branch and open a new issue about it.

Sounds great.

Something that I've noticed is that when I clear, flush, add new waypoints, and flush again is that the Home location is set to the first standard waypoint (not takeoff or change speed, etc) in the command sequence. I've created a workaround by just adding the Home waypoint after I clear the list, but I'm gathering from reading this and the Dronekit.io documentation that this is a bug.
Just letting y'all know about it. I can replicate the problem if you need it. My last pull request was a few days ago, so this may have been solved since then anyway.

Hi @jamesbellaero
Yes, if you clear the vehicle.commands then you clear out all waypoints including the home position. When you upload, the bug reported here means that the first command you added will be lost. There are two solution - either add a dummy waypoint first or do a download after clearing in order to populate the first waypoint. I submitted https://github.com/dronekit/dronekit-python/pull/260/ (not published yet) which is supposed to add some clarity on this subject.
The implementation of the autopilot to use the first waypoint as the home position is a misinterpretation of the protocol. In any case though, a user should not need to think about this when using the API. Our plan to address this is to make the API hide the command - a user will have an api to access and handle home, and a separate API to handle missions. https://github.com/dronekit/dronekit-python/issues/252
Thanks for reporting this.

Fixed by #384


[NL NumberWithUnit] Fix Kilobyte definition and enable more tests

Bugfix: Change KB to kB
Activate disabled tests

Still one test failing, which is logged in issue #800

I agree. I added KB back and added a testcase for it. However, this is something that should be fixed in the English patterns and testcases also. The variants used there are incorrect at the moment.
See issue #800


rancher-monitoring k8s 1.27 chart support

This issue is to implement the chart changes & perform the testing required to upgrade to k8s 1.27. Please perform below steps.

[ ] Check whether the chart has any references to the below deprecated features of k8s 1.27. You can find information about deprecated APIs here on Kubernetes Deprecation guide.

storage.k8s.io/v1beta1 API version of CSIStorageCapacity

[ ] Update kube-version annotation in Chart.yaml. catalog.cattle.io/kube-version: '>= 1.23.0-0 < 1.28.0-0'

Note: If this is an upstream chart and if the upstream chart doesn't support k8s 1.27 then do not update the kube-version annotation or update the k8s version accordingly.

[ ] Perform below tests

Fresh install/uninstall/upgrade/downgrade of the charts on K8s 1.27 provisioned on Rancher. Test the chart functionality.
(Please refer to the https://github.com/rancher/rancher/issues/41395 to know what testing was done for k8s 1.26 upgrade for your chart)

Provision rancher on k8s 1.26 or lesser

Install chart on this cluster

Upgrade this cluster in rancher to k8s 1.27

Make sure the existing chart and the app is functional after cluster upgrade

@dbason , Are you blocked on something to pick up this task?

@joshmeranda is currently working on rebasing the monitoring stack.  This will be picked up as part of that work.  The pull request should be up in the next couple of days.

Tested as part of https://github.com/rancher/rancher/issues/42019 by me and @geethub97. Can be closed!

Tested the following test cases of rancher-monitoring chart.
Fresh install checks
RKE1, RKE2, and K3S (Node driver and Custom)
RKE1, RKE2, and K3S (hardened)
RKE1 Custom Cluster
RKE1 imported cluster.
Upgrade checks were performed on RKE1 normal and harden cluster. Cluster upgrade was performed from k8s v1.26 to v1.27 and chart functionality was assessed.
Chart is working as expected hence closing the issue.


Using Paroller on elements at the top of the page

Hi!
I am hoping to have an element positioned at the top of the page (a Hero Banner) and I want to implement paroller so that its' content parallaxes.
At the moment it seems on page-load paroller kicks in and applies the transform despite not started scrolling yet. For example transform: translateY(-84px);, which causes an unwanted gap.
How is it possible to have transform initiate at 0 or null so that it is only applied when the user starts scrolling.
Thank you and great job

Bump @tgomilar?

@elliott-impression please look at this issue.

Super, thanks!


Added columns of json type check and Codecov badge

https://github.com/mfvanek/pg-index-health/issues/153

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@0940cec). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #26   +/-   ##
=========================================
  Coverage          ?   98.55%
  Complexity        ?       29
=========================================
  Files             ?        9
  Lines             ?       69
  Branches          ?        0
=========================================
  Hits              ?       68
  Misses            ?        1
  Partials          ?        0

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Add ability to filter on multiple columns.

Add ability to filter on multiple columns.
applyFilter(filters) {
  filters.forEach(filter => this.applyThisFilter(filter, items));
}

Implemented.  Will be available in aurelia-datagrid v0.1.4


what tigger HTTPProtocol on_message_complete?

what tigger HTTPProtocol on_message_complete?

Do you mean this?
https://github.com/channelcat/sanic/blob/79df52e519588112e43dd771dcd42ea245e732d2/sanic/server.py#L283
The HTTPProtocol is a subclass/extension of the asyncio.Protocol class. It is passed to the asyncio TCP socket data handler to act as the receiving protocol implementation.
It is the asyncio TCP service (in the case of Sanic, it is the one provided by uvloop) that calls on_message_complete after all packets of the TCP message has been received by the socket.

@GzRichChen "on_message_complete" is a callback from HttpRequestParser (httptools library). It's triggered after the HTTP parser consumes all the HTTP request body (Sanic will feed the HTTP parser as data is received from asyncio). @ashleysommer your answer is wrong.


Refactor all the code

The code needs refactoring. Check https://github.com/objectionary/aoi2/pull/21#issuecomment-1448472208

@c71n93 5 puzzles #24, #25, #26, #27, #28 are still not solved.


Tuya/SmartLife Ceiling Fan Controller (FSD) Support

Feature Description
I added a new Tuya/SmartLife Ceiling Fan Controller (FSD) and was surprised that I am not getting a HomeKit accessory added.
I noticed that in the Homebridge UI -> Accessories screen, the fan doesn't show there either, so is it the case that this particular kind of fan + light combo is not yet supported?
To provide context w.r.t. the different types of controls on this composite device, here is a visual of the Smart Life app controls:

Here are some logs from the homebridge-tuya-platform plugin, showing its sending device information and state change requests, which AFAICT are completely ignored by Homebridge:
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"447833e2-41fd-4975-b565-dcfa6a3ddde0","productKey":"puj7wgpctcgpn70b","status":[{"60":true,"code":"fan_switch","t":1702687150,"value":true}]},"t":1702687150,"pv":"2.0","sign":"cd14ced7e9cb05bd595e6a2ff284a783"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"7c5ef505-65a4-4920-8682-8539ba7cebc9","productKey":"puj7wgpctcgpn70b","status":[{"62":1,"code":"fan_speed","t":1702687155,"value":1}]},"t":1702687155,"pv":"2.0","sign":"c5d5c05f2c3739d725dcca9693f20933"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"c0e28d3f-e774-49ed-b3e1-7548167736d9","productKey":"puj7wgpctcgpn70b","status":[{"62":2,"code":"fan_speed","t":1702687156,"value":2}]},"t":1702687156,"pv":"2.0","sign":"942f57e24c3cfd01485f504015fe5474"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"58b5ada6-875f-43e1-9499-be7ac6af05b8","productKey":"puj7wgpctcgpn70b","status":[{"62":3,"code":"fan_speed","t":1702687157,"value":3}]},"t":1702687157,"pv":"2.0","sign":"e41758fb4bdd8c01cd416679556f0149"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"1cd8c92f-0a87-41a7-9916-b40326b765ae","productKey":"puj7wgpctcgpn70b","status":[{"60":false,"code":"fan_switch","t":1702687159,"value":false}]},"t":1702687159,"pv":"2.0","sign":"9e31314995beadbb3e891c486a0076df"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"30c1ef92-0058-4498-8ac1-6d16d96f9e1e","productKey":"puj7wgpctcgpn70b","status":[{"20":true,"code":"switch_led","t":1702687161,"value":true}]},"t":1702687161,"pv":"2.0","sign":"e0da152b6c30a2b864f13cc4e9036cf8"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"89aa26b9-fcc4-428b-ad89-c573d950f998","productKey":"puj7wgpctcgpn70b","status":[{"22":43,"code":"bright_value","t":1702687166,"value":43}]},"t":1702687166,"pv":"2.0","sign":"4a79a48fbe6ca546cff9e08fd8d60c7e"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"4bf6ad60-27d5-4a3d-a59b-f19cec1c1cee","productKey":"puj7wgpctcgpn70b","status":[{"22":100,"code":"bright_value","t":1702687169,"value":100}]},"t":1702687169,"pv":"2.0","sign":"613759e61eb59b1fc38d84afc7deda37"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"c3390f71-6c4f-4ce3-8cd2-9181d7c4af01","productKey":"puj7wgpctcgpn70b","status":[{"23":2,"code":"temp_value","t":1702687171,"value":2}]},"t":1702687171,"pv":"2.0","sign":"01dbb3277aa11cf0926ca06c2cc7791d"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"093543b4-e679-4a27-ac98-f5c5ef7cc24f","productKey":"puj7wgpctcgpn70b","status":[{"23":3,"code":"temp_value","t":1702687173,"value":3}]},"t":1702687173,"pv":"2.0","sign":"605be9a09f8d3c9d2c98f9e5af94914b"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"ba931224-5ec4-4aba-ad87-bad28966386b","productKey":"puj7wgpctcgpn70b","status":[{"20":false,"code":"switch_led","t":1702687175,"value":false}]},"t":1702687175,"pv":"2.0","sign":"36c2a5f822d2d5fa834e60af3057de4a"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"be2b9150-91c0-43b4-876a-34ab72412af9","productKey":"puj7wgpctcgpn70b","status":[{"60":true,"code":"fan_switch","t":1702687180,"value":true}]},"t":1702687180,"pv":"2.0","sign":"a5c87f903d1c70ca02929c108853d680"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"1a37b573-de7c-4596-bce0-6739ab7ebf0b","productKey":"puj7wgpctcgpn70b","status":[{"62":2,"code":"fan_speed","t":1702687183,"value":2}]},"t":1702687183,"pv":"2.0","sign":"1f5c7d0b4753d20f26e7b08fc6bb0475"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"0e1707c3-1986-4bd4-97c5-d6b3c0c88ad7","productKey":"puj7wgpctcgpn70b","status":[{"62":1,"code":"fan_speed","t":1702687184,"value":1}]},"t":1702687184,"pv":"2.0","sign":"d962e4e6a550a44844dbbe85d0daaba7"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"76c39fc5-858c-477c-a2ea-79bc2a2fe200","productKey":"puj7wgpctcgpn70b","status":[{"63":"reverse","code":"fan_direction","t":1702687186,"value":"reverse"}]},"t":1702687186,"pv":"2.0","sign":"63f172dfd67a6f0ec235ad9e60217487"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"34f3f7f8-1e3a-4454-8e79-6a3128cad6f1","productKey":"puj7wgpctcgpn70b","status":[{"63":"forward","code":"fan_direction","t":1702687188,"value":"forward"}]},"t":1702687188,"pv":"2.0","sign":"dfe57ab1d97cde1977f7e462f514c90b"}
TuyaOpenMQ onMessage: topic = cloud/token/in/**redacted**, message = {"protocol":4,"data":{"devId":"eb2347163b32527c6byj9n","dataId":"87b1fb65-7c02-4dd3-9e75-49bb896a9ca5","productKey":"puj7wgpctcgpn70b","status":[{"60":false,"code":"fan_switch","t":1702687189,"value":false}]},"t":1702687189,"pv":"2.0","sign":"9c26c0489cfda954f2224ebc6c4e520d"}

Here is a sample payload for the Tuya FSD device type.  This is their API, but it gives good hints as to what "knobs" are available for the FSD device type:
[
    {
        "code": "switch_led",
        "value": false
    },
    {
        "code": "bright_value",
        "value": 100
    },
    {
        "code": "temp_value",
        "value": 3
    },
    {
        "code": "fan_switch",
        "value": false
    },
    {
        "code": "fan_speed",
        "value": 1
    },
    {
        "code": "fan_direction",
        "value": "forward"
    },
    {
        "code": "fan_countdown_set",
        "value": "cancel"
    }
]

Here are some further details on the values allowed in the payload:
| Code              | Type    | Values                                                                    |
|-------------------|---------|---------------------------------------------------------------------------|
| switch_led        | Boolean | "{true,false}"                                                            |
| bright_value      | Integer | {   "unit": "",   "min": 1,   "max": 100,   "scale": 0,   "step": 1 }     |
| temp_value        | Integer | {   "unit": "",   "min": 1,   "max": 3,   "scale": 0,   "step": 1 }       |
| fan_switch        | Boolean | "{true,false}"                                                            |
| fan_speed         | Integer | {   "unit": "",   "min": 1,   "max": 3,   "scale": 0,   "step": 1 }       |
| fan_direction     | Enum    | {   "range": [     "forward",     "reverse"   ] }                         |
| fan_countdown_set | Enum    | {   "range": [     "cancel",     "1h",     "2h",     "4h",     "8h"   ] } |

Is support for this type of composite fan + light controller a feature that's been requested before?
Could this be mapped to a standard HomeKit light dimmer, and separate HomeKit Fan or Fanv2 device?

@0x5e Is this issue on your radar?

Solved this by simply stopping (not disable!) my old homebridge-tuya-platform, installed this other version:
https://www.npmjs.com/package/@0x5e/homebridge-tuya-platform
Then checked that my configuration got copied over from the old plugin to the new one.
Then disabled the old version.
Couldn't be easier!   Closing this issue!


Adding PathType under Route for Ingress resources

What this PR does / why we need it?:

Adding PathType under route field - mandatory field  for Ingress

Does this PR introduce a user-facing change?

[ ] User guide
[ ] CHANGELOG.md

Which issue(s) this PR fixes:

Fixes #266

Merging the code changes


IOS null token error

Hi,
I'm only opening this issue because I tried everything else I could find.  So, it works normally for Android, but not for IOS.

Already have the Firebase configuration OK for IOS (tried both: APN certificate and Auth Key).
Using code to getToken from the docs:

  async getToken() {
    let token;

    if (this.platform.is('android')) {
      token = await this.firebase.getToken();
    }

    if (this.platform.is('ios')) {
      token = await this.firebase.getToken();
      await this.firebase.grantPermission();
    }

    this.saveToken(token);
  }

Tried the "cordova-plugin-firebase-analytics" fix.
My specs:
Ionic 5.2.3
Cordova 9.0.0<EMAIL_ADDRESS>"cordova-plugin-firebase-analytics": "^4.0.1",
"cordova-plugin-firebasex": "^7.0.1",
"cordova-ios": "^5.1.1"

Another problem I have is that I dont have a MAC, so I use an online tool to build and deploy. So basically I need to debug this on production basically=/
Thank you in advance =]

It turns out that I can run custom cordova hooks on the online build environment, I just need to specify them. Could you please guide me to which hooks I need to execute?
Thank you sir.

You can find the hook scripts used by this plugin in the plugin.xml

First of all @dpa99c, great work on this plugin mate. I would like to buy you a beer (please tell me how).
Now, I have tried everything, but still getting null on ios. I am not sure if this was closed correctly? I am using mac to build with cocoa pods, but still getToken is null.
Did everything that this issue states. Still null. Works on android.
Any reference would be great!

@mtnoronha  did you found a solution ??


Update test expectations

Fixes the test cli/test_cli.sh: scope ps by updating expectations to match cli output

I needed this fixed for #117 so I applied the fix there.


exporting/importing takes longer the mysqldump

exporting and importing a database takes longer than CLI mysqldump. Is there a way to speed this up?
Nothing immediate jumps out in Config.php
specifics: mysqldump created a file of 840M and took approximately 1 minute
this library took just over 15 minutes and created a file size of 930 M

It depends on how mysqldump create inserts. Mysqldump uses multiple rows in one insert command. This is nice for export and size of dump files, but size of one insert depends on setting of mysql server. When you have same setting (net_buffer_length) all work fine, but when source server have bigger value than destination server, insert fails.
Is more secure to make single insert at one row.
You can emulate same behavior with myslqdump --skip-extended-insert option.