[10.x] Add toRawSql, dumpRawSql() and ddRawSql() to Query Builders

For quite some time, many developers have requested to get SQL queries with merged bindings from the query builder (#38027, #39053, #39551, #45705, #45189).
They want to have something like this:
User::where('email', 'foo@example.com')->ddRawSql();
// "SELECT * FROM users WHERE email =<EMAIL_ADDRESS>
Instead of:
User::where('email', 'foo@example.com')->dd();
// "SELECT * FROM users WHERE email = ?"
// [
//  0 =><EMAIL_ADDRESS>// ]

Prior Implementations
All prior implementations had the same issues that prevented them from being merged:

The bindings had been inserted into the query as strings without any further processing. Each of those generated queries was vulnerable to SQL injection attacks.
They replaced all question marks with values that would break on raw calls, e.g. ->whereRaw("description = 'foo?'")

Improved Implementation
These new ways of generating SQL queries with embedded bindings are available:
$sql = User::where('email'<EMAIL_ADDRESS>  ->toRawSql();// $sql = "SELECT * FROM users WHERE email =<EMAIL_ADDRESS>
User::where('email'<EMAIL_ADDRESS>  ->dumpRawSql(); // "SELECT * FROM users WHERE email =<EMAIL_ADDRESS>
User::where('email'<EMAIL_ADDRESS>  ->ddRawSql(); // "SELECT * FROM users WHERE email =<EMAIL_ADDRESS>
$sql = DB::connection()->getQueryGrammar()->makeRawSql(
  'SELECT * FROM users WHERE email = ?',
<EMAIL_ADDRESS>); // $sql = "SELECT * FROM users WHERE email =<EMAIL_ADDRESS>
1. SQL Injections
I've built an extension for the database layer that can escape any values for safe embedding into SQL queries (#46558) that is already merged into Laravel 10.x. Based on that code, any binding is escaped before being injected into the SQL query:
User::where('name', "Robert'; drop table users; --")->dd();
// "SELECT * FROM users WHERE email = 'Robert\'; drop table users; --'"

2. Ambiguous Question Marks
Simple search-and-replace operations are not enough to reliably generate a raw SQL statement. With raw expressions anyone can embed more question marks into a SQL query that are clearly no placeholders:
User::whereRaw("abc = 'Hello World?'")->where('name', 'Robert')->dd();
// "SELECT * FROM users WHERE abc = 'Hello WorldRobert' name = ?"

But this can also be solved relatively easily. The generated SQL string with placeholders is parsed by a very simple LL(1) parser (just 20 lines) to watch for string escape sequences and only replace question not being escaped in string literals:

The occurrence of ' starts a string literal -> no question marks will be replaced
The occurrence of '' and \' marks escaped quotes -> they are copied
The occurrence of ' ends a string literal -> question marks will be replaced again.

That way the generated raw SQL string have no problem with question marks in string literals:
User::whereRaw("abc = 'Hello World?'")->where('name', 'Robert')->dd();
// "SELECT * FROM users WHERE abc = 'Hello World?' name = 'Robert'"

3. Executability of Raw SQL Queries
A generated query by these new functions should be able to be copied and pasted into any query tool and execute without problems. This is guaranteed for any database except PostgreSQL. Because PostgreSQL has special operators involving a question that needs to be doubled because of some PDO behaviour:
User::where('json', '?', 'abc')->dd(); // json object contains key "abc"
// "SELECT * FROM users WHERE json ?? 'abc'"

The query can not be executed as Laravel (correctly!) doubles the question mark (double ones are exempt from replacement 😉). To also make these special queries copy-able any (1) PostgreSQL operator containing a question mark that is (2) included in Laravel's operator information is decoded again:
User::where('json', '?', 'abc')->dd(); // json object contains key "abc"
// "SELECT * FROM users WHERE json ? 'abc'"

Final
This implementation solves any known problems of generating raw SQL string known until today (including mine added for PostgreSQL).
I am open to a different naming for these new methods.

The failing test is of a different module than I have been working on. So not relevant for the PR.

This opens the doors for devs to do stuff like DB::raw($query->toRawSql()); which now depends on your (complex) escaping logic.
Meanwhile I can oneline this output for simple ? replacements (ignoring edgecases) which works in most cases when you quickly need to debug a query:
$sql = str_replace_array('?', $query->getBindings(), $query->toSql());

Thirdly there are query logs for your RDBMS of choice and there's also a nice Laravel debugbar :) (although only useful after execution of the query).

This opens the doors for devs to do stuff like DB::raw($query->toRawSql()); which now depends on your (complex) escaping logic.
Sure. You can always do dumb stuff, but it will still be safe.

Meanwhile I can oneline this output for simple ? replacements (ignoring edgecases) which works in most cases when you quickly need to debug a query:
$sql = str_replace_array('?', $query->getBindings(), $query->toSql());

As you said, there are edge cases that will fail on some queries.

Thirdly there are query logs for your RDBMS of choice and there's also a nice Laravel debugbar :) (although only useful after execution of the query).
You can think of this implementation to also be the foundation stone for these packages. Debugbar and so many packages can use the included implementation that also handles edge cases they all don‘t have to implement again and again.

I'm looking to create a raw query that could then be passed to something like pgsql2shp. @tpetry, could you please clarify if this would work? From your description above it's not immediately apparent to me if Postgres is fully supported. Thanks!

This implementation can generate a raw SQL query for all databases supported by Laravel: MySQL, MariaDB, PostgreSQL, SQLite and SQL Server.

This implementation can generate a raw SQL query for all databases supported by Laravel: MySQL, MariaDB, PostgreSQL, SQLite and SQL Server.

Alright, that should work! pgsql2shp is a PostGIS binary. It creates a shapefile for you based on the raw query you pass to it.

I've had this on a big project a few years ago where we did a home-grown solution. Just as a heads up (and possible feature request) that if you're planning to log your queries, make sure you insert some conditional logic to redact personal/risky information from the queries so that you don't log stuff like passwords, or any data that might violate GDPR.

In which Laravel version is this going to be available? I'm on v10.14.1 and still don't see the changes.

@ARehmanMahi I'm guessing it's not released yet, since this was merged a few hours after v10.14.1 was released.

@ARehmanMahi I'm guessing it's not released yet, since this was merged a few hours after v10.14.1 was released.

\Illuminate\Database\Events\QueryExecuted would be improved if raw sql query was provided.

@tpetry There will be error below on some drivers: The database driver's grammar implementation does not support escaping values. on this line.
I am not sure about side effects but it could be fixed by calling setConnection() on grammer property:
public function toRawSql()
{
    return $this->grammar->setConnection($this->getConnection())->substituteBindingsIntoRawSql(
        $this->toSql(), $this->connection->prepareBindings($this->getBindings())
    );
}

and
    public function getRawQueryLog()
    {
        return array_map(fn (array $log) => [
            'raw_query' => $this->queryGrammar->setConnection($this)->substituteBindingsIntoRawSql(
                $log['query'],
                $this->prepareBindings($log['bindings'])
            ),
            'time' => $log['time'],
        ], $this->getQueryLog());
    }

It is working for all database drivers provided by Laravel. If you‘re using a custom one, it has to make the needed changes.

@AmirHossein same problem on MySQL with multiples databases. It's not documented what is needed to be done since everything else works fine.

When the connection creates the grammar, it has to also pass itself to the grammar:
protected function getDefaultQueryGrammar(): QueryGrammar
{
    $grammar = new QueryGrammar();
    if (method_exists($grammar, 'setConnection')) {
        $grammar->setConnection($this);
    }

    return $grammar;
}

Thanks I didn't notice I was using a custom mysql package for connection, we will make the needed change above in their project.

Please how to use it in create or update or delete instructions?


Added support lowering a new type of "accelerator-only" functions.

This is needed for function-typed attributes in TF ops, such as the dataset_factory
attribute in OneShotIterator.
An accelerator-only function can only be called/used by other TF ops, and never
from host (via TF session API). As such, it has the following requirements:
a) It cannot contain sends/recvs, or any host-side logic.
b) It must be lowered to a single TF device.
To summarize the implementation:

Added a code path to the partition and graph lowering passes to handle
accelerator-only functions:

In partitioning, skip host-side code rewrite in
TFFunctionPartition::insertTensorComputationStartEndTerminate() and
PartitionCloner::finalizeOriginal().

In graph lowering, skip generating top level graph nodes in
TFGraphLowering::buildGraphNodesForTopLevelFunctionCall().

Added support for processing FunctionRefInst as an operand to tfop insts in
the partitioning and graph lowering passes.

@swift-ci please test tensorflow

@rxwei was working on something similar, have you two been talking about this?

@lattner Yes I discussed with Richard before starting this PR -- his frontend work (https://github.com/apple/swift/pull/17460 and the Swift side API design for dataset) compliments well with my backend work here.

Thank you Chris and Richard for the review!
@rxwei Richard, do you think this PR is ready to go in? I believe it's mostly orthogonal to #17460, but if you prefer to land it first, I can hold off on this one.

Great, thanks.


Two works running simultaneously

Describe your issue here.
Steps to reproduce
Change the button to "Cancel Pomodoro"
Expected behaviour
As soon as we start a task from notification bar, the "Start Pomodoro" button should change to "Cancel

Pomodoro".
Actual behaviour
If we start a task from the notification bar, then also we are allowed to start another task simultaneously from the main app
Would you like to work on the issue?
Tell us if you would like to work on this issue

Interested in taking up this issue? Leave a comment expressing your interest.
Each issue expects a 24hr turnaround time by default. In case where a student doesn't come back with an update after 24 hours have elapsed, the issue is up for grabs by anyone.

This is connected to #49


Get existing project properties

Close https://github.com/jhipster/jhipster-lite/issues/2696

This needs https://github.com/jhipster/jhipster-lite/pull/2770 before merging


Fix 177 issue (Core)

This one fixes #177 issue for Core branch.

Related to: https://github.com/poanetwork/deployment-playbooks/pull/181 https://github.com/poanetwork/deployment-playbooks/pull/180 https://github.com/poanetwork/deployment-playbooks/pull/179


Add Dockerfile and docker-compose.yml

Hi! Thanks for making umami. I've just started to play with it, but here's a starting point with some docker files.
To start it, you should be able to just run docker-compose up.
The postgres image has a initdb folder that you can drop any .sql file into, so I mounted the sql/schema.postgresql.sql file into there to have it run on first start up.
Some gotchas that I can think of:

If you change some of the code and want to build a new container, you'll want to rebuild and invalidate the cache with: docker-compose build --no-cache umami.

Oh and I wasn't sure what to do for documentation, so feel free to change that (or any of this for that matter).
Linking relevant issue: https://github.com/mikecao/umami/issues/6.

Looks good, just give me some time to test it out first.

Yep no problem :+1:

Could you please create a CapRover One Click App to make using umami easier for beginners like me?

@Jean-Cote I'm actually working on it. Will be creating a PR there soon.

That is awesome!
Gesendet: Donnerstag, 20. August 2020 um 19:02 Uhr
Von: "Salmanul Farzy"<EMAIL_ADDRESS>An: "mikecao/umami"<EMAIL_ADDRESS>Cc: "Jean Cote"<EMAIL_ADDRESS>"Mention"<EMAIL_ADDRESS>Betreff: Re: [mikecao/umami] Add Dockerfile and docker-compose.yml (#14)

@Jean-Cote I'm actually working on it. Will be creating a PR there soon.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.


[NR-165205] Add codecov

SQUASH TO MERGE

Added codecov & GHA

@cthomas-newrelic ready for review, and will squash merge. Thank you.


ipv6 not working with NPM in Docker

Checklist

Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image?

Yes

Are you sure you're not using someone else's docker image?

Yes

Have you searched for similar issues (both open and closed)?

Yes

Describe the bug
til now i only used ipv4 and npm was working fine, even with an Access List. Now i am beginning to switch to ipv6.
When no Acces List is set, NPM works fine with ipv6. But when i activate an Access List, i get a 403 Forbidden. Even when i allow my ipv6 Subnet in the Access List.
Nginx Proxy Manager Version
v2.9.12
To Reproduce
Steps to reproduce the behavior:

add your ipv6 and ipv4 to an access list
add this access list to a Website that is routed throug npm
open the Website, that is routed through npm and has an access list.
see 403 Forbidden

Expected behavior
open the Website even when coming from ipv6, only if in access list allowed
Screenshots

my Access List

Operating System
nmp is installed on a debian 10 (buster 10.10) in docker 20.10.8, build 3967b7d with enabled ipv6
Additional context
For testing i installed nmp on a alpine Proxmox LXC, with the same Access List. There it was working like a charme.
I found out, that when i am coming from a ipv6 subnet, then this ipv6 IP is natted to a Docker-Internal ipv4 address.

From my understanding the Docker gateway is converting IPv6 traffic to IPv4, thus why you see <IP_ADDRESS> as your client.
I'm in the same boat and have yet to find a functional solution.

yes you are right. why npm-docker is doing that, i dont know.
my solution is, i use an alpine linux VM/LXC. With that it is working and ipv4 and ipv6 are working.
as soon as i have my setup running, i can report.
regards

ok.
i've installed a complet new alpine-lxc on my Proxmox Server and deployed npm.
now it is working as expected.
the only problem was, that i had to configure all my domains an certificates once again, so it would be very goot, if there is a export and import funktionality.
regards

I believe I have also an issue with jc21/nginx-proxy-manager and ipv6.
I have setup a reverse proxy for Nextcloud (besides others). Everything works well, except of the Nextcloud Android App used remotely, not via the local WIFI. I know that this is related to ipv6 as I already had an issue with it about one year ago which I fixed by properly setting up the ipv6 routing in my router (Fritz!Box). Any issue there I can exclude as route case and clearly link it to the Nginx-Proxy-Manager as everything works when disabling NPM and setting up a reverse proxy via the local Nginx of my openmediaVault system.
However, this I did only for testing as I there miss the auto-generated LetsEntscrypt certificate. Therefore I hope that anybody can help with a solution.

I just found that it works with jlesage/nginx-proxy-manager.


local cat alias breaks d in normal mode

I've got cat aliased to bat locally, and it seems that cat is used internally to parse keys using the -v key which isn't supported by bat.

ref: https://github.com/jeffreytse/zsh-vi-mode/blob/b74c0dfc712593b14c4dff6449e58623d7dc7b8f/zsh-vi-mode.zsh#L251
Not sure if this is workable as I know it's not really standard to expect cat to be aliased. Maybe if you go for /usr/bin/cat, but then that might make it harder to support other systems...
I think I could work around by using a function that checks for valid bat args and if it finds something that won't work with bat then defer to cat.

Hi @BerkeleyTrue
🎉 Firstly, thank you very much for your issue reporting, and now this issue has been addressed and settled down. Welcome to star this project for further features in the future.
Thanks and regards

Hi @BerkeleyTrue
🎉 Firstly, thank you very much for your issue reporting, and now this issue has been addressed and settled down. Welcome to star this project for further features in the future.
Thanks and regards

Woah! very nice! Works like a charm. I'll be switching over now. Thanks for the hard work!

Woah! very nice! Works like a charm. I'll be switching over now. Thanks for the hard work!


[SPARK-36223][SQL][TEST] Cover 3 kinds of join in the TPCDSQueryTestSuite

What changes were proposed in this pull request?
In current github actions we run TPCDSQueryTestSuite for tpcds benchmark. But it's only tested under default configurations. Since we have added the spark.sql.join.forceApplyShuffledHashJoin config. Now we can test all 3 join strategies in TPCDS to improve the coverage.
Why are the changes needed?
Improve the coverage of join strategies in the TPCDS.
Does this PR introduce any user-facing change?
No, only for testing.
How was this patch tested?
No need.

Can one of the admins verify this patch?

ok to test

Test build #145066 has started for PR 33510 at commit 8e2f4fb.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49535/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49535/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49535/

Test build #145066 has finished for PR 33510 at commit 8e2f4fb.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145066/

Test build #145119 has started for PR 33510 at commit 343fb58.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49589/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49590/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49589/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49589/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49590/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49590/

Test build #145119 has finished for PR 33510 at commit 343fb58.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145119/

Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49590/
The test failure seems not to be caused my pr.

If there aren't many query files, I think we should better stick to the existing way

I have check all the queries in the tpcds.The query 9 in tpcds don't have join, all the other queries have join.

If we need to add the same CONFIG_DIM for almost all files, I prefer hardcoding this directly.

If we need to add the same CONFIG_DIM for almost all files, I prefer hardcoding this directly.

+1, it's nearly impossible to accurately find out which TPC queries have JOIN and need to add CONFIG_DIM. It's easier to run all the queries 3 times with different join strategies.

Test build #145161 has started for PR 33510 at commit 59ead04.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49632/

Test build #145164 has started for PR 33510 at commit bf6b575.

Test build #145164 has finished for PR 33510 at commit bf6b575.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145164/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49632/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49635/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49632/

Test build #145167 has started for PR 33510 at commit cb0d3bc.

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49635/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49637/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49635/

Test build #145170 has started for PR 33510 at commit cf38cab.

q97 with shuffleHashJoinConf failed because codegen failed. let me take a look.
File 'generated.java', Line 302, Column 1: Thrown exception of type "java.io.IOException" is neither caught by a "try...catch" block nor declared in the "throws" clause of the declaring function

I revert  Revert "[SPARK-32567][SQL] Add code-gen for full outer shuffled hash join",  the q97 passed.

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49637/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49640/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49637/

Test build #145161 has finished for PR 33510 at commit 59ead04.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49640/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145161/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49640/

Test build #145167 has finished for PR 33510 at commit cb0d3bc.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145167/

Test build #145170 has finished for PR 33510 at commit cf38cab.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145170/

Test build #145222 has started for PR 33510 at commit c4ff4ab.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49692/

Test build #145223 has started for PR 33510 at commit 510fa61.

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49692/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49693/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49692/

thanks, merging to master!

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49693/

Thank you @cloud-fan @linhongliu-db and @HyukjinKwon  for review.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49693/

Test build #145222 has finished for PR 33510 at commit c4ff4ab.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145222/

Test build #145223 has finished for PR 33510 at commit 510fa61.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145223/


consensus: impl recover_signers for BlockBody<TxEnvelope>

Related https://github.com/paradigmxyz/reth/issues/11208
The ideal for the reth issue is to restrict the functions to be implemented as much as possible to the alloy Transaction trait however it doesn't make much sense to add a recover_signer or recover_signers method to the Transaction trait because we can only fetch the signer in the Signed<Tx> case, which is almost never the case when trying to implement Transaction.
So I think the simplest approach to replace https://github.com/paradigmxyz/reth/blob/0270128d4f7a9f6fad27dff69273095abdfa7452/crates/primitives/src/block.rs#L564-L566 is to do the similar implementation here in alloy using what already exists in TxEnvelope to recover the signers from an iterator of transactions TxEnvelope.

@emhane Maybe we should keep this for a follow up PR right? My reasoning was as follows:

Currently in reth, the BlockBody contains a vector of TransactionSigned
In alloy we don't have a TransactionSigned type but we have the TxEnvelope type which is equivalent because it is an enum that contains all the Signed variants.

In addition we already have the recover_signer method which is already implemented for TxEnvelope.
So I thought it would be rather logical to make an implementation of BlockBody<TxEnvelope> to use it in reth. But maybe if we want to be even more generic we could open a follow up issue to move the recover_signer and recover_signers methods to the appropriate traits right?


chore: add ci and fix test case

fix #18
Add CI check. The test case cannot run normally in the latest apisix version. Therefore, this PR fix the test case.

nice PR


Add setLayer to TileEngine

Adds a function to set the data to an entire layer at once.

Coverage remained the same at 100.0% when pulling 9259a284ffa335d350e73b2a59704f739201bb4f on Tollefsen:master into 5383a6ba21e7200dbbf503a0ba59712269d4d647 on straker:master.

Amazing! Thanks for even adding docs and tests!


De-meta GPP mods

See https://github.com/Galileo88/Galileos-Planet-Pack/pull/69:
The compatibility data for GPP, GPPClouds*, GPPSecondary and GPPTextures are a mess:

This was due to hardcoded compatibility in the metanetkans.
I've had the PR open for several months, but got not a single person of the Galileo team to look at it, even after several pings on the PR itself, or the forum thread. Several team members have been active both on the GitHub repo and the forum thread in the meantime. Maybe they are purposefully ignoring us, I don't know.
In any case, since this is a live problem, with a worse impact than our average "forgot a comma in the version file" upstream issues and causes people to have broken installs, with several people messaging me about this in the last few months, I decided to de-meta these metanetkans and fix them here.
Changes
The hardcoded compatibilities in all netkan files are removed. That's what the AVC .version file included in the download (and its online copy in the repo) is for, which CKAN will use as source of compatibility data from now on.
The .version file got a new KSP_VERSION_MAX to make sure the releases don't have unbounded future compatibility. This means a tiny bit of extra maintenance to update the maximum compatibility when new KSP versions drop, but gives an extra safeguard to avoid users installing it even if it is broken for the new release.
We can't de-meta version files. Since the version file itself looks good and the unbounded future compat being a minor issue (especially now after the final KSP release), I've decided to keep the vref instead of hardcoding it in our netkans. But I'm going to monitor it, as we can't rely on the GPP team not breaking it.
The zip for GPPTextures doesn't have a version file, so it's now unbounded, which shouldn't be much of a problem since it is dependent on GPP anyway.
To reflect this in CKAN, GPP is added to the dependencies of GPPTextures, as well as GPPCloudsHighRes, GPPCloudsLowRes and GPPSecondary.
The hardcoded download link of GPPTextures is replaced with a kref+asset_match.

Missing comma, I take it as a sign to convert them to YAML.

Ugh, it was inflating all those frozen modules from #8641, another case of https://github.com/KSP-CKAN/xKAN-meta_testing/issues/73 :(

Whoops, I think I put the filter for the Grannus Ribbon Pack on the wrong install stanza :grimacing:
It should be on the Optional Mods/GPP_FinalFrontier/GameData/Nereid one, not the main GPP one:

https://github.com/Galileo88/Galileos-Planet-Pack/pull/64/files


fix:(legacy-plugin-chart-country-map)Fix the territories of Karelia and Murmansk(#30326)

SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

BEFORE

AFTER

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

rusackas
I don't think it's a good idea to add such changes to the Jupyter notebook. Since I found this flaw in the NaturalEarth database, and have already created an issue to fix it. (https://github.com/nvkelso/natural-earth-vector/issues/929)
Therefore, when the corrections appear in NaturalEarth, then it will be correct in Superset.


Rename CoreStage::First to CoreStage::PreStartup

Objective

I just learned that CoreStage::First runs before startup systems. This is incredibly surprising, and explains a number of frame-delay bugs I've been unable to diagnose for a very, very long time.
I couldn't find this in the documentation, and the name of the stage label does not imply it is a special case. By their name, I expected startup systems to run before all normal stages.

Solution

Rename the variant to make it explicit that this is not just the first stage, but a stage that runs before the startup stages.
Alternatively we could update the docs, but I don't think it's sufficient, as the name itself is misleading.

Changelog

CoreStage::First has been renamed to CoreStage::PreStartup to make it clear that the stage runs before startup systems.

Migration Guide

Rename all occurrences of  CoreStage::First to  CoreStage::PreStartup. The behavior of this stage is unchanged.

IMO this can come out of draft mode.

Had to clean up a few other comments I found. Ready now.

There's already a startup stage named PreStartup https://dev-docs.bevyengine.org/bevy/app/enum.StartupStage.html#variant.PreStartup.
First also isn't a startup stage and runs every frame so including Startup in the name is confusing.

I'm happy to consider other suggestions, but it has startup in the name because it is a special case of stage that runs before startup stages. The whole point of using the word "startup" is to communicate that fact. I find it's much more confusing not to make any mention of startup systems in its name. Worst case, it will drive users to seek out the docs if confused by the name, instead of making bad assumptions because the name is misleading.
I considered the name BeforeStartup to avoid the aforementioned enum variant conflict, but it seemed odd to mix Pre and Before in the same enum.

I think I'd prefer to duplicate those systems, keep the name, and remove the confusing behavior.
This will be easier to port directly to stageless.

Yeah, I'm not sure this is the right fix. There's clearly an issue with First running before startup, but PreStartup doesn't really make sense once the startup stage is done.
It seems to me like a better fix would be to not run First before the startup stage.

Oh wow, this is new to me too.
I suppose that First was created as a sort of workaround for the sake of DRY ("Don't Repeat Yourself" ideology), to not duplicate the systems, because Bevy should have working events during startup stages/systems, but also during normal frame updates.
I agree with the comments that the better solution IMO is to just duplicate the systems. Having a special magic stage that runs both before the other core stages and also before startup stages is just confusing. It would be confusing even if documented. If we think about the tradeoff: how much complexity are we introducing to the design, for what benefit it brings ... the current design is not good.
I say remove this special stage.

It sounds like (correct me if I'm wrong) stageless would solve the problem of needing commands to run before other systems to prevent frame delays when, e.g. spawning entities. In that case, CoreStage::First is no longer needed.
So the conclusion is:

Duplicate the event update systems that use First. There would now be event systems in both StartupStage::PreStartup as well as CoreStage::PreUpdate.
Delete CoreStage::First entirely.

Yep agreed!

Are these systems safe to run multiple times in a frame?

It seems to me like a better fix would be to not run First before the startup stage.

Yeah, just schedule CoreStage::First after the StartupSchedule stage. First pretty much exists to update Time. I was already intending to have the startup schedule run before the update schedule once schedule_v3 module is added.

Yup as others have said, First exists for two purposes:

Update Time for this update. This should happen "first"
Flip Event buffers for this update. This should happen "first"

It is a separate stage to ensure that these systems always run "before" all other systems in the schedule (in stageless this will be a system set with before ordering constraint relative to the PreUpdate set).
These constraints cannot and should not change. I believe "First" is a pretty good name for things in this bucket.
The question to answer is: Where do "startup" systems exist in the app logic?
We have a number of options:

Consider startup systems "a part of the first schedule update" (the current impl). Under those constraints startup systems must:

Run after the "systems that must run before all other systems in the schedule" (ex: after time is "updated" and event buffers are flipped).
Run before "user app update" logic

Consider startup systems "before" the first update. Time hasn't been ticked yet. Event buffers havent been flipped yet. This doesn't seem particularly bad.
Consider startup systems "their own separate update". Time is ticked for startup systems, then it is ticked again when the update starts. Same goes for event buffer flips. We won't lose events created this update with the double buffer flip as startup happens before other app logic. However, we will lose events if any were created during app setup (ex: plugin init). These events will be dropped when the second event update happens.  Given that this is possible, I don't think  (3) is viable. It also doesn't make much logical sense to me conceptually, as we have a "one update() corresponds to a frame" conceptual model, but we wouldn't be rendering just "startup stage" results.

Either (1) or (2) feel valid to me. I chose (1) for Bevy because:

Given that "startup systems" are triggered as result of schedule.update, I consider them a part of that update
It seemed like conceptually startup systems should happen "after" we did the initial bookkeeping for a given update.
Having a stage called "First" that ran after systems in an update felt confusing / a misnomer.

I do see why First running before startup could be a bit confusing. But not running "bookkeeping that should happen before anything else in a schedule.update" / considering startup systems "outside" of a schedule.update (despite still running as a result of schedule.update), is also confusing from my perspective. Both states of confusion come from conceptual model mismatches (ex: thinking "startup systems" aren't part of a schedule update vs thinking that they are).
Another consideration: right now it seems logically ok to run our "systems that must run before all other systems in an update" after startup systems (ex: not ticking Time doesn't break its functionality for startup systems. same goes for not flipping event buffers). But thats not necessarily going to be true for the next system we add in that category.

So the conclusion is: [...]

I don't think moving time updates to PreUpdate is a good idea, as it forces everything reliant on time to order itself relative to it. Forgetting to do this would result in those systems sometimes using the "old" time and sometimes using the "new" time. "First" seems like an important concept.
Likewise I don't think double-updating events (once in prestartup and once in preupdate) is a good idea, as it would erase events created during app setup (from the perspective of user app logic on the first update).

Yeah, just schedule CoreStage::First after the StartupSchedule stage. CoreStage::First must exist to guarantee that Time updates before anything else. I think the weird ordering just happened because it's called "first" lol (i.e. it's not out of necessity).

This is option (2) as described above. I think this is valid (currently) but I still think (1) is valid (and maybe preferable). Curious what everyone thinks.

I have a pretty strong preference for 2, because I think it's much easier to teach and remember :)

Closing due to stageless.


Translated into Romanian!

Not sure if I should've replaced those en variables with ro, be sure to check that out and tell me

Nice job @Vladutzu27. Thanks for the contribution


New cls file for LSST DESC Notes

Check it out, @kadrlica  - I adapted aastex61 to make a latex PDF that looks like it was written in Markdown. Ready for review, in the issue/11/descnotecls branch. Do make tex to compile the PDF given your template (which I also extended and cookiecuttered).

cookiecutter uses the prompt's description field to populate the variables if none are entered, so that's good! Thanks for the review.

Yeah, that's what I was saying we might want to change, but I don't have strong feelings.

Closes #11


Memory leaks in subscription socket

I recieve memory leaks (few kilobytes per message recieved) when using pub-sub pattern at the subscriber side. It looks independent from recieved message size
My code is (simplified):
while (true)
{
    var eventData = _socket.Receive();
    if (eventData == null)
    {
           continue;
    }
    HandleData(eventData);
}

I use version from NuGet

_socket is SubscribeSocket

I expect this is an issue with nanomsg itself because NNanomsg is all manged code. We are using the last nanomsg release (v0.5). I believe there are substantial fixes after this in master, though I also believe it currently doesn't compile under Windows. You could try compiling a more recent version of nanomsg and see if that fixes the issue.

But actually, how are you measuring that? if you're just watching the process size grow, it'll just be that the GC isn't freeing memory allocated in Receive very quickly - it will eventually.

I use memory profiler (dotMemory) and see that manage memory is freed but unmanaged slowly grows. I'll try to build latest release of nanomsg and check leaks there

Sergey - Is this specific to a transport?

I found an issue with the handling of zero-byte messages and subscriptions which could produce a native leak.  Seems unlikely, but any chance that this is what you're seeing Sergey?
If not, please include some more details about your platform and scenario.  I couldn't reproduce this for  non-zero length messages on any of the transports.

I've found memory leak with non-zero messages (I've not tested zero-byte messages). And it is seems to be independent of message size - something like unmanaged descriptor leak.
I see memory leaks on Windows 7 Pro x64, but where are no any leaks on Windows 8.1 Pro x64. I'll check this more carefully
I use tcp transport for pub-sub sockets and localhost connection

Any updates on this issue?

There is a new version of nanomsg out (0.6) that I haven't added binaries for because I don't currently have a windows machine ... This version evidently fixes a number of issues so it might be worth trying that. The new binaries should be able to be just dropped in without changes to NNanomsg (compile instructions are in the NNanomsg readme). I've been thinking it might also be worth trying to wrap mangos - the go implementation which is evidently much more stable at this point. I haven't looked into it but I believe it to be possible.


Aligning within SliverFillRemaining

This layout can be achieved using a SliverFillRemaining to put the button on the bottom, but it's not clear if or how the cat picture could be centered in this case.

The use case here is that the layout should not really scroll if the screen is big enough and the image small enough, but should scroll if the image is especially tall or the screen too small. It's possible to precompute all of this and decide which widget to build (a scroll view or just a column), but that seems clunky and harder to test for developers.
See also https://github.com/evelyne24/flutter_layout_weirdness
/cc @goderbauer @Piinks @slightfoot @evelyne24

Loosely related: #38640

@Piinks  says there's a way to do this

Playing with it now. I'll share a repro when I have it ready for feedback. :)

So there is a bunch of extra code here, but I wanted to share the full picture of what is going on in this gif. I believe this layout is possible with the refactoring that was done to SliverFillRemaining in #35810.
Let me know what you think! 😃

import 'package:flutter/material.dart';
import 'package:flutter/rendering.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(home: HomePage());
  }
}

class HomePage extends StatefulWidget {
  @override
  _HomePageState createState() => _HomePageState();
}

class _HomePageState extends State<HomePage> {
  bool fillOverscroll = false;
  bool hasScrollBody = false;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: Text('Layout Demo')),
      body: CustomScrollView(
        slivers: <Widget>[
          SliverToBoxAdapter(
            child: Container(
              color: Colors.amberAccent,
              child: Column(
                mainAxisAlignment: MainAxisAlignment.center,
                children: <Widget>[
                  Padding(
                    padding: const EdgeInsets.all(8.0),
                    child: Row(
                      mainAxisAlignment: MainAxisAlignment.center,
                      children: <Widget>[
                        Checkbox(
                          value: fillOverscroll,
                          onChanged: (bool newValue) => setState(() => fillOverscroll = newValue),
                        ),
                        Text(
                          'fillOverscroll',
                          style: TextStyle(fontSize: 15),
                        ),
                      ],
                    ),
                  ),
                  Padding(
                    padding: const EdgeInsets.all(8.0),
                    child: Row(
                      mainAxisAlignment: MainAxisAlignment.center,
                      children: <Widget>[
                        Checkbox(
                          value: hasScrollBody,
                          onChanged: (bool newValue) => setState(() => hasScrollBody = newValue),
                        ),
                        Text(
                          'hasScrollBody',
                          style: TextStyle(fontSize: 15),
                        ),
                      ],
                    ),
                  ),
                ],
              ),
            ),
          ),
// The Good Stuff -----
          SliverFillRemaining(
            hasScrollBody: hasScrollBody,
            fillOverscroll: fillOverscroll,
            child: Column(
              mainAxisSize: MainAxisSize.min,
              children: <Widget>[
                Flexible(
                  child: Center(child: FlutterLogo(size: 100)),
                  fit: FlexFit.loose,
                ),
                Padding(
                  padding: const EdgeInsets.all(8.0),
                  child: RaisedButton(
                    child: Text('Bottom'),
                    onPressed: () {},
                  ),
                ),
              ],
            ),
          ),
// ---------------
        ],
      ),
    );
  }
}

Great, I'll give that a go! Thanks for the code sample! 🎉


Adding Bytesafe to the OSPO Tools/SCA section

Pre-submission checklist:
Please check each of these after submitting your pull request:

[x] Are you only including a repo_url if your project is 100% open source? If so, you need to pick the single best GitHub repository for your project, not a GitHub organization.
[x] Have you picked the single best (existing) category for your project?
[x] If submitting a tool, is your project closed source or, if it is open source, does your project have at least 100 GitHub stars?
[x] Does it follow the other guidelines from the new entries section?
[x] Have you included a URL for your SVG or added it to hosted_logos and referenced it there?
[x] Does your logo clearly state the name of the project/product and follow the other logo guidelines?
[x] Does your project/product name match the text on the logo?
[x] Have you verified that the Crunchbase data for your organization is correct (including headquarters and LinkedIn)?
[ ] ~10 minutes after opening the pull request, the CNCF-Bot will post the URL for your staging server. Have you confirmed that it looks good to you and then added a comment to the PR saying "LGTM"?

LGTM

FYI: adding @caniszczyk to double-check, but I think everything seems in order and this tool can be merged


featureset nereids cbo weights by session var

Proposed changes
good for tune cost model
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall


Mixing direct/inverse precedence locks doesn't yield the expected result

We have a jenkins job that uses locks to avoid running a specific stage in parallel even if the job has multiple instances running at the same time.
We normally lock the resource with inversePrecedence: false to have a FIFO-like behaviour. In some cases, however, we want one specific run to 'jump' over the queue, and that is possible with a build-specific parameter. However, the build that acquires the lock with inversePrecedence: true is still executed after all the other builds are done.
The LIFO-like behaviour is only respected when all the locks are acquired with inversePrecedence: true.
It would be great if the plugin could handle this mixed behaviour.

A similar use case is low priority builds:

you normally run builds with inversePrecedence: true
you can sometime specify inversePrecedence: false and they go at the back of the queue
Although, it seems if we are mixing the LIFO/FIFO model a more flexible feature could be a priority field (defaults to 100) so that locks are acquired in order of priority.

It doesn't solve the current issue with inversePrecedence doesn't react as expected, but if you want a work around for allowing jumping the queue, here an exemple with a optional lock in front of the real one:
pipeline {
    options {
        ansiColor('xterm')
        timestamps()
    }
    parameters {
        booleanParam(name: 'FASTLANE', defaultValue: false, description: 'Toggle this value for fastlane deployment')
    }
    stages {
        stage('Slowlane? lock') {
            options {
                lock(extra: (params.FASTLANE ? [[]] : [resource: 'hyperloop-deployment-slowlane']))
            }
            stages {
                stage('Deploy') {
                    options {
                        lock(resource: 'hyperloop-deployment')
                    }
                    steps {
                          script {
                              echo 'Doing the deployment'
                              sleep(time: 120, unit: "SECONDS")
                          }
                    }
                }
            }
        }
    }
}

looks to be fixed


TensorFlow master build failing on Ubuntu18.04 x86

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: NA
TensorFlow installed from (source or binary): source
TensorFlow version: master as on today
Python version:  2.7.x
Bazel version (if compiling from source): 0.19.0 ( installed using bazel-0.19.0-installer-linux-x86_64.sh)
GCC/Compiler version (if compiling from source): gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
CUDA/cuDNN version: NA
GPU model and memory: NA

Describe the problem
Tensorflow build fails with an error:
In file included from external/hwloc/include/private/private.h:29:0,
from external/hwloc/hwloc/traversal.c:11:
external/hwloc/include/private/misc.h:491:10: fatal error: xlocale.h: No such file or directory
#include "xlocale.h"
^~~~~~~~~~~
compilation terminated.
Steps to reproduce:
git clone https://github.com/tensorflow/tensorflow/
cd tensorflow
./configure
./configure
Extracting Bazel installation...
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.19.0 installed.
Please specify the location of python. [Default is /usr/bin/python]:
Found possible Python library paths:
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
Please input the desired Python library path to use.  Default is [/usr/local/lib/python2.7/dist-packages]
Do you wish to build TensorFlow with XLA JIT support? [Y/n]: N
No XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: N
No OpenCL SYCL support will be enabled for TensorFlow.
Do you wish to build TensorFlow with ROCm support? [y/N]: N
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]: N
No CUDA support will be enabled for TensorFlow.
Do you wish to download a fresh release of clang? (Experimental) [y/N]: N
Clang will not be downloaded.
Do you wish to build TensorFlow with MPI support? [y/N]: N
No MPI support will be enabled for TensorFlow.
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: N
Not configuring the WORKSPACE for Android builds.
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
--config=mkl            # Build with MKL support.
--config=monolithic     # Config for mostly static monolithic build.
--config=gdr            # Build with GDR support.
--config=verbs          # Build with libverbs support.
--config=ngraph         # Build with Intel nGraph support.
--config=numa           # Build with NUMA support.
--config=dynamic_kernels        # (Experimental) Build kernels into separate shared objects.
Preconfigured Bazel build configs to DISABLE default on features:
--config=noaws          # Disable AWS S3 filesystem support.
--config=nogcp          # Disable GCP support.
--config=nohdfs         # Disable HDFS support.
--config=noignite       # Disable Apache Ignite support.
--config=nokafka        # Disable Apache Kafka support.
--config=nonccl         # Disable NVIDIA NCCL support.
Configuration finished
bazel --host_jvm_args="-Xms512m" --host_jvm_args="-Xmx1024m"  build  -c opt  //tensorflow/toockage:build_pip_package

Looks like glibc version is upgraded on Ubuntu 18.04
ldd --version
ldd (Ubuntu GLIBC 2.27-3ubuntu1) 2.27
As mentioned [here](https://sourceware.org/ml/libc-alpha/2017-08/msg00010.html , https://sourceware.org/glibc/wiki/Release/2.26#Removal_of_.27xlocale.h.27) ,  The nonstandard header <xlocale.h> has been removed.
This comes through third_party code hwloc (https://github.com/tensorflow/tensorflow/tree/master/third_party/hwloc)
Any workaround for this? I am not sure about functionality impact  if we disable including xlocale.h.

Or should downgrade gcc ( build from source or installing particular version) ?

A hack I've seen elsewhere is:
ln -s /usr/include/locale.h /usr/include/xlocale.h
Really the build should be using locale.h and not xlocale.h

@wdirons ya..This hack worked.
I was also looking for gcc version which might mentioned by TensorFlow community.

Btw looks like community has made changes around xlocale.h through this commit.
Will try building TensorFlow master again .

@Nayana-ibm Is this still an issue? Did you get a chance to build TF again? Thanks!

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

I think we can also work around the problem by unsetting HAVE_XLOCALE_H
however, I cannot even find where we are setting it in the first place :/

Looks like https://github.com/tensorflow/tensorflow/pull/26156 fixed this.


[feat] Implement fee function

Implement function to calculate transaction fee:
https://github.com/keep-starknet-strange/raito/blob/4bf3aaba4d04f53255f32d022d3ee860ad20da8d/src/validation.cairo#L44
ZeroSync version:
https://github.com/ZeroSync/ZeroSync/blob/f1bd4736b0bac190ac48cbc8d5fcd92e2e1be7b9/src/transaction/transaction.cairo#L262

Hi, can I take this?


RFC: Periodic full compaction (#110)

/cc @tonyxuqqi @v01dstar

/cc @v01dstar

Added graphs. Ready to merge.

/cc @Connor1996 @bufferflies

Added a bit on disk metrics to the future work section.

/merge

Added a bit on disk metrics to the future work section.

It should be better if we can che

Added a bit on disk metrics to the future work section.

If an internal API interface could be provided to inform the caller that TiKV's resources are relatively idle, I believe it would be quite useful. We could rely on this interface to accomplish various tasks, such as increasing scheduling speed, performing compaction, and increasing the concurrency of analyze operations.

/merge


how to generate dataset structure like you mentioned

Hi, thanks for your repo. I have a question about how to generate dataset structure like this?

i downloaded dataset from this website http://salicon.net/challenge-2017/, and fixation data is in .mat file, how could i generate .png fixation files?


Ability to manage groups through the provider

Hi,
I couldn't find any mention of this elsewhere, so I'm opening an issue.
The provider currently does not have to capability to manage OneLogin groups.
Is this planned by any chance?

This provider looks unmaintained. I created a maintained version here with a few bugfixes and am planning on releaseing some features (related to auth servers - scopes, claims and client apps)

Hey @kda-jt,
Would you help me out and clarify which part of the Onelogin group you are seeking to manage through the provider?
Users are able to be added and removed from groups through the user endpoint OneLogin groups
With some clarification we are more than willing to fix any issues and begin investigating

@ebarti

This provider looks unmaintained.

It sure looks like it.

I created a maintained version here with a few bugfixes and am planning on releaseing some features

Kudos to you, looking forward to seeing the new goodies

This provider looks unmaintained.

It sure looks like it.

@Jmfwolf will be maintaining this repo.

We are in the process of standardizing our SDKs by using OpenAPI specifications.
I have an OpenAPI generated terraform-provider-onelogin in prerelease if you would like t try it out.
https://github.com/onelogin/terraform-provider-onelogin/tree/openapi

@Jmfwolf I'm no longer a OneLogin user.
I'll keep the issue open as I do believe this is an interesting feature.


Human Annotation Analysis

The annotations with prompt, classification, overall_rating, image_text_alignment_rating, fidelity_rating are released. Will the annotations of image issues (i.e. repeated generation, body problems, fuzzy images, toxic, pornographic, violent, or violating protected groups in your papers) be released? Thank you!

Thanks for your attention! Please refer to #75 to find other annotation results.

Thanks for your reply. However, I found that 'problem_data.json' contains only prompts. Does it contains the detail classifaction of problem (e.g. 'xxxxxx.webp' has body problems, 'xxxxxx.webp' has pornographic problems)?

+1, I also couldn't find overall_rating, image_text_alignment_rating, fidelity_rating

谢谢你的回复。但是我发现'problem_data.json'只包含提示。它是否包含问题的详细分类（例如'xxxxxx.webp'有身体问题，'xxxxxx.webp'有色情问题）？Did you find the file with the image problem annotations?


AtlasEngine: Improve appearance of curly underlines

We'd previously subtract one underline-height from the curly line
offset, even though we already had subtracted its complete height.
Additionally, the pixel shader received some fine tuning:

Shrink the stroke width so that the anti-aliasing can be seen
all the way up to the horizontal edges of the bounding box.
Add a phase shift to break apart the symmetry of the curve.

Closes #17482

All of these are with the Hack font:
Before, 10pt, 100% scale, 800% zoom:

After, 10pt, 100% scale, 800% zoom:

Before, 10pt, 150% scale, 800% zoom:

After, 10pt, 150% scale, 800% zoom:

Before, 12pt, 300% scale, 400% zoom:

After, 12pt, 300% scale, 400% zoom:

Regarding the last image: The line is actually supposed to be 1px wide (according to the font's underline width). This probably best demonstrates how the previous pixel shader drew the line too "fat".


Trying to compile for webassembly

I have created a hello_wasm project using https://developer.mozilla.org/en-US/docs/WebAssembly/Rust_to_Wasm
This is Cargo.toml
[package]
name = "hello-wasm"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[lib]
crate-type = ["cdylib"]

[dependencies]
wasm-bindgen = "0.2"
rust-crypto-wasm = "0.3.1"
aes-gcm = "0.10.3"
getrandom = { version = "0.2", features = ["js"] }
ece-native = "0.4.0"
once_cell = "1.19.0"
base64ct = { version = "1.6.0", features = ["alloc", "std"] }
web-push-native = "0.4.0"

This is html
<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
    <script type="module">
        import init, { test_encryption_decryption_with_serialization } from "./pkg/hello_wasm.js";
        init().then(() => {
            test_encryption_decryption_with_serialization();
        });
    </script>
</head>

<body>

</body>

</html>

This is src/lib.rs
extern crate crypto;

use crypto::{digest::Digest, sha2::Sha256};
use wasm_bindgen::prelude::*;

use base64ct::{Base64UrlUnpadded, Encoding};

use web_push_native::{
    decrypt, encrypt,
    jwt_simple::{self, reexports::serde_json, algorithms::ES256KeyPair},
    p256::{self, PublicKey}, Auth, WebPushBuilder,
};

use web_push_native::p256::elliptic_curve::sec1::ToEncodedPoint;

#[wasm_bindgen]
pub fn test_encryption_decryption_with_serialization() {
    // Placeholders for variables provided by individual clients. In most cases,
    // these will be retrieved in-browser by calling `pushManager.subscribe` on
    // a service worker registration object.
    const ENDPOINT: &str = "";
    const P256DH: &str = "";
    const AUTH: &str = "";

    // Placeholder for your private VAPID key. Keep this private and out of your
    // source tree in real projects!
    const VAPID: &str = "";

    let key_pair = ES256KeyPair::from_bytes(&Base64UrlUnpadded::decode_vec(VAPID).unwrap()).unwrap();
    let builder = WebPushBuilder::new(
        ENDPOINT.parse().unwrap(),
        PublicKey::from_sec1_bytes(&Base64UrlUnpadded::decode_vec(P256DH).unwrap()).unwrap(),
        Auth::clone_from_slice(&Base64UrlUnpadded::decode_vec(AUTH).unwrap()),
    )
    .with_vapid(&key_pair, "mailto:john.doe@example.com");

    let len = builder.build("content");
}

compile, run and open in browser
wasm-pack build --target web --debug
python -m http.server

error in console log in chrome dev tools
Uncaught (in promise) RuntimeError: unreachable
    at __rust_start_panic (hello_wasm_bg.wasm:0x17ab2b)
    at rust_panic (hello_wasm_bg.wasm:0x17a7bc)
    at std::panicking::rust_panic_with_hook::h62090d3fc0630473 (hello_wasm_bg.wasm:0x13ddaa)
    at std::panicking::begin_panic_handler::{{closure}}::h823c80ef219a166d (hello_wasm_bg.wasm:0x14d820)
    at std::sys_common::backtrace::__rust_end_short_backtrace::hcc7222b556723586 (hello_wasm_bg.wasm:0x17aad1)
    at rust_begin_unwind (hello_wasm_bg.wasm:0x170a9f)
    at core::panicking::panic_fmt::hd79411a297d06dc8 (hello_wasm_bg.wasm:0x174955)
    at core::panicking::assert_failed_inner::h003d4b5b03930769 (hello_wasm_bg.wasm:0x110a73)
    at core::panicking::assert_failed::h985a4a1c1b7563ab (hello_wasm_bg.wasm:0x160a2b)
    at <&generic_array::GenericArray<T,N> as core::convert::From<&[T]>>::from::h68a9a2e49c3eb98d (hello_wasm_bg.wasm:0x1344df)

Hi! I am personally not a Rust WASM expert, but from what I can tell this is simply a result of you not providing values for the placeholder variables in the example. You can read the more fully-fledged example at ./example to find how to populate them correctly.
Also, I don't know what you are trying to do exactly, but usually one would use the Push Web API in the browser and a library like this on the server. Compiling this library to WASM should not be necessary most of the time.

Thanks for reply.
I want to use this for getting request header and body parts. I have tried many implementation nodejs, other c project ompiled using emiscripten and none of those working.
I have tried your library and it works for this example.
use aes_gcm::{
    aead::{Aead, AeadCore, KeyInit, OsRng, rand_core::RngCore},
    Aes256Gcm,
    Key, // Or `Aes128Gcm`
};

#[wasm_bindgen]
extern "C" {
    pub fn alert(s: &str);
}

#[wasm_bindgen]
pub fn greet(name: &str) {
    // alert(&format!("Hello, {}!", name));
}

#[wasm_bindgen]
pub fn test_crypto(str: &str) {
    let mut sha = Sha256::new();
    sha.input_str(str);
    // alert(&format!("Sha256, {}!", sha.result_str()));
}

#[wasm_bindgen]
pub fn aes_gcm_128_encrypt() {
    // The encryption key can be generated randomly:
    let key = Aes256Gcm::generate_key(OsRng);

    // Transformed from a byte array:
    let key: &[u8; 32] = &[42; 32];
    let key: &Key<Aes256Gcm> = key.into();

    // Note that you can get byte array from slice using the `TryInto` trait:
    let key: &[u8] = &[42; 32];
    let key: [u8; 32] = key.try_into().unwrap();

    // Alternatively, the key can be transformed directly from a byte slice
    // (panicks on length mismatch):
    let key = Key::<Aes256Gcm>::from_slice(&key);

    let cipher = Aes256Gcm::new(&key);
    let nonce = Aes256Gcm::generate_nonce(&mut OsRng); // 96-bits; unique per message
    let ciphertext = cipher
        .encrypt(&nonce, b"plaintext message".as_ref())
        .unwrap();
    // alert(&format!(
    //     "encrypted, {:?}!",
    //     str::from_utf8(ciphertext.as_ref())
    // ));
    let plaintext = cipher.decrypt(&nonce, ciphertext.as_ref()).unwrap();
    // alert(&format!("decrypted, {:?}!", str::from_utf8(&plaintext)));
}

So, I want to use your library to get encrypted and send using fetch in browser. It is very convenient without using any server. Just running a static a web page like github pages, fetch subscription data and use your library to encrypt message and then send to subsriber using fetch.
If possible can you create a library just for get request header with options and encrypted data.

My hypothesis in above error is that plaintext encrypted is not able stored because of limited space, may be?

Hmmm, so you want to use the Web Push API entirely on the client side, without a server? This seems unnecessary and not like what the API is designed to do. If you just want to send desktop notifications from a static web page, maybe instead use the Notifications API.
Maybe I'm misunderstanding what you want to achieve. Do you want to send push messages from one web browser to a different one? That's currently the only use-case I can think of where this approach might make sense.

Do you want to send push messages from one web browser to a different one?

No
I have created a static blog website and I have used google sheet to get subscription data using google script. Currently not planning to get hosting service. I will fetch the subscription data from google sheet and use my "private" static page to send notification any user on new blog post.

You do understand that you will have to store the notification endpoint, secret, etc. for every user you want to send a Web Push notification to, yes? It is unclear to me how you intend to achieve this without a server.

Okay I am providing the exact JS code I am trying. In below code I can use fetch send the request. The only problem I am facing is to get     body: cipherText in request header. So, I just need ciphertext using provided key. Thats all. I will delete this message just in case may contain sensitive info.
const jws = require('jws');
const asn1 = require('asn1.js');
const url = require('url');
const Buffer = require('buffer/').Buffer;
const ece = require('./ece.js');
const createECDH = require('./createECDH.js');
const crypto = require('crypto-js');

const ECPrivateKeyASN = asn1.define('ECPrivateKey', function () {
    this.seq().obj(
        this.key('version').int(),
        this.key('privateKey').octstr(),
        this.key('parameters').explicit(0).objid()
            .optional(),
        this.key('publicKey').explicit(1).bitstr()
            .optional()
    );
});

function toPEM(key) {
    return ECPrivateKeyASN.encode({
        version: 1,
        privateKey: key,
        parameters: [1, 2, 840, 10045, 3, 1, 7] // prime256v1
    }, 'pem', {
        label: 'EC PRIVATE KEY'
    });
}

function getFutureExpirationTimestamp(numSeconds) {
    const futureExp = new Date();
    futureExp.setSeconds(futureExp.getSeconds() + numSeconds);
    return Math.floor(futureExp.getTime() / 1000);
}

const privateKey = "rWt2iY4Zg-WvgA0I9O7DqDspYc4iZeehlcX4xeWc-8U";
const privateKeyBuffer = Buffer.from(privateKey, 'base64');
const publicKey = "BAaMW3K7zlPJW7XZUXbfI1an7CmodF9CrcuQQfvbX8ThP8v4LAPeE8OTDMvosy-p3C9fuJnwV0IWiOc4Pd6dE_8";

const subscription = {
    endpoint: 'https://fcm.googleapis.com/fcm/send/cVE7R8hJyy8:APA91bGvrk0VdBosG1uGD5btPrZDHbfk9LN5CGePfUDD-UfHnuPbnuhX748NTLl4iNzugWhNv5lk-8O3LrS2wiHjmSaCRY9O3W3KqlSPPBm-bi1tMaatJ2O9xs7Ad73R8bdRjX2hzMpn',
    keys: {
        auth: 'y9A0RuJwUMuylFKpBSjv3Q',
        p256dh: 'BJnDYSESgGPRnWon7yv_oBrL8uRttrJ7oZSKNo8LOYh65EqoXwIeCHI-0P4qX4RYe3zWjs9U_g9NUQUdOlyTQq0'
    }
};

const parsedUrl = url.parse(subscription.endpoint);
const audience = parsedUrl.protocol + '//' + parsedUrl.host;
const DEFAULT_EXPIRATION_SECONDS = 12 * 60 * 60;

const header = {
    typ: 'JWT',
    alg: 'ES256'
};

const jwtPayload = {
    aud: audience,
    exp: getFutureExpirationTimestamp(DEFAULT_EXPIRATION_SECONDS),
    sub<EMAIL_ADDRESS>};

const jwt = jws.sign({
    header: header,
    payload: jwtPayload,
    privateKey: toPEM(privateKeyBuffer),
});

// userPublicKey - subscription.keys.p256dh
// userAuth - subscription.keys.auth
// contentEncoding - 'aes128gcm'

const localCurve = createECDH('prime256v1');
const localPublicKey = localCurve.generateKeys();
// const salt = crypto.lib.WordArray.random(16);

console.log("localPublicKey", localPublicKey);
console.log("salt", salt);

const payload = Buffer.from("Your Push Payload Text");

const cipherText = ece.encrypt(payload, {
    version: 'aesgcm',
    dh: subscription.keys.p256dh,
    privateKey: localCurve,
    salt: salt,
    authSecret: subscription.keys.auth,
});

const requestDetails = {
    method: 'POST',
    headers: {
        'Content-Type': 'application/octet-stream',
        'Authorization': 'vapid t=' + jwt + ', k=' + publicKey,
        'Content-Encoding': 'aes128gcm',
        'Urgency': 'normal'
    },
    body: cipherText
};

console.log(requestDetails);

// send the push notification
fetch(subscription.endpoint, requestDetails);

Ok, if you simply want to use my library to generate the encrypted message to send to the subscription endpoint, I would recommend you use the web_push_native::encrypt function. You only need to give it the message, user public-key and user auth and it should return the encrypted message.

Thanks for reply. I will try and update.


How can we use formula with this

Kindly tell us how to use formula with this.
Regards
gmchaturvedi
Thanks for filing an issue!
If at all possible, please provide a Codepen to demonstrate the problem you're having with React Quill. Here's a template to get you started.
Quill version:

[ ] master
[ ] 0.4.1
[ ] 1.0.0-beta-1
[ ] 1.0.0-beta-2
[ ] 1.0.0-beta-3
[ ] 1.0.0-beta-4
[ ] 1.0.0-beta-5
[ ] 1.0.0-rc-1
[ ] Other (fork)

It should work if you add formula to the modules.toolbar and the formats props? https://github.com/zenoamaro/react-quill#custom-toolbar


Storage: Use only one param for handling the ID

SiteStorage -> use only siteIDs and remove siteID
Do the same for all storage classes

Done


Unable to show the first item as image in the container in the mobile view port(probably because it is not responsive)

https://www.loom.com/share/0ff093bf68ef4dc99eb2a60141647c36

@aswathkk  comment : Observations from DOM:

The height of .t--widget-imagewidget becomes “auto” in the mobile viewport is the issue

Working as expected. Moving to done.


Fixed mysql connection

For all drivers aside from pymysql, using a context manager like with db.connect() as connection returned a connection object. This was then passed to pandas which retrieved the cursor, executed the query and closed the cursor.
Pymysql's connection object returns a cursor object when used in a context manager. This PR adds a class which overrides the pymysql connection object to ensure that the connection object is returned on enter and closed on exit (which was not happening on the pymysql version). Note: This class could have been nested in the connect method, but that would make it hard to test, so I extracted it into a separate method.

Coverage decreased (-0.4%) to 90.201% when pulling b844ed7987fce3589e35cb10fc6721198b07a7f5 on mikeengland:master into 526e93905ac3081f1002f273b59e84860726402c on kayak:master.

Coverage decreased (-0.4%) to 90.201% when pulling b844ed7987fce3589e35cb10fc6721198b07a7f5 on mikeengland:master into 526e93905ac3081f1002f273b59e84860726402c on kayak:master.


Build against NiFi version 0.5

Update the project to build against the Version 0.5 API

Closed by #6.


Improve paket's conflict reporting during resolution time

Paket is being odd and downloading a lot of unnecessary packages.  Sorry I can't give you a repro repo right now.  I'll try to later tonight.
Paket version <IP_ADDRESS>

It doesn't download these packages. It tries to find a resolution. It seems
there are very narrow dependency restrictions.
On Oct 26, 2015 8:30 PM, "Jimmy Byrd"<EMAIL_ADDRESS>wrote:

Paket is being odd and downloading a lot of unnecessary packages. Sorry I
can't give you a repro repo right now. I'll try to later tonight.
Paket version <IP_ADDRESS>
[image: screen shot 2015-10-26 at 3 25 46 pm]
https://cloud.githubusercontent.com/assets/1490044/10739982/03f0eef0-7bf6-11e5-8ca9-0cb36fdf0981.png
—
Reply to this email directly or view it on GitHub
https://github.com/fsprojects/Paket/issues/1166.

Turn on verbose mode - usually gives the answer as to why this is.

Ah ok.  Will do.

Ok, so I was pinned to Nancy 1.2.0 in my paket.dependencies but when I added a reference to Nancy.Serialization.JsonNet, it started spinning its wheels looking for the 1.3.0.  Updated Nancy to 1.3.0 and it was fine.

So I guess if we would report like every nth conflict during resolution
(without using -v) this would already have helped you. Right?
On Oct 26, 2015 8:49 PM, "Jimmy Byrd"<EMAIL_ADDRESS>wrote:

Closed #1166 https://github.com/fsprojects/Paket/issues/1166.
—
Reply to this email directly or view it on GitHub
https://github.com/fsprojects/Paket/issues/1166#event-445987147.

@forki was just thinking this.

Yeah that would be a good idea.

could you please retry that failing state with paket 2.17.2 and check if it's better?

Sure, I'll take a look.

It started trying to resolve about 10 minutes ago, and I haven't gotten a warning yet.  I'll try to get you a repro repo.

Just found my bug. But yes please send repro.
On Oct 27, 2015 14:59, "Jimmy Byrd"<EMAIL_ADDRESS>wrote:

It started trying to resolve about 10 minutes ago, and I haven't gotten a
warning yet. I'll try to get you a repro repo.
—
Reply to this email directly or view it on GitHub
https://github.com/fsprojects/Paket/issues/1166#issuecomment-151504457.

This is the simplest case I could come up with.
https://github.com/TheAngryByrd/ResolveWarningRepro
I added a few dependencies and pinned Nancy to 1.2.0
nuget FSharp.Formatting
nuget Microsoft.AspNet.SignalR.Core
nuget Microsoft.AspNet.SignalR.JS
nuget Microsoft.Owin.Hosting
nuget Microsoft.Owin.Security.Cookies
nuget Nancy 1.2.0
nuget Nancy.MSOwinSecurity
nuget NUnit
nuget NUnit.Runners
nuget FAKE
nuget SourceLink.Fake

To reproduce what I was seeing just run
.paket/paket.exe add nuget Nancy.Serialization.JsonNet

The output I got was:
Resolving packages for group Main:
 - Octokit 0.16.0
 - Nancy 1.2.0
 - FAKE 4.7.2
 - FSharp.Formatting 2.12.0
 - Microsoft.AspNet.SignalR.Core 2.2.0
 - Microsoft.AspNet.SignalR.JS 2.2.0
 - Microsoft.Owin.Hosting 3.0.1
 - Microsoft.Owin.Security.Cookies 3.0.1
 - Nancy.MSOwinSecurity 1.0.1
 - Nancy.Serialization.JsonNet 1.3.0
 - NUnit 2.6.4
 - NUnit.Runners 2.6.4
 - SourceLink.Fake 1.1.0
 - FSharpVSPowerTools.Core 2.1.0
 - FSharp.Compiler.Service <IP_ADDRESS>
 - Newtonsoft.Json 7.0.1
 - Microsoft.Owin 3.0.1
 - Microsoft.Owin.Security 3.0.1
 - Owin 1.0.0
 - jQuery 2.1.4
 - jQuery 2.1.3
 - jQuery 2.1.2
 - jQuery 2.1.1
 - jQuery 2.1.0
 - jQuery 2.0.3
 - jQuery 2.0.2
 - jQuery <IP_ADDRESS>
 - jQuery 2.0.1
 - jQuery 2.0.0
 - jQuery 1.11.3
 - jQuery 1.11.2
 - jQuery 1.11.1
 - jQuery 1.11.0
 - jQuery 1.10.2
 - jQuery 1.10.1
 - jQuery <IP_ADDRESS>
 - jQuery 1.10.0
 - jQuery 1.9.1
 - jQuery 1.9.0
 - jQuery 1.8.3
 - jQuery 1.8.2
 - jQuery 1.8.1
 - jQuery 1.8.0
 - jQuery 1.7.2
 - jQuery <IP_ADDRESS>
 - jQuery 1.7.1
 - jQuery 1.7.0
 - jQuery 1.6.4
 - Newtonsoft.Json 6.0.8
 - Newtonsoft.Json 6.0.7
 - Newtonsoft.Json 6.0.6
 - Newtonsoft.Json 6.0.5
 - Newtonsoft.Json 6.0.4
 - SourceLink.Fake 1.0.0
 - SourceLink.Fake 0.6.1
 - SourceLink.Fake 0.6.0
 - SourceLink.Fake 0.5.0
 - SourceLink.Fake 0.4.2
 - SourceLink.Fake 0.4.0
 - SourceLink.Fake 0.3.4
 - SourceLink.Fake 0.3.3
 - SourceLink.Fake 0.3.2
 - SourceLink.Fake 0.3.1
 - SourceLink.Fake 0.3.0
 - NUnit.Runners 2.6.3
 - NUnit.Runners 2.6.2
 - NUnit.Runners 2.6.1
 - NUnit.Runners <IP_ADDRESS>51
 - NUnit 2.6.3
 - NUnit 2.6.2
 - NUnit 2.6.1
 - NUnit <IP_ADDRESS>54
 - NUnit <IP_ADDRESS>92

and I just stopped it after a few minutes, not sure if it would keep going.

I wonder if we can find a little bit of text that clarifies this even further.
At this stage we can't say for sure that there is absolutely no resolution possible, but as the resolver goes on and on we have less confidence to find a resolution. But as I said to be 100% sure the resolver needs to finish (look at all possible configurations).
Anyone have suggestion for expressing this?

funny note: after waiting a couple of minutes the case above actually resolves to.
NUGET
  remote: https://nuget.org/api/v2
  specs:
    FAKE (4.7.2)
    FSharp.Compiler.Service (<IP_ADDRESS>)
    FSharp.Formatting (2.12.0)
      FSharp.Compiler.Service (<IP_ADDRESS>)
      FSharpVSPowerTools.Core (2.1.0)
    FSharpVSPowerTools.Core (2.1.0)
      FSharp.Compiler.Service (>= <IP_ADDRESS>)
    jQuery (2.1.4)
    Microsoft.AspNet.SignalR.Core (2.2.0)
      Microsoft.Owin (>= 2.1.0)
      Microsoft.Owin.Security (>= 2.1.0)
      Newtonsoft.Json (>= 6.0.4)
      Owin (>= 1.0)
    Microsoft.AspNet.SignalR.JS (2.2.0)
      jQuery (>= 1.6.4)
    Microsoft.Owin (3.0.1)
      Owin (>= 1.0)
    Microsoft.Owin.Hosting (3.0.1)
      Microsoft.Owin (>= 3.0.1)
      Owin (>= 1.0)
    Microsoft.Owin.Security (3.0.1)
      Microsoft.Owin (>= 3.0.1)
      Owin (>= 1.0)
    Microsoft.Owin.Security.Cookies (3.0.1)
      Microsoft.Owin (>= 3.0.1)
      Microsoft.Owin.Security (>= 3.0.1)
      Owin (>= 1.0)
    Nancy (1.2.0)
    Nancy.MSOwinSecurity (1.0.1)
      Microsoft.Owin (> 2.0)
      Nancy (> 0.21)
    Nancy.Serialization.JsonNet (1.2.0)
      Nancy (>= 1.2.0)
      Newtonsoft.Json
    Newtonsoft.Json (7.0.1)
    NUnit (2.6.4)
    NUnit.Runners (2.6.4)
    Owin (1.0.0)
    SourceLink.Fake (1.1.0)

Apart from the reporting, it seems something is wrong with the conflict boosting. ==> created https://github.com/fsprojects/Paket/issues/1169

Nice!  Probably something along the lines of what you said about resolution not 100% sure.
Quote-alpha-0.0.1
The resolution for %s is taking longer than expected.  This may still resolve, but this might take a while.

https://github.com/fsprojects/Paket/issues/1169 resolves this really really quick. so I need to change that conflict tracing again.


No invoice after subscription order is placed

When the Mollie subscription system calls the webhook to place the subscriptoin order in the webshop, the order is placed correctly and visible in the Magento backend.
We have configured Mollie payment module to create the invoice after payment authorization was successfull, but when the order is created during a subscription run, no invoice is created https://github.com/mollie/magento2-subscriptions/blob/main/Controller/Api/Webhook.php#L193
So we have a payment, but we do not have an automatic invoice created. Why don't you use the same logik as in the payment module? https://github.com/mollie/magento2/blob/2d5295974b0fa684e5be2354bb6e3c9f8b62445b/Model/Client/Orders/Processors/SuccessfulPayment.php#L125-L128

Hello, @ThomasNegeli!
We appreciate you bringing up this matter. We have received your email containing the necessary details as well, and we will respond to you promptly. Thank you!

Thanks for your confirmation @ThomasNegeli , closing this issue.


Upgrade failed on yum rpm corrupted database without the ability to recover

Environment info

Version: 2.3.0-1798557
Deployment: AZURE
Customer: QA

Actual behavior

Upgrade failed on yum rpm corrupted database without the ability to recover

we can recover with manual intervention.

Expected behavior

When yum rpm corrupted database appear we should be able to recover.

No longer relevant due to OVA deployment support removal. Upgrade will be done via k8s (either initiated by the user or not, it will still be an unmanaged path)


How to improve the topic(/fix) output precision, retaining 10 decimal places

Hello everyone, please tell me how to improve the topic(/fix) output accuracy to 10 decimal places?. When I run the demo, the altitude read always only has 3 digits after the decimal point. , longitude only has 7 digits of data after the decimal point. How to change to retain 10 decimal places?
OS: [Ubuntu 18.04]
ROS Version [ROS1 Melodic]

Nothing in the driver itself reduces the precision of the altitude or longitude, so the precision you're seeing is the same precision output by your navsat device.

Nothing in the driver itself reduces the precision of the altitude or longitude (aside from the limits of floating point precision in Python itself), so the precision you're seeing is the same precision output by your navsat device.

Now I get it,thank you for your answers.


Improve SSSE3 and AVX2 encoding speed

The enc_translate function now uses less instructions and only 3 constants.
Speed-up on Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz using Apple LLVM version 8.0.0 (clang-800.0.38)
AVX2: +45% compared to previous version
SSSE3: +54% compared to previous version
The AVX2 flavor uses:

3 add/sub instructions
1 comparison instruction
1 shuffle instruction

The SSSE3 flavor uses:

3 add/sub instructions
1 comparison instruction
1 shuffle instruction
3 copy instructions

Full results before & after modifications follows.
Before:
Filling buffer with 10.0 MB of random data...
Testing with buffer size 10 MB, fastest of 10 * 1
AVX2	encode	5772.70 MB/sec
AVX2	decode	6373.58 MB/sec
plain	encode	1344.26 MB/sec
plain	decode	1446.26 MB/sec
SSSE3	encode	3562.93 MB/sec
SSSE3	decode	3315.56 MB/sec
Testing with buffer size 1 MB, fastest of 10 * 10
AVX2	encode	5824.07 MB/sec
AVX2	decode	6292.53 MB/sec
plain	encode	1491.26 MB/sec
plain	decode	1430.49 MB/sec
SSSE3	encode	3594.22 MB/sec
SSSE3	decode	3485.10 MB/sec
Testing with buffer size 100 KB, fastest of 10 * 100
AVX2	encode	5860.74 MB/sec
AVX2	decode	6439.77 MB/sec
plain	encode	1470.44 MB/sec
plain	decode	1478.82 MB/sec
SSSE3	encode	3598.01 MB/sec
SSSE3	decode	3531.33 MB/sec
Testing with buffer size 10 KB, fastest of 100 * 100
AVX2	encode	5796.35 MB/sec
AVX2	decode	6395.72 MB/sec
plain	encode	1498.13 MB/sec
plain	decode	1476.94 MB/sec
SSSE3	encode	3592.39 MB/sec
SSSE3	decode	3580.89 MB/sec
Testing with buffer size 1 KB, fastest of 100 * 1000
AVX2	encode	4768.07 MB/sec
AVX2	decode	5258.08 MB/sec
plain	encode	1417.29 MB/sec
plain	decode	1440.80 MB/sec
SSSE3	encode	3369.49 MB/sec
SSSE3	decode	3397.11 MB/sec

After:
Filling buffer with 10.0 MB of random data...
Testing with buffer size 10 MB, fastest of 10 * 1
AVX2	encode	7736.49 MB/sec
AVX2	decode	6393.49 MB/sec
plain	encode	1425.47 MB/sec
plain	decode	1510.67 MB/sec
SSSE3	encode	5576.68 MB/sec
SSSE3	decode	3568.17 MB/sec
Testing with buffer size 1 MB, fastest of 10 * 10
AVX2	encode	8728.56 MB/sec
AVX2	decode	4699.43 MB/sec
plain	encode	1408.76 MB/sec
plain	decode	1556.61 MB/sec
SSSE3	encode	5679.42 MB/sec
SSSE3	decode	3596.99 MB/sec
Testing with buffer size 100 KB, fastest of 10 * 100
AVX2	encode	8752.18 MB/sec
AVX2	decode	5936.24 MB/sec
plain	encode	1500.53 MB/sec
plain	decode	1560.69 MB/sec
SSSE3	encode	5529.50 MB/sec
SSSE3	decode	3492.68 MB/sec
Testing with buffer size 10 KB, fastest of 100 * 100
AVX2	encode	8889.15 MB/sec
AVX2	decode	6421.75 MB/sec
plain	encode	1501.56 MB/sec
plain	decode	1515.71 MB/sec
SSSE3	encode	5513.91 MB/sec
SSSE3	decode	3481.16 MB/sec
Testing with buffer size 1 KB, fastest of 100 * 1000
AVX2	encode	6395.93 MB/sec
AVX2	decode	5184.36 MB/sec
plain	encode	1377.24 MB/sec
plain	decode	1519.36 MB/sec
SSSE3	encode	5048.06 MB/sec
SSSE3	decode	3397.37 MB/sec

Wow! This is absolutely astounding! I'm blown away by the fact that you were able to realize that degree of speedup. And the whole thing is so wickedly clever! Using saturating subtraction, interpreting the 0xFF mask value as signed -1... this is really really cool. You win this game.
I'm obviously honoured to merge this, but one small remark: could you please fix the indentation to use tabs, and could you please insert a blank line before comment lines? You can force-push the result, it will update the merge request automatically. Thanks!

I'll do that later in the day (probably in 12 hours or so).
To be fair, I had a version that used one shuffle instruction but with many comparisons (same as before in fact) that was faster than the previous version (this lead to encode being a bit faster that decode). Given the link to Wojciech Muła articles in README.md, I went to his site and saw that he already had a faster version using the saturation substraction trick. I just improved it a little with the mask value as signed -1 trick.
Also, it uses less constants than the latest version by Wojciech Muła and I guess it will benefit 32bits builds (reduce register pressure) but I did not test that.

I pushed the small style changes. Tell me if that's what you're expecting. There might be 1 line break that you don't wan't between 2 comments. I did it that way because the first refers to the block of 3 statements that follows.

Yes, that line break is superfluous in my opinion. Otherwise it's ready to merge. Let me know if you want to push a fix.

Done. I also modified the commit log: the store instruction is not from that inlined function.

Merged, thanks!


Another BashException

Using latest PySyncDroid error:
Copying /media/joe/Media/Music/Skillet/Comatose/folder.jpg to /run/user/1000/gvfs/mtp:host=%5Busb%3A005%2C003%5D/Card/Music/Skillet/Comatose/folder.jpg
Traceback (most recent call last):
File "/usr/local/bin/pysyncdroid", line 9, in
load_entry_point('pysyncdroid==0.0.1', 'console_scripts', 'pysyncdroid')()
File "build/bdist.linux-x86_64/egg/pysyncdroid/cli.py", line 50, in main
File "build/bdist.linux-x86_64/egg/pysyncdroid/sync.py", line 215, in sync
File "build/bdist.linux-x86_64/egg/pysyncdroid/sync.py", line 139, in gvfs_wrapper
File "build/bdist.linux-x86_64/egg/pysyncdroid/gvfs.py", line 54, in cp
File "build/bdist.linux-x86_64/egg/pysyncdroid/utils.py", line 38, in run_bash_cmd
pysyncdroid.exceptions.BashException: Command "gvfs-copy /media/joe/Media/Music/Skillet/Comatose/folder.jpg /run/user/1000/gvfs/mtp:host=%5Busb%3A005%2C003%5D/Card/Music/Skillet/Comatose/folder.jpg" failed: Error copying file /media/joe/Media/Music/Skillet/Comatose/folder.jpg: The name :1.1976 was not provided by any .service files
Needed to remount after the above error.

Needed to remount after the above error.

Why not leave out line 141 and 146 in sync.py?

So, in case of the The name :<name> was not provided by any .service files we can just retry the copy and then continue the sync, right?

Yes it appears that that "might" work.  I'll do more testing.
On 03/03/2016 07:29 AM, Dušan Maďar wrote:

So, in case of the |The name : was not provided by any .service
files| we can just retry the |copy| and then continue the sync, right?
—
Reply to this email directly or view it on GitHub
https://github.com/DusanMadar/PySyncDroid/issues/3#issuecomment-191739903.

OK, there is only one problem. I didn't came across this issue so far, therefore I cannot test it. But I still have to try copy a couple of GBs from computer to phone, as that is what you are doing when this happens.

Latest Error:
pysyncdroid -v -V Samsung -M GT-I9100 -s /media/joe/Media/Music/ -d Card/Music/ -u remove
Gathering the list of files to synchronize, this may take a while ...
Traceback (most recent call last):
File "/usr/local/bin/pysyncdroid", line 9, in
load_entry_point('pysyncdroid==0.0.1', 'console_scripts', 'pysyncdroid')()
File "build/bdist.linux-x86_64/egg/pysyncdroid/cli.py", line 50, in main
File "build/bdist.linux-x86_64/egg/pysyncdroid/sync.py", line 204, in sync
OSError: [Errno 5] Input/output error: '/run/user/1000/gvfs/mtp:host=%5Busb%3A005%2C002%5D/Card/Music/'

Since I had problems with PySyncDroid I needed to do some testing.
I do BASH and C, rather than Python, so I wrote a little BASH script to do the syncronization from Computer to Samsung and easily do testing (something that I understand and am comfortable with).  I take advantage of find, sed, grep, and awk to make the script very simple.
I'm intending to do syncronization from computer to Samsung (of a large amount of music) and delete files on Samsung that are not found on Computer and "update" any "newer" file from Computer to Samsung (Computer being the "master").
First - Android issues.
Samsung Galaxy Tab 3 has Android 4.4 KitKat and does not allow writing to the external SD card (without rooting the device).  gvfs-ls, gvfs-tree, etc. will work, but anything that tries to write to "Card" will not work.
Samsung SM-G30V has Android version 5.1.1 Lollipop which does allow reading and writing to Card.
Finding all the mp3's on the Computer is just a simple: find $musicPathOnComputer -type f -iname "*.mp3".  Finding all of the mp3's on the Samsung is another issue. There are different ways of doing this.
There are 7453 files in 2664 directories currently in the Music folder (36.5GB), for a total of 10,096 executions of gvfs-ls (one possible approach) to find all of the files on the Samsung Device.  I have an additional ~10GB of music, currently in an Archive folder that I may also like to sync.  A "large" amount of music.
Music is arranged as Artist(s)/Album(s)/Song(s).
Findings:

Using gvfs-tree mtp://[usb:005,005]/Card/Music fails about a third of the way through the 10,096 folders and files with the error - "(gvfs-tree:20778): GLib-ERROR **: Creating pipes for GWakeup: Too many open files Trace/breakpoint trap"

Using gvfs-ls to loop through the Artists and then using gvfs-tree within each Artist also fails part way through.

Using gvfs-ls to loop through Artists/Albums/Songs speeds along through about the first 1000 entries and then the following error appears: "Error: Error while getting peer-to-peer dbus connection: Error receiving data: Connection reset by peer".  After this, gvfs-ls becomes very slow!  It would take hours to complete the process (maybe the Windoze transfer - that I described earlier - suffered from a similar problem).

"Small" file transfers, using gvfs-cp, etc. seem to work without problems.

I've tried to explore solutions (inserting time delays in the code, etc.) to the above "problems" but have not found a solution yet.
Highlighting mtp://[usb:005,005]/Card/Music in Linux file manager (Caja 1.12.1) and the pressing * * * to expand the Music folder to display all Artists, Albums, and Songs takes several minutes, but eventually completes and displays the entire folder structure including all mp3's!  So, I'm not sure what Caja uses to do this successfully?
Currently trying a different approach.  Computer is Music "MASTER".
New Approach:

Use find and stat to create a timestamped (timestamp for each mp3) list of all music on Computer for transfer to Samsung.
Copy timestamped list to Samsung.
Do "full" copy of all Music from computer to Samsung Card (by extracting the microSD card from phone and doing copy).  Computer and Samsung identical at this point.
At next sync - generate updated timestamped list of all Computer music, retrieve list from Samsung, compare lists then update new/changed music to Samsung, update Samsung timestamped music list and then transfer new list to Samsung.
Repeat.

Comments?

Can't reproduce The name :<name> was not provided by any .service files.
When trying to sync PC to Device and it takes a bit longer (10+ minutes), I get the following:
Message did not receive a reply (timeout by message bus)
and the device is disconnected. I need to unlock the device again to re-connect it back again and then continue the sync process.
It seems like the device drops the MTP connection after a certain amount of time.

As long as there are no gvfs* errors, then things run to ok.
My new (successful) procedure is to initially copy all music from pc to card (so they are identical).  Then, I run a very small bash script, ~50 lines, to keep track of music on computer and card.  Then script syncs new music to card and deletes music from card.  Log files on computer and card (with timestamp for each song) are used for each subsequent sync.
Can't beat the simplicity of bash!  Just wrote bash script to populate the lyrics frames in my entire music library from the web.


done some code cleanup -> PEP8, fixed some logical stuff like not use…

done some code cleanup -> PEP8, fixed some logical stuff like not used variables in all sensors
@eagle00789 : Could you review this pull request as it involves some minor changes to your sensor code as well. Feedback appreciated.

Looks good to me...

Thanks, so I will merge now


Update usage_FAQ.md (language edit)

Description
Copy editing for style, readability and clarity.
Motivation and Context
There was nothing "wrong" with the answers as they were, but changes of this kind are always useful and valid.
How Has This Been Tested?
The changes were to language only, not meaning.
Checklist:

[x] I have added a Signed-off-by
[ ] I have run gometalinter, gofmt, and goimports and have resolved all errors and warnings.
[x] If applicable, I have updated the documentation accordingly.
[ ] If applicable, I have added tests to cover my changes.

I was invited to copy edit some files in github by Josh Horton and Mark Parzygnat.
Signed-off-by:
Joe Alewine

Thanks Joe! Feel free to continue, and/or to look at potential for better GitHub doc displays as per https://github.com/hyperledger/fabric/issues/1262

Josh,
I've submitted another pull request for the "identity management" FAQ. Each
one takes a while to complete because I want to make sure I'm being very
careful not to change the meaning of any of the answers but my plan is to
do them all.
I like the "beautiful docs" image you were able to create. It has more of
an uncluttered wiki look which has become sort of the standard. How
specifically can I help?
Joe
On Wed, May 11, 2016 at 1:06 PM, Josh Horton<EMAIL_ADDRESS>wrote:

Thanks Joe! Feel free to continue, and/or to look at potential for better
GitHub doc displays as per #1262
https://github.com/hyperledger/fabric/issues/1262
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/hyperledger/fabric/pull/1399#issuecomment-218524418


Add generatordir to output-dirs

The user/project should be able to control where generators are placing/creating files.
Proposal is to add a generatordir to the output-dirs field.
The generatordir would then update the destination for the generator input file
In the PDSC we can also introduce a $G parameter that would expand to the generator input file path so that generators easily can find it.
The Generator could still potentially create files anywhere it wants, but assuming the generator is a good citizen it can at least use the destination to try follow user wishes.

Is the proposed $G the path to the cbuild.yml file?

Is the proposed $G the path to the cbuild.yml file?

Yes, it probably should be. For now the https://github.com/Open-CMSIS-Pack/devtools/pull/462 implements $G expansion to the currently implemented ...generate.yml file, to connect the $G to something.
Once we have the .cbuild.yml implemented, I think we can start to change $G to point to this and remove/replace the current ".generate.yml"-code.
Then we can see if we need to create a temporary more explicit copy of .cbuild.yml or not (basically just with all paths resolved absolutely everywhere instead of relying on relative/variables) but this can be discussed later, since the format would not require change, only the values.

Closing this issue since it has now been implemented and merged.


remove bin directory from npm package

the 2.0.0 release added some development scripts in the bin directory that have been included in the NPM package. add bin to the .npmignore file to remove that directory to reduce the size of the package further.

Verified fixed in starter kit after deploy to NPM.


Add fantinel.dev

Thanks for suggesting the site ;)

Very polished blog! Thanks for this addition. I wish I could design like that. 😄


Resource Center Template Refactor

Created a new approach to handling sidebar pages - this effectively uses a dummy block to indicate where the content should split between left & right.

Sidebar Right -> 75/25 column split
Sidebar Left - 25/75 Column split (CSS not defined yet but easy to add)

Fix #141
Test URLs:

After:

https://bug-resource-center-template--shredit--stericycle.aem.page/en-us/resource-center/videos/data-breaches-understanding-todays-threats
https://bug-resource-center-template--shredit--stericycle.aem.page/en-us/resource-center/infographics/6-tips-to-protect-your-business-from-fraud

@MarquiseRosier, Kindly squash and merge the changes instead of just merging them.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/incorporating-changes-from-a-pull-request/about-pull-request-merges


Support for EF Core 7

What are the plans for EF Core 7 support?

I dunno. I haven't followed its development too closely beyond high level feature announcements etc. Anything I need to think about and/or update?

I think you just need to update dependencies. I can try a PR?

Sure, go for it! ☺🙏

(I use your extension in EF Core Power Tools)

I think an update of https://www.nuget.org/packages/Verify.EntityFramework is needed first @SimonCropp ?

Why exactly does EF Core require these version bumps? If a library you depend on (in this case Verify.EntityFramework) requires >= 6.0.5, shouldn't you just be able to pin 7.0.0 and everything should just work? Is it because of breaking changes?

@ErikEJ what update do u need in Verify.EntityFramework  ?

@SimonCropp None apparently! :smile:


Const warnings

This PR fixes a few more compiler warnings about constness

Any more thoughts about this PR?

No thoughts. Just a lack of pulling time.

To the tune of Gimme Shelter:
:notes: Merge, children, it's just a click away :notes: