Android: Deadlock when Read RSSI Doesn't Callback

Pixel 7, current version.
Running local build of Kable from HEAD of main.

I'm encountering what appears to be a deadlock when I have two active connections (different peripheral devices) that poll on the remote RSSI.  It looks to me like Android can fail to callback with the remote RSSI.  When that occurs, Kable waits indefinitely for that callback, blocking subsequent synchronous operations over the connection (e.g. peripheral.write). Note that Kable is still providing our app with updates from notifications (e.g. onCharacteristicChanged from Android).
Please let me know if there is additional information that I can provide.

Does this only happen when you have 2 (or more) active peripheral connections? In other words, do you ever see this issue with only a single peripheral connected?

I don't think so.

@twyatt -- happy new year!  we're starting to plan around a feature that will need to read RSSI while connected.  I just wanted to touch base to see what you thought about the timing for this.  We can fork and dig into it, but would only really want to do that if you think this bug is further out.

Happy New Year to you as well! 😁
I have two big-ish projects for work that take priority, so I'm guessing I won't have a lot of time to dedicate to Kable for at least a month.
I try to find time off hours to patch bugs, but this one may require a bit of investigation. Anything you can do to add clarity to the issue would be super helpful.
I'll try to find time sooner, but no promises.


add more e2e and integration test for manifestworkreplicaset rollout

Describe the enhancement
A clear and concise description of what the enhancement request is.
When trying the RolloutConfig of manifestworkreplicaset, meet below issues

when the type is ProgressivePerGroup and the MinSuccessTime meets, the rollout doesn't continue to the next group
when the failure exceeds MaxFailures the rollout doesn't stop.

need more  e2e and integration tests for manifestworkreplicaset rollout, to verify if the above feature works and this could also be an example for the usage.

cc @haowells

both work-controller on hub and spoke cluster have this error messages
E1211 12:54:27.375595       1 base_controller.go:266] "ManifestWorkReplicaSetController" controller failed to sync "default/guestbook-progressive-failure-pause", err: Operation cannot be fulfilled on manifestworkreplicasets.work.open-cluster-management.io "guestbook-progressive-failure-pause": the object has been modified; please apply your changes to the latest version and try again
the workaround to let rollout continue is keep annotating MWRS object..

/assign

The mentioned issues are fixed and more testing cases are added by PR https://github.com/open-cluster-management-io/ocm/pull/337, feel free to have a try. @haowells

@haoqing0110  thanks your effort !


Unable to see the strings for SAX

I am not sure if this is a bug but I am unable to see the string representation for SAX. I can group by PAA but the whole point of SAX is to transform a series into a string representation.
Is there something I am missing?

Hi @eemamedo
If you use SAX transform as done below:
sax = SymbolicAggregateApproximation(n_segments=n_paa_segments,
                                     alphabet_size_avg=n_sax_symbols)
dataset_sax = sax.fit_transform(dataset)

your variable dataset_sax will be made of integer indices (0, 1, 2, etc.) that are your symbols. If you really want a string, could can always map the 2d array of integers to an array of strings.


Rustup

For https://github.com/rust-lang/miri/pull/3103

@bors r+

:pushpin: Commit 90744dc2f90b06e3be751cdaa1cbec85723e4953 has been approved by saethlin
It is now in the queue for this repository.

:hourglass: Testing commit 90744dc2f90b06e3be751cdaa1cbec85723e4953 with merge 133e9895e1a0dae2bf05ce133489f538a757350f...

:sunny: Test successful - checks-actions
Approved by: saethlin
Pushing 133e9895e1a0dae2bf05ce133489f538a757350f to master...


Add Alert (event) support for MediaConnect

EMX has one minute log poll time in CW currently, Alarms are VERY slow to populate - events would be best. Similar to MediaLive in MSAM today.

Query from EMX team a few of weeks ago:
Q. Does mediaconnect do cloudwatch events or just metrics/logs? i can't remember...
A. We have a few Events, but they haven’t been released in the CW Events console or documentation yet. Hopefully we will get to that soon. We don’t have any CW Logs that we expose to customers.

JSON
{
"source": ["aws.mediaconnect"],
"detail-type": ["MediaConnect Alert"]
}

Is this JSON Schema consistent with other Media Services?

We'll hold on this one until we get some feedback from MediaConnect on the data structures they're using for their Alerts.

Support added.
Alert event looks like this:
{
"version": "0",
"id": "638c00d8-ccd9-f2e7-ae10-f11cfc5515a0",
"detail-type": "MediaConnect Alert",
"source": "aws.mediaconnect",
"account": "xxxx",
"time": "2020-04-22T20:42:11Z",
"region": "us-west-2",
"resources": [
"arn:aws:mediaconnect:us-west-2:xxxx:flow:1-WgQJAQMJV1MIUFhf-bc9b5327f68e:alerttestflow2"
],
"detail": {
"error-code": "ValidationException",
"error-id": "ANiyJOI6U1O+4em1CH2O3ss3Y8Y",
"error-message": "The decryption key for source stdsrc is invalid....).",
"errored": False
}
}


The table of contents is broken

My linter caught that "Online Services" is missing

submitted a PR


Readme example doesn't run

Gt\Database\DatabaseClient::__construct() must be an instance of Gt\Database\Query\QueryCollectionFactory, instance of Gt\Database\Connection\Settings given
The ability to specify a QueryCollectionFactory is great so I can use a different folder for queries (src/sql/query is my personal preference so I can have migrations in src/sql/migrations).
On which note, looking at the source, am I right in thinking that the default base directory for query files is actually getcwd() rather than src/query as specified in the readme example?

Will be handled by #21.


docker service failure with error 'ERROR: DockerService is not a class'

I have a simple recipe which does:
docker_service 'default' do
  		action [:create, :start]
	end

I see the following error:
Generated at 2015-10-20 13:13:41 +0000
TypeError: DockerService is not a class
/var/chef/cache/cookbooks/docker/libraries/provider_docker_service.rb:3:in `<class:Provider>'
/var/chef/cache/cookbooks/docker/libraries/provider_docker_service.rb:2:in `<class:Chef>'
/var/chef/cache/cookbooks/docker/libraries/provider_docker_service.rb:1:in `<top (required)>'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:191:in `load'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:191:in `block in load_libraries_from_cookbook'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:188:in `each'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:188:in `load_libraries_from_cookbook'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:99:in `block in compile_libraries'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:98:in `each'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:98:in `compile_libraries'
/opt/chef/embedded/apps/chef/lib/chef/run_context/cookbook_compiler.rb:71:in `compile'
/opt/chef/embedded/apps/chef/lib/chef/run_context.rb:97:in `load'
/opt/chef/embedded/apps/chef/lib/chef/policy_builder/expand_node_object.rb:87:in `setup_run_context'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:486:in `setup_run_context'
/opt/chef/embedded/apps/chef/lib/chef/client.rb:266:in `run'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:270:in `block in fork_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:258:in `fork'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:258:in `fork_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:224:in `block in run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/local_mode.rb:39:in `with_server_connectivity'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:212:in `run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application/client.rb:402:in `block in interval_run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application/client.rb:392:in `loop'
/opt/chef/embedded/apps/chef/lib/chef/application/client.rb:392:in `interval_run_chef_client'
/opt/chef/embedded/apps/chef/lib/chef/application/client.rb:382:in `run_application'
/opt/chef/embedded/apps/chef/lib/chef/application.rb:60:in `run'
/opt/chef/embedded/apps/chef/bin/chef-client:26:in `<top (required)>'
/bin/chef-client:55:in `load'
/bin/chef-client:55:in `<main>'

What version of chef-client ?

@someara
Chef Client, version 12.4.1

Did you put an "include 'docker'" in your metadata.rb?
On Tue, Oct 20, 2015 at 10:30 AM, Avinash<EMAIL_ADDRESS>wrote:

@someara https://github.com/someara
Chef Client, version 12.4.1
—
Reply to this email directly or view it on GitHub
https://github.com/someara/chef-docker/issues/525#issuecomment-149584634
.

depends 'docker', '= 2.1.2'

The code in your stack trace is from the old (1.0.x) version of the cookbook.
Try cleaning your chef-client cache
-s

@someara
Dont think so. Because the following file
/var/chef/cache/cookbooks/docker/CHANGELOG.md has the following content
v2.1.2
--------------------
- Adding why_run support

v2.1.1
--------------------
#511 - fix container wait state checking
#512 - wait for registry ports to be open in test recipe
#519 - updating README to include labels#511 - fix container
...

chef-client -l debug will tell you at the top when it syncs cookbooks what
version it is going to use
On Tuesday, October 20, 2015, Avinash<EMAIL_ADDRESS>wrote:

@someara https://github.com/someara
Dont think so. Because the following file
/var/chef/cache/cookbooks/docker/CHANGELOG.md has the following content
v2.1.2

Adding why_run support

v2.1.1
#511 - fix container wait state checking
#512 - wait for registry ports to be open in test recipe
#519 - updating README to include labels#511 - fix container
...
—
Reply to this email directly or view it on GitHub
https://github.com/someara/chef-docker/issues/525#issuecomment-149597489
.

Loading cookbooks<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

This smells like a bug in chef-client.
It is definitely the old code in the stack-trace.
provider_docker_service.rb no longer exists in 2.x, and it no longer uses LWRPBase.
Are you using a chef-server?
-s

@someara
figured out the problem. this is a chef screwup.
I had 2 versions of the docker cookbook locally. When i uploaded using knife, it merged the files from both the cookbooks and uploaded them as the latest version on chef server.
I cleanup my local box, chef server and did a fresh upload. Things are fine now.

That shouldn't have happened. Can you file a bug against chef-server, then
paste the link into this thread?
That way, people that find this via google can follow it.
-s
On Tue, Oct 20, 2015 at 11:39 AM, Avinash<EMAIL_ADDRESS>wrote:

@someara https://github.com/someara
figured out the problem. this is a chef screwup.
I had 2 versions of the docker cookbook locally. When i uploaded using
knife, it merged the files from both the cookbooks and uploaded them as the
latest version on chef server.
I cleanup my local box, chef server and did a fresh upload. Things are
fine now.
—
Reply to this email directly or view it on GitHub
https://github.com/someara/chef-docker/issues/525#issuecomment-149608113
.

Filed issue - https://github.com/chef/chef/issues/4077

Thanks Avinash!


Update ELE_2018.json

correcting specific ele 208b requirement.

Could you provide a source link (as in, where this requirement is explained)?
Thanks!

http://ee.princeton.edu/sites/default/files/UG_Handbook.pdf

This was merged into master instead of dev. I removed it from master and added it to dev instead.


perf(transformer): SparseStack always keep minimum 1 entry

Optimize SparseStack. Initialize it with a single entry, and ensure the stack is never emptied. This allows making take, get_or_init and get_mut_or_init methods infallible, since there is always an entry to read.

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#5962  👈
#5960
#5959
#5942
#5940
#5945
#5941
#5944
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @overlookmotel and the rest of your teammates on  Graphite

This series of PRs for SparseStack gains about +1% on transformer benchmarks in total. Not as much as I'd hoped, but still 1% here and 1% there adds up. And I think SparseStack is a useful abstraction, as we're using stacks which fit this description a lot in the transformer.
I'll follow up with another PR later on to use 1 stack shared between all these transforms, rather than each maintaining their own stack.


Account pagination error when going to next page

Version information

v0.31.2
Describe the bug

When querying accounts, the nextId is incorrect and due to that next pagination querying results in error.
To Reproduce

Expected behavior

Additional context

This is fixed.

This is fixed.


Move clientId from constant.

Sorry, but i don't understand why string value must be typed as new type? Could you please explain me why identifier like this Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0 must be of type ClientId?


Notify about DataBroker Disconnect

Closes: #12

Getting the following error message in the logs after starting / disconnecting the data broker
Previous channel {0} was not shutdown properly!!! ~~~*

The java mode does not seem to work with disconnecting

Good catch postCreate was "to early" to register the listener, because the DataBrokerEngine is switched when toggling the Java-Compatibility mode. Registering now in connect() and unregistering in disconnect() resp. in onDestroy() in case the connection is not manually closed.

Getting the following error message in the logs after starting / disconnecting the data broker Previous channel {0} was not shutdown properly!!! ~*~~

I can't reliably reproduce this issue, however I always got it once in a few tries.
I added some logic to always "shutdownNow" in case the Connection is closed, either by calling disconnect or when the state changes (unexpectedly) to anything else then ConnectivityState.SHUTDOWN. I couldn't see the issue since then, but unfortunately it's a devil's proof, as I could simply have been lucky with it.

LGTM


will it work for  multivariate  time series classification for example mixture of categorical and continues data?

will it work for  multivariate  time series classification for example mixture of categorical and continues data?
for example at time t1 we have observation: red, 2.4 , 5, 12.456 and time t2: green, 3.5, 2, 45.78; time t3: black, 5.6, 7, 23.56; t4: red, 2.1, 5, 12.6 ?

No, it's addressed to realtime 3D/2D gesture recognition with different Sensors. By now it supports only Kinect v1 and v2


Create criteria for Alpha ui-component release

As a: Consumer of the design system documentation
I want: Understand that I am looking at an alpha component, what that means, and how I can use it
so that: I can determine, when, how, where to use a ui-component in my service.
Acceptance criteria

[x] Approach allows backlog of ui-components to make their way into "Alpha" quickly
[x] "Alpha" criteria, will be reviewed with design system team and feedback will be incorporated before going into the final criteria
[x] Criteria will be published to Confluence

Presentable outcomes

[x] N/A, will be presented along with Beta component, which will allow for better comparison by consumers.

Sprint Ready Checklist

Acceptance criteria defined
Team has defined steps to satisfy acceptance criteria
Acceptance criteria is verifiable / testable
External / 3rd Party dependencies identified

Related issues:
GitHub issue: #417 #421
Resources:
Mockups: [If applicable a URL to or the name of the mockup(s)]
Testing URL: [If applicable a URL to the testing branch]

@Spark450 here's a couple suggestions from the technical side that we have in the Developer Guide: https://goa-dio.atlassian.net/wiki/spaces/DS/pages/2206924906/Developer+Guide#Web-Components

Link the confluence
https://goa-dio.atlassian.net/l/c/oDC1wPbi


Typo in config.py - set_sun_time

The automatic switching according to the local sunset / sunrise does not work correctly (gets wrong times).
I have looked into the config.py, and it seems that in set_sun_time, both latitude and longitude values are pulled from the setting fro the latitude (get("latitude")).

Just saw that there's a PR for this
https://github.com/daehruoydeef/Yin-Yang/pull/73#issue-524979851

just merged :)


Add 4etire tridcat by Violetta Sokolova

Song Name
4etire tridcat
Artist Name
Violetta Sokolova
Source
Youtube
Youtube Link
KdZzWEfrHIBf4JzM
Direct File Link
No response
Song ID
54558
Start Offset [ms]
0
Extra
Created from Auto Nong.
Created for level: 108621832

reject
invalid YouTube link


Signaltable/Debug interference bug

Changing the signaltable while having an active simulation will make the debug mode unusable unless you press the reset button. (Maybe cache the "old" simulation or terminate the "old" simulation when you update the signaltable?)

You are not supposed to change anything during a running simulation. But maybe I will implement a check and warning dialog.


WIP: msbuild targets for publish IIS

This is an initial cut of an msbuild replacement for dotnet-publish-iis. It needs additional work, as noted below, but I wanted to get extra eyes on this to make sure this is the right direction.
This draft puts targets in the runtime package (Microsoft.AspNetCore.Server.IISIntegration) which would require a runtime update to deliver this.
TODO

E2E testing using msbuild 15
figure out package layout and distribution.
configure build to use nuspec instead of project.json

cref #258
cc @davidfowl @moozzyk @Tratcher

Hi @natemcmaster, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by .NET Foundation and real humans are currently evaluating your PR.
TTYL, DNFBOT;

cc @anurse could you take a look at the MSBuild part of this?

Thanks for initial review. I'm closing for now and will keep this work in the feature/msbuild branch. We'll do a review again later when we're ready to actually release some of this. We expect lots of churn before this is ready for release.

I looked at "the other part" - most of it is basically a copy of what we already had. I am not an expert in creating custom MSBuild tasks but what I saw looked good.


Fix : Operand of null-aware operation '!'

I got this with previous version : ../../../flutter/.pub-cache/hosted/pub.dartlang.org/slide_to_act-2.0.1/lib/slide_to_act.dart:385:20: Warning: Operand of null-aware operation '!' has type 'WidgetsBinding' which excludes null. and i fixed this issue by removing the '!' null checking.


sdk build issues with some versions of lwip

I was experimenting with different versions of lwip and it seems that this source file src/core/ipv4/acd.c doesn't exists in some older stable releases. I think we should handle this?
Also, these only exist in 2.2.x
            ${PICO_LWIP_PATH}/src/apps/mdns/mdns_out.c
            ${PICO_LWIP_PATH}/src/apps/mdns/mdns_domain.c

Fixed in develop branch


Leaving GitHub?

At the moment a lot of people switch to GitLab, because GitHub has been acquired by Microsoft. We all know that this acquisition does not immediately mean any concrete threat, so leaving GitHub now is more a matter of attitude than anything else.
I personally vote for leaving. But it would definitely mean a lot of work, especially for @nitriques. (Moving the repos is simple, but I am sure that there would be a lot of side effects on the whole ecosystem!) So I would not want to push anybody. @nitriques is the chief developer, so IMHO it's his decision.
I just wanted to say: If Symphony moves to GitLab, I will follow.

We are using Gitlab for another project. Github is better and I‘d recommend to continue using it.
Besides reservations against Microsoft, what’s the reason behind this idea besides self-hosting?

Microsoft wouldn't acquire GitHub if it wasn't about control and influence. (And they wouldn't pay so much money either.) It will by a step-by-step process, of course. But IMHO it is not desirable, no matter how small single steps will be.
(BTW, I am not sure at all that self-hosting would be better for a project like Symphony.)

I understand that feeling and it’s good to know alternatives and to be prepared. But right now the product hasn’t changed and it’s still the better product in my eyes.
That said, I’m not an active contributor anymore. But please don’t make rushed decisions.

Thanks! I am not for rushed decisions either; that's why I tried to be careful with my initial post.

No wonder if gitlab's gradually getting attention of another GIANTSOFT ..

Github is better and I‘d recommend to continue using it.

Elaborate! :)

No wonder if gitlab's gradually getting attention of another GIANTSOFT ..

Any source for that info?

I'm strictly against this unless Microsoft shows clear signs they're going to spoil the party here (which I hope and believe they won't do).
Moving to Gitlab might me a manageable task for the Symphony main repo, but it won't be so for lots of more or less abandoned or seldom-used extension repos. Trying to move all symphony related repos to gitlab will therefore fail and we will end up with having the main repo and most current and actively developed extensions on gitlab while lots of older extensions will remain on github.
In my eyes Symphony currently hasn't the "powers" to make such a move in a way the eco-system could actually benefit from it. We had the powers to make the move from self-maintained to github back in the days and should still be thankful for what we achieved with that move and the great additions to the symphony-eco-system that were built on top of the new gained possibilities of a central git-based repo-platform.
And speaking about symphonyextensions.com we should be especially thankful that it's still there, running reliable and giving an up-to-date overview of a solid majority all extensions from the symphony-cosmos though its codebase nearly hasn't been touched since @nickdunn built it in the heydays of Symphony. Moving to gitlab will definitively break this too and my fear is that if that happens it won't get it fixed as we simply don't have the resources (just like getsymphony.com is more or less in kinda "dead"-mode since the blog and forum were ditched for gitter).
And while I don't really know which parts of github-based data and infrastructure we would loose while moving (I'm talking about prs, issues, discussions, stars, etc. from people that won't make the move) I'm pretty certain trying to move everything sympohny-related to github will spoil a lot on this level also.
But apart from my fear that such a move will harm the already fragile symphony-eco-system I simply don't share your sentiments against Microsoft  (though I'ma Mac user since 10 years):

Microsoft wouldn't acquire GitHub if it wasn't about control and influence.

»MS Github« will most definitely make use of all the newly available data in order to maximise their profits and market share in ways I can hardly imagine

Do you really think? I'm not so sure… regarding all the work MS has put into Visual Code Studio recently and all the praise they've earned from developers for it I think their motivation in buying github lies in attracting developers rather than scaring or exploiting them. I found this article (linked by @nitriques on twitter?) pretty interesting and I would propose to at least give them an honest chance reagarding their vision for github.
If Microsoft proves me wrong and real threats arise we will still have the time to move on! Until then I'd suggest we spend the time we can afford for symphony more productive ;)

By the way: As I just adopted the geocoding-extension yesterday and as @klaftertief was so kind to give @nitriques access to all his other extension-repos I have to say that regaining access to abandoned extension-reops by adopting them (if you use them yourself) or getting them moved to the symphonists-account is something we should really enforce and do as much as we can!
Not only does it help to keep our extension-eco-system on github and symphonyextensions.com healthy but it would also greatly improve our "starting position" in case we'd really decide to make the move to gitlab.
I know just forking and fixing stuff for oneself is always easier, but having a well-maintained and solid-documented main-repo is so much better than a couple of different forks with different fixes.

Elaborate! :)

My main point: the UI and UX is confusing, cumbersome and buggy. It all looks quite similar at first sight and the general functionality is the same (a pull request becomes a merge request – things you get accustomed to quite quickly). But too often views don't load or you are completely lost because the navigation seems to be split and randomly distributed in all cardinal directions.
Generally, things have a tendency to behave slowly due to the JavaScript based interface. A problem that Github faces as well well with there new activity stream layout though. But still, I'm often surprised how well Github works compared to Gitlab when coming back after a while.

Thanks for elaborating Niels. I have yet to make those experiences.

I didn't intend to discuss which one is better. This is not the question.
And despite my personal opinion I just said "@nitriques should decide" and "if Symphony leaves, I will follow". I don't want to create any pressure, and I don't want to be engaged in an emotional discussion. (Why emotional? Because we can only discuss what we believe or want to be true. No hard facts.)
So I think that this issue should be closed as soon as @nitriques has chimed in and posted his decision on the matter.

Yeah, it's cool to talk about it, I appreciate the feedback really ;)
I am typing this on a Windows computer, wearing a Microsoft jacket.
Github is a wonderful software and I am pretty sure nothing bad will happen to it. Many of their employees work with it and they will not want to piss their devs.
I've used Bitbucket and gitlab (both recently) and I prefer github.
I feel like Microsoft is the best company to acquire it. VC wants their money back, si it was inevitable for a 10 years old start up. And I do trust Nat Friedman.
If you fear Microsoft, you must also fear Google, Amazon, Facebook, Twitter and many more ;)

If you fear Microsoft, you must also fear Google, Amazon, Facebook, Twitter and many more ;)

I do! ;)
Thanks for that clear statement.

Thanks for that clear statement.

No problem. And do no be confused: I know what does companies can do. I sometimes fear them. They can do real creepy and crapy things. They have immense power. We must accept the risk or live disconnected.

If you fear Microsoft, you must also fear Google, Amazon, Facebook, Twitter and many more ;)

I do! ;)

So do I.

I am typing this on a Windows computer, wearing a Microsoft jacket.

I hope that I won't dream of this tonight.
:-)
Thanks, @nitriques!

I think it’s most likely my utter unspeakable hate of Word® that makes me distrust MS.

It looks like you're trying to leave Github

https://twitter.com/_disco/status/1003684420526669826

@nilshoerrmann you could join Gitlab and improve it ;) —> https://twitter.com/andreasmarc/status/1006243013801271296


feat: Opening hyperlinks externally

This is a stab at https://github.com/nvim-orgmode/orgmode/issues/563 and https://github.com/nvim-orgmode/orgmode/issues/506. <Leader>oo now opens linked files with xdg-open.
There is also an option to specify custom: links, which I would like to read your thoughts on:
[[custom:nsxiv:~/some-gif.gif][]] - This link opens a GIF file in nsxiv
[[file:~/some-gif.gif][]] - This link opens a GIF file with the specified default opener
[[file:~/org/notes.org]] - Regular behaviour on open

Issues
This functionality depends on nvim's vim.filetype.match() behaviour, which returns nil on unrecognized filetypes inferred from the filename. The issue with this is that some plaintext (i.e. .txt or files without an extension) would be opened externally. Directories are also treated as externally openable (https://github.com/nvim-orgmode/orgmode/issues/556). If this is not desired behaviour for directories, this could be fixed by checking the type with vim.loop.fs_stat and made configurable.

Is custom: something that exists in the Emacs Orgmode?
Also, does Emacs have a way to provide this custom executable for opening things? I didn't find anything in the docs, only some custom functions that allow this https://emacs.stackexchange.com/questions/57506/open-file-in-orgmode-with-external-application-check-if-file-exists-and-create

custom: is not something implemented in Emacs, I was mostly looking at the types of links listed here: https://orgmode.org/guide/Hyperlinks.html.
And here https://orgmode.org/manual/External-Links.html.
The shell: type is pretty much what custom: is, but with an admittedly prettier name (which is easily fixed on our end, of course). Most of the remaining link types are either not relevant for nvim or are already implemented.
There is also this feature: https://orgmode.org/manual/Adding-Hyperlink-Types.html.
Perhaps a more true-to-Emacs way to do this would be to have the link types and their respective openers defined in the config, which I could give a try.

Renamed custom: to shell:, added a config option (that I couldn't find a good short descriptive name for) to make the external opening behaviour optional. That should make the changes non-breaking.
When it is enabled, however, the issue with some plaintext not recognized as such is still there. Do you know of a better way of inferring filetypes other than with vim.filetype.match()?
I could also give implementing user-defined links a shot, if this is desired functionality.

Added an option replicating what is described in Adding Hyperlink Types, or at least a way of describing how to follow links:
org_link_types = {
    -- Opens shell: links
    -- For example [[shell:notify-send Ding!]] sends a notification on open-at-point
    shell = {
      handler = function(url)
        url = string.gsub(url, '^shell:', '')
        vim.fn.jobstart(url, { detach = true })
        return
      end,
    },
    -- Opens print: links
    -- An arbitrary example, just prints the entire link
    print = {
      handler = function(url)
        print(url)
      end,
    },
  }

This allows the user to define any prefix in a table and give it any kind of handler one would want.

@kristijanhusak, @jgollenz What should we do with this PR? Actually I would like to have the possibility to open file links in orgmode with XDG_OPEN (e.g. PDFs, images), because it doesn't make any sense to open them in Neovim as text. I am used to reference PDFs in my notes, but this is currently useless. While I already thought to implement something myself, I found this open PR, which looks promising.

@seflue we will probably close this.
I plan to refactor hyperlinks to support multiple sources, like it's done for the autocompletion https://github.com/nvim-orgmode/orgmode/tree/master/lua/orgmode/org/autocompletion, and then expose the API to add own sources. I just didn't get to it yet.

@seflue nothing to add from my part, except a sincere sorry to @maidl0ver for the radio silence on this MR.

@seflue we will probably close this. I plan to refactor hyperlinks to support multiple sources, like it's done for the autocompletion https://github.com/nvim-orgmode/orgmode/tree/master/lua/orgmode/org/autocompletion, and then expose the API to add own sources. I just didn't get to it yet.

I wasn't aware how much you actually refactored autocompletion - this just looks much more structured and extendable then what I remember from when I did my first refactoring on hyperlinks.

Hey, I'm looking forward to this possibly landing in orgmode. A note I'd love to make is how orgmode has the hyper link types (which you can see here). If the refactor could in some way include adding hyperlink types, that would be awesome.
That would also (I imagine) give direction towards custom link transformation when exporting a pdf (which you can see here). These are some powerful org-mode utils.


[regression] doom-one and doom-vibrant not rendering background correctly

What happened?

What did you expect to happen?

Steps to reproduce

configure to use doom-vibrant: (setq doom-theme 'doom-vibrant)
start doom-emacs, edit some file

System Information

Emacs version: 27.1
Emacs distro: Doom Emacs
Operating system: mosh from Windows to Linux
Installed commit of doom-themes: b2c0ea0
Which theme are you using: doom-vibrant

More information
from https://www.reddit.com/r/DoomEmacs/comments/nvrvoz/the_doomone_theme_is_not_rendered_correctly_in/

I have met the problem on Konsole, too.
Because of this line https://github.com/hlissner/emacs-doom-themes/blob/b2c0ea0f0778a6ea85b87e1b87916572e98e1fe7/themes/doom-one-theme.el#L52,
"brightblack", which is the default background, is rendered as gray on Konsole, perhaps on other terminal emulators, too.
It's back to black after setting the 256 bg color to nil
You should open an issue at https://github.com/hlissner/emacs-doom-themes/issues
Edit: it's due to this commit https://github.com/hlissner/emacs-doom-themes/commit/7ce9be332fa6b5f805ee54c47b65e07060aec37c

As of 5221b06 this issue should be resolved. Let me know if that isn't the case and I'll reopen it. Thanks for bringing it to my attention!


Update training operators doc

TFJob And PyTorchJob pages are updated with latest training operator repo changes
Related: #2915
/cc @Jeffwan

Thanks @johnugeorge!
/lgtm
/cc @RFMVasconcelos

I will help  update the MXNet part. @andreyvelich


Improve error visibility in stress test

We should make sure we log out the error message.
We should also gracefully handle a load node peer failing. Currently we end up logging out a bunch of socket hangup messages that are not super helpful.


About using hybrid task for run_cifar10.sh

Hi @LuChengTHU, thank you for the great work, which is very inspiring.
I run your code by defaulting settings, which work very well.
Now, I wanted to train your model for the hybrid task. Specifically, I run
run_cifar10.sh with adding --task hybrid, then I got the following error

Traceback (most recent call last):
File "train_img.py", line 507, in
model(x, restore=True)
File "/home/miniconda3/envs/research/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/GitHub/implicit-normalizing-flows/lib/implicit_flow.py", line 198, in forward
x = self.transforms[idx].forward(x, restore=restore)
File "/home/GitHub/implicit-normalizing-flows/lib/layers/container.py", line 15, in forward
x = self.chain[i](x, restore=restore)
File "/home/miniconda3/envs/research/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'restore'

Can someone help me with the error? Thank you very much.

Hi @musikisomorphie , thanks for your interest in our work!
We didn't run the hybrid modeling task in our experiments. Our code is based on the released code of 'Residual Flows' and we didn't remove the unused parts.
Nevertheless, you can try to train the tasks to check the performance. The error above is because we need to add a new keyword "restore" for the "forward" function to all flow blocks in "self.trainsforms". You can check "self.transforms" to see what transforms are used and modify them. In our experiments we only modify the transforms that we needed.

Thank you for the reply, @LuChengTHU. I will give it a try.


Redis Express App

This PR is related to the following issue:
https://github.com/keploy/keploy/issues/2296
Request for Review: @khareyash05  @Sonichigo  – Kindly take a moment to review this PR. Your feedback will be highly appreciated.

@khareyash05 I have made the required changes in the readme file , if there need be any changes let me know

Also please attach the screenshots of the markdown after your commit, will be easier to check

Added instructions for docker container and cleaned the readme file using online editor.


Currency Converter Not Working

Describe the bug
Developer who has Done currency converter added api but not api key
so it is giving something went wrong at each time
To Reproduce
Steps to reproduce the behavior:
open extension and try any operation you will get this error
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots

code mistak

Desktop (please complete the following information):

OS: Windows
Browser Chrome

Additional context
we can add API key of same source is it is free else i will use another API and will amke proper implmenration

@Sulagna-Dutta-Roy
kindly assign me this issue


on TRY step, add ability to fast forward upgrade sequence when counterparty is ahead

Description
closes: #3794

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

[ ] Targeted PR against correct branch (see CONTRIBUTING.md).
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Code follows the module structure standards and Go style guide.
[ ] Wrote unit and integration tests.
[ ] Updated relevant documentation (docs/) or specification (x/<module>/spec/).
[ ] Added relevant godoc comments.
[ ] Provide a commit message to be used for the changelog entry in the PR description for review.
[ ] Re-reviewed Files changed in the Github PR explorer.
[ ] Review Codecov Report in the comment section below once CI passes.

Superseded by https://github.com/cosmos/ibc-go/pull/4362#pullrequestreview-1581947747.


Prepare release for Azure.ResourceManager.NetApp

API change check
API changes are not detected in this pull request.


File Upload Pattern Design

Following pattern framework criteria the scope as follows:

[x] Create pattern site page for review
[ ] Define if Figma sticker component or example is required. (Or both if the case)

@bhoppers2008 the the file upload pattern site page is ready for review.

Frappe Goal: Publish pattern to site using Static List

Ben reviewing Mon 30th Sep

Ready for dev

Reviewed the figma file and left comments for review.

Reviewed. Updated. Ready for dev

Figma sticker outstanding. Complete in Lungo.

File upload sticker branch reviewed with @yunjungyeh. QA to be performed yet.

@ivan-calderon completed design and stickers.
Closing this ticket and @joshwooding can create a new one for the dev portion.


Basic Usage Docs

Create some basic docs on how to use the CLI tooling.  Especially what PAT scopes are needed in order to run the various commands.
I've been testing with All Orgs + Full Access for the ADO PAT, and similarly broad scopes for the GH PAT.
The scopes needed will be diff for diff commands (e.g. generate-script only does ADO reads, but obviously some other commands are writing stuff).
There's another issue #20 to code in some explicit tests for the right permissions, this issue represents just figuring out what's needed and documenting it.

We'll need to condense this down into something easier for customers to consume, but here's the detailed info on what scopes are needed for which commands:
generate-script
ADO: Build (Read)
ADO: Code (Read)
ADO: Service Connections (Read)
generate-script (--repos-only)
ADO: Build (Read) -- REMOVING THIS in v0.5
ADO: Code (Read)
migrate-repo
ADO: Code (Read)
GH: Repo (all)
GH: read:org
** ADO: Identity (Read)
** ADO: Work Item (Read)
add-team-to-repo
GH: Repo (All)
configure-autolink
GH: Repo (All)
create-team
GH: Repo (All)
lock-ado-repo
ADO: Code (Read)
ADO: Identity (Read)
ADO: Security (Manage)
disable-ado-repo
ADO: Code (Read, write & manage)
rewire-pipeline
ADO: Build (Read & execute)
share-service-connection
ADO: Full Access
integrate-boards
ADO: Full Access (for GetGithubHandle API)
GH: Repo (All)
grant-migrator-role
GH: read:org
revoke-migrator-role
GH: read:org


User tools fallback to default zone/region

This fixes https://github.com/NVIDIA/spark-rapids-tools/issues/1018.
This PR fallsback to default  region/zone(where applicable)  for CLI command when Region/Zone is not set by the user.
Earlier it would throw an error which doesn't show the exact reason though. With this PR, it  continues with the default region with a warning that Region was not set and using the default values from environment variable.
In addition to it, updated the way the remaining environment variables are set in sp_types.py. Earlier condition would miss some of the environment variables.
Tested it on platform:

dataproc
emr
databricks-aws

databricks-azure has  already default defined.

 Dataproc failure

 spark_rapids qualification --eventlogs=gs://< PATH TO EVENTLOGS>  --platform=dataproc
RuntimeError: Error invoking CMD :
        | ERROR: (gcloud.compute.machine-types.describe) Could not fetch resource:
        |  - Invalid value for field 'zone': 'None'. Must be a match of regex '[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?'

 EMR failure

spark_rapids qualification --eventlogs=s3://{PATH_TO_EVENTLOGS} --platform=emr --verbose
File "/home/test/spark-rapids-tools/user_tools/src/spark_rapids_pytools/common/utilities.py", line 333, in exec
    raise RuntimeError(f'{cmd_err_msg}')
RuntimeError: Error invoking CMD :
        |
        | Could not connect to the endpoint URL: "https://elasticmapreduce.None.amazonaws.com/"

 EMR completion with this PR

        Instance types conversions:
------------  --  ----------
m6gd.4xlarge  to  g5.4xlarge
------------  --  ----------
To support acceleration with T4 GPUs, switch the worker node instance types
Processing Completed!

Since this updates region in our env_vars and not the actual CLI configuration, CLI cmds such as
aws emr describe-cluster --cluster-id {cluster_id} might crash because it will try to get region from the CLI config. We might have to add the region explicitly in these CLI cmds.

Yes @parthosa ! You are correct. Region becomes mandatory when --cluster is provided as the argument. We get a better error now to set it explicitly:
Error invoking CMD <aws emr list-clusters --query 'Clusters[?Name==`emr_perfio_on_filecache_on_us_east_1a`]'>:
        |
        | You must specify a region. You can also configure your region by running "aws configure".


customViewForEmptyDataSet in 1.8.1

when I create CustomeView in customViewForEmptyDataSet delegate , there is nothing display

I'm facing the same issue. I'm creating a  custom UIView and setting its frame to the frame size of my tableview and nothing appears.

It use  Autolayout, and the frame doesn't work
Maybe you need to create a subclass that inherits from  UIView, and then override this method:
- (CGSize)intrinsicContentSize;

For example:
@interface CustomView : UIView

@end

@implementation CustomView

- (CGSize)intrinsicContentSize {
    return CGSizeMake(0, 300);
}
@end

- (UIView *)customViewForEmptyDataSet:(UIScrollView *)scrollView {
    CustomView *emptyView = [[CustomView alloc] init];
    emptyView.backgroundColor = [UIColor redColor];
    return emptyView;
}

it works like that:

its work for me, thx!

It use Autolayout, and the frame doesn't work
Maybe you need to create a subclass that inherits from UIView, and then override this method:
- (CGSize)intrinsicContentSize;
For example:
@interface CustomView : UIView

@end

@implementation CustomView

- (CGSize)intrinsicContentSize {
    return CGSizeMake(0, 300);
}
@end

- (UIView *)customViewForEmptyDataSet:(UIScrollView *)scrollView {
    CustomView *emptyView = [[CustomView alloc] init];
    emptyView.backgroundColor = [UIColor redColor];
    return emptyView;
}

it works like that:

it is work for me, thx!

its work for me, thx!


Empty sets should be allowed as input

Currently the codegen adds an assertion (e.g., for python)
assert isinstance(E, list) and len(E) > 0

Shouldn't this be
assert isinstance(E, list) and len(E) >= 0

?
For example, it is perfectly valid to build the (all-zero) adjacency matrix of an empty set of edges.

We used set in C++, so maybe E = frozenset(E) is a better choice.
Do we need to check the size of each element in a set for Python?
In C++, the type definition ensures this.
std::set<std::tuple< int, int > > or std::set<std::tuple< int, int, int > >

We could check the size in Python with assert all( len(el) == n for el in E )

Fixed


Minor map bugs

Monza
Monza tire stack without collision
Location: Pit entrance

Monza grass, gravel, sand without speed loss
Driving on any non-track surface doesn't incur a speed penalty like on Party Parks.
Monza audio issue
The audio shifts from stereo to the right side only when driving through Curva Grande.
Then, at Variante Ascar, it shifts to the left side only.
Spa
Spa crash barrier without collision
Location: Eau Rouge

Spa grass, gravel, sand without speed loss
Driving on any non-track surface doesn't incur a speed penalty like on Party Parks.
Spa track markers / objects don't have physics
Objects completely lack physics on Spa.

https://github.com/PartyKarts-io/UnityGame/pull/31
This PR fixes the collider issues you've reported - I will close this bug and re-open another for tracking the friction material issue at the new tracks


Python: OOP-init #566

Description 📜
In this documentation I've added described about init method along with some examples
Fixes #566

Type of change 📝

[ ] Au#dio (Should be in mp3 format Includes speech clarity, Concise ,Low distortion)
[ ] Vi#deo (Animations, screen-recordings, presentations and regular explanatory films are all possibilties etc)
[x] Documentation (Content Creation in the form of codes or tutorials)
[ ] Other (If you choose other, Please mention changes below)

Domain of Contribution 📊

[ ] Android Dev #(Flutter)
[ ] Android Dev #(Java)
[ ] Android Dev #(Kotlin)
[ ] Backend Dev #(Java)
[ ] Backend Dev #(.NET)
[ ] Backend Dev #(PHP)
[ ] Backend Dev #(Python)
[ ] C/#CPP
[ ] Competitive #Programming
[ ] Cyber #Security
[ ] DS#A
[ ] Data#base
[ ] Datascience with #Python
[ ] Datascience with #R
[ ] Frontend Dev #HTML/CSS/JS
[ ] Frontend Dev #React/Angular/Vue
[ ] Go#lang
[ ] Interview #Prep
[ ] Java_#Domain
[ ] Java#Script
[ ] ME#RN
[ ] Machine #Learning
[ ] Open #Source
[x] Python_Domain
[ ] Ru#st
[ ] Statis#tics
[ ] UI/#UX

Checklist ✅

[x] I follow Contributing Guidelines & Code of conduct of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GWOC'21 contributor

Screenshots / Gif (Optional) 📸

Hello
My name is Rohit Goyal
pls assign the documentation part of this issue to me
i am from python batch-6

Hello
My name is Ankit Kumar
i am from python batch-5
pls assign the documentation part of this issue to me

@Ankit-Tec40 @rohit-goyal, this is a PR not an issue.
@SaijyotiTripathy, your PR needs more details for both the files and Change your project folder to Issue Title : OOP-Init


feat: add fancy collapsible header logic

Description
This PR adds logic to support a fancy collapsible header. This is needed for the upcoming work on the Activity screen and potentially for other similar screens. Ideally, we can have a similar logic for all the screens 🤞
How to use

Customization

Demo
https://github.com/artsy/eigen/assets/11945712/7a198180-9646-4073-bf2b-03bacd4d086a

PR Checklist

[x] I have tested my changes on iOS and Android.
[x] I hid my changes behind a feature flag, or they don't need one.
[x] I have included screenshots or videos, or I have not changed the UI.
[x] I have added tests, or my changes don't require any.
[x] I added an app state migration, or my changes do not require one.
[x] I have documented any follow-up work that this PR will require, or it does not require any.
[x] I have added a changelog entry below, or my changes do not require one.

To the reviewers 👀

[ ] I would like at least one of the reviewers to run this PR on the simulator or device.

Changelog updates
Changelog updates

Cross-platform user-facing changes

add fancy collapsible header logic - mounir

iOS user-facing changes

Android user-facing changes

Dev changes

Need help with something? Have a look at our docs, or get in touch with us.

This PR contains the following changes:

Cross-platform user-facing changes (add fancy collapsible header logic - mounir)

  Generated by :no_entry_sign: dangerJS against 410c2cd63be8884412ba78997628d4c230031229

since we do have the Screen and Screen.AnimatedHeader components in palette are we thinking of embedding this logic there as well?

Nice idea, I think it should be possible to add this there as well. I had this here initially because it's only used in the Activity screen but I will talk to design to see if this is something we would like to add everywhere to include it in palette.

This looks great! 🌟

Thank you!

Doesn't PageWithSimpleHeader has the same functionality (just without this great animation)?

Thanks for the feedback as well. Yes, this is accurate, but it's unfortunately not as nice as a UX as I believe this is. My goal here is to slightly elevate the UX and hopefully we can have this in many other screens once Barney approves it

Thanks for the feedback as well. Yes, this is accurate, but it's unfortunately not as nice as a UX as I believe this is. My goal here is to slightly elevate the UX and hopefully we can have this in many other screens once Barney approves it.

Yes, your header component looks way better 🤩. Then, I will approve this. Feel free to add it to the new Activity screen.
But anyway, I think we should update PageWithSimpleHeader and the Palette's Screen component to use this new animation/implementation. If we only add it to the Activity screen, we will have an inconsistent header throughout the app, which isn't ideal.

I am on it 🤝

I will close this in favor of this one https://github.com/artsy/palette-mobile/pull/176


Figure out package version

Currently we are targeting 1.0.0, should it be 2.0.0 ?

FYI @AlexGhiondea

Version decided is 3.0
We need to make sure to update our docs and package with this information


Hot-cold-assets

Summary
Add the ability to specify whether a static file should be encoded in the hot package or the cold package.
Motivation
Fixes #116
Test Plan

[X] Compiles
[x] Path following works with example path
[x] Project with lemlib installed compiles with example.txt

Additional Notes
This fixes the only confirmed error in LemLib

Download the template for this pull request:

[!NOTE]
This is auto generated from Add Template to Pull Request

via manual download<EMAIL_ADDRESS>via PROS Integrated Terminal:

curl -o<EMAIL_ADDRESS>https://nightly.link/LemLib/LemLib/actions/artifacts/1796943050.zip;
pros c fetch<EMAIL_ADDRESS>pros c apply<EMAIL_ADDRESS>rm<EMAIL_ADDRESS>

Hardware testing successful, ready for review


Update ProcessContext.java: fixed crazy bug with pointless vararg for List

Quickfix for odd API bug I found during code-review.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>2
Details

0 of 0   changed or added relevant lines in 0 files are covered.
2 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.02%) to 66.754%

Files with Coverage Reduction
New Missed Lines
%

com/devonfw/tools/ide/process/ProcessContext.java
1
40.0%

com/devonfw/tools/ide/url/model/file/json/ToolDependencies.java
1
76.92%

Totals

Change from base Build<PHONE_NUMBER>3:
-0.02%

Covered Lines:
6266

Relevant Lines:
9039

💛  - Coveralls


Enums loaded as part of transaction after UPDATE, slowing down transaction.

Bug description
Running on Cockroach, we're noticing times when a transaction gets created that leads to row contention and takes 1-2s to complete. Here's the transaction:
SELECT public."GatherEvent".id FROM public."GatherEvent" WHERE public."GatherEvent".id = $1
UPDATE public."GatherEvent" SET progress = $1, "updatedAt" = $2 WHERE public."GatherEvent".id IN ($3,)
# I believe these are all enum lookups, a little surprising that's not cached? We're not changing the enums ever.
 FROM public."GatherEvent" WHERE public."GatherEvent".id = $1 LIMIT $2 OFFSET $3

Our application code is extremely simple, there's only one place where we're doing an UPDATE:
  const event = await prismaDb.gatherEvent.update({ ...byId(eventId), data: patch });

which we then await in a controller, where that is the only interaction with prisma.
Our HTTP servers only create one Prisma client per instance as suggested in the docs, and reuses the client across all connections to that server instance.
I dug in with the Cockroach team and they're fairly confident it's not due to anything weird on their end.
So now I'm checking in here, wondering if it's possible that Prisma would be taking that UPDATE query and lumping it into a transaction with aSELECT query by mistake?
How to reproduce
Unclear, we're seeing it occur regularly in prod, but didn't notice this behavior in testing.
Expected behavior
The UPDATE runs alone in a transaction.
Prisma information
generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "cockroachdb"
  url      = env("DATABASE_URL")
}

model GatherEvent {
  id String @id @default(dbgenerated("replace(replace(encode(substr(gen_random_uuid()::BYTES, 0, 16), 'base64'::STRING), '/'::STRING, '_'::STRING), '+'::STRING, '-'::STRING)"))
  // ...
}

enum GatherEventCategory {
  // ...
}

// we have to combine all possible enum values into one since SQL doesn't support multiple enum types for a column
enum GatherEventAttendanceNumber {
  // ...
}

enum GatherEventPrivacy {
  // ...
}

enum GatherEventProfessionalCategory {
  // ...
}

enum GatherEventUseCase {
  // ...
}

Environment & setup

OS: Occurs presumably on linux instances, it's the CRDB cloud servers
Database: Cockroach 22.1.5
Node.js version: 12.17.4

Prisma Version
prisma                  : 4.2.1
@prisma/client          : 4.2.1
Current platform        : darwin
Query Engine (Node-API) : libquery-engine 2920a97877e12e055c1333079b8d19cee7f33826 (at<EMAIL_ADDRESS>Migration Engine        : migration-engine-cli 2920a97877e12e055c1333079b8d19cee7f33826 (at node_modules/@prisma/engines/migration-engine-darwin)
Introspection Engine    : introspection-core 2920a97877e12e055c1333079b8d19cee7f33826 (at node_modules/@prisma/engines/introspection-engine-darwin)
Format Binary           : prisma-fmt 2920a97877e12e055c1333079b8d19cee7f33826 (at node_modules/@prisma/engines/prisma-fmt-darwin)
Default Engines Hash    : 2920a97877e12e055c1333079b8d19cee7f33826
Studio                  : 0.469.0

Hm, not good. I think I need a bit more info:
Where does this list of SQL queries at the beginning of the issue come from?
Is this SQL query logging of Prisma Client? (The enum lookups make me think it is not...)
Do you know if some/any of these queries are actually slow?
Or are you sure there is some contention going on? Where exactly?
Where do you see "unexpected transaction grouping" really (see note at end of comment)?
The enum SQL queries are something that our database driver does under the hood, and which is usually not visible at all. Only in context of CockroachDB I have actually seen those before to be really visible.
Is this issue reproducible in any way?
Side note:
Generally Prisma's job is to do to your data what you tell it to, you say update and at the end the data needs to be updated. If it does that with a SELECT query in between (or at the end) that usually has a reason, as for example getting any automatically changed values or similar. Prisma can and does not guarantee that an update only creates a UPDATE query. So by itself such "unexpected" SQL should not considered a bug - unless of course it leads to undesirable performance behavior as you describe it.

Thanks for the quick response!

Where does this list of SQL queries at the beginning of the issue come from?

Sorry, let me give you more context, it's coming from the CRDB admin dashboard.
The queries by themselves are definitely very fast. Here's the SQL statements table of fingerprints, they're all very fast normally:

Here's a few examples with detailed views:

It's only when it's part of this transaction where it fails.

One of the CRDB field engineers did some digging and found it's part of this transaction that gets locked. Here's the table of transaction fingerprints, notice how they're all 1+ sec, while all the queries from before were ~20-40ms:

Here's a couple detailed views. Many of them actually don't even involve the enums, so while that's pretty weird it might be a red herring to look at another day.

Or are you sure there is some contention going on? Where exactly?

The detail views show how each statement is 1-5ms generally, but the transaction is 1+ sec, so Bob @ Cockroach was betting on it being contention.

Where do you see "unexpected transaction grouping" really

Ah that helps clarify. I wasn't expecting to see transactions since we don't manually create transactions with Prisma, and that particular controller only does an update.
It seems to be happening without heavy traffic, our qps is quite low.

reproducible

Hmm this part is tougher. I haven't had the chance to run this locally yet and see if I can repro it. It's a little tricky since macOS has different behaviors from what's deployed in the cloud. It's a basic 3-node cluster.

I spun up another cluster in the cloud, and tried to reproduce it with a simple script that would randomly read and update the same record in the DB every 1-10ms.
Script
  const MAX_ITERATIONS = 300;
  let readCount = 0;
  let updateCount = 0;

  const read = async () => {
    readCount++;
    await prismaDb.gatherEvent.findUnique({ ...byId(id) });
    if (readCount >= MAX_ITERATIONS) {
      console.log("done reading");
      return;
    }
    setTimeout(read, Math.random() * 10);
  };

  const update = async () => {
    updateCount++;
    await prismaDb.gatherEvent.update({ ...byId(id), data: { progress: updateCount } });
    if (updateCount >= MAX_ITERATIONS) {
      console.log("done updating");
      return;
    }
    setTimeout(update, Math.random() * 10);
  };

  read();
  update();

I wasn't able to reproduce it in small 30s batches, but I think I'll write it so it just runs longer for an hour and see what happens. So far things look pretty normal.
One thing that surprised me while digging is just how many statements are involved with a single .update call. To repro this, I wiped the SQL activity history, and ran a single query:
await prismaDb.gatherEvent.update({ ...byId("some-id"), data: { hostId: "42" } });

With this, I got a transaction that involved two SELECTs and one UPDATE.

I definitely get that sometimes an ORM will generate some extraneous queries, but this seems quite heavy, is a SELECT happening before and after every .update? Experimentally, this seems to be happening. Here's the activity list after I run that same statement 100 times:

(This is on local, running in memory, so the speeds were quite a bit higher.)

Okay, so it's run for a few hours, and I'm not able to reproduce it with a simple script. Here's the script, for sake of record keeping:
Script
import { prismaDb } from "gather-prisma-backend/dist/src/public/client";
import { byId } from "gather-prisma-backend/dist/src/public/queryBuilders";
import {
  GatherEventAttendanceNumber,
  GatherEventCategory,
  GatherEventProfessionalCategory,
  GatherEventUseCase,
  PrismaTypes,
} from "gather-prisma-types/dist/src/public/client";

export const event: PrismaTypes.GatherEventCreateInput = {
  category: GatherEventCategory.EducationOrAcademia,
  coverImage: "foo",
  description: "foo",
  draft: true,
  endDate: new Date(),
  hostId: "foo",
  hostName: "foo",
  minutesEarlyToOpen: 0,
  minutesLateToClose: 0,
  name: "foo",
  numAttendees: GatherEventAttendanceNumber.num_25_to_50,
  professionalCategory: GatherEventProfessionalCategory.external,
  progress: 0,
  spaceId: "foo",
  startDate: new Date(),
  type: GatherEventUseCase.ClassOrWorkshop,
};

const run = async () => {
  const { id } = await prismaDb.gatherEvent.create({ data: { ...event, hostId: "abc" } });

  let readCount = 0;
  let updateCount = 0;

  const read = async () => {
    readCount++;
    await prismaDb.gatherEvent.findUnique({ ...byId(id) });
    if (readCount % 1000 === 0) {
      console.log("Read count:", readCount);
    }
    setTimeout(read, Math.random() * 10);
  };

  const update = async () => {
    updateCount++;
    await prismaDb.gatherEvent.update({ ...byId(id), data: { progress: updateCount } });
    if (updateCount % 1000 === 0) {
      console.log("Update count:", updateCount);
    }
    setTimeout(update, Math.random() * 10);
  };

  read();
  update();
};

run().catch();

Results don't have the same spiky look:

I'll now try running it without a random delay in between requests, see if that changes anything...
So, things I'm thinking about:

It's not an rps thing
It's not a "making repeated, fast updates one after another" thing
It's probably not "occasionally we read and update to the same document at the same time", though this would be better simulated by multiple actors
It seems to be driven by traffic/user patterns somehow. The usual workflows:

An index page, showing all events
User clicks an event, displays the one event
Makes some edits, saves to update, goes back to show the event
When users are editing events, usually it's before it's published, so it's highly unlikely multiple users are looking at the same public event. More than likely these are one person looking at one event, and somehow that's causing contention

Unless somehow there's contention when doing read/writes to different documents. That seems completely unlikely though.

Copying over some comments from the CRDB engineer:
"For transactions like this one:
SELECT public."GatherEvent".id FROM public."GatherEvent" WHERE public."GatherEvent".id = $1
UPDATE public."GatherEvent" SET progress = $1, "updatedAt" = $2 WHERE public."GatherEvent".id IN ($3,)
SELECT public."GatherEvent".id, ... FROM public."GatherEvent" WHERE public."GatherEvent".id = $1 LIMIT $2 OFFSET $3

this TX pattern would be improved by using SELECT FOR UPDATE. Changing that first statement to SFU would go ahead and create the locks that the subsequent UPDATE has to create, so the transaction can be ordered correctly at the outset. The mean transaction time in that fingerprint is 114ms, but the statements themselves are low, single-ms times.
Is there a way to instrument the client code to look at the inter-statement times?
What I’m seeing in the console suggests that transactions are being held open quite a bit longer than it takes to run the SQL commands, and that UPDATE in the middle of the TX would block other accesses to the same id.  I think we’re seeing this, given the contention numbers seen on SELECT in the other transaction signatures."

After more debugging today, it looks like the really bad transaction times are due to long network latencies across the globe. Our DB cluster is in NYC, and we have servers in NYC, FRA, and SGP that access that DB. If we look at calls from just NYC are fast:

From SGP, the transactions involving an UPDATE become way too slow, up to 4.5s.

The theory Bob and I have is that it's because Prisma is making five separate round-trip messages with the DB for an .update:

BEGIN
SELECT
UPDATE
SELECT
COMMIT

We don't know if that's actually true, we poked around in the source code but it was hard to see if each of these statements in the transaction were submitted as a single block or one at a time...
If this is true, it'd explain why SGP calls are many times slower than the nominal RTT of SGP <-> NYC; 5 * 250ms = 1.25s in a perfect case. That still doesn't explain how it can continue to grow to 4+ sec, but would at least help narrow down the problem.

Okay, I have a way to reproduce it pretty easily.

Set up a CRDB cluster on the other side of the world from where you are. e.g. if you're in NYC, set one up in SGP.
Connect to it and run with this schema:

Schema
generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "cockroachdb"
  // Even though we don't use DATABASE_URL to actually connect to our DB, we need to specify a `url` here so the
  // Prisma Migrate tool knows where to connect to run the migration. Do not delete this line, otherwise it will
  // break the generation of the `.prisma` client folder.
  url      = env("DATABASE_URL")
}

model GatherEvent {
  // the @default value is ugly, but we can't create our own functions to abstract it. The string literals have to
  // be cast as ::STRING to avoid Prisma Migrate always thinking a new migration is required.
  // https://github.com/prisma/prisma/issues/14917
  id String @id @default(dbgenerated("replace(replace(encode(substr(gen_random_uuid()::BYTES, 0, 16), 'base64'::STRING), '/'::STRING, '_'::STRING), '+'::STRING, '-'::STRING)"))
  category GatherEventCategory? // enum
  coverImage String? // url
  description String?
  draft Boolean @default(true)
  endDate DateTime
  hostId String
  hostName String?
  minutesEarlyToOpen Int?
  minutesLateToClose Int?
  name String
  numAttendees GatherEventAttendanceNumber?
  privacy GatherEventPrivacy?
  professionalCategory GatherEventProfessionalCategory?
  progress Int @default(0) // 0 - 100
  spaceId String?
  startDate DateTime
  type GatherEventUseCase?

  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
}

enum GatherEventCategory {
  ProfessionalOrBusiness
  CommunityOrOrganization
  EducationOrAcademia
  FriendsOrFamily
  Other
}

// we have to combine all possible enum values into one since SQL doesn't support multiple enum types for a column
enum GatherEventAttendanceNumber {
  num_0_to_25
  num_25_to_50
  num_50_to_100
  num_100_to_200
  num_200_orMore
  num_26_to_100Legacy
  num_101_orMoreLegacy
}

enum GatherEventPrivacy {
  public
  private
}

enum GatherEventProfessionalCategory {
  internal
  external
  NA
}

enum GatherEventUseCase {
  PartyOrSocialGathering
  TalkOrSeminar
  ConferenceConventionOrExpo
  MeetingOrNetworkingEvent
  ClassOrWorkshop
  Other
}

Open up the monitoring tab of CRDB, you'll need to add a SQL user and acceptlist your IP to access the monitoring page

Run this script:

Script
import { PrismaClient } from "@prisma/client";

export const db = new PrismaClient();

export const event = {
  category: "EducationOrAcademia",
  coverImage: "foo",
  description: "foo",
  draft: true,
  endDate: new Date(),
  hostId: "foo",
  hostName: "foo",
  minutesEarlyToOpen: 0,
  minutesLateToClose: 0,
  name: "foo",
  numAttendees: "num_25_to_50",
  professionalCategory: "external",
  progress: 0,
  spaceId: "foo",
  startDate: new Date(),
  type: "ClassOrWorkshop",
};

const run = async () => {
  const { id } = await db.gatherEvent.create({ data: event  });

  let updateCount = 0;

  const update = async () => {
    updateCount++;
    const start = Date.now();
    await db.gatherEvent.update({ where: { id }, data: { progress: updateCount } });
    console.log("done in", Date.now() - start);
    setTimeout(update, 0);
  };

  update();
};

run().catch();

in Monitoring console, go to [SQL Activity] > [Transactions]. Let the script run, it'll eventually show a few transactions that are 4+ seconds long. Most will be ~900ms.

It also turns out that the long transaction happens consistently when Prisma client connects for the first time. If you reset SQL activity, modify the script to only run the update once (comment out this line), and run the script, you'll see a  4+sec transaction run each time.

I'm still investigating whether the long transaction shows up more than once, or if it's only on initialize...

It looks like the enums are consistently what slows the transaction so much on the initial call. If I create the same table but without enums:
model EventWithoutEnum {
  id String @id @default(dbgenerated("replace(replace(encode(substr(gen_random_uuid()::BYTES, 0, 16), 'base64'::STRING), '/'::STRING, '_'::STRING), '+'::STRING, '-'::STRING)"))
  category String? // enum
  coverImage String? // url
  description String?
  draft Boolean @default(true)
  endDate DateTime
  hostId String
  hostName String?
  minutesEarlyToOpen Int?
  minutesLateToClose Int?
  name String
  numAttendees String?
  privacy String?
  professionalCategory String?
  progress Int @default(0) // 0 - 100
  spaceId String?
  startDate DateTime
  type String?

  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
}

and then run the script, it shows that the first .update still has a slower-than-normal transaction time (~1400ms vs ~900ms), but not nearly the same as the equivalent call but for a table with enums.
So I guess the immediate question then becomes: can we get confirmation that prisma is loading each enum separately over the wire? If so, is there a way to change that behavior?

It also looks like "pre-loading" the enums by grabbing a random record off the table fixes the issue. The problem was that the enum load when it comes from .update will occur after the UPDATE has created a write intent and locked the row, and then it sits there for ~3s loading enums one at a time, and so the whole transaction blocks. By doing a simple load before any other calls, the enums appear to be cached into the prisma and the subsequent .update call doesn't require the enums to be loaded.
 const run = async () => {
+  await db.gatherEvent.findFirst(); // pre-load enums
   const { id } = await db.gatherEvent.create({ data: event });

   let updateCount = 0;

   const update = async () => {
     updateCount++;
     const start = Date.now();
     await db.gatherEvent.update({ where: { id }, data: { progress: updateCount } });
     console.log("done in", Date.now() - start);
   };

   update();
 };

Here's what the transactions look like with that one line tweak:

Notice how the pg_catalog.pg_type call (the one calling for 420KB of memory) is now split away from the UPDATE transaction, and the slowest transaction is now 1.2 sec, down from 3.5-4.5 sec.

If so, is there a way to change that behavior?

Unfortunately nothing instant and trivial. This is default behavior of our database driver we are using here, and was never a problem in the past. The distributed nature of CockroachDB seems to add another layer here, which makes these queries even noticable (I doubt many PostgreSQL users even are aware these happen on connect).
We'll need to investigate and see if and how we can fix this. Your manual workaround give a good pointer into which direction we should probably go.
Do you consider the workaround good enough for now to proceed here?

The distributed nature of CockroachDB seems to add another layer here

I haven't tested against Postgres yet, but I'm pretty sure the distributed SQL aspect is unrelated. I think it happens when you have a server talking to a DB very far away. (I also realize that's pretty atypical... it was our existing server architecture, which I'm currently working on changing.)

We'll need to investigate and see if and how we can fix this. Your manual workaround give a good pointer into which direction we should probably go.

Great to hear it! That'd be much appreciated.

Do you consider the workaround good enough for now to proceed here?

Yeah I think we're set for now, we can work around it. It's not my favorite to remind engineers "every time you add a table with enums on it, make sure to preload it", but it's way better than the system not working at all.

Internal note:
The gist of this issue/problem here are additional queries our PostgreSQL Rust driver is doing to get some information about types. These do not appear in our usual query logs, but need to be logged on the database side (examples above). On globally distributed databases these queries can be slow enough to cause trouble, but probably they make a lot of other queries and connection creations slow in general. We had similar issues in the past, where this also surface with CockroachDB.

After some additional experimentation, our original workaround doesn't seem to be as effective as expected. Right now we have DB in NYC and http-servers in NYC + SFO. The difference in performance for calls SFO -> NYC vs NYC -> NYC is linearly measurable: (normal NYC-NYC perf) + (roundtrip NYC + SFO) * (number of enums).

http-server
latency (ms)

NYC
10

SFO (raw SQL)
100

SFO (prisma)
500

So, while I agree it's fair to say Prisma shouldn't have to solve for a topology where a http-server is talking to a DB halfway around the world, I think having a server talking across the US country isn't uncommon, and the impact is considerable for models with enums.

Linking this other investigation that landed on the same issue: https://github.com/prisma/prisma/issues/11317#issuecomment-1065414426

I haven't dug into this issue recently, I imagine #5043 helps but it seems there's still many heavy queries against for metadata about enums and types. They're in the top 20-ish in terms of execution count...

and in 4th and 11th place in terms of runtime execution (relative amount of compute time)


pass async functions to transaction

Looking at the code here I'm certain this is a feature request:
https://github.com/JoshuaWise/better-sqlite3/blob/master/lib/transaction.js#L58

my pseudo code looks like this:
const run = db.transaction(async () => {
  const rows = await sheet.getRows({ offset: 0, limit: 300 }) // <-- this step is async
  for (const row of rows) db.prepare('INSERT ...').run(row)
})
run()

I don't have enough memory to retrieve all rows at once outside the transaction

with the verbose option enabled the console output looks like this:
BEGIN
COMMIT
INSERT ...

Wanted behavior is
BEGIN
INSERT ...
COMMIT

This is not possible or wanted given the design of better-sqlite3.
https://github.com/JoshuaWise/better-sqlite3/issues/319#issuecomment-549099116

You should never keep SQLite3 transactions open across event loop ticks in Node.js. SQLite3 serializes all statements/transactions. This means, if you try to run an asynchronous function within an SQL statement, you'll block all other statements/transactions across your entire program. better-sqlite3 doesn't support it, and it never will, because it's a very bad idea.

Instead, you should gather all information required for your transaction (reading files, waiting on sockets, etc.) BEFORE you start the transaction. Then, when everything is ready, run the transaction synchronously, within a single event loop tick.

I assume sheet.getRows retrieves data from another system? Do you need to run the above pseudo code regularly or just once to import the data? One way would be to do batch fetches of sheet.getRows, so that the batch fits in memory. And then do a bulk insert in a single transaction. Sure, this doesn't make the whole import transactional, but it's probably the closest you can get. Using UNIQUE keys you can at least prevent importing the same row twice in case your import crashes midway.

Ah true, I haven't thought about it but it's probably bad design if you're aiming for concurrency :/

regularly or just once to import the data
once on process start (importing) and once on process end (exporting).

this doesn't make the whole import transactional
yeah, this was what I was aiming for but doing it in bulks is probably good enough

Thanks a lot :1st_place_medal:


Cut Helm Control Flow,  Conditionals and Templates into two sections

Should be fixed with #41, will update slides now.

Slides have been updated.


Added maps subsetting data for tags released in American Samoa and Hawaii fisheries

Added maps subsetting data for tags released in American Samoa and Hawaii fisheries.  Also added a reference insert map for the American Samoa fishery as the tag map is zoomed in without land and hard to visualize on its own.

Merging tag map updates which includes subsetting data for AS and HI longline fisheries and adding reference map for AS fishery.


Tesla really needs a guidebook

Description

For things like engines it's extremely important to have a proper guide entry, should've been done before merge

Resolved by #26851; Tesla is intergrated into the same entry as singulo due to the setup similarities (and because the TG Wiki did the same thing)


Add support for unknown PGP key types

This PR adds support for OpenPGP (v6) keys with unknown public key algorithms.
The v6 test key
-----BEGIN PGP PUBLIC KEY BLOCK-----

xhQGZl7182MAAAAKAAECAwQFBgcICQ==
=anvn
-----END PGP PUBLIC KEY BLOCK-----

contains a key with unknown algorithm 99 and consists of the octets 0-9:
Public-Key Packet, new CTB, 2 header bytes + 20 bytes
    Version: 6
    Creation time: 2024-06-04 11:09:39 UTC
    Pk algo: Unknown algo 99
    Fingerprint: ACD6CEC3A014079F78DA9A60B98F9E3D4E7321834E1EBEC68A1C3701F882D5A3
    KeyID: ACD6CEC3A014079F

    00000000  c6                                                 CTB
    00000001     14                                              length
    00000002        06                                           version
    00000003           66 5e f5 f3                               creation_time
    00000007                       63                            pk_algo
    00000008                           00 00 00 0a               public_len
    0000000c                                       00 01 02 03   rest
    00000010  04 05 06 07 08 09

Handling unknown keys gracefully is possible with v6 keys, since the key encoding length is known.
TODO: Figure out, if we can also support unknown v4/v5 keys.

We can only gracefully parse v5, v6 keys, since secret key packets simply append the secret key material after the public key material, so when parsing a v4 secret key, we don't know, when to stop parsing the public key material.
V5 and v6 keys include an additional field encoding the public key length to enable graceful parsing (as implemented here).


[SPARK-31372][SQL][Test] Display expression schema for double check.

What changes were proposed in this pull request?
Although SPARK-30184 Implement a helper method for aliasing functions, developers always forget to using this improvement.
We need to add more powerful guarantees so that aliases outputed by built-in functions are correct.
This PR extracts the SQL from the example of expressions, and output the SQL and its schema into one golden file.
By checking the golden file, we can find the expressions whose aliases are not displayed correctly, and then fix them.
Why are the changes needed?
Ensure that the output alias is correct
Does this PR introduce any user-facing change?
'No'.
How was this patch tested?
Jenkins test.

Test build #121140 has started for PR 28194 at commit 36a7ce8.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25828/
Test PASSed.

Test build #121141 has started for PR 28194 at commit 8d976a2.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25829/
Test PASSed.

Test build #121141 has finished for PR 28194 at commit 8d976a2.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Test build #121140 has finished for PR 28194 at commit 36a7ce8.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class ExpressionsSchemaSuite extends QueryTest with SharedSparkSession
 protected case class QueryOutput(sql: String, schema: String)

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121141/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121140/
Test FAILed.

retest this please

Test build #121145 has started for PR 28194 at commit 8d976a2.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25833/
Test PASSed.

Test build #121145 has finished for PR 28194 at commit 8d976a2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121145/
Test PASSed.

Test build #121165 has started for PR 28194 at commit 31c7984.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25852/
Test PASSed.

Test build #121165 has finished for PR 28194 at commit 31c7984.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121165/
Test FAILed.

retest this please

Test build #121177 has started for PR 28194 at commit 31c7984.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25864/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25881/
Test PASSed.

Test build #121194 has started for PR 28194 at commit c7b565b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25882/
Test PASSed.

Test build #121194 has finished for PR 28194 at commit c7b565b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121194/
Test FAILed.

retest this please

Test build #121205 has started for PR 28194 at commit c7b565b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25893/
Test PASSed.

Test build #121177 has finished for PR 28194 at commit 31c7984.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121177/
Test PASSed.

Test build #121205 has finished for PR 28194 at commit c7b565b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121205/
Test FAILed.

Test build #121245 has started for PR 28194 at commit 42910c2.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25932/
Test PASSed.

Merged build finished. Test PASSed.

Test build #121245 has finished for PR 28194 at commit 42910c2.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121245/
Test FAILed.

retest this please

Test build #121258 has started for PR 28194 at commit 42910c2.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25945/
Test PASSed.

Test build #121258 has finished for PR 28194 at commit 42910c2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121258/
Test PASSed.

cc @gatorsmile @cloud-fan @maropu

discussed this with @beliefer offline. Let us generate a MD file as the golden file. We can build a MD table to present the schema info.

Test build #121359 has started for PR 28194 at commit c6ed125.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26043/
Test PASSed.

Test build #121359 has finished for PR 28194 at commit c6ed125.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  protected case class QueryOutput(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121359/
Test FAILed.

retest this please

Test build #121363 has started for PR 28194 at commit c6ed125.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26047/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26049/
Test PASSed.

Test build #121365 has started for PR 28194 at commit 57f78fd.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26050/
Test PASSed.

Test build #121363 has finished for PR 28194 at commit c6ed125.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
  protected case class QueryOutput(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121363/
Test PASSed.

Test build #121365 has finished for PR 28194 at commit 57f78fd.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121365/
Test PASSed.

Merged build finished. Test PASSed.

Test build #121911 has started for PR 28194 at commit da0adba.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26581/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26583/
Test PASSed.

Test build #121914 has started for PR 28194 at commit e0dde74.

Test build #121916 has started for PR 28194 at commit b6f32ca.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26586/
Test PASSed.

Test build #121914 has finished for PR 28194 at commit e0dde74.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121914/
Test FAILed.

Test build #121916 has finished for PR 28194 at commit b6f32ca.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121916/
Test FAILed.

Test build #121911 has finished for PR 28194 at commit da0adba.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121911/
Test FAILed.

Test build #121940 has started for PR 28194 at commit 9803e4a.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26613/
Test PASSed.

Test build #121940 has finished for PR 28194 at commit 9803e4a.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121940/
Test FAILed.

retest this please

Test build #121973 has started for PR 28194 at commit 9803e4a.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26648/
Test PASSed.

Test build #121973 has finished for PR 28194 at commit 9803e4a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121973/
Test PASSed.

Test build #122006 has started for PR 28194 at commit a7bc72d.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26679/
Test PASSed.

Test build #122006 has finished for PR 28194 at commit a7bc72d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122006/
Test PASSed.

The other parts look fine.

Test build #122024 has started for PR 28194 at commit f38f942.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26697/
Test PASSed.

Test build #122028 has started for PR 28194 at commit c4d45ea.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26701/
Test PASSed.

Test build #122030 has started for PR 28194 at commit 2173536.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26703/
Test PASSed.

Test build #122030 has finished for PR 28194 at commit 2173536.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Test build #122024 has finished for PR 28194 at commit f38f942.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Test build #122028 has finished for PR 28194 at commit c4d45ea.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122030/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122028/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122024/
Test FAILed.

Test build #122044 has started for PR 28194 at commit 460da00.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26720/
Test PASSed.

Test build #122049 has started for PR 28194 at commit 133456d.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26724/
Test PASSed.

Test build #122044 has finished for PR 28194 at commit 460da00.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122044/
Test FAILed.

Test build #122063 has started for PR 28194 at commit e571667.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26739/
Test PASSed.

Test build #122065 has started for PR 28194 at commit a4d4de9.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/26740/
Test PASSed.

Test build #122049 has finished for PR 28194 at commit 133456d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122049/
Test PASSed.

Test build #122065 has finished for PR 28194 at commit a4d4de9.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122065/
Test PASSed.

Test build #122063 has finished for PR 28194 at commit e571667.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122063/
Test PASSed.

thanks, merging to master/3.0!

@cloud-fan @gatorsmile @maropu Thanks for all your help!

Hi, guys.
This seems to break branch-3.0 UT. Could you take a look?

org.apache.spark.sql.ExpressionsSchemaSuite.Check schemas for expression examples

@beliefer could you do follow-up?

https://github.com/apache/spark/pull/28427

To recover 3.0 asap, I opened a PR to fix it.


Remove cd command in VM-Install-Shortcut

This fixes what was referenced here: https://github.com/mandiant/VM-Packages/issues/676#issuecomment-1877578921
After some testing, it looks like we do not need the cd C:\Users\User\Desktop piece of the command as we will already be in there due to WorkingDirectory being set there for Install-ChocolateyShortcut.

@Ana06 Turns out that Start In is explicitly ignored for elevated command prompts to mitigate a potential security issue: https://superuser.com/a/1067979
For shortcuts that are command line based and require admin privileges, we should do what we were doing before where we set the Target to perform cmd /K "cd $executableDir"
Otherwise, we can just use the WorkingDirectory field. :)

@emtuls thanks for the research! I was really confused and this clarifies. I agree we should keep it so that it works for all cases and that we can close this PR.


Fix windows-nightly upload step

This should fix missing bash command in upload step on windows-nightly
CC @aneeshusa @larsbergstrom

This change is

Thank you, this looks like it might work!  Travis might fail if lints catch the missing newline at the end of environments.py, though.

Fixed!

@bors-servo r+

:pushpin: Commit a094cad has been approved by edunham

oh wait, Bors isn't actually doing the thing on this repo at the moment. Travis passed, so I'll impersonate Bors for now.