fix: error causes if you press button OK when there’s no text input in RS_InputDialog.js

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

see below link :
https://twitter.com/Fujitsuki15/status/1691441151247892480
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Link
https://github.com/biud436/MV/blob/36d1713a81a4cfa2e1c296e972715ec8f2f46cce/RS_InputDialog.js#L1093-L1104
https://github.com/biud436/MV/blob/36d1713a81a4cfa2e1c296e972715ec8f2f46cce/RS_InputDialog.js#L1202-L1223

This appears to be an error unrelated to this plugin, but if it's null or undefined, I think we should assign a default value to make it safer.


Additional repo, docs/discussion around Node.js+JS interactive reviews

Discussed in CommComm meeting.  We want a place we can post info/discussion around selection process to make it transparent.
We thought it should be in OpenJS org as it should be across the projects.
Would something like "Conference Management" make sense?

Conference Management will be cool but then it may house other future conferences, if the use case is only for Node.js+Js interactive then this may confuse other conference organizer.
I would have suggested Node.js+JS interactive as it name but then this will lead to many others with similar purpose.
Yeah in my opinion Conference Management is :+1:

I agree perhaps conference management with a directory which is named Node.js+JS Interactive

We're hoping to publish the CFP early this week. Would be great to get more input on the conference overview, topics, and types of sessions.
December 10-12, 2019 Montreal, Canada
https://events.linuxfoundation.org/events/nodejs-interactive-2019/
At Node+JS Interactive, attendees collaborate face-to-face, network, and learn how to better their JS skills in serverless, IoT, backend and frontend contexts. The program will cover a broad spectrum of the JavaScript ecosystem including Appium, Dojo, jQuery, Node.js, or webpack, as well as frameworks, best practices, and stories from successful end-users.
Submissions must be received by 11:59pmPST on April 20, 2018.
Topics:
Architectures in Node.js
Expanding JS to JavaScript
Community Building
Diagnostics (Performance, Debugging)
Workshops
Emerging Use Cases (AI, Big Data)
Front-end Engineering
Node.js in Production
Deep Dive on Appium, Dojo, jQuery, Node.js, or webpack projects
Security
Operations (Serverless, Containers, Orchestration)
Wildcard
JavaScript Ecosystem
JavaScript in Production
JS in Multimedia & Arts
JS in Serverless
The Case for Appium, Dojo, jQuery, Node.js, or webpack
IoT
Frameworks (Lessons in Selection, Considerations, Migration, Comparisons)
Tooling
Testing Frameworks (Appium, Mocha)
Internationalization
Accessibility
Building and Evolving APIs
Mobile
PWAs
Other, please specify…
Submission Type*
BoF
Lightning Talk
Panel (*maximum 4 panelists & 1 moderator)
Presentation (*maximum 2 presenters)
Hands-On Workshop

Is this an example of last year's text or a copy/paste error in the new one?
"Submissions must be received by 11:59pmPST on April 20, 2018."

That's a copy and paste error. Submissions must be received by 11:59 pmPST on June 21, 2019.

A request was made in the last CommComm meeting to summarize key CFP review information.
Please consider this a work in progress document.
https://docs.google.com/document/d/1c0y-XSssZdEZGyCo0FVU6Wnd2TM3Ye-63xi18xjKVGM/edit
It includes the following information:
-people who have volunteered to be part of the CFP review process this year. Call for volunteers still open.
-changes for this year's review CFP process
-key CFP review dates
-feedback from last year's Node+JSInteractive attendee survey

-people who have volunteered to be part of the CFP review process this year. Call for volunteers still open.

Where is the call for volunteers? Are there any limitations - can somebody send a proposal and evaluate as well?

If anyone is interested in joining the CFP review process, please volunteer here. We are also going to ask for volunteers from the TSC. Anyone interested can send their name to me. I'll make sure they are added to the CFP review team.
No limitations. Someone can submit talks as well as review proposals.

@SarahKConway I'll volunteer :)

I'd like to help

Thanks! @MylesBorins and @Jonahss. Will add you to the list.

@SarahKConway I will be happy to help too.

@SarahKConway would love to volunteer too!

@SarahKConway I'd like to volunteer to review proposals too. :-)

Please also add me to the list @SarahKConway 🙂

@SarahKConway I'm happy to join as well.

Thanks all. Volunteers are being tracked in this document. https://docs.google.com/document/d/1c0y-XSssZdEZGyCo0FVU6Wnd2TM3Ye-63xi18xjKVGM/edit
@hackygolucky and christian-bromann will be the event co-chairs this year. They will be reviewing input from the CFP committee and responsible for making the final decisions on talks/speakers who get accepted. Thank you both!

I'd like to participate in the review process.

Happy to join as well.

I would also like to participate in the review process.

I would like to volunteer too, thanks!

I am also happy to help here, thanks :)

@SarahKConway is there anything else to do for the people who have asked to be added to the document in the last few comments?

Let's try this again... @mhdawson is there any update here?

@cjihrig I've not heard anything yet, will send email to @SarahKConway to ask for a follow up.

UPDATE: If you haven't already done so, please nominate yourself to the review committee here if you'd like to participate ( even if you've raised your hand elsewhere) This is the best way to ensure you get notified of next steps.  https://events.linuxfoundation.org/events/nodejs-interactive-2019/program/program-committee/
The deadline to add yourself to this list is June 14 (tomorrow)
The deadline for CFPs is June 21 and the review process will start Monday, June 24th.

@RRomoff the link to the form looks broken

@codeekage Updated my comment to include the form to add yourself as well as the program committee details. Should be good now! LMK if not.

Thank you! Working now.

FYI, an email with next steps for the Node+JS Interactive program committee was sent out this afternoon from Shannon Jessee, from the Linux Foundation. If you did not receive that email, and you'd like to be on the committee, please let me know. We pulled everyone who raised their hand to help from this discussion, as well as the form, but if someone was unintentionally missed, we want to get you added and ensure everyone who wants to participate can! Thanks all!!

Closing as reviewers have been identified and given access to review site. Please reopen if this was a mistake


Feature request: Make reconnection logic configureable

First of all thanks for the great work here!
We have a use-case where the client can be disconnected from the network for more than 60 seconds.
In this case the livekit client gives up reconnecting and stops all tracks.
Instead of hardcoding it here: https://github.com/livekit/client-sdk-js/blob/a492d213ac9c2e20caaf7147cb4930c65bff839b/src/room/RTCEngine.ts#L28
I would like to be able to configure it myself & specify if the tracks should be stopped.
Example how SignalR handles that.
I'm looking for something like this
export class MyReconnectPolicy implements IRetryPolicy {
   public nextRetryDelayInMilliseconds(retryContext: RetryContext): number | null {
      if (retryContext.previousRetryCount > 10) {
         retryContext.stopTracks = false;
         return null;
      }

      return retryContext.previousRetryCount * 1000;
   }
}

this.room = new Room({
 reconnectPolicy: new MyReconnectPolicy()
});

yeah, this is a great suggestion! we could provide a few common reconnect policies in the SDK while making it easy for users to override that behavior.
are you interested in creating a PR for this?


refactor: Update links to organization discussions in issue templates

Update the links to organization discussions in the issue templates for feature requests and ideas as well as for bug reports. The new links now point to the organization discussions instead of the upptime repo discussions which don't exist.

Thanks!


Add more debugging information to error messages

File: engine/reference/commandline/image_build.md
Add information about the path and url parameters.
I have discovered (empirically) that if my requirements.txt file is in my current directory, then the command
sudo docker image build . will build the image.  I have not yet figured out what the url parameter does.
This might not be the best place to note this, and if not, then please: let me know where it should go.  The error message failed to dial gRPC: cannot connect to the Docker daemon. Is 'docker daemon' running on this host?: dial unix /var/snap/microk8s/current/docker.sock: connect: no such file or directory has several possible errors including that the daemon is not running.  However, docker might need to run as root with sudo or there might a protection issue with the socket that the docker client is connecting to.  These kinds of problems could (should) be checked programmatically, e.g. sudo ss -a -p | fgrep docker can verify that the docker server is running.  I have looked over the internet and I find lots of different ways of troubleshooting.

Thanks for sharing the issue.
It looks like the error message is generated from the Engine. If the issue is still applicable, please help create an issue at https://github.com/moby/moby/issues


Tracks download support

Planned, shouldn't be much work. I need to do playlists as well, though.

Tracks and user playlists done. I'll keep this issue open for a bit just in case.
Sorry for the wait, I only just started working on it today.


Document redirectRules in the README.

Added in 04215a5ce6c727ced2b8572e4b4febacf146e0be, this should be documented like the other fields in the manifest.

Done in https://github.com/timonus/OpenerManifest/commit/6600dd02ec04de0e57f762429f2c03c5e52e5c5c.


bug(python): failed to infer column name from the schema

LanceDB version
v0.13.0
What happened?
Query except FTS requires vector column name to be passed in.
Failed to infer vector column at
https://github.com/lancedb/lancedb/blob/main/python/python/lancedb/table.py#L1635
the vector_column_name is None without throwing exception.
vector_column_name = inf_vector_column_query(self.schema)

Proposal

early error out if failed to infer vector column name from schema
check why schema does not have vector column

Are there known steps to reproduce?

E   TypeError: expected bytes, NoneType found

schema.get_field_index(column)  throws exception
pyarrow schema
'inner_id: int64, has_vector: bool, vector: fixed_size_list<item: double>[384], item_id: int64, category: string, numeric_int: int64, numeric_float: double, category_set: list<item: string>'

column is None
(Pdb) pp nearest
{'column': None,
 'k': 3,
 'metric': 'L2',
 'nprobes': 20,
 'q': <vector_values>,
 'refine_factor': None
}

After pass in search(query, vector_column_name), the test passed.

related: https://github.com/lancedb/lancedb/pull/1575
the tests in the PR all supply the vector_column_name argument, which seems to be a functional temporary workaround


Support SPDY and HTTP/2 with the C++ HTTP server

I don't even know if that's possible with HTTP/2. I keep reading articles here and there that HTTP/2 never added back the "Upgrade" header.

Maybe it is only SPDY then. I get all scrambled up headers when using SSL, they need to get decompressed to make sense.

I'm pretty sure it's the same for SPDY, considering it was essentially HTTP/2 pre-alpha. http://stackoverflow.com/a/12103981

AH! I was trying to connect SSL to a non-SSL server and got all scrambled up headers so I thought it had to be SPDY stuff. I simply couldn't get Chrome to connect via SSL. So yeah, this SPDY thing is not really needed at all - it works now that I actually enabled SSL again hahaha

Ueah, afaik WebSockets don't travel over HTTP2. A real shame that.

I connected SSL to a non-SSL server and since SPDY is part of the SSL negotiation process, integrated into OpenSSL kind of, I got scrambled binary with holes of text where it said "spdy/3.1" or something so I got confused :wink:


Revert ambient capabilities for 1.13

There is currently no way to support backward compatible behaviour  with containers that use suid binaries such as sudo while also supporting ambient capabilities.
Plan to fix this for 1.14 as it requires runc changes to support a compatibility mode, and the freeze for 1.13 is rather soon...
Reopens #8460
Includes based version of #27634 from 1.12 branch for tests

Errors are due to the runc version I vendored apparently having an incorrect fix for anotehr issue, will look into that...

@justincormack is this good to be reviewed again?

Now https://github.com/opencontainers/runc/pull/1172 is merged I will update

@justincormack shouldn't we vendor same version of runc in docker? Just for the case.

Needs squashing, but just going to run tests first

@LK4D4 done
@crosbymichael yes, good to go now.

LGTM

Amended to use docker.runc not opencontainers/runc so we can branch for patch releases more easily.

LGTM

Opened a PR to revert docs changes in https://github.com/docker/docker/pull/28453

Thanks, on mg todo list.
On 15 Nov 2016 21:08, "Sebastiaan van Stijn"<EMAIL_ADDRESS>wrote:

Opened a PR to revert docs changes in #28453
https://github.com/docker/docker/pull/28453
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/docker/docker/pull/27737#issuecomment-260768579, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAdcPAvgP4EVSefdWnj-tOYGDzWM_8Wdks5q-h9cgaJpZM4Kf-oz
.


Create Tags.html

pls merge my prs

Follow the CONTRIBUTING.md file before putting any unnecessary PR.


File exclusion and inclusion are behaving differently in search with same regex.

VSCode Version:1.1.0-alpha
OS Version: Windows10

Setup:
My Directory structure for CSS file.
{$workspaceRoot}\timer-styles1.css
{$workspaceRoot}\Scripts\timer-styles3.css
{$workspaceRoot}\Contents\app.css
Steps to Reproduce:

Launch app and open any folder which contains .CSS and .ts file with keyword "width".
Perform search for keyword: width.

Include .css file with regex: *.css
-Exclude .css file with regex: *.css

Verify search result in both the conditions.

Actual: File exclusion is only excluding .css file which is located at the root dir (in my case {$workspaceRoot}\timer-styles1.css)  whereas file inclusion is displaying result for all the available css file (root dir as well as sub directories)
Expected: File include and exclude filter should work uniformly for same regex.

Make sure you uncheck the "use glob pattern" toggle that you have enabled. This changes the pattern.


Taskcontroller logic (EXPOSUREAPP-2743)

Replacing #1345

Hi, looks great.
I have a question: How would future debugging look like, if a task is buggy or runs into a crash? With transactions it was more or less easy, if a stack trace contained the clear name of a crashed transaction (from non-minified code). In the current implementation there is the random UUID only to identify a buggy task. I wonder if there is a benefit if a task could additionally be registered with a clear name, especially when many tasks are in the pipe or are running in parallel (if supported later). Or did I miss something?


Project seems dead ... what fork to use instead?

No activity on Pull requests for years and no releases pushed to PyPI recently, and code is completely broken on recent versions of Django.  Please comment here if you know of a fork or drop-in replacement.  Thanks!

For a drop in replacement, consider https://github.com/inducer/django-bootstrap3-datetimepicker. It's a compatible fork with several js and python changes for recent compatibility. It's also available in this repo as PR #32 .
For slightly different functionality, you could consider this incompatible fork: https://github.com/tutorcruncher/django-bootstrap3-datetimepicker . That fork is on PyPI as django-bootstrap3-datetimepicker-2 . It works a bit differently, but is up to date with both the latest Django and bootstrap datetime picker.


Select File button should be in the Dark in the Dark theme.

Terms

[X] Before you openning this issue, i have checked if the issue has already been reported.

Bagisto Version(s) affected
master
Issue Description
Select File button should be in the Dark in the Dark theme.
Preconditions

Admin should be logged in.

Steps to reproduce

Go to 'Admin Panel'
Click on 'Design' inside the Configurations/Design
See the error

Expected Result
It should be a show button in the dark theme.
Actual Result
Not showing button dark in the dark theme. Here
Screenshots
No response
Additional context
No response

@Amitk-webkul can you assign this issue to me , as the last issue #8671 that you assigned me is already solved.

Hi @sayyedarib, I have assigned this issue. Please continue with this issue.

I couldn't find this choose file button anywhere. Please help me to figure out how to get this button?

@sayyedarib, Please take a reference from here.

Hi @sayyedarib, Is there any update regarding this issue?

@sayyedarib, No problem at all. Thank you for dedicating your previous time to attempting to find a solution.

https://developer.mozilla.org/en-US/docs/Web/CSS/::file-selector-button
This is a new selector that can be used to style the file selector button. It has full support on recent browser versions.
@sayyedarib Take this example at https://codepen.io/spetsnaz/pen/bGzGWbO and try to fix the issue 🤝

Hey there I am willing to contribute to this issue can you assign this to me :)

@prajjwalkumar17 Sure, I'm assigning this issue to you.

Thankyou I have started working on it :)

Checked and Found that the issue has been fixed and PR has been merged. Hence I am closing the issue from here.
Please check the screenshot here


Update access_policies_access-advisor-view-data.md

I noted that this URL and others point to AWS docs with a message stating: AWS CLI version 2, the latest major version of AWS CLI, is now stable and recommended for general use. To view this page for the AWS CLI version 2, click here. For more information see the AWS CLI version 2 installation instructions and migration guide.
if this is true then the URL should probably point to the new docs, and the other URLs in this doc should be updated. I did note though that the original URL points to /latest, so maybe this should be updated elsewhere in AWS docs and this PR closed instead?
Issue #, if available: not available
Description of changes: see above
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Thanks for the feedback. I've passed along your feedback to the CLI Documentation team. For now, pointing to CLI V1 URLs is the recommended approach.


add save/update/delete usage for @Table annotations

Coverage remained the same at 67.565% when pulling 784b3da6b7bc7f2a8692780e58c9166f2ac0c865 on redgetan:master into 1ca7267e17887b60f9e8218dfaa2609eaf6f4d1e on satyan:master.

thanks


Where do I add the Google Maps API

js:35 Google Maps API error: ApiNotActivatedMapError https://developers.google.com/maps/documentation/javascript/error-messages#api-not-activated-map-error
I am using with /react-google-maps https://github.com/kriasoft/react-starter-kit

As the error message you are getting suggests, this doesn't look related to react-google-maps or React; you just need to enable the Google Maps Javascript API.
You can do that for your project by navigating https://console.developers.google.com/apis. Are you trying to pass an API key when loading the script? You should create a new one just for that project, and pass that along.

It looks like the Google account and project you are using doesn't have Google maps activated.
You need to set it up from Google console.
Maybe this link can help you setting it up.

@jmagrippis sorry I saw your comment just after posting mine :)

Then where do you need to pass your Google Maps API key to react-google-maps? Is it even supported to pass an API key? I guess we should add instructions for this in the README ;)

@ClementParis016 You would add it where you import the script for the Google maps API, so if you just include it synchronously, you'd do something like:
<script src="https://maps.googleapis.com/maps/api/js?key=YOUR_API_KEY"   type="text/javascript"></script>
If you are using the ScriptjsLoader to load the script asynchronously, you can sorta derive from the async example that you should pass it in its "query" property. Something like:
<ScriptjsLoader
  hostname={"maps.googleapis.com"}
  pathname={"/maps/api/js"}
  query={{ key: 'YOUR_API_KEY', libraries: 'geometry,drawing,places' }}
  //...

I do feel including that script is something a lot of people using this library might struggle with, or be neglecting to add their own keys, so I do agree it'd be helpful to add a section to the README.

@jmagrippis that's exactly the explanation I was looking for, thanks!
+1 for adding this to the README, I could write a PR once I've set this up for my project

@ClementParis016 @jmagrippis feel free to make a pull request for the README. I think this issue can be closed.

https://github.com/tomchentw/react-google-maps/pull/284 new pull request made, please review it.

Could you still use <script src="https://maps.googleapis.com/maps/api/js?key=YOUR_API_KEY" type="text/javascript"></script> if you are using withGoogleMap and supplying prop googleMapURL? Just wondering fig readme could be updated to reflect the correct API_KEY usage., it would save a lot of developers' time.

The readme is very confusing and incomplete, I reverted to using Google-map-react it's a much simpler API, this one is badly architected and confusing, to say the least.
I mean lines like this: const MyMapComponent = withScriptjs(withGoogleMap((props) are cringe..
It should just be a simple component that handles the initializations for you (optionally of course).
Ah well 👎


Improving code coverage collection and general usability

The purpose of this pull request is to make some minor improvements to how Modelsim is executed. With this work the simulation flow is changed such that vsim will source a do-file when started in order to control execution. The behavior of the do-file is customized by setting variables. The do-file includes a function reload that can be used to reload the simulation image.
The work simplifies how vsim is invoked as the commands related to gathering and storing coverage data are now in the do-file instead of as command-line parameters.

Indeed the big problem with the initial implementation was that I would create the DO file in the QuestaCmd function causing the Ninja build file to be regenerated whenever a simple parameter was changed. I was trying to come up with a more elegant solution that would sort of combine the best of both worlds: static Ninja build files (regardless of options used with the test and run commands), but still with the ability to customize the behavior.
I'm sure I haven't really succeeded yet, but it's a step on the way.


Image path not resolved if used with the basePath property

Hi and congrats for taking the initiative of this package!
I'm often facing use-case where I have to deploy static pages under a subfolder in a shared hosting.
Unfortunately it doesn't work with the needed next.conf.js basePath property, to resolve asset. The built path doesn't take that into account.
I would like to help but I'm not sure being skilled enough for this 🤭.
Tell me if you have recommandation regarding this bug or needed contribution, I'm really interested to get this package working.

Hi @lansolo99, glad you like the package!
Regarding your problem:
Let's say you are using the standard out directory. Currently, all images in public/images will be copied to out/images. If your basePath is "example", would it be sufficient if the optimized images get copied to out/example/images?
(You could try it by manually creating this directory after the Next.js export command and the next-image-export-optimizer command.)

@Niels-IO Ok my fault, after digging a bit, I realized it was because I omitted to use the prop trailingSlash to true within next.config.js. It works like a charm then, you can consider it resolved 🙂.
By the way, I also noted 2 things:

There is no TS file declaration for the package, as I think most of people work with it nowadays.
The storePicturesInWEBPset to false still produce webp files (for a jpg file), not a big deal but weird

Hi @lansolo99,
Thanks for all the feedback! Glad it is now working for you. :)
The storePicturesInWEBP flag actually had a bug, and it is resolved now. Thanks for noting it!
Regarding TS: to rewrite it from JS to TS would be the next step for this package. To be honest, I have to get more into TS first. :)

Hi, this package is great and it helps me with my images, my problem is my basePath wherein if I try to go to the url of my basePath it gives me a 404 error. also I have a lot of images on multiple components per page so is there a way to set the basePath dynamically?
@Niels-IO


Error out when management kubeconfig is not present for workload cluster operations

When management cluster kubeconfig is not set during workload cluster create/upgrade, eks-a doesn't fail and incorrectly sets management cluster to same as workload cluster when clusterspec.ManagementCluster value is nil which is determined based on kubeconfig file being explicitly passed or set as env variable. This change tries to additionally find the kubeconfig file locally if the env variable is not set for kubeconfig and errors out when not able to find any valid kubeconfig file.
Issue #, if available:
Description of changes:
Testing (if applicable):
Documentation added/planned (if applicable):
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve

/cherrypick release-0.17


The option to "Show all audios and subtitles" does not work.

As it says in the title of this post, the option to show audios and subs is not working, I am trying to get Netflix to show me all available audios and subs but it does not, can you solve it please? I will be very grateful.

This option will be removed


LIHADOOP-19609: Make Spark Event Log Directory configurable

Make Spark Event log directory configurable
Fix a bug: When Spark log size was configured (in MB), it was converted to bytes twice.

+1

Merged!


Sub-menu sidebar

How to add sub menus in the sidebar?

Hello there @jdk1997 ,
Thank you for your interest in working with our products.
This is a PRO feature, and cannot be added in the FREE product. I am sorry about this.
Here is a link for the PRO product:

Buy link
Live preview link

Best,
Manu


unable to collect mobs in buckets

Making an issue as your advice with the lang file didn't work.
Let me know what logs ect you require, i  haven't edited my config so not sure what the issue is being caused by

I haven't forgot about this ;)
My PC got some malware last week so I've shut it down until I can reformat it.


Cannot create a string longer than 0x1fffffe8 characters

Description
The main problem occurs during bundling in android OS
After running npx react-native bundle --platform android --dev false --entry-file index.js --bundle-output android/app/src/main/assets/index.android.bundle, process exited with thrown error Failed to construct transformer: Error: Cannot create a string longer than 0x1fffffe8 characters - screenshot -

Have included kotlinVersion = "1.6.0" in android/build.gradle but still not working
Please help !!!
Version
0.66.5
Output of npx react-native info
System:
OS: Windows 10 10.0.19044
CPU: (4) x64 Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz
Memory: 2.53 GB / 7.89 GB
Binaries:
Node: 16.15.0 - C:\Program Files\nodejs\node.EXE
Yarn: Not Found
npm: 8.5.5 - C:\Program Files\nodejs\npm.CMD
Watchman: Not Found
SDKs:
Android SDK: Not Found
Windows SDK: Not Found
IDEs:
Android Studio: Version     2<IP_ADDRESS> AI-211.7628.21.2111.8309675
Visual Studio: Not Found
Languages:
Java: 18.0.1 - C:\Program Files\Common Files\Oracle\Java\javapath\javac.EXE
npmPackages:
@react-native-community/cli: Not Found
react: ^17.0.1 => 17.0.1
react-native: ^0.66.5 => 0.66.5
react-native-windows: Not Found
npmGlobalPackages:
react-native: Not Found
Steps to reproduce
npx react-native bundle --platform android --dev false --entry-file index.js --bundle-output android/app/src/main/assets/index.android.bundle
Snack, code example, screenshot, or link to a repository

Running into the same issue, did you ever figure out a solution or workaround? @anirbanhazra

I ran into this issue after editing my assets (icons, splash screens etc)

Same here. I have created android flavors with different resources including splash. Getting exact same issue.

same issue here,
When I run detox build IOS or Android, I got this issue

After creating a bundle I ran into this error.
I fixed it by deleting the base.zip file it created inside the folder:
android/app/build/intermediates/module_bundle/release


Open Issue 19: UDAP

Note: Support for HL7 Security for Scalable Registration, Authentication, and Authorization (aka UDAP)  when it gets published

UDAP is still not published. so will leave this as an open issue


Stacked bar chart on barclick not working.

Hi Guys, I have used Stacked bar chart into my project I want to get clicked bar position.
@Override     public boolean onTouchEvent(MotionEvent event) {         if (event.getAction() == MotionEvent.ACTION_DOWN) {             performClick();             return true;         } else {             return false;         }     }
from this ovveride method not getting the expected result.

I am facing same issue please provide any solution.


wait for deployments in perf test

wait for deployments to complete in perf test

@rkpagadala: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/daily-perf-tests.sh
32db2d542f8f72ae998a062de134bcb7fa70c22c
link
/test daily-perf-tests

prow/daily-unit-tests.sh
80dde02464a1dd6a7d7d1a8182fbd06e05ac86a0
link
/test daily-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rkpagadala: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/daily-e2e-rbac-no_auth.sh
80dde02464a1dd6a7d7d1a8182fbd06e05ac86a0
link
/test daily-e2e-rbac-no_auth

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


bison: only run install check

While reviewing #180825, I noticed that many tests were being ran twice.

Install check appears to run
the normal check target as well.
This introduces a scenario where
a large test suite is being ran twice.

Also hurts a lot since the test suite isn't thread safe, so I have to run it on a single core to avoid flakiness.
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Also hurts a lot since the test suite isn't thread safe, so I have to run it on a single core to avoid flakiness.

I run bison tests on -j16 and with a shuffle 2-3 times a day every day. I never saw any flakes on bison. Looking at CPU usage they are visibly parallel in both checkPhase and installCheck. Do you have an example build log for parallel failures?
Given that the tests take a lot more time that the build itself another option would be to move to doCheck = false; by default and use passthru.tests = { check = bison.override { doCheck = true; }; };.

Also hurts a lot since the test suite isn't thread safe, so I have to run it on a single core to avoid flakiness.

I run bison tests on -j16 and with a shuffle 2-3 times a day every day. I never saw any flakes on bison. Looking at CPU usage they are visibly parallel in both checkPhase and installCheck. Do you have an example build log for parallel failures?
Given that the tests take a lot more time that the build itself another option would be to move to doCheck = false; by default and use passthru.tests = { check = bison.override { doCheck = true; }; };.

It was 435: glr.cc %union { float fval; int ival; };        ok which failed twice in a row for me. I can't seem to reproduce it even with 128 threads. but it seemed to always pass if I did cores 1. It could also be that I was doing a large number of other jobs at the time (e.g. ~40 jobs, 32 cores each), so there may have been other issues

Also hurts a lot since the test suite isn't thread safe, so I have to run it on a single core to avoid flakiness.

I run bison tests on -j16 and with a shuffle 2-3 times a day every day. I never saw any flakes on bison. Looking at CPU usage they are visibly parallel in both checkPhase and installCheck. Do you have an example build log for parallel failures?
Given that the tests take a lot more time that the build itself another option would be to move to doCheck = false; by default and use passthru.tests = { check = bison.override { doCheck = true; }; };.

It was 435: glr.cc %union { float fval; int ival; }; ok which failed twice in a row for me. I can't seem to reproduce it even with 128 threads. but it seemed to always pass if I did cores 1. It could also be that I was doing a large number of other jobs at the time (e.g. ~40 jobs, 32 cores each), so there may have been other issues

Looking at individual processes bison uses up to 4GB RAM per test (quite a bit). Could it be it OOMed for you?


Incorrect .npmignore generation

Coming from https://github.com/ventojs/vento/issues/48
dnt adds the src/ folder to the .npmignore file, to don't include this folder in the package.
The problem is this matches also with esm/src/ folder. It should include a slash at the begining (/src/, to make sure only the root src folder is ignored.
Probably all paths in this file should start with slash, to prevent other issues like that.

I'll take a look into this now because I'm goign to do a release soon.


Randomness appears panic

Randomness appears panic
What version of Go are you using (go version)?

$ go version
go version go1.17.7 linux/amd64

Does this issue reproduce with the latest release?
just run in 1.17.7
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
**os：**
Linux host-6891d0637088 4.18.0-6.8.0 #1 SMP Tue May 31 06:06:47 CST 2022 x86_64 GNU/Linux
cpu：
processor       : 23
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
stepping        : 2
microcode       : 0x2b
cpu MHz         : 2600.163
cache size      : 15360 KB
physical id     : 1
siblings        : 12
core id         : 5
cpu cores       : 6
apicid          : 27
initial apicid  : 27
fpu             : yes
fpu_exception   : yes
cpuid level     : 15
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bt                                  s rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c r                                  drand lahf_lm abm cpuid_fault epb invpcid_single intel_ppin tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida ara                                  t pln pts
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips        : 4799.90
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

code：
    8: func Recover(cleanups ...func()) {
     9:         for _, cleanup := range cleanups {
=>  10:                 cleanup()
    11:         }
    12:
   13:         if p := recover(); p != nil {
   14:                 logx.ErrorStack(p)
   15:         }

    recover.go:10   0x825acb        488b1a          mov rbx, qword ptr [rdx]
    recover.go:10   0x825ace        ffd3            call rbx

Recover use example：
defer Recover（func(){...}）
panic occur in line no 10
What did you do?

What did you expect to see?
no panic
What did you see instead?
panic

Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.
For questions please refer to https://github.com/golang/go/wiki/Questions


Create hook for user removal, delete removed users from mailchimp list.

As per issue.
This change crate a new hook "onRemoveUserAsync".

Ok have now added in user config settings and a hook for when the user config changes. New hooks are:
onRemoveUserAsync
and
onUpdateUserAsync

Oh nice but can please keep the names format as used in telescope docs
userAccountRemoved
and
profileUpdateSync
Thanks
if we keep it like that we can fix and improve plenty of things

I don't see these hooks in the docs. I can only find this list of hooks, are there more somewhere else?

Thanks for the great work. I'll probably wait until the next release to merge this in though, just to avoid having even more stuff change at the same time. Hope that's ok?
And yeah, we should have a standard syntax for defining hooks names. onRemoveUserAsync is not a bad name, but it's true it doesn't really follow the same pattern as the other ones. If we wanted to be consistent, it would be userDeleteAsync. I'm happy to discuss this though.

Ha sure I was just blatting issues.
cool yea that is a better name for it. I don't mind what the name is just nice if there is a scheme. I'll update to your suggested names.
What for the update hook? userUpdateAsync?

maybe userEditAsync?


Upgrade to 5.5

We should upgrade to the latest 5.5 wordpress version

There is currently no instructions to upgrade the WordPress version. Something should be written in the README.


proberta_model.py missing from modeling directory

The code in train.py wants to make use of proberta_model (in src/modeling), proberta_model.py doesn't currently exist in the current repo.

Hi, sorry for the late reply. proberta_model.py was added to this repo by mistake, sorry for that. No, we've updated train.py so everything should work now. Thanks for noticing!


Review sourcemaps

The sourcemaps need to be checked in dev and the output for production that gets uploaded to sentry also.

These work as expected!


Enable debug logging for datastore

This is turned on when you set verbose to true in your AppScalefile

https://ocd.appscale.com:8080/job/Daily Build/837/

https://ocd.appscale.com:8080/job/Daily Build/843/


GET call to POST call

We currently make a GET call thats works fine with small query, but the issue is when the query is too long its not able to handle. So i want to change it to a POST call on my code side but i am not sure how it works in PostgREST, can someone help me with how to do that. Thanks for your help in advance.

Hey @Pavan-anonymous,
Can you share the request you're doing? Does it have a lot of filters or an in filter with lots of values?

@steve-chavez Thanks for the response. I cant share the request but yes it has two 'in' filters and one among them has values ranging from any where between 1 - 3000 (and each value about 15 characters). Can you suggest on this

@Pavan-anonymous I see. In that case you could convert the GET to POST with a function. It should be like:
create or replace function projects_in(vals int[]) returns setof projects as $$
  select * from projects
  where id = any(vals);
$$ language sql;

POST /rpc/projects_in # you can add any of the GET filters to POST "?id=eq.2&select=id,name,clients(*)"

{"vals": [1,2,3,4,5,6,7,8,9,10,....]} # any size of IN filters here

Thanks @steve-chavez

@Pavan-anonymous It should be like this:
create or replace function sales_locations_in(vals text[]) returns setof sales as $$
  select * from sales
  where locations = any(vals);
$$ language sql;

Then you can use POST for the locations:
POST /rpc/sales_locations_in?practice=in.(practice_type)

{"vals": ["florida/Falachua", "florida/baker", "etc"]}

The upcoming HTTP QUERY method could help us with this issue.


nhttp with react ssr refreshes pages constantly.

nhttp with react ssr refreshes the pages constantly.

yes, it uses HR not HMR.


Build is failing on GHC 7.10.1

Configuring wreq-<IP_ADDRESS>...
Building wreq-<IP_ADDRESS>...
Preprocessing library wreq-<IP_ADDRESS>...
[ 1 of 14] Compiling Network.Wreq.Lens.Machinery ( Network/Wreq/Lens/Machinery.hs, dist/build/Network/Wreq/Lens/Machinery.o )
[ 2 of 14] Compiling Paths_wreq       ( dist/build/autogen/Paths_wreq.hs, dist/build/Paths_wreq.o )
[ 3 of 14] Compiling Network.Wreq.Cache.Store ( Network/Wreq/Cache/Store.hs, dist/build/Network/Wreq/Cache/Store.o )
[ 4 of 14] Compiling Network.Wreq.Internal.Types ( Network/Wreq/Internal/Types.hs, dist/build/Network/Wreq/Internal/Types.o )
Network/Wreq/Internal/Types.hs:47:1: Warning:
The import of ‘mconcat’ from module ‘Data.Monoid’ is redundant
[ 5 of 14] Compiling Network.Wreq.Internal.Lens ( Network/Wreq/Internal/Lens.hs, dist/build/Network/Wreq/Internal/Lens.o )
[ 6 of 14] Compiling Network.Wreq.Internal.AWS ( Network/Wreq/Internal/AWS.hs, dist/build/Network/Wreq/Internal/AWS.o )
Network/Wreq/Internal/AWS.hs:130:42:
Couldn't match expected type ‘time-<IP_ADDRESS>:Data.Time.Format.Locale.TimeLocale’
with actual type ‘System.Locale.TimeLocale’
NB: ‘time-<IP_ADDRESS>:Data.Time.Format.Locale.TimeLocale’
is defined in ‘Data.Time.Format.Locale’ in package ‘time-<IP_ADDRESS>’
‘System.Locale.TimeLocale’
is defined in ‘System.Locale’ in package ‘old-locale-<IP_ADDRESS>’
In the first argument of ‘formatTime’, namely ‘defaultTimeLocale’
In the first argument of ‘(.)’, namely
‘formatTime defaultTimeLocale "%Y%m%dT%H%M%SZ"’
cabal: Error: some packages failed to install:
wreq-<IP_ADDRESS> failed during the building phase. The exception was:
ExitFailure 1

The hackage version also fails for me, but the latest github version now seems to build fine.

So there is only need to upload a new version on hackage.

also fails similarly on 7.8.3

Pinging @bos , any preview when version 0.4 will be in hackage?
Thanks

A hackage update would nice, so I can update my libraries than depend on it.

We have the same problem in NixOS: http://hydra.cryp.to/build/785343/nixlog/1/raw.

Did @bos upload this?

Still can not install. here is my shelllog:
Preprocessing library wreq-<IP_ADDRESS>...
[ 1 of 14] Compiling Network.Wreq.Lens.Machinery ( Network/Wreq/Lens/Machinery.hs, dist/build/Network/Wreq/Lens/Machinery.o )
[ 2 of 14] Compiling Paths_wreq       ( dist/build/autogen/Paths_wreq.hs, dist/build/Paths_wreq.o )
[ 3 of 14] Compiling Network.Wreq.Cache.Store ( Network/Wreq/Cache/Store.hs, dist/build/Network/Wreq/Cache/Store.o )
[ 4 of 14] Compiling Network.Wreq.Internal.Types ( Network/Wreq/Internal/Types.hs, dist/build/Network/Wreq/Internal/Types.o )

Network/Wreq/Internal/Types.hs:47:1: Warning:
    The import of ‘mconcat’ from module ‘Data.Monoid’ is redundant
[ 5 of 14] Compiling Network.Wreq.Internal.Lens ( Network/Wreq/Internal/Lens.hs, dist/build/Network/Wreq/Internal/Lens.o )
ld: warning: directory not found for option '-L/private/tmp/ghc20150401-90656-rxy9sj/ghc-7.10.1/gmp-static'
[ 6 of 14] Compiling Network.Wreq.Internal.AWS ( Network/Wreq/Internal/AWS.hs, dist/build/Network/Wreq/Internal/AWS.o )

Network/Wreq/Internal/AWS.hs:130:42:
    Couldn't match expected type ‘time-<IP_ADDRESS>:Data.Time.Format.Locale.TimeLocale’
                with actual type ‘System.Locale.TimeLocale’
    NB: ‘time-<IP_ADDRESS>:Data.Time.Format.Locale.TimeLocale’
          is defined in ‘Data.Time.Format.Locale’ in package ‘time-<IP_ADDRESS>’
        ‘System.Locale.TimeLocale’
          is defined in ‘System.Locale’ in package ‘old-locale-<IP_ADDRESS>’
    In the first argument of ‘formatTime’, namely ‘defaultTimeLocale’
    In the first argument of ‘(.)’, namely
      ‘formatTime defaultTimeLocale "%Y%m%dT%H%M%SZ"’
cabal: Error: some packages failed to install:
wreq-<IP_ADDRESS> failed during the building phase. The exception was:
ExitFailure 1

Dmitrij-Dorofeevs-MacBook-Pro:Books deem0n$ cabal --version
cabal-install version <IP_ADDRESS>
using version <IP_ADDRESS> of the Cabal library
Dmitrij-Dorofeevs-MacBook-Pro:Books deem0n$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.10.1

Dmitrij-Dorofeevs-MacBook-Pro:Books deem0n$ uname -a
Darwin dmitrijfeevsmbp 13.4.0 Darwin Kernel Version 13.4.0: Wed Mar 18 16:20:14 PDT 2015; root:xnu-2422.115.14~1/RELEASE_X86_64 x86_64

@deem0n use wreq-sb instead.

What is this wreq-sb package you're talking about ? According to the hackage page it is maintained by @bos but it appears to just be a fork ?

The actual maintainer is taking his sweet time uploading wreq 0.4 to
hackage. Someone got impatient, and so right now we're using wreq-sb.
On 10 May 2015 at 04:16, Leon Mergen<EMAIL_ADDRESS>wrote:

What is this wreq-sb package you're talking about ? According to the
hackage page it is maintained by @bos https://github.com/bos but it
appears to just be a fork ?
—
Reply to this email directly or view it on GitHub
https://github.com/bos/wreq/issues/61#issuecomment-100625386.

Thanks @bos!
On 10 May 2015 at 12:14, bos<EMAIL_ADDRESS>wrote:

Closed #61 https://github.com/bos/wreq/issues/61.
—
Reply to this email directly or view it on GitHub
https://github.com/bos/wreq/issues/61#event-301232161.


mapboxapi geocoder is used in the app

as mapbox dependency has been removed temporarily, we need another package for geocoding

Next pull request will resolve this issue. mapboxapi has been reinstated.

necessary linux packages added; mapboxapi can now be loaded without issues


Stopping an application pool that is already stopped causes an error

It would be nice if the App Management task could determine the status of an app pool before it tries to start or stop it. The reason is that it tends to cause errors if the state isn't what it expects to see. This causes issues if you are trying to act on errors.

PR - https://github.com/microsoft/azure-pipelines-extensions/pull/615

Closing this issue as PR merged.


Remove base tests from python precommits

GCP and Cython test variants provide the same coverage and also ensure
that test could run with or without GCP or Cython components installed.
Did not remove the actual gradle tasks, only removed from precommit tasks.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
Post-Commit Tests Status (on master branch)

Lang
SDK
Apex
Dataflow
Flink
Gearpump
Samza
Spark

Go

---
---

---
---

Java

Python

---

---
---

XLang
---
---
---

---
---
---

Pre-Commit Tests Status (on master branch)

---
Java
Python
Go
Website

Non-portable

Portable
---

---
---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

@yifanzou do I need to run seed job for these changes to take effect?

No need to run the seed job for this change to take effect. AFAICT, the non job non-gcp job will NOT be running regularly (say, in postcommits) after this change. I assume this is the intent.

No need to run the seed job for this change to take effect. AFAICT, the non job non-gcp job will NOT be running regularly (say, in postcommits) after this change. I assume this is the intent.

Yes, that is correct. I want to make sure that changes to gradle files do not have obvious mistakes and some tests are ran using them. It seems like precommit on this job already ran with this change (https://scans.gradle.com/s/laaik6wiyqrko).


typescript: usePlaybackState has not exported

Describe the bug
I am getting waring (Module '"../../../../node_modules/react-native-track-player"' has no exported member 'useTrackPlayerProgress'.) when I am importing hook like usePlaybackState, useTrackPlayerProgress and  useTrackPlayerEvents.
would you help me how i could fix this problem?
Code
`import TrackPlayer, {
usePlaybackState,
useTrackPlayerProgress,
useTrackPlayerEvents,
} from 'react-native-track-player';`

@itsgracian hey did you find a solution to this? I'm getting it also..

Hello, @TowhidKashem yes. uninstall react-native-track-player and install yarn add react-native-track-player@next

@TowhidKashem  this hooks useTrackPlayerProgress was changed to useProgress

That will install v2 of the library though and I'm not 100% sure but I think that's still in beta (or at least it doesn't seem to me very stable from some comments). I think I'll just deal with the TS error with a ts-ignore ignore comment for now, although it doesn't make sense how it's even working with the renaming but it is.

I'm able to import it by reaching directly into the package's hooks directory:
import { useTrackPlayerProgress } from "react-native-track-player/lib/hooks"

I know the issue is closed, but maybe that will help anyone else who winds up reading this issue.

I'm able to import it by reaching directly into the package's hooks directory:
import { useTrackPlayerProgress } from "react-native-track-player/lib/hooks"

I know the issue is closed, but maybe that will help anyone else who winds up reading this issue.

Hi, would you please release a new version with the fix?

The full path "react-native-track-player/lib/hooks" hook only works in iOS if we directly import then it works in android what to do in such a situation ?


Valid Palindrome

🚀 Feature
Given a string s, determine if it is a palindrome, considering only alphanumeric characters and ignoring cases
Have you read the Contributing Guidelines on Pull Requests?
yes
Pitch
Example 1:
Input: s = "A man, a plan, a canal: Panama"
Output: true
Explanation: "amanaplanacanalpanama" is a palindrome.

@soumik2012 Do you need it in all languages mentioned? Can I contribute this Issue on C, C++, Python language?

It depends upon you @SanthoshS20 if you can solve the question in all the languages.
Yes, you can.

Hello @soumik2012 I can contribute towards validating the palindrome using C++, with proper explanation through comments.
Could you assign the task to me ?


Several Api fixes

Validator tab:
Endpoint - statistics/get_statistics_validators

[ ] Restake enabled (yes,no)
[ ] Validator Rank, according to voting power
[ ] Return list of active set validators
[ ] Additional endpoint Commission Earned as static value
[ ] Voting participation. Currently we have number of proposals, in which  validator took a participation. We need also to show amount of proposals, in which validator could participate. Example: Last proposal is 150. Bro_n_Bro joined Cosmos Hub, when there were already 100 proposals passed, we couldn't participate in them, so we need to count proposals, that we can participate. So, there are should be returned votes such as 45/50.
[ ] Add uptime stats. Static value for 10k blocks and historical average by 500 blocks.
[ ] Add commission earned

Endpoint - /statistics/get_statistics_validators__operator_address_

[ ] Voting participation. Currently we have number of proposals, in which  validator took a participation. We need also to show amount of proposals, in which validator could participate. Example: Last proposal is 150. Bro_n_Bro joined Cosmos Hub, when there were already 100 proposals passed, we couldn't participate in them, so we need to count proposals, that we can participate. So, there are should be returned votes such as 45/50.
[ ] Add uptime stats. Static value for 10k blocks and historical average by 500 blocks.

New endpoint:

[ ] List of our delegators(wallets) with their balances. Input address to get staked amount, the similar as get_statistics_validators__operator_address_. Api should be able to accept array of addresses and return a sum of native tokens.

Endpoint - /statistics/get_statistics_inactive_accounts

[ ] Additional endpoint inactive_accounts as historical value

Endpoint -/statistics/get_statistics_total_accounts

[x] Additional endpoint total_accounts as historical value

Endpoint - statistics/get_statistics_blocks_time

[x] Additional endpoint get_statistics_blocks_time  as historical value

Account tab:
Add additional endpoint with static value.

[x] /statistics/get_statistics_active_accounts
[x] /statistics/get_statistics_new_accounts
[x] /statistics/get_statistics_gas_paid
[x] /statistics/get_statistics_transactions

@bro-n-bro-0
Additional endpoint inactive_accounts as historical value
Please provide sql

Restake enabled (yes,no)
spacebox.exec_message
msgs type message delegate -> delegator address
invalid json, must be fixed in DB before

Validator Rank, according to voting power
from 1 to n based on validator_voting_power

active validators
select * FROM spacebox.validator_status FINAL WHERE status = 3 and jailed = FALSE

validator created - select * FROM spacebox.create_validator_message cvm
proposal created - select * FROM spacebox.proposal submit_type

select * FROM spacebox.validator_precommit vp for last 10000
count of each validator / 10000
for 500: return like true/false for each validator does he sign each block

@bro-n-bro-0
can't find a way to calculate proposals , that were created after validator was created. here is sql example, how i thought it could work
select t.*, (select count(*) from spacebox.proposal as p where p.submit_time > t.timestamp ) as value from ( select * from spacebox.create_validator_message as cvm left join (select * from spacebox.block) as b on cvm.height = b.height) as t

@bro-n-bro-0

Add uptime stats. Static value for 10k blocks
Don't really get it, looks like some validators appears more often then 1 time in each block, let's discuss

@bro-n-bro-0 please help with it, any ideas?
Unique delegators historical data

@bro-n-bro-0
Add commission earned as static
Timeout error


No livereload and no backend styles

          Hee Chris,

In my setup this doesn't seem to work. Maybe this has anything to do with the fact that I'm using 'normal' urls instead of localhost. So by default it's opening localhost:3000 when running npm run dev. But this gives back a dns error.
Originally posted by @bdfpplinsinga in https://github.com/cearls/timberland/issues/39#issuecomment-2097097177

Hey, the Vite dev server runs at localhost:3000 when running npm run dev, but you wouldn't use it to preview the site. The WordPress theme in development mode pulls in assets served from localhost:3000. I use local URLs like mysite.test.
You can see how those assets are used when in development mode:
https://github.com/cearls/timberland/blob/main/theme/functions.php#L102-L108

Hee Chris,
Indeed on the front end all is working, but i cant seem to get the backend working. I tried to add these vite script links to the admin_head with
add_action('admin_head', 'vite_head_module_hook');
``` but this doesnt seem to work. The only thing that works is running a build command.

Am i missing something? Or aren't you using the vissual backend on development?

I have been running the build command to preview in the admin. I typically develop templates on the frontend and the backend comes later as I split the templates into reusable blocks.
Let me give this some thought as I can see how it would be annoying since that's how the theme used to work prior to Vite.


[SPARK-28945][SPARK-29037][CORE][SQL] Fix the issue that spark gives duplicate result and support concurrent file source write operations write to different partitions in the same table.

What changes were proposed in this pull request?
For InsertIntoHadoopFsRelation operations.
Case A:
Application appA insert overwrite table table_a with static partition overwrite.
But it was killed when committing tasks, because one task is hang.
And parts of its committed tasks output is kept under /path/table_a/_temporary/0/.
Then we rerun appA. It will reuse the staging dir  /path/table_a/_temporary/0/.
It executes successfully.
But it also commit the data reminded by killed application to destination dir.
Case B:
Application appA insert overwrite table table_a(non-partitioned table).
Application appB insert overwrite table table_a, too.
They execute concurrently, and they may all use /path/table_a/_temporary/0/ as workPath.
And their result may be corruptted.
In this PR, we set a staging output path for InsertIntoHadoopFsRelation operation.
The output path is a multi level path and is composed of specified partition key-value pairs formatted .spark-staging-${depth}/p1=v1/p2=v2/.../pn=vn.
We can detect the conflict by checking the existence of relative staging output path.
For example:
table_a(c1 string, a string, b string,...) partitioned by (a, b , ...)
When writing to partition a=1, we need to check the existence of

.spark-staging-0
.spark-staging-1/a=1
.spark-staging-2/a=1
...
.spark-staging-numPartitions/a=1

When we found relative staging output path is existed, an exception would be thrown.
Thanks @advancedxy for providing the prototype of solution and valuable suggestions.
Why are the changes needed?
Data may be corrupted without this PR.
Does this PR introduce any user-facing change?
User can see the exists of staging output path and may need clean up these path manually, when this path is belong to a killed application and has not been cleaned up gracefully.
How was this patch tested?
Added unit test.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

How about just clean up the staging dir before write job starts?

ok to test

Test build #111047 has started for PR 25863 at commit fa60fc9.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16135/
Test PASSed.

Test build #111047 has finished for PR 25863 at commit fa60fc9.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111047/
Test FAILed.

@turboFei it seems your description does not reflect changes in this PR.

How about just clean up the staging dir before write job starts?

The staging dir may be a used by another job, so we can not clean up it directly.

Test build #111061 has started for PR 25863 at commit 0b1323d.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16147/
Test PASSed.

Test build #111061 has finished for PR 25863 at commit 0b1323d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111061/
Test FAILed.

Test build #111073 has started for PR 25863 at commit 9fb68e7.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16158/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16159/
Test PASSed.

Test build #111075 has started for PR 25863 at commit eedad23.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16160/
Test PASSed.

Test build #111073 has finished for PR 25863 at commit 9fb68e7.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111073/
Test FAILed.

Test build #111075 has finished for PR 25863 at commit eedad23.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111075/
Test FAILed.

retest this please

Test build #111109 has started for PR 25863 at commit a4f33d9.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16190/
Test PASSed.

Test build #111109 has finished for PR 25863 at commit a4f33d9.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111109/
Test FAILed.

In this PR, for the partition overwrite operation, I set a unique output for that.
I have test the UT failed before locally, they all passed.

retest this please.

Test build #111116 has started for PR 25863 at commit 98aa212.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16196/
Test PASSed.

Test build #111116 has finished for PR 25863 at commit 98aa212.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111116/
Test PASSed.

@cloud-fan
This PR can resolve the duplicate result issue.
In this PR, I set a unique output dir for partition overwrite operation, both dynamic and static partition overwrite.

After this PR, the  InsertOverwrite operation for partitioned table will not reuse _temporary/0.
InsertInto operation for partitioned table and InsertOverwrite/InsertInto for non-partitioned  table will still use _temporary/0 as  job attempt path.

In the new commit.
Not only for partition overwrite operation,  but for all InsertIntoHadoopFsRelation operation, I set a unique output path for it.

Test build #111125 has started for PR 25863 at commit 6ee11b8.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16205/
Test PASSed.

Test build #111125 has finished for PR 25863 at commit 6ee11b8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111125/
Test PASSed.

I  will improve the details of this pr.
Firstly， I will invoke mergePaths by using reflect.
Secondly， I will set a output path with checksum value according to specified partition key values.

I have implement the function to detect conflict when several InsertIntoHadoopFsRelation operations executes concurrently.

retest this please.

Test build #111163 has started for PR 25863 at commit 2567ffe.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16244/
Test PASSed.

Test build #111163 has finished for PR 25863 at commit 2567ffe.

This patch fails to generate documentation.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111163/
Test FAILed.

Test build #111166 has started for PR 25863 at commit e3eb9d6.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16246/
Test PASSed.

Test build #111166 has finished for PR 25863 at commit e3eb9d6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111166/
Test PASSed.

I haven't looked at the code yet, but after reading the PR description, it seems a little contradictory to what we discussed before.
The new proposal here can avoid duplicated data if a write job failed without cleaning the staging dir. What if there are 2 write jobs running at the same time?

@cloud-fan
Sorry, I just updated the PR description.
In this PR, we set a unique staging dir for InsertIntoHadoopFsRelation operation and detect the conflict when there are several insetIntoHadoopFsRelation operations execute concurrently.
The detail implement:
For all InsertIntoHadoopFsReltion operation,  we will transfer three parameters into HadoopMapReduceCommitProtocol.

isInsertIntoHadoopFsRelation
isOverwrite
staticPartitionKVs

We will create a new staging dir related with the parameters above.
.spark-staging-\${jobId}-{overwrite or append}-${staticPartitionKVs.size}
With this new stagingDir, we can detect the operation type, overwrite or append.
We will set a subDir under staging dir as output path.
Its file tree is .spark-staging-\${jobId}-{overwrite or append}-${staticPartitionKVs.size}/appId/sp_pk1=v1/sp_pk2=v2/.../sp_pkn=vn
The appId is used to record the appId of InsertIntoHadoopFsRelation and is useful when we detect a conflict.
And ${staticPartitionKVs.size} is used to find a longest path tree with static prefix, and can prevent be confused when there is a partition key started with static prefix.
As descripted above, we set a outputPath for each insertIntoHadoopFsRelation operation,
After commitJob we just need invoke mergePaths(fs, outputPath, tablePath)  and clean up stagingDir.
About detecting conflict:

Firstly, we need get current operation's outputPath, such as sp_pk1=v1/sp_pk2=v2.

If current operation is an overwrite operation, we need to find the longest path with static prefix of all staging dirs(include overwrite and append staging dirs ).

If current operation is an append operation, we just need to find that of all overwrite staging dirs.

Judge whether current output path contains other existing outputs or be contained.

private def pathIsContained(path1: String, path2: String): Boolean = {
  path1 == "" ||path2 == "" || path1 == path2 ||
    (path1.startsWith(path2) && path1.charAt(path2.length) == '/') ||
    (path2.startsWith(path1) && path2.charAt(path1.length) == '/')
}

If  exists pathIsContained=true, we will throw an exception.

We can get the appId of relative staging and its last modification time. If it run on yarn, wen can get its running duration and judge whether this application is running, if not, we can clean up this staging dir or we can notify that user can clean up it manually.

After commitJob we just need invoke mergePaths(fs, outputPath, tablePath) and clean up stagingDir.

How expensive is it? Previously the output path is table path directly, right?

After commitJob we just need invoke mergePaths(fs, outputPath, tablePath) and clean up stagingDir.

How expensive is it? Previously the output path is table path directly, right?

I have checked the code of mergePaths method in FileOutputCommitter.
When commitJob, mergePaths is invoked to merge the files under taskAttemptPaths.
Its args is fs(fileSystem), from and  to.
If to is null, it just need  rename.
For example, for the sql insert overwrite table ta partition(p1=1,p2=2) select ....
After commitJob to stagingDir, it just need rename stagingDir/appId/staticPrefixPath/p1=1/p2=2 to tablePath/p1=1/p2=2, because tablePath/p1=1/p2=2 has been dropped.
For the sql insert overwrite table ta select ... (ta is a partitioned table).
After commitJob to stagingDir, there will be many partitions under stagingDir/appId/staticPrefixPath/, such as p1=v1, p2=v2, pn=vn.
And we just need to rename these pn=vn to tablePath/pn=vn, which has been dropped before.
In conclusion, for the overwrite operation of partitioned table, its additional cost is  a serial of  directory rename operations.
For the overwrite operation of non-partitioned table, its cost is double.
For the append operation, its cost is double.
In fact, we can skip commit all task output to stagingDir, we can merge the output of all tasks from taskAttemptPaths to tablePath directly.

I'm a little worried about relying on file listing in the commit protocol. AFAIK s3 file listing may not give you the latest file list.
Can we think of a simple approach? e.g. use a fixed-path staging dir and fail the job if the staging dir already exists.

Can we think of a simple approach? e.g. use a fixed-path staging dir and fail the job if the staging dir already exists.

@cloud-fan
We can transfer each specified partitionKey Value to a checkSum, which is fixed-width, such as CRC8.
And name the staging dir .spark-staging-${staticKVs.size}-CRC1-CRC2-...-CRCN.
For example, for the sql insert overwrite table ta partition(p1=v1,p2=v2,p3) select ....
We need to check the existence of:

.spark-staging-0
.spark-staging-1-CRC1
.spark-staging-2-CRC1-CRC2

But we can not check the existence of .spark-staging-3-CRC1-CRC2-CRC3.
Do you have suggestion? @advancedxy

@turboFei this is a good idea. We can make it simpler: spark-staging-2-a=1-b=2. Then we don't need to worry how to encode partition values to a unique string.

how about
.spark-staging-0
.spark-staging-1/a=1
.spark-staging-2/a=1/b=2

When writing to partition a=1, we need to check the existence of

.spark-staging-0
.spark-staging-1/a=1
.spark-staging-2/a=1
...
.spark-staging-numPartitions/a=1

When there are two write can execute currently.
such as  A:Insert overwrite table ta partition(a=1,b=2) select ... and
B:Insert overwrite table ta partition(a=1,b=3) select ... .
The file tree is:
.spark-staging-2/a=1/b=2
                    /b=3

When A is complete,  we should delete b=2 firstly.
Then we need list files under a=1, if there is not another partition path, we should delete a=1.
In this case, B is not completed, we can not delete a=1.
So, in this way, we also rely on file listing in the commit protocol.

how about we call FileSystem#delete(/path/to/staging/dir/a=1, recursive = false)?

Sounds good. I will try to implement it. Thanks

Test build #111268 has started for PR 25863 at commit 88243dc.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16335/
Test PASSed.

Test build #111268 has finished for PR 25863 at commit 88243dc.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111268/
Test FAILed.

I will try to set our own commitJob method to FileOutputCommitter by using reflection to fix the UT issue.

I will try to set our own commitJob method to FileOutputCommitter by using reflection to fix the UT issue.

For the UT issue, there  is sub class of ParquetOutputCommitter, which overrides the commitJob method.
So, I can not pass the UT with our own commitJob method.
I have try to set tablePath to field of committer named  "outputPath", when FileOutputCommitter has been initialed.
But the  getAllCommittedTaskPaths method will use outputPath  to get all committed task paths.
So there is not a perfect solution.
I think we can set mapreduce.fileoutputcommitter.algorithm.version to 2 implicitly for InsertIntoHadoopFsRelation operation, which would commit task output to stagingOutputPath directly.
Then we can merge the output under stagingOutputPath to tablePath.
And the cost is same with previous implementation when mapreduce.fileoutputcommitter.algorithm.version is set to 1.
And it also would not cause partial result.

Test build #111322 has started for PR 25863 at commit dbd6466.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16372/
Test PASSed.

Test build #111322 has finished for PR 25863 at commit dbd6466.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111322/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16375/
Test PASSed.

Test build #111325 has started for PR 25863 at commit 4640861.

Test build #111325 has finished for PR 25863 at commit 4640861.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111325/
Test FAILed.

The failed SQLQueryTestSuite in sql module  passed locally.

Test build #111329 has started for PR 25863 at commit ed2af4a.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16378/
Test PASSed.

Test build #111329 has finished for PR 25863 at commit ed2af4a.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111329/
Test FAILed.

Test build #111335 has started for PR 25863 at commit 30cc2cc.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16383/
Test PASSed.

Test build #111335 has finished for PR 25863 at commit 30cc2cc.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111335/
Test PASSed.

Thanks a lot @advancedxy .
I have updated the comment, code and UT.
cc @cloud-fan

Test build #111350 has started for PR 25863 at commit d130bc8.

Merged build finished. Test PASSed.

Test build #111353 has started for PR 25863 at commit 55c0df3.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16399/
Test PASSed.

Test build #111350 has finished for PR 25863 at commit d130bc8.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class FileSourceWriteDesc(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111350/
Test FAILed.

Add three-args constructor for HadoopMapReduceCommitProtocol to fix the UT.

Test build #111356 has started for PR 25863 at commit 9eebd49.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16402/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16404/
Test PASSed.

Test build #111358 has started for PR 25863 at commit 3914515.

Test build #111353 has finished for PR 25863 at commit 55c0df3.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111353/
Test FAILed.

Test build #111356 has finished for PR 25863 at commit 9eebd49.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111356/
Test FAILed.

Test build #111358 has finished for PR 25863 at commit 3914515.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111358/
Test FAILed.

Test build #111363 has started for PR 25863 at commit 0ae4a30.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16408/
Test PASSed.

Test build #111363 has finished for PR 25863 at commit 0ae4a30.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111363/
Test PASSed.

Test passed.
cc @advancedxy @cloud-fan

Test build #111392 has started for PR 25863 at commit dacd0f8.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16433/
Test PASSed.

Test build #111392 has finished for PR 25863 at commit dacd0f8.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111392/
Test FAILed.

Test build #111408 has started for PR 25863 at commit 6fc8e7b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16447/
Test PASSed.

Test build #111408 has finished for PR 25863 at commit 6fc8e7b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111408/
Test PASSed.

Test build #111435 has started for PR 25863 at commit a048a5b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16471/
Test PASSed.

Test build #111435 has finished for PR 25863 at commit a048a5b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111435/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16501/
Test PASSed.

Test build #111467 has started for PR 25863 at commit 256aceb.

Test build #111467 has finished for PR 25863 at commit 256aceb.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class FileSourceWriteDesc(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111467/
Test FAILed.

Test build #111476 has started for PR 25863 at commit 4f2a804.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16507/
Test PASSed.

Test build #111476 has finished for PR 25863 at commit 4f2a804.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class FileSourceWriteDesc(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111476/
Test FAILed.

Test build #111484 has started for PR 25863 at commit abb7fad.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16512/
Test PASSed.

I have tried make dir before doing this check, and thrown exception when it existed.
But when exception thrown  after that, I can not ensure it would be cleaned up.
And it may cause other job fail.
I will try to find other solution.

Test build #111484 has finished for PR 25863 at commit abb7fad.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111484/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16518/
Test PASSed.

Test build #111492 has started for PR 25863 at commit 4d6c228.

Test build #111492 has finished for PR 25863 at commit 4d6c228.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111492/
Test FAILed.

Test build #111513 has started for PR 25863 at commit f87aeb3.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16533/
Test PASSed.

Test build #111513 has finished for PR 25863 at commit f87aeb3.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111513/
Test FAILed.

Test build #111516 has started for PR 25863 at commit 8098a8f.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16536/
Test PASSed.

Test build #111516 has finished for PR 25863 at commit 8098a8f.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class FileSourceWriteDesc(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111516/
Test PASSed.

refactor the code.

retest this please.

Test build #111569 has started for PR 25863 at commit 81aff54.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16576/
Test PASSed.

Test build #111569 has finished for PR 25863 at commit 81aff54.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class FileSourceWriteDesc(

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/111569/
Test PASSed.

This scares me. The FileOutputCommitter code scares me already. It's why when I added a new committer extension point I chose not to go near that code and instead add a new superclass. It's scary code because

it's co-recursive in a way that hard to follow.
it's used in the MRv1 and V2 APIs
it implements to different commit algorithms in the same methods
there is no documentation other than that code.
one of those commit algorithms, doesn't have the failure semantics spark expects (network partition during task commit)
it's been widely subclassed in ways people aren't aware of.
there's lots of references to static methods which you can't subclass when writing your own committer.

mergePaths() was one of the most obtuse bits of code I've ever had to step through with a debugger and a pen and paper. And I've stepped through how Windows NT boots.
You are proposing using reflection to get at private superclass methods that are not part of the public API, where we aren't even going to be aware of the fact that they are being used this way and have no guarantees of stability whatsoever.
And of course, for anything which implement the new superclass, PathOutputCommitProtocol it's not going to work.
Overall then doing things during commit operations for which it is going to be very hard to prove correctness -and vulnerable two people doing things in the MR codebase we could fundamentally break things, either a link time, or worse, in the semantics of the operations you are trying to perform.
I don't see an easy solution here, but as someone with write access to the MR codebase, I'm happy to discuss how we could make things easier.
For example, someone could actually pull out the mergePath code into its own class, with its own tests which could then be reused. We can also look about whether we could extend PathOutputCommitter to offer more here. However mergePath contains some big assumptions about the nature of the destination and what the commit algorithms do.
In ideal world, how we commit should be opaque to the caller. You commit a task and its work is promoted into the set of committed tasks work, or you don't get a response from the task's process, in which case you can safely commit another task, confident that irrespective of what has gone wrong with the task which failed/partitioned/hung only one tasks output will ever become visible -and after the job has been committed or aborted the output of a partitioned task will never appear. V2 doesn't offer those guarantees, which is why I consider it broken.
One thing I have thought about adding to PathOutputCommitter is a predicate isTaskCommitRepeatable() which would at least tell the spark driver, MR AM etc whether it is safe to attempt to retry a task if it failed what timed out during that commit. the V2 protocol would say false, as should the new EMR spark committer. Not sure if that would help here.

I've looked at the code a bit more. As I noted earlier, this scares me.
in the FileOutputCommitter, the FS itself is synchronization point, with assumptions about atomicity and performance implicitly implemented in the code. The application which use the committer have their own assumptions about atomicity and performance which are derived transitively from those of the file system and then extended by assumptions about the correctness of the algorithms.
Things are bad enough as they are.
I am not convinced that relying on the internals of FileOutputCommitter Versions is the safe way to do this.
I think you better off specifying a commit protocol which is explicitly designed for writing files into the destination, and then implementing it.  For S3A, knowing the destination path lets is initiate but not complete the upload to the final path; we would then propagate the information needed to manifest that file to the job committer. The "Magic" committer does exactly this by recognising that when someone writes to dest/__magic/$job_attemptID/$task-attempt-id/__basepath/year=2019/month=10/day=1/part-0001.csv is to be have a final destination of dest/year=2019/month=10/day=1/part-0001.csv and the output stream, rather than create that file o close(), is only to save the metadata for the task commit to find.
That is very much black magic in the FS connector. Having a commit protol where you could ask the committer for an output path to use when writing to a specific destination we let you eliminate that trick completely and possibly help with this problem.
The other thing to consider is that spark permits committed tasks to pass serialized data back to the driver, as an alternative to relying on the file system. Scale issues notwithstanding, task committers should be able to provide the information needed to include the task in the jobs final output. For example, rather have task commit renaming files, it could just return a path too where it has stored its list of files to commit. Job commit becomes a matter of loading those files and moving the output into the final destination. Again, this is essentially what we do with the S3A committer -we just save, propagate and reload those files within the committer.
This all gets complicated fast -even without worrying about concurrent jobs. But trying to use FileOutputCommitter internals to do what you want to do here strikes me is very very dangerous. I have issues with one of the commit algorithms implemented in it already; trying to use it for other purposes is only going to make things worse.

Thanks @steveloughran
I will try my best to understand the content of  you comments,  thanks a lot.

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/16942/
Test PASSed.

Test build #112119 has started for PR 25863 at commit fac7b7b.

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17114/
Test PASSed.

Test build #112119 has finished for PR 25863 at commit fac7b7b.

This patch passes all tests.
This patch does not merge cleanly.
This patch adds no public classes.

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112119/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19713/
Test PASSed.

Test build #114890 has started for PR 25863 at commit fa66a5b.

Test build #114890 has finished for PR 25863 at commit fa66a5b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114890/
Test FAILed.

Test build #114891 has started for PR 25863 at commit 79d59bd.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19714/
Test PASSed.

Test build #114891 has finished for PR 25863 at commit 79d59bd.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114891/
Test FAILed.

Test build #114901 has started for PR 25863 at commit 0433977.

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19723/
Test PASSed.

Test build #114901 has finished for PR 25863 at commit 0433977.

This patch passes all tests.
This patch does not merge cleanly.
This patch adds no public classes.

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114901/
Test PASSed.

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19864/
Test PASSed.

Test build #115045 has started for PR 25863 at commit f45ca9b.

Test build #115045 has finished for PR 25863 at commit f45ca9b.

This patch passes all tests.
This patch does not merge cleanly.
This patch adds no public classes.

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115045/
Test PASSed.


Implementaiton of functions() in "Services"

I really appreciate the work you've done in the Security and Privacy Guide.
I'm interested in how you implement the "Services" section.
Do you incorporate the functions there via a login script (.profile, .bashrc, etc.)
Or do you run them manually?
When I run them manually, I'm required to authenticate some of the services, which would make login script implementation cumbersome.
Just wondering how you do it, and perhaps adding those instructions to the README.MD.
Thanks again for a great guide!

Hey there,
Sorry it isn't clear. It is just a bash function with calls to it. You can just copy paste the lines into a Terminal window to run them one by one. I'll try and make that understandable in the guide.


no-internal-module raises an error for namespaces with nested paths

Consider the following example:
namespace Example.Utils { }

with no-internal-module set to to true an error is thrown starting at the Utils fragment. This can be applied to an arbitrary number of nested namespace paths, each of which raises an error of its own. A possible workaround is to rework the code-block as:
namespace Example {
    namespace Utils { }
}

If this is the intended way to apply the option it seems to add unnecessary verbosity and nesting to code. If it's not, my best bet is that the module/namespace declaration node is split into 2 or more elements which are visited, the first of which passes the namespace check and the rest end up as the error.

fixed in v2.5.0-dev.5