Having youtube-dl Grab From Source Instead of URL

[X] I've verified and I assure that I'm running youtube-dl 2016.08.31

Before submitting an issue make sure you have:

[X] At least skimmed through README and most notably FAQ and BUGS sections
[X] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[X] Feature request (request for a new functionality)

Description of your issue, suggested solution and other information
This might be the second or third time that I request this feature because it would be so extremely useful to have.
There are tons of reasons to why this would be useful to have, but the main reason is that this way I could have youtube-dl extract video URLs on member restricted pages such as Facebook videos set to private.
YouTube videos are fine as I can use the username and password parameters to log in, but for other sites (like Facebook and Instagram among others) I'm shit out of luck so I have to resort to installing software to sniff the URLs.
If there was just a simple parameter like --get-from-source C:\location\to\website_source.html which then dumped the video links then I might be free from bundled adware at the end of the day.

Duplicate of #5768.

I went through the #5768 and what @jaimeMF wrote doesn't completely make sense and I'll explain why.
Here are the reasons he posted to why this feature could not be added
Reason 1) It would unnecessarily complicate the code, we would need to modify the extractors so that they work both by giving an url and the webpage. (I gess that you are problably only interested on youtube, but we support more sites).
Answer 1) Not really, it would be extremely simple to add as it would only be a matter of adding an if-statement checking if the --get-from-source parameter was set.  If it is set then it doesn't download the page source, but uses the data from the parameter.  Example:
if(get-from-source == true){

   /* Use specified source */

}

else{

   /* Download the page source as usual */

}

The main page source is only downloaded once so this if-statement would also only have to be used once.  Most hosts also don't restrict the direct video link to the video so youtube-dl would have no issue working with the source as a usual request after that point.
Reason 2) What happens if we decide to use another page for getting the info? You would get error messages and we would need to explain what has changed, and this is internal behaviour so we shouldn't have to.
Answer 2) Like I explained in my previous answer, nothing would change other than adding that if-statement so no additional errors would have to be caught.  Everything would be exactly as a normal request only that the user was allowed to input the source.
Reason 3) You would need to specify an extractor, which is another source of bug reports because people would use the wrong extractor and we sometimes split extractors or add a special one for handling some urls.
Answer 3) There's a point here, but it would be very easy to solve as well.  The user would simply add the URL along with the parameter just like any other command.  For example youtube-dl http://web.com/video --get-from-source

Even if it were like in your fancy pseudocode (it's actually not, there are lots of places that rely on URL heavily, lots of complicated codes that will require heavy refactoring) this would require such branching in every extractor for every downloaded source used during extraction process. That's a lot of mess for nothing thus not feasible and senseless to maintain.
Do you even understand the problem? Extraction scenario may involve downloading arbitrary set of webpages/assets/APIs/whatever that is technically an implementation detail end user should not even bother about or be aware of. All of this set items you will have to provide as input to extractor in correct order. But before this extractor should somehow point you out which files from which locations in which order it accepts as input and how to obtain all this stuff. Moreover this set of files may change during time since websites' layouts and structures change leading to extraction scenarios' changes. Each such change will result in different set of input files with different constraints. Needless to say providing such input set may not be even possible at all due to sessioning, authentication, different kind of protections, URL signing, expiry and so on.

I was under the impression that youtube-dl first had the core files download the URL and then passed it on to the specific extractor, since nearly every single extractor downloads the URL anyway (as you pointed out yourself).  This is a flaw in the youtube-dl structure as the same code/process is repeated over every single extractor (hundreds of files), but it's still possible to fix by removing all the requests to the main URL, moving the part to the core files and then handing it over to the extractors through a variable.

To start off, everything would be exactly as it is working now if the parameter wasn't used.  Second off I think you've missed the whole point of how this works or you didn't read my answer properly.

I think we can agree that currently youtube-dl usually works something like this:
User provide a URL  -> Youtube-dl -> Downloads URL source
-> Downloads other assets which were LOADED THROUGH THE PAGE SOURCE
With the new page source parameter you would simply only change the "Downloads URL source" part which leaves the rest of the process untouched.  It would look like this:
User provide a URL -> Youtube-dl -> page source PROVIDED THROUGH PARAMETER
-> Downloads other assets which were LOADED THROUGH THE PAGE SOURCE
Now as I said in my PREVIOUS answer, youtube-dl would in most cases get the direct link RIGHT AWAY (this is the case for Facebook and Instagram and many more as most pages have the video URL right on the page) from the page source WITHOUT having to download any additional assets or it would download a few assets first.  Now like I said, the assets usually are NOT restricted by the user session as they're on static file servers which in very rare cases just check the IP.

I was under the impression that youtube-dl first had the core files download the URL and then passed it on to the specific extractor, since nearly every single extractor downloads the URL anyway (as you pointed out yourself). This is a flaw in the youtube-dl structure as the same code/process is repeated over every single extractor (hundreds of files), but it's still possible to fix by removing all the requests to the main URL, moving the part to the core files and then handing it over to the extractors through a variable.

we don't always request the url passed by the user, sometimes we use other sources to extract information, these are some examples of the extractors that do this: CBSIE, CWTVIE, CTV, TouTvIE, WATIE...

@remitamine: I was expecting this to be mentioned sooner or later.  My answer is that this is very, very rare and almost never happens.  Take a look at the size of the sites that you mentioned for example and besides 1 extra request for these rare cases won't really cause any performance issues.

I was under the impression that youtube-dl first had the core files download the URL and then passed it on to the specific extractor

Do you even read my post? Core does not and should not be aware of which URL or set of URLs extractor needs and which actions this extractor should take it to obtain them, which headers and cookies to provide and so on.

since nearly every single extractor downloads the URL anyway (as you pointed out yourself)

No it's not. And I did not say that.

This is a flaw in the youtube-dl structure as the same code/process is repeated over every single extractor (hundreds of files), but it's still possible to fix by removing all the requests to the main URL, moving the part to the core files and then handing it over to the extractors through a variable.

No it's not. Even if this subset of dumb extractors that pull only the original URL without any HTTP request tweaking, without custom headers and cookies is generalized there are still extractors that does not follow this pattern and can't be generalized this way.

User provide a URL -> Youtube-dl -> Downloads URL source
-> Downloads other assets which were LOADED THROUGH THE PAGE SOURCE

No, not necessarily.

With the new page source parameter you would simply only change the "Downloads URL source" part which leaves the rest of the process untouched.

How come? What if somebody wants to provide two first pages? Solution should be generic.

User provide a URL -> Youtube-dl -> page source PROVIDED THROUGH PARAMETER
-> Downloads other assets which were LOADED THROUGH THE PAGE SOURCE

Fetching page source sets some cookies required for further extraction that are not available as bare file => extraction won't work.
Page source contains some expired hash, that is used in subsequent requests => extraction won't work.
etc.
Does not scale.

youtube-dl would in most cases get the direct link RIGHT AWAY

No it's not. Even assuming this to be true for a second, what about other extractors? Say, those that don't download source URL at all. Then the way you suggest it to be implemented does not fit them at all and this feature won't be available for them that leads to even more confusion.

assets usually are NOT restricted by the user session

Everything else may require.
So this does not worth the effort.

@dstftw has explained things well. Here are just some interesting statistics:

The number of extractors that downloads the user-input URL: 475

$ grep -r 'self\._download_webpage(url' youtube_dl/extractor/*.py | wc -l
475

The number of total extractors: 900 (the first line in this file is the title)

$ wc -l docs/supportedsites.md
901 docs/supportedsites.md

@yan12125: no reason to continue discussing this then as editing 900 files would take way too much time.
Unfortunate that this feature couldn't be added though, it would be really useful to get the download links on pages that youtube-dl can't reach due to login restrictions.

Well, the point is not that we have lots of extractors, but your solution applies to few websites. If you want some website supporting logging in, open new issues for each one.

This is still completely doable and the solution does actually apply 475 websites/extractors as you mentioned, but then again you'd have to add the if-statement to every single one of those extractors.

I count extractors that downloads the web page, not extractors that only downloads the web page. I didn't find a quick way to count the latter, so I skip it. It should be much less than 475.

@dstftw: now over to your reply:

Do you even read my post? Core does not and should not be aware of which URL or set of URLs extractor needs and which actions this extractor should take to obtain them, which headers and cookies to provide and so on.

Your post mentioned the core AFTER I wrote that, but it doesn't matter much as the solution could be applied to each extractor individually although it would be quite some amount of work.
How come? What if somebody wants to provide two first pages? Solution should be generic.
The solution is generic, the only difference is that this would only work with a single URL.
If they want to download a video located on two URLs then they'd simply have to run two commands with youtube-dl.
Fetching page source sets some cookies required for further extraction that are not available as bare file => extraction won't work. Page source contains some expired hash, that is used in subsequent requests => extraction won't work. etc. Does not scale.
I've explained this part twice now.
The source that has been fed through the parameter is almost always the only one that requires cookies sent through headers, the other assets don't require the login or any other cookies.
There is no reason for the page source to contain an expired hash as the source would be recently taken.
And like I said most websites don't require authentication for the direct video URL or the assets on that page while they do for the video page so there wouldn't be any issues with the cookies (try it for yourself).
`Everything else may require.
So this does not worth the effort.`
"I've explained this part twice now.
The source that has been fed through the parameter is almost always the only one that requires cookies sent through headers, the other assets don't require the login or any other cookies."
There is also no reason for there

Again, this is completely doable, the only thing in the way is that the if statement would have to be added to every single extractor that downloads the page source and like @yan12125 pointed out it's a few hundred files.

The source that has been fed through the parameter is almost always the only one that requires cookies sent through headers, the other assets don't require the login or any other cookies.

This is the key point whether this is doable or not. Please give one example URL that you think this is doable.

And like I said most websites don't require authentication for the direct video URL or the assets on that page while they do for the video page so there wouldn't be any issues with the cookies (try it for yourself).

Likely, it's your work to give examples to prove that your proposal works, not ours.

It is indeed my duty to provide examples, but I didn't bother as I don't think anyone is interested in editing hundreds of files just to add this parameter (although it'd be extremely useful to have).

Your post mentioned the core AFTER I wrote that

I pointed out that extraction process is an implementation detail that should be known only to extractor itself in the very first reply. You are just speculating with terms in order to continue stubborn arguing with facts.

The solution is generic, the only difference is that this would only work with a single URL.

Generic solution offers arbitrary control. By definition. With generic solution one should be able to control any depth of extraction process not only special case with N=1.

The source that has been fed through the parameter is almost always the only one that requires cookies sent through headers, the other assets don't require the login or any other cookies.

"almost always" and "usually" does not mean always. These are concrete extraction scenarios of concrete extractors in youtube-dl codebase.

There is no reason for the page source to contain an expired hash as the source would be recently taken.

No, you can't make such assumptions. You are also not allowed to make any assumptions on IP and way this page was obtained thus making it again impossible in case of IP-bound source page.

It is indeed my duty to provide examples, but I didn't bother as I don't think anyone is interested in editing hundreds of files just to add this parameter (although it'd be extremely useful to have).

The point is not adding this parameter but adding support for these unsupported URLs (that you haven't even provided so far) directly, adding support for authentication for extractors that does not support it, adding support for password protected videos and so on.


Issues with Node Registration (PROXY_REREGISTRATION)

We moved to the latest release v1.12.12 recently and everything worked great the first two days but now we're plagued with PROXY_REREGISTRATION errors that I can't seem to resolve. There is very little information online about this particular error code.
In the hub logs, I see a lot of activity - registering nodes and cleaning up stale(?) node sessions:
02:30:49.003 INFO - Unregistering the node http://xxx.xxx.182.128:5551 because it's been down for 10015 milliseconds 02:30:49.003 WARN - Cleaning up stale test sessions on the unregistered node http://xxx.xxx.182.128:5551 02:30:49.050 INFO - Registered a node http://xxx.xxx.182.128:5551 02:30:49.972 INFO - Registered a node http://xxx.xxx.58.224:5550 02:30:52.372 INFO - Unregistering the node http://xxx.xxx.30.73:5552 because it's been down for 10022 milliseconds 02:30:52.372 WARN - Cleaning up stale test sessions on the unregistered node http://xxx.xxx.30.73:5552 02:30:52.419 INFO - Registered a node http://xxx.xxx.30.73:5552 02:30:54.053 INFO - Marking the node http://xxx.xxx.182.128:5551 as down: cannot reach the node for 1 tries 02:30:54.976 INFO - Marking the node http://xxx.xxx.58.224:5550 as down: cannot reach the node for 1 tries 02:30:57.431 INFO - Marking the node http://xxx.xxx.30.73:5552 as down: cannot reach the node for 1 tries
I bumped up the hub's cleanUpCycle to 10s and that seemed to help a bit, but not enough.
In Selenium Grid's console UI, each of my nodes shows this possibly related error message:

SetupTeardownProxy unknown version,server response code : 500

It appears that each node successfully registers with the hub, then 5 seconds later they are marked as down (probably due to the SetupTeardownProxy HTTP 500), and finally 10 seconds later the node is removed from the hub's list of registered nodes... and the cycle repeats as the node attempts to re-register.
This is the error I'm seeing in my test runner (Visual Studio Team Services / Visual Studio Test Runner):

System.InvalidOperationException: Session [87a9dff5-aeba-4a15-b36c-64276527ef38] was terminated due to PROXY_REREGISTRATION

I'm using WebDriver 3.3.0 on all hubs and nodes, with Selenium Grid Extras v1.12.12.
Thank you

On a related note, my grid_extras hub log included these events:

09 Mar 2017 03:10:00 [qtp1891546521-24] INFO  proxies.SetupTeardownProxy  - Attaching node http://xxx.xxx.58.224:5550
09 Mar 2017 03:10:02 [qtp1891546521-26] INFO  proxies.SetupTeardownProxy  - Attaching node http://xxx.xxx.30.73:5552
09 Mar 2017 03:10:07 [qtp1891546521-19] INFO  proxies.SetupTeardownProxy  - Attaching node http://xxx.xxx.182.128:5551
09 Mar 2017 03:10:15 [qtp1891546521-26] INFO  proxies.SetupTeardownProxy  - Attaching node http://xxx.xxx.58.224:5550
09 Mar 2017 03:10:17 [qtp1891546521-22] INFO  proxies.SetupTeardownProxy  - Attaching node http://xxx.xxx.30.73:5552
09 Mar 2017 03:10:22 [qtp1891546521-22] INFO  proxies.SetupTeardownProxy  - Attaching node http://xxx.xxx.182.128:5551
09 Mar 2017 03:10:32 [qtp1891546521-26] INFO  proxies.SetupTeardownProxy  - Attaching node http://xxx.xxx.30.73:5552
09 Mar 2017 03:10:35 [qtp1891546521-26] INFO  proxies.SetupTeardownProxy  - Attaching node http://xxx.xxx.58.224:5550
09 Mar 2017 03:10:37 [qtp1891546521-19] INFO  proxies.SetupTeardownProxy  - Attaching node http://xxx.xxx.182.128:5551

There is a compatibility issue with Grid Extras v1.12.12 and WebDriver 3.3.0 where the Proxy version is not recognized
You need to bump down WebDriver to 3.2.0

You are the man, thank you for saving me a ton of time! :)

I wonder if its a grid extras issue, or if its a selenium grid issue in general.

+1

@alltheoptions Hi, what is the +1 for ?

@smccarthy the problem still exists


urlBase64Decode issue

Hi. We stumbled upon case when urlBase64Decode fails decoding valid token. when Im looking inside that function I see you using return decodeURIComponent(escape(window.atob(r))). According to cods "escape" is deprecated and you should use encodeURIComponent instead. When I replaced escape with encodeURIComponent our problematic token was decoded correctly.

This is still an issue


test failed while building go on debian 8(debian jessie)

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go version go1.7.3 linux/amd64
What operating system and processor architecture are you using (go env)?
My operation system is debian 8
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/zhw2101024/.local/share/go"
GORACE=""
GOROOT="/home/zhw2101024/go1.7"
GOTOOLDIR="/home/zhw2101024/go1.7/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build424418262=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1
What did you do?
build go from source using following command:
GOROOT_BOOTSTRAP=/home/zhw2101024/go1.7 GOROOT_FINAL=/home/zhw2101024/go1.7 ./all.bash
What did you expect to see?
Go version is "go1.7.3", ignoring -next /mnt/floppy/src/go/api/next.txt
ALL TESTS PASSED

Installed Go for linux/amd64 in /mnt/floppy/src/go
Installed commands in /mnt/floppy/src/go/bin
The binaries expect /mnt/floppy/src/go to be copied or moved to /home/zhw2101024/go1.7
What did you see instead?
--- FAIL: TestLoadFixed (0.08s)
time_test.go:943: Now().In(loc).Zone() = "-01", -3600, want "GMT+1", -3600

Dup of #17276, already fixed on tip.


Update infra-operator kuttl tests set up to extend test coverage

With this PR, now we deploy KeystoneAPI before we run kuttl tests for infra-operator. This is required so that we can test OpenStackClient CR which depends on KeystoneAPI.
Also, this adds the step to ensure the Memcached CR deployed via make memcached_deploy has the expected name, so that we can change the name in the sample config file. This is needed so that we can deploy two instances in infra-operator kuttl tests. One is used by KeystoneAPI CR while the other is created for tests.

recheck

recheck


Add Revert reason to Trace APIs

The debug_* series trace operations record the revert reasons when a method call reverts.  However the trace_* series do not.  When a call fails I would like the reason to show up in the vmTrace at the operation that made the call and in the flat trace.  When a precompiled call returns a revert reason that also should be included as a trace, even though precompiled calls are not normally in the vmTrace.

@davemec is this closed by #1921?

I did a quick review of the PR and I believe it does. @shemnon you opened this ticket, are you happy to close it?


Configure target directory for start scripts

Hi,
I would propose a new feature: to be able to configure the location of the start scripts. Currently it is only possible to set its name, it would be great to be able to set the target directory too.
Expected Behavior

You should be able to set the complete location of startup script, either with applicationName and a new property applicationDirectory, or a single new variable applicationPath.
For example,
task example1(type: CreateStartScripts, dependsOn: jar) {
  applicationName = "example1"
  applicationDirectory = ""
}

would produce:
|
+--+-- libs/
   +-- example1
   +-- example1.bat

And this config
task example2(type: CreateStartScripts, dependsOn: jar) {
  applicationName = "example2"
  applicationDirectory = "foo/bar"
}

would produce:
|
+--+-- libs/
   +-- foo/
      +-- bar/
         +-- example2
         +-- example2.bat

Current Behavior

The target directory is hard-coded to bin:
See this line. It is not configurable.
Context

I am trying to output my build scripts at the root of the zip produced by distZip instead of being in a subdirectory. I'd like to be able to do something like in the first example. This is also similar to the start-script used by gradle itself: ./gradle is at the root instead of being inside the bin directory.

See https://discuss.gradle.org/t/change-startscripts-relative-path-from-bin-to-something-else/22912. You can extend CreateStartScripts and chang the relative script path. However, I agree that it'd be better to configure this via the DSL.

Thank you for this link, it helped me to solve my issue but I am happy that we agree that it should be simpler. I was about to implement this solution myself but I wasn't comfortable importing classes from a package named "internal".

I think this could be solved fairly easily by exposing a new property. Would you be interested in providing a pull request for this?

I am not familiar enough with Gradle or Groovy and to be honest I don't have much time right now to learn how the link between the DSL and the Java code I found works. I did not research it yet, but if you have an article explaining how it works, I'd be interested. Java is not my main technology so it was a punctual problem for me: even if Gradle is great, I do not expect to use it much.
What I mean by that is that I'd like to help, but I cannot guarantee that I'll provide a PR.

OK, thanks for your feedback. I added the tag help-wanted if somebody from the community wants to pick this up.

Hi @bmuschko , I'd like to work on this feature. This is my first time changing code inside gradle code. I have implemented the feature but still need to add tests / documentation.
Can you help me out with a few things?

How do I run one specific integration test from the command line?
Can I submit a WIP pull request for a code review of the feature (before having written any tests)? I can add the tests / documentation later once the implementation passes code review.

Here are my changes so far:
https://github.com/gradle/gradle/compare/master...xz64:custom_start_scripts_dir

I'm closing this issue now that #2977 was merged.


Use Laravel Pint  & gomzyakov/php-code-style

https://github.com/gomzyakov/php-code-style

Used in https://github.com/gomzyakov/payture/pull/5


Just a Test

1. Why is this change necessary?
2. What does this change do, exactly?
3. Describe each step to reproduce the issue or behaviour.
4. Please link to the relevant issues (if any).
5. Checklist

[ ] I have written tests and verified that they fail without my change
[ ] I have squashed any insignificant commits
[ ] I have written or adjusted the documentation according to my changes
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[ ] I have read the contribution requirements and fulfil them.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.1 out of 13 committers have signed the CLA.:white_check_mark: shyim:x: taltholtmann:x: keulinho:x: tobiasberge:x: OliverSkroblin:x: PaddyS:x: Lennart Tinkloh:x: SebastianFranze:x: pweyck:x: J-Rahe:x: ssltg:x: leichteckig:x: raknisonLennart Tinkloh seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Unnecessary SERVICE mode

Is your feature request related to a problem? Please describe.
Unncessary SERVICE mode when it could be DOCKSTORE.YML instead.
descriptorType being service allows us to know if a workflow is service,
Describe the solution you'd like
Just use DOCKSTORE.YML instead
Describe alternatives you've considered
Not have any mode for services since it's known to use .dockstore.yml (at least for now)
Additional context
webservice_test=# select count(*) from service where descriptorType='service';
 count
-------
    73
(1 row)

webservice_test=# select count(*) from service where mode='SERVICE';
 count
-------
    73
(1 row)

webservice_test=# select count(*) from service where mode='SERVICE' and descriptorType='service';
 count
-------
    73
(1 row)

webservice_test=# select count(*) from service;
 count
-------
    73
(1 row)

webservice_test=# select count(*) from workflow where descriptorType='service';
 count
-------
     0
(1 row)

webservice_test=# select count(*) from workflow where mode='SERVICE';
 count
-------
     0
(1 row)

┆Issue is synchronized with this Jira Story
┆Issue Type: Story
┆Fix Versions: Dockstore 1.9
┆Sprint: Sprint 34 Isopods
┆Issue Number: DOCK-1325

A bit odd to review since mode isn't displayed in the info tab for the UI (not even on 1.8/prod). However the api response on a published service shows that mode = DOCKSTORE_YML as of API - 1.9.0-alpha.9.


Builders bindings tests for python&java

Rebased #764
So far in this pr:

Java builders tests
Python builders tests
gradle integration for java

Codecov Report

Merging #792 into trunk/bindings-test will not change coverage.
The diff coverage is n/a.

@@                 Coverage Diff                  @@
##           trunk/bindings-test     #792   +/-   ##
====================================================
  Coverage                40.17%   40.17%
====================================================
  Files                      456      456
  Lines                    15502    15502
  Branches                  8594     8594
====================================================
  Hits                      6228     6228
  Misses                    1554     1554
  Partials                  7720     7720

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6089aad...15680e6. Read the comment docs.


I think you missed an include file!

Thanks for updating the plugin!
When I build I think that
#include "OscMessageStruct.h"
Is missing from the git.
Looking forward to trying this!
Dan

I should never commit at 1am...
2016-03-02 11:58 GMT+01:00 Dannington<EMAIL_ADDRESS>
Thanks for updating the plugin!
When I build I think that
#include "OscMessageStruct.h"
Is missing from the git.
Looking forward to trying this!
Dan
—
Reply to this email directly or view it on GitHub
https://github.com/monsieurgustav/UE4-OSC/issues/22.

OK, done!
Sorry about that...


Addresses #610 - geodesic plotting

Addresses #610

[/] Capability to pass an optional title argument into plot2D(), parametric_plot(), and plot() for the interactive and static modules.
[/] Capability to pass an optional aspect argument for plot() method for the interactive and static modules, along with optional aspect_ratio for interactive module.

Appreciate if you can help review this PR @JeS24. Thank you in advance for your time.

Also, thanks for the PR. Could you run tox -e reformat in the base directory? That should help with the CI quality checks. See this for more details - https://github.com/einsteinpy/einsteinpy/wiki/Development-tips.
I think the additions with some minor changes as listed in the review are proper. We will need a few more additions here:

[ ] Couple of unit-tests about title, aspect and aspect_ratio for Static and Interactive plotters to maintain code coverage. Feel free to ask me if you have any doubts.
[ ] These tutorial notebooks need to be edited to reflect this change.

[ ] NB 1 - Do not change the paragraph about the existence of the fig attribute, but rather add the title and aspect arguments to the *_plot*() methods, to make them conspicuous.
[ ] NB 2 - Add another interactive plot with aspect set to data.
[ ] NB 3 - Edit the (static) 3D plot so that it has a title and aspect set to equal.

I think, this should be sufficient for this PR.

Also, some notes for personal reference:

Wondering, whether we should show different options for aspect or try to equalize the options across the Static & Interactive plotters, since their API is so similar from the user's perspective. Matplotlib's equal nomenclature does make more sense than Plotly's data. Perhaps, we can wait till matplotlib==3.7 adds more options for the aspect ratios. Could be an extension to #610.

@mshumayl Your tests on Python 3.7 aren't passing because matplotlib has dropped support for Python 3.7. We should too. Give me some time. I will make the necessary changes on the CI side.

I see, no wonder I got NotImplementedError for all py37 CI checks.
Thank you very much for your help. Let me know if I need to push again to pass the CI tests.

@mshumayl Checks should pass now.

Apologies, please ignore while I properly sync my local repo with the remote branch.

@mshumayl By CHANGELOG, I am referring to this file - https://github.com/einsteinpy/einsteinpy/blob/main/CHANGELOG.

@mshumayl Try force-pushing. Not sure, why there is a codecov failure.

@mshumayl So, the docs check failure is fixed. However, docs/source/examples/Visualizing Precession in Schwarzschild Spacetime.ipynb is conflicting with the one on main, because I had changed the interactive plots to static plots, as the interactive ones aren't showing up in the documentation. And manual conflict resolution is a no-go given the notebook is a massive JSON file.
Can you sync with main on your end, merge locally (just accept "current changes" for now) and then push the changes? After that, I should be able to merge the PR. The update regarding aspect to the notebook can be separate.

Alright, in the meanwhile please ignore the mess while I figure this out.

Thanks for the changes @mshumayl. Merging now.

Thanks for your help and pointers @JeS24, I really appreciate it.

Thanks for your help and pointers @JeS24, I really appreciate it.

No problem. Glad I could help. : )

@JeS24 lets remove old Python versions from CI

@JeS24 lets remove old Python versions from CI

Hey. I have removed 3.7 from CI and package requirements. At the moment, we are testing against 3.8, 3.9 & 3.10. I think, we can drop 3.8 around the next release (hopefully soon). Or, we can pin our drop schedule to numpy's drop schedule. There have been issues with dropping 3.7 though - Google Colab is still annoyingly on 3.7 and Binder defaults to it (the latter is no longer a problem since we are specifying required Python version in runtime.txt now).

@shreyasbapat Also, before I forget, if you have some time, can you regenerate the CircleCI SSH keys (for merged PRs). Currently, the Linux build is failing because of it. It requires admin permission, it seems.


fix(ci): Fix indentation of condition so it applies correctly

Description
The condition was under the inputs node, where it doesn't do its job.
Reviewer Guidance
The review process is outlined on this wiki page.

Any idea when this broke? Has it always been this way?

As far as I can tell it's always been this way.


Expose GetName extension on IFormFile

Fixes #499. These names should probably be cached somewhere. Anyone opposed to exposing a Name property on IFormFile backed by a Lazy<string>?

This isn't how I think it should be done. We should update FormFile to have a name property on it. Make it lazy there.

@davidfowl Hence my "These names should probably be cached somewhere. Anyone opposed to exposing a Name property on IFormFile backed by a Lazy<string>?" :wink:

https://github.com/aspnet/HttpAbstractions/blob/dev/src/Microsoft.AspNet.Http/Features/FormFeature.cs#L150-L160

@khellang Lazy is overkill, it doesn't need to be thread safe. Also add a test for this once you're done.

Lazy is overkill, it doesn't need to be thread safe.

So are you saying parse the name eagerly? Or just _name ?? (_name = GetName())?

Lazy is fine but using Lazy<T> is just overhead since we don't care about thread safety.

@davidfowl How does it look now?

Much better!

What about FileName as well?

Go for it

I think I change my mind on the lazy. Not worth it. Parse it out and just assign eagerly.

Have you looked at the getters for Name and FileName in CDHV? Granted, it's not often you send files, and not very many of them, so it probably won't matter much.

Added 5d1cd10 to close #352 as well.

Ok this PR is good to go once the CI passes. Then we'll have to see what repos break. If I had to guess. MVC would be broken


Add org_id/app_id/endpoint_id/msg_id (or a subset of these) to the worker tracing span

Just adding them to the tracing span so that the logs are nicely annotated.

I think it's being addressed in #704

Done.


Refactor _.intersection, More readable, Less lines

Uses underscore to make _.intersection shorter and more understandable.

Re: https://github.com/jashkenas/underscore/pull/1517

Oops my bad


args.suffix is undefined when esbuild's onTransform hook is triggered

Environment
<EMAIL_ADDRESS>Reproduction
repo
And add any unplugin-related plugins.
Describe the bug

Additional context
No response
Logs
No response

Please provide a minimal reproduction. Thanks.

Why reproduction is required

Please provide a minimal reproduction. Thanks.

Why reproduction is required

Step 1, download super-dev;
Step 2, install unplugin-auto-import and use it;
Step 3, npm run prod;
Then in 049f99c, we can find that esbuild only provides args.suffix from v0.14.3. The esbuild version used in super-dev is v0.12.24.

I guess you can try to upgrade esbuild version, and the issue will be fixed, right?


Can This Be Used To Converted A Local Hosts File To An LSRuleset?

Hi. How might I adapt this to replace SB's list URL with that of one of my local hosts files?

You can run the snitcher.sh script against any URL, including a local file:// URL. The only criteria is that any blacklisted host names should have the IP address <IP_ADDRESS> in the hosts file.
So you can clone the repository and then run something like this:
URL='file:///path/to/my/hosts' NAME='My hosts file' ./snitcher.sh > my_hosts.lsrules

Note: this is similar to this line in update.sh.


negative value when 'too many login attempts' happens

Laravel Version: 5.3.15
PHP Version: 7.0.11
Database Driver & Version: mysql  Ver 15.1 Distrib 10.1.17-MariaDB, for Linux (x86_64) using readline 5.1
memcached: 1.4.31-1

Description:
I don't know how this happens. I did a composer update some minutes ago and tested some functions locally, also the login. After 5 attempts with a wrong password I still can not login.
Too many login attempts. Please try again in -127 seconds.
Steps To Reproduce:
Try to login with the same wrong password (i used 'wwww') with 'remember me' checkbox unchecked, until the throttle message appears.
Try again with correct password, 3 sec before waiting time is over, with 'remember me' checked. And the again with correct password and remember me checked. The I see something like: "Try again after -15 seconds".
Too many login attempts. Please try again in -684 seconds.
My cache driver is memcached. The :lockout value stays in memcached even after 4000 seconds. 'Normal' Cache::forget() and other cache functions work in the application.
When I do php artisan cache:clear I can login again, and with cache driver file everything seems to be okay.

I don't seem to be able to replicate this using the file driver, so maybe it's only a memcached issue?

Thank you. Today, after a reboot, I can't reproduce it, too.

💃 :D

FYI, just duplicated this on 5.4

I can reproduce this in 5.6.16 backed by memcached, flip to file cache (or flush memcached) and it's resolved.


Refactor formatAettosToAe

Please describe the problem that your request should resolve.
Usage is mixed in adapter and template
Describe the solution you'd like
Can be used in adapters.js
Is it already possible to achieve the same outcome in any other way? If so, how?

Additional context

This cannot be implemented yet as it highly interfering with this
https://github.com/aeternity/aescan/pull/637 ( <- should be merged first)


try to document the build

Closes #2 .

Hi! Thanks for submitting this, but I've gone and reworked the build process so it's a lot more automated, then documented that.


Add initial box view, with pokemon cards.

Initial box view PR.

[x] Update pk6parse and use the ball name to get sprite
[ ] Add some more tests
[x] Null check on heldItemName
[x] Pad ID
[x] Switch OT and ID
[ ] Link the box card, rather than the title
[ ] Link the pokemon card, rather than the title (or remove it at this point because it does nothing?)

Looks good apart from a few comments.
I published the latest version of pk6parse and merged the greenkeeper PR, so it should be possible to uncomment that line. Be sure to replace the é character with e when fetching the pokeapi url, though.

I pushed a commit that fixes some of these, see the updated checkboxes.

Ok, this should be ready to merge, feel free to play around with it.
One known issue is that only the top half of the Pokémon cards are hyperlinked; the bottom half doesn't link to anything.


Fix broken link

Fix broken 'data annotation documentation' link.

Closing, realized this is only an issue if navigating between the docs files in GitHub. The main use for these MD files is to build the actual documentation site, and all is working fine there.


App immediately closes

I had been running the program for over a week and suddenly today me laptop froze.  when i tried to run the program after turning my laptop back on the program wouldn't run.  i get a window that reads "bling has stopped working" but it closes to quickly to do anything.  i have uninstalled and reinstalled and nothing changes

Remove folder Bling.exe inside Documents and install everything fresh again. It may be a problem of compatibility between viewer and desktop.


Security Contact/Policy

Please provide a security contact or a security policy. Thank you!

Write me here<EMAIL_ADDRESS>


Integrate PagedAttention Optimization custom kernel into vLLM

This PR introduces the custom optimized PagedAttention on ROCm. To use the Custom PagedAttention use the following env variable VLLM_USE_ROCM_CUSTOM_PAGED_ATTN=1.
Currently this PagedAttention kernel is supported with fp16 datatype, with GQA ratio=1 to 16 and Head-size=128 and block-size=16.
Kernel was authored by : @sanyalington
cc: @shajrawi @sunway513

@shajrawi added into the documentation and enabled by default. The context length <= 32k support will be added into the next PR after this one.


[BugFix] Set mv insert overwrite job enableInsertStrict to false

Fixes #issue
set seesion variable enableInsertStrict to false.
eg.
if base table has partition [2020-07-15, 2020-08-15),
mv has partition [2020-08-01, 2020-09-01),  mv need to insert from base table data which partition value at [2020-01-01, 2020-08-15)
What type of PR is this:

[ ] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[ ] Yes, this PR will result in a change in behavior.
[x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.2
[x] 3.1
[ ] 3.0
[ ] 2.5

@mergify backport branch-2.5


No such file or directory error

⬇ Pulling screenshots from your connected device!
rm: /sdcard/screenshots/nz.co.myappid.test/screenshots-default/: No such file or directory
Any Idea how to fix this?
Thanks

Can you provide more information about how to reproduce the error?


Provide a way to change the type of $root

What problem does this feature solve?
We use root component as a global store.
createApp({
  data: () => ({ loginInfo: { id: 1, name: 'a' } }),
});

We an access loginInfo easily in child components

User name: {{$root.loginInfo.name}}

It used to work in Vue 2 and older version of Vue 3, until we migrated to vue-tsc + volar recently
error: 类型“ComponentPublicInstance<{}, {}, {}, {}, {}, {}, {}, {}, false, ComponentOptionsBase<any, any, any, any, any, any, any, any, any, {}>>”上不存在属性“userInfo”

https://github.com/vuejs/vue-next/blob/d955cfacd64217e381e17d0e1e1fbf476a431080/packages/runtime-core/src/componentPublicInstance.ts#L189
What does the proposed API look like?
$root should be correctly typed as what user specified in createApp. In my case ComponentPublicInstance<{}, {}, { loginInfo:{ id: number; name: string; } }

$root and $parent and generally typed as ComponentPublicInstance | null, which makes them almost unusable in vue template directly. Even any can be a better option than ComponentPublicInstance | null

$parent cannot be typed because it depends on the runtime as a component could be the child of different components at runtime
$root being a type, I don't think you can extend it in your codebase to add properties 🤔
You could however add a custom global property (https://v3.vuejs.org/api/application-config.html#globalproperties) which can be typed by extending the global types. In both scenarios, you will have to manually type your root app as there is no way to directly infer the global type of $root in vue from a createApp() call.
Note you can also do $root as Root in templates by defining a global type Root in one of your global d.ts files type Root = ComponentPublicInstance<{}, {}, { loginInfo:{ id: number; name: string; } }

In both scenarios, you will have to manually type your root app as there is no way to directly infer the global type of $root in vue from a createApp() call.

I was aware of it. In addition, there can be multiple createApps in one project.
Can we just type $root as any? Using $root as the common ComponentPublicInstance feels meaningless in Vue 3.
While in Vue 2, you may use $root as an event bus.


Explicitly set php supported version to composer.json

Set the php version require help to get sync composer with IDE and other
dev tools.

@widoz Thanks Guido, that was missing. Just a thing, Composer does not support to have the same package required in both require and require-dev and even if PHP is a platform requirement, it follows same rules. If you require PHP 7+ in require, that's already enough.
You can verify yourself by running composer validate...

@gmazzap Thank you for the advice.
I removed the dev recurrency.


[BUG] Flink batch is written to hudi. When the number of slots is less than write.tasks, it cannot be written

jira: https://issues.apache.org/jira/browse/HUDI-5100

Hello, why close it then ?

Can this be added to the 0.13 plan?

Yes, this is a known bug, sorry to tell you that it is not supported in 0.13.0 release.


cmd/go: go mod does support directory packages

What version of Go are you using (go version)?

$ go version
go version go1.11 darwin/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/mrj/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/mrj/Desktop/vendor"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/mrj/Desktop/echo-basic/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/4v/knjv8mkn7931qlfdfc7s3v140000gn/T/go-build634382221=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I want to see download from github.com
What did you see instead?
https fetch failed: Get https://golang.org/x?go-get=1: dial tcp <IP_ADDRESS>:443: i/o timeout
Fetching https://golang.org?go-get=1
How should I write the go.mod file correct ?

CC @bcmills

@myitcv Yeah. Im in china , golang.org was blocked ! Im use github.com/labstack/echo depends on golang.org/x/crypto/acme/autocert
see here
so I want to replace golang.org/x/crypto/acme/autocert to github.com/golang/crypto/acme/autocert
but the go-get fetch from https://github.com/golang/crypto/acme/autocert is not a vcs .
how can I make the correct replace in my go.mod

Thanks for confirming @clearcodecn. This is, in effect, a duplicate of #28652. So I'll close this and we can continue conversation there.


Trunk-based build for preview Kotlin + Compose compiler versions

We're finding out about problems in the JB Compose compiler for JS targets way too late. We need to see these much earlier so that they can be fixed before the releases.
Trunk builds should also have a variant that attempts to use the in-development Kotlin and Compose compiler versions. The success or failure of this variant of the trunk build should be reflected in the open or closed status of a single issue. This will lower the noise as it will only see activity when the status changes. Plus it means anyone can see the status at any time.
This will be tricky because we currently use both the AndroidX Compose compiler and the JetBrains Compose compiler. We could replace the AX one with JB to simplify things.

We probably also want the ability to maintain a branch with changes for these versions.

This is less of an issue now. We test Kotlin EAPs with branches and that brings the new Compose compiler.


Installing make

Hello,
I just did a clean install of NixOS 19.03(Koi) and wanted to compile dwm, but make was missing, so i typed in sudo nix-env -i make, and got an error. After doing sudo nix-env -qa make, make wasn't there. Does anyone know what package i have to install?
Thanks in Advance

Perhaps the attribute you want is gnumake?
[nix-shell:~]$ nix search gnumake
* nixpkgs.gnumake (gnumake)
  A tool to control the generation of non-source files from sources
[nix-shell:~]$ nix-shell -p gnumake
[nix-shell:~]$ make --version
GNU Make 4.2.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

I was also struggling to find make at first, and gnumake what was I was searching for.


Circular dependency in Makefile

From https://github.com/pypa/warehouse/pull/10278#issuecomment-974902013:

I believe this change has introduced a circular dependency, as perceived by these messages:
$ make -v | head -1
GNU Make 3.81
$ make build
make[1]: Circular requirements/docs.txt <- .state/env/pyvenv.cfg dependency dropped.
...

For funsies, I tried the same with remake, which uses a newer version of make:
$ remake --version | head -1
GNU Make 4.3+dbg-1.5
$ remake build
remake[1]: Entering directory '/Users/miketheman/workspace/miketheman/warehouse'
remake[1]: Circular requirements/docs.txt <- .state/env/pyvenv.cfg dependency dropped.
remake[1]: Circular requirements/lint.txt <- .state/env/pyvenv.cfg dependency dropped.
remake[1]: '.state/docker-build' is up to date.
...

I don't know if this is harmful yet, but I'm having other issues getting started with development, and wondered if this is somehow related or not.

Originally posted by @miketheman in https://github.com/pypa/warehouse/issues/10278#issuecomment-974902013

Reposting my comment:
I think we could mostly solve the problem by making the targets more granular, but at core we need to solve something: in order to install pip-compile, we need to know which version to install it too, which means we need to run pip-compile and so on.
If we install pip-compile without the hashes, it kinda defeats the purpose of using hashes in the first place.
So one solution could be to not declare to make that the pip-compile target needs the pip-compile requirement file, considering that it doesn't often change, and the venv gets recreated once in a while anyway.
So we would have:

A target for venv creation but not installation of dependencies
a target for pip-tools that would depend on that one, which would actually use the requirements/pip-tools.txt file but wouldn't say it
requirements/*.txt would depend on that one
the full environement target, the one everyone uses, would use the relevant .txt files and declare it.

I believe it would work.

Or, we could take this as an opportunity to move more towards #4948, since it doesn't make a ton of sense right now for us to compile dependencies on one platform (the host OS) to be used on a different platform (inside the container images), which removes the need to build dependencies in the virtual environment.

Why not to vendorize pip-compile?

For example with pex.
pip install pex
pex pip-tools --console-script pip-compile --output-file pip-compile.pex

-rwxr-xr-x 1 root root 4750609 Dec  7 10:53 pip-compile.pex


Document update conflicts showing in logs

I'm getting Document update conflict  a lot (30%?) of the time while use the Tutorial messaging code (which is taken directly from the Web Sandbox).
An example is this:
Uncaught (in promise)
Object { status: 409, name: "conflict", message: "Document update conflict", error: true, id: "v8644a9100f541b841d54deef10ab0d6ba2abd7a13e87fe117bebfcb725af6e85", docId: "v8644a9100f541b841d54deef10ab0d6ba2abd7a13e87fe117bebfcb725af6e85", stack: "" }
docId: "v8644a9100f541b841d54deef10ab0d6ba2abd7a13e87fe117bebfcb725af6e85"
error: true
id: "v8644a9100f541b841d54deef10ab0d6ba2abd7a13e87fe117bebfcb725af6e85"
message: "Document update conflict"
name: "conflict"
stack: ""
status: 409
<prototype>: Object { toString: toString(), stack: "", … }
vendors-node_modules_mdx-js_react_dist_esm_js-node_modules_verida_account-web-vault_dist_inde-af8651.js line 104741 > Function:48:18
    sendMessage http://localhost:3000/vendors-node_modules_mdx-js_react_dist_esm_js-node_modules_verida_account-web-vault_dist_inde-af8651.js line 104741 > Function:48
    AsyncFunctionThrow self-hosted:699

Context name is Verida: Tutorial and changing it fixes the problem

The underlying issue here was due to a Verida account created using an old version of the client-ts library.
Within the client-ts library there is a datastore that saves an index of all the databases opened within the current application context. The older client-ts library version used a different schema name for this datastore than the schema used in the current client-ts version.
This datastore is configured to always set the primary key (id ) of the database row to match the name of the database. However, in this instance there is already a record that exists (but with an incorrect schema) so when the library forces an attempt to save a new record it creates this conflict error.
The solution here is to:

Create a new Verida account (which won't have any previous database indexes with the wrong schema)
Change the context name (which also won't have any previous database indexes)


bug: Error deleting a message after bot sends DM to user

What happened?
Revolt.js 7 seems to not handle deleting messages using message.delete() or channel.deleteMessages() correctly after a DM is sent.
Error message:
/home/dumpling/Documents/Code/stationbot/packages/revolt.js/lib/cjs/classes/Message.js:96
        return __classPrivateFieldGet(this, _Message_collection, "f").getUnderlyingObject(this.id).authorId;
                                                                                ^

TypeError: Cannot read properties of undefined (reading 'authorId')
    at get authorId [as authorId] (/home/dumpling/Documents/Code/stationbot/packages/revolt.js/lib/cjs/classes/Message.js:96:99)
    at Timeout._onTimeout (/home/dumpling/Documents/Code/stationbot/packages/base/dist/index.js:244:53)
    at listOnTimeout (node:internal/timers:564:17)
    at process.processTimers (node:internal/timers:507:7)

Node.js v18.7.0

Code is available here
Removing message.delete()/channel.deleteMessages() seems to fix the issue.
Currently using revolt.js 7.0.0-beta.1. Built locally.
EDIT: Add version information

Issue persists in revolt.js 7.0.0-beta.4

Can't reproduce on revolt.js 7.0.0-beta.6. (Built locally)

Assuming this was fixed.


[VC]: Sync Kubernetes ResourceQuota to Tenant Namespace

ResourceQuota restricts tenant from using too much resources in super cluster. Would there be interest in applying resource quota to tenant namespace?

First shall we sync resource quota upward to tenant namespace. This way tenant can monitor his/her resource usage and tenant's resource request can be restricted in VC space.
Second, if Resource Quota of virtual cluster has been configured, can Syncer apply the same quota or a subset quota to tenant’s namespace?

@christopherhein @Fei-Guo @charleszheng44

Thanks for filing this @weiling61 I think syncing ResourceQuotas could be good but it heavily depends on how it was implemented for each supercluster as cluster admins, similarly to the priorityclasses. I could see this being done as one of the optional syncers.
Some notes we need to consider:

if the tenant control plane has the quota admission controller turned on will it update the status.used on the ResourceQuota and how will, assuming we're not going to be syncing downward we'll need to make sure that those stay in sync somehow.
Because we want to stay true to core kubernetes implementation if we sync them I think we need to make sure we're syncing the entire object and not a subset quota like you called out

If in sync is required, we probably only allow tenants to monitor their quota usages, but not allow them to modify them
I did not explain clearly about "the subset of quota" idea. Yes we will always sync entire quota object to tenant namespace. But we can choose a smaller quota value if needed.

I see, thanks for the clarification. The smaller quota value seems like it would be out of the scope for syncing resources since it might be used in different ways depending on how someone is using and deploying virtualcluster.

Agree, small value is an example

@christopherhein @weiling61  I agree. How to synce resource quota or priority class really depends on how supercluster implements the resource management mechanism for vc.
In my opinion, maybe we can make the resource quota an optional resource for syncing. Or more flexibly, provide new syncer options like --uws-resources, --dws-resources, which allow the user to decide how they want to sync the resource.

In my opinion, maybe we can make the resource quota an optional resource for syncing. Or more flexibly, provide new syncer options like --uws-resources, --dws-resources, which allow the user to decide how they want to sync the resource.

Sounds like a good idea, from your experiences, do you see the syncers as being independent enough where you could have that level of options, ie could you implement dws without uws or vice versa and it wouldn't cause issues? or is that a lot more effort to maintain?

In many cases, tenants want to control their quota independently in tenant master. Normally super does not need to be aware of tenant quota for function correctness (I do see request to sync them because some monitoring tools may watch super master only for tenant quota usage though). I tend not to sync quota from tenant to super.
Syncing from super to tenant is interesting. This is something that tenant is not expected, hence it is less likely tenant has any tool depending on the quota object. In this case, the difference between syncing and not syncing quota is that, the former may prevent Pod objects from creating and the latter may make the Pod stuck in pending status for a long time (syncer will keep retry until some other tenant Pods are gone which free the quota). In the latter case, we can also send event to tenant to notify user that the quota in super is used up.  Overall, the question is that which is the preferred behavior or can the behavior of not syncing quota be acceptable.
I don't see a strong need to sync quota resource. If it has to be synced in some cases I think the source of truth should be tenant.  And one important thing is that we should keep the Pod resource configuration consistent between tenant and super if resource quota is synced. This means any sidecar webhook cannot be installed in super and it has to be installed in each tenant.

In my opinion, maybe we can make the resource quota an optional resource for syncing. Or more flexibly, provide new syncer options like --uws-resources, --dws-resources, which allow the user to decide how they want to sync the resource.

@charleszheng44 Sounds like a good idea, from your experiences, do you see the syncers as being independent enough where you could have that level of options, ie could you implement dws without uws or vice versa and it wouldn't cause issues? or is that a lot more effort to maintain?

In my opinion the resource sync direction has to be deterministic and predefined (the other direction should not be supported). Note that the syncer work for all tenants, if we open the sync direction flexibility, what if some tenants want one direction but the others want the opposite direction. This is way too complicated to manage.

In my opinion the resource sync direction has to be deterministic and predefined (the other direction should not be supported). Note that the syncer work for all tenants, if we open the sync direction flexibility, what if some tenants want one direction but the others want the opposite direction. This is way too complicated to manage.

Yea, this was my concern as well regarding complexity for maintenance.

I agree that resource downward (tenant to super) syncing will bring quite some confusions. Upward syncing (super to tenant) makes more sense. ResourceQuota is an isolation mechanism super wants to use to control tenant behaviors. This is similar to OS system ----  admin can set quotas for user,  user can check the quotas usage but cannot change the quota.
If super wants “the source of truth should be tenant”, they just do not turn on --extra-syncing-resources=resourcequota and api-server in each vc cluster can handle quota restriction.  If super turns on ResourceQuota syncing. It means tenant should not be allowed to modify assigned quotas at all. Resource quota UWS will over-write the quota modified by tenant, which provides super an enforced resource isolation mechanism.

I agree that resource downward (tenant to super) syncing will bring quite some confusions. Upward syncing (super to tenant) makes more sense.
In most cases, ResourceQuota is an isolation mechanism super wants to use to control tenant behaviors. This is similar to OS system ---- admin can set quotas for user, user can check the quotas usage but cannot change the quota.
If super wants “the source of truth should be tenant”, they just do not turn on --extra-syncing-resources=resourcequota and api-server in each vc cluster can handle quota restriction. If super turns on ResourceQuota syncing. It means tenant should not be allowed to modify assigned quotas at all. Resource quota UWS will over-write the quota modified by tenant, which provides super an enforced resource isolation mechanism.

My question is why we need to sync the quota to tenant? The tenant cannot consume over the super resource quota anyway.  Will sending an event to tell tenant that the Pod cannot be run because it violates the super limitation be enough? More importantly, tenant is fully open to tenant, so any overwrite to tenant created quota resources will be very confusing to the tenant users.
Normally the capacity planning has to be done between cluster-admin and tenant and it is fairly reasonable that tenant choose a smaller assigned quota value in tenant namespace for different reasons. The tenant quota and super quota are not necessary to be identical.
Another thought is that it is not practical for super to assign quota to each tenant namespace since the number of namespace can grow. It is hard to control overall tenant resource usage by setting the quota per namespace assuming this is the ending goal. Tenant can just bypass the limitation by creating more namespaces. Super needs to implement another webhook to do the check instead of relying on the quota system.

"sync the quota to tenant" from super is just one option. If this use case is not very common, we probably do not need bother to implement resource quota UWS.
Cluster-admin and tenant do their capacity planning/quota management separately without sync is another option.


New login to Twitter from ChromeMobileAndroid on Android

I want to automate an account. Everytime I login I get this notification from twitter. Will I be banned if I login frequently? Like every one to two minutes to publish a tweet?

It might look a bit suspicious. I've tested this before and I've gotten a few email confirmation challenges. It's hard to predict the frequency at which they issue these challenges, or what indicators they have. I would suggest just modifying your program, letting your account object have global scope for example.


WIP: ENH: Detect the GRIB files by the filename extension.

_get_default_engine chooses the engine suitable to open the path, ATM it only detects URL's. In this PR I propose to also detect GRIB files by the extension.
The whole engine detection strategy looks a bit fragile as the .gz extension is recognised outside of _get_default_engine and the nested ifs and try / except blocks make the code poorly readable. I can add a bit of a refactor in this area if you think it is appropriate.

[ ] Closes #xxxx
[ ] Tests added
[ ] Fully documented, including whats-new.rst for all changes and api.rst for new API

Hello @alexamici! Thanks for submitting the PR.

There are no PEP8 issues in the file xarray/backends/api.py !

There are no PEP8 issues in the file xarray/core/utils.py !

The whole engine detection strategy looks a bit fragile as the .gz extension is recognised outside of _get_default_engine and the nested ifs and try / except blocks make the code poorly readable. I can add a bit of a refactor in this area if you think it is appropriate.

Agreed! Yes, this would be welcome.
I'm happy with this change, but it would be good to ensure we have test coverage. It looks like we currently don't have any unit tests, but I'm pretty sure the other code is exercised by integration tests. If you're up for adding unit tests for _get_default_engine that would be awesome.

Mm... the private functions in api.py are not unit tested as they are not part of the API, I'll add a test_backends_api.py for the _get_default_engine tests and then other can use it to unit test the other _ functions.

Actually, thinking about this a little more, I'm not entirely sure it makes sense to fallback from using pynio to cfgrib. The problem is that these libraries may interpret GRIB files differently, e.g., with different variable names. It could make for a poor user experience for open_dataset('mydata.grib') to work differently when different dependencies are installed.
(please let me know if I'm mistaken here!)

@shoyer you are right, the GRIB translation is not unique and with cfgrib we have no plan to keep compatibility with PyNIO. I also agree that this will be really poor user experience 😞in some circumstances (even if in general it will work just fine). Do not merge yet!
We may default to one of the two without fall-back if it is not present (you may guess that I have a bias on which one to choose 😄), or keep the current behaviour.
In any even we can improve the error message by suggesting to add the engine= corresponding to the installed backends.
I'll show you what I mean.

As someone who hasn't opened up a grib file for several years, I'm not in a good position to choose which backend to use by default. For now, I would be inclined to hold off on making a default choice (unless there is a clear consensus among other xarray devs/users) in favor of requiring users to be explicit. This is somewhat inline with the Zen of Python: "In the face of ambiguity, refuse the temptation to guess."
Probably the best user experience would be to raise an informative error suggesting the use of cfgrib or pynio when opening a grib but no engine is specified, e.g.,
>>> xarray.open_dataset('data.grib')
ValueError: cannot open GRIB files without explicitly setting engine='pynio' or engine='cfgrib' in xarray.open_dataset()

Totally agree with what @shoyer just wrote. I will add the pynio team is considering mothballing pynio in the relatively near future. So long term, we may not have to make a choice here but for now, I support asking the user to make an explicit decision.
cc @khallock

@shoyer, I also agree that requiring a backend to be explicitly selected for grib files would be a better solution from a user experience perspective, as opposed to silently replacing PyNIO with cfgrib (inconsistent variable names would be a major concern).
As @jhamman mentioned, we have been evaluating the long-term viability of PyNIO and are considering putting it into "maintenance mode". Unfortunately, the lead developer of PyNIO retired last year, and we don't have the budget to replace him at this time.
We intend to continue fixing bugs in PyNIO as necessary (versions 1.5.1, 1.5.2, and 1.5.3 have all been released in the last three months), and we will also continue supporting the conda-forge/pynio-feedstock recipe. However, no new functionality (such as updating/adding grib tables) is part of our current long-term plan for PyNIO.
(cc'ing @marylhaley in case she has any additional input to offer)

And we "refuse to guess"!
I broke the forrest of try / except to different functions with no real change (hopefully!) except _get_default_engine_grib.
Now opening a GRIB file without explicit engine raises, but suggests the user to use the installed backend or to install one if none is present.
@shoyer if this too complex? Alternatively I can always return the same, generic, error message.

@shoyer I consider https://github.com/pydata/xarray/pull/2492/commits/486a391915d3bf1af685ae9ef908c30d5e053851 in a mergiable state if you like it.
I updated the description of the PR to sync with the new aim.


Polar to Cartesian and vice-versa conversion

We probably cannot settle on an aligned approach on how to convert Polar to Cartesian systems and vice-versa. However can we at least propose something everybody can stand behind so that when someone uses ADM-OSC, the same results are achieved?

More detail in section 10.1 of this: https://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.2127-0-201906-I!!PDF-E.pdf
These maths are part of the EBU renderer here: https://github.com/ebu/ebu_adm_renderer/blob/master/ear/core/objectbased/conversion.py

That is the only official recommendation for conversion I'm aware of.  That is what we should recommend if we want to be most compatible with ADM.
Note that this projects polar coordinates onto a 5.0.4 layout with corner speakers at (0º, +/-30ª, +/-110ª).  Elevation is idiosyncratic, too.

Perhaps we should supply some boiler-plate code to help implementations? The EBU reference is Python. Should we add C/C++ too?

This is good for implicit listener position at x=0, y=0, z=0 and straight to the front (Y axis) head orientation.
Maybe worth to follow EBU work on AR, VR, head tracking and game engine compatibility to be able to define a more general approach to the transition from the physical world to individual perception and vis versa ?

I have created an open source project, written in Swift, that performs the conversion between polar and cartesian coordinate systems. You can find a link to the source code here.
I have attempted to keep the implementation as simple as possible, with the conversion happening within two small functions. The project comes with a demo app that allows the user to view the conversion. I hope that this approach will allow people to easily convert the maths to other languages, or use directly within their application.

@Daniel-Higgott This is a good start! The ADM spec implements a more complicated conversion: https://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.2127-0-201906-S!!PDF-E.pdf The ADM-OSC stakeholders haven't made a decision about whether to require that math.

I have created an open source project, written in Swift, that performs the conversion between polar and cartesian coordinate systems. You can find a link to the source code here.
I have attempted to keep the implementation as simple as possible, with the conversion happening within two small functions. The project comes with a demo app that allows the user to view the conversion. I hope that this approach will allow people to easily convert the maths to other languages, or use directly within their application.

Awesome!

I would propose that refering to the EBU recommendation and Daniel's example code we can safely close this issue, once both has been added to the official documentation.

I would be opposed to using a basic co-ordinate conversion algorithm as it doesn't represent positions within cartesian space from an allocentric perspective. The EBU algorithm should be implemented in order to ensure that, perceptually, we maintain spatial accuracy between rendering systems using the two different coordinate systems.
We implemented C++ code in the EPS to do conversion; https://github.com/ebu/ear-production-suite/tree/main/reaper-adm-extension/src/reaper_adm/coordinate_conversion


Defer extension cable system connection next frame

Why / Balance

To fix https://github.com/space-wizards/space-station-14/issues/22966 and to have more deterministic connectivity between power receivers and providers, which isn't dictated by the order in which entities are serialized/deserialized in map files.
Technical details

I believe that during a map initialization and probably when any grid is deserialized, the extension cable system makes confusing connections, dictated by the order in which entities are deserialized and initialized. The deferring allows all the other entities to spawn-in and participate as valid candidates for the extension cable system connections.
Media
I verified that the airlock on the Fland station, showcased in https://github.com/space-wizards/space-station-14/issues/22966 now connected to the cable that is directly underneath it, though I kinda forgotten to record that.

[x] this PR does not require an ingame showcase

Changelog

:cl:

fix: Power receivers now connect to the closest cable during a map initialization.

Funnily, the updated connection order broke the entity deletion.
Caught exception while trying to detach parent of entity 'LV power cable (19222/n19222, CableApcExtension)' to null.
System.Collections.Generic.KeyNotFoundException: Entity 19124 does not have a component of type Robust.Shared.GameObjects.MetaDataComponent
   at Content.Server.Power.EntitySystems.ExtensionCableSystem.ResetReceivers(ExtensionCableProviderComponent provider) in /home/runner/work/space-station-14/space-station-14/Content.Server/Power/EntitySystems/ExtensionCableSystem.cs:line 139
   at Content.Server.Power.EntitySystems.ExtensionCableSystem.Disconnect(EntityUid uid, ExtensionCableProviderComponent provider) in /home/runner/work/space-station-14/space-station-14/Content.Server/Power/EntitySystems/ExtensionCableSystem.cs:line 111
   at Content.Server.Power.EntitySystems.ExtensionCableSystem.OnProviderAnchorStateChanged(EntityUid uid, ExtensionCableProviderComponent provider, AnchorStateChangedEvent& args) in /home/runner/work/space-station-14/space-station-14/Content.Server/Power/EntitySystems/ExtensionCableSystem.cs:line 90
   at Robust.Shared.GameObjects.EntityEventBus.<>c__DisplayClass50_0`2.<SubscribeLocalEvent>g__EventHandler|0(EntityUid uid, IComponent comp, TEvent& args) in /home/runner/work/space-station-14/space-station-14/RobustToolbox/Robust.Shared/GameObjects/EntityEventBus.Directed.cs:line 282
   at Robust.Shared.GameObjects.EntityEventBus.<>c__DisplayClass62_0`1.<EntSubscribe>b__0(EntityUid uid, IComponent comp, Unit& ev) in /home/runner/work/space-station-14/space-station-14/RobustToolbox/Robust.Shared/GameObjects/EntityEventBus.Directed.cs:line 432
   at Robust.Shared.GameObjects.EntityEventBus.EntDispatch(EntityUid euid, Type eventType, Unit& args) in /home/runner/work/space-station-14/space-station-14/RobustToolbox/Robust.Shared/GameObjects/EntityEventBus.Directed.cs:line 584
   at Robust.Shared.GameObjects.EntityEventBus.RaiseLocalEventCore(EntityUid uid, Unit& unitRef, Type type, Boolean broadcast) in /home/runner/work/space-station-14/space-station-14/RobustToolbox/Robust.Shared/GameObjects/EntityEventBus.Directed.cs:line 217
   at Robust.Shared.GameObjects.EntityEventBus.RaiseLocalEvent[TEvent](EntityUid uid, TEvent& args, Boolean broadcast) in /home/runner/work/space-station-14/space-station-14/RobustToolbox/Robust.Shared/GameObjects/EntityEventBus.Directed.cs:line 195
   at Robust.Shared.GameObjects.EntitySystem.RaiseLocalEvent[TEvent](EntityUid uid, TEvent& args, Boolean broadcast) in /home/runner/work/space-station-14/space-station-14/RobustToolbox/Robust.Shared/GameObjects/EntitySystem.cs:line 160
   at Robust.Shared.GameObjects.SharedTransformSystem.Unanchor(EntityUid uid, TransformComponent xform, Boolean setPhysics) in /home/runner/work/space-station-14/space-station-14/RobustToolbox/Robust.Shared/GameObjects/Systems/SharedTransformSystem.Component.cs:line 156
   at Robust.Shared.GameObjects.SharedTransformSystem.SetCoordinates(Entity`2 entity, EntityCoordinates value, Nullable`1 rotation, Boolean unanchor, TransformComponent newParent, TransformComponent oldParent) in /home/runner/work/space-station-14/space-station-14/RobustToolbox/Robust.Shared/GameObjects/Systems/SharedTransformSystem.Component.cs:line 464
   at Robust.Shared.GameObjects.SharedTransformSystem.DetachParentToNull(Entity`2 entity, TransformComponent oldXform, Boolean terminating) in /home/runner/work/space-station-14/space-station-14/RobustToolbox/Robust.Shared/GameObjects/Systems/SharedTransformSystem.Component.cs:line 1415
   at Robust.Shared.GameObjects.EntityManager.RecursiveDeleteEntity(EntityUid uid, MetaDataComponent metadata, TransformComponent transform, TransformComponent parentXform) in /home/runner/work/space-station-14/space-station-14/RobustToolbox/Robust.Shared/GameObjects/EntityManager.cs:line 557

My best guess is that the LinkedReceivers list isn't maintained correctly and also depends on the initialization order.

Cant wait until this gets merged though, this will be so useful.


cfreate custom role in questionnare

create custom role in questionare
uncomment this line https://github.com/EscolaLMS/ts-models/blob/main/config/typescript.php#L111

https://github.com/EscolaLMS/Questionnaire/commit/509ff0e772f5960280d1985de92f2693c7d9bc74

you mean custom rule in this task


Bug: POST with data in it fails a lot

If use JQuery:
$.post(url,{value:someData},
       function(data,status){
       console.log("\nData: " + data + "\nStatus: " + status);
});

This successes every time. And I can get the right data from back-end server.
But if I use this:
this.$http.post(url, {value:someData}).then(function (response) {

 }, function (response) {

 }
);

Almost failed every time. And I can NOT even get the data from back-end server.
url is the url I want to send. someData is the data I want to send.

Why does it fail? What errors are logged in the console and what is the response from the server?

Make sure your server can work with application/json requests, see https://github.com/vuejs/vue-resource/blob/master/docs/config.md#legacy-web-servers


[zh]update daemonset.md

It seems a invalid link:/zh/docs/concepts/configuration/pod-priority-preemption/
 ./scripts/lsync.sh content/zh-cn/docs/concepts/workloads/controllers/daemonset.md
content/zh-cn/docs/concepts/workloads/controllers/daemonset.md is still in sync

This page was still in sync before directory renaming.
/lgtm
/approve


Purple Heroku button

The Purple Heroku button leads to a empty deployment page

It's working for me:

Make sure you actually log in when if redirects you to their login page after you click it


FENews in 2019

Codecov Report

Merging #17 into WIP will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##              WIP      #17   +/-   ##
=======================================
  Coverage   87.05%   87.05%
=======================================
  Files          28       28
  Lines         139      139
  Branches        7        7
=======================================
  Hits          121      121
  Misses         18       18

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b97d323...77adc2c. Read the comment docs.


HDDS-7858. implement getStatus

What changes were proposed in this pull request?
Report correct quota, used bytes and available quota by implementing getStatus in both BasicRootedOzoneFileSystem and BasicOzoneFileSystem.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7858
How was this patch tested?
Unit tests/Manual tests

Thanks @sumitagrawl for the review, you mean handling the volume case is not required or implementing getStatus is not useful at the moment?


Strange javac "automatic module" warning for benchmark-jmh

Description
I suddenly noticed this warning when running top-level ./gradlew jar in Lucene main:
/s1/l/trunk/lucene/benchmark-jmh/src/java/module-info.java:20: warning: requires directive for an automatic module
  requires jmh.core;
              ^
1 warning

I think it is related to java modules but I'm not sure how to fix/suppress it :)

@mikemccand gave it a shot at #13997


coredump in SSL_dup

We got a coredump issue when using SSL_dup, the test code is primarily as follows:
ctx = SSL_CTX_new(TLS_client_method());
ssl = SSL_new(ctx);
tcp connect...
SSL_set_fd(ssl, fd);
ssl_dup = SSL_dup(ssl);
all the previous operations are ok, but we got :

the input of BIO_up_ref is a NULL pointer, corresponding code is ,
https://github.com/openssl/openssl/blob/a28e4890eed847e6122a1c4d50653566e0813f45/ssl/ssl_lib.c#L3841
now s->rbio and s->wbio are the same and not NULL (set by SSL_set_fd). and the following code tries to duplicate rbio from s->rbio, but we find that BIO_dup_state does nothing for option BIO_CTRL_DUP,
https://github.com/openssl/openssl/blob/a28e4890eed847e6122a1c4d50653566e0813f45/ssl/ssl_lib.c#L3833
https://github.com/openssl/openssl/blob/a28e4890eed847e6122a1c4d50653566e0813f45/crypto/bio/bss_sock.c#L175
so ret->rbio keeps NULL, causing Segmentation fault when passed to BIO_up_ref.

Congratulations! AFAICT you seem to have run into a bug that is over 20 years old and predates OpenSSL itself (it comes from its predecessor SSLeay)! It amazes me that no one has encountered and fixed this before now.
It looks to me like you only run into this bug if you attempt an SSL_dup() on an SSL object that hasn't yet been used to attempt a connection, but after the BIOs have been set.
BIO_dup_state is supposed to duplicate the state from one BIO object to another. However it assumes that the destination BIO has already been created. Looking at the SSL_dup code it doesn't attempt to create new BIOs at all! It just goes straight into BIO_dup_state() - which as you note is unimplemented for a socket BIO. If it had been implemented it probably would have crashed there since the destination BIO is NULL at this point.
I guess the fix would be to just use BIO_up_ref instead and copy the BIO pointers in the source SSL to the destination SSL directly without calling BIO_dup_state() at all.

Fix in #8161

Issue seems fixed


change in icons for the menu

Issue Type: Bug
after the latest update observed tick marks in the menu instead of the ">" sign to indicate there is a submenu.
is this expected change? i have pinned the menu in the sidebar.

VS Code version: Code - Insiders 1.45.0-insider (6fc9bd19c5a3c9c42d9a5c23152b1680bc308a57, 2020-04-22T06:26:58.340Z)
OS version: Windows_NT x64 10.0.17134
Remote OS version: Linux x64 4.15.0-96-generic

System Info

Item
Value

CPUs
Intel(R) Core(TM) i5-4210U CPU @ 1.70GHz (4 x 2394)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: disabled_offprotected_video_decode: unavailable_offrasterization: enabledskia_renderer: disabled_off_okvideo_decode: enabledviz_display_compositor: enabled_onviz_hit_test_surface_layer: disabled_off_okwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.88GB (9.00GB free)

Process Argv

Screen Reader
no

VM
0%

Item
Value

Remote
SSH: angular_serve

OS
Linux x64 4.15.0-96-generic

CPUs
Intel Xeon E312xx (Sandy Bridge, IBRS update) (4 x 2399)

Memory (System)
7.79GB (4.41GB free)

VM
0%

Extensions (20)

Extension
Author (truncated)
Version

remote-containers
ms-
0.112.0

remote-ssh
ms-
0.51.0

remote-ssh-edit
ms-
0.51.0

remote-wsl
ms-
0.44.2

vscode-remote-extensionpack
ms-
0.20.0

debugger-for-chrome
msj
4.12.6

ng-template
Ang
0.901.2

vscode-eslint
dba
2.1.5

gitlens
eam
10.2.1

EditorConfig
Edi
0.14.5

prettier-vscode
esb
4.4.0

node-debug2
ms-
1.42.2

typescript-javascript-grammar
ms-
0.0.53

vscode-typescript-tslint-plugin
ms-
1.2.3

vsliveshare
ms-
1.0.2000

vscode-xml
red
0.11.0

vscode-yaml
red
0.7.2

LiveServer
rit
5.6.1

sonarlint-vscode
Son
1.15.0

vscodeintellicode
Vis
1.2.6

I don't think that this one is intended.
/cc @misolori
May be something with codicons or styles.
I noticed that I have sometimes spinning codicon-tree-item-expanded instead of codicon-loading because former overrides styles of loader. Haven't noticed this problem before

@aeschli related to product icon overrides?

@misolori sorry for mention, I was not aware about changes in approach to use Codicons and thought that it might be unintentional font update.
Yes, looks like it was part of #95435 . PR with fix is available


Handle OSError due to in-use web server port automatically for the user

If the user does not specify a port when initializing VitessceWidget() and if the default port 8000 is already allocated, then try subsequent ports until an available one has been found.
This should allow the average user to ignore the port argument.

It might be good to show a warning if the default port is in use.


latex.rs:53-56: Add support of @ExtendsWith from JUnit in...

The puzzle 41-8e9aa4b2 from #41 has to be resolved:
https://github.com/h1alexbel/fakehub/blob/fe9cf78769e95b1d2a27a298988922a95950044b/server/src/report/latex.rs#L53-L56
The puzzle was created by @h1alexbel on 26-Jun-24.
Estimate: 60 minutes,  role: DEV.
If you have any technical questions, don't ask me, submit new tickets instead. The task will be "done" when the problem is fixed and the text of the puzzle is removed from the source code. Here is more about PDD and about me.

@0pdd thanks for the report, here is a feedback: * The todo seems to be asking for adding support of...

I see you're talking to me, but I can't reply since I'm not a chat bot.

@l3r8yJ can you help here?


Helix fail if exceptions thrown during test discovery

Summary of the changes (Less than 80 chars)

Detail 1
Detail 2

Addresses #bugnumber (in this specific format)

Oops meant to do draft


Add option to configure AppUserId from Initialize

Currently Initialize method in RevenueCatBillingiOS (I'm not sure about Android impl detail) doesn't accept AppUserId, but Purchases.ConfigureWithAPIKey accepts that parameter and we can initialize the AppUserId from startup instead of doing that via Login.

So If I understand this correctly you would like to have 2 methods:
current one: Initialize(string apiKey) and
new one: Initialize(string apiKey, string appUserId)
Correct ?

Yes, exactly. This way, we can initialize the plugin using appUserId in first place in the DI, and as far as I understood, that doesn't need to be async unlike the Login method.

This month I am quite busy so I wont be able to implement this. But feel free to create a PR

Ok I got to that sooner because I needed to adjust one thing.
Your proposal was Implemented in:
https://github.com/Kebechet/Maui.RevenueCat.InAppBilling/commit/772a40375b011c884ea0940c3c9262cf4d23db95
and will be released as v4.5.0

@MoienTajik does it work correctly ?


Master installation failling - "Unable to find sql file"

Hello,
I am trying to install latest master Midas on Ubuntu 14.04 LTS.
$ git clone https://github.com/midasplatform/Midas.git Midas
$ chmod a+w core/configs/ data/ log/ tmp/
$ curl -sS https://getcomposer.org/installer | php
$ php composer.phar install

I go to the index page to run the installation step but during step2, it fails given at step3 the following message:

Unable to find sql file
In /media/data/felixveysseyre/Projects/Midas/core/controllers/InstallController.php, line: 115
At 11:36:53 2016-02-10

I do not know if it is related, but the install.step2.js is not loaded correctly:

Thanks,
Félix

The latest changes corrected it.


700 extend vuejs small variant filter form with remaining tabs

Codecov Report
Base: 78.19% // Head: 78.19% // No change to project coverage :thumbsup:

Coverage data is based on head (6573ced) compared to base (7a303f4).
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #738   +/-   ##
=======================================
  Coverage   78.19%   78.19%
=======================================
  Files          15       15
  Lines        1330     1330
  Branches      307      307
=======================================
  Hits         1040     1040
  Misses        290      290

Flag
Coverage Δ

javascript
78.19% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Unable to create BTS Azure integration

Hello
I have similar to this issue (RP V5.7 | What is Project Key and Which API Key to Use #40)
My Azure is not located on the dev.azure.com, I have link smth like https://tfs.projectName.com/tfs/projectNameCollection/
Do I have any ability to use in "Link to BTS" any url that differs from dev.azure.com?

Response:
{"errorCode":40302,"message":"Impossible interact with integration. Unable to connect to Azure DevOps. Code: 400, Message: A potentially dangerous Request.Path value was detected from the client (:)."}

@Dalalaler, have you managed to resolve the issue?

@Dalalaler, have you managed to resolve the issue?

Suddenly, no(


Firebase Admin SDK sends empty message with native image

I'm not quite sure if this issues belongs here or to the firebase-admin-java repo, so please let me know and i will open this issue there.
I have a small java app which sends a basic FCM message to a specific device token.
I'm using firebase-admin:7.1.1 and google-cloud-graalvm-support:0.3.0 as dependencies and this code snippet to send the message:
FirebaseApp app = FirebaseApp.initializeApp();
Message message = Message.builder()
            .setToken(devicetoken)
            .putData("body", "This is a test message")
            .build();
FirebaseMessaging.getInstance(app).send(message);

This works fine when building and running with Java 11.0.10.
But when I build and run a native image with GraalVM Java 11 <IP_ADDRESS> I get a "400 Bad Request" response from the FCM backend (see exception below).
So I took a look at the network traffic and found that the message sent (with the native image) is empty.
I'm quite new to GraalVM and have no idea how to debug or find the cause of this.
Java:
Request body (token omitted)
{
  "message":{
    "data":{
      "body":"This is a test notification"
    },
    "token":"[...]"
  }
}

Request headers (Authorization omitted)
Accept-Encoding: gzip
Authorization: Bearer [...]
User-Agent: Google-HTTP-Java-Client/1.36.0 (gzip)
x-goog-api-format-version: 2
x-firebase-client: fire-admin-java/7.1.1
Content-Type: application/json; charset=UTF-8
Host: fcm.googleapis.com
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
Content-Length: 233

Native:
Request body
{
  "message":{}
}

Request header (Authorization omitted)
Accept-Encoding: gzip
Authorization: Bearer [...]
User-Agent: Google-HTTP-Java-Client/1.36.0 (gzip)
x-goog-api-format-version: 2
x-firebase-client: fire-admin-java/7.1.1
Content-Type: application/json; charset=UTF-8
Host: fcm.googleapis.com
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
Content-Length: 14

Exception
Exception in thread "main" com.google.firebase.messaging.FirebaseMessagingException: Unexpected HTTP response with status: 400
{
  "error": {
    "code": 400,
    "message": "Recipient of the message is not set.",
    "status": "INVALID_ARGUMENT",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.BadRequest",
        "fieldViolations": [
          {
            "field": "message",
            "description": "Recipient of the message is not set."
          }
        ]
      },
      {
        "@type": "type.googleapis.com/google.firebase.fcm.v1.FcmError",
        "errorCode": "INVALID_ARGUMENT"
      }
    ]
  }
}

        at com.google.firebase.messaging.FirebaseMessagingClientImpl$MessagingErrorHandler.createException(FirebaseMessagingClientImpl.java:293)
        at com.google.firebase.messaging.FirebaseMessagingClientImpl$MessagingErrorHandler.createException(FirebaseMessagingClientImpl.java:282)
        at com.google.firebase.internal.AbstractHttpErrorHandler.handleHttpResponseException(AbstractHttpErrorHandler.java:57)
        at com.google.firebase.internal.ErrorHandlingHttpClient.send(ErrorHandlingHttpClient.java:108)
        at com.google.firebase.internal.ErrorHandlingHttpClient.sendAndParse(ErrorHandlingHttpClient.java:72)
        at com.google.firebase.messaging.FirebaseMessagingClientImpl.sendSingleRequest(FirebaseMessagingClientImpl.java:127)
        at com.google.firebase.messaging.FirebaseMessagingClientImpl.send(FirebaseMessagingClientImpl.java:113)
        at com.google.firebase.messaging.FirebaseMessaging$1.execute(FirebaseMessaging.java:137)
        at com.google.firebase.messaging.FirebaseMessaging$1.execute(FirebaseMessaging.java:134)
        at com.google.firebase.internal.CallableOperation.call(CallableOperation.java:36)
        at com.google.firebase.messaging.FirebaseMessaging.send(FirebaseMessaging.java:104)
        at com.google.firebase.messaging.FirebaseMessaging.send(FirebaseMessaging.java:86)
        at fcmnativetest.Application.main(Application.java:23)
Caused by: com.google.api.client.http.HttpResponseException: 400 Bad Request
[response body again (omitted)]
        at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1113)
        at com.google.firebase.internal.ErrorHandlingHttpClient.send(ErrorHandlingHttpClient.java:96)
        ... 9 more

Site Note
To get the firebase-admin-sdk (the message sending) working at all I had to add
{
  "resources": [
    {
      "pattern": "admin_sdk\\.properties"
    }
  ]
}

to my META-INF/native-image/resource-config.json

Thanks for sharing this. Indeed looks like a missing configuration issue on our end.
We just released version 0.4.0 -- Could you try that and see if it fixes the problem? We changed how we registered JSON-related classes for reflection so it might help.
Let me know if that helps; if not I can try to reproduce on my end. We can also add the admin_sdk resource as one of the built-in resources.

Thanks for the quick response. I don't know how I missed the new release 😞
Unfortunately it didn't fix the problem.
I have pushed a minimal reproducible example here: veN1337/fcm-native-test
The container from the provided Dockerfile needs two environment variables:

GOOGLE_APPLICATION_CREDENTIALS path to the service account private key
FCM_DEVICETOKEN device token to send the message to

Great, thank you for the reproducer. I'll take a look.

Hey, I think I might have found a fix. For the temporary solution, I forked your project and added an additional reflect-config.json which seemed to work for this case.
For the long term solution, I think we will need more time to figure out all of the classes being used reflectively in the library to add this to the project. What makes this tricky is that the reflection used here does not throw error immediately. The firebase library uses reflection to iterate over the fields of an object to populate a JSON request, so if reflection fails, it will simply iterate over an empty list in the first part, and then the error shows up as a "missing request field" later down the line.
There is a good tool to help with debugging called the Native Image Agent that we use, maybe this can help you too. In the fork, I ran the app with normal JVM:
mvn package
java -agentlib:native-image-agent=config-output-dir=native-image-config -jar target/fcmnativetest-0.1.jar

It will generate native-image config files (often generates too much) but it narrows down the choices of what config you need to add.
I'll keep this issue open for now until we add full support for the fire base libs.

By the way, is there a way to generate FCM_DEVICETOKEN? (Or perhaps is there an emulator we should use?) Not too familiar with Firebase APIs.
I would be interested in adding in a sample application for our native-image tests to support this library.

Thank you so much! The additional reflect-config.json did the trick and sending messages works now.
Also the Native Image Agent you mentioned looks really helpful for (my) future GraalVM problems. Thanks again :)
I'm not quite sure if I understand your last question correctly and I only have very, very basic knowledge of Firebase messaging. If it helps I would be happy to provide a small Android app which, when run in an Android emulator, generates a FCM_DEVICETOKEN and receives messages.

No problem, glad it works!
Gotcha, when I was debugging your app, I didn't have a valid FCM_DEVICETOKEN to pass to the API; so I just made one up, however once I got past your error I started getting an "invalid token error". So I was curious if there was a way to register test/dummy tokens in Firebase or something to let you try calling the APIs and see what happens.
I don't think we are looking for an Android app but rather an sample native-image application which exercises some of those Firebase app calls on a test device token. But don't worry about trying to contribute this; I'm just curious on how to exercise the APIs from an integration testing context.

Closing as issue has been resolved.


Use CMAKE_CROSSCOMPILING_EMULATOR to run cross compiled executables

Brief Issue Summary
When a program is cross compiled and I try to run it (Run in terminal), CMakeTools run it directly without taking into consideration that there is an emulator set.
Example:
set(CMAKE_CROSSCOMPILING_EMULATOR ${CMAKE_SYSROOT}/tools/HEXAGON_Tools/8.0.10/Tools/bin/hexagon-sim FORCE "The simulator")
Expected:
The emulator should be used to run the executable
Apparent Behavior:
The executable is run directly on the host system
Platform and Versions
**Operating System**: Linux
**CMake Version**: 3.10.2
**VSCode Version**: 1.27.2
**CMake Tools Extension Version**: 1.1.2
**Compiler/Toolchain**: Custom (Based on CLang 3.9.0)

Good idea! I'll add it to 1.2.0.

Is there any news on this issue?
I would really love to have it. In combination with docker multi-platform support this would faciliate cross platform development extremely and give a great IDE experience.
In the meantime, is there any workaround to use an emulator when using VSCode and CMake with presets?