Concept for Workflow "Invite new CX participant by member" (ext: CXAR-806)

Background

in the current implementation (Release 3.0) organizations /  companies are invited via the  CX operating environment (Role CX-Admin).
in the invitation process the following data is entered by the CX-Admin:
** e-Mail address of representative of the company to be invited
** First- & Last-name of the representative of the company to be invited
** company name

Target scenario

in extension to the scenario "invite by CX Organization / Admin" a scenario "invite by existing member of Catena-X" shall  be provided
in the scenario "invite by existing member" a already registered member shall be able to send an invitation eMail to a not yet registered potential participant.

For the new scenario a concept is to be delivered which includes:

Roles and authorization concept for the “invitation” function.
Screens for data input of contact and sending invitation (see current screens for CX admin with eMail, Adress, Company Name)
Check based on provided contact may already be registered or is in the invitation / registration process
** If it is determined (based, for example, on company name ord  the domain of the e-mail address provided) that the organization is already registered, a corresponding message will be sent and the invitation must be confirmed again
Sending eMail with parallel notification of an operating company (notification configurable by operating company)

Components: CPLP
Sprints: N/A
Fix Versions: N/A
StoryPoints:N/A
Attachments:N/A

still valuable ? or should be close this.

@MaximilianHauer in my opinion it still makes sense, maybe that's already existing with the onboarding service provider functionality in some points?


Changing loadChildren syntax (From string to import(...)) breaks pre-rendering

🐞 bug report
Description
Hi,

I recently updated my angular application from v7 to v8

Upon review, i noticed the "module" and "target" fields in tscongif did not update to "esnext" and "es2015" respectively - so I manually updated these keys.

I also noticed update did not automatically convert my lazy loading import syntax from string

(loadChildren: './home/home.module#HomeModule')
to functional
(loadChildren: () => import('./home/home.module').then(m => m.HomeModule)) , so I  proceeded to update lazy loading import syntax my self.
When I served the app locally, everything was working. However, when I tried to SSR the application for testing, an error was thrown =>

ERROR Error: Uncaught (in promise): TypeError: Cannot read property 'call' of undefined
TypeError: Cannot read property 'call' of undefined
at webpack_require (webpack:///./dist/myDayPwaApp-server/main.js?:26:30)
at Function.requireEnsure [as e] (webpack:///./dist/myDayPwaApp-server/main.js?:45:25)
at ɵ0 (webpack:///./dist/myDayPwaApp-server/main.js?:2517:38)
at RouterConfigLoader.loadModuleFactory (webpack:///./node_modules/@angular/router/fesm5/router.js?:3684:39)
at RouterConfigLoader.load (webpack:///./node_modules/@angular/router/fesm5/router.js?:3669:35)
at MergeMapSubscriber.eval [as project] (webpack:///./node_modules/@angular/router/fesm5/router.js?:2672:47)
at MergeMapSubscriber._tryNext (webpack:///./node_modules/rxjs/_esm5/internal/operators/mergeMap.js?:71:27)
at MergeMapSubscriber._next (webpack:///./node_modules/rxjs/_esm5/internal/operators/mergeMap.js?:61:18)
at MergeMapSubscriber.Subscriber.next (webpack:///./node_modules/rxjs/_esm5/internal/Subscriber.js?:63:18)
at Observable.eval [as _subscribe] (webpack:///./node_modules/rxjs/_esm5/internal/util/subscribeToArray.js?:7:24)
at resolvePromise (webpack:///./node_modules/zone.js/dist/zone-node.js?:851:31)
at resolvePromise (webpack:///./node_modules/zone.js/dist/zone-node.js?:808:17)
at eval (webpack:///./node_modules/zone.js/dist/zone-node.js?:912:17)
at ZoneDelegate.invokeTask (webpack:///./node_modules/zone.js/dist/zone-node.js?:422:31)
at Object.onInvokeTask (webpack:///./node_modules/@angular/core/fesm5/core.js?:24730:33)
at ZoneDelegate.invokeTask (webpack:///./node_modules/zone.js/dist/zone-node.js?:421:60)
at Zone.runTask (webpack:///./node_modules/zone.js/dist/zone-node.js?:194:47)
at drainMicroTaskQueue (webpack:///./node_modules/zone.js/dist/zone-node.js?:600:35)
at ZoneTask.invokeTask (webpack:///./node_modules/zone.js/dist/zone-node.js?:501:21)
at ZoneTask.invoke (webpack:///./node_modules/zone.js/dist/zone-node.js?:486:48) {
rejection: TypeError: Cannot read property 'call' of undefined
at webpack_require (webpack:///./dist/myDayPwaApp-server/main.js?:26:30)
at Function.requireEnsure [as e] (webpack:///./dist/myDayPwaApp-server/main.js?:45:25)
at ɵ0 (webpack:///./dist/myDayPwaApp-server/main.js?:2517:38)
at RouterConfigLoader.loadModuleFactory (webpack:///./node_modules/@angular/router/fesm5/router.js?:3684:39)
at RouterConfigLoader.load (webpack:///./node_modules/@angular/router/fesm5/router.js?:3669:35)
at MergeMapSubscriber.eval [as project] (webpack:///./node_modules/@angular/router/fesm5/router.js?:2672:47)
at MergeMapSubscriber._tryNext (webpack:///./node_modules/rxjs/_esm5/internal/operators/mergeMap.js?:71:27)
at MergeMapSubscriber._next (webpack:///./node_modules/rxjs/_esm5/internal/operators/mergeMap.js?:61:18)
at MergeMapSubscriber.Subscriber.next (webpack:///./node_modules/rxjs/_esm5/internal/Subscriber.js?:63:18)
at Observable.eval [as _subscribe] (webpack:///./node_modules/rxjs/_esm5/internal/util/subscribeToArray.js?:7:24),
promise: ZoneAwarePromise [Promise] {
__zone_symbol__state: 0,
__zone_symbol__value: TypeError: Cannot read property 'call' of undefined
at webpack_require (webpack:///./dist/myDayPwaApp-server/main.js?:26:30)
at Function.requireEnsure [as e] (webpack:///./dist/myDayPwaApp-server/main.js?:45:25)
at ɵ0 (webpack:///./dist/myDayPwaApp-server/main.js?:2517:38)
at RouterConfigLoader.loadModuleFactory (webpack:///./node_modules/@angular/router/fesm5/router.js?:3684:39)
at RouterConfigLoader.load (webpack:///./node_modules/@angular/router/fesm5/router.js?:3669:35)
at MergeMapSubscriber.eval [as project] (webpack:///./node_modules/@angular/router/fesm5/router.js?:2672:47)
at MergeMapSubscriber._tryNext (webpack:///./node_modules/rxjs/_esm5/internal/operators/mergeMap.js?:71:27)
at MergeMapSubscriber._next (webpack:///./node_modules/rxjs/_esm5/internal/operators/mergeMap.js?:61:18)
at MergeMapSubscriber.Subscriber.next (webpack:///./node_modules/rxjs/_esm5/internal/Subscriber.js?:63:18)
at Observable.eval [as _subscribe] (webpack:///./node_modules/rxjs/_esm5/internal/util/subscribeToArray.js?:7:24)
},
zone: Zone {
_parent: Zone {
_parent: null,
_name: '',
_properties: {},
_zoneDelegate: [ZoneDelegate]
},
_name: 'angular',
_properties: { isAngularZone: true },
_zoneDelegate: ZoneDelegate {
_taskCounts: [Object],
zone: [Circular],
_parentDelegate: [ZoneDelegate],
_forkZS: null,
_forkDlgt: null,
_forkCurrZone: [Zone],
_interceptZS: null,
_interceptDlgt: null,
_interceptCurrZone: [Zone],
_invokeZS: [Object],
_invokeDlgt: [ZoneDelegate],
_invokeCurrZone: [Circular],
_handleErrorZS: [Object],
_handleErrorDlgt: [ZoneDelegate],
_handleErrorCurrZone: [Circular],
_scheduleTaskZS: [Object],
_scheduleTaskDlgt: [ZoneDelegate],
_scheduleTaskCurrZone: [Circular],
_invokeTaskZS: [Object],
_invokeTaskDlgt: [ZoneDelegate],
_invokeTaskCurrZone: [Circular],
_cancelTaskZS: [Object],
_cancelTaskDlgt: [ZoneDelegate],
_cancelTaskCurrZone: [Circular],
_hasTaskZS: [Object],
_hasTaskDlgt: [ZoneDelegate],
_hasTaskDlgtOwner: [Circular],
_hasTaskCurrZone: [Circular]
}
},
task: ZoneTask {
_zone: Zone {
_parent: [Zone],
_name: 'angular',
_properties: [Object],
_zoneDelegate: [ZoneDelegate]
},
runCount: 0,
_zoneDelegates: null,
_state: 'notScheduled',
type: 'microTask',
source: 'Promise.then',
data: ZoneAwarePromise [Promise] {
__zone_symbol__state: 0,
__zone_symbol__value: TypeError: Cannot read property 'call' of undefined
at webpack_require (webpack:///./dist/myDayPwaApp-server/main.js?:26:30)
at Function.requireEnsure [as e] (webpack:///./dist/myDayPwaApp-server/main.js?:45:25)
at ɵ0 (webpack:///./dist/myDayPwaApp-server/main.js?:2517:38)
at RouterConfigLoader.loadModuleFactory (webpack:///./node_modules/@angular/router/fesm5/router.js?:3684:39)
at RouterConfigLoader.load (webpack:///./node_modules/@angular/router/fesm5/router.js?:3669:35)
at MergeMapSubscriber.eval [as project] (webpack:///./node_modules/@angular/router/fesm5/router.js?:2672:47)
at MergeMapSubscriber._tryNext (webpack:///./node_modules/rxjs/_esm5/internal/operators/mergeMap.js?:71:27)
at MergeMapSubscriber._next (webpack:///./node_modules/rxjs/_esm5/internal/operators/mergeMap.js?:61:18)
at MergeMapSubscriber.Subscriber.next (webpack:///./node_modules/rxjs/_esm5/internal/Subscriber.js?:63:18)
at Observable.eval [as _subscribe] (webpack:///./node_modules/rxjs/_esm5/internal/util/subscribeToArray.js?:7:24)
},
scheduleFn: undefined,
cancelFn: undefined,
callback: [Function],
invoke: [Function]
}
}

I then reverted back to the string syntax for lazy loading children routes, and everything was working again.
I'd like to use the functional syntax for lazy loading children routes.
Thank you for any help.

I'm sorry, but we can't reproduce the problem following the instructions you provided.
Remember that we have a large number of issues to resolve, and have only a limited amount of time to reproduce your issue.
Short, explicit instructions make it much more likely we'll be able to reproduce the problem so we can fix it.
If the problem persists, please open a new issue following our submission guidelines.
A good way to make a minimal repro is to create a new app via ng new repro-app and add the minimum possible code to show the problem. Then you can push this repository to github and link it here.

This seems like a bug but we'll need to look at a reproduction to find and fix the problem. Can you setup a minimal repro please?
You can read here why this is needed. A good way to make a minimal repro is to create a new app via ng new repro-app and adding the minimum possible code to show the problem. Then you can push this repository to github and link it here.
This might be related to your directory structure so its really important to get an accurate repro to diagnose this.

Hi @alan-agius4 , I have set up a repro here =>
https://github.com/mazlano27/repro-app
To pre-render application for deployment -> please run 'npm run build:all'. Build will succeed.
However, when I change the way the routing is set up in 'app-routing.module.ts' (details commented in file lines 8 - 9), build fails.
I hope this helps to diagnose the issue?

Thanks @mazlano27, I can replicate the problem.

Hi, I looked at this and the problem is that in the compilerOptions of your tsconfig.server.json you don't have "module": "commonjs", which is fall backing to esnext modules which don't work on the server.

I had the same problem and I was watching this issue.
Creating a new project using ng new and adding ssr using ng add, doesn't add "module": "commonjs"  to tsconfig.server.json

@alan-agius4 Thank you for finding the fix - the issue is solved! With the updated syntax of how children routes are loaded, there is more consistency in how code is written which makes it more fulfilling to use Angular to develop apps.


Make handshake timeout configurable

Change to allow configuring handshake timeout to be a value other than 5 seconds.

Note that default value of ws.timeout is also 5 seconds.

👍 Seems reasonable,  thanks for the PR


Feature Request: Support Adobe Connect

@AlistairKeiller Can you provide an example URL of the Adobe Connect meeting?

The main scheme is https://pcadobeconnect.hosting.example/roomID, where hosting.example would change depending on who is hosting it ( for me it is the domain of my educational institution ).

Closed in https://github.com/Leko/crx-gcal-url-opener/tree/v1.2.0


fixtures: remove s3_fake_creds_file, allow for finer boto3 client configuration

By passing client_kwargs to the s3 temp upath factory, we can avoid having to monkeypatch s3 environment variable for the whole test session.

Codecov Report
Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (33fc10c) 85.96% compared to head (dba2b30) 84.59%.

Files
Patch %
Lines

src/pytest_servers/factory.py
87.50%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #137      +/-   ##
==========================================
- Coverage   85.96%   84.59%   -1.38%
==========================================
  Files          12       12
  Lines         399      370      -29
  Branches       57       52       -5
==========================================
- Hits          343      313      -30
- Misses         46       47       +1
  Partials       10       10

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Will it break things on the dvc-s3 side

Hopefully not :smile:: the API is unchanged, and the upath should have all it needs to properly communicate with the moto server

Hopefully not 😄: the API is unchanged, and the upath should have all it needs to properly communicate with the moto server

yep, just thinking if some code was relying on those env vars present. Anyways, we can find out that.

@efiop @dtrifiro @skshetry can we merge and release this please? :)

Thanks @dtrifiro ! 🙏

Looks like we did have some code that relied on the env vars and I'd expect that to be normal. It's probably reasonable for users to set them if they need them, although if they create a path with tmp_upath_factory.mktemp, they'll need to know to take values from path.fs.client_kwargs


The search cannot limit the scope, for example, just search for a folder

VSCode Version: 1.13.1
OS Version:  windows 7

Reproduces without extensions: Yes/No

You can right click on a folder and select "Find in folder", or expand the search details and use the search 'include' box.


any updates?

Hi, do you plan to update?

done just now lol

done just now lol

Hey thanks a lot!

yes


CEO-216 Add 'admin mode' switch

Purpose

Enable project admins to toggle on "Admin Mode." Includes:

Removing the "All" navigation option
New Switch component
Renames "My Analyzed Plots" to "Analyzed Plots"
Stores adminMode selection per project in localStorage.
Passing adminMode to the get-collection-plot endpoint
Displays the admin mode warning when initially viewing a project.

Related Issues
Closes CEO-216
Submission Checklist

[x] Commits include the JIRA issue and the #review hashtag (e.g. CEO-### #review <comment>)
[x] Code passes linter rules (npm run eslint/clj-kondo --lint src)

Testing

Given I am an admin, When I view a project, Then I can toggle on "admin mode."

Screenshots

Warning message

Admin mode switch

src/css/custom.css
src/js/components/Switch.js
geo-dash.css already has a .switch class so this will conflict.  This update needs to be matched with the geodash.  I was thinking radio buttons for "toggle".  The switch looks good, it just now adds an issue with the geodash.  If its quick to just update the geodash and remove the .switch css from geo-dash.css thats good.
src/js/collection.js
isAdminMode -> inAdminMode
168, this adds /n when its just the admin mode warning.  Put the /n on 172, 178.
326, Ill update my PR to use inAdminMode so you can use the shorthand notation for the object.
src/js/utils/preferences.js
You have other code to get and save from the local storage for the home help modal. Make sure that this file is generic enough to work with that and update the help modal local storage code.


Provide a way to refresh augury when using webpack Hot Module Replacement

I get the following message after updating with hmr:
Cannot find element associated with node 0 0 0

@aitboudad My initial thought is this is because HMR hotpatches your code and whatever that does, breaks augury from correctly parsing the Angular root element. We will need to research this in order to confirm it. A potential fix for you could be to introduce polling with webpack, for a full page-refresh.

check https://github.com/jschwarty/angularcli-hmr-example
npm run serve:hmr

@aitboudad Thank you, we will try to get to this as soon as possible

when updating the title of AppComponent the state component disappear

I have a difference behavior on my project but I couldn't reproduce it on the example app.

I solved my issue with hacky way :)
module.hot.dispose(() => {
    ....
    ngModule.injector.get(TestabilityRegistry)._applications.clear();
    ngModule.injector.get(NgZone)._onStable.emit();
   ...
});


Upgrade Hadoop version to 3.3.4

Which Delta project/connector is this regarding?

[X] Spark
[ ] Standalone
[ ] Flink
[ ] Kernel
[ ] Other (fill in here)

Description
Upgrade Hadoop version to 3.3.4 to alain with spark 3.5.0 version

How was this patch tested?
Unit test - SUCCEEDED

Does this PR introduce any user-facing changes?
No

@rbushri this is great. I think this is currently not working in 3.0.0 and also 2.4.0, since Spark 3.4 also uses this Hadoop version.
Have you synced with @scottsand-db? He has a PR open opened 1688, with additional changes.
Thanks. Happy to see this fixed.

@rbushri this is great. I think this is currently not working in 3.0.0 and also 2.4.0, since Spark 3.4 also uses this Hadoop version.
Have you synced with @scottsand-db? He has a PR open opened 1688, with additional changes.
Thanks. Happy to see this fixed.

I reviewed @scottsand-db's PR, and the additional changes involve adding an info log and making small cosmetic changes. I'm uncertain if these should be included in this PR since they're not directly related to the Hadoop version. However, if you think they should be included, I'll add them.

It is good to me as is

Could you add a "Fix #1935" to the PR description?
Any plans to merge this before 3.1 cut?

Could you add a "Fix #1935" to the PR description?
Any plans to merge this before 3.1 cut?

done.
I hope so too.


[mesheryctl] Fix for perf view

Signed-off-by: Piyush Singariya<EMAIL_ADDRESS>Description
Few stuff fixed with perf view
This PR fixes #
Notes for Reviewers
Signed commits

[ ] Yes, I signed my commits.

Codecov Report

Merging #3762 (410c015) into master (b64d564) will increase coverage by 0.45%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master    #3762      +/-   ##
==========================================
+ Coverage   20.59%   21.05%   +0.45%
==========================================
  Files          45       45
  Lines        3807     3819      +12
==========================================
+ Hits          784      804      +20
+ Misses       2817     2803      -14
- Partials      206      212       +6

Flag
Coverage Δ

gointegrationtests
10.02% <0.00%> (+0.44%)
:arrow_up:

unittests
12.75% <66.66%> (-0.05%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

mesheryctl/internal/cli/root/perf/list.go
25.56% <0.00%> (-0.72%)
:arrow_down:

mesheryctl/internal/cli/root/perf/view.go
64.34% <100.00%> (+0.27%)
:arrow_up:

mesheryctl/pkg/utils/helpers.go
18.58% <0.00%> (-0.17%)
:arrow_down:

mesheryctl/internal/cli/root/system/stop.go
40.00% <0.00%> (ø)

mesheryctl/internal/cli/root/system/error.go
0.00% <0.00%> (ø)

mesheryctl/internal/cli/root/system/start.go
54.81% <0.00%> (+0.16%)
:arrow_up:

mesheryctl/internal/cli/root/system/logs.go
21.59% <0.00%> (+21.59%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b64d564...410c015. Read the comment docs.


chore: collect static log features to a separate feature

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.
What's changed and what's your intention?
As titled. More discussion: #5721 .
Checklist

[x] I have written necessary rustdoc comments
[x] I have added necessary unit tests and integration tests
[x] All checks passed in ./risedev check (or alias, ./risedev c)

Refer to a related PR or issue link (optional)
close #5721

Or we can combine static-link and static-log-level into one crate called dist later.

@BugenZhao What about now?

LGTM. How about introducing the compile_error check?

LGTM. How about introducing the compile_error check?

Oh, I forgot it. Let me add it. 😀

It seems that the DEBUG log of aws_config::meta::credentials::lazy_caching in the release build doesn't suppress after this PR.  cc @MrCroxx
compactor log:
2022-10-12T03:23:15.535807Z DEBUG send_operation:provide_credentials: aws_config::meta::credentials::lazy_caching: loaded credentials from cache operation="PutObject" service="s3" provider=default_chain
2022-10-12T03:23:15.784546Z DEBUG send_operation:provide_credentials: aws_config::meta::credentials::lazy_caching: loaded credentials from cache operation="PutObject" service="s3" provider=default_chain
2022-10-12T03:23:15.955515Z DEBUG send_operation:provide_credentials: aws_config::meta::credentials::lazy_caching: loaded credentials from cache operation="PutObject" service="s3" provider=default_chain
2022-10-12T03:23:16.341963Z  INFO risingwave_storage::hummock::compactor: Finished compaction task in 1126.990332ms:
Compaction task id: 9, target level: 0
Compaction watermark:<PHONE_NUMBER>512000
Compaction target_file_size: 33554432
Compaction # splits: 1
Compaction task status: 2
Compaction Sstables structure:
Level 0 ["[id: 50, 9114KB]"]
...


preload does not work and downloaded twice on firefox

Describe the bug

Here is the firefox bug of module preload: https://bugzilla.mozilla.org/show_bug.cgi?id=1425310, i.e. firefox does not support modulepreload. There is a similar issue for sapper, https://github.com/sveltejs/sapper/issues/1387.

The new vite already has a builtin preload plugin https://github.com/vitejs/vite/blob/main/packages/vite/src/node/plugins/importAnalysisBuild.ts#L156-L162 on the client side. It will insert preload links despite sveltejs/kit also does that: https://github.com/sveltejs/kit/blob/1007f6730600b3f770c5062d385d539661d6d0c6/packages/kit/src/runtime/server/page/render.js#L104-L106.

Links from svelte are like /./_app/pages/$layout.svelte-3b4aa0d5.js, but vite checks /_app/pages/$layout.svelte-3b4aa0d5.js. So vite does not think the inserted link is duplicated. One fix is:
---<EMAIL_ADDRESS>+++<EMAIL_ADDRESS>@@ -229,8 +229,8 @@ async function build_server(

                find_deps(file, js_deps, css_deps);

-               const js = Array.from(js_deps).map((url) => prefix + url);
-               const css = Array.from(css_deps).map((url) => prefix + url);
+               const js = Array.from(js_deps).map((url) => path.join(prefix, url));
+               const css = Array.from(css_deps).map((url) => path.join(prefix, url));

                const styles = config.kit.amp
                        ? Array.from(css_deps).map((url) => {

Vite inserted links like <link rel="preload" as="script" crossorigin="" href="/_app/pages/$layout.svelte-a23c4e9e.js">, but it does not take effect somehow. So when sveltekit import scripts, resource will be downloaded again.

To Reproduce
Create an app with layout, build & start, and you should be able to see the twice downloading on the latest firefox.
Expected behavior
Despite of preloading or not, I only want one time downloading.
Information about your SvelteKit Installation:

Diagnostics
-
  System:
    OS: Linux 5.8 Noname Linux
    CPU: (16) x64 AMD Ryzen 7 4800H with Radeon Graphics
    Memory: 20.62 GB / 30.36 GB
    Container: Yes
    Shell: 5.0.7 - /bin/bash
  Binaries:
    Node: 14.13.1 - /bin/node
    Yarn: 1.22.10 - ~/.npm-packages/bin/yarn
    npm: 6.14.8 - /bin/npm
  Browsers:
    Firefox: 86.0.1
  npmPackages:
    @sveltejs/kit: next => 1.0.0-next.74
    svelte: ^3.37.0 => 3.37.0
    vite: ^2.1.5 => 2.1.5

Your browser: firefox 86.0.1

Your adapter: vercel, but I am experimenting with npm run start.

Severity
I think it can be a major bug, nobody like downloading things twice.

@benmccann PTAL

are you 100% sure you're not just seeing firefox request stuff from the service worker, and the service worker request stuff from the network?

(PS, don't ever ping maintainers with "PTAL" or similar. this is your first and final warning :)

are you 100% sure you're not just seeing firefox request stuff from the service worker, and the service worker request stuff from the network.

I am sure. One comes from preload-helper-xxx.js, which looks exactly same to the source code of vite. And the second request comes from the entry/startup script block.

don't ever ping maintainers with "PTAL" or similar

Are there some unspoken rules that I don't know :O? Sorry if that bothers you.

Are there some unspoken rules that I don't know :O? Sorry if that bothers you.

There are two reasons that that's considered rude. First, maintainers are automatically pinged when an issue is created, so it's redundant to ping them and ask them to take a look. It just adds to the email in their overflowing inbox, without adding anything new to the conversation, so it's essentially spam even if you didn't intend to spam them. Second, it feels like demanding that they pay attention to your issue instead of some other issue that they might think is more important right now. If you have a support contract with a company then it's fair to say "Hey, please prioritize this issue I'm having, because I'm paying you to get my issues considered as high-priority." But this is an open-source project where the maintainers are donating their time. Demanding that they fix your issue, instead of whatever issue they are currently working on, is rude when you're making that demand of volunteers. You probably didn't intend to sound demanding, but when you ping project maintainers and ask them to take a look, it comes across as demanding, especially when you ping them two minutes after creating your issue. If an issue has languished for two months then it is fair to ping someone because they may have forgotten about it, but pinging them two minutes after creating it (when there's no way they've had time to look at it yet) comes across as saying "Hey! I'm more important than everybody else and you should pay instant attention to anything I do!", even if that wasn't your intent.
So that's the unspoken rule that you didn't realize you were breaking. However, you did follow the issue template (well, almost: you forgot to fill out the "Information about your SvelteKit Installation" section, which would be useful information in this case since it might be important to know which version of Firefox you're using), so thank you for that. I've seen a lot of people just ignore the issue template (not just in the svelte-kit repo, but in many other repositories as well), so thank you for actually paying attention and filling it out. Please do fill out the one section you missed, the "Information about your SvelteKit Installation" section, as well, so that we can know your browser version and other relevant info. (See #1049 for an example of what it should look like).

First, maintainers are automatically pinged when an issue is created.

Oh, the subscription... I typically will not subscribe all the messages in a repository. And if any, there is mail aggregation and mail labeler, which makes it much less spamming.

Second, it feels like demanding that they pay attention to your issue instead of some other issue that they might think is more important right now.

Is it acceptable to ping someone in the issue body, instead of comments? benmccann filed that sapper issue, so I think he can help with the issue. Will it be better if it is started with do you have free time? Or maybe a chat room?

the "Information about your SvelteKit Installation" section

Updated.
BTW: I think I figured out the problem:

modulepreload is not working since firefox does not support.
Downloaded twice since the url is different:

<link rel="preload" as="script" crossorigin="" href="/_app/pages/$layout.svelte-7efdaaad.js"> // inserted by vite
nodes: [ // startup script
     import("/./_app/pages/$layout.svelte-7efdaaad.js"),

The relevant sveltejs/kit code here, changing to path.join(prefix, url) solved the problem.
But error.svelte is still one downloaded twice... I am trying to fix it.

Is it acceptable to ping someone in the issue body, instead of comments? benmccann filed that sapper issue, so I think he can help with the issue. Will it be better if it is started with do you have free time? Or maybe a chat room?

That will depend on the repo and how the maintainer team prefers to work. In some repos that can be acceptable, but it's usually unnecessary since benmccann is also a maintainer and will be auto-notified of new issues (unless he's chosen to turn that off). Here, since you have been requested by the repo owner not to do it, I'd avoid it altogether.
The Svelte Discord chat room is a good place to ping people if you think they will want to know about an issue. It's generally acceptable to ping people in a Discord chat room, since that does not create an email notification and they'll only see it when they next visit the chatroom. Though there, too, you should wait a few days before pinging anyone, since pinging people immediately comes across as demanding their time. (And "a few days" should be understood as "a few business days", e.g. allow extra time if it's the weekend, or if it's a major holiday in the country/countries the maintaner(s) are from.)
The Discord chat room is also a good place to mention something if you're not quite sure whether it's a bug or not. Good rule of thumb if you're not sure is to create a minimal example that reproduces the bug, either on the Svelte REPL or in a GitHub repo if you can't create a REPL reproduction (for example, anything related to routing can't be reproduced in the REPL). Then post a link to your bug reproduction on Discord and ask if that's a bug, or if you're making a mistake somewhere. Then if others agree that it's a bug, open an issue here.

Some discussion of the double download behavior here: https://bugzilla.mozilla.org/show_bug.cgi?id=1425310#c3

I think the line here should detect if it's Chrome vs Firefox/Safari and return modulepreload vs preload as appropriate:
https://github.com/sveltejs/kit/blob/61d7fa00022aa8e368b83b4e58b0694e90024ecc/packages/kit/src/runtime/server/page/render.js#L99

Support for modulepreload is pretty scattered according to https://caniuse.com/link-rel-modulepreload, wouldn't it be safer to opt-in Chrome and Chromium-based Edge specifically? I just ran into this double download issue with a bone stock npm init @svelte@next with starter app, but I may be the only person left on Earth who uses FF as their daily driver.

56k of JS while on Chrome I get 40k.

Well now I'm just confused, I commented out the modulepreload tabs but still see that double load. I'm not sure I trust the Firefox dev tools now. I might try running this through a proxy like Charles proxy.

And you're right @benmccann I see in the compiled client JS code it's inspecting link.relList.supports('modulepreload'), which I guess comes from https://github.com/vitejs/vite/blob/4112c5d103673b83c50d446096086617dfaac5a3/packages/vite/src/node/plugins/importAnalysisBuild.ts#L39, but for some reason I'm seeing modulepreload in Firefox even though it definitely reports no support for it. I don't know.

Oh wait, this is in the prerendered index.html file, client-side checks like this wouldn't and couldn't be in effect. Anyway this just sounds like a Vite issue, I'll look into their bug tracker.

I have filed an issue for vite, check https://github.com/vitejs/vite/issues/3133.

OK, but now I'm not sure if it's Vite or SvelteKit, not certain it has anything to do with modulepreload, because I comment out or just delete those link tags and I still see the issue. I confirmed the double load in Charles, here's a single page load:

You need to hack around the source code of sveltekit and vite both to eliminate the double download issue.

So I tried this on a somewhat more substantial project, something I'm working on at work, and it looks like only the vendor chunk and those two tiny layout and error files are duplicated, everything is single loaded, so it's like a 10kb penalty for Firefox that seems fairly constant as the app grows. I'm not too worried about this issue at the moment anymore.

i am seeing the same problem with latest FF 89.0.1

if anyone wants to check himself. https://notific.at (might be down, the site is online for testing for now)
unfortunately it actually loads every component two times for me. i haven't been able to check it with charles myself, but will do so.

This issue is still there and makes the lighthouse performance score lower. Even in the official SvelteKit site this is happening. After checking this on different browsers, what @benmccann is mentioning on https://github.com/sveltejs/kit/issues/1042#issuecomment-821918949 , it seems that the bug still exists.

Would anyone be able to test this with Vite 2.5 beta? They've added a modulepreload polyfill: https://github.com/vitejs/vite/pull/4058

I just tested with<EMAIL_ADDRESS>and this appears fixed now

Not sure if I should open a new issue on this, but I am seeing this issue with Firefox (103.0.2) on latest sveltekit/vite.
It seems it has crept up again.
To reproduce, you can check against https://kit.svelte.dev/ where the issue is visible in devtools/network. Some files are loaded twice and even three times (favicon). This is making showing 202kB (Chrome) vs 240kB (Firefox) transferred (with cache disabled) in 36 (Chrome) vs 50 (Firefox) requests.

Can reproduce on a very simple kit project. Reopening.

This seems like something that will have to be fixed in Vite. As a test, I removed all the <link rel="modulepreload"> elements using transformPageChunk, and saw the same result, which suggests the issue is Vite's module preloading logic. Not immediately sure what the fix is though.

Not sure if I should open a new issue on this, but I am seeing this issue with Firefox (103.0.2) on latest sveltekit/vite. It seems it has crept up again.
To reproduce, you can check against https://kit.svelte.dev/ where the issue is visible in devtools/network. Some files are loaded twice and even three times (favicon). This is showing 202kB (Chrome) vs 240kB (Firefox) transferred (with cache disabled) in 36 (Chrome) vs 50 (Firefox) requests.

I've the same issue on my website

Maybe this will help fix the issue?
https://github.com/vitejs/vite/pull/9938

This seems like something that will have to be fixed in Vite. As a test, I removed all the <link rel="modulepreload"> elements using transformPageChunk, and saw the same result, which suggests the issue is Vite's module preloading logic. Not immediately sure what the fix is though.

As someone that was involved in this Vite issue, I it was fixed in https://github.com/vitejs/vite/pull/9970, which is applied from vite>=3.2.0-beta.0.
Please test again and report if the issue continues.

Thanks for the update! I'll go ahead and close this now since it's been fixed in Vite. 3.2.0 final should be released soon

Vite 3.2 has been released


fix: Resolve doctrine deprecation notices

Resolves the following Doctrine deprecations on the EventArgs:

getEntityManager() -> getObjectManager()
getEntity() -> getObject()

Copied pull request here: https://github.com/DoctrineEncryptBundle/DoctrineEncryptBundle/pull/2
Please see these comments:

https://github.com/absolute-quantum/DoctrineEncryptBundle/pull/65#issuecomment-1476301156
https://github.com/absolute-quantum/DoctrineEncryptBundle/pull/65#issuecomment-1638006829

The new package that I hope will be maintained by more than 1 person


zh: Content/docs/reference/commands/node agent

add

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: fleeto
If they are not already assigned, you can assign the PR to them by writing /assign @fleeto in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

content_zh/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Hi @loverto. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

CI failed, please re-lint.
- ./public/about/community/customers/index.html
  *  External link https://www.trulia.com failed: 403 No error
- ./public/blog/2018/announcing-1.0/index.html
  *  External link https://www.trulia.com/ failed: 403 No error
- ./public/docs/reference/config/policy-and-telemetry/adapters/cloudmonitor/index.html
  *  External link https://cloudmonitor.console.aliyun.com/ failed: response code 0 means something's wrong.
             It's possible libcurl couldn't connect to the server or perhaps the request timed out.
             Sometimes, making too many requests at once also breaks things.
             Either way, the return message (if any) from the server is: Couldn't resolve host name
htmlproofer 3.9.2 | Error:  HTML-Proofer found 3 failures!
LINTING FAILED
Exited with code 1

/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: rootsongjc
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

content_zh/OWNERS [rootsongjc]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


orca not found on path using conda install

I just tried to install orca using the conda installation method on Windows 10 using Miniconda. Python version is 3.6.6 and the installed orca version is 1.2.0. E.g., I did:
conda install -c plotly plotly-orca
It installs fine, but does not put the executable on path nor in the Scripts folder of the environment. A quick search also reveals no executable file called orca (only some orca.py files inside the package).
npm install -g<EMAIL_ADDRESS>orca worked fine though (orca found on path afterwards) and also installs version 1.2.0.

Should be fixed now for<EMAIL_ADDRESS>Would you mind giving:
conda install -c plotly plotly-orca

another shot?

Done. It works!


Add license file

You mention the license in the README, but there's no license file.  This pull request adds a license file.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.James Vasile seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Sanitise embeded HTML from edusharing?!

See https://github.com/serlo/ece-as-a-service/blob/main/src/plugins/edusharing-asset.tsx#L117-L123

What is the thread here? XSS?

<script> is needed for H5P...


Token issues

So I am using token custom bars, token mold and cub. I believe a combination of these mods is causing a huge issue that your module is creating..
lLcSh0y698s2teZo which is a copy of a token that multi level token created.
foundry.js:2242 Error: No actor or token exists with id 'lLcSh0y698s2teZo'     at m.getEntity (TokenHotbar.js:1)     at m.get (TokenHotbar.js:1)     at T.removeTokenMacros (TokenHotbar.js:1)     at TokenHotbar.js:1     at Function._call (foundry.js:2239)     at Function.call [as __furnace_original_call] (foundry.js:2224)     at Function.callOriginalFunction (Patches.js:57)     at Function.<anonymous> (Debug.js:42)     at foundry.js:10380     at Array.reduce (<anonymous>)
Foundry VTT | Deleted 5 Tokens [lLcSh0y698s2teZo,YHtjnEChi6r12P0a,kL7AGpTLDr9MNCRD,dVHXK2lT48HUZQ3Z,4OH2TMxR6KIMqkHO] from parent Scene 0zIkmnGLn1ej4EkG
Then token mold throws up an error..
foundry.js:2242 TypeError: Cannot read property 'data' of null     at o._setCreatureSize (token-mold.js:537)     at o._setTokenData (token-mold.js:257)     at token-mold.js:240     at Function._call (foundry.js:2239)     at Function.call [as __furnace_original_call] (foundry.js:2224)     at Function.callOriginalFunction (Patches.js:57)     at Function.<anonymous> (Debug.js:42)     at Scene.createEmbeddedEntity (foundry.js:10137)     at multilevel.js:570
Probably trying to get the data off the fake token multi level created.
Cub also gets into the mix trying to read the fake token..
foundry.js:2242 TypeError: Cannot read property 'items' of null     at Function.hasFeat (actor.js:12)     at Function._checkForFeat (mighty-summoner.js:13)     at Function._onPreCreateToken (token.js:30)     at signal.js:113     at Function._call (foundry.js:2239)     at Function.call [as __furnace_original_call] (foundry.js:2224)     at Function.callOriginalFunction (Patches.js:57)     at Function.<anonymous> (Debug.js:42)     at Scene.createEmbeddedEntity (foundry.js:10137)     at multilevel.js:570

This is a somewhat-known issue. By necessity, MLT creates fake tokens that have no associated actor - this is something Foundry allows (you can set the "Actor" of a token to "None" in base foundry). Unfortunately, since it's not the usual case, it often trips other modules up, but there isn't much I can do about this in MLT itself.
I believe the fixes in Token Mold and CUB are simple, so I've put up PRs for those: https://github.com/Moerill/token-mold/pull/58 and https://github.com/death-save/combat-utility-belt/pull/260
I'm much less sure what's going on with the Token Hotbar module. What are you trying to do when that error shows up? It looks like the error must be coming from the preDeleteToken hook in Token Hotbar, but I'm not sure why that would get run for a fake token anyway.

Thanks for the mention! I'm not sure why it would show this error. Are the fake tokens part of the global array of tokens (canvas.tokens.placeables)? I suppose I technically don't need to throw an error, but the error is nice in case there is unexpected behaviour on Token Hotbar's side.
Apart from it filling up the console, there is nothing to worry about though. Maybe I'll change it into a warning in the meantime so it looks less severe.

@janssen-io although the tokens are "fake", apart from the missing actor link they're created like any other and shouldn't behave in an observably different way.
Your mention of the canvas gave me an inkling though. canvas.tokens.placeables only ever stores data for tokens on the scene you are currently viewing. Because Multilevel does automation in the background, it can end up creating and deleting tokens on scenes you are not actually looking at right now. Those tokens only have data in game.scenes.xxx.data.tokens, they won't show up in the canvas!

[...] Because Multilevel does automation in the background, it can end up creating and deleting tokens on scenes you are not actually looking at right now. Those tokens only have data in game.scenes.xxx.data.tokens, they won't show up in the canvas!

Ah that must be it! I'll have to see how I can fix that problem. Ideally, I don't want to go looking through all scenes to find the token that's being deleted...

Closing in favour of https://github.com/grandseiken/foundryvtt-multilevel-tokens/issues/34 which is specifically about the token hotbar issue


⚠ Bofrost.es has degraded performance

In 99a7f24, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1231 ms

Resolved: Bofrost.es performance has improved in d105c39 after 8 minutes.


Chrome Extension doesn't work on player.spotify.com, only on play.spotify.com

I am not sure where the issue is. My spotify web player is player.spotify.com even if I type play.spotify.com. It resolves back to player...
I sent the spotify html file to Paul and I hope that will provide some insight.
When I click the chrome soundbounce extension from the spotify webplayer tab, nothing happens.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/23274864-chrome-extension-doesn-t-work-on-player-spotify-com-only-on-play-spotify-com?utm_campaign=plugin&utm_content=tracker%2F9578624&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F9578624&utm_medium=issues&utm_source=github).

OK Yes this is definitely to do with player.spotify.com.  I've updated wiki to mention this as caveat.

I've updated the chrome extension description to warn of this limitation too.

@RossBlakeD Hi Ross, can you do me a favour, and follow the instructions here https://basecamp.com/help/guides/troubleshooting/creating-a-HAR-file to create a HAR file.
Please do it on the player.spotify.com tab - hit record in dev tools network tab, refresh page, click soundbounce extension, then "save with content" and post either here or to<EMAIL_ADDRESS>cheers.

There you go: https://copy.com/FVDpJT7JzCkrayay

Thanks @Papipo  - what is your spotify username?

@Papipo or anyone with player.spotify.com access - please can you do me a favour and open dev tools once the player has loaded, and paste in here your localStorage object once logged in, or just Object.keys(localStorage) if you're concerned about privacy.  I'll pick this up tomorrow.

I don't have access to the beta, but when I browse to that domain I seem to get redirected by javascript, not by a 302 header from the server. See HAR: https://www.dropbox.com/s/6y4cced5bsi80t1/andrew-player.spotify.com-load-including-redirect-to-play.spotify.com.har?dl=0

OK I have a potential fix here.  @Papipo please can you run this snippet in your spotify player dev tools, and it should display your spotify username.  If this works, I'll update the extension and we'll see if it logs in...
for(var key in localStorage){ if(key.indexOf('notifications-')==0) console.log(key.substr('notifications-'.length));}

On Windows 7, I currently can launch the extension from https://player.spotify.com and see the rooms.
When I click on a room, it recognizes and says, "Joining.." but nothing happens after that.

@RossBlakeD - glad we seem to have solved the authentication issue then.  To work out the joining issue, please can you confirm A) that you are not logged into soundbounce elsewhere and B) http://websocketstest.com/ works for you for port 443.

On MacOS X v 10.10.4, I can use extension to login to soundbounce and join and a room. I join the room and can chat and see tracks playing - no sound.
On win 7 - same as before. I can login. I can see rooms, click to join rooms but I am not able to join room or hear tracks.
A) Not logged in SB elsewhere
B) I think this means port 443 is supported. (see screenshot)

Unfortunately we don't support websockets using SSL, for now.  Support is planned in future but I don't have a timescale for that at all, sorry...

If / when we do move to wss:// (web sockets with ssl), we'll be able to embed soundbounce inside the spotify tab.  Lack of support for SSL is the reason it's a separate tab at the moment.

OK, that explains windows, I suspect that is a antivirus measure and port 443 can be openned up on windows. I'll see if I can figure that piece out.
On Mac, that socket is open. Still does not play.

btw: is it the player that is closing down port 443 because of https?
https://player.spotify.com/

Yes, that's why we have to have Soundbounce in a diff tab.  If we supported wss:// for the websocket, then it could be embedded in an https site (i.e. the spotify player) as an iframe.
So the issue (and same thing has been reported to me on facebook) is now player.spotify.com is not receiving the commands properly from the extension.
Please can you launch the extension from player.spotify.com, join a room (so it's trying to play music), then go to the spotify player tab, show dev tools, and paste any error messages in the console log here - that'd really help me work out what is different in that player.

Failed to update icon badge Error: Request failed with status code 404
at WebResolver.onSuccess (https://player.spotify.com/app/zlink/fbb6617/main.js:32758:23)
at executeDeferreds (https://player.spotify.com/app/zlink/fbb6617/main.js:35386:17)
at handlePostMessage (https://player.spotify.com/app/zlink/fbb6617/main.js:49862:18)
pubads_impl_65.js:153 Mixed Content: The page at 'https://player.spotify.com/app/browse/7.1.0/index.html?v=0.4.0&locale=en' was loaded over HTTPS, but requested an insecure resource 'http://www.rgeneratorbannerhosting.com/spotify/acdc/english_og/hpto/ACDC_HPTO_3.html'. This request has been blocked; the content must be served over HTTPS.rh @ pubads_impl_65.js:153mh @ pubads_impl_65.js:154jh @ pubads_impl_65.js:151kh @ pubads_impl_65.js:148(anonymous function) @ pubads_impl_65.js:166(anonymous function) @ ads?gdfp_req=1&correlator=12913509138432&output=json_html&callback=googletag.impl.pubads.setAdConte…:1
data:text/html;char…:1 Mixed Content: The page at 'data:text/html;charset=utf-8,%3Ciframe%20src%3D%22http%3A%2F%2Fwww.rgenerat…%22235%22%20frameborder%3D%220%22%20scrolling%3D%22no%22%3E%3C%2Fiframe%3E' was loaded over HTTPS, but requested an insecure resource 'http://www.rgeneratorbannerhosting.com/spotify/acdc/english_og/hpto/ACDC_HPTO_3.html'. This request has been blocked; the content must be served over HTTPS.
Uncaught TypeError: Cannot read property 'contentWindow' of null
(program):1<PHONE_NUMBER>
(program):1 Uncaught TypeError: Cannot read property 'contentWindow' of null
(program):1 Uncaught TypeError: Cannot read property 'contentWindow' of null
(program):1 Uncaught TypeError: Cannot read property 'contentWindow' of null
(program):1 Uncaught TypeError: Cannot read property 'contentWindow' of null
main.js:9822 Failed to update icon badge Error: Request failed with status code 404
at WebResolver.onSuccess (https://player.spotify.com/app/zlink/fbb6617/main.js:32758:23)
at executeDeferreds (https://player.spotify.com/app/zlink/fbb6617/main.js:35386:17)
at handlePostMessage (https://player.spotify.com/app/zlink/fbb6617/main.js:49862:18)
VM18636:1<PHONE_NUMBER>
(program):1 Uncaught TypeError: Cannot read property 'contentWindow' of null
(program):1 Uncaught TypeError: Cannot read property 'contentWindow' of null

Thanks, this is useful - I can see it's failing to find the "app-player" iframe that is there on play.spotify.com:

Does your player iframe have a different id maybe?  Or perhaps it's nested inside another iframe?
A screenshot of the "Elements" tab of dev tools with the player iframe selected would be great.
Thanks again for your help!

I don't have the little docked player the way you have illustrated.
I am not even sure where to find that option.

Oh they've completely moved the player - it's now at the bottom!  It's the bit with the play / skip + seek bar I'm after, similar screenshot with that selected would be great. THanks again

That captures the class plus I expended the relevant controls underneath. Let me know if that isn't it.

btw: on windows 7, after having the room joining fail on the chrome extension, I went to launch the soundbounce app and now have the same problem with the app.
I tried switching to all 3 ports. No difference.
Probably need to reboot.

It looks like there might be an iframe called #main that contains that div perhaps?  Please can you paste this line into the console and see if any music plays (Jackson 5!):
document.getElementById('main').contentWindow.window.eval('window.top.postMessage(JSON.stringify({"type":"cosmos-request","resolver":1,"id":"cosmos_99999999","name":"cosmos_request_create",      "payload":{"action":"POST","uri":"sp://player/v1/main","headers":{},                "body":JSON.stringify({"action":"play","context":"spotify:track:3b0EOvScbZUc0qJx0E1L2z","tracks":["spotify:track:3b0EOvScbZUc0qJx0E1L2z"],    "options":{"repeat":false,"shuffle":false,"can_repeat":true,"can_shuffle":true,"can_skip_prev":true,"can_skip_next":true,"can_seek":true,"use_dmca_rules":false},                    "play_origin":{"source":"unknown","reason":"playbtn","referrer":"spotify:app:now-playing-recs","referrer_version":"2.2.2","referrer_vendor":"com.spotify"}})}}), "*")');

I want you back! :laughing:

Yep. Jackson 5 plays
ixed Content: The page at 'https://player.spotify.com/app/browse/7.1.0/index.html?v=0.4.0&locale=en' was loaded over HTTPS, but requested an insecure image 'http://d3rt1990lpmkn.cloudfront.net/unbranded/03fa7db6aae8ffb4a936d0fe5a4f419199c313a9'. This content should also be served over HTTPS.
runner.js:2183 Load playlist: 13276.963ms
document.getElementById('main').contentWindow.window.eval('window.top.postMessage(JSON.stringify({"type":"cosmos-request","resolver":1,"id":"cosmos_99999999","name":"cosmos_request_create",      "payload":{"action":"POST","uri":"sp://player/v1/main","headers":{},                "body":JSON.stringify({"action":"play","context":"spotify:track:3b0EOvScbZUc0qJx0E1L2z","tracks":["spotify:track:3b0EOvScbZUc0qJx0E1L2z"],    "options":{"repeat":false,"shuffle":false,"can_repeat":true,"can_shuffle":true,"can_skip_prev":true,"can_skip_next":true,"can_seek":true,"use_dmca_rules":false},                    "play_origin":{"source":"unknown","reason":"playbtn","referrer":"spotify:app:now-playing-recs","referrer_version":"2.2.2","referrer_vendor":"com.spotify"}})}}), "*")');
undefined

OK thanks for info.  I've pushed an update that should be live in the Chrome Store within the hour - v0.4.2 - once you're on this version, let me know if the music plays and we can hopefully close this issue... (fingers crossed!)

Fix commit was here btw https://github.com/pdaddyo/soundbounce/commit/ef5fdd0bce448a91960588959a8dfd052b04adf4

There are two results: Good & Bad
Good first.
MacOS is now playing. It has some of the same issues as the app. It plays through the spotify player tab for some period of time and then stalls. You have to pause and play to twerk it back into audio. During that time, the speaker on the spotify player tab is gone. So I do not know why that happens but it happens on both the app and the extension.
Windows. After reinstalling both the app and chrome extension on Windows 7, neither soundbounce platform will join a room. When I click on the room, it says "Joining" for a few sec and nothing happens. I am not sure how to resolve this.

OK, did we realise Windows 7 was to do with lack of websockets on 443 support?
Re: spotify player audio cutting out - does it do it when you play a track manually in the spotify player tab?

Windows7
Attempt to join room as captured by dev tools | Network:
Request URL:ws://app.soundbounce.org:443/myy4VG4z
Request Method:GET
Status Code:101 Switching Protocols
Response Headers
view parsed
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: D7F0RdrD7Xv0NdmXxuzsJq+fguc=
Request Headers
view source
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:Upgrade
Cookie:__cfduid=d8fc3469cf0dfe023ea134172606800701435764333; _ga=GA1.2.1382316177.1435764320; _gat=1; connect.sid=s%3AxLzZ_LYf-R5yk6UxEeQkWJU-sUxvbyay.kHWqlJAULH1p6KvosVf9l%2BVoQYKKoGs5v6VDyIc5sto
Host:app.soundbounce.org:443
Origin:http://app.soundbounce.org
Pragma:no-cache
Sec-WebSocket-Extensions:permessage-deflate; client_max_window_bits
Sec-WebSocket-Key:36MyQ0vFhbPocsCwg5J0CQ==
Sec-WebSocket-Version:13
Upgrade:websocket
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.130 Safari/537.36

WebSocket connection to 'ws://app.soundbounce.org:443/myy4VG4z' failed: WebSocket is closed before the connection is established.

And yes, the spotify web player is reliable and does not seem to have the same audio interruptions.

Why is player.spotify.com able to establish a websocket while app.soundbounce.org cannot?
Request URL:wss://sjc1-linkap-a8.ap.spotify.com/
Request Method:GET
Status Code:101 Switching Protocols
Response Headers
view source
Connection:upgrade
Date:Sun, 05 Jul 2015 17:42:21 GMT
Sec-WebSocket-Accept:svz1s61UiPROBf2vv9VItkhwMUE=
Server:nginx
Upgrade:websocket
wss://sjc1-linkap-a8.ap.spotify.com/

Really strange that a track triggered via Soundbounce chrome extension would have buffering issues vs one launched in the player, as far as the player is concerned they are the same thing.  Would be great if you could do a bit more testing and see if this is definitely the case....much appreciated.

Please be specific. What is the test?

Why is player.spotify.com able to establish a websocket while app.soundbounce.org cannot?

Because spotify is using wss:// (websocket over TLS - i.e. secure websockets) - we're using ws:// because I don't have an SSL certificate set up on the server, and the node library we're using doesn't seem to support wss:// out the box.
I'd like to move to wss:// because then we can embed soundbounce app inside the spotify tab, as mentioned earlier in this thread.  I'll probably do this by using nginx as a reverse proxy and keep the node server as is.

Test is: please check if the "stalls" reported on osx are consistent and reproducable, and if any console errors happen in spotify when it's stalling etc.

wss:// issue now open here #272

Test results: Console on the spotify player side is inactive. The console on the soundbounce extension side shows calls to play individual tracks: call - play - done. Nothing shows in the console on the spotify side. I may be looking in the wrong place.
Additionally, with MacOS, what seems to be happening is this: Left idle, the two seem to play reliably.
Once I started doing this, I got the spotify player to fail audio on a track. When the next track was called from the extension, it began to play again.
Not sure how to isolate this. I suspect that track preview is the culprit but it does not reliably cause a fail. Some amount of time goes by before the audio fails. It i s not immediate.

I don't know if it is related but I did get this alert in the player.spotify.com console while audio was failing:
Mixed Content: The page at 'https://player.spotify.com/app/browse/7.1.0/index.html?v=0.4.0&locale=en' was loaded over HTTPS, but requested an insecure image 'http://d3rt1990lpmkn.cloudfront.net/unbranded/03fa7db6aae8ffb4a936d0fe5a4f419199c313a9'. This content should also be served over HTTPS.
runner.js:2183 Load playlist: 8432.791ms

another error (related?) occurred while audio was failing:
Uncaught TypeError: Cannot read property 'reason' of null(anonymous function) @ main.js?v=0.2.0:3c.emitEventSync @ spotify.sdk.5.8.3-8be3a59.min.js:1197(anonymous function) @ spotify.sdk.5.8.3-8be3a59.min.js:1196a @ spotify.sdk.5.8.3-8be3a59.min.js:560h @ spotify.sdk.5.8.3-8be3a59.min.js:561

One last error:
Uncaught SyntaxError: Unexpected token ILLEGAL         VM4874057:1
clicking on VM4874057:1 link:
playerFrame){ playerFrame = document.getElementById('main'); }  if(!playerFrame){ alert('Unable to find player?'); return; }  playerFrame.contentWindow.window.eval('document.getElementById('app-player').contentWindow.window.eval('window.top.postMessage(JSON.stringify({"type":"bridge_request","id":1436027256787,"name":"player_pause","args":["main"],"appVendor":"com.spotify","appVersion":"4.2.0"}), "*");');');  }());

I've seen a lot of those errors - it throws them even when soundbounce is disabled so I'm ignoring them, thanks for posting though


sample-encode: measure sample video stream size instead of file size

When calculating the predicted encode size percent we compare the sample file size sum to their encoded counterparts. This works fairly well since the samples & encoded samples contain only a video stream.
However, there may also be some container overhead for each sample that introduces inaccuracy. So we could instead measure the lossless sample video stream size sum and compare that to the encoded video streams specifically avoiding any container overhead on each side. Also see discussion in #79.
Measuring video streams a little harder than measuring file sizes and may be a bit less reliable too as we have to trust what ffmpeg tells us. It also seems to be in "kB" precision only, I'm not sure if we can configure that or not.
My current feeling is that this won't significantly improve the sample-encode video stream size predictions over the noise of using sample sizes to deduce the total predicted stream size. So to begin with we should gather some test cases where calculating the samples vs encoded samples using video stream instead of full sample size would be more accurate over many different sample counts.

I was getting very weird results with mp4 files, but then I saw that the sample duration was not being correctly detected.
It was being reported like this:

Duración :
11 s 0 ms

Duración original :
20 s 0 ms
That directly correlates to the differences in stream size being reported. Just to be sure, I remuxed the mp4 video stream into mkv, and the duration was showing correctly (20s) and stream/file size were virtually identical and matched the full mp4 file size too with insignificant overhead.
So false alarm I guess.
Both mp4/mkv containers seem to have minimal overhead, and according to all documentation on the internet (such as the matroska overhead comparison tests), it should remain fairly low, below 3% for sure, being more common about 2% or maybe less for some files.
It does not entail a big difference, but knowing the general overhead percentage and being fairly easy to implement as a correction, it is a free gain to get closer predictions.
I will download and start using the new version to see how it works.

I've been testing the new method to calculate the size prediction and in general it makes much more sense and it seems a lot more consistent, specially the logic improvement to account the real duration of the samples.
However, I've still getting pretty weird inconsistencies with mp4 files now in the estimated percentage that I think might be inherited from the previous version, and maybe also related to how the lossless sample duration is incorrectly detected, as shown above, but I'm not sure.
Here is an example of the same video file tested in 3 different ways: the original file in mp4 as reference (how anybody would usually run ab-av1), then remuxed into mkv with all the streams (maintaining the same filesize, to discard or confirm a suspected container issue), and also remuxed into mkv only with the video stream (to check prediction consistency in general, but specially between using the input size in the old method vs the estimated video stream size in the new method).
Old prediction method:

All streams remuxed into mkv: crf 17 VMAF 95.00 predicted full encode size 249.05 MiB (30%) taking 15 minutes
Only video stream remuxed into mkv: crf 17 VMAF 95.00 predicted full encode size 233.11 MiB (30%) taking 15 minutes
Original mp4 file as reference: crf 17 VMAF 95.12 predicted full encode size 153.14 MiB (18%) taking 10 minutes

The mkv files more o less make sense according to the old method (one is a bit bigger because of the included audio stream), but the mp4 file does not, having in mind that the input file size is almost exactly the same as the first mkv that includes all the streams, so the results should have been very similar between the mkv and mp4 files, being both all the same video and audio streams and same size, so the problem comes from that 18% estimated percentage.
That is more clearly shown when we test the new prediction method for only vstream size:

All streams remuxed into mkv: crf 17 VMAF 95.00 predicted video stream size 225.15 MiB (30%) taking 15 minutes
Only video stream remuxed into mkv: crf 17 VMAF 95.00 predicted video stream size 225.14 MiB (30%) taking 15 minutes
Original mp4 file as reference: crf 17 VMAF 95.12 predicted video stream size 230.57 MiB (18%) taking 16 minutes

Now the predicted sizes are a lot more consistent because all of them estimate the vstream size, which is the same, but oddly the mp4 file still shows that weird 18% estimated percentage that now does not correspond to the approximate size at all. So there is something wrong in the percentage there inherited from the previous version, and only seems to be inaccurate for mp4 files.
I've tested some more mp4 vs mkv files, and not all of them are inconsistent in the same degree, but they are not correct. I think the inaccuracy depends on how correctly the duration of the mp4 samples is detected. In some samples the detected duration was almost 20s, like 18-19s, and then the estimated percentage was only inaccurate by 1-2% points. That is why I believe there is a direct correlation, but I'm not sure, it's difficult to say.
You will know better how that works, maybe you can reproduce the issue and confirm comparing mp4 vs mkv containers for the same video.

This is the mediainfo of one of the lossless mp4 samples of that test file. Two different durations, two bitrates, two reported stream sizes... I'm guessing the problem is somewhere here not being correctly recognized and messing with ab-av1's estimations:

MP4 Vídeo

ID :
1

Formato :
AVC

Formato/Info :
Advanced Video Codec

Formato del perfil :
<EMAIL_ADDRESS>
Ajustes del formato :
CABAC / 1 Ref Frames

Ajustes del formato, CABAC :
Sí

Ajustes del formato, RefFrames :
1 fotograma

ID códec :
avc1

ID códec/Info :
Advanced Video Coding

Duración :
11 s 0 ms

Duración original :
20 s 0 ms

Tasa de bits :
3 927 kb/s

Tasa de bits máxima :
3 733 kb/s

Ancho :
1 280 píxeles

Alto :
720 píxeles

Relación de aspecto :
16:9

Modo velocidad fotogramas :
Constante

Velocidad de fotogramas :
25,000 FPS

Espacio de color :
YUV

Submuestreo croma :
4:2:0

Profundidad bits :
8 bits

Tipo barrido :
Progresivo

Bits/(píxel*fotograma) :
0.170

Tamaño de pista :
5,15 MiB (62%)

Cantidad de pistas original :
8,32 MiB (100%)

Idioma :
Ruso

Codec configuration box :
avcC
Meanwhile, the samples from the file remuxed into mkv don't seem to have those kind of problems and they are correctly identified by ab-av1.

If the lossless sample containers are inflated we can only really fix the incorrect percent prediction by properly reading the video stream size. That could be possible just from parsing the ffmpeg output that is already happening, I'm just not sure how reliable it'll be in the end.

@iPaulis can you run ffmpeg -i sample.mp4 -c copy -f null /dev/null (or maybe just ffmpeg -i sample.mp4 -c copy tmp.mp4 if /dev/null doesn't work on Windows).
The last line of output should list the video stream size according to ffmpeg, that's the info we should be able to get at fairly easily.

Yes sure, here are the last lines of the output for the source file: ffmpeg -i source.mp4 -c copy -f null /dev/null

frame=43486 fps=4510 q=-1.0 Lsize=N/A time=00:28:59.49 bitrate=N/A speed= 180x
video:792718kB audio:54030kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

The sizes and duration are 100% correct and it took just about 10 seconds to read the whole file and give the results.
And these are the results for a sample of that file: ffmpeg -i sample.mp4 -c copy -f null /dev/null

frame=  500 fps=0.0 q=-1.0 Lsize=N/A time=00:00:10.96 bitrate=N/A speed=93.7x    s speed=N/A
video:8522kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

There is a problem there. The size is correct (the same as the lossless samples generated from the mkv), but the time is not; it should be 20s, I checked.
I saw you released a new version with more fixes, including a nice improvement for accuracy in size prediction if the vstream size method gave higher predictions than the whole input size method, which was happening in some cases, but should never happen.
So I compiled this version and rerun this mp4 file, and it is indeed changing the prediction method, but for the wrong reasons this time as the percentage is not correct:

crf 17 VMAF 95.12 predicted video stream size 153.14 MiB (18%) taking 16 minutes

This is the real encoded result:

Encoded 291.02 MiB (35%, video:237.36 MiB, audio:52.76 MiB)

I don't know where the percentage inaccuracy is exactly coming from for mp4 files, and it would be great to understand and be able to fix the source of the issue itself.
But if it is just an mp4 inconsistency issue, maybe the simplest and easiest way to fix it could be to generate the lossless samples in mkv instead, even for mp4 source files; it could end up giving more reliable results.

I've sourced some >20s samples to test with from 4kmedia & pixabay (it'll be useful to have a set of test videos to investigate this kind of thing). They are all .mp4, I've used them to test mp4 or mkv lossless samples.
Tests

4kmedia-sony-new-york-fashion-demo.mp4 results
4kmedia-sony-new-york-fashion-demo.mp4
mp4 -> mp4 lossless samples
$ ab-av1 sample-encode -i test-vids/4kmedia-sony-new-york-fashion-demo.mp4 --crf 28 --preset 12
VMAF 95.83 predicted video stream size 248.06 MiB (26%) taking 3 minutes

==> 4kmedia-sony-new-york-fashion-demo.sample45+1199f.mp4
file: 186560 KiB
frame= 1199 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.36 bitrate=N/A speed=1.46e+03x
video:186547kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> 4kmedia-sony-new-york-fashion-demo.sample45+1199f.crf28.p12.ivf
file: 49360 KiB
frame= 1164 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.41 bitrate=N/A speed=2.17e+03x
video:49347kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

mp4 -> mkv lossless samples
$ ab-av1 sample-encode -i test-vids/4kmedia-sony-new-york-fashion-demo.mp4 --crf 28 --preset 12
VMAF 95.75 predicted video stream size 255.98 MiB (27%) taking 3 minutes

==> 4kmedia-sony-new-york-fashion-demo.sample45+1199f.mkv
file: 186557 KiB
frame= 1199 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.97 bitrate=N/A speed=1.36e+03x
video:186547kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> 4kmedia-sony-new-york-fashion-demo.sample45+1199f.crf28.p12.ivf
file: 50936 KiB
frame= 1200 fps=0.0 q=-1.0 Lsize=N/A time=00:00:20.01 bitrate=N/A speed=2.54e+03x
video:50922kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

4kmedia-spacex-launch-demo.mp4 results
4kmedia-spacex-launch-demo.mp4
mp4 -> mp4 lossless samples
$ ab-av1 sample-encode -i test-vids/4kmedia-spacex-launch-demo.mp4 --crf 28 --preset 12
VMAF 99.04 predicted video stream size 21.21 MiB (12%) taking 82 seconds

==> 4kmedia-spacex-launch-demo.sample53+600f.mp4
file: 28657 KiB
frame=  600 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.85 bitrate=N/A speed=5.23e+03x
video:28653kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> 4kmedia-spacex-launch-demo.sample53+600f.crf28.p12.ivf
file: 3418 KiB
frame=  597 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.91 bitrate=N/A speed=9.66e+03x
video:3411kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

mp4 -> mkv lossless samples
$ ab-av1 sample-encode -i test-vids/4kmedia-spacex-launch-demo.mp4 --crf 28 --preset 12
VMAF 99.03 predicted video stream size 21.15 MiB (12%) taking 85 seconds

==> 4kmedia-spacex-launch-demo.sample53+600f.mkv
file: 28659 KiB
frame=  600 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.95 bitrate=N/A speed=4.39e+03x
video:28653kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> 4kmedia-spacex-launch-demo.sample53+600f.crf28.p12.ivf
file: 3425 KiB
frame=  600 fps=0.0 q=-1.0 Lsize=N/A time=00:00:20.01 bitrate=N/A speed=1.02e+04x
video:3419kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

pixabay-bridge-23544.mp4 results
pixabay-bridge-23544.mp4
mp4 -> mp4 lossless samples
$ ab-av1 sample-encode -i test-vids/pixabay-bridge-23544.mp4 --crf 28 --preset 12
VMAF 96.36 predicted video stream size 29.29 MiB (30%) taking 23 seconds

==> pixabay-bridge-23544.sample5+600f.mp4
file: 65739 KiB
frame=  600 fps=0.0 q=-1.0 Lsize=N/A time=00:00:14.90 bitrate=N/A speed=2.45e+03x
video:65731kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> pixabay-bridge-23544.sample5+600f.crf28.p12.ivf
file: 19409 KiB
frame=  450 fps=0.0 q=-1.0 Lsize=N/A time=00:00:15.00 bitrate=N/A speed=3.99e+03x
video:19404kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

mp4 -> mkv lossless samples
$ ab-av1 sample-encode -i test-vids/pixabay-bridge-23544.mp4 --crf 28 --preset 12
VMAF 96.53 predicted video stream size 45.11 MiB (45%) taking 22 seconds

==> pixabay-bridge-23544.sample5+600f.mkv
file: 65737 KiB
frame=  600 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.90 bitrate=N/A speed=2.76e+03x
video:65731kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> pixabay-bridge-23544.sample5+600f.crf28.p12.ivf
file: 29893 KiB
frame=  600 fps=0.0 q=-1.0 Lsize=N/A time=00:00:20.00 bitrate=N/A speed=4.35e+03x
video:29886kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

pixabay-elevator-3735.mp4 results
pixabay-elevator-3735.mp4
mp4 -> mp4 lossless samples
$ ab-av1 sample-encode -i test-vids/pixabay-elevator-3735.mp4 --crf 28 --preset 12
VMAF 96.24 predicted video stream size 16.57 MiB (78%) taking 8 seconds

==> pixabay-elevator-3735.sample12+500f.mp4
file: 9047 KiB
frame=  500 fps=0.0 q=-1.0 Lsize=N/A time=00:00:17.88 bitrate=N/A speed=1.03e+04x
video:9041kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> pixabay-elevator-3735.sample12+500f.crf28.p12.ivf
file: 7016 KiB
frame=  450 fps=0.0 q=-1.0 Lsize=N/A time=00:00:18.00 bitrate=N/A speed=7.88e+03x
video:7011kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

mp4 -> mkv lossless samples
$ ab-av1 sample-encode -i test-vids/pixabay-elevator-3735.mp4 --crf 28 --preset 12
VMAF 96.28 predicted video stream size 17.47 MiB (87%) taking 8 seconds

==> pixabay-elevator-3735.sample12+500f.mkv
file: 9044 KiB
frame=  500 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.88 bitrate=N/A speed=1.1e+04x
video:9041kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> pixabay-elevator-3735.sample12+500f.crf28.p12.ivf
file: 7837 KiB
frame=  500 fps=0.0 q=-1.0 Lsize=N/A time=00:00:20.00 bitrate=N/A speed=9.07e+03x
video:7832kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

pixabay-lemon-82602.mp4 results
pixabay-lemon-82602.mp4
mp4 -> mp4 lossless samples
$ ab-av1 sample-encode -i test-vids/pixabay-lemon-82602.mp4 --crf 28 --preset 12
VMAF 97.03 predicted video stream size 15.81 MiB (17%) taking 34 seconds

==> pixabay-lemon-82602.sample16+500f.mp4
file: 35574 KiB
frame=  500 fps=0.0 q=-1.0 Lsize=N/A time=00:00:13.88 bitrate=N/A speed=3.81e+03x
video:35567kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> pixabay-lemon-82602.sample16+500f.crf28.p12.ivf
file: 6220 KiB
frame=  350 fps=0.0 q=-1.0 Lsize=N/A time=00:00:14.00 bitrate=N/A speed=7e+03x
video:6217kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

mp4 -> mkv lossless samples
$ ab-av1 sample-encode -i test-vids/pixabay-lemon-82602.mp4 --crf 28 --preset 12
VMAF 96.91 predicted video stream size 23.45 MiB (26%) taking 32 seconds

==> pixabay-lemon-82602.sample16+500f.mkv
file: 35571 KiB
frame=  500 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.88 bitrate=N/A speed=4.78e+03x
video:35567kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> pixabay-lemon-82602.sample16+500f.crf28.p12.ivf
file: 9223 KiB
frame=  500 fps=0.0 q=-1.0 Lsize=N/A time=00:00:20.00 bitrate=N/A speed=8.69e+03x
video:9218kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

pixabay-nature-31377.mp4 results
pixabay-nature-31377.mp4
mp4 -> mp4 lossless samples
$ ab-av1 sample-encode -i test-vids/pixabay-nature-31377.mp4 --crf 28 --preset 12
VMAF 96.67 predicted video stream size 332.43 MiB (324%) taking 22 seconds

==> pixabay-nature-31377.sample4+480f.mp4
file: 72065 KiB
frame=  480 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.72 bitrate=N/A speed=3.34e+03x
video:72062kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> pixabay-nature-31377.sample4+480f.crf28.p12.ivf
file: 233506 KiB
frame=  474 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.76 bitrate=N/A speed= 717x
video:233501kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

mp4 -> mkv lossless samples
$ ab-av1 sample-encode -i test-vids/pixabay-nature-31377.mp4 --crf 28 --preset 12
VMAF 96.65 predicted video stream size 329.75 MiB (325%) taking 22 seconds

==> pixabay-nature-31377.sample4+480f.mkv
file: 72067 KiB
frame=  480 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.97 bitrate=N/A speed=2.84e+03x
video:72062kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> pixabay-nature-31377.sample4+480f.crf28.p12.ivf
file: 234560 KiB
frame=  480 fps=0.0 q=-1.0 Lsize=N/A time=00:00:20.02 bitrate=N/A speed=1.05e+03x
video:234555kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

pixabay-sunrise-83880.mp4 results
pixabay-sunrise-83880.mp4
mp4 -> mp4 lossless samples
$ ab-av1 sample-encode -i test-vids/pixabay-sunrise-83880.mp4 --crf 28 --preset 12
- Sample 1 (55%) vmaf 96.13
VMAF 96.13 predicted video stream size 5.33 MiB (55%) taking 20 seconds

==> pixabay-sunrise-83880.sample6+2398f.mp4
file: 5577 KiB
frame= 2398 fps=0.0 q=-1.0 Lsize=N/A time=00:00:18.14 bitrate=N/A speed=5.33e+03x
video:5549kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> pixabay-sunrise-83880.sample6+2398f.crf28.p12.ivf
file: 3072 KiB
frame= 2181 fps=0.0 q=-1.0 Lsize=N/A time=00:00:18.19 bitrate=N/A speed=3.28e+03x
video:3047kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

mp4 -> mkv lossless samples
$ ab-av1 sample-encode -i test-vids/pixabay-sunrise-83880.mp4 --crf 28 --preset 12
VMAF 96.15 predicted video stream size 5.36 MiB (59%) taking 20 seconds

==> pixabay-sunrise-83880.sample6+2398f.mkv
file: 5566 KiB
frame= 2398 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.97 bitrate=N/A speed=5.48e+03x
video:5549kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> pixabay-sunrise-83880.sample6+2398f.crf28.p12.ivf
file: 3266 KiB
frame= 2401 fps=0.0 q=-1.0 Lsize=N/A time=00:00:20.02 bitrate=N/A speed=3.47e+03x
video:3238kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

pixabay-turntable-8453.mp4 results
pixabay-turntable-8453.mp4
mp4 -> mp4 lossless samples
$ ab-av1 sample-encode -i test-vids/pixabay-turntable-8453.mp4 --crf 28 --preset 12
- Sample 1 (15%) vmaf 94.14
VMAF 94.14 predicted video stream size 2.34 MiB (15%) taking 7 seconds

==> pixabay-turntable-8453.sample14+500f.mp4
file: 6542 KiB
frame=  500 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.52 bitrate=N/A speed=1.48e+04x
video:6537kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> pixabay-turntable-8453.sample14+500f.crf28.p12.ivf
file: 974 KiB
frame=  491 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.64 bitrate=N/A speed=1.42e+04x
video:969kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

mp4 -> mkv lossless samples
$ ab-av1 sample-encode -i test-vids/pixabay-turntable-8453.mp4 --crf 28 --preset 12
VMAF 93.95 predicted video stream size 2.29 MiB (15%) taking 8 seconds

==> pixabay-turntable-8453.sample14+500f.mkv
file: 6541 KiB
frame=  500 fps=0.0 q=-1.0 Lsize=N/A time=00:00:19.88 bitrate=N/A speed=1.3e+04x
video:6537kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

==> pixabay-turntable-8453.sample14+500f.crf28.p12.ivf
file: 972 KiB
frame=  500 fps=0.0 q=-1.0 Lsize=N/A time=00:00:20.00 bitrate=N/A speed=1.48e+04x
video:967kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

Analysis
In all cases frame count and duration of lossless vs encoded samples are closer using mkv and closer to 20s than with mp4.
In particular pixabay-bridge-23544.mp4, pixabay-lemon-82602.mp4 reproduce the case where the mp4 sample is significantly under 20s. Playing the lossless versions with mpv I can see 4-5s of "negative time" before 0, so there is ~20s of video but with strange timing. When ffmpeg converts this to yuv streams for encoding or vmaf though this "negative time" doesn't seem to be used. This means the encoded samples have significantly fewer frames and the file/video stream comparison is misleading as the lossless samples contain more video than they let on. In both cases mkv samples had no issues.
So according to these results for mp4 files using .mkv for lossless samples is a clear win.
Video stream size vs file size
Getting back to what this issue is really about this testing also shows file size vs video size info. We can compare encode-percent calculations using either. (We'll use mkv samples results since they're better).

4kmedia-sony-new-york-fashion-demo.mp4

file size (50936 / 186557) = 27.30%
vid size (50922 / 186547) = 27.30%

4kmedia-spacex-launch-demo.mp4

file size (50936 / 186557) = 11.95%
vid size (50922 / 186547) = 11.93%

pixabay-bridge-23544.mp4

file size (29893 / 65737) = 45.47%
vid size (50922 / 186547) = 45.47%

pixabay-elevator-3735.mp4

file size (7837 / 9044) = 86.65%
vid size (50922 / 186547) = 86.63%

pixabay-lemon-82602.mp4

file size (9223 / 35571) = 25.93%
vid size (9218 / 35567) = 25.92%

pixabay-nature-31377.mp4

file size (234560 / 72067) = 325.47%
vid size (234555 / 72062) = 325.49%

pixabay-sunrise-83880.mp4

file size (3266 / 5566) = 58.68%
vid size (3238 / 5549) = 58.35%

pixabay-turntable-8453.mp4

file size (972 / 6541) = 14.86%
vid size (967 / 6537) = 14.79%

This supports my initial feeling that there isn't a significant win in using video stream size over file size. If they were equally easy to do I would still do video stream comparison, however measuring file size is easier as it doesn't involve ffmpeg output parsing.

Great spot regarding mkv vs mp4 samples @iPaulis!
With #90 merged I think we can close this issue as I plan to continue using the file-size comparison until video stream size can be demonstrated to be better enough to be worth digging out of the ffmpeg output.

i think this must be an edge case with ffmpeg where is only encoding 11s because of some quirk with the lossless samples. So the encoded percent comes out too low. Just to clarify the video stream size there is pretty much the same as the file size then?

Yes, completely accurate.

I haven't seen this with mp4s myself though. It's interesting that using mkv solves it, that could be a solution as you suggest. However, I'll need to test more widely in case mp4->mkv sampling causes other issues.

I've been testing more mp4 files and almost all of their samples have this issue where the duration is not correctly detected to some degree. For some files it is just 18-19s instead of 20s samples, which introduces a smaller inaccuracy, but there also are some files like the example above where the duration difference is higher like 11-12s or 15-16s, and the inaccuracy in the estimated percentage seems proportional to the duration inaccuracy.
I found a video file I could share with you so you could try to reproduce the issue and see if it also happens in your system.
I used the --samples 10 parameter to detect the issue more easily: ab-av1 crf-search --samples 10 --pix-format yuv420p10le --min-vmaf 96 --vmaf pool=harmonic_mean -i '.\webproject (5h zuzenketak).mp4' --preset 6
I'm adding a screenshot of the samples of both files, source mp4 and remuxed mkv.

The sample sizes are equivalent and correct, but you can see the durations are not consistent. I confirm that all samples are in fact 20 seconds long.

This supports my initial feeling that there isn't a significant win in using video stream size over file size. If they were equally easy to do I would still do video stream comparison, however measuring file size is easier as it doesn't involve ffmpeg output parsing.

Please, I would highly recommend you against it. I've also been testing the input size method vs the video stream method and using the whole input has an inherent inaccuracy issue. It is the most reliable and accurate prediction when the video stream size is most of the whole file size (probably the case for the testfiles you used), but the bigger the percentage of the audio streams, the higher the inaccuracy it introduces.
That is specially noticeable in files with multiple audio tracks, high quality audios that take up much space, etc.
Very clear example with a very common type of file (real encoded video stream size is 6,1GB):

Input size method: crf 18 VMAF 96.51 predicted full encode size 9.37 GiB (32%) taking 2 hours
Vstream size method: crf 18 VMAF 96.51 predicted video stream size 5.90 GiB (32%) taking 2 hours

New method is close enough, that's fine.
Another file (real encoded video stream size is 6,0GB):

Input size method: crf 17 VMAF 96.47 predicted full encode size 11.97 GiB (45%) taking 2 hours
Vstream size method: crf 17 VMAF 96.47 predicted video stream size 6.81 GiB (45%) taking 2 hours

So, in this case the new method overestimates size (which could be improved), but it is closer than the old method anyway.
We can find a more reliable and accurate way than the current method avoiding having to regress to the input method and without needing to involve ffmpeg output parsing.
I think I have an alternative idea that could work (and should be fairly simple to implement and have no impact in ab-av1 runtime), I just have to think about it a bit more, I can come back later in the afternoon to explain.

Please, I would highly recommend you against it. I've also been testing the input size method vs the video stream method and using the whole input has an inherent inaccuracy issue. It is the most reliable and accurate prediction when the video stream size is most of the whole file size (probably the case for the testfiles you used), but the bigger the percentage of the audio streams, the higher the inaccuracy it introduces.

This is a different issue though. I'm talking about lossless sample file size vs encoded sample file size which according to these tests is pretty much the same as using the video stream while much easier to do.
The full input video stream size isn't relevant to calculating the encode percentage. It could make the full predicted video stream size more accurate, but it involves a full scan of the input which isn't desirable. The predicted size isn't as important as the percentage either, since the latter is used as part of a crf-search.

I've raised #91 so we can investigate further getting the full input video stream size for the predicted size calc (since you won't let me avoid it :laughing:)

Please, I would highly recommend you against it. I've also been testing the input size method vs the video stream method and using the whole input has an inherent inaccuracy issue. It is the most reliable and accurate prediction when the video stream size is most of the whole file size (probably the case for the testfiles you used), but the bigger the percentage of the audio streams, the higher the inaccuracy it introduces.

This is a different issue though. I'm talking about lossless sample file size vs encoded sample file size which according to these tests is pretty much the same as using the video stream while much easier to do.
The full input video stream size isn't relevant to calculating the encode percentage. It could make the full predicted video stream size more accurate, but it involves a full scan of the input which isn't desirable. The predicted size isn't as important as the percentage either, since the latter is used as part of a crf-search.

Oh, you meant only for the samples, sorry for the misunderstanding. Yes, sure, as the samples don't have audio streams, sample file size should be pretty much the same as video stream size of the sample, with an almost insignificant overhead difference.
Anyway, I believe there is a way to improve the final predicted video stream size, but I have to leave now, I'll explain later and I hope you find it useful.
Sorry to bother you, it is not my intention, I just want to be helpful.

Sorry to bother you, it is not my intention, I just want to be helpful.

It's been super helpful mate, I appreciate the input!

Sorry to bother you, it is not my intention, I just want to be helpful.

It's been super helpful mate, I appreciate the input!

I'm glad to help. I know you do, thank you for listening to all these requests and doing all the hard work.


New flag --oidc-providers-disable to disable OIDC providers

Summary
This PR adds a new command line flag --oidc-providers-disable to cosign sign and cosign sign-blob to disable the internal OIDC providers. This does not break compatibility with the current cli and skips the providers logic to jump straight to the OIDC flow.
/cc @di @eddiezane
Ticket Link
Fixes https://github.com/sigstore/cosign/issues/1819
Release Note
New command line flag `--oidc-providers-disable` to `cosign sign` and `cosign sign-blob` to disable the internal OIDC providers. This does not break compatibility with the current cli and skips the providers logic to jump straight to the OIDC flow.

Codecov Report

Merging #1832 (99c1db0) into main (88b68f4) will decrease coverage by 0.01%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main    #1832      +/-   ##
==========================================
- Coverage   32.98%   32.96%   -0.02%
==========================================
  Files         147      147
  Lines        9350     9355       +5
==========================================
  Hits         3084     3084
- Misses       5911     5916       +5
  Partials      355      355

Impacted Files
Coverage Δ

cmd/cosign/cli/fulcio/fulcio.go
21.42% <0.00%> (ø)

cmd/cosign/cli/options/oidc.go
0.00% <0.00%> (ø)

cmd/cosign/cli/sign.go
0.00% <0.00%> (ø)

cmd/cosign/cli/signblob.go
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 88b68f4...99c1db0. Read the comment docs.


INIT(READ);
  req->file = file;

  req->nbufs = nbufs;
  req->bufs = req->bufsml;
  if (nbufs > ARRAY_SIZE(req->bufsml))
    req->bufs = uv__malloc(nbufs * sizeof(*bufs));

  if (req->bufs == NULL)
    return -ENOMEM;

But when uv_fs_read meat ENOMEM error , should we do some housekeeper that call
uv__req_unregister?
Otherwise, active_reqs will explode.

Yes, we need to undo what uv__req_init did. Thanks for looking into these issues!

yes, I will send PR,  but lots to fix.

:heart_eyes_cat:

Fixed by #543


Failed to build the site

Hello.
I exported my Wordpress blog to .md files and like to experiment with Zola and Zolarwind theme. What trying to zola serve I got the message:
Building site...
Checking all internal links with anchors.
> Successfully checked 1 internal link(s) with anchors.
-> Creating 73 pages (0 orphan) and 2 sections
Error: Failed to serve the site
Error: Failed to render pager 7
Error: Reason: Failed to render 'blog.html'
Error: Reason: Variable `res.empty_page_summary` not found in context while rendering 'partials/list-element.html'

What should I check?

Hi! You found a bug: in the template list-element.html was a wrong reference to the resource value res.empty_page_summary, which should have been res.empty_page_description.
I have commited a fix to the repository.
You can also edit line 21 in templates/partials/list-element.html and replace res.empty_page_summary with res.empty_page_description.

Thank you, your fix works :)