Readme and documentation differ on require example

On the http://google.github.io documentation pages this way of requesting the library is given:
const google = require('googleapis'); const tagmanager = google.tagmanager('v2');
I found this didn't work, and went confirm that this was the same on the readme. I found that it differs slightly:
var {google} = require('googleapis'); var urlshortener = google.urlshortener('v1');
Interestingly, the use of const is more modern than var, but the example with var is the one that works 😄
It would be a good idea to update the http://google.github.io pages.
Thanks!

Apologies for the troubles here!  The docs on http://google.github.io are wildly out of date.  To get them updated, we need to complete a few things around the TypeScript transition.
This is blocked by:

#503 - Better TypeScript d.ts support
#1009 - Fix doc generation my moving to TypeDoc

In the meantime, I submitted #1084 to help clean up some of the old code in the README :)


List of all the available features

Hi there :)
I am confused by all the available options sometimes, especially since they are named similar sometimes and without any additional description.
Is it appropriate to display the different options with a short description in an own Readme?

I think we can just add a header in the JS files so the docs are directly linked to the feature names and don’t risk being forgotten.

Very good

@bfred-it any specific style?

I think we can probably just copy-paste the text from the readme, then place the screenshot link(s) on separate lines under it. Additional caveats and notes can go under it.
I think this is Sindre’s style?
/*
Text
*/

Maybe a GIF?

Most of our features already appear in the readme. Those that need it have a gif or image attached.

??
No new wiki page. Just comments in the headers of JavaScript files. There’s already a link to the feature list in the options.

This has been happening for new features and old ones are updated when we touch thouse files.


optional attrs

Describe the feature
Actual data from the database, especially from nosql, do not contain complete data for each item.
I'm not sure how or if it can be implemented in pinia orm.
I need some attrs to be optional, i.e. if I create a new model, so that all attrs in the model are not filled with the default value (which is null), but are undefined.
And null is not even an allowed value for these attrs, so not only is the model normalized to some data, but I can't save it.
I tried to create my own Cast, but if there is something undefined in the data, the Cast value will be always an empty string.
Additional information

[ ] Would you be willing to help implement this feature?

Final checks

[X] Check existing discussions and issues.

@ahoyahoy Completed, but no comment or commit? Still trying to figure out how to have optional attrs.


If technically possible, remembering the view setting (zoom/pan) for a specific image.

First, I want to say thank you, very great extension! :)
So my idea is that the extension can remember the last view of an image (specifically for my need the level of zoom/pan).
If I close the chrome tab and I open the same URL, the zoom and pan will be the same.
This is very useful as I use this for very big jpg map.
That's it!
Thank you again.

I just realize also that sometime the zoom/pan is reset if I let the tab open for a couple of minute and navigate to other tabs (which is maybe a bug?).


Bug Fixes

My system was reaching unreachable code because of a bug, changed to pass an error instead of panicking.
Fixed the error in calculating PDU length for write_multiple for TCP

Thanks for your PR!
Would you mind to do a rebase?

Thanks for your PR!
Would you mind to do a rebase?

Done. Looks like CI failed because cargo was not able to update itself though.

@dtjones190 sorry for not taking care of this for such a long time.
I hopefully will manage to take a second look at it within the next days.

@dtjones190 I'd release a v0.1.0 if you don't mind?


[Agent] fix the interceptor package name is incorrect

Please answer these questions before submitting pull request

Why submit this pull request?

[x] Bug fix

[ ] New feature provided

[ ] Improve performance

Related issues

Bug fix

Bug description.

How to fix?

New feature or improvement

Describe the details and related test reports.

Coverage remained the same at 19.29% when pulling 689e86b0931b7abe0741be6b87d8d282fe4eae23 on ascrutae:feature/fegin-issue into be4a8db0c9c069f78742be9141aa9aec3c0f5bf4 on apache:master.


Mac support

Not sure if it is obvious, but - would this work on Mac ?

so, a quick and simple way to install neptune on macos (yes, it is doable)
https://github.com/uwu/neptune/archive/refs/heads/master.zip
download the repo
go to /Applications/TIDAL.app/Contents/Resources
rename app.asar in that folder to original.asar
move the injector directory from the downloaded and extracted main repo into said resources folder, rename the injector directory to "app"
profit

Magic, thanks!
Question :
Is this something I might have to re-do each time Tidal installs an update?
Thanks!

depends on the type of update

ok!


Crash In Node3DEditorPlugin Selecting On Viewport

Tested versions

Reproducable in 4.3-rc2, rc3, 88f3b5f9d52f740b24fabfb8bc01b8b7026ba279
Not reproducable in 4.2.2-stable

System information
Windows 11/64, RTX 3070, Vulkan
Issue description
Downstream bug report: https://github.com/TokisanGames/Terrain3D/issues/449
Drawing a selection box in the viewport can cause a crash under certain conditions. Does not affect 4.2.2-stable. The script errors are mine and of no consequence. I'm looking for what conditions in my code flushes out the engine bug.
Godot Engine v4.3.rc.custom_build.88f3b5f9d (2024-08-09 21:25:56 UTC) - https://godotengine.org
Vulkan 1.3.278 - Forward+ - Using Device #0: NVIDIA - NVIDIA GeForce RTX 3070 Laptop GPU

SCRIPT ERROR: Invalid call. Nonexistent function 'get_camera' in base 'Nil'.
          at: update_decal (res://addons/terrain_3d/src/ui.gd:269)
SCRIPT ERROR: Invalid call. Nonexistent function 'get_camera' in base 'Nil'.
          at: update_decal (res://addons/terrain_3d/src/ui.gd:269)
SCRIPT ERROR: Invalid call. Nonexistent function 'get_camera' in base 'Nil'.
          at: update_decal (res://addons/terrain_3d/src/ui.gd:269)
ERROR: Condition "!is_ancestor_of(p_start_node)" is true. Returning: p_start_node
   at: Node::get_deepest_editable_node (scene\main\node.cpp:2525)
WARNING: Terrain3D#3596:_notification: NOTIFICATION_CRASH
     at: VariantUtilityFunctions::push_warning (core\variant\variant_utility.cpp:1112)

================================================================
CrashHandlerException: Program crashed
Engine version: Godot Engine v4.3.rc.custom_build (88f3b5f9d52f740b24fabfb8bc01b8b7026ba279)
Dumping the backtrace. Please include this when reporting the bug to the project developer.
[0] Node3DEditorViewport::_select_region (C:\GD\godot\editor\plugins\node_3d_editor_plugin.cpp:1085)
[1] Node3DEditorViewport::_sinput (C:\GD\godot\editor\plugins\node_3d_editor_plugin.cpp:1951)
[2] CallableCustomMethodPointer<Node3DEditorViewport,Ref<InputEvent> const &>::call (C:\GD\godot\core\object\callable_m)[3] Callable::callp (C:\GD\godot\core\variant\callable.cpp:58)
[4] Object::emit_signalp (C:\GD\godot\core\object\object.cpp:1190)
[5] Node::emit_signalp (C:\GD\godot\scene\main\node.cpp:3896)
[6] Control::_call_gui_input (C:\GD\godot\scene\gui\control.cpp:1818)
[7] Viewport::_gui_call_input (C:\GD\godot\scene\main\viewport.cpp:1570)
[8] Viewport::_gui_input_event (C:\GD\godot\scene\main\viewport.cpp:1836)
[9] Viewport::push_input (C:\GD\godot\scene\main\viewport.cpp:3259)
[10] Window::_window_input (C:\GD\godot\scene\main\window.cpp:1680)
[11] CallableCustomMethodPointer<Window,Ref<InputEvent> const &>::call (C:\GD\godot\core\object\callable_method_pointer)[12] Callable::callp (C:\GD\godot\core\variant\callable.cpp:58)
[13] Callable::call<Ref<InputEvent> > (C:\GD\godot\core\variant\variant.h:876)
[14] DisplayServerWindows::_dispatch_input_event (C:\GD\godot\platform\windows\display_server_windows.cpp:3557)
[15] Input::_parse_input_event_impl (C:\GD\godot\core\input\input.cpp:775)
[16] Input::flush_buffered_events (C:\GD\godot\core\input\input.cpp:1056)
[17] DisplayServerWindows::process_events (C:\GD\godot\platform\windows\display_server_windows.cpp:3025)
[18] OS_Windows::run (C:\GD\godot\platform\windows\os_windows.cpp:1666)
[19] widechar_main (C:\GD\godot\platform\windows\godot_windows.cpp:181)
[20] _main (C:\GD\godot\platform\windows\godot_windows.cpp:208)
[21] main (C:\GD\godot\platform\windows\godot_windows.cpp:220)
[22] __scrt_common_main_seh (D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288)
[23] <couldn't map PC to fn name>
-- END OF BACKTRACE --
================================================================

The crash is in node_3d_editor_plugin.cpp in void Node3DEditorViewport::_select_region()

node is null and it isn't checked
L1070 sp as a Node* is not null, but the Object within is null.
L1058 instances.size() is 1, which is accurate in the steps to produce. Only Terrain3D is in the scene. However the selection box is intentionally empty.

In 4.2 this situation doesn't crash but does report the same error:
  scene\main\node.cpp:2350 - Condition "!is_ancestor_of(p_start_node)" is true. Returning: p_start_node
The crash was caused by https://github.com/godotengine/godot/pull/92188
Steps to reproduce
Setup

Load the MRP
Restart when prompted
In Project settings enable Terrain3D
Restart a second time

Bug

Create a new scene
Add a Terrain3D Node
In our toolbar, click any tool other than the first (region) so the cursor decal shows on the terrain
Click Escape to deselect Terrain3D
Click and drag a selection box in the viewport (doesn't matter on sky/terrain). Upon releasing the mouse Godot will crash.

Minimal reproduction project (MRP)
Includes a stripped down version of terrain3d from this specific build w/ windows & linux builds
test-select-crash.zip

The problem is setup by this line. If it is commented out it won't crash. In this current setup it might only queue up 3-10 timers. Unnecessary but not crash worthy. Not sure how it allows the selection issues below.
# project/addons/terrain_3d/src/ui.gd: update_decal()

		await get_tree().create_timer(.05).timeout

When the problem arises the Object* returned by get_instance is a decal, which is our mouse cursor. It shouldn't be selectable.
//node_3d_editor_plugin.cpp:void Node3DEditorViewport::_select_region() L1059

	for (int i = 0; i < instances.size(); i++) {
		Node3D *sp = Object::cast_to<Node3D>(ObjectDB::get_instance(instances[i]));

The decal is a descendant of our EditorPlugin, not the edited scene, so it errors here.
		Node *node = Object::cast_to<Node>(sp);
		if (node != edited_scene) {
			node = edited_scene->get_deepest_editable_node(node);
		}

The comment is good but what actually happens is it traces the parent up to EditorPlugin, EditorNode, Godot's Window, then null.
		// Prevent selection of nodes not owned by the edited scene.
		while (node && node != edited_scene->get_parent()) {
			Node *node_owner = node->get_owner();
			if (node_owner == edited_scene || node == edited_scene || (node_owner != nullptr && edited_scene->is_editable_instance(node_owner))) {
				break;
			}
			node = node->get_parent();
		}

Since there is no parent above Godot's window, accessing this null node causes a crash.
		// Replace the node by the group if grouped
		if (node->is_class("Node3D")) {

So we can fix our code. But Godot shouldn't be crashing, nor recursing up the tree beyond Godot's window, nor even looking at nodes not within the scene. It quite plausible that other editor plugins will create meshes in the scene for debug purposes that should not be selectable unless owned by the edited scene.


remove thrift protocol references

[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[X] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

Not needed. Thrift protocol support has been removed long back. These references still exist.


log add doris version info to query profile

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

rebase and rerun the all test

run buildall

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.04 seconds
stream load tsv:          448 seconds loaded<PHONE_NUMBER>9 Bytes, about 159 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          57 seconds loaded<PHONE_NUMBER> Bytes, about 18 MB/s
stream load parquet:          28 seconds loaded 861443392 Bytes, about 29 MB/s
insert into select:          25.8 seconds inserted 10000000 Rows, about 387K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230709174612_clickbench_pr_175334.html

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 54.05 seconds
stream load tsv:          502 seconds loaded<PHONE_NUMBER>9 Bytes, about 142 MB/s
stream load json:         19 seconds loaded<PHONE_NUMBER> Bytes, about 118 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size:<PHONE_NUMBER>7 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230710015320_clickbench_pr_175336.html

run build all

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 54.06 seconds
stream load tsv:          502 seconds loaded<PHONE_NUMBER>9 Bytes, about 142 MB/s
stream load json:         19 seconds loaded<PHONE_NUMBER> Bytes, about 118 MB/s
stream load orc:          64 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size:<PHONE_NUMBER>3 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711171141_clickbench_pr_176335.html

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 52.04 seconds
stream load tsv:          501 seconds loaded<PHONE_NUMBER>9 Bytes, about 142 MB/s
stream load json:         19 seconds loaded<PHONE_NUMBER> Bytes, about 118 MB/s
stream load orc:          64 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size:<PHONE_NUMBER>0 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711171147_clickbench_pr_176331.html

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 56.09 seconds
stream load tsv:          502 seconds loaded<PHONE_NUMBER>9 Bytes, about 142 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          64 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select:          29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size:<PHONE_NUMBER>2 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711200103_clickbench_pr_176513.html

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 52.59 seconds
stream load tsv:          505 seconds loaded<PHONE_NUMBER>9 Bytes, about 141 MB/s
stream load json:         19 seconds loaded<PHONE_NUMBER> Bytes, about 118 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          28.7 seconds inserted 10000000 Rows, about 348K ops/s
storage size:<PHONE_NUMBER>9 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711200216_clickbench_pr_176515.html

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 49.62 seconds
stream load tsv:          443 seconds loaded<PHONE_NUMBER>9 Bytes, about 161 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          57 seconds loaded<PHONE_NUMBER> Bytes, about 18 MB/s
stream load parquet:          28 seconds loaded 861443392 Bytes, about 29 MB/s
insert into select:          25.8 seconds inserted 10000000 Rows, about 387K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711131341_clickbench_pr_176512.html


UpdateQueue function now matches API docs and allows a Queue's name t…

…o be updated
Provide a general summary of your changes in the title above. You should
remove this overview, any sections and any section descriptions you
don't need below before submitting. There isn't a strict requirement to
use this template if you can structure your description and still cover
these points.
Description
The existing UpdateQueue implementation was not quite correct. The API endpoint only allows for updating a Queue's name. The existing name is passed in the URL, while the new name is passed as a parameter in the body. This PR updates the function to match the API expectations.
Has your change been tested?

Updated unit test
Tested manually with local terraform provider changes to update a Queue's name

Screenshots (if appropriate):
Types of changes
What sort of change does your code introduce/modify?

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ x] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[x ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x ] I have added tests to cover my changes.
[ x] All new and existing tests passed.
[ x] This change is using publicly documented (api.cloudflare.com or developers.cloudflare.com) and stable APIs.

Codecov Report

Merging #1188 (9452119) into master (6153c1e) will decrease coverage by 0.05%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##           master    #1188      +/-   ##
==========================================
- Coverage   49.40%   49.36%   -0.05%
==========================================
  Files         127      128       +1
  Lines       12290    12430     +140
==========================================
+ Hits         6072     6136      +64
- Misses       4840     4890      +50
- Partials     1378     1404      +26

Impacted Files
Coverage Δ

access_organization.go
64.70% <ø> (ø)

cloudflare.go
68.08% <0.00%> (-0.63%)
:arrow_down:

cloudflare_experimental.go
0.00% <0.00%> (ø)

magic_transit_ipsec_tunnel.go
46.15% <ø> (ø)

teams_rules.go
44.94% <0.00%> (-0.52%)
:arrow_down:

utils.go
72.72% <ø> (ø)

mtls_certificates.go
26.59% <26.59%> (ø)

workers_bindings.go
67.44% <40.00%> (-2.31%)
:arrow_down:

origin_ca.go
57.26% <90.90%> (-2.09%)
:arrow_down:

dns.go
72.72% <96.36%> (+9.00%)
:arrow_up:

... and 12 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

thanks!


extract() function broken due to GdsLibrary API change

Calling gdspy.GdsLibrary.extract() results in the following error:
File "/usr/local/lib/python3.7/site-packages/gdspy/library.py", line 2598, in extract
  cell = self.cell_dict.get(cell, cell)
AttributeError: 'GdsLibrary' object has no attribute 'cell_dict'

According to the docs, self.cell_dict is now self.cells.
This error is easily fixed by changing line 2598 in library.py:
cell = self.cell_dict.get(cell, cell)
to:
cell = self.cells.get(cell, cell)
While gdspy.GdsLibrary.extract() will be deprecated as discussed in Issue #113, having it broken is probably not desired.

Thanks! Fixed!


Create bug, feature and support issue templates

Instead of one general issue template we can use multiple issue templates to sort issue from the beginning. Especially useful are pre-defined labels such as kind/bug, kind/feature and kind/support.
/cc @brancz @mxinden @squat @paulfantom @s-urbaniak

GitHub docs specify uppercase name for an issue template directory. Does it work with lowercase?

@paulfantom Good guess, I expected it to work with lowercase but turns out it doesn't. :+1:
I've renamed the folder and you can now check it out at metalmatze/prometheus-operator-issue-templates.

lgtm


Announcement:  Highlighting RealSense ROS wrapper release 2.2.13 / YouTube demo of On-Chip Calibration

Hi everyone,
I would like to highlight that there is a new release of the RealSense ROS wrapper (version 2.2.13).
https://github.com/IntelRealSense/realsense-ros/releases
It supports the following features:

Librealsense SDK version 2.33.1
The forthcoming new RealSense L515 lidar depth camera
Simultaneous use of multiple T265 devices
Kinetic and Melodic ROS distributions

The installation instruction for Melodic is:
sudo apt-get install ros-melodic-realsense2-camera

Edit:  In other news, the official RealSense YouTube channel has published a demonstration video of the new On-Chip Calibration feature.
https://www.youtube.com/watch?v=pcoYc4YPVMY
An accompanying article has been published on the official RealSense blog:
https://www.intelrealsense.com/self-calibration-for-depth-cameras/
Edit 2  More news: the official RealSense YouTube account posted a new video clip of the Cubemos Skeleton Tracking SDK.
https://www.youtube.com/watch?v=cdPdoueaC-0

I really appreciate this. What kernel would you recommend for Ubuntu 18.04 for a production system, stability?

@antoan  You are very welcome!  The newest kernel supported in the RealSense SDK at the time of writing this is 5.3.  If production stability is a priority then a good general rule of thumb is to use "one version back", whether it be SDK, firmware or kernel.  So the second newest kernel supported is Kernel 5.

Would very be grateful if you could recommend a way to 'hold' a HWE 5.0 kernel which came with Ubuntu 14.04.3 (if I understand things correctly,  the hardware enablement stack is necessary in order to get a 5.0 kernel on Ubuntu 18.04), to prevent it changing to the next point release kernel - 5.3, during upgrades.
Was not able to find clear information on how to go about this elsewhere

The link below describes a manual cloning process for HWE.
https://github.com/IntelRealSense/librealsense/issues/4464#issuecomment-513343764


MudSelect not working in v6.0.17 WASM Hosted

Bug type
Component
Component name
MudSelect
What happened?
I made a MudSelect and added enum values which resultated in an error.
Complex value = error.
String value = error.
I have been searching the forums for this bug without luck.
Expected behavior
MudSelect working and not throwing errors
Reproduction link
https://try.mudblazor.com/snippet/mkwQbkGZCauEFEfy
Reproduction steps

Upgrade to 6.0.17
Create a MudSelect with any type
Try and select a value
...

Relevant log output
Microsoft.AspNetCore.Components.WebAssembly.Rendering.WebAssemblyRenderer[100]
      Unhandled exception rendering component: Could not find 'mudKeyInterceptor.updatekey' ('updatekey' was undefined).
      Error: Could not find 'mudKeyInterceptor.updatekey' ('updatekey' was undefined).
          at https://localhost:44342/_framework/blazor.webassembly.js:1:328
          at Array.forEach (<anonymous>)
          at a.findFunction (https://localhost:44342/_framework/blazor.webassembly.js:1:296)
          at _ (https://localhost:44342/_framework/blazor.webassembly.js:1:2437)
          at https://localhost:44342/_framework/blazor.webassembly.js:1:3325
          at new Promise (<anonymous>)
          at Object.beginInvokeJSFromDotNet (https://localhost:44342/_framework/blazor.webassembly.js:1:3306)
          at Object.St [as invokeJSFromDotNet] (https://localhost:44342/_framework/blazor.webassembly.js:1:59938)
          at _mono_wasm_invoke_js_blazor (https://localhost:44342/_framework/dotnet.6.0.9.u26l3xc0xr.js:1:195300)
          at wasm://wasm/00971d1a:wasm-function[219]:0x1a492
Microsoft.JSInterop.JSException: Could not find 'mudKeyInterceptor.updatekey' ('updatekey' was undefined).
Error: Could not find 'mudKeyInterceptor.updatekey' ('updatekey' was undefined).
    at https://localhost:44342/_framework/blazor.webassembly.js:1:328
    at Array.forEach (<anonymous>)
    at a.findFunction (https://localhost:44342/_framework/blazor.webassembly.js:1:296)
    at _ (https://localhost:44342/_framework/blazor.webassembly.js:1:2437)
    at https://localhost:44342/_framework/blazor.webassembly.js:1:3325
    at new Promise (<anonymous>)
    at Object.beginInvokeJSFromDotNet (https://localhost:44342/_framework/blazor.webassembly.js:1:3306)
    at Object.St [as invokeJSFromDotNet] (https://localhost:44342/_framework/blazor.webassembly.js:1:59938)
    at _mono_wasm_invoke_js_blazor (https://localhost:44342/_framework/dotnet.6.0.9.u26l3xc0xr.js:1:195300)
    at wasm://wasm/00971d1a:wasm-function[219]:0x1a492
   at Microsoft.JSInterop.JSRuntime.<InvokeAsync>d__16`1[[Microsoft.JSInterop.Infrastructure.IJSVoidResult, Microsoft.JSInterop, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=adb9793829ddae60]].MoveNext()
   at Microsoft.JSInterop.JSRuntimeExtensions.InvokeVoidAsync(IJSRuntime jsRuntime, String identifier, Object[] args)
   at MudBlazor.Services.KeyInterceptor.UpdateKey(KeyOptions option)
   at MudBlazor.MudSelect`1.<OpenMenu>d__159[[System.String, System.Private.CoreLib, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at MudBlazor.MudSelect`1.<ToggleMenu>d__158[[System.String, System.Private.CoreLib, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at Microsoft.AspNetCore.Components.ComponentBase.CallStateHasChangedOnAsyncCompletion(Task task)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.GetErrorHandledTask(Task taskToHandle, ComponentState owningComponentState)
window.Module.s.printErr @ blazor.webassembly.js:1

Version (bug)
6.0.17
Version (working)
6.0.9
What browsers are you seeing the problem on?
Chrome
On what operating system are you experiencing the issue?
Windows
Pull Request

[ ] I would like to do a Pull Request

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Did you try to clear cache? Can press ctrl + f5 or clean cache manually. The error look like not about enum or wasm hosted.

Hi @mckaragoz I do not know why I did not think about clearing the cache! Silly me! It worked as a charm :)
Yeah I did not really expect it to be related to the value type, but thought it would be worth mentioning.
Thanks a mil!! Guess I will dedact this issue...


Error executing golangci-lint

Trying to figure out why I'm getting this error :(
Error: Dependencies file is not found in /home/runner/work/kvstok/kvstok. Supported file pattern: go.sum

I'd like to take this up! Can you send some documented guide or any tutorial which I can follow before proceeding to it?

Hi @nikzayn I'll describe

Hi @nikzayn the main goal here, is to fix golanci-lint action to execute on every commit check to certify that everything is correct according. As a reference you can take a look here: https://github.com/golangci/golangci-lint-action

Fix error on execute lint.

Removed config file and adjust workflow script

@holopin-bot @nikzayn

@holopin-bot @nikzayn

@holopin-bot @nikzayn

@holopin-bot @nikzayn

@holopin-bot @nikzayn


Create Saved Cart adapter

The only required call to use for the feature is :
saveCart() using /{baseSiteId}/users/{userId}/carts/{cartId}/save
We should create an adapter for save cart that contains this call and in the future might contain other.
This feature requires to use the "adapter pattern". Therefore, create a connector, adapter and occ adapter. Please see the cart adapter for an example.
Note: At the moment this call returns an error for anonymous cart

Will not be tested individually


PHP xsl extension broken

Describe the bug
Install php with xsl extension
To Reproduce
Steps to reproduce the behavior:
PHP Warning:  PHP Startup: Unable to load dynamic library '/nix/store/frr0k4dmfd02y4b6nq7shpw3q4gcjb7s-php-xsl-7.4.16/lib/php/extensions/xsl.so' (tried: /nix/store/frr0k4dmfd02y4b6nq7shpw3q4gcjb7s-php-xsl-7.4.16/lib/php/extensions/xsl.so (/nix/store/frr0k4dmfd02y4b6nq7shpw3q4gcjb7s-php-xsl-7.4.16/lib/php/extensions/xsl.so: undefined symbol: dom_node_class_entry), /nix/store/6r68y0fcr9k50lznk493w7iwh6jgna17-php-7.4.16/lib/php/extensions//nix/store/frr0k4dmfd02y4b6nq7shpw3q4gcjb7s-php-xsl-7.4.16/lib/php/extensions/xsl.so.so (/nix/store/6r68y0fcr9k50lznk493w7iwh6jgna17-php-7.4.16/lib/php/extensions//nix/store/frr0k4dmfd02y4b6nq7shpw3q4gcjb7s-php-xsl-7.4.16/lib/php/extensions/xsl.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
PHP 7.4.16 (cli) (built: Mar  2 2021 10:35:27) ( ZTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
with Zend OPcache v7.4.16, Copyright (c), by Zend Technologies
Expected behavior
The extension is installed without errors
Screenshots
Additional context
Notify maintainers
Metadata

system: "x86_64-linux"
host os: Linux 5.10.0-0.bpo.3-amd64, Ubuntu, 21.04 (Hirsute Hippo)
multi-user?: no
sandbox: no
version: nix-env (Nix) 2.3.10
channels(root): "nixpkgs-21.05pre273941.a2b0ea6865b"
channels(dev): "nixpkgs-21.05pre286880.7cb76200088"
channels(projector-user): "nixpkgs-21.05pre282843.dcdf30a78a5"
nixpkgs: /home/dev/.nix-defexpr/channels/nixpkgs

Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute: php
# a list of nixos modules affected by the problem
module:

Whats your config for installing it?  I got same error when installing using:
(php.withExtensions ({ all, ... }: with all; [ xsl ]))
but it works for me when using:
(php.withExtensions ({ enabled, all }: enabled ++ [ all.xsl ]))
code is from https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/php.section.md
My problem is composer is not finding xsl, and the error message suggests its not loading the same ini files as cmdline php somehow.

Status of this issue ?


Xnero: Watch buzztowns(?!.com)

Xnero requests the watch of the watch_keyword buzztowns(?!\.com). See the MS search here and the Stack Exchange search in text, in URLs, and in code.

buzztowns(?!\.com) has been seen in 0 true positives, 0 false positives, and 0 NAAs.

Approved by Spevacus in Charcoal HQ


Oxygene language lexer

Added a lexer for the Oxygene language (http://www.remobjects.com/oxygene). Will also be submitting to https://bitbucket.org/birkenfeld/pygments-main, as well.

The files in vendor/ is not a part of pygments.rb. They should be managed by python package managers.


Visual Studio Crashes when opening solution with Audit.net

Every time a solution is loaded it crashes immediately when audit.net is enabled. More details can be found here - https://developercommunity.visualstudio.com/content/problem/99433/visual-studio-crashes-when-opening-solution-3.html
This has already been fixed via https://github.com/OSSIndex/audit.net/commit/cf24459deca6a7898d5248cc22242ad98a89c38b so please take care of this.

Thanks for the report. It appears we have not kept the GitHub tags up to date with what has been uploaded to the marketplace. I believe that the code you have linked is already in the uploaded release. I am currently upgrading Visual Studio and will try to reproduce the problem. Even if I cannot reproduce the issue I will re-spin a release since the previous was generated using a pre-release version of Visual Studio 2017.
I'll do my best to resolve this as quickly as possible.

I have reproduced the issue and am working on a solution.

It seems that the problem resides in the code which checks to see if a project supports Nuget packages. The used method throws exceptions of various sorts depending on the status of the project. Originally we were only checking for InvalidOperationException, but it appears that other project types can cause other exceptions.
For that reason we now catch any exception, like so:
            try
            {
                // FIXME: This should not happen
                if (project == null) return false;
                IVsPackageInstallerServices locator = ServiceLocator.GetInstance<IVsPackageInstallerServices>();
                // FIXME: This should not happen
                if (locator == null) return false;
                locator.IsPackageInstalled(project, "__dummy__");
                return true;
            }
            catch (InvalidOperationException)
            {
                return false;
            }
            catch (Exception)
            {
                // FIXME: A variety of project types which do not work with the IsPackageInstalled method will throw exceptions of various sorts.
                // FIXME: Surely there is a better way to check for Nuget support?
                return false;
            }

We kept the original catch, since that is the one we expected. This new catch should prevent any crashing, anyhow, even if it results in the plugin not working in some situations. I am generating a new distribution now.

What is curious is that I know this was working in earlier versions, including the 2017RC. It seems that something changed since then.


Support marathon 1.5+

In marathon 1.5, metric collection have been switched to kamon library.
Gauge implementation in kamon is:

Our Gauge is a mix between the Histogram and the MinMaxCounter, taking
measurements of a given value every 100 milliseconds by default and
storing the observed values in a Histogram.
(http://kamon.io/documentation/0.6.x/kamon-core/metrics/instruments/)

Since kamon metrics are "flushed" every 10 seconds, using max (or min)
of the kamon gauge gives one observed value of the gauge.
This patch makes this exporter not compatible with marathon 1.5+
Upstream ticket (https://jira.mesosphere.com/browse/MARATHON-7673) has
not been fixed yet, it seems reasonnable to assume this will stay like
this.

cc @clems4ever

Any progress on this?

apparently not. We've been using this with success for quite a while now

@matt-deboer could you consider this PR?

@kamaradclimber : I no longer maintain or have ownership of this repo (or work with Marathon for that matter). The best I can suggest is that you create/maintain your own fork.
I don't think anyone in the getty_images org still uses/maintains this. If you would like to take over ownership of marathon_exporter, we could request @mapitman to add a link to your fork from the readme of this repo...

@kamaradclimber would you like to take ownership of this repo? I believe I can transfer it to you. It may be prudent for me to accept any merge requests you would want to allow first.

I'll merge this now. Still would like to pass this repo off to someone who wants to maintain it!

@mapitman yes I would be interested in maintaining this repository. I'll probably maintain it as part of my day job (criteo) since we heavily use marathon.
How could transfer work?


[Feature Request] make SLM support quant by CPU in weight convert process

🚀 Feature
make SLM support quant by CPU in weight convert process
Motivation
legacy way by using mlc_llm.build can use CPU to quant weight in android model build process .
but current SLM's mlc_chat convert_weight  process doesn't support CPU quantization.
when user try to build model on a machine without enable x86's TVM GPU build option or convert model without GPU env, the process will fail.

The modification I tried was to add in check_device.py to break out of the current loop when device_str == "cpu". Then you can use cpu quantization.

The modification I tried was to add in check_device.py to break out of the current loop when device_str == "cpu". Then you can use cpu quantization.

thanks, I modified auto_target.py and convert_weight.py then quantization passed


update 0.10.0

Executed sync_proto.sh & generate_proto_go.sh.
Now tests fail with
go test ./...
# github.com/qdrant/go-client/examples
examples/main.go:58:20: invalid operation: cannot indirect 4 (untyped int constant)
examples/main.go:59:20: invalid operation: cannot indirect pb.Distance_Dot (constant 3 of type go_client.Distance)

I push a quick fix. Tests shuld pass now.
Background info: VectorSize and Distance fields are typed with pointers to their values. Primitive literals and other constants have no addresses because there's no memory allocated for them, so we cannot use & operator directly. Instead we create helper variables to get a pointer to them.
This is a well known issue in Go that is actively worked on.

@monatis Thank you very much for your help!

go run examples/main.go
2022/09/13 10:14:52 Qdrant version: 0.9.1
2022/09/13 10:14:52 Collection test_collection deleted
2022/09/13 10:14:53 Collection test_collection created
2022/09/13 10:14:53 List of collections: [name:"test_collection" name:"test" name:"stress_collection"]
2022/09/13 10:14:53 Field index for field city created
2022/09/13 10:14:53 Field index for field count created
2022/09/13 10:14:53 Upsert 6 points
2022/09/13 10:14:53 Retrieved points: [id:{num:1} id:{num:2}]
2022/09/13 10:14:53 Found points: [id:{num:4} score:1.362 version:2 id:{num:1} score:1.273 version:2 id:{num:3} score:1.208 version:2]
2022/09/13 10:14:53 Found points: [id:{num:4} score:1.362 version:2 id:{num:2} score:0.871 version:2]


Run doctests on examples in the docstrings in the source code

Set the doctests to run not only on the README.md but also on the examples that are now baked into the source code.

@gsarma Hey, what do you think about this?


Cleaning requirement + git fix

What does this PR do?
Resolving a special case for developers who were affected by #14638 and later fix #14644
Normally clean clone would also help :otter:
mypy: set connector as skip as a method without proper typing was moved in #14861
Before submitting

[ ] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[ ] Did you make sure to update the documentation with your changes? (if necessary)
[ ] Did you write any new necessary tests? (not for typos and docs)
[ ] Did you verify new and existing tests pass locally with your changes?
[ ] Did you list all the breaking changes introduced by this pull request?
[ ] Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
Make sure you had fun coding 🙃
cc @borda

Maybe @williamFalcon can test this first on his side and then merge the PR if it's good?
Probably needs to pip install -e after pulling that branch, right @Borda?

Maybe @williamFalcon can test this first on his side and then merge the PR if it's good? Probably needs to pip install -e after pulling that branch, right @Borda?

there is some problem with using -e :/, see: https://github.com/Lightning-AI/lightning/issues/14847#issuecomment-1255106639

@Borda it's broken again...
(l1) ➜  lightning git:(master) pip install -e .
Obtaining file:///Users/williamfalcon/Developer/lightning
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... error
  error: subprocess-exited-with-error

  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> [25 lines of output]
      Traceback (most recent call last):
        File "/Users/williamfalcon/opt/miniconda3/envs/l1/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/Users/williamfalcon/opt/miniconda3/envs/l1/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/Users/williamfalcon/opt/miniconda3/envs/l1/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 144, in get_requires_for_build_editable
          return hook(config_settings)
        File "/private/var/folders/78/_1r_8s7d74l4gsgt9w25s0000000gn/T/pip-build-env-_6bjsfp9/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 445, in get_requires_for_build_editable
          return self.get_requires_for_build_wheel(config_settings)
        File "/private/var/folders/78/_1r_8s7d74l4gsgt9w25s0000000gn/T/pip-build-env-_6bjsfp9/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/private/var/folders/78/_1r_8s7d74l4gsgt9w25s0000000gn/T/pip-build-env-_6bjsfp9/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/78/_1r_8s7d74l4gsgt9w25s0000000gn/T/pip-build-env-_6bjsfp9/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 482, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/private/var/folders/78/_1r_8s7d74l4gsgt9w25s0000000gn/T/pip-build-env-_6bjsfp9/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 100, in <module>
        File ".actions/setup_tools.py", line 526, in _load_aggregate_requirements
          requires = [
        File ".actions/setup_tools.py", line 527, in <listcomp>
          load_requirements(d, file_name="base.txt", unfreeze=not freeze_requirements)
        File ".actions/setup_tools.py", line 120, in load_requirements
          with open(os.path.join(path_dir, file_name)) as file:
      FileNotFoundError: [Errno 2] No such file or directory: 'requirements/docs/base.txt'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
(l1) ➜  lightning git:(master) ✗

@williamFalcon are you on clean master? It seems like you have a directory requirements/docs, but no file requirements/docs/base.txt. The install script tries to install all the base requirements of all subdirectories present in requirements folder.
Btw, I think it should not loudly fail, when such a requirement file is not found, but only warn


Scroll to top button

Describe the bug
Scroll to top button doesn't work properly
@Amit366 can i work on this issue??

@kumarishalini6 it works properly

Ok


cmake: linker: make passing -no-pie configurable

This adds a new linker property specifically for passing
"-no-pie" to linker. Some linkers (such as the one in
xcc-clang) do not support this flag and will behave erratically
if set (such as parsing it as "-n" and "-o"). By having it
as a linker property, we can avoid passing it to problematic
linkers.
Signed-off-by: Daniel Leung<EMAIL_ADDRESS>

Need a bit more description to fully understand the use-case.
Also, this should be having corresponding Kconfig setting, and not rely on board testing directly in CMake.

The Kconfig part may not be required.
Seems to be toolchain related, and hence the toolchain properties should be defined properly.
So that's first point.
Of course if a better user-facing handling of when to apply the properties is desired, then those should go through Kconfig as described. But the immediate problem described in #54333 doesn't seem to need it.

I get that if Zephyr SDK and other currently do not produce PIE binaries per default, but what is the problem in specifying the flag in those cases then ?

Others can comment with more expertise, but the core issue is that this flag isn't supported compatibly across all binutils/gcc/clang/lld variants.  And we don't really want it anyway, it's not a feature we enable at code generation so we don't want the linker to have to care about it.
The sole exception (I think! @yperess caught this originally) is on Debian when using the distro-provided clang/lld to produce native_posix binaries.  There, the toolchain is hard-wired to generate pic/pie by default, so we need the flag to the linker to unbreak things.  But that requires per-toolchain hackery.
I think.

@tejlmand I have updated the PR to use linker property. I tried this previously but was having issue with using zephyr_ld_options(). Using zephyr_link_libraries() got past that issue.

There [on Debian], the toolchain is hard-wired to generate pic/pie by default, so we need the flag to the linker to unbreak things. But that requires per-toolchain hackery.

Stating the obvious: this was done to increase security
https://wiki.debian.org/Hardening/PIEByDefaultTransition
https://wiki.ubuntu.com/SecurityTeam/PIE
A lot of background information on these pages, including this (not) fun bit:

Note that some of these packages have already taken patches upstream from distros that were using patched versions of GCC that enabled PIE by default and added a GCC option -nopie to disable it. However, upstream GCC has settled on the -no-pie option as the means for disabling PIE when GCC is configured to use it by default.

Similar headaches:

https://github.com/mesonbuild/meson/pull/6415

I think something is wrong in cmake scripts... the error said it was using /usr/bin/ld, but yet the LLD linker cmake files were parsed instead.

I tried this previously but was having issue with using zephyr_ld_options().

Yes, because that function was made to be able to check if a flag is valid, so it only handles simply flags and not generator expressions.

Using zephyr_link_libraries() got past that issue.

yes, and this is the function to use now, because code is updated to use linker properties.

I think something is wrong in cmake scripts... the error said it was using /usr/bin/ld, but yet the LLD linker cmake files were parsed instead.

Sounds strange. Will see if I can re-produce.

I think something is wrong in cmake scripts... the error said it was using /usr/bin/ld, but yet the LLD linker cmake files were parsed instead.

Sounds strange. Will see if I can re-produce.

Turns out if you only have LLVM/clang/lld 15 installed, the symlink /usr/bin/ld.lld would not be there as it comes from package lld (which pulls in lld-14). Therefore, using find_program() to find ld.lld would not work since it's no in PATH. CMake then defaults to using /usr/bin/ld. I think this is what is causing the build failure. I have added a commit to expand the search to LLVM_TOOLCHAIN_HOME/bin (which basically is TOOLCHAIN_HOME).
However, another issue is that CONFIG_LLVM_USE_LD=y does not have any effect. I am not totally sure why... my guess is that the cmake scripts under cmake/toolchain is parsed before kconfigs, and none of CONFIG_* exist.

Errr... the new commit doesn't seem to work with the Clang/LLVM workflow.

Can you please rename the brand new xcc-clang/ directory to xt-clang/?
There are two toolchains variants: xt-xcc (gcc-based) and xt-clang, so naming anything xcc-clang does not seem to make sense.
See
https://thesofproject.github.io/latest/introduction/index.html#toolchain-faq
https://github.com/thesofproject/sof/pull/7027#discussion_r1095792670

Can you please rename the brand new xcc-clang/ directory to xt-clang/?
There are two toolchains variants: xt-xcc (gcc-based) and xt-clang, so naming anything xcc-clang does not seem to make sense.
See https://thesofproject.github.io/latest/introduction/index.html#toolchain-faq thesofproject/sof#7027 (comment)

That should be a follow PR since xcc-clang already exists in the tree.

That should be a follow PR since xcc-clang already exists [ELSEWHERE] in the tree.

Missed the other places sorry.
But... why couldn't this be fixed before? PIE flavors are complex already, "mashup" toolchain names don't really help IMHO.
It's not like anyone uses xt-clang yet, is it? Then better fix this quickly before anyone does and complains.

But... why couldn't this be fixed before? PIE flavors are complex already, "mashup" toolchain names don't really help IMHO.
It's not like anyone uses xt-clang yet, is it? Then better fix this quickly before anyone does and complains.

The rename can come before or after, just shouldn't get mixed up in this PR.  Its 2 completely unrelated changes.

Some linkers (such as the one in xcc-clang) do not support this flag and will behave erratically if set (such as parsing it as "-n" and "-o").

I don't think this commit message makes sense. As found by @andyross in https://github.com/zephyrproject-rtos/zephyr/pull/54331#issuecomment-1416016501, -no-pie has been "ping-ponging around quite a bit in the last few months". If this fix can't be described clearly then the "ping-ponging" will surely continue.
First there is no such thing as xcc-clang, see above. I get that we don't want to fix the directory names yet but that does not require the commit messages and comments in the source to keep using the wrong name and spread the confusion even further. English text can start using the accurate name already. "xcc-clang" is especially confusing considering this issue at hand affects xt-clang but not xt-xcc.
A more important issue with this commit message is that the linker in xt-clang 2022 has not changed. xt-clang 2022 still uses GNU ld just like in xt-xcc did. So this PR is not fixing a problem with "some linkers", at least not as far as xt-clang is concerned. If anything it's more about clang's inability to pass a GNU ld linker option to GNU ld without using -Wl

Its 2 completely unrelated changes.

This PR is trying to fix some compatibility issue(s) between "xcc" and "clang" so I think carrying forward a super confusing and wrong "xcc-clang" name that concatenates the two is actually quite relevant.

Ok. I think I got it mostly figured out. The CI failure is due to https://github.com/zephyrproject-rtos/zephyr/issues/54720 (which qemu_arc_em failed tests/kernel/mem_protect/syscalls/).

@evgeniy-paltsev / @ruuddw  can you verify on metaware?

can you verify on metaware?

@nashif  Not sure what your question is, test the CI failure on nSIM or check if this PR breaks anything for MetaWare?
Anyway, I think for both the answer is: no problems.

@nashif MetaWare doesn't have the '-no-pie' option, but no_postion_independent is already handled by the arcmwdt cmake files. I don't see any conflicts. Thanks for checking!

Hi there,
we have some issues when building SOF project on Windows with xt-clang toolchain and this PR.
I've cherry picked your commits and run a build on my PR https://github.com/thesofproject/sof/pull/7027 .
I've got following error log:
output.log
Uploading also part of my environment setup:
env.log
Please note, that on Windows we use MSYS to provide gcc implementation for Windows and other linux tools.
This is required by both Zephyr SDK and SOF project that builds firmware tools using native linux toolchains.
So when I set all the environmetal variables to select xt-clang toolchain, C:\msys64\usr\bin\ld.bfd.exe linker is used (what is not desirable since xt-clang linker should be selected).

Ok, it looks like we need to set TOOLCHAIN_HOME env variable from now on to avoid those issues...

Any update on this? We have a strong dependency to this PR.

Any update on this? We have a strong dependency to this PR.

Have you tried #54836? That should get rid of the -no-pie warning.

Have you tried #54836? That should get rid of the -no-pie warning.

Would like to see this cleanup merged as need it form arm-llvm as well ;).

Any update on this? We have a strong dependency to this PR.

Have you tried #54836? That should get rid of the -no-pie warning.

Thanks for pointing me in right direction!

Hm... compliance check complains about CONFIG_LLVM_* because they are not available unless LLVM is used. Not sure if I should add these to the kconfig exempt list...

I have updated the PR to use find_package() for both ld and lld.

Compliance failure is due to the kconfig file being selectively included (based on which toolchain being used), and thus any documentation on those kconfigs generated check errors as clang is not used by default (and thus not the checked).

The last change was simply adding the kconfigs to the compliance check ignore list.

In some conditions, the commit that adds GNULD_LINKER_IS_BFD causes very confusing "-fuse-ld=bfd: unknown flag failures:

#56501


Error in if (n > 0) c(NA_integer_, -n) else integer()

I'm connected to the cluster.
sc <- spark_connect(master = "yarn-client", version = "2.3.1")
and i create one data.frame and copy to tbl_spark:
df<- copy_to(sc, data.frame(x=c(1, NA, 2), y=c(NA, 2, NA), z=42))
the following error arises, when I try to view or do any operation:
df

Error in if (n > 0) c(NA_integer_, -n) else integer() :
missing value where TRUE/FALSE needed

Fix on its way, CRAN will be also patched in the next couple days.

@hanfernandes fix available by reinstalling with:
devtools::install_github(“rstudio/sparklyr”)


How to make gen_pose_map_cano_smplx.py

Very nice work! I tried using smplx to train my own videos, but I found that the gen_pose_map_cano_smplx.py is missing. So, I tried to mimic the model's call to make this file, but I failed. Can you tell me how to make this file or provide it? Thank you.
Here is the code I imitated：
smpl_model = smplx.SMPLX(model_path='../assets/smpl_files/smplx', use_pca=False, num_pca_comps=45, flat_hand_mean=True, batch_size=1)
cano_smpl = smpl_model.forward(betas = smpl_data['beta'],
global_orient = smplx_cpose_param[:, :3],
transl = torch.tensor([[0, 0.30, 0]]),
body_pose = smplx_cpose_param[:, 3:66],
jaw_pose = smplx_cpose_param[:, 66:69],
leye_pose=smplx_cpose_param[:, 69:72],
reye_pose=smplx_cpose_param[:, 72:75],
left_hand_pose= smplx_cpose_param[:, 75:120],
right_hand_pose= smplx_cpose_param[:, 120:])
However, doing so would result in errors in the body_model.py or lbs.py files. Could you tell me where the problem is?  I would greatly appreciate it if you could reply.

Hello, have you solved this problem?

Hi. @superhero177  I found that the batch_size of "betas" in gen_pose_map_cano_smpl is only 1, while "betas" in smplx I provided are not 1. Therefore, I used the first frame's "betas" or the average "betas" as input for smpl_model, which may cause some issues. If you have a better solution, you can share it.

"betas": results["smpl_betas"].mean(axis=0),
I just discovered that this is done in the pre-processing file of InstantAvatar, which may be a better option on average.

ooh！Thank you,  I think this is a good approach.

Hi, sorry for the delayed response. It would help if you modified the function 'save_obj' in gen_pose_map_cano_smpl.py to save the corresponding SMPL-X obj and set uv_template_fn=the path of SMPL-X template.
This is the save_obj code for SMPLX-X:
smpl_data = torch.load( data_path + '/smpl_parms.pth')
smpl_model = smplx.SMPLX(model_path='/mnt/disk/avatar/gaussian_map/assets/smpl_files/smplx',gender = 'neutral', use_pca=False, num_pca_comps=45, flat_hand_mean=True, batch_size=1)

cano_dir = os.path.join(data_path,)
live_smpl = smpl_model.forward(betas = smpl_data['beta'][0][None],
                            transl = torch.tensor([[0, 0.35, 0]]),
                            global_orient=smplx_cpose_param[:, :3],
                            body_pose=smplx_cpose_param[:, 3:66],
                            jaw_pose=smplx_cpose_param[:, 66:69],
                            leye_pose=smplx_cpose_param[:, 69:72],
                            reye_pose=smplx_cpose_param[:, 72:75],
                            left_hand_pose=smplx_cpose_param[:, 75:120],
                            right_hand_pose=smplx_cpose_param[:, 120:])

ori_vertices = live_smpl.vertices.detach().cpu().numpy().squeeze()
joint_mat = live_smpl.A
print(joint_mat.shape)
torch.save(joint_mat ,join(cano_dir, 'smplx_cano_joint_mat.pth'))

mesh = trimesh.Trimesh(ori_vertices, smpl_model.faces, process=False)
mesh.export('%s/%s.obj' % (cano_dir, 'cano_smplx'))

Thanks for your reply. But I am still a bit confused about the role of the second stage training. I noticed that other issues mentioned that the role of the second stage training is not obvious. Could you tell me the significance of two-stage training.


Feat/516

this PR introduces a force git init flag --force-git-init to make sure that a fresh git repo is initialise, without the parent repository discovery mechanism.
In case the target folder has already a git repo, it would not be touched.
It fixes #516

/rebase


amplify pull sandbox fails with amplify-meta.json does not exist error

How did you install the Amplify CLI?
curl
If applicable, what version of Node.js are you using?
No response
Amplify CLI Version
12.4.0
What operating system are you using?
mac
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
no
Describe the bug
Creating a sandbox application and running a pull causes the following error
Error: File at path: '/Users/<path>/myapp/amplify/backend/amplify-meta.json' does not exist
Successfully generated models. Generated models can be found in /Users/<path>/rn/myapp/src

causing being unable to add resources to the project
Expected behavior
not error on pull
Reproduction steps

cerate a app on https://sandbox.amplifyapp.com/
copy and run the pull command example: amplify pull --sandboxId a51fb968-6335-4d95-9ad3-b024878001e7
observe error

Project Identifier
No response
Log output

# Put your logs below this line

Additional information
No response
Before submitting, please confirm:

[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

confirmed this is a bug

note for fix: this flow does a partial "pull" in the sense that gitignored files such as amplify-meta are not created (likely because we do not yet have an App ID). If amplify codegen models is the operation that is dependent on amplify-meta.json we can remove this execution.
It is also important to note that Amplify Studio no longer automatically enables DataStore, removing the need for modelgen

closing as wont fix


One cp command is sufficient to copy both files.

Removed multiple cp commands.

#summerhack

Congratulations @ankitkachru for your PR getting merged! Thanks for taking part in OpenEBS Hackfest.
Fill up the doc form to get the Hackfest T-Shirts and Swags
https://docs.google.com/spreadsheets/d/1TZYIdERnD72YPeUsdS5hDfI6xgXXCax4-DHpXh-o6r0/edit?usp=sharing
Looking for some more awesome contributions from you and win more prizes


WIP Fix findByProductsId

WIP Fix findByProductsId

This method was searching with the `productId` but matching on the `id` of
the `metadataEntity` rather than the `productEntity`. The tests were
passing because they only ever created one `productEntity` and one
`metadataEntity` so the sequential table ids always matched.

TODO:
I don't think we need this dao function since the corresponding service function isn't used by anything other than a test: https://github.com/alphagov/pay-products/blob/0ebadf89d9df6ff82fc22ae14aed89fe7ee1b737/src/main/java/uk/gov/pay/products/service/ProductsMetadataFinder.java#L23
Check whether that's right and if so remove the function and its tests. Might be worth checking out the other functions.
WHAT YOU DID
Worth validating this since I'm on holiday tomorrow and I'm scribbling this down before I go....if I'm mistaken the please close it and carry on about your day
This was highlighted when testing on Concourse. This test consistently failed: https://github.com/alphagov/pay-products/blob/master/src/test/java/uk/gov/pay/products/persistence/dao/ProductMetadataDaoIT.java#L58
For some reason the truncating of tables doesn't seem to work quite as expected running on Concourse which means there are multiple products and associated metadata. This results in productEntity.id != metadata.id which highlights this problem. When the IT tests run as expected the metadata.id == product.id (they are both 1) and this isn't noticed. In the example below the test using the unmodified doa function is trying to find rows where the id is 23 but it should be looking in the product_id column instead, however it finds nothing and the test fails.
postgres=# select * from products_metadata;
 id | product_id | metadata_key | metadata_value | version
----+------------+--------------+----------------+---------
  1 |          5 | key          | value          |       0
  2 |          5 | secondkey    | value2         |       0
  3 |         23 | a key        | a value        |       1

Would be worth checking the other methods and tests to be sure.

https://github.com/alphagov/pay-products/pull/538 think I'm going to do this instead

Thanks for your investigation and thorough notes though!


Hello, thank you so much for this new update but I was wondering if you are working on a new update for the new cards available?

Hello, thank you so much for this new update but I was wondering if you are working on a new update for the new cards available after the recent update released my master duel on February 14th?

There's a pinned issue called "New updates" https://github.com/pixeltris/YgoMaster/issues/130 with this information


[Isolate.exit] Unpredictable performance.

This issue is about the efficiency guarantees of the new Isolate.exit feature.
Isolate.exit (via #36097) as a possible solution to #40653 seems to work ~50% of the time.
The following example uses the new Isolate.exit functionality to encode and decode json on a different isolate to not drop any frames (Timer.tick is used as a proxy for frames).

Code
import 'dart:async';
import 'dart:convert';
import 'dart:isolate';
import 'dart:math';

Future<void> main() async {
  for (int i = 0; i < 10; i++) {
    await run();
  }
}

Future<void> run() async {
  final frames = <int>[];
  final timer = Timer.periodic(
    const Duration(milliseconds: frameDurationIsMS),
    (final timer) => frames.add(timer.tick),
  );
  final jsonString = () {
    final rnd = Random();
    final map = <String, dynamic>{
      for (int i = 0; i < 500000; i++)
        i.toString(): [
          rnd.nextInt(1000),
          () {
            final rand = Random();
            final codeUnits = List.generate(
              10,
              (final index) => rand.nextInt(33) + 89,
            );
            return String.fromCharCodes(codeUnits);
          }()
        ],
    };
    return json.encode(map);
  }();
  final map = await runAsyncOperation<dynamic>(
    description: "json.decode",
    operation: () => asyncCompute<String, dynamic>(
      fn: json.decode,
      input: jsonString,
    ),
    getTimerTick: () => timer.tick,
  );
  await Future<void>(() {});
  final encodedMap = await runAsyncOperation<dynamic>(
    description: "json.encode",
    operation: () => asyncCompute<Map<dynamic, dynamic>, String>(
      fn: json.encode,
      input: map.result as Map<dynamic, dynamic>,
    ),
    getTimerTick: () => timer.tick,
  );
  timer.cancel();
  output(
    ops: [
      map,
      encodedMap,
    ],
    ticks: frames,
  );
}

Future<O> asyncCompute<I, O>({
  required final FutureOr<O> Function(I) fn,
  required final I input,
}) async {
  final receivePort = ReceivePort();
  final inbox = StreamIterator<dynamic>(
    receivePort,
  );
  await Isolate.spawn<_IsolateMessage<I, O>>(
    (final message) => Isolate.exit(
      message.sendPort,
      message.fn(
        message.input,
      ),
    ),
    _IsolateMessage(
      sendPort: receivePort.sendPort,
      fn: fn,
      input: input,
    ),
  );
  final movedNext = await inbox.moveNext();
  assert(
    movedNext,
    "Call to moveNext is expected to return true.",
  );
  final typedResult = inbox.current as O;
  receivePort.close();
  await inbox.cancel();
  return typedResult;
}

class _IsolateMessage<I, O> {
  final SendPort sendPort;
  final FutureOr<O> Function(I) fn;
  final I input;

  const _IsolateMessage({
    required final this.sendPort,
    required final this.fn,
    required final this.input,
  });
}

const int frameDurationIsMS = 1000 ~/ 120;

Future<Operation<T>> runAsyncOperation<T>({
  required final String description,
  required final FutureOr<T> Function() operation,
  required final int Function() getTimerTick,
}) async {
  final startTick = getTimerTick();
  final value = await operation();
  final endTick = getTimerTick();
  return Operation<T>(
    description: description,
    frameInfo: FrameInfo(
      startTick: startTick,
      endTick: endTick,
    ),
    result: value,
  );
}

void output({
  required final List<Operation<Object?>> ops,
  required final List<int> ticks,
}) {
  print("== Operation Info ==");
  print(
    "Operation".padRight(30) + "Start Frame".padRight(30) + "End Frame".padRight(30),
  );
  for (final op in ops) {
    print(
      _operationToText(
        operation: op,
      ),
    );
  }
  print("== Frame Info ==");
  print("Skipped Frames".padRight(30) + "Frame Range".padRight(30));
  for (final skippedFrame in _findSkippedFrames(
    frames: ticks,
  )) {
    print(
      ((skippedFrame.endTick - skippedFrame.startTick).toString()).padRight(30) +
          (skippedFrame.startTick.toString() + " to " + skippedFrame.endTick.toString()).padRight(30),
    );
  }
}

String _operationToText<T>({
  required final Operation<T> operation,
}) =>
    operation.description.padRight(30) +
    (operation.frameInfo.startTick.toString()).padRight(30) +
    (operation.frameInfo.endTick.toString()).padRight(30);

class Operation<T> {
  final String description;
  final FrameInfo frameInfo;
  final T result;

  const Operation({
    required final this.description,
    required final this.frameInfo,
    required final this.result,
  });
}

List<FrameInfo> _findSkippedFrames({
  required final List<int> frames,
}) {
  final skippedFrames = <FrameInfo>[];
  frames.fold<int?>(
    null,
    (final previousValue, final element) {
      if (previousValue == null) {
        return element;
      } else if (previousValue + 1 != element) {
        skippedFrames.add(
          FrameInfo(
            startTick: previousValue,
            endTick: element,
          ),
        );
      } else {
        return element;
      }
    },
  );
  return skippedFrames;
}

class FrameInfo {
  final int startTick;
  final int endTick;

  const FrameInfo({
    required final this.startTick,
    required final this.endTick,
  });
}

Output
/usr/local/opt/dart-beta/libexec/bin/dart --version
Dart SDK version: 2.15.0-178.1.beta (beta) (Tue Oct 12 11:11:28 2021 +0200) on "macos_x64"

/usr/local/opt/dart-beta/libexec/bin/dart snippet.dart
snippet.dart: Warning: Interpreting this as package URI, (...)
== Operation Info ==
Operation                     Start Frame                   End Frame
json.decode                   0                             255
json.encode                   257                           347
== Frame Info ==
Skipped Frames                Frame Range
3                             132 to 135
4                             149 to 153
4                             159 to 163
2                             170 to 172
5                             175 to 180
4                             193 to 197
3                             202 to 205
2                             255 to 257
2                             337 to 339
== Operation Info ==
Operation                     Start Frame                   End Frame
json.decode                   0                             228
json.encode                   234                           337
== Frame Info ==
Skipped Frames                Frame Range
2                             128 to 130
4                             141 to 145
3                             151 to 154
3                             166 to 169
2                             180 to 182
6                             228 to 234
2                             325 to 327
== Operation Info ==
Operation                     Start Frame                   End Frame
json.decode                   0                             225
json.encode                   225                           311
== Frame Info ==
Skipped Frames                Frame Range
2                             122 to 124
2                             131 to 133
5                             142 to 147
4                             171 to 175
3                             180 to 183
44                            225 to 269
2                             300 to 302
== Operation Info ==
Operation                     Start Frame                   End Frame
json.decode                   0                             225
json.encode                   226                           325
== Frame Info ==
Skipped Frames                Frame Range
2                             126 to 128
4                             139 to 143
3                             149 to 152
3                             163 to 166
2                             177 to 179
58                            226 to 284
2                             312 to 314
== Operation Info ==
Operation                     Start Frame                   End Frame
json.decode                   0                             227
json.encode                   227                           315
== Frame Info ==
Skipped Frames                Frame Range
3                             122 to 125
2                             132 to 134
5                             143 to 148
4                             172 to 176
2                             181 to 183
46                            227 to 273
2                             304 to 306
== Operation Info ==
Operation                     Start Frame                   End Frame
json.decode                   0                             224
json.encode                   225                           324
== Frame Info ==
Skipped Frames                Frame Range
2                             125 to 127
3                             138 to 141
4                             147 to 151
3                             162 to 165
3                             176 to 179
59                            225 to 284
== Operation Info ==
Operation                     Start Frame                   End Frame
json.decode                   0                             225
json.encode                   226                           313
== Frame Info ==
Skipped Frames                Frame Range
2                             122 to 124
2                             131 to 133
4                             143 to 147
4                             172 to 176
2                             181 to 183
45                            226 to 271
2                             303 to 305
== Operation Info ==
Operation                     Start Frame                   End Frame
json.decode                   0                             226
json.encode                   231                           329
== Frame Info ==
Skipped Frames                Frame Range
2                             126 to 128
4                             139 to 143
4                             149 to 153
3                             164 to 167
2                             178 to 180
5                             226 to 231
== Operation Info ==
Operation                     Start Frame                   End Frame
json.decode                   0                             224
json.encode                   224                           311
== Frame Info ==
Skipped Frames                Frame Range
3                             121 to 124
2                             131 to 133
5                             142 to 147
4                             171 to 175
2                             180 to 182
45                            224 to 269
2                             301 to 303
== Operation Info ==
Operation                     Start Frame                   End Frame
json.decode                   0                             226
json.encode                   231                           330
== Frame Info ==
Skipped Frames                Frame Range
2                             127 to 129
5                             139 to 144
3                             150 to 153
3                             164 to 167
3                             178 to 181
5                             226 to 231

The Frame Info section shows that some runs caused ~40 dropped frames and some didn't. I expected to see no large ranges of dropped frames.

Not certain I should be adding to this issue as I don't think my issue is GC but does related to unexpected lag with a call to Isolate.exit.
From the documentation:

Further, the receiving isolate will in most cases be able to receive the message in constant time.

I read this that the time to exit and return data should be consistent regardless of the size of the data being returned.
I've written a test to explore this:

import 'dart:isolate';

void main() async {
  await test(size: 1);
  await test(size: 10);
  await test(size: 20);
  await test(size: 30);
  await test(size: 40);
  await test(size: 50);
  await test(size: 60);
  await test(size: 70);
  await test(size: 80);
  await test(size: 90);
  await test(size: 100);
}

Future<void> test({required int size}) async {
  final result = await Isolate.run(() {
    // allocate size * MB.
    final list = List.filled(1024 * 1024 * size, 0);
    return Tuple(DateTime.now(), list);
  });
  final completed = DateTime.now();
  final interval = completed.difference(result.t1);
  print('Time for Isolate.exit size: $size, ms:  ${interval.inMilliseconds} ');
  print('');
}

class Tuple<T1, T2> {
  Tuple(this.t1, this.t2);

  T1 t1;
  T2 t2;
}

This is a fairly typical run of the benchmark:
Time for Isolate.exit size: 1, ms:  2

Time for Isolate.exit size: 10, ms:  9

Time for Isolate.exit size: 20, ms:  17

Time for Isolate.exit size: 30, ms:  45

Time for Isolate.exit size: 40, ms:  33

Time for Isolate.exit size: 50, ms:  73

Time for Isolate.exit size: 60, ms:  49

Time for Isolate.exit size: 70, ms:  103

Time for Isolate.exit size: 80, ms:  66

Time for Isolate.exit size: 90, ms:  128

Time for Isolate.exit size: 100, ms:  82

Whilst we have some lumps, there is a fairly linear relationship between the size of the memory allocated and the time it
takes isolate.exit to return it.
I would expects some bumps (due to gc) but I wouldn't expect the linear relationship we are seeing.

Whilst we have some lumps, there is a fairly linear relationship between the size of the memory allocated and the time it takes isolate.exit to return it.

In general Isolate.exit() is just like SendPort.send() a O(n) operation, i.e. linear in message size. That's because it has to verify that the message being sent doesn't contain objects that are unsendable. Though compared to SendPort.send() it does avoid the copy.
This is vaguely mentioned in the API docs of Isolate.exit() as: The system may be able to send this final message more efficiently than normal port communication between live isolates. In these cases this final message object graph will be reassigned to the receiving isolate without copying.
As part of https://github.com/dart-lang/sdk/issues/46752 there's a bullet point:

Explore making send-and-exit O(1) - transferring receive port ownership as well

The issue right now is that certain native objects (e.g. sockets, timers, ...) cannot be sent or moved from isolate to isolate.

That is disappointing.
I would suggest the current documentation is misleading in that it
describes isolate.exit as operating in constant time, which clearly it
doesn't. Perhaps we could get this updated.
I was rather excited by isolate.exit but it appears we are still left with
isolates being  of very limited utility given any sizable response will
cause frame drops.
Oh but for a thread.
On Fri, 24 Mar 2023, 8:10 pm Martin Kustermann, @.***>
wrote:

Whilst we have some lumps, there is a fairly linear relationship between
the size of the memory allocated and the time it takes isolate.exit to
return it.
In general Isolate.exit() is just like SendPort.send() a O(n) operation,
i.e. linear in message size. That's because it has to verify that the
message being sent doesn't contain objects that are unsendable. Though
compared to SendPort.send() it does avoid the copy.
This is vaguely mentioned in the API docs of Isolate.exit() as: The
system may be able to send this final message more efficiently than normal
port communication between live isolates. In these cases this final message
object graph will be reassigned to the receiving isolate without copying.
As part of #46752 https://github.com/dart-lang/sdk/issues/46752 there's
a bullet point:

Explore making send-and-exit O(1) - transferring receive port
ownership as well

The issue right now is that certain native objects (e.g. sockets, timers,
...) cannot be sent or moved from isolate to isolate.
—
Reply to this email directly, view it on GitHub
https://github.com/dart-lang/sdk/issues/47508#issuecomment-1482477993,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAG32OB7CENRG55WC4BYSDTW5VQJVANCNFSM5GLEUNIA
.
You are receiving this because you commented.Message ID:
@.***>

@bsutton the O(n) overhead is entirely on the sending side, which does verification inside Isolate.exit call. Documentation does not mention that, but it correctly says that receiving side pays only O(1) per message. Your benchmark is measuring both the cost of Isolate.exit and the cost of receiving the message. So you see both sending and receiving costs which add up to O(n) + O(1) = O(n). If you want to measure just receiving costs you could set a timer which fires, say, every 10ms in the main isolate and then observe that there should be no significant drift between events even when you send large payloads.
That being said - building large payloads likely comes with some GC costs (e.g. due to the cost of evacuation of large number of survivors from the nursery or due to GC inability to make appropriate progress when allocation rate is too high) and GCs will pause all isolates in a group.

That is disappointing.

@bsutton an alternative solution to this issue is to allocate unmanaged memory via ffi and to pass around a pointer to that memory between isolates (see: #50457).

As part of https://github.com/dart-lang/sdk/issues/46752 there's a bullet point:

Explore making send-and-exit O(1) - transferring receive port ownership as well

This sounds very exciting to me. @mkustermann, thank you for the update (and your work on these issues!).
Is the work on making send-and-exit O(1) intended to help with the "skipped frames" issue here? (I've ran the code in the issue description and it continues to report some skipped frames.)

Is the work on making send-and-exit O(1) intended to help with the "skipped frames" issue here? (I've ran the code in the issue description and it continues to report some skipped frames.)

The pauses may still be GC related (see my older comment at https://github.com/dart-lang/sdk/issues/47508#issuecomment-947541183). Though investigate it again with newest VM - @aam could you have a look?

I looks like Michael has a more up-to-date benchmark related to this issue here.

@mraleph
So I rewrote my own benchmark.
It spawns the isolate and then starts a loop checking for any gaps in the performance of the loop:
import 'dart:async';
import 'dart:isolate';

void main() async {
  await test(size: 1);
  await test(size: 10);
  await test(size: 20);
  await test(size: 30);
  await test(size: 40);
  await test(size: 50);
  await test(size: 60);
  await test(size: 70);
  await test(size: 80);
  await test(size: 90);
  await test(size: 100);
}

Future<void> test({required int size}) async {
  final completer = Completer<bool>();

  final future = Isolate.run(() {
    // allocate size MB.
    final list = List.filled(1024 * 1024 * size, 0);
    return Tuple(DateTime.now(), list);
  });

  unawaited(future.whenComplete(() {
    completer.complete(true);
  }));

  final frameDrops = <Duration>[];
  final stopwatch = Stopwatch()..start();

  while (!completer.isCompleted) {
    final elapsed = stopwatch.elapsed;
    if (elapsed > const Duration(milliseconds: 10)) {
      frameDrops.add(elapsed);
    }
    stopwatch.reset();
    // give async tasks a chance to run.
    await Future.delayed(const Duration(milliseconds: 1), () => null);
  }

  final completed = DateTime.now();
  final interval = completed.difference((await future).t1);
  print(
      'Time for Isolate.exit size: $size MB, ms:  ${interval.inMilliseconds} ');

  final totalDuration = frameDrops.isNotEmpty
      ? frameDrops.reduce((a, b) => a + b)
      : Duration.zero;

  final averageDuration = frameDrops.isNotEmpty
      ? Duration(
          microseconds: totalDuration.inMicroseconds ~/ frameDrops.length)
      : Duration.zero;

  if (frameDrops.isEmpty) {
    print('All Good');
  } else {
    print(
        'ERROR: frame drops: ${frameDrops.length} average duration: ${averageDuration.inMilliseconds}');
  }
  print('');
}

class Tuple<T1, T2> {
  Tuple(this.t1, this.t2);

  T1 t1;
  T2 t2;
}

Here is a test run:
Time for Isolate.exit size: 1 MB, ms:  4
All Good

Time for Isolate.exit size: 10 MB, ms:  17
All Good

Time for Isolate.exit size: 20 MB, ms:  34
ERROR: frame drops: 1 average duration: 21

Time for Isolate.exit size: 30 MB, ms:  50
ERROR: frame drops: 1 average duration: 34

Time for Isolate.exit size: 40 MB, ms:  69
ERROR: frame drops: 1 average duration: 47

Time for Isolate.exit size: 50 MB, ms:  87
ERROR: frame drops: 1 average duration: 56

Time for Isolate.exit size: 60 MB, ms:  104
ERROR: frame drops: 1 average duration: 60

Time for Isolate.exit size: 70 MB, ms:  64
ERROR: frame drops: 1 average duration: 60

Time for Isolate.exit size: 80 MB, ms:  140
ERROR: frame drops: 1 average duration: 92

Time for Isolate.exit size: 90 MB, ms:  152
ERROR: frame drops: 1 average duration: 102

Time for Isolate.exit size: 100 MB, ms:  174
ERROR: frame drops: 2 average duration: 65

This test still shows a linear relationship.
I don't think this is a GC issue.
Each isolate does a single allocation. So the number of objects that need to be GC'd are constant for each test.
My assumption from this is that GC overhead should be constant.
Of course, the timing of when GC kicks in will be somewhat variable and the larger block allocations will be more likely to cause a GC.
So I did a run reversing the order of the sizes.  The idea is that if we do the largest allocation first, we should be less likely to see a GC whilst it runs. My assumption (?) is that  a single 100MB allocation shouldn't cause a GC to run.
Here are the results of the reversed run:
Time for Isolate.exit size: 101 MB, ms:  173
ERROR: frame drops: 1 average duration: 111

Time for Isolate.exit size: 91 MB, ms:  153
ERROR: frame drops: 1 average duration: 91

Time for Isolate.exit size: 81 MB, ms:  139
ERROR: frame drops: 1 average duration: 90

Time for Isolate.exit size: 71 MB, ms:  125
ERROR: frame drops: 2 average duration: 46

Time for Isolate.exit size: 61 MB, ms:  106
ERROR: frame drops: 1 average duration: 69

Time for Isolate.exit size: 51 MB, ms:  84
ERROR: frame drops: 1 average duration: 57

Time for Isolate.exit size: 41 MB, ms:  69
ERROR: frame drops: 1 average duration: 46

Time for Isolate.exit size: 31 MB, ms:  53
ERROR: frame drops: 1 average duration: 36

Time for Isolate.exit size: 21 MB, ms:  37
ERROR: frame drops: 1 average duration: 24

Time for Isolate.exit size: 11 MB, ms:  16
ERROR: frame drops: 1 average duration: 14

Time for Isolate.exit size: 1 MB, ms:  1
All Good

As you can see we still have a linear relationship, which suggests this isn't a GC issue and that (at least some) of the performance cost is worn by the calling isolate.

@bsutton What Dart version are you running? I get no frame drops neither on 2.19.5 nor 3.0.0-336.0.dev even if I go as high as 300 Mb.

@mraleph
Dart SDK version: 2.19.5 (stable) (Mon Mar 20 17:09:37 2023 +0000) on "linux_x64"

@mraleph
I am running it from the vscode debugger but I've just doen a second run from the cli and get the same results:
dart bin/simple.dart

@mraleph
AMD® Ryzen 7 3700x 8-core processor × 16
AMD® Radeon rx 5500 xt
Memory: 32.0 GiB
Ubuntu 22.04.2 LTS (64 bits)

Interesting, I can indeed reproduce this on Linux and it is GC related:
On Mac I get:
[ main         ,  StartCMark(   new space),   28,   2.18,   0.0,   0.0,   0.0,   2.0,   2.0, 0.0, 0.0, 1524.6, 1524.6, 1525.9, 1525.9,   0.0,   0.0,   0,   0,   0.0,    0.0, ]
main: hard_threshold=8796093022208MB, soft_threshold=1005MB, idle_threshold=806MB, reason=gc
[ main         ,   MarkSweep(    finalize),   29,   2.29,   0.2,   0.1,   0.1,   2.0,   2.0, 0.0, 0.0, 1524.6,  804.6, 1525.9, 1525.9,   0.0,   0.0,   0,   1,   0.0, -720.0, ]
Time for Isolate.exit size: 100 MB, ms:  77

on Linux I get:
 main         ,  StartCMark(   new space),   28,   7.44,   0.1,   0.0,   0.0,   2.0,   2.0, 0.0, 0.0, 1524.6, 1524.6, 1526.8, 1526.8,   0.0,   0.0,   1,   1,   0.0,    0.0, ]
main: hard_threshold=8796093022208MB, soft_threshold=1005MB, idle_threshold=806MB, reason=gc
[ main         ,   MarkSweep(    finalize),   29,   8.37, 107.1,   0.4,   0.4,   2.0,   2.0, 0.0, 0.0, 1524.6,  804.6, 1526.8, 1526.8,   0.0,   0.0,   0,   1,   0.0, -720.0, ]
Time for Isolate.exit size: 100 MB, ms:  140

@rmacnak-google any ideas? seems like we are hitting MS finalize in some very unfortunate moment which causes large synchronous pause.

Here is a 3.x run:
bsutton@slayer4:~/git/onepub_isolates$ dswitch use dev
Switched to dev (3.0.0-263.0.dev)

You may need to restart your terminal session.
bsutton@slayer4:~/git/onepub_isolates$ dart bin/simple.dart
Time for Isolate.exit size: 101 MB, ms:  326
ERROR: frame drops: 1 average duration: 117

Time for Isolate.exit size: 91 MB, ms:  295
ERROR: frame drops: 2 average duration: 65

Time for Isolate.exit size: 81 MB, ms:  208
ERROR: frame drops: 1 average duration: 57

Time for Isolate.exit size: 71 MB, ms:  230
ERROR: frame drops: 2 average duration: 49

Time for Isolate.exit size: 61 MB, ms:  131
ERROR: frame drops: 2 average duration: 47

Time for Isolate.exit size: 51 MB, ms:  163
ERROR: frame drops: 1 average duration: 61

Time for Isolate.exit size: 41 MB, ms:  131
ERROR: frame drops: 1 average duration: 46

Time for Isolate.exit size: 31 MB, ms:  98
ERROR: frame drops: 1 average duration: 39

Time for Isolate.exit size: 21 MB, ms:  47
ERROR: frame drops: 1 average duration: 48

Time for Isolate.exit size: 11 MB, ms:  41
ERROR: frame drops: 1 average duration: 15

Time for Isolate.exit size: 1 MB, ms:  3
All Good

@mraleph how do you dump out the gc stats?

how do you dump out the gc stats?

You provide --verbose-gc command line option to dart.

FYI: if I aot compile the benchmark, then no more dropped frames and the raw exit time (time from the last line of the lambda to primary isolate resuming) drops to a consistent 1-2 ms.
I've added warmup logic for calls to isolate.run and the GC to get the jit compiler primed.
Of course, I would assume that the GC is written in C so the concept of warming up the GC is probably moot.
With the jit run, we do see an improvement to the first 100MB call (110ms down to 70ms).
But essentially the rest of the remaining stats are the same.
I can see from the --verbose-gc output that the warm-up causes the gc to run hard.
Jit test:
warming: 0
warming: 100
warming: 200
warmup complete
Time for Isolate.exit size: 101 MB, raw Exit time: 99ms
ERROR: frame drops: 1 average duration: 73

Time for Isolate.exit size: 91 MB, raw Exit time: 151ms
ERROR: frame drops: 1 average duration: 64

Time for Isolate.exit size: 81 MB, raw Exit time: 88ms
ERROR: frame drops: 1 average duration: 67

Time for Isolate.exit size: 71 MB, raw Exit time: 122ms
ERROR: frame drops: 1 average duration: 82

Time for Isolate.exit size: 61 MB, raw Exit time: 106ms
ERROR: frame drops: 1 average duration: 64

Time for Isolate.exit size: 51 MB, raw Exit time: 89ms
ERROR: frame drops: 1 average duration: 59

Time for Isolate.exit size: 41 MB, raw Exit time: 72ms
ERROR: frame drops: 1 average duration: 37

Time for Isolate.exit size: 31 MB, raw Exit time: 55ms
All Good

Time for Isolate.exit size: 21 MB, raw Exit time: 36ms
ERROR: frame drops: 1 average duration: 36

Time for Isolate.exit size: 11 MB, raw Exit time: 16ms
ERROR: frame drops: 1 average duration: 16

Time for Isolate.exit size: 1 MB, raw Exit time: 2ms
All Good

AOT run
Note: for a direct comparison, the aot run included the warm-up logic, however, it didn't need to as it delivered the same results without the warmup (as expected).
warming: 0
warming: 100
warming: 200
warmup complete
Time for Isolate.exit size: 101 MB, raw Exit time: 0ms
All Good

Time for Isolate.exit size: 91 MB, raw Exit time: 1ms
All Good

Time for Isolate.exit size: 81 MB, raw Exit time: 1ms
All Good

Time for Isolate.exit size: 71 MB, raw Exit time: 0ms
All Good

Time for Isolate.exit size: 61 MB, raw Exit time: 2ms
All Good

Time for Isolate.exit size: 51 MB, raw Exit time: 0ms
All Good

Time for Isolate.exit size: 41 MB, raw Exit time: 0ms
All Good

Time for Isolate.exit size: 31 MB, raw Exit time: 0ms
All Good

Time for Isolate.exit size: 21 MB, raw Exit time: 0ms
All Good

Time for Isolate.exit size: 11 MB, raw Exit time: 0ms
ERROR: frame drops: 1 average duration: 14

Time for Isolate.exit size: 1 MB, raw Exit time: 1ms
All Good

Updated benchmark with warmup logic.
import 'dart:async';
import 'dart:isolate';

void main() async {
  await warmup();
  print('warmup complete');

  for (var size = 101; size > 0; size -= 10) {
    await test(size: size);
  }
}

Future<void> test({required int size}) async {
  final completer = Completer<bool>();

  final future = Isolate.run(() {
    // allocate size MB.
    final list = List.filled(1024 * 1024 * size, 0);
    return Tuple(DateTime.now(), list);
  });

  unawaited(future.whenComplete(() {
    completer.complete(true);
  }));

  final frameDrops = <Duration>[];
  final stopwatch = Stopwatch()..start();

  while (!completer.isCompleted) {
    final elapsed = stopwatch.elapsed;
    if (elapsed > const Duration(milliseconds: 10)) {
      frameDrops.add(elapsed);
    }
    stopwatch.reset();
    // give async tasks a chance to run.
    await Future.delayed(const Duration(milliseconds: 1), () => null);
  }

  final completed = DateTime.now();
  final interval = completed.difference((await future).t1);
  print('Time for Isolate.exit size: $size MB, raw Exit time: '
      '${interval.inMilliseconds}ms');

  final totalDuration = frameDrops.isNotEmpty
      ? frameDrops.reduce((a, b) => a + b)
      : Duration.zero;

  final averageDuration = frameDrops.isNotEmpty
      ? Duration(
          microseconds: totalDuration.inMicroseconds ~/ frameDrops.length)
      : Duration.zero;

  if (frameDrops.isEmpty) {
    print('All Good');
  } else {
    print(
        'ERROR: frame drops: ${frameDrops.length} average duration: ${averageDuration.inMilliseconds}');
  }
  print('');
}

Future<void> warmup() async {
  for (var i = 0; i < 300; i++) {
    // 10 MB
    List.filled(1024 * 1024 * 10, 0);
    if (i % 100 == 0) {
      print('warming: $i');
    }
    // hopefully the gc will run
    await Future.delayed(const Duration(milliseconds: 1), () => null);
  }
}

class Tuple<T1, T2> {
  Tuple(this.t1, this.t2);

  T1 t1;
  T2 t2;
}

I'm wondering, if the observations here are related to GC, then it might make sense to first establish what's expected behavior?
@mraleph wrote:

That being said - building large payloads likely comes with some GC costs (e.g. due to the cost of evacuation of large number of survivors from the nursery or due to GC inability to make appropriate progress when allocation rate is too high) and GCs will pause all isolates in a group.

This sounds to me like it's currently technically impossible (ignoring ffi-related techniques) to e.g. parse arbitrarily large json in a separate isolate B and to pass that back to the main isolate without having the main isolate freeze (because of operations that happened on isolate B)?
So the skipped frames that are being reported by bsuttons benchmark and my benchmark are expected behavior?
If that's true:

are there's plans to improve GC-related performance for inter-isolate communication? (does it even make sense to optimize for this?)
are there plans to work on removing that fundamental limitation?

Note: This issue was motivated by the following use-case:

I needed to parse a large chunk of json
doing that on the main isolate caused my Flutter UI to freeze and therefore drop frames.
I decided to offload parsing to a separate isolate
The main isolates was still dropping frames despite the work happening on a separate isolate.
I was hoping to be able to parse arbitrary large strings to json on a separate isolate and have the main isolate never drop any frames. (The skipped frames here were meant to be a proxy metric for the dropped frames that I would observe in Flutter apps)

if one isolate is not doing anything and it's timer "jitters" then this jitter is most likely introduced by GC in another isolate.

@mraleph For me the light weight Isolates feature has been a huge benefit for Dart server side applications! but what I'm also concerned about is that GC affects all the Isolates in a group. I (vaguely) understand that all the Isolates in a group share a heap, but they are still all isolated afaik, so I'm not clear as to why a long running GC needs to stop all Isolates in a group? ie. an Isolate in group creating alot of garbage can still cause performance issues for other Isolates in the group, so there seems to be no way at the moment to offload memory intensive work from Isolate(s) that are latency sensitive?

@maks isolates in a group only look isolated to the Dart developer, which means that at the Dart language level there is no way to observe that two isolates work on the same heap. They do exist in the same heap (which is what enables things like Isolate.exit and sharing various internal data structures and immutable objects like String, int, double instances, deeply immutable Lists and compile time constants), which means that you can't reliably partition the graph of objects into isolate specific parts (i.e. given an arbitrary object in the heap you can't say which isolate owns it). GC is a heap level operation - which means all isolates that share the heap need to cooperate. We did consider if we want to experiment with implementing isolate local young generation collection, but so far this never bubbled to the top of the list.
We do want our GC pauses to be as short as possible (and if you see large pauses you should report this), but today we don't have any plans for providing any sort of hard or soft real-time time guarantees.

Thanks @mraleph for the detailed explanation! That makes things a lot clearer for me, as I hadn't considered that there is no book keeping for mem allocations per Isolate so then the GC has to run over the entire heap and hence applies to all Isolates.
I definitely agree that GC pauses be as short as possible. Infact for my usecase (a backeend system using large numbers of Isolates) its really only the stop-the-world GC that is of concern for the young generation GC and not any slow down from the old gen concurrent one as that would still allow all my Isolates to make some progress, which what I'm most interested in.
I'll try to get some benchmarks done with my current WIP application and report back.
Btw I found this quite detailed doc about GC, would that be the best reference at the moment on how Dart GC currently works?

Btw I found this quite detailed doc about GC, would that be the best reference at the moment on how Dart GC currently works?

Yes.


Add support for raiplay.it

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.10.07. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2016.10.07

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[ ] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[x] Feature request (request for a new functionality)
[ ] Question
[ ] Other

Description of your issue, suggested solution and other information
Is it possible to use youtube-dl to download videos from http://www.raiplay.it ? This is an official website by the Italian national television (there are probably other videos at http://rai.it). If it's not possible, would you please consider adding support for it?
Thanks
PS: considering how many websites are now supported by youtube-dl, have you ever considered renaming the project?

Could you paste a concrete video URL?

have you ever considered renaming the project

If you have an idea, open a new issue for discussion!


Fix warnings and errors

Overview

This suppresses several warnings and fixes two errors in the existing suite.
Testing recommendations

mix test --trace
Related Issues or Pull Requests

Checklist

[x] Code is written and works correctly;
[x] Changes are covered by tests;
[ ] Documentation reflects the changes;

@iilyak can you merge this for me please?


Is the tc_mysql library thread-safe?

E：Thread is not safe.
C：线程不安全。

If using the tc_mysql library, does each servant need to instantiate an object?

如果使用tc_mysql的库，每个servant都需要实例化一个对象吗？

yes

是的

Generally mysql has limited links, multiple server deployment, each server has N servants, there will be many links, is it easy to exceed the number of links?

一般mysql有限定链接，多server部署，每个server有N个servant，便会有很多链接，链接数容易超限吗？

Not all services need to connect to the datebase, or you can have one mysql per partition. If it is a global service, you can have a proxy service to specifically communicate with mysql, so that the number of links will not exceed the limit

不是所有的服务都要连接datebase，或者可以每个分区一个mysql。如果是全局服务，可以做个代理服务专门去和mysql通讯，这样链接数就不会超限了