Envoy cluster creation issue with conflicting service names

Describe the bug
When emissary is creating envoy clusters, emissary is evidently constrained by a character limit for the length of the cluster and must handle potential naming conflicts. If a mapping and service are named identically in different namespaces, and the name of the service is over a certain number of characters, there seems to be a condition that causes envoy cluster creation for one of those mappings to fail.
To Reproduce

Create a service that maps to a pod with a name that is over 30 characters or so, but in different namespaces. I've reproduced this issue in minikube with the following manifest.

apiVersion: v1
kind: Service
metadata:
  name: hello-this-name-is-too-long-for-emissary
  namespace: alpha
spec:
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
  selector:
    app: hello-alpha
---
apiVersion: v1
kind: Service
metadata:
  name: hello-this-name-is-too-long-for-emissary
  namespace: beta
spec:
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
  selector:
    app: hello-beta

I have a simple nginx pod in each namespace with hello-alpha and hello-beta as the matching label selector.

Create a mapping in the beta namespace:

apiVersion: getambassador.io/v3alpha1
kind: Mapping
metadata:
  name: hello
  namespace: beta
spec:
  host: "*"
  prefix: /hello-beta
  service: http://hello-this-name-is-too-long-for-emissary

The following route will be created in envoy according to the debug GUI
{
  "match": {
    "case_sensitive": true,
    "prefix": "/hello-beta",
    "runtime_fraction": {
      "default_value": {
        "denominator": "HUNDRED",
        "numerator": 100
      },
      "runtime_key": "routing.traffic_shift.cluster_http___hello_this_name_is_too_lo-0"
    }
  },
  "route": {
    "cluster": "cluster_http___hello_this_name_is_too_lo-0",
    "prefix_rewrite": "/",
    "timeout": "3.000s"
  }
}

Create a similar mapping but in the alpha namespace.

apiVersion: getambassador.io/v3alpha1
kind: Mapping
metadata:
  name: hello
  namespace: alpha
spec:
  host: "*"
  prefix: /hello-alpha
  service: http://hello-this-name-is-too-long-for-emissary

Once the mapping is created, it will now "take over" the envoy cluster with the -0 suffix, and the endpoints for this cluster will be updated to point to the alpha pod.
{
  "match": {
    "case_sensitive": true,
    "prefix": "/hello-alpha",
    "runtime_fraction": {
      "default_value": {
        "denominator": "HUNDRED",
        "numerator": 100
      },
      "runtime_key": "routing.traffic_shift.cluster_http___hello_this_name_is_too_lo-0"
    }
  },
  "route": {
    "cluster": "cluster_http___hello_this_name_is_too_lo-0",
    "prefix_rewrite": "/",
    "priority": null,
    "timeout": "3.000s"
  }
}

However, the mapping for beta will still be pointing to the envoy cluster named cluster_http___hello_this_name_is_too_lo-0. This causes traffic destined for the beta instance to be routed to the alpha instance instead.
Before creating the alpha mapping, if I hit the ClusterIP of the emissary-ingress service, requests for /hello-beta are routed to the beta pod correctly.
 testpod ~ $ curl -s emissary-ingress.emissary.svc.cluster.local/hello-alpha | grep name

Server name: hello-alpha

 testpod ~ $ curl -s emissary-ingress.emissary.svc.cluster.local/hello-beta | grep name

Server name: hello-beta

After creating the alpha mapping, requests for /hello-beta are send to the hello-alpha pod
 testpod ~ $ curl -s emissary-ingress.emissary.svc.cluster.local/hello-alpha | grep name

Server name: hello-alpha

 testpod ~ $ curl -s emissary-ingress.emissary.svc.cluster.local/hello-beta | grep name

Server name: hello-alpha

Additionally, the emissary pod logs the following error:
2022-07-26 16:05:18 diagd 3.0.0 [P17TAEW] ERROR: {
...
<trim long JSON dump of envoy config>
...
could not validate the envoy configuration above after 5 retries, failed with error
[2022-07-26 16:05:18.276][140][info][main] [source/server/server.cc:786] runtime: layers:
  - name: static_layer
    static_layer:
      re2.max_program_size.error_level: 200
      envoy.reloadable_features.no_extension_lookup_by_name: false

[2022-07-26 16:05:18.280][140][critical][main] [source/server/config_validation/server.cc:63] error initializing configuration '/ambassador/snapshots/econf-tmp.json': route: unknown cluster 'cluster_http___hello_this_name_is_too_lo-1'

(exit code 1)
Aborting update...

This would suggest to me that the beta mapping should get a route that points to a new envoy cluster named cluster_http___hello_this_name_is_too_lo-1. However, this cluster is not being created correctly. I haven't been able to spot anything in the logs that would suggest why this cluster is not being created.
Expected behavior
The new envoy cluster should be created and the "beta" route should be correctly updated to point to the new cluster.
Versions:

emissary-ingress 3.0.0 (can reproduce on 2.3.1 as well)
Kubernetes 1.21, minikube 1.26.0

@kflynn has requested debug logs via Slack, so I'm attaching them here. I began capturing these logs from just prior to applying the mapping in the alpha namespace.
alphamapping.txt

Here's the smoking gun from the logs:
2022-07-26 21:13:04 diagd 3.0.0 [P17TAEW] DEBUG: COLLISION: compress cluster_http___hello_this_name_is_too_long_for_emissary_alpha_alpha to cluster_http___hello_this_name_is_too_lo
2022-07-26 21:13:04 diagd 3.0.0 [P17TAEW] DEBUG: COLLISION: compress cluster_http___hello_this_name_is_too_long_for_emissary_beta_beta to cluster_http___hello_this_name_is_too_lo
2022-07-26 21:13:04 diagd 3.0.0 [P17TAEW] DEBUG: COLLISION: mangle cluster_http___hello_this_name_is_too_long_for_emissary_alpha_alpha => cluster_http___hello_this_name_is_too_lo-0

The service name hello-this-name-is-too-long-for-emissary is long enough all by itself that the namespace name is not being taken into account when name mangling happens. 🤦♂
There are a few possible fixes, but I think I would start by trying this patch:
diff --git a/python/ambassador/ir/ir.py b/python/ambassador/ir/ir.py
index abd6f4bee..7993a52d7 100644
--- a/python/ambassador/ir/ir.py
+++ b/python/ambassador/ir/ir.py
@@ -11,7 +11,7 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License
-import json
+import hashlib
 import logging
 import os
 from ipaddress import ip_address
@@ -480,7 +480,11 @@ class IR:
             if len(name) > 60:
                 # Too long. Gather this cluster by name prefix and normalize
                 # its name below.
-                short_name = name[0:40]
+                h = hashlib.new("sha1")
+                h.update(name.encode("utf-8"))
+                hd = h.hexdigest()[0:16].upper()
+
+                short_name = name[0:40] + "-" + hd

                 cluster = self.clusters[name]
                 self.logger.debug(f"COLLISION: compress {name} to {short_name}")

Unfortunately I won't be able to test that until at least tomorrow, but I think it should work – it makes sure that the short name both has some human-readable aspects for matching clusters to Mappings, and takes the whole name into account.

I can confirm that the above patch has resolved our issues in our dev cluster. We'll attempt to deploy a custom docker image with changes to production in the coming days.
Thanks a ton for the quick turnaround!

We deployed a locally built patched version of emissary to our production cluster 2 weeks ago and are happy to report that the patch resolved the issue for us. Thanks again for the quick turnaround on this.


[Event Request] codeunit 80 "Sales-Post" PostItemJnlLineBeforePost

Please make function PostItemJnlLineBeforePost externally available. Also please add the parameter TrackingSpecification as the parameter. Parameter neede for following Event:
OnPostItemJnlLineOnBeforeTransferReservToItemJnlLine
https://github.com/microsoft/ALAppExtensions/issues/17937
current:
local procedure PostItemJnlLineBeforePost(var ItemJnlLine: Record "Item Journal Line"; SalesLine: Record "Sales Line"; var TempWhseJnlLine: Record "Warehouse Journal Line" temporary; var PostWhseJnlLine: Boolean; QtyToBeShippedBase: Decimal)
desired:
procedure PostItemJnlLineBeforePost(var ItemJnlLine: Record "Item Journal Line"; SalesLine: Record "Sales Line"; var TempWhseJnlLine: Record "Warehouse Journal Line" temporary; var PostWhseJnlLine: Boolean; QtyToBeShippedBase: Decimal; TrackingSpecification: Record "Tracking Specification")

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 40900.


V2 doesn't support bound enum generics

Initial Checks

[X] I confirm that I'm using Pydantic V2 installed directly from the main branch, or equivalent

Description
Provided code example raises the following error:
pydantic_core._pydantic_core.SchemaError: Error building "model" validator:
  SchemaError: Error building "typed-dict" validator:
  SchemaError: Field "bar":
  SchemaError: Error building "lax-or-strict" validator:
  SchemaError: Error building "function-after" validator:
  SchemaError: Error building "literal" validator:
  SchemaError: "expected" should have length > 0

Example Code
from enum import Enum
from typing import Generic, TypeVar
from pydantic import BaseModel

T = TypeVar('T', bound=Enum)

class Foo(BaseModel, Generic[T]):
    bar: T

Python, Pydantic & OS Version
pydantic version: 2.0a2
        pydantic-core version: 0.24.0 release build profile
                 install path: C:\Program Files\Python\Lib\site-packages\pydantic
               python version: 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)]
                     platform: Windows-10-10.0.22621-SP0
     optional deps. installed: ['typing-extensions']

Update: the fix for this should now be available in the latest v2 alpha release v2.0a3


Points or Lines are shifted when save 'pdf'

I used the export_fig to save a picture generated by 'imagesc'  together with 'plot'.  However, I find the points (generated by 'plot') has a little position error. Hope this can be corrected in the near future.
Here is the pdf by export_fig.m
ByExport_fig.pdf
But it should be like this:
BySave2pdf.pdf
Thank you!

This appears to me to be an internal bug in Matlab's print command (which is used by export_fig to save the figure). I suggest that you send this example to MathWorks for review and possible fix. It would help if you could replicate the issue with a simple Matlab command for example, comparing the outputs of the following commands:
print(gcf, '-loose', '-painters', '-r864', '-depsc2', 'output1.pdf');  % used by export_fig
saveas(gcf, 'output2.pdf');  % builtin save-to-PDF

Thanks for reply.
I draw a image by 'imagesc' command, and then add points and a line by 'plot' command, and then save the image by three command as following:
export_fig(gcf, 'ExportFig.pdf','-painters');
print(gcf, '-loose', '-painters', '-r864', '-dpdf', 'Print.pdf');
saveas(gcf, 'Saveas.pdf');
where Print.pdf and Saveas.pdf have the same results as show in matlab figure window (ScreenCapture.png). But in ExportFig.pdf , the white line and points have a little shift (down direction).
However,  if the image drawn by imagesc is very sparse, the three command almost have the same results (ExportFigSparse.pdf, PrintSparse.pdf, SaveasSparse.pdf, ScreenCaptureSparse.png).
So, I think there is something wrong with 'export_fig.m'.
ExportFig.pdf
Print.pdf
Saveas.pdf

ExportFigSparse.pdf
PrintSparse.pdf
SaveasSparse.pdf

reopened based on the @MrGang-UESTC 's detailed comment above

To give notice to anyone interested but altmany, who received my answer by eMail on 29.1., this is my code. I know it's not short, by I couldn't make the bug appear reliably and with good visibility otherwise. Might still be possible, of course...
%% parameter
n = 81;
% if displacement doesn't show, go up by 10s (91, 101, ...)
%% create symmetric bit data, ignore
n2 = ceil(n/2);
nr = n2^2;
data = round(rand(nr,1));
c = 1;
data_symm = zeros(n);
for i = 1:n2
    data_symm(1:i,i) = data(c:c+i-1);
    data_symm(i,1:i) = data(c:c+i-1);
    data_symm(n:-1:n-i+1,n-i+1) = data(c:c+i-1);
    data_symm(n-i+1,n:-1:n-i+1) = data(c:c+i-1);
    c = c+i;
end
for i = n2-1:-1:1
    data_symm(1:i,n-i+1) = data(c:c+i-1);
    data_symm(n-i+1,1:i) = data(c:c+i-1);
    data_symm(n:-1:n-i+1,i) = data(c:c+i-1);
    data_symm(i,n:-1:n-i+1) = data(c:c+i-1);
    c = c+i;
end
%% visualize
figure(1);clf;
set(gcf,'Units','centimeters');
set(gcf,'Position',[0 0 21 29.7]);
subplot('Position',[0.1 0.7 0.2 0.2]); %1st one is enough to show bug
% data
imagesc((1:n)+0.5, (1:n)+0.5, 1-data_symm);
colormap(gray);
axis equal;
set(gca, 'XTick', [], 'YTick', [], 'XLim', [1 n+1], 'YLim', [1 n+1], 'XGrid', 'off', 'YGrid', 'off');
hold on;
% symmetry axis
red = [0.8 0.4 0.4];
plot3([1 n+1],[1 n+1],[2 2],'Color',red,'LineWidth',0.6);
plot3([1 n+1],[n+1 1],[2 2],'Color',red,'LineWidth',0.6);
%% export and open
export_fig 'Test_Export_Fig_Bug.pdf' -nocrop;
winopen('Test_Export_Fig_Bug.pdf');

@Spiegel-Leser  - you mentioned an email to me on Jan 31, but I never received your email. Perhaps you sent it to the wrong email address - please email it to altmany at gmail

Using the code supplied above by @Spiegel-Leser I was able to confirm that indeed, as per my original suspicion, this is due to an internal bug in Matlab's print() function, which duplicates the bottom row and rightmost column of pixels in its output when the image resolution is low enough (the exact threshold depends on the image physical display size vs. its rows/cols size).
I filed a bug report with MathWorks (Case #02961062). I will follow-up here if and when I learn anything useful.

I would like to present similar problem with shifted x-axis. Attached files generated by:
export_fig -jpeg -pdf plot_spectrogram
saveas(gcf, 'plot_spectrogram2.pdf');

While export_fig JPEG and saveas PDF are the same, export_fig PDF has x-axis ticks and tick labels shifted.
Note, XTick and XTickLabel were change by code from default values.
plot_spectrogram.jpg
plot_spectrogram2.pdf
plot_spectrogram.pdf

Any updates? I tried to find the indicated case number (#02961062), but could not find anything on Mathworks?

02961062 is the internal MathWorks case number. In my experience, MathWorks often does not publish bug reports until they are actually fixed. Sad but true...

That's unfortunate, indeed. Thanks though.
In the meanwhile, just FYI for anyone stumbling on this later on, this issue (naturally, I suppose) is absent in versions below (and excluding) 2014b.
... it's a pity that 2014b introduced so many degradations to the vectorized outputting capabilities! From my perspective, their "improvements" to the plotting capabilities were anything but that.

I also have the same issue (in my case, errorbar and plot function drawed shifted marker) on R2017a.
But it is solved on R2018a. I think that bug was solved in R2018a

I can confirm that the problem seems to be solved in Matlab release R2018b (perhaps also in R2018a, I'm not sure). I'm closing this issue - if anyone sees that the problem is still not solved, then let me know and I will reopen it.


:gift: Configurability of Unicode Control Characters check

Changes

:gift: Configurability of Unicode Control Characters check

/kind enhancement
Follows up on https://github.com/knative-sandbox/.github/pull/166

/cc @pierDipi


remove getset and url

getset got removed, because it's a very immature library
the url library doesn't support uris and it'd be too complicated for this library to parse uris.

closes #19
closes #21

Codecov Report

Merging #30 into master will decrease coverage by 0.52%.
The diff coverage is 53.77%.

@@            Coverage Diff            @@
##           master     #30      +/-   ##
=========================================
- Coverage   65.12%   64.6%   -0.53%
=========================================
  Files          43      43
  Lines        2747    2794      +47
=========================================
+ Hits         1789    1805      +16
- Misses        958     989      +31

Impacted Files
Coverage Δ

src/error.rs
17% <ø> (+1.25%)
:arrow_up:

src/tags/media_segment/date_range.rs
1.01% <0%> (ø)
:arrow_up:

src/media_segment.rs
45.07% <100%> (ø)
:arrow_up:

src/tags/media_segment/byte_range.rs
87.03% <100%> (ø)
:arrow_up:

src/tags/master_playlist/session_key.rs
77.55% <100%> (+2.55%)
:arrow_up:

src/tags/media_segment/key.rs
85.71% <100%> (+1.4%)
:arrow_up:

src/tags/master_playlist/i_frame_stream_inf.rs
65.57% <28.57%> (-11.51%)
:arrow_down:

src/tags/master_playlist/session_data.rs
79.81% <45.16%> (-16.66%)
:arrow_down:

src/types/byte_range.rs
84.74% <50%> (-13.04%)
:arrow_down:

src/line.rs
48.25% <50%> (-0.36%)
:arrow_down:

... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fb44c88...71361ff. Read the comment docs.


Video and audio tracks does not start

Describe the bug
When using the Livekit demo (on iPhone 13), sometimes video does not start for local participant (and of course is not streamed to the remote participant).

Reproduction

Macbook: Join the room with cam & mic: https://meet.livekit.io/rooms/mob5-6jrt

iPhone: Open link https://meet.livekit.io/rooms/mob5-6jrt

iPhone: Accept permissions

iPhone: Write the name

iPhone: Click "Join room"

iPhone: Check if camera feed is shown - here's the bug. If not, refresh the page.

iPhone: Accept the permissions.

iPhone: go back to step 5.

I have uploaded two videos, which shows the whole flow (including logs in video).
Logs
Logs are in the [videos](https://drive.google.com/drive/folders/1-icuHdRj9SKC2yHS_HeVa-cV95mQPzEn?usp=share_link).

System Info
iPhone 13,
  iOS 17.4
    Safari
MacBook Pro M1, 14.3.
  Safari 17.3 (19617.2.3).

Link: https://meet.livekit.io/rooms/mob5-6jrt

Severity
blocking all usage of LiveKit
Additional Information
We have the same issue on our platform, which uses Livekit SDK.

Thanks for the report.
It looks like the second time the camera permissions re-acquiring the camera stream is blocked for some reason. I just tried with iOS 17.2 and cannot reproduce this behaviour.
Will try also with 17.4 and see if there's any difference.

It's hard to follow the logs in the video, could you post them in their entirety as text?

[Log] rawFacingMode – {rawFacingMode: "user"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] awaiting getUserMedia promise (dba1bf7f-511427f33cbdd780.js, line 1, x2)
[Log] reacquiring camera track – {trackSid: undefined, trackSource: "camera", trackMuted: false, …} (dba1bf7f-511427f33cbdd780.js, line 1)
{trackSid: undefined, trackSource: "camera", trackMuted: false, trackEnabled: true, trackKind: "video"}Object
[Log] restarting track with constraints – {trackSid: undefined, trackSource: "camera", trackMuted: false, …} (dba1bf7f-511427f33cbdd780.js, line 1)
{trackSid: undefined, trackSource: "camera", trackMuted: false, trackEnabled: true, trackKind: "video", …}Object
[Log] video playback aborted, likely due to new play request (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] re-acquired MediaStreamTrack – {trackSid: undefined, trackSource: "camera", trackMuted: false, …} (dba1bf7f-511427f33cbdd780.js, line 1)
{trackSid: undefined, trackSource: "camera", trackMuted: false, trackEnabled: true, trackKind: "video"}Object
[Log] track mute bounce got cancelled by an unmute event – {trackSid: undefined, trackSource: "camera", trackMuted: false, …}
{trackSid: undefined, trackSource: "camera", trackMuted: false, trackEnabled: true, trackKind: "video"}Object
[Log] setting video track unmuted – {trackSid: undefined, trackSource: "camera", trackMuted: false, …} (dba1bf7f-511427f33cbdd780.js, line 1)
{trackSid: undefined, trackSource: "camera", trackMuted: false, trackEnabled: true, trackKind: "video"}Object
[Log] track mute bounce got cancelled by an unmute event – {trackSid: undefined, trackSource: "camera", trackMuted: false, …}
{trackSid: undefined, trackSource: "camera", trackMuted: false, trackEnabled: true, trackKind: "video"}Object
[Log] {url: "wss://meet-uylq8icb.livekit.cloud"} ([name]-405583ddfedead02.js, line 1)
[Log] room event connectionStateChanged – {room: "", roomSid: "", identity: "", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "", roomSid: "", identity: "", event: "connectionStateChanged", args: ["connecting"]}Object
[Log] connecting to wss://meet-uylq8icb.livekit.cloud/rtc?access_token=eyJhbGciOiJIUzI1NiJ9.eyJuYW1lIjoiRGhkaGRoIiwidmlkZW8iOnsicm9vbSI6Im1vYjUtNmpydCIsInJvb21Kb2luIjp0cnVlLCJjYW5QdWJsaXNoIjp0cnVlLCJjYW5QdWJsaXNoRGF0YSI6dHJ1ZSwiY2FuU3Vic2NyaWJlIjp0cnVlfSwiaXNzIjoiQVBJVmVNQnBxd3lMa01OIiwiZXhwIjoxNzA3NDAwMTY4LCJuYmYiOjAsInN1YiI6IkRoZGhkaCJ9.npoyKyWHXwQVTiRAeeP2zBJoklZkVc7FpLQ4pYXvSxg&auto_subscribe=1&sdk=js&version=1.15.11&protocol=11&adaptive_stream=1 – {room: undefined, roomSid: undefined, identity: undefined} (dba1bf7f-511427f33cbdd780.js, line 1)
[Warning] Could not resume audio context – {room: "", roomSid: "", identity: "", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "", roomSid: "", identity: "", error: InvalidStateError: Failed to start the audio device}Object
[Log] room event audioPlaybackChanged – {room: "", roomSid: "", identity: "", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "", roomSid: "", identity: "", event: "audioPlaybackChanged", args: []}Object
[Log] {url: "wss://meet-uylq8icb.livekit.cloud"} ([name]-405583ddfedead02.js, line 1)
[Log] ping config – {room: undefined, roomSid: undefined, identity: undefined, …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: undefined, roomSid: undefined, identity: undefined, timeout: 15, interval: 5}Object
[Log] clearing ping interval – {room: undefined, roomSid: undefined, identity: undefined} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] start ping interval – {room: undefined, roomSid: undefined, identity: undefined} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] connected to Livekit Server edition: 1, version: 1.5.2, protocol: 13, region: Germany, nodeId: NM_DFRA1A_QG6MNjuLf5Qv, debugInfo:  – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] room event signalConnected – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", event: "signalConnected", args: []}Object
[Log] setTrackEnabled – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", participantSid: "PA_bjRDKDUyfAEu", participantId: "Dhdhdh", …}Object
[Log] setTrackEnabled – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", participantSid: "PA_bjRDKDUyfAEu", participantId: "Dhdhdh", …}Object
[Log] setTrackEnabled – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", participantSid: "PA_bjRDKDUyfAEu", participantId: "Dhdhdh", …}Object
[Log] received server offer – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", RTCSdpType: "offer", sdp: "v=0\r↵o=-<PHONE_NUMBER>015216362<PHONE_NUMBER> IN IP4 0.0…jlp\r↵a=ice-pwd:MXCLxNbCkLJynePBatqIjNzUHnDYlySK\r↵", …}Object
[Log] awaiting getUserMedia promise (dba1bf7f-511427f33cbdd780.js, line 1, x2)
[Info] publishing track – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", participantSid: "PA_bjRDKDUyfAEu", participantId: "Dhdhdh", …}Object
[Info] publishing track – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", participantSid: "PA_bjRDKDUyfAEu", participantId: "Dhdhdh", …}Object
[Warning] Could not resume audio context – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", error: InvalidStateError: Failed to start the audio device}Object
[Log] using video encoding – {maxBitrate: 1445000, maxFramerate: 30, priority: undefined} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] using svc with scalabilityMode L3T3_KEY (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] encodings – [Object, {rid: "h", maxBitrate:<PHONE_NUMBER>666667, maxFramerate: 30}, {rid: "q", maxBitrate:<PHONE_NUMBER>5555556, maxFramerate: 30}] (3) (dba1bf7f-511427f33cbdd780.js, line 1)
[Object, {rid: "h", maxBitrate:<PHONE_NUMBER>666667, maxFramerate: 30}, {rid: "q", maxBitrate:<PHONE_NUMBER>5555556, maxFramerate: 30}]Array (3)
[Log] room event audioPlaybackChanged – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", event: "audioPlaybackChanged", args: [true]}Object
[Log] received trackPublishedResponse – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", cid: "8aeb3b33-001b-44d4-9811-467e85f35450", track: "TR_AMFgHQzoxR7kuT"}Object
[Log] update publication info – {trackSid: "TR_AMFgHQzoxR7kuT", trackName: "", track: undefined, …} (dba1bf7f-511427f33cbdd780.js, line 1)
{trackSid: "TR_AMFgHQzoxR7kuT", trackName: "", track: undefined, trackEnabled: true, trackEncrypted: false, …}Object
[Log] publishing audio with encodings – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", participantSid: "PA_bjRDKDUyfAEu", participantId: "Dhdhdh", …}Object
[Log] setting munged local description – {} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] sending answer – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", answerSdp: "v=0\r↵o=-<PHONE_NUMBER>491493167 2 IN IP4 <IP_ADDRESS>\r↵s…:78:54\r↵a=ice-options:trickle\r↵a=sctp-port:5000\r↵"}Object
[Log] pc state change: from NEW to CONNECTING – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] primary PC state changed 1 – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] received trackPublishedResponse – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", cid: "ec58dfdf-8e49-463f-8f9f-6acea898ab74", track: "TR_VCADDE3rBdR5XA"}Object
[Log] update publication info – {trackSid: "TR_VCADDE3rBdR5XA", trackName: "", track: undefined, …} (dba1bf7f-511427f33cbdd780.js, line 1)
{trackSid: "TR_VCADDE3rBdR5XA", trackName: "", track: undefined, trackEnabled: true, trackEncrypted: false, …}Object
[Log] publishing video with encodings – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", participantSid: "PA_bjRDKDUyfAEu", participantId: "Dhdhdh", …}Object
[Log] get sender capabilities – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", cap: Object}Object
[Warning] silence detected on local audio track – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", participantSid: "PA_bjRDKDUyfAEu", participantId: "Dhdhdh", …}Object
[Log] starting to negotiate – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] setting munged local description – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] sending offer – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", offerSdp: "v=0\r↵o=-<PHONE_NUMBER>705487063 2 IN IP4 <IP_ADDRESS>\r↵s…le\r↵a=sctp-port:5000\r↵a=max-message-size:262144\r↵"}Object
[Log] received server answer – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", RTCSdpType: "answer"}Object
[Log] setting munged remote description – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] room event localTrackPublished – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", event: "localTrackPublished", args: Array}Object
[Log] room event localTrackPublished – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", event: "localTrackPublished", args: Array}Object
[Log] rawFacingMode – {rawFacingMode: "user"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] on data channel 1, _reliable – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] on data channel 3, _lossy – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] video playback aborted, likely due to new play request (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] pc state change: from CONNECTING to CONNECTED – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] primary PC state changed 2 – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Log] updating track subscription permissions – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", participantSid: "PA_bjRDKDUyfAEu", participantId: "Dhdhdh", …}Object
[Log] room event connectionStateChanged – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", event: "connectionStateChanged", args: ["connected"]}Object
[Log] room event connected – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", event: "connected", args: []}Object
[Log] rawFacingMode – {rawFacingMode: "user"} (dba1bf7f-511427f33cbdd780.js, line 1)
[Warning] silence detected on local audio track – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", participantSid: "PA_bjRDKDUyfAEu", participantId: "Dhdhdh", …}Object
[Log] room event localAudioSilenceDetected – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", event: "localAudioSilenceDetected", args: Array}Object
[Log] room event connectionQualityChanged – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", event: "connectionQualityChanged", args: Array}Object
[Log] upstream paused – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", participantSid: "PA_bjRDKDUyfAEu", participantId: "Dhdhdh", …}Object
[Log] upstream paused – {room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", …} (dba1bf7f-511427f33cbdd780.js, line 1)
{room: "mob5-6jrt", roomSid: "RM_5YA67X9Muney", identity: "Dhdhdh", participantSid: "PA_bjRDKDUyfAEu", participantId: "Dhdhdh", …}Object

I tried to reproduce this again and wasn't able to, are you still seeing this issue on meet.livekit.io?

I tried to reproduce this again and wasn't able to, are you still seeing this issue on meet.livekit.io?

No, seems to be solved. Closing


Feature request: Create new file by right clicking on existing file

A nice alternative to
Right click on a directory in the sidebar-tree > ANF New file here
Would be to
Right click on a file in the sidebar-tree > ANF New file in same dir
Which would use the existing file's path, with the cursor placed at the end of the name, before the extension dot. I'm often creating files that have very similar names to their neighbors.
Preventing accidental overwrite with a yes-no prompt would work.
One of my most important plugins. Thanks again for it.

Sorry for the delay in responding. I've implemented a version of this already. Will try to push the change soon.

My sincerest apologies, I thought I had pushed this a long time ago, but instead had it sitting on a branch on my machine. Anyways, I have  pushed an update for this. There is a new command AdvancedNewFileNewAtCommand that can be mapped to in the side bar menu. In addition there is a new setting cursor_before_extension which can be set to true, which should place the cursor prior to the extension. When a file exists, it simply tries to open that file, so there should be no risk of accidently overriding an existing file.
Again, my apologies. Thank you for using AdvancedNewFile.

Excited to start using it! Thank you for doing this.


[Meta] Mechanism to publish the plugin zips to maven repo.

Is your feature request related to a problem? Please describe
With current setup the plugins zips that are used as dependencies cannot be downloaded dynamically during runtime, as they are not part of version controlled maven system. Only the respective Java jars are available to download using maven coordinates.
More details on https://github.com/opensearch-project/opensearch-build/issues/716
Describe the solution you'd like
Publish the release candidate zips, distribution build zips (snapshots) to maven repo, so that a user who is dependent on the plugin zip, can download using maven coordinates. There needs to be a process implemented for uploading zips to snapshot and release repos
Proposed Solution [1]:
With the current build system the plugin zips are stored under builds/plugins/, these are not stored or ordered in maven folders, so that maven can publish them with right artifactID, groupID and version.
for each plugin use install:install-file to create the maven coordinates and publish them to local file system maven repo, passing the right artifactID, groupID and version along with proper qualifier if required.
Then from local file system publish them to respective snapshot or release repos, adhering to existing workflows.
To achieve this integrate this solution with existing CI.
Proposed Solution [2]: ---> (The chosen one)
Add a gradle sub-project in each component, so that root gradle project can include them, that converts the zips generated inside distribution folder to maven format, then publish them to local maven repo and staging repo, this way all the maven coordinate's (groupID, artifactID, version) generation and other maven heavy lifting is done at component (plugin) level and using CI with already existing code, publish them to maven nexus repo.
Additional context
For Proposed Solution [1]:
Need to come up with an idea to add either a python script,  a groovy shared library or a gradle sub tasks for publishing the zips.
Snapshot zips:
Upload the snapshot plugin zips to https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/plugin/
<plugin_name>/-SNAPSHOT/<.zip_file>
Edit the plan https://github.com/prudhvigodithi/opensearch-build/blob/slack-publish-v1/jenkins/opensearch/distribution-build.jenkinsfile#L82 to upload zips to nexus maven repo
Release zips:
The Release candidates will be signed before staging to make sure files are unaltered.
Also there needs to be an additional tags inside the POM as example for the zips pom file
https://aws.oss.sonatype.org/service/local/repositories/releases/content/org/opensearch/common-utils/<IP_ADDRESS>/common-utils-<IP_ADDRESS>.pom
Create a staging repo nexus-staging:rc-open   this will create a new new staging repository on the remote Nexus
Upload zips from the local filesystem to a staging repository using plugin in built deploy-staged-repository
Close the created staging repo nexus-staging:rc-close
For Proposed Solution [2]:
Add the gradle subproject to each repo and include in root project, for the generated zip to get published to local staging maven repo.
Implementation tasks

[ ] Upload the distribution snapshot plugin zips to nexus maven snapshot repo.
[ ] Sign and Upload the release plugin zips to nexus maven release repo.
[ ] Document publishing to maven central
[ ] Add Plugin design and usage doc

Associated Issues

[ ] https://github.com/opensearch-project/opensearch-build/issues/716
[ ] https://github.com/opensearch-project/opensearch-build/issues/1823

Issue with integrating the gradle java custom plugin:

[x] https://github.com/opensearch-project/OpenSearch/issues/3025

opensearch-plugin-template-java jdk and gradle version version upgrade

[x] https://github.com/opensearch-project/opensearch-plugin-template-java/issues/30

Document Create

[x] https://github.com/opensearch-project/opensearch-plugins/issues/143

Campaign Issues:

[x] https://github.com/opensearch-project/opensearch-plugins/issues/145
[ ] https://github.com/opensearch-project/alerting/issues/435
[ ] https://github.com/opensearch-project/anomaly-detection/issues/525
[ ] https://github.com/opensearch-project/asynchronous-search/issues/140
[x] https://github.com/opensearch-project/common-utils/issues/164
[ ] https://github.com/opensearch-project/dashboards-reports/issues/351
[ ] https://github.com/opensearch-project/index-management/issues/348
[x] https://github.com/opensearch-project/job-scheduler/issues/175
[ ] https://github.com/opensearch-project/k-NN/issues/378
[ ] https://github.com/opensearch-project/performance-analyzer/issues/202
[ ] https://github.com/opensearch-project/performance-analyzer-rca/issues/175
[ ] https://github.com/opensearch-project/security/issues/1801
[ ] https://github.com/opensearch-project/sql/issues/592
[ ] https://github.com/opensearch-project/observability/issues/694
[ ] https://github.com/opensearch-project/cross-cluster-replication/issues/384
[ ] https://github.com/opensearch-project/ml-commons/issues/306
[ ] https://github.com/opensearch-project/geospatial/issues/56

Associated PR's

[x] https://github.com/opensearch-project/job-scheduler/pull/167
[x] https://github.com/opensearch-project/security/pull/1757
[x] https://github.com/opensearch-project/opensearch-build/pull/1953

With custom plugin:

[x] https://github.com/opensearch-project/OpenSearch/pull/2988/files
[x] https://github.com/opensearch-project/OpenSearch/pull/3095
[x] https://github.com/opensearch-project/OpenSearch/pull/3096
[ ] https://github.com/opensearch-project/opensearch-build/pull/1999
[x] https://github.com/opensearch-project/OpenSearch/pull/3252

Plugin design and usage doc

[x] https://github.com/opensearch-project/opensearch-plugins/pull/144

opensearch-plugin-template-java jdk and gradle version version upgrade

[x] https://github.com/opensearch-project/opensearch-plugin-template-java/pull/29

opensearch-plugin-template-java add opensearch.pluginzip details to template

[x] https://github.com/opensearch-project/opensearch-plugin-template-java/pull/31

Looks like a dup of https://github.com/opensearch-project/opensearch-build/issues/716

Hey @dblock, since there are two aspects for this maven publish, one for snapshots repo and other for actual release repo, the two repo workflows are not constant, as release repo has more wider process, like pre staging, promoting, signing, additional maven tags and then releasing, so I have created a meta issue https://github.com/opensearch-project/opensearch-build/issues/1916 that expands with more details based on the issue https://github.com/opensearch-project/opensearch-build/issues/716.
Right now the snapshot zips for maven snapshot repo, does not have an enforced workflow to publish a zip, unlike the actual release repo, so I have separated into two tasks for snapshot plugin zips and release plugin zips and together tracked in https://github.com/opensearch-project/opensearch-build/issues/1916.

@dblock I am hesitating a bit to acknowledge that publishing plugin ZIPs to Maven repositories is a good idea (it is certainly possible - no doubts). But from other side - how people are supposed to use it? It looks to me we should align the publishing (at least, of the standard plugins) with the opensearch-plugin tooling: the plugin ZIP (snapshot / release / ...) should end up in the catalog and be downloadable from there.
In scope of this issue, if you need those dependencies in Maven projects - the direct URL to the catalog could be used along with plugin (fe [1]), they should not be only artifacts. Also, it covers well external 3rd party plugins, often hosted o Github and not published to Maven repos (afaik).
Hope I didn't introduce the confusion, what do you think?
[1] https://github.com/maven-download-plugin/maven-download-plugin

@reta I think as a consumer I would ideally want to do something like this in Gradle:
dependencies {
    opensearchPlugin 'org.opensearch.job-scheduler.spi:2.0-SNAPSHOT'
}

Latest snapshot build would be downloaded from Maven and automatically installed into an integration test cluster before I run integTest.
Is this crazy?

Hey @dblock, I see this PR overrides the requirement to download a zip in a maven way, it uses directly GET protocol to download the latest zip, also as suggested by @reta  maven-download-plugin download different files on different protocol and this can connect with above change, a user can download straight from URL without giving maven coordinates as
        <uri>https://example.com/file1.zip</uri>
        <uri>https://example.com/file2.zip</uri>
        <uri>https://example.com/file3.zip</uri>
        <!-- and so on -->
    </uris>

So from consumer perspective if we can document this we can skip zips to maven publish, else once its in maven a user can always download using maven coordinates as well as with GET protocols.
@bbarani @peterzhu1992

Once ZIPs are published to maven the URL will be changing every time, you'll need to be able to resolve something like https://aws.oss.sonatype.org/service/local/repositories/snapshots/content/org/opensearch/opensearch-job-scheduler-spi/<IP_ADDRESS>-alpha1-SNAPSHOT/opensearch-job-scheduler-spi-<IP_ADDRESS>-alpha1-20220406.220005-79.zip (here's the jar published today). So the solution in that PR doesn't work.

@reta I think as a consumer I would ideally want to do something like this in Gradle:
dependencies {
    opensearchPlugin 'org.opensearch.job-scheduler:2.0-SNAPSHOT'
}

Latest snapshot build job-scheduler-2.0-SNAPSHOT.zip would be downloaded from Maven and automatically installed into an integration test cluster before I run integTest. All this support would be implemented in Gradle in OpenSearch min.
Is this crazy?

No, it is not, but:

what about Maven / SBT / ... build tools?
where the opensearch-plugin downloads plugins from?

Something like maven-download-plugin works well as a middle-ground solution, but @prudhvigodithi it looks like you chose the maven route, I imagine you have a clear idea of how those artifacts will be used?

There are plenty of projects which use this route (publishing ZIP/TAR/...), it is on the table for sure and it works. BUT that would mean, very likely, we have to ask all external plugin authors to do that for their plugins (and as such, they would have to look for publishing options).

@dblock I dont see date format is added to zips with current setup, the plugins are just dumped as regular files without date,  so you are saying if we add ./gradlew publishShadowPublicationToStagingRepository it will add date as well to the zip, just like it adds for jar ?, but gradle should always pull the latest if we pass org.opensearch.job-scheduler:2.0-SNAPSHOT even though there are multiple date formatted zips, also I guess it would be better to add plugins under https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/plugin/

@dblock I dont see date format is added to zips with current setup, the plugins are just dumped as regular files without date, so you are saying if we add ./gradlew publishShadowPublicationToStagingRepository it will add date as well to the zip, just like it adds for jar ?, but gradle should always pull the latest if we pass org.opensearch.job-scheduler:2.0-SNAPSHOT even though there are multiple date formatted zips, also I guess it would be better to add plugins under https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/plugin/

Yes, it will add the date to the file name, that's how maven works for snapshots. Take a look at https://aws.oss.sonatype.org/service/local/repositories/snapshots/content/org/opensearch/opensearch-job-scheduler-spi/<IP_ADDRESS>-alpha1-SNAPSHOT/, that's all the artifacts for many snapshot builds. To find out what the latest one is, maven looks in this file.
When we publish a final released version of 2.0, there's only 1, so maven doesn't need to append the date or anything.

> > Is this crazy?

No, it is not, but:

what about Maven / SBT / ... build tools?

Maven could use maven-download-plugin, etc. Either way we're only debating whether ZIPs are published like JARs in maven, or they are published as ZIPs somewhere else. We can do both and let people decide what they like.

where the opensearch-plugin downloads plugins from?

It piggybacks on maven repository resolution 100%. It's effectively a fancy wrapper over (the equivalent of) maven-download-plugin, a syntax sugar that knows to install the plugin into an OpenSearch cluster before running integ tests.

@dblock sounds good, as far as different tools pull ZIPs consistently using same conventions, we could use Maven as the distribution mechanism.

Hey @dblock, I have tested the idea to work backward from individual repo, to be able to add assembled zip's also to maven format just like how the jars are added. I was able to do this Draft. Please check the following implementation steps

Directly publishing the plugins zips in to current maven repo(the place where the jars are published ), would cause some problem, as it appends with -spi suffix, but the distribution zip (actual plugin does not create this way), we should actually separate the maven coordinates for the plugins, rather that having same maven articatID, groupID and version just like the jar files. Example as this is default if we directly take the zip and add it as part of publish

f build.gradle.

Since we are using shadow plugin to publish, as per the functionality it will publish both the sourcesJar and the javadocJar (along with the shadowjars), along with the zip ----> we need to remove those 2 jars as it will duplicate the sourcesJar and the javadocJar,
Example if seen the below screenshot, we are seeing the sourcesJar and the javadocJar are pushed twice

Since we dont need those  sourcesJar and the javadocJar files for zip's, and we since we also cant exclude in with shadow publishing, I have added a quick shortcut, to clean them using find cli, this will not update the metadata.xml file generate for the local repo.

Now with this a user should be able to download opensearch-job-scheduler plugin zip with maven coordinates as
  <groupId>org.opensearch.plugin</groupId>
  <artifactId>opensearch-job-scheduler</artifactId>
  <version><IP_ADDRESS>-alpha1-SNAPSHOT</version>

The above values of groupId, artifactId, version are all inherited from default job-scheduler build system, as all heavy lifting is already been done here.

Thanks for the update @prudhvigodithi ,

Now separating out the maven coordinates for plugin Zips, Since we are using shadow plugin to publish, as per the functionality it will publish both the sourcesJar and the javadocJar (along with the shadowjars), along with the zip ----> we need to remove those 2 jars as it will duplicate the sourcesJar and the javadocJar,
Example if seen the below screenshot, we are seeing the sourcesJar and the javadocJar are pushed twice

I think there should be a way to strip those artifacts (javadoc and sources) from ZIP publishing without CLI scripting. May be we should strip them by default and only configure with JAR publisher?

I like the draft! I obviously dislike the custom elimination of sources and jar, which is hopefully something that can be disabled in shadow. Then, job scheduler has this spi vs. non-spi build gradle so this may be muddying the waters. Should this be going into root buld.gradle? Should this be in the distribution gradle? A new subfolder dedicated to publishing the zip to maven?
I think you should iterate on a PR into job-scheduler for this and let's see if we can merge something?

@reta, yes we can directly push them with jar, then all the zips, jar javadoc and sources jars's will be under same maven location, which could confuse users to use those zip plugins, hence I wanted to separate the maven location for plugin zips, there already exists [plugin]( https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/plugin/,)  prefix.
ya thats a good point @reta I have this question as well, do we actually use these javadoc and sources jars's ? @dblock @bbarani @reta
If not we dont need to publish them, I see for example opensearch-ml-client  does not have these javadoc and sources jars's (it depends on app to app as well).
@dblock I liked the idea for a new subfolder, let me explore this and yes my idea was 1st once we have the job-scheduler working, then we can have the same implementation for other components as well. 👍

ya thats a good point @reta I have this question as well, do we actually use these javadoc and sources jars's ?

they are useful for developers (for example if you develop a plugin which depends on another plugin), so it is better to have them published.

@dblock I liked the idea for a new subfolder, let me explore this and yes my idea was 1st once we have the job-scheduler working, then we can have the same implementation for other components as well.

We could actually follow the other projects and publish ZIPs as -distribution artifacts, fe [1]
[1] https://repo1.maven.org/maven2/org/eclipse/jetty/jetty-distribution/9.4.9.v20180320/

Cool, thanks for the info @reta
I found something, I actually dont see these  javadoc and sources jars's  for [opensearch-job-scheduler] (https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/opensearch-job-scheduler/<IP_ADDRESS>-alpha1-SNAPSHOT/) as well, but I so see them on my local when I build them.

I see whats happening so this block of code  eliminates from uploading them :)
So I guess its useful @reta like the main jar is itself built with all dependancies, please correct me if i'm wrong
@dblock

@dblock @prudhvigodithi since I am lacking the context here, sorry for possible derailing, but using custom made scripts for publishing artifacts is not what the community (since we are discussing the plugins) would expect: gradle publish (and its variations) should be enough and the only way (imho)

Wanted to chime in with a few callouts & questions.

@dblock @prudhvigodithi since I am lacking the context here, sorry for possible derailing, but using custom made scripts for publishing artifacts is not what the community (since we are discussing the plugins) would expect: gradle publish (and its variations) should be enough and the only way (imho)

Agree 100%.  This stems from a current limitation with our signing process not wired into gradle.  So the build system builds and pushes the artifacts that are then after the fact picked up, signed, and promoted with these scripts.

I see whats happening so this block of code eliminates from uploading them :)

We've had an issue out for killing this script for a while.  We should be pushing directly from gradle publish.  It doesn't look like we are signing every snapshot build so we don't have the limitation there.  I'm thinking maybe we can update the opensearch publish plugin to automatically give all plugins a task to push snapshots, and they only have to define their maven publication.
As far as placing maven publications in the same group/artifact id with different packaging. I'm hesitant about this because I'm not sure the jars produced by each plugin match the produced plugin zips that are installed today. Meaning the artifact is inherently different and not simply differentiated by packaging type.  If this is the case I think we should treat them as two separate artifacts under the same group ID.

Hey @dblock, @reta, please check this Draft, I have modified to add the zip publish task to root gradle project, I'm initially ok with creating a new folder and subproject, but there will be some duplicate code that needs to be re added like injecting version, qualifiers, snapshot flags and local repo stage information, so I went ahead with root project since these are already being passed here. With I see the maven coordinates for zip plugin being created as Example
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.opensearch.plugin</groupId>
  <artifactId>opensearch-job-scheduler</artifactId>
  <version><IP_ADDRESS>-alpha1-SNAPSHOT</version>
  <packaging>zip</packaging>
  <name>OpenSearch Job Scheduler Plugin zip</name>
  <url>https://github.com/prudhvigodithi/job-scheduler.git</url>
  <scm>
    <url>https://github.com/prudhvigodithi/job-scheduler.git</url>
  </scm>
  <inceptionYear>2021</inceptionYear>
  <licenses>
    <license>
      <name>The Apache Software License, Version 2.0</name>
      <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
    </license>
  </licenses>
  <developers>
    <developer>
      <name>OpenSearch</name>
      <url>https://github.com/opensearch-project/job-scheduler</url>
    </developer>
  </developers>
</project>

Since this is added as part of root project, we call a task publishAllPublicationsToStagingRepository that by default checks for all maven publish tasks.
I have also added a flag ext.publish = 'skip', just in case in future if a user wants ti disable to publish the zips to maven
So my plan is to get this merged to job-scheduler and will no other code change we should be see job-scheduler plugin zips under nexus maven repo plugin location.

As far as placing maven publications in the same group/artifact id with different packaging. I'm hesitant about this because I'm not sure the jars produced by each plugin match the produced plugin zips that are installed today. Job-scheduler-spi vs job-scheduler as an example. Meaning the artifact is inherently different and not simply differentiated by packaging type. If this is the case I think we should treat them as two separate artifacts under the same group ID.

Hey @reta, thats true, we should not use same groupID for different package types, I have separated the zips's as
  <groupId>org.opensearch.plugin</groupId>
  <artifactId>opensearch-job-scheduler</artifactId>
  <version><IP_ADDRESS>-alpha1-SNAPSHOT</version>

@dblock, I think this is much better solution :)
I have created a new gradle subproject so each component(plugin team) can include this, this was its easy to add this project if required to publish the zip and no change needed on the main root project.
I have tested for job-scheduler, that uses its own build script and security that uses default build script, in both scenarios this idea worked.
This was the new subproject does not publish testJar, sourcesJar and the javadocJar to the zip package, as its not a java project (Once generated testJar, sourcesJar and the javadocJar and if its a Java project, gradle publish does not exclude them)
The another advantage with this is, each plugin can easily add this project to existing setup with no logic change of this subproject.
Job-scheduler:
https://github.com/opensearch-project/job-scheduler/compare/main...prudhvigodithi:main
Security:
https://github.com/opensearch-project/security/compare/main...prudhvigodithi:main
NOTE: Since Security uses default build script, there need be a small change in opensearch-build repo as well i.e https://github.com/opensearch-project/opensearch-build/compare/main...prudhvigodithi:zipsmaven
I have included a switch ext.publish = 'skip', with this ext even if the subproject is added, the zips wont be published to maven repo. This was a team need not delete the subproject if at all they have decided not the publish the zips.
@bbarani

Hey @dblock, @reta, please check this Draft, I have modified to add the zip publish task to root gradle project. With this I see the maven coordinates for zip plugin being created as Example

Thanks @prudhvigodithi , it looks much better than previous scripted version (at least, to me).

Thanks @prudhvigodithi , it looks much better than previous scripted version (at least, to me).

Hey @reta, please check my most recent comment that has even a better way of handling this :)

Better! I would rename zipsmaven to just maven or publish to accommodate future publications of non-zips.

The another advantage with this is, each plugin can easily add this project to existing setup with no logic change of this subproject.

This would be great and reduce a lot of duplication in each plugin.  Where would the subproject live and be imported?  Is it distributed from core as a plugin with configuration required by each plugin or imported as a project?

@mch2 right now the subproject will be added as part of every plugin repo (Ex for security, job-scheduler), as a future improvement, we can make sure it can be imported from a central location and maven coordinates (artifcatID, groupID, version and name) can be manipulated during runtime, this would be even more robust as the maintenance of code is also easy at single place.

@mch2 right now the subproject will be added as part of every plugin repo Ex for security, job-scheduler
include 'pluginZips'
project(':pluginZips').projectDir = file('maven/plugin-zips')

as a future improvement, we can make sure it can be imported from a central location and maven coordinates (artifcatID, groupID, version and name) can be manipulated during runtime, this would be even more robust as the maintenance of code is also easy at single place.

Because you're doing this for each plugin, maybe worth the effort to move common code into core?

@dblock, @reta I have tested with publishing job-scheduler (1.3.0 working version) plugin zip to nexus maven repo (snapshots)
Maven repo URL: https://aws.oss.sonatype.org/content/repositories/snapshots/org/opensearch/plugin/opensearch-job-scheduler/<IP_ADDRESS>-SNAPSHOT/
Able to also fetch the plugin zip using maven coordinates
mvn org.apache.maven.plugins:maven-dependency-plugin:2.1:get  -DrepoUrl=https://aws.oss.sonatype.org/content/repositories/snapshots/  -Dartifact=org.opensearch.plugin:opensearch-job-scheduler:<IP_ADDRESS>-SNAPSHOT:zip

This looks good to me! Get https://github.com/opensearch-project/OpenSearch/pull/2988 to a ready state and let's get it merged?
For pom.xml updates, if that's a requirement, you probably want to expose a DSL that dynamically adds anything to the pom.
pom {
    inceptionYear: '2021'
    developers: [
       developer {
          name: 'Bob'
       }
    ]
}

you get the idea ... it's definitely something that can be added later too.

Hey @dblock I have initially started it by adding it to the plugin extension, as
zipmavensettings {
}

This extension already exists now with custom zippublish plugin
But I see the already existing plugin expects in the same format, just to be consistent (as I have already seen few repos using them, in the format I have shared) I have added this way.

Continuing with plugin test:
cross-cluster-replication:
Add a gradle property: zipArtifact=opensearch-cross-cluster-replication
https://github.com/prudhvigodithi/cross-cluster-replication/blob/gradleplugin/gradle.properties
Apply the plugin: apply plugin: 'opensearch.zippublish'
https://github.com/prudhvigodithi/cross-cluster-replication/blob/gradleplugin/build.gradle#L96
Run the task to publish the zips: (security uses default build scripts from opensearch-build repo): https://github.com/opensearch-project/opensearch-build/pull/1999

@dblock
I have observed the assemble plugins zip location for each repo is different
Example
ml-commons add's the zip to ./plugin/build/distributions/*.zip
Alerting add's the zip to  ./alerting/build/distributions/opensearch-alerting-<IP_ADDRESS>-alpha1-SNAPSHOT.zip
Job-scheduler, security, AD add's the zip to /build/distributions/
I have added to look for /build/distributions/
Error I have observed
ml-common
> Task :publishMavenzipPublicationToZipstagingRepository FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':publishMavenzipPublicationToZipstagingRepository'.
> Failed to publish publication 'mavenzip' to repository 'zipstaging'
  > Invalid publication 'mavenzip': artifact file does not exist: '/tmp/tmpxa59alv1/ml-commons/build/distributions/opensearch-ml-<IP_ADDRESS>-alpha1-SNAPSHOT.zip'

@prudhvigodithi @peterzhuamazon  Plugin zip files can be downloaded with urls today, why is this issue needed?

@prudhvigodithi @peterzhuamazon Plugin zip files can be downloaded with urls today, why is this issue needed?

Hey @cliu123, this issue describes to download the zip's using maven coordinates, this can be used with dependency model as well as can downloaded directly using the URL's once its in maven repo. More examples and details on this meta issue as well as on https://github.com/opensearch-project/opensearch-build/issues/716

@prudhvigodithi This has not been communicated as a requirement for 2.0.0 GA can we disconnect this deliverable with the v2.0.0 release?  It seems like a valid ask, but I'm not sure the time pressure associated with this work is accomplishing what we need.
In an ideal world this would be part of the 2.0.0 preparation phase and agreed before started coding, so plugin teams can confirm the resourcing available and ensure they know how to do the discussed work.

@prudhvigodithi This has not been communicated as a requirement for 2.0.0 GA can we disconnect this deliverable with the v2.0.0 release? It seems like a valid ask, but I'm not sure the time pressure associated with this work is accomplishing what we need.
In an ideal world this would be part of the 2.0.0 preparation phase and agreed before started coding, so plugin teams can confirm the resourcing available and ensure they know how to do the discussed work.

Hey @peternied, adding zip's to maven repo add's lot of benefits, thats the community expectation as well. The heavy lifting is all done by the opensearch.pluginzip, it wont be a complex integration for plugin team and should be able to implement this with few steps as shown in the above doc.

We are asking plugin teams to integrate our plugin to publish the Zips to Maven and all the heavy lifting is already done by our team.

@prudhvigodithi what do you think about integrating the required build.sh changes for the opensearch.pluginzip into the default build.sh script? For index management, we use the default build script and I would prefer to keep it default so we don't need to make a change in our plugin repo whenever there is a central build change. As we are on-boarding this to all plugins anyways, I would prefer this.

Hey @downsrob there is already some discussion out there regarding this, please check this PR
@dblock @bbarani any thoughts ?

@prudhvigodithi dB looks to be blocking that PR until all plugins get this change in to avoid adding custom logic to the default build script, but that would mean introducing a custom build script in every plugin, then changing the default build script and then removing the custom build script from each plugin. I would prefer to only add the publishing zip task for plugins with the default build script, and then once all plugins have onboarded, the default build script can change to introduce the change without requiring an interim build script for all plugins.

What @downsrob is proposing works fine as well - get all plugins to onboard without adding a custom build script, then change the default to enable it for everyone.

Hey @dblock @downsrob, if I understood correct, 1st make sure all plugins use the gradle plugin opensearch.pluginzip (make all changes in build,.gradle file as per document) without executing the task publishPluginZipPublicationToZipStagingRepository and then make change to default build.sh to add publishPluginZipPublicationToZipStagingRepository, is that what the suggestion is ?

Yes
if this is the case, the zip's wont be published, as the task publishPluginZipPublicationToZipStagingRepository should be called out to ensure the zips are published.
i am not sure i understand what you’re saying :) once you’ve added the task to the default build.sh without conditionals it will work for all plugins
Note that the default script is still used for older versions, so we may not be able to modify it after all

I'm fine until all plugins are onboarded to use opensearch.pluginzip and then include in default build.sh file, but the catch here is the plugins team should also backport the changes to older versions if not it would break the old versions when the default build.sh is called that has new added task publishPluginZipPublicationToZipStagingRepository.
The other better way is create a build.sh file to each plugin and manage the assemble and publish tasks, this could be a starting point for this change, but it would benefit long term as each plugin team can manage the set of tasks to run during build process and we can keep the generic default build.sh for infrastructure automations, thoughts? @downsrob @dblock @bbarani

If you want a published zip of the older versions anyways, say, for BWC testing, then I can see the merit in backporting the task to all versions and moving on.  However, it seems natural that eventually the default build.sh would no longer encompass all versions, if not from this change, maybe the next. What do you think about versioning the default script on the infra side?

If you want a published zip of the older versions anyways, say, for BWC testing, then I can see the merit in backporting the task to all versions and moving on. However, it seems natural that eventually the default build.sh would no longer encompass all versions, if not from this change, maybe the next. What do you think about versioning the default script on the infra side?

That's proposed in https://github.com/opensearch-project/opensearch-build/issues/1975, add your comments there.
I want to make sure we don't try to solve the kitchen sink at the same time, so maybe cloning the default build script for your plugin is the most expedient path?

Starting with release 2.1.0, we now have plugin zips in maven repo.
alerting
anomaly-detection
asynchronous-search
reports-scheduler
index-management
job-scheduler
k-NN
performance-analyzer
security
sql
observability
cross-cluster-replication
ml-commons
Thank you

Closing this META as associated tasks are marked as complete, please feel free to re-open this issue if required. :)
Thank you
@dblock @CEHENKLE @bbarani


Expose experimental binder transport API

New headers are added
grpcpp/create_channel_binder.h : interfaces for creating client
channel
grpcpp/security/binder_credentials.h: interfaces for binder server
credentials
grpcpp/security/binder_security_policy.h: interfaces for binder
security policy, which is used by both server and client. Individual
security policies are merged into this single header.
Users can now depend on the grpc++_binder target to use the headers
listed above.

Quick Note: The merge conflict is because we moved the whole header in this PR, and there are minor changes in the header. Shouldn't affect the readiness of code review

PHP CI run fails in every recent PRs / ObjC iOS infra timeout


refactor: make CCTX tracking browser-compatible

This refactor removes dependence on Node.js-only libraries making it possible to use CCTX tracking from browser environment.

Merged into https://github.com/zeta-chain/toolkit/pull/74


Power off

I set up mi pi Zero W with raspotify and I am very happy with it. However, I wonder how people usually power the pi off when using it as a Spotify connect device. I don't want to just unplug it from the wall every time I am not using it because the SD card will eventually get corrupted.

Not related to librespot / raspotify, however, you could try executing sudo shutdown or click shutdown in the UI.


Just a "only" too much

In the first "Note"-section it says:
"For Azure SQL Managed Instance, the supported option FORCE_LAST_GOOD_PLAN can only be configured through T-SQL only.
I think the last "only" should be removed.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5ef45327-7cdf-fc60-2a04-b785bc12f4cf
Version Independent ID: b4f8c915-9651-14df-92d7-fa1387780a41
Content: Enable automatic tuning - Azure SQL Database
Content Source: articles/azure-sql/database/automatic-tuning-enable.md
Service: sql-db-mi
Sub-service: performance
GitHub Login: @danimir
Microsoft Alias: danil

@2ndcahl Thank you for your feedback. We are reviewing this and will get back shortly.

@2ndcahl - I am fine with this suggestion. Content editors can go ahead and make this style change.

@danimir I will raise a PR to update the doc. Thanks

2ndcahl I have processed a change to change this in the documentation that should be live later this week. Thanks for helping us improve our documentation.


Need a new release to support the https_verify changes

The latest release doesn't have the https_verify changes.  Please create a new rubygems release :)

Will do it soon.

Here you go https://rubygems.org/gems/rhn_satellite/versions/0.1.2


Use generic ItemView type for buttons instead of AnyView

Memory fails me as I can't recall where I've heard it (probably WWDC2021 talk Demystifying SwiftUI), but SwiftUI uses the type system to determine the structured identity of nodes in the view tree. AnyView erases this type information, and so should be avoided as much as possible.
I'm a bit uncomfortable about wrapping views in AnyView for the keyboard button factory. I propose to make the relevant view generic, and to leverage the compiler feature of some View to calculate the exact type.
I'm not deep enough in SwiftUI to have an intuition about whether it actually makes a dent in performance, but I think it is more legible like this anyway (just an opinion).
I've made a little proof of concept in this branch. All code still works exactly as before, thanks to the fallback convenience initializer providing a default view factory.
What do you think about it? If you like it, we could proceed to remove all or most AnyView casting.

Thanks for the suggestion, but what happens when you want to return different views for different actions?

I've made a preview sample. Do you mean this:
    @ViewBuilder
    static func systemPreviewButtonBuilder(
        action: KeyboardAction,
        appearance: KeyboardAppearance,
        context: KeyboardContext
    ) -> some View {
            switch action {
                case .backspace:
                    Text("bksp")
                default:
                    SystemKeyboardActionButtonContent(
                        action: action,
                        appearance: appearance,
                        context: context)
            }
    }

Or something else?

Also tried an alternative where I've replaced buttonBuilder with a rowItem lambda so that we can have control over the entire button, not just the content.
https://github.com/digitalheir/KeyboardKit/tree/also-parameterize-row-item-for-system-keyboard

Wow, sorry for not looking before asking 😅 I was building stuff for the Pro library and had my nose deep in code. I think this looks absolutely amazing, can't wait to get rid of the AnyViews!
I will give it a look and merge it asap. Could I just ask that you change the context menu type names from T and V to ItemType and ViewType?

I'll clean up the code today, and see if I can extend apply this technique to the suggestion bar as well!

There was a slight API change. I've made the convenience initializer of SystemKeyboard into a function called standardSystemKeyboard, because I could not figure out how to make the compiler use some View as a generic type parameter. (I think it's not possible.)
I backwards compatibility is very important to you, we could still add a convenience initialiser which uses AnyView as the button type.
public extension SystemKeyboard where RowItem == AnyView {
  public init(
        layout: KeyboardLayout,
        appearance: KeyboardAppearance,
        actionHandler: KeyboardActionHandler,
        context: KeyboardContext,
        inputContext: InputCalloutContext?,
        secondaryInputContext: SecondaryInputCalloutContext?,
        width: CGFloat = KeyboardInputViewController.shared.view.frame.width,
        buttonContent: @escaping ButtonBuilder<ButtonContent>
) -> some View {
    self.init(
            layout: layout,
            appearance: appearance,
            actionHandler: actionHandler,
            context: context,
            inputContext: inputContext,
            secondaryInputContext: secondaryInputContext,
            width: width,
            rowItem: { layout, item, keyboardWidth, inputWidth in
                       AnyView(
                        SystemKeyboardButtonRowItem(
                                content: buttonContent(item.action, appearance, context),
                                item: item,
                                context: context,
                                keyboardWidth: keyboardWidth,
                                inputWidth: inputWidth,
                                appearance: appearance,
                                actionHandler: actionHandler
                        )
                                // Add locale context menu for `nextLocale` action
                                .withLocaleContextMenu(for: item.action == .nextLocale ? context : nil)
                      )
            }
    )
}

But it's so ugly that I left it out. 😅

BTW, sorry that so many whitespace changes crept in. You can ignore whitespaces in Github by appending ?w=1 to the the URL.
https://github.com/KeyboardKit/KeyboardKit/pull/357/files?w=1

Hi again, sorry for this taking so long for me. I'm swamped with other stuff and haven't had time to look at it yet.
I am a bit concerned about the changes to the SystemKeyboard, since they are pretty massive, but I am at the same time very excited about dropping all the usage of AnyView.
I added a few comments, but will take more time to look at this once things calm down a bit.

Hi, no problem. Do you really feel it's massive? I tried to be conservative about it. 😅
I feel like the most problematic change might be the breaking API change, but it could be fixed by adding the initialiser which uses AnyView as the button type. Looking back, I think it may be good to add it so that code continues working, but should should be marked as deprecated (favoring the strongly-typed standardSystemKeyboard function).

@digitalheir Haha, no I think it's perfect in size, but it's just many new considerations. I 100% agree on the deprecated AnyView initializer. It would be great to be able to add this and have the demo app work just like before, with just the deprecation warning.

Added in the default (deprecated) initializer for SystemKeyboard

Hi @digitalheir
I was going to merge this now, after releasing the 5.3 yesterday, but noticed that there are some merge conflicts after my rebasing, sorry about that :/
Would it be possible for you to fix these conflicts in your branch? Sorry for the inconvenience!

I'll take a look

@digitalheir  I merged this into a new branch, but noticed that the demo doesn't build. I'll fix it.

@digitalheir I first had time today so look at this. There are renamings that cause the demo not to build (for instance, buttonBuilder became buttonContent), and when looking at the new code, I'm not really sure how to best solve this.
I also wanted to ask you about the builder signature, where you've added a layout parameter. I'm curious, in what way do you use the layout in your keyboard?

I'll write down some adjustments here, that I do while working on merging your PR:

For the locale context menu, I really don't see the point of having a generic "keyboardContextMenu" since it just wraps anslready generic contextMenu view modifier. I move the keyboard locale stuff back there, but keep the generic fix.

For the locale context menu, I really don't see the point of having a generic "keyboardContextMenu"

It's because I wanted it to switch subtypes in a different way!

For me, demo builds fine for the last commit in my branch (rev 2f3bfb92c1b191c468751f0b655242cb754d2b13). I'll try to see what went wrong...

Did you try pointing the demo to your specific branch? In the merged PR, it believe it was pointing at the master branch. Changing to your branch (or the merge branch in my case), the demo failed.
I think it's just that the buttonBuilder argument was renamed. You can try the last commit in the remove-any-views branch in the public repo, since I've made some changes.

It's because I wanted it to switch subtypes in a different way!

In this case, I don't think it brings much value, since it basically just creates a context menu that iterates over the items.

Ok, I see. This makes a lot more sense. I'll fix my branch now so that the current demo code keeps working.

I'm fairly certain that afa5a2bfe5fd551a0d5d951328c27e75884b3906 fixes the issues on my branch, but I'm having trouble building the demo in XCode, looks like a persistent package caching issue (a lot of 'missing symbol' errors). I'll try and see if clearing cache / restarting everything fixes it.

Yep, restarting did the trick and I can build the demo with my branch. I'll try and merge with your remove-any-views now.


Day170

Resolves #353.

Pull Request Test Coverage Report for Build 652

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 650:
0.0%

Covered Lines:
5

Relevant Lines:
5

💛  - Coveralls

Pull Request Test Coverage Report for Build 661

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 650:
0.0%

Covered Lines:
5

Relevant Lines:
5

💛  - Coveralls


[ALLUXIO-2546] Improve fault tolerance + HDFS API quality of life

https://alluxio.atlassian.net/browse/ALLUXIO-2546

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/13234/
Test PASSed.

Merged build finished. Test PASSed.

@calvinjia comments addressed, PTAL

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/13237/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/13238/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/13239/
Test PASSed.

LGTM


MAX32660: SPI v2

Hello msdk developers,
I have a quick questions about the MAX32660 and the usage of the new SPI v2 SDK. Is the v2 meant to be targetable for any of the MAX326xx chips? Or is it specifically for MAX7xxxx series? I have tried following the MSDK User Guide about updating the build configuration variables and that does not seem to change my code to V2. Is there anything else that must additionally be done to utilize the SPI v2? I have verified my MSDK is updated to the newest release and can see the v2 code files in the PeriphDrivers in the MaximSDK folder location.

Hi @trevorcmit,
SPI v2 isn't fully ported to all the micros yet.  I thought we had added the current status to the developer note already but I guess not.
It needs a somewhat non-trivial implementation file to be created for each micro.  (see spi_ai87_v2.c vs spi_ai87.c)
@sihyung-maxim can you update the UG with the list of SPI v2 supported parts?

Thanks for the response @Jake-Carter. It does seem very non-trivial; is the plan for all the micros to eventually be implemented?

Yes, the plan is to eventually migrate them all.  We're working with a very small team atm so it may take some time.  First we need to catch up on our backlog.  Contributions are very much welcome :)


Failure to start: ValidateConfig: error normalizing listen addrs

Background
Lnd fails to start after 7dfe4018ce2f95c2f6a9ff04e370f668fc755138 has been merged. There are no messages in the log right after the start but after 2 minutes this is returned:
[WRN] LTND: Error validating config: ValidateConfig: error normalizing REST listen addrs: parse address <IP_ADDRESS>:8081 failed: EOF
failed to load config: ValidateConfig: error normalizing REST listen addrs: parse address <IP_ADDRESS>:8081 failed: EOF

Your environment

version of lnd 7dfe4018ce2f95c2f6a9ff04e370f668fc755138
which operating system (uname -a on *Nix) Debian 11.2 amd64 (and also Raspbian arm64)

I bisected lnd and found that this is the bad commit. My config has been working fine for half a year. Maybe there's something expected but missing in my lnd.conf but it shouldn't freeze for 2 minutes and then drop this hard to understand error. On my main node (on Raspbian/amd64) the error is slightly different:
[WRN] LTND: Error validating config: ValidateConfig: error normalizing p2p listen addrs: parse address <IP_ADDRESS>:9735 failed: EOF
failed to load config: ValidateConfig: error normalizing p2p listen addrs: parse address <IP_ADDRESS>:9735 failed: EOF

But otherwise the behavior is the same.

. My config has been working fine for half a year. Maybe there's something expected but missing in my lnd.conf but it shouldn't freeze for 2 minutes and then drop this hard to understand error.

Thanks for reporting this! If you're running the master branch, then you may occasionally run into things like this, with the expectation that by the final release (aided by issue reports like this) these type of issues will have been caught.
That commit is interesting, as it's mainly updating to use new Bitcoin libraries, and this looks to be a network address parsing issue: https://github.com/lightningnetwork/lnd/blob/master/lncfg/address.go#L34-L41
Perhaps it's actually some other commit instead?

Has anything else changed in your set up? Are you running a tor node?

I ran a regular git bisect using master as bad and 72aec1722b4a04b3df144b3363ecb1c59e5078a8 as good (the last commit I built lnd from before). It eventually pointed at 7dfe4018ce2f95c2f6a9ff04e370f668fc755138. It might be a dependency issue as well.
I haven't changed anything really, I even tried it on two different nodes. One runs on RPi 4/Raspbian/arm64 mainnet, the other runs on Debian stable/amd64 testnet. Both show the same issue.
Tor seems to be the problem though! After I disabled it lnd managed to start normally (and after enabling it back the issue is also back). At least on one of the nodes (testnet), that on RPi (mainnet) takes a long while to launch so I prefer to keep it running.

Are you running with the Tor health check active? Without it, if tor restarts then lnd's connection won't be properly re-established. Here's the config section: https://github.com/lightningnetwork/lnd/blob/master/sample-lnd.conf#L1000

The whole heathcheck section is commented. I don't restart Tor though.

Gotcha, I'd recommend turning on the Tor one, as it'll also detect any unexpected disruption of the control port connection and re-create any relevant onion services as well. Or if for w/e reason the tor daemon restarts, then this makes sure lnd either also restarts or can heal things. Otherwise, you'll need to manually restart your lnd node each time something weird happens with the tor daemon.

Okay, but the issue isn't solved. I added the healthcheck lines from the sample, enabled Tor and lnd doesn't start again with the same error. I don't see how this is fixed because if I build lnd from the previous commit it runs just fine on the same node with zero changes to the config or Tor. Also the error is very confusing. Also, I don't think this behavior is good for the default config with these lines commented.

Tor seems to be the problem though! After I disabled it lnd managed to start normally (and after enabling it back the issue is also back). At least on one of the nodes (testnet), that on RPi (mainnet) takes a long while to launch so I prefer to keep it running.

Missed this the first time around: so the issue is you can't start w/ tor at all w/ that set up?

so the issue is you can't start w/ tor at all w/ that set up?

Exactly, if it's enabled lnd freezes for exactly 2 minutes, then errors out with the message I posted in the OP. If Tor is disabled it starts normally.

Still don't see how those commits are related (could be something with bisect not being aware of merge commits the way it's being used?), but I think this section needs to be updated to allow the "all interfaces" IP to pass through to normal TCP resolution: https://github.com/lightningnetwork/lnd/blob/262591c3331c2170697fe89b8c49ab857db0adf7/lncfg/address.go#L235

Well, no idea but 8ee9fc837b1a4a575cb37b413ae11ee97dd397fe is working and this 7dfe4018ce2f95c2f6a9ff04e370f668fc755138 (which is the next one) is not.

Current hunch is that we fixed a bug on the btcd/btcsuite side somewhere, that unmasked a bug here where we attempt to use Tor DNS resolution for the "all interfaces" IP. Likely need to ignore that here: https://github.com/lightningnetwork/lnd/blob/262591c3331c2170697fe89b8c49ab857db0adf7/lncfg/address.go#L235


Multiple port issue in OpenDB

Hi, it seems like OpenDB can't parse multiple ports.
I'm currently using real Foundary PDK and this design contains a lot of multiple ports.
I got an error (DEFPARS-6010) while processing my DEF file.
Are there any options that I can do?

Do you get any other messages?  Can you reduce this to a minimal test case that you can make public (you don't need to make the LEF public, just a reduced DEF since it is a port issue).

Before I share my content, sorry I'm a super newbie in this physical design area.
multiple ports of DEF file looks like this.
- VSS + NET VSS + SPECIAL +DIRECTION INOUT + USE GROUND
 + PORT
     +LAYER D6 ( -188 0 ) (188 376 )
     +FIXED (5600 0) N
 + PORT
     +LAYER D6 ( -188 0 ) (188 376 )
     +FIXED (5600 1490400 ) S
 + PORT
     +LAYER D6 ( -188 0 ) (188 376 )
     +FIXED (9930 0 ) N
 + PORT
     +LAYER D6 ( -188 0 ) (188 376 )
     +FIXED (9930 1490400 ) S
 + PORT
     +LAYER D6 ( -188 0 ) (188 376 )
     +FIXED (14250 0 ) N
 + PORT
     +LAYER D6 ( -188 0 ) (188 376 )
     +FIXED (14250 1490400 ) S

... and so on. I'm not sure why the Innovus tool exports the DEF file like this. Again, sorry for my lack of knowledge about VLSI physical design.


docs: fix broken link to query-keys docs

:tada: This PR is included in version 3.6.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Documentation for using REX-Ray Docker Plug-in

Create documentation for using REX-Ray with Docker's new plug-in architecture

#736


docs: fixes to create valid RST

CC @phlax
Signed-off-by: Kuat Yessenov<EMAIL_ADDRESS>

/assign @markdroth

@snowp Can you help this get merged?

Happy to approve but I think we need @htuch or @markdroth to do the actual merge


Investigate the weird zeroing of the version

I would have thought installing setuptools-scm would've fixed this but I guess not? Note that build isolation is disabled since mypy isn't specified in Black's build-system requirements, maybe that has something to do with it?

And it turns out setuptools-scm breaks when operating within a shallow clone (actions/checkout defaults to --depth 1). Pulling the whole history allows setuptools-scm to generate the right version. Sadly I was not able to find a way to do a treeless or blobless clone but thankfully psf/black is only ~5 MBs.
Pushing those changes and looking at this workflow, everything looks fine now :D


Envoyv2 ingress

convert ingress into gateway + virtual service (no merging yet)
TODO: plumb these two configs into config registry
gateway code to generate http listeners
Signed-off-by: Shriram Rajagopalan<EMAIL_ADDRESS>

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: ayj
Assign the PR to them by writing /assign @ayj in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

cc @ZackButcher ?

@rshriram: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
c4ebd02dfa18ee3b0a80a161e37fed8a4ded68fb
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
c4ebd02dfa18ee3b0a80a161e37fed8a4ded68fb
link
/test istio-unit-tests

prow/istio-pilot-e2e-v1alpha3.sh
c4ebd02dfa18ee3b0a80a161e37fed8a4ded68fb
link
/test istio-pilot-e2e-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
c4ebd02dfa18ee3b0a80a161e37fed8a4ded68fb
link
/test istio-unit-tests

prow/istio-pilot-e2e-v1alpha3.sh
c4ebd02dfa18ee3b0a80a161e37fed8a4ded68fb
link
/test istio-pilot-e2e-v1alpha3

prow/istio-pilot-e2e.sh
c4ebd02dfa18ee3b0a80a161e37fed8a4ded68fb
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
c4ebd02dfa18ee3b0a80a161e37fed8a4ded68fb
link
/test istio-pilot-e2e

prow/istio-pilot-e2e-v1alpha3.sh
0c0145268055d90d63829f2fdd79a1b65189e1bf
link
/test istio-pilot-e2e-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e-v1alpha3.sh
0c0145268055d90d63829f2fdd79a1b65189e1bf
link
/test istio-pilot-e2e-v1alpha3

prow/istio-pilot-e2e.sh
0c0145268055d90d63829f2fdd79a1b65189e1bf
link
/test istio-pilot-e2e

prow/istio-unit-tests.sh
07e01f6f4f5f688dcb703811d3c8bcf6f0d1be18
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
0c0145268055d90d63829f2fdd79a1b65189e1bf
link
/test istio-pilot-e2e

prow/istio-pilot-e2e-v1alpha3.sh
8520c1c1be7a27ee46cf5c2f6e89cecf4ccd203d
link
/test istio-pilot-e2e-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@costinm SNI is there in gateway spec..

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e-v1alpha3.sh
8520c1c1be7a27ee46cf5c2f6e89cecf4ccd203d
link
/test istio-pilot-e2e-v1alpha3

prow/istio-pilot-e2e.sh
8520c1c1be7a27ee46cf5c2f6e89cecf4ccd203d
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
8520c1c1be7a27ee46cf5c2f6e89cecf4ccd203d
link
/test istio-pilot-e2e

prow/istio-pilot-e2e-v1alpha3.sh
77f8ff7b2a89c8fc8ea5b1db80f90245a19cae95
link
/test istio-pilot-e2e-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e-v1alpha3.sh
77f8ff7b2a89c8fc8ea5b1db80f90245a19cae95
link
/test istio-pilot-e2e-v1alpha3

prow/istio-pilot-e2e.sh
77f8ff7b2a89c8fc8ea5b1db80f90245a19cae95
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: costinm
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS [costinm]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e.sh
77f8ff7b2a89c8fc8ea5b1db80f90245a19cae95
link
/test istio-pilot-e2e

prow/istio-pilot-e2e-v1alpha3.sh
397517b65909f644cfa8d8ee9e3bb19ed221eec0
link
/test istio-pilot-e2e-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rshriram: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-e2e-v1alpha3.sh
397517b65909f644cfa8d8ee9e3bb19ed221eec0
link
/test istio-pilot-e2e-v1alpha3

prow/istio-pilot-e2e.sh
397517b65909f644cfa8d8ee9e3bb19ed221eec0
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Force merging due to cla issue.


DNF: best, excludepgks, and skip_if_unavailable

Description
Adds the following attributes for DNF configuration:

best
excludepkgs
skip_if_unavailable

Signed-off-by: Matthew Newell<EMAIL_ADDRESS>See https://dnf.readthedocs.io/en/latest/conf_ref.html for more information on these configuration options.
Please let me know if any changes are needed.
Issues Resolved
Fixes #202
Check List

[x] A summary of changes made is included in the CHANGELOG under ## Unreleased
[x] New functionality includes testing.
[x] New functionality has been documented in the README if applicable.

Released as: 7.4.0