west.yml: Update libmetal and open-amp for v2023.10 release

Update to the last relase of open-amp library
The libmetal is updated to a more recebnt commit to integrate 2 zephyr commits on top of the v2023.10 release.

INFO    - /__w/zephyr/zephyr/twister-out/nrf5340bsim_nrf5340_cpuapp/samples/subsys/logging/multidomain/sample.logging.multidomain.ipc_static_vrings.simu/handler.log
ERROR   - d_00: @00:00:00.000000  (CPU:1): *** Booting Zephyr OS build zephyr-v3.5.0-1999-gc4cb59af6464 ***
d_00: @00:00:00.000000  (CPU:1): IPC-service REMOTE [INST 1] demo started
d_00: @00:00:00.037842  (CPU:0): *** Booting Zephyr OS build zephyr-v3.5.0-1999-gc4cb59af6464 ***
d_00: @00:00:00.037842  (CPU:0): Hello World! nrf5340bsim_nrf5340_cpuapp
d_00: @00:00:00.037842  (CPU:0): [00:00:00.000,000] <inf> app: IPC-service HOST [INST 1] demo started
d_00: @00:00:02.000031  (CPU:0): [00:00:01.000,030] <inf> app: ipc open 0
ipc_service_register_endpoint() failure

INFO    - /__w/zephyr/zephyr/twister-out/nrf5340bsim_nrf5340_cpuapp/samples/subsys/logging/multidomain/sample.logging.multidomain.ipc_static_vrings.simu/handler.log
ERROR   - d_00: @00:00:00.000000  (CPU:1): *** Booting Zephyr OS build zephyr-v3.5.0-1999-gc4cb59af6464 ***
d_00: @00:00:00.000000  (CPU:1): IPC-service REMOTE [INST 1] demo started
d_00: @00:00:00.037842  (CPU:0): *** Booting Zephyr OS build zephyr-v3.5.0-1999-gc4cb59af6464 ***
d_00: @00:00:00.037842  (CPU:0): Hello World! nrf5340bsim_nrf5340_cpuapp
d_00: @00:00:00.037842  (CPU:0): [00:00:00.000,000] <inf> app: IPC-service HOST [INST 1] demo started
d_00: @00:00:02.000031  (CPU:0): [00:00:01.000,030] <inf> app: ipc open 0
ipc_service_register_endpoint() failure

Regression is introduced by https://github.com/OpenAMP/libmetal/pull/244
but something seems not clean in the test as the libmetal_setup is called twice time per CPU.
Here is a trace with some added printk.
@00:00:00.000000  (CPU:1): *** Booting Zephyr OS build zephyr-v3.5.0-1999-gc4cb59af6464 ***
d_00: @00:00:00.000000  (CPU:1): IPC-service REMOTE [INST 1] demo started
d_00: @00:00:00.000000  (CPU:1): ipc1_entry, 59
d_00: @00:00:00.000000  (CPU:1): ipc_service_open_instance, 21
d_00: @00:00:00.000000  (CPU:1): libmetal_setup, 81 start
d_00: @00:00:00.000000  (CPU:1): metal_init: _metal.common.ref_count 0
d_00: @00:00:00.000000  (CPU:1): libmetal_setup, 111 end
d_00: @00:00:00.037842  (CPU:0): *** Booting Zephyr OS build zephyr-v3.5.0-1999-gc4cb59af6464 ***
d_00: @00:00:00.037842  (CPU:0): Hello World! nrf5340bsim_nrf5340_cpuapp
d_00: @00:00:00.037842  (CPU:0): [00:00:00.000,000] <inf> app: IPC-service HOST [INST 1] demo started
d_00: @00:00:00.037842  (CPU:0): ipc1_entry, 58
d_00: @00:00:00.037842  (CPU:0): ipc_service_open_instance, 21
d_00: @00:00:00.037842  (CPU:0): libmetal_setup, 81 start
d_00: @00:00:00.037842  (CPU:0): metal_init: _metal.common.ref_count 0
d_00: @00:00:00.037842  (CPU:0): libmetal_setup, 111 end
d_00: @00:00:00.037842  (CPU:0): [00:00:00.000,000] <err> ipc_service: Register endpoint logging
d_00: @00:00:00.037842  (CPU:0): register_ept : 431 start
d_00: @00:00:00.037842  (CPU:0): register_ept : 452 end
d_00: @00:00:01.000040  (CPU:1): register_ept : 431 start
d_00: @00:00:01.000040  (CPU:1): ipc_rpmsg_register_ept : 55 start
d_00: @00:00:01.000040  (CPU:1): rpmsg_create_ept : 275 start
d_00: @00:00:01.000040  (CPU:1): ipc_service_open_instance, 21
d_00: @00:00:01.000040  (CPU:1): libmetal_setup, 81 start
d_00: @00:00:01.000040  (CPU:1): metal_init: _metal.common.ref_count 1
d_00: @00:00:01.000040  (CPU:1): libmetal_setup, 111 end
d_00: @00:00:01.000040  (CPU:1): register_ept : 431 start
d_00: @00:00:01.000040  (CPU:1): ipc_rpmsg_register_ept : 55 start
d_00: @00:00:01.000040  (CPU:1): rpmsg_create_ept : 275 start
d_00: @00:00:02.000031  (CPU:0): ipc1_entry, 63
d_00: @00:00:02.000031  (CPU:0): ipc_service_open_instance, 21
d_00: @00:00:02.000031  (CPU:0): libmetal_setup, 81 start
d_00: @00:00:02.000031  (CPU:0): metal_init: _metal.common.ref_count 1
d_00: @00:00:02.000031  (CPU:0): libmetal_setup, 111 end
d_00: @00:00:02.000031  (CPU:0): [00:00:01.000,030] <inf> app: ipc open 0
d_00: @00:00:02.000031  (CPU:0): [00:00:01.000,030] <err> ipc_service: Register endpoint ipc1
d_00: @00:00:02.000031  (CPU:0): register_ept : 431 start
d_00: @00:00:02.000031  (CPU:0): register_ept : 452 end
d_00: @00:00:16.564026  (CPU:1): rpmsg_create_ept : 313 end
d_00: @00:00:16.564026  (CPU:1): ipc_rpmsg_register_ept : 69  end ret = -2002
d_00: @00:00:16.564026  (CPU:1): ipc_service_register_endpoint() failure ( -22)
d_00: @00:00:16.564026  (CPU:1): rpmsg_create_ept : 313 end
d_00: @00:00:16.564026  (CPU:1): ipc_rpmsg_register_ept : 69  end ret = -2002
d_00: @00:00:16.564026  (CPU:1): ASSERTION FAIL [err >= 0] @ WEST_TOPDIR/zephyr/subsys/logging/backends/log_multidomain_backend.c:202
d_00: @00:00:16.564026  (CPU:1): @ WEST_TOPDIR/zephyr/lib/os/assert.c:43

@carlocaione, @carlescufi: Do you have any idea what could be causing this behavior? I don't see anything wrong in the sample project that could explain it.

@carlocaione, @carlescufi: Do you have any idea what could be causing this behavior? I don't see anything wrong in the sample project that could explain it.

No, not directly. Perhaps @emob-nordic or @nordic-krch may know something about it.

@carlocaione, @carlescufi: Do you have any idea what could be causing this behavior? I don't see anything wrong in the sample project that could explain it.

No, not directly. Perhaps @emob-nordic or @nordic-krch may know something about it.

Not an expert in openAMP

I think I found the issue:
meta_init is acalled a first time in ipc1_entry()
    ipc_service_open_instance()
        backend->open_instance()  that point to subsys/ipc/ipc_service/backends/ipc_rpmsg_static_vrings.c : open()
            ipc_static_vrings_init()
                libmetal_setup()
                    metal_init()

asecond time in [https://elixir.bootlin.com/zephyr/v3.5.0/source/subsys/logging/backends/log_backend_ipc_service.c#L49)
    init()
       backend_remote->transport_api->init(backend_remote); that point to backend_ipc_service_init()
           ipc_service_open_instance()
              backend->open_instance()  that point to subsys/ipc/ipc_service/backends/ipc_rpmsg_static_vrings.c : open()
                  ipc_static_vrings_init()
                      libmetal_setup()
                         metal_init()

What is strange is that the patch added in lmetal_init prevent to reinitialise if alrteady initialised.
And the issue here is that the test fails if the libmetal is not reinitialised by backend_ipc_service_init
From my analysis the bug is in the zephyr part... What is the objective of the log backend? can we disable it for test?

unfortunately I have not idea what's going on there. There are two different instances of ipc backends one for sample and one for logging so ipc_service_open_instance() is called twice but with different instace.

@arnopo

What is strange is that the patch added in metal_init prevents to reinitialize, if already initialized.
And the issue here seems that the test fails if the libmetal is not reinitialized by backend_ipc_service_init

I'm a bit lost here. The libmetal patch is simply returning 0 if the metal singleton has been already initialised, how that is making the sample failing?
@nordic-krch

unfortunately I have not idea what's going on there. There are two different instances of ipc backends one for sample and one for logging so ipc_service_open_instance() is called twice but with different instace.

Yes, but why that would be a problem?

@arnopo

What is strange is that the patch added in metal_init prevents to reinitialize, if already initialized.
And the issue here seems that the test fails if the libmetal is not reinitialized by backend_ipc_service_init

I'm a bit lost here. The libmetal patch is simply returning 0 if the metal singleton has been already initialised, how that is making the sample failing?

Yes that why I suspect an issue in Zephyr.
whithout the libmetal patch

@arnopo

What is strange is that the patch added in metal_init prevents to reinitialize, if already initialized.
And the issue here seems that the test fails if the libmetal is not reinitialized by backend_ipc_service_init

I'm a bit lost here. The libmetal patch is simply returning 0 if the metal singleton has been already initialised, how that is making the sample failing?

That why i suspect a bug in the zephyr part.
the test is working well if following code is executed twice time:
https://github.com/OpenAMP/libmetal/blob/main/lib/init.c#L17-L26
I perform further tests. seems that the issue is linked to the _metal.common.generic_device_list
Following dirty patch fix the issue:
@@ -11,8 +11,11 @@ int metal_init(const struct metal_init_params *params)
 {
 	int error = 0;

-	if (_metal.common.ref_count++ != 0)
+	if (_metal.common.ref_count++ != 0) {
+		metal_list_init(&_metal.common.generic_device_list);
 		return 0;
+	}

So the issue is around the metal_register_generic_device call

@arnopo
I'm confused how metal_init() is supposed to be accessed concurrently:
// _metal.common.ref_count is 0

if (_metal.common.ref_count++ != 0)
    return 0;
// _metal.common.ref_count is 1

memset(&_metal, 0, sizeof(_metal));
// _metal.common.ref_count is 0 (again)

...
++_metal.common.ref_count;
// _metal.common.ref_count is 1

@arnopo
I'm confused how metal_init() is supposed to be accessed concurrently:
// _metal.common.ref_count is 0

if (_metal.common.ref_count++ != 0)
    return 0;
// _metal.common.ref_count is 1

memset(&_metal, 0, sizeof(_metal));
// _metal.common.ref_count is 0 (again)

...

++_metal.common.ref_count;
// _metal.common.ref_count is 1

It is a standard call counter to avoid reinitializing several times, but also to deinitialize only when all users have released the libmetal.
The confusion can come from the memset(&_metal, 0, sizeof(_metal)) that resets the _metal.common.ref_count, leading to it be increased a  second time.

@carlocaione @arnopo any progress here? do we need to discuss this in a meeting?

@carlocaione @arnopo any progress here? do we need to discuss this in a meeting?

I took a look at the problem but it is very, very puzzling to me why that commit is breaking bsim. I honestly do not exclude that the problem is in bsim itself. I spent a few hours but was unable to find the root cause.
On my side I need a bit more time to debug this because so far nothing stands out as wrong on the Zephyr / libmetal / open-amp side.

@carlocaione @arnopo any progress here? do we need to discuss this in a meeting?

I took a look at the problem but it is very, very puzzling to me why that commit is breaking bsim. I honestly do not exclude that the problem is in bsim itself. I spent a few hours but was unable to find the root cause.
On my side I need a bit more time to debug this because so far nothing stands out as wrong on the Zephyr / libmetal / open-amp side.

I don't have any more bandwidth this week for a deeper investigation ( and probably not the week after also). If it's blocking, we can revert the commit on top of the release and create an issue.
However, there is a risk that the issue will be given low priority.

@carlocaione @carlescufi
I think I found the issue.
The issue is in In ipc_static_vrings.c
We are calling metal_device_open() twice with the same device name SHM_DEVICE_NAME
From libmetal percpective there is only one instance of libmetal with several devices. These devices are identified by a name.
Therefore, since we have two device instances, we should have two different names.
Prior to the fix in libmetal, it appeared to be working because we initilaze libmetal and allocated resources for one instance. then we clean-up libmetal and reinitialized libmetal for the second instance.
I fixed the issue with following quick and dirty patch:
@@ -7,13 +7,18 @@
 #include <zephyr/ipc/ipc_static_vrings.h>
 #include <zephyr/cache.h>

 #define SHM_DEVICE_NAME		"sram0.shm"

+#define SHM_DEVICE_NAME1	"sram1.shm"
+
 #define RPMSG_VQ_0		(0) /* TX virtqueue queue index */
 #define RPMSG_VQ_1		(1) /* RX virtqueue queue index */

+
+static unsigned int static_vring_cnt = 0;
+
 static void ipc_virtio_notify(struct virtqueue *vq)
 {
 	struct ipc_static_vrings *vr;

 	vr = CONTAINER_OF(vq->vq_dev, struct ipc_static_vrings, vdev);
@@ -86,11 +91,12 @@ static int libmetal_setup(struct ipc_static_vrings *vr)
 	err = metal_register_generic_device(&vr->shm_device);
 	if (err != 0) {
 		return err;
 	}

-	err = metal_device_open("generic", SHM_DEVICE_NAME, &device);
+	err = metal_device_open("generic",  static_vring_cnt? SHM_DEVICE_NAME1 : SHM_DEVICE_NAME,
+				&device);
 	if (err != 0) {
 		return err;
 	}

 	vr->shm_io = metal_device_io_region(device, 0);
@@ -164,11 +170,11 @@ int ipc_static_vrings_init(struct ipc_static_vrings *vr, unsigned int role)

 	if (!vr) {
 		return -EINVAL;
 	}

-	vr->shm_device.name = SHM_DEVICE_NAME;
+	vr->shm_device.name = static_vring_cnt? SHM_DEVICE_NAME1 : SHM_DEVICE_NAME;
 	vr->shm_device.num_regions = 1;
 	vr->shm_physmap[0] = vr->shm_addr;

 	metal_io_init(vr->shm_device.regions, (void *) vr->shm_addr,
 		      vr->shm_physmap, vr->shm_size, -1, 0, NULL);
@@ -176,10 +182,12 @@ int ipc_static_vrings_init(struct ipc_static_vrings *vr, unsigned int role)
 	err = libmetal_setup(vr);
 	if (err != 0) {
 		return err;
 	}

+	static_vring_cnt++;
+
 	return vq_setup(vr, role);
 }

I think the  ipc_static_vrings_init  caller should provide the "vr->shm_device.name" based on a specific name (based on DT?)

@carlescufi @carlocaione @arnopo any progress on this? What are the next steps? I'm interested, since this seems to be indirectly blocking the progress of #65908 :)

I'll try to find sometimes to fix this next week. Please note that this should be trivial to fix if you want to progress yourself: it is simply matter of setting vr->shm_device.name to the instance name gathered from DT.

I have something on my side, I'm preparing a PR
--------- subsys/ipc/ipc_service/backends/ipc_rpmsg_static_vrings.c ----------
index dcbff7b360..20e29604fa 100644
@@ -555,6 +555,7 @@ static int open(const struct device *instance)

 	data->vr.notify_cb = virtio_notify_cb;
 	data->vr.priv = (void *) conf;
+	data->vr.shm_device.name = instance->name;

 	err = ipc_static_vrings_init(&data->vr, conf->role);
 	if (err != 0) {

---------------- subsys/ipc/ipc_service/lib/ipc_static_vrings.c ----------------
index be6c365a02..f6ca553fcc 100644
@@ -88,7 +88,7 @@ static int libmetal_setup(struct ipc_static_vrings *vr)
 		return err;
 	}

-	err = metal_device_open("generic", SHM_DEVICE_NAME, &device);
+	err = metal_device_open("generic", vr->shm_device.name, &device);
 	if (err != 0) {
 		return err;
 	}
@@ -166,7 +166,8 @@ int ipc_static_vrings_init(struct ipc_static_vrings *vr, unsigned int role)
 		return -EINVAL;
 	}

-	vr->shm_device.name = SHM_DEVICE_NAME;
+	if ( vr->shm_device.name == NULL)
+		vr->shm_device.name = SHM_DEVICE_NAME;
 	vr->shm_device.num_regions = 1;
 	vr->shm_physmap[0] = vr->shm_addr;

Pull request that should fix the issue: https://github.com/zephyrproject-rtos/zephyr/pull/66525

@carlescufi,  @carlocaione
I rebased on top of the main branch. Now  the CI is Ok

My bad, seems that i made a merge error in https://github.com/zephyrproject-rtos/libmetal/commit/55aac79513bd7a81c71040e7c8986f8d5d3aa496
Files that have been deleted in the libmetal library repo are still present  in the Zephyr libmetal repo.
I will send a new PR to fix that and update this one

update after merge of https://github.com/zephyrproject-rtos/libmetal/pull/24

Waiting for a re-ack from @carlocaione and I'll merge

Waiting for a re-ack from @carlocaione and I'll merge

On second thought, nothing has changed and Carlo may be on holiday so merging.


Profile the AltR Data Model to indicate objects types to be used for graph traversal

Add the profile write-up to https://w3id.org/profile/

@rob-metalinkage: can you add to this?


Sync infranstructure with Dawn

Hi @lisa0314 I rebased the PR to the latest code, and there are below files need to be more flexible for WebNN, so could you please take it from top to end, i will make it from end to top, thanks.

[ ] ObjectType.h
[ ] WireCmd.h
[ ] WireCmd.cpp
[ ] ApiObjects.h
[ ] ApiProcs.cpp
[ ] ClientBase.h
[ ] ClientHandlers.cpp
[ ] ClientPrototypes.inc
[ ] ServerBase.h
[ ] ServerDoers.cpp
[ ] ServerHandlers.cpp
[ ] ServerPrototypes.inc

@fujunwei no problem, I will start to modify the templates according to your list.


upgrade to EO 0.28.2

We should upgrade to 0.28.2 and release a new version. There will be some issues, I suspect.

@Graur please, help with this one

@yegor256 the puzzle #37 is still not solved.

@yegor256 the only puzzle #37 is solved here.


Add Builder::append_data supporting long paths

This method lets one specify a manual header like append, but
incorporates the special handling needed for setting long paths.  If
given path doesn't fit in the header name, it will write a GNU
././@LongLink entry beforehand.
Fixes #108.
Fixes #109.

👍
Looks great, thanks!

Mind testing this out upstream before I publish a new version? If it works I'll publish pronto

I tried it here https://github.com/cuviper/rust-installer/commit/5ad2d00703fd0ad52f9a8bef2adeaa8a6eb42976, and updated into the rust submodule, and it seems to work well.  I don't love a whitelist, but I don't have a better option. 🙂

Awesome! I've published 0.4.13


Fix ClinGen ingest clinical_assertion_observation table fields in parser

clinical_assertion_observation table fields that are missing/wrong:
• clinical_assertion_trait_set_id missing
• Id is different from DSP version

Id field appears to have already been fixed by Kyle previosly.


WiX Builds are broken with recent update to virtual environments

Description
Our WiX setup builds are suddenly broken. The build system is reporting that the following file is missing:

C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\WiX\v3.x\Wix.targets

Recently WiX was updated - is the Visual Studio extension missing/broken now?

https://github.com/actions/virtual-environments/releases

Virtual environments affected

[ ] Ubuntu 18.04
[ ] Ubuntu 20.04
[ ] macOS 10.15
[ ] macOS 11
[ ] Windows Server 2016
[X] Windows Server 2019
[ ] Windows Server 2022

Image version and build link
Agent name: 'Azure Pipelines 2'
Agent machine name: 'WIN-3155C9SFUN7'
Current agent version: '2.195.0'
Operating System
Virtual Environment
Virtual Environment Provisioner
Current image version: '20211122.1'
Is it regression?
no
Expected behavior
I expect the build to complete successfully.
Actual behavior
The build breaks because of recently missing files in the build environment.
Repro steps
Build a WiX Setup project using Visual Studio.

@svnscha , Please take a look at https://github.com/actions/virtual-environments/issues/4419

Can confirm that this is an issue. Switching the agent to windows-2022 or windows-latest is a workaround for this issue.

Closing this issue as a duplicate of #4419

The comments in issue #4419 indeed helped. But the actual fix is changing the *.wixproj file as mentioned by someone in that thread.
<Import Project="$(WixTargetsPath)" Condition=" '$(WixTargetsPath)' != ''" />
  <Import Project="$(MSBuildExtensionsPath32)\Microsoft\WiX\v3.x\Wix.targets" Condition=" '$(WixTargetsPath)' == '' AND Exists('$(MSBuildExtensionsPath32)\Microsoft\WiX\v3.x\Wix.targets') "/>
  <Target Name="EnsureWixToolsetInstalled" Condition=" '$(WixTargetsImported)' != 'true' ">
     <Error Text="The WiX Toolset v3.11 (or newer) build tools must be installed to build this project. To download the WiX Toolset, see http://wixtoolset.org/releases/" />
  </Target>

For reference, the required change and windows-2019 still works with that, too.

@dibir-magomedsaygitov I don't think this is a duplicate of #4419 because adding the Wix tools to the windows-2022 vm image should not have broken the windows-2019 vm image.

@AlmightyFuzz , This https://github.com/actions/virtual-environments/pull/4448 PR affected all Windows Server 2016,2019,2022 images.

@al-cheb Why? The pull request title suggests it should only affect 2022?

@al-cheb Why? The pull request title suggests it should only affect 2022? The #4419 issue also says that it should only affect 2022.
Because of this change our builds on windows-2019 are now failing.

It's a single script for all images - https://github.com/barnson/virtual-environments/blob/main/images/win/scripts/Installers/Install-Wix.ps1

I understand what has changed now, as part of the update to the windows-2022 image you have removed the Wix Toolset Studio 2019 Extension from the windows-2019 image. See win-2019 readme changes
If the script is used by all images then shouldn't a Test-IsWin22 check have been added to that script file rather than removing something that was already installed on the vm image that people were already using and relying on?

Adjusting the files has not helped us at the moment. We are now running the script that @barnson removed again manually in the CI process.
Warning. This could extend every build by at least 20 minutes.
This issue cost us today and the next day's a lot of time 😑
Have to remove the test call because file is not there
Choco-Install -PackageName wixtoolset -ArgumentList "--force"

if (Test-IsWin19)
{
    $extensionUrl = "https://wixtoolset.gallerycdn.vsassets.io/extensions/wixtoolset/wixtoolsetvisualstudio2019extension/<IP_ADDRESS>/1563296438961/Votive2019.vsix"
    $VSver = "2019"
}
else
{
    throw "Invalid version of Visual Studio is found. 2019 are required"
}

$extensionName = "Votive$VSver.vsix"

#Installing VS extension 'Wix Toolset Visual Studio Extension'
Install-VsixExtension -Url $extensionUrl -Name $extensionName -VSversion $VSver

Please don't get me wrong - I completely understand that changes can be necessary f.e. switching from windows-2019 to windows-2022; this is not a problem as it can be handled via regular change & deployment process to make those changes ahead of time.
And while everybody understands that bugs can occur, (intentionally) breaking a released image is a completely different situation;
We ship build pipelines and deployment pipelines to customers - some repos we don't event have access to.
At the moment it's a lot of work to update all of our repositories and their branches and to ship emergency fixes to customers.
Once the current situation is dealt with we'll have to answer why we chose a platform that can break at a moment's notice - and we'll likely need to move away from hosted Azure.


Fastlane Evolution

Fastlane is an amazing tool that helps developers to automate a lot of tasks, saves many hours of developers work and handles boring stuff for them. Thank you Fastlane ❤
That being said, I am very curios about the future of the tool. I've been looking at the opened issues to figure it out, however there is not many enhancements (9 issues) or features (4 issues) labeled issues. There is a few swift issues as well. Based on that it's quite hard to see what is the overall direction / vision the tool is heading to.
So basically, a few questions came to my mind 😊
1. Is there any document / forum / Slack or any other channel of communication that people could follow the next milestone / goal for Fastlane?
It's always great to get a sneak peak of exciting stuff coming and it adds extra dose of excitement for contributors 😅 I really like the way Swift Evolution handles it 👏
2. Swift support
There's Fastlane.swift beta 🎊 already. The next big thing seems to be plugin support:

Currently, Fastlane.swift does not have support for plugins. This is a work in progress and we will continue to update this doc with the current working condition of each feature as we move from beta to general availability.

Is there any plan to migrate some Fastlane tools to Swift in the future? 👍Why Fastlane decided to support Swift in the first place? 🤔
So sorry, if that's a wrong place to ask this type of questions, although didn't know where I could ask them 🙈If so, don't hesitate to close this issue.
Thank you for your time! 🙇🏻♂

there is not many enhancements (9 issues) or features (4 issues) labeled issues.

That is simply because we don't use those labels that much. I mostly label the issues by "code area" (which effectively are the legacy tool names that correspond to folders in the codebase) so it is easier to work on several things in the same area of the code in batches, and so that pattern become visible.
Fastlane is also a pretty mature tool, lots of things are already implemented or available via plugins. There is not much functionality left to add, and if there is, it often is to niche to find the support of someone that could write the Ruby code for it.

Based on that it's quite hard to see what is the overall direction / vision the tool is heading to.

Right now most of the development is either 1. pain/bug/issue or 2. contributor driven.
1 effectively means that contributors like you and me just look at what is present here in the issues. If something is broken, people write about it and we try to help them and they or we implement a solution.
2 is whatever a contributor wants to see in fastlane. I am working on Windows support for example, as I use Window as my main machine every day. Swift was implemented for similar reasons, a contributor just wanted to be able to write their fastlane configuration and lanes in Swift (afaik).
So if you want fastlane to do a specific thing, your best bet is to open an issue, see if there is interest (and agreement that such a feature would be merged), then implement it yourself or get someone else to implement it.

Thank you @janpio 🙇 Your answer gave me a context I was missing.

There is not much functionality left to add, and if there is, it often is to niche to find the support of someone that could write the Ruby code for it.

So you think there's no many Ruby developers left 🤔 Back in the day Ruby was a first choice for automation tools, i.e. CocoaPods to not looking far away. Wondering what would've been chosen today for Fastlane? Another question is if Fastlane might port Ruby code to other language in the future? 🤔 (even though right now it might seem like a crazy idea).

Nah, you misunderstood me here. Not ruby is to niche, but the problems that are not solved by fastlane.
If the usecase for a feature is literally just 1 person, it will be hard to find some contributor (like you or me) that wants to implement it and then commit to supporting the feature for the next weeks, months or years.
When there are big changes like with force 2FA or a new API from Apple, we usually find enough contributors (like you and me!) that want and can work on it.

@mollyIV Thanks for bring this up! This is definitely a topic that isn't very transparent to the fastlane community 😊 @janpio's answers were pretty on point there. fastlane is a pretty mature tool at this point. It interacts most of the Apple APIs that it currently can as well as most of the Google Play APIs that it currently can. We also offer the third party plugin system for anybody to add features they want to add to fastlane without going through the fastlane PR process which also helps limit the amount of knowledge that maintainers need to know 😇
That's being said... I think each of the individual contributors have a personal list of things they would like to see added to or changed in fastlane. Those usually come through as WIP PRs that prototype what we want changed. @janpio's Window's support (as he mentioned) is one of the bigger ones and we greatly appreciate it 😍 I have goals to clean up action/plugin configuration option interface and add some improved usage around lane options. One of our other combined team goals is to eventually switch over to the new App Store Connect API (but it currently does not offer everything that we need). fastlane at this point is pretty big and each of the core contributors has the areas of fastlane in which they are experts in so that is where you will tend to see new features appear more often.

Swift support

Swift support was added as something that was in beta as it was more/less a prototype to see the possibilities of adding new language interfaces on top of fastlane as well as seeing the demand that Fastlane.swift would generate without rewriting all of fastlane in a different language.
Fastlane.swift was a success from the standpoint that its a relatively small amount of code that generates the Swift interface and also there are relatively few issues that arise from it. It also was developed in a way where it fully leverages the Ruby implementation of fastlane and actually ended up enhancing the Ruby action option interface by cleaning up and explicitly specifying the different types of options that can be passed into actions. After this thought it was kind of left alone as it was working.
The hard part about making plugins work with Fastlane.swift is that the plugins Swift interface would need to be generated a little more dynamically than the actions are (since the actions get generated during a fastlane release. I would totally love adding plugin support but it sadly comes lower in priority to most of the other issues that come in. We would be happy to see this being worked on by someone 😊
There is no vision right now to migrate any of the codebase of fastlane into Swift. Ruby is a pretty good fit for fastlane right now as it runs nicely on any platform, its a scripting language which fits the Fastfile, and also because of the enormous amount of gems that are available for us to make use of for actions and plugins. Not to say that some of these don't exist on Swift for us to use but at the time when fastlane was first being developed, it was much easier to get running on Ruby.
Rewriting the whole fastlane codebase in Swift (or a different language) wouldn't provide much benefit for the amount of work it would take to replicate everything. The reason was Swift chosen as a second interface for fastlane was that is would be a natural language for new iOS developers to write code in and that it was fairly straight forward for our team to implement as an interface because of iOS experience with Swift 🙃
TL;DR
Sooooooo, this seemed to turn more into a blog post than a response to an issue 🙈

There is currently no one, single vision for fastlane's future in terms of new features but we each have individual goals for fastlane within our areas of expertise.
Swift was the more natural language for new iOS developers who want to adopt fastlane


Anonymous PL/SQL Block execution as script results in error

Hello,
With version 3.8.0 64 bit Linux running the following statement as script
DECLARE
    V_COUNT NUMBER := 0;
BEGIN
    /* TIBCOBPMAPP */
    SELECT COUNT(1) INTO V_COUNT FROM ALL_TABLES WHERE UPPER(OWNER) = 'SOME_TABLE';

    IF V_COUNT >= 1 THEN
      EXECUTE IMMEDIATE 'CREATE OR REPLACE SYNONYM SOME_TABLE_SYN FOR SOME_TABLE';
    END IF;
END;

results in the following error:

SQL Error [6550] [65000]: ORA-06550: line 10, column 11:
PLS-00103: Encountered the symbol "end-of-file" when expecting one of the following:
( begin case declare end exception exit for goto if loop mod
null pragma raise return select update while with

 << continue close current delete fetch lock
insert open rollback savepoint set sql execute commit forall
merge pipe purge
SQL Error [6550] [65000]: ORA-06550: line 10, column 11:
PLS-00103: Encountered the symbol "end-of-file" when expecting one of the following:
( begin case declare end exception exit for goto if loop mod
null pragma raise return select update while with

 << continue close current delete fetch lock
insert open rollback savepoint set sql execute commit forall
merge pipe purge
ORA-06550: line 10, column 11:
PLS-00103: Encountered the symbol "end-of-file" when expecting one of the following:
( begin case declare end exception exit for goto if loop mod
null pragma raise return select update while with

 << continue close current delete fetch lock
insert open rollback savepoint set sql execute commit forall
merge pipe purge
ORA-06550: line 10, column 11:
PLS-00103: Encountered the symbol "end-of-file" when expecting one of the following:
( begin case declare end exception exit for goto if loop mod
null pragma raise return select update while with

 << continue close current delete fetch lock
insert open rollback savepoint set sql execute commit forall
merge pipe purge

Selecting the whole statement and running it as a single statement works.
Thanks for a great SQL Client :-)

UPPER(OWNER) => UPPER(TABLE_NAME)

PL/SQL processing will be improved in the next version.


Embedded scanner

@dbmeneses Is there a way to grab the scanner zip at build time instead of storing it in the Git repo?

I'll look into it.
On 20 September 2016 at 18:04, Julien HENRY<EMAIL_ADDRESS>wrote:

@dbmeneses https://github.com/dbmeneses Is there a way to grab the
scanner zip at build time instead of storing it in the Git repo?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/SonarSource-VisualStudio/sonar-scanner-msbuild/pull/245#issuecomment-248348291,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AL6fgfqJwJGIY01NvSMYVdAXJGIMxwt3ks5qsAP7gaJpZM4KBn7c
.

--
*Duarte MENESES | *SonarSource
http://sonarsource.com


[backport] Add Sequential class for easy model definition of a single-stream computational graph

Backport of #2918

LGTM. We need to backport #4605 too.


WSL.exe in System32 Folder Always Launches New Window

Version
Microsoft Windows [Version 10.0.19044.2604]
WSL Version

[X] WSL 2
[ ] WSL 1

Kernel Version
<IP_ADDRESS>-microsoft-standard-WSL2
Distro Version
Ubuntu-22.04 (but also Ubuntu-20.04 and others)
Other Software
Docker Desktop 4.16.3 (also 4.15.0)
Repro Steps
Update wsl to latest version or latest pre-release:
wsl --update --web-download or wsl --update --web-download --pre-release
Launch wsl.exe from cmd without admin rights

%userprofile%\AppData\Local\Microsoft\WindowsApps\wsl.exe

In the same window the wsl distro starts running

C:\Windows\System32\wsl.exe

A new window opens with wsl distro running

Launch wsl.exe from cmd with admin rights

C:\Windows\System32\wsl.exe

In the same window the wsl distro starts running

%userprofile%\AppData\Local\Microsoft\WindowsApps\wsl.exe

In the same window the wsl distro starts running

Docker Desktop
This seems to affect Docker Desktop, as it cannot start up in non-admin rights. With admin rights it works.
My guess after chasing this problem a few days since it stopped working:
Docker Desktop cannot start because it's using wsl.exe from system32 (found out via taskmanager) and is obviously not expecting to have this new-window indirection.
Expected Behavior
C:\Windows\System32\wsl.exe should not open a new window - regardless if the lauching console has or hasn't admin rights.
Actual Behavior
C:\Windows\System32\wsl.exe opens a new window if the launching console has no admin rights.
C:\Windows\System32\wsl.exe does not open a new window if the launching console has admin rights.
Diagnostic Logs
No response

Fortunately, this issue has nothing to do with WSL.
The issue was a user rights management software (AppSense in my case) used by our corporation, which injects itself the processes. This caused wsl.exe in system32 to behave differently.
Once wsl was excluded, everything went back to normal.


Python Go-To-Definition Error: Error: Unable to resolve nonexistent file '\*** unresolved ***'

Does this issue occur when all extensions are disabled?: Yes/No
Yes(With Only "Python" Extension Enabled)

Version: 1.66.2 (user setup)
Commit: dfd34e8260c270da74b5c2d86d61aee4b6d56977
Date: 2022-04-11T07:46:01.075Z
Electron: 17.2.0
Chromium: 98.0.4758.109
Node.js: 16.13.0
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.19043
Steps to Reproduce:

Install package lief with pip install lief, or from source https://github.com/lief-project/LIEF
create empty .py file, and typing:

import lief
print(lief)

Press F12 on lief, right bottom error window appears, and Log(Window) Output:

[2022-04-21 09:08:54.223] [renderer3] [error] Error: Unable to read file '\*** unresolved ***' (Error: Unable to resolve nonexistent file '\*** unresolved ***')
    at p.doReadFileStream (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:33842)
    at async doRead (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2666:110433)
    at async d.readStream (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2666:110108)
    at async ji.resolveFromFile (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2661:66651)
    at async u.doResolve (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2666:104975)
    at async ya.doResolveAsText (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2132:89317)
    at async g.setInput (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2636:111961)
    at async i.doSetInput (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2636:26823)
    at async i.doOpenEditor (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2636:25196)
    at async i.openEditor (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2636:24822)
    at async vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2842:17260
    at async C.doOpenCodeEditor (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:1346:31316)
    at async Cr._openReference (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:790:97714)
    at async Cr._onResult (vscode-file://vscode-app/c:/Users/Admin/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:790:97437)

PS: tested on pycharm-community edition with same steps, and successfully navigated to definition of lief.

I believe this is because lief is using pyd files. This discussion has more information:
https://github.com/microsoft/pylance-release/issues/166
Some features don't work with pyd files. What does Pycharm show for goto definition as there is no .py file to go to?

https://github.com/microsoft/pyright/blob/main/packages/pyright-internal/src/analyzer/binder.ts#L2455
https://github.com/microsoft/pyright/blob/main/packages/pyright-internal/src/analyzer/binder.ts#L2518

@debonte, do you remember what you linked to for this? line numbers probably don't match anymore

Oh, I think it was the two uses of *** unresolved *** in binder.ts.

I don't think so? It's working as expected:

Lief is a pyd file.


Clean up main Client class

Currently, the main Client class contains all logic related to building and processing the responses from the API. The size of the class is acceptable in the current implementation, but could easily grow as the number of API methods increases.
Instead, use a request/response factory model similar to Omnipay:
https://github.com/thephpleague/omnipay-stripe
https://github.com/thephpleague/omnipay-common

Fixed in version 0.2.0.


Add a way to disable hash prefix when using consistent_snapshot

Fixes issue #: https://github.com/theupdateframework/tuf/issues/1080
Description of the changes being introduced by the pull request:
Currently, if the repository is consistent_snapshot,
Updater will prefix the target filename with the hash
when constructing the download URL.
For some adopters of TUF (like Warehouse) this is not wanted
(warehouse target file paths are "consistent",
even if the filenames are not).
For example, Warehouse doesn't follow what tuf
(the reference implementation and specification) advice for naming
consistent filenames, which is to prefix the filename with the hash
of the files contents.
However, the target filenames it does use are consistent,
only the hash is part of the target's file path
not the target's file name.
Please verify and check that the pull request fulfills the following
requirements:

[x] The code follows the Code Style Guidelines
[x] Tests have been added for the bug fix or new feature
[ ] Docs have been added for the bug fix or new feature

I decided to add use_existing_hash_prefixes argument to the updater.download_target() function instead as an argument to the __init__ function for the Updater because updater.download_target() is really the only place where this argument is needed.
Plus, this made it easier for testing.

I updated my pr adding the necessary explanations in the test and changing the variable name.

Thanks for the patches @MVrachev and for the review @jku

Thanks for the patches @MVrachev and for the review @jku


code-server exits when refreshed during the login process

code-server version: 2.1692-vsc1.39.2
OS Version:  Ubuntu 18.04.3 LTS by BandwagonHost

Description

When the browse was loading the website, I try to stop the browser and reload the website because of slowly loading ,  The browser displays CONNECTION_REFUSED. And an the sametime  code-server on the server will be stoped liked press CTRL + C.
The server will show error informations while stoped reload the  code-server if the code server has been loaded

Steps to Reproduce

open code-server in new tab or reload while code-server already load

stoped the browser load

Me too, tried Ubuntu 16.04 LTS & 18.04 LTS. Nether works.

Can't replicate issue, are you sure nothing else runs on port 443?

Can't replicate issue, are you sure nothing else runs on port 443?

Nothing  else runs on port 443.
After OS reboot

After Code-server start

Made the issue clear now, @code-asher this might be a show breaking bug.

I've seen this before but have been having a hard time replicating it.

Related: https://github.com/cdr/code-server/issues/1009

I'm having a similar issue. It closes just as I am entering my login password. The only parameters I use are the --port, --cert and --cert-key.
This is on Ubuntu 18.04.3 LTS as well.

We are also experiencing this problem. Only during startup.   No obvious errors, either on command output, or in the "remoteagent.log" log file, with --verbose enabled.

See #1009


[0.17] Constant reconciliation for KafkaSource when it is created as v1beta1

Describe the bug
Constant reconciliation for KafkaSource when it is created as v1beta1
 {"level":"info","ts":"2020-10-27T12:13:04.174Z","logger":"kafka-controller.knative.dev-eventing-contrib-kafka-source-pkg-reconciler-source.Reconciler","caller":"controller/controller.go:520","msg":"Reconcile succeeded. Time taken: 22.018803ms","knative.dev/traceid":"f438352f-8da9-40b4-9e32-524d9d5f2c40","knative.dev/key":"default/kafka-source"}

Expected behavior
No constant reconciliation
To Reproduce
Create Kafka topic
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaTopic
metadata:
  name: my-topic
  labels:
    strimzi.io/cluster: my-cluster
spec:
  partitions: 100
  replicas: 1

Create ksvc
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: event-display
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/target: "5"
    spec:
      containers:
        - image: quay.io/openshift-knative/knative-eventing-sources-event-display

Create KafkaSource
apiVersion: sources.knative.dev/v1beta1
kind: KafkaSource
metadata:
  name: kafka-source
spec:
  bootstrapServers:
   - my-cluster-kafka-bootstrap.kafka:9092
  topics:
   - my-topic
  sink:
    ref:
      apiVersion: serving.knative.dev/v1
      kind: Service
      name: event-display

Knative release version
Tried with eventing v0.17.7 and eventing-contrib v0.17.4
Additional context
Debugging session with Ville: https://knative.slack.com/archives/CQBKVH4QY/p1603801829059700

/close
Fixed by https://github.com/knative/eventing-contrib/pull/1650


Modular Onboarding for Agentless ACA/ACI

Azure ACA/ACI agentless scanning requires no resources to be created in the customer's infra.
This onboarding just loops the subscriptions (coming from management group ids, root management group or single subscription id provided by the client) and creates the assignments in cloud auth between account and feature.
I've also decided to keep in source control the text of what should be presented in the UI, on the files ui_*.txt.

Hey Mig, I don't think we need a submodule here. TF provisioning should take care of creating cloud side resources, and as a consequence any backend representations of those created resources. We shouldn't be doing feature enablement here without any corresponding provisioning

You are absolutely right @nkraemer-sysdig . I've updated the PR including ONLY the UI terraform that does feature enablement.
Does this look right to you @nkraemer-sysdig ?


Graphic for 1.10 release

Related Issue
No response
Description
Key highlights on 1.10 releases:

FlowForge now available on AWS Marketplace
Device management improvements
Import of environment variable templates

@joepavitt are you able to help since Yndira is out.

PR #1011

Can we create a back up graphic in case we don't have AWS Marketplace approval in time. I would suggest something like:
'FlowForge 1.10 makes it easier to import Node-RED environment variables.'

Device management improvements

It's a bit vague and underwhelming, is there a better spin we can put on what was achieved?

Improved Device Editor Tunnel Reliability
Team Library Access on Devices

Provisional tile, will update accordingly as required

It looks like we need to remove AWS Marketplace since it is not available.
The challenge with using Improve Device Editor Tunnel Reliability and Team Library access on devices is that it requires you to actually know the detailed features of FlowForge. Most people reading this graphic won't so I suggest we focus on the high level features.

Most people reading this graphic won't so I suggest we focus on the high level features.

Understood, as requested:


Work with JetBrains to get JetGradlePlugin compatible with Isolated Projects

[x] Report existing issues to the shared Slack channel (ping Nikita and Sergei Vorobyov)
[ ] IP safe access to project buildscript classpath #27195

Waiting on IDEA-341827 which in turn requires #22514

The upcoming IntelliJ IDEA release is expected to be free of incompatibilities.
We also track the existing violations in the sync smoke tests.


[Bug] - context.globals.theme is empty string when default

Describe the bug
When using withThemeFromJSXProvider, attempting to grab the current theme from the context using the story render function, the context.globals.theme is an empty string when set to the default theme.
export const Primary: Story = {
  render: (args, context) => {
    console.log("context", context)
    return <Box>some text</Box>;
  }
}

export const decorators: Decorator[] = [
  withThemeFromJSXProvider({
    themes,
    Provider: ThemeProvider,
    defaultTheme: 'v3',
  }),
];

{
    "backgrounds": null,
    "measureEnabled": false,
    "outline": false,
    "theme": ""
}

Expected behavior
context.globals.theme should always represent the current theme name regardless of whether it is the default or not.

@jeremydrichardson! Oh no, that's odd.
Can you provide me with a reproduction repo for this problem?

I can't with the project I'm working on, it's an internal project.
I'll see if I can spin up a demo version that demonstrates the issue.

For anyone else who find this, my workaround was to set a global for theme and set it to what I wanted the default theme to be. I used a variable so I could make sure the withThemeFromJSXProvider was the same as the global I set.
const defaultTheme = 'mytheme';

export const globals = {
  themes,
  theme: defaultTheme,
};

export const decorators: Decorator[] = [
  withThemeFromJSXProvider({
    themes,
    Provider: ThemeProvider,
    defaultTheme: defaultTheme,
  }),
];

Hey @jeremydrichardson 👋
Were you ever able to reproduce this in another project? I still haven't come across this.
Also, what version of the addon are you using?


[SPARK-23559][SS] Add epoch ID to DataWriterFactory.

What changes were proposed in this pull request?
Add an epoch ID argument to DataWriterFactory for use in streaming. As a side effect of passing in this value, DataWriter will now have a consistent lifecycle; commit() or abort() ends the lifecycle of a DataWriter instance in any execution mode.
I considered making a separate streaming interface and adding the epoch ID only to that one, but I think it requires a lot of extra work for no real gain. I think it makes sense to define epoch 0 as the one and only epoch of a non-streaming query.
How was this patch tested?
existing unit tests

@tdas @rdblue @cloud-fan
I haven't forgotten that we need a design doc before finalization; SPARK-23556 tracks that.

Can one of the admins verify this patch?

Test build #87862 has started for PR 20710 at commit 5bbd497.

Test build #87862 has finished for PR 20710 at commit 5bbd497.

This patch fails to generate documentation.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87862/
Test FAILed.

Test build #87909 has started for PR 20710 at commit cb6b2cf.

Test build #87909 has finished for PR 20710 at commit cb6b2cf.

This patch fails to generate documentation.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87909/
Test FAILed.

Test build #87912 has started for PR 20710 at commit 544eb1b.

Test build #87915 has started for PR 20710 at commit 79495b1.

Test build #87917 has started for PR 20710 at commit 9fb74e2.

Test build #87918 has started for PR 20710 at commit 215c225.

Test build #87912 has finished for PR 20710 at commit 544eb1b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87912/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87915/
Test PASSed.

Merged build finished. Test PASSed.

Test build #87918 has finished for PR 20710 at commit 215c225.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87918/
Test PASSed.

Test build #87917 has finished for PR 20710 at commit 9fb74e2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87917/
Test PASSed.

Merged build finished. Test PASSed.

LGTM

@rdblue @jose-torres arrgh... i didnt notice that you guys were still commenting before i merged it.
feel free to continue discussion and if any change is needed we will deal with this accordingly. sorry about it!

@tdas, thanks for letting us know. I'm really wondering if we should be using the same interfaces between batch and streaming. The epoch id strikes me as strange for data sources that won't support streaming. What do you think?

My primary concern with splitting the interfaces is that it makes it easy for Spark changes to accidentally do the wrong thing. Callers of DataWriterFactory.createDataWriter() won't necessarily notice that there's a StreamingDataWriterFactory which needs to be supported; they'd likely just end up writing code which will break with an opaque internal error in a streaming query.

@jose-torres, can you explain that more for me? Why would callers only use one interface but not the other? Wouldn't streaming use one and batch the other? Why would batch need to know about streaming and vice versa? The simplification is for implementers. It seems odd for implementations to deal with parameters that are for something else (e.g., don't worry about this for batch).

There isn't a currently a distinction between streaming and batch in the places where this interface is called, except in the experimental continuous processing streaming mode. The streaming engine executes a sequence of WriteToDataSourceV2Exec plans, in the same way that a sequence of unrelated batch queries would. Only the DataSourceWriter implementation knows that it needs to do something special because it's in a streaming query.

Could the non-continuous streaming mode just use the batch interface, since each write is basically separate?

I'm not certain I understand the question.
From the perspective of query plan execution, the non-continuous streaming mode does just use the batch interface. The motivation of adding epoch ID to DataWriterFactory is to allow it to continue using the batch interface, rather than adding a StreamingDataWriterFactory which it must use instead of the batch interface.
From the perspective of the writer, the batch interface is not sufficient. Epoch ID is relevant to the data writer for the same reason that partition ID is; they need to identify to the remote sink which segment of the overall data they're responsible for.

My question is: why can't we use a batch interface for batch and micro-batch (which behaves like batch) and add a separate streaming interface for continuous streaming? I see no reason to have epoch ID for batch, and it seems janky to add options that implementers should know to ignore.

Spark may need to distinguish between different segments of the data when talking to the remote sink.

For which case, continuous or micro-batch?

For either case. Any streaming execution model has to know that epoch 1 and epoch 2 are part of the same query, for the same reasons it has to know that task attempt 0 and task attempt 1 are iterations of the same task.

Partitions are a better example than task attempts, but it's still roughly the same idea. Data source writers need to be able to reason about what progress they've made, which is impossible in the streaming case if each epoch is its own disconnected query.

Data source writers need to be able to reason about what progress they've made, which is impossible in the streaming case if each epoch is its own disconnected query.

I don't think the writers necessarily need to reason about progress. Are you saying that there are guarantees the writers need to make, like ordering how data appears?
I'm thinking of an implementation that creates a file for each task commit and the driver's commit operation makes those available. That doesn't require any progress tracking on tasks.
As far as a writer knowing that different epochs are part of the same query: why? Is there something the writer needs to do? If so, then I think that is more of an argument for a separate streaming interface, or else batch implementations that ignore the epoch might do the wrong thing.

As you say, there's no strict semantic need to have createDataWriter() take arguments. We could simply have each DataWriter identify itself by a random UUID, and require upstream components to keep track of which UUIDs map to which of the writers they care about. But the current API design is to enable each data writer to identify its logical place in the query, and epoch ID is an important part of that. (I expect it would be infeasible to migrate existing sources to an API which didn't provide things like partition ID or attempt number.)
StreamWriter is the separate streaming interface, and DataWriterFactory implementations in streaming queries will always come from a StreamWriter.

Epoch ID is not a valid part of the logical place in a query for batch. I think we should separate batch and streaming, as they are already coming from different interfaces. There's no need to pass useless information to a batch writer or committer.
Implementations can choose to use the same logic if they want, but we should keep the API focused on what is needed, to keep it reasonable for implementers.

I still maintain that it's sensible to say a batch query is a query that has only one epoch, and that the ship has sailed on passing useless information. But I'm bikeshedding here. Created https://github.com/apache/spark/pull/20752 to split the interfaces.


fix fee sharing percentage

fixes DASH-476

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#5525  👈 (View in Graphite)
#5524
#5523
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


Many non-optional optional arguments

I'm trying to use this right now and I'm finding that a lot of "optional" args are actually required with defaults that are in the CLI code. For instance the dos_options in the bandstructure/dos plotting is by default None in the code. This breaks and I have to set it to a series of values I figured out by scouring the CLI code. The methods should initialize to intelligent defaults if not specified.

Thank you for using Sumo! That doesn't sound ideal, we'll take a look at this.

Hi @shyamd
This is a bug. There shouldn't be any optional arguments that require modification for the code to run without an error. In general, I've tried to make the default value for the optional arguments the same as the default in the command line scripts.
The dos_options is a special case, in that it is the only optional argument that can accept a dictionary containing sub-options. You shouldn't have to look through the CLI code to see what the valid keys for this dictionary are. There are detailed in the docstring for the function, namely:
            dos_options (:obj:`dict`, optional): The options for density of
                states plotting. This should be formatted as a :obj:`dict`
                containing any of the following keys:
                    "yscale" (:obj:`float`)
                        Scaling factor for the y-axis.
                    "xmin" (:obj:`float`)
                        The minimum energy to mask the energy and density of
                        states data (reduces plotting load).
                    "xmax" (:obj:`float`)
                        The maximum energy to mask the energy and density of
                        states data (reduces plotting load).

                    ... etc

I've fixed the bug you mentioned (namely that supplying None for dos_options doesn't work). The following code snippet should now successfully produce a combined band structure and dos plot:
from pymatgen.io.vasp.outputs import BSVasprun
from sumo.electronic_structure.dos import load_dos
from sumo.plotting.dos_plotter import SDOSPlotter
from sumo.plotting.bs_plotter import SBSPlotter

vr = BSVasprun('vasprun.xml.gz')
bs = vr.get_band_structure(line_mode=True)

dos, pdos = load_dos('vasprun.xml.gz')
dos_plotter = SDOSPlotter(dos, pdos)
bs_plotter = SBSPlotter(bs)

plt = bs_plotter.get_plot(dos_plotter=dos_plotter)
plt.savefig('band_dos.pdf', bbox_inches='tight')

You mentioned that you had found several cases where the default optional arguments didn't work. Would you be able to tell me the others so I can fix that behaviour.
Cheers!

That actually fixed all my other optionals as well. I think this one looks good.


Latest agent version does not mitigate Log4j vulnerability

According to Datadog's official announcement, released versions 6.32.2 and 7.32.2 of the Agent that initializes the JVM for our JMX monitoring client with the following flag: -Dlog4j2.formatMsgNoLookups=True
A new CVE has been released since: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-45046
Note that previous mitigations involving configuration such as to set the system property log4j2.noFormatMsgLookup to true do NOT mitigate this specific vulnerability. Log4j 2.16.0 fixes this issue by removing support for message lookup patterns and disabling JNDI functionality by default. This issue can be mitigated in prior releases (<2.16.0) by removing the JndiLookup class from the classpath (example: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class).

Yes, can this to be addressed asap? Thanks!

Hi @jenle-chime / @jduan-highnote!
We are trying to coalesce/unify all the information together about this together so all official communication updates around log4j will be available at https://www.datadoghq.com/log4j-vulnerability/ as they become available.
I'll leave the issue open for visibility for now but keep in mind that most up-to-date info will be provided in the mentioned link.

@sgnn7 I am trying to report an issue with Datadog's latest update. Is there another place you would like us to report log4j issues?

Datadog agent is downloading jmxfetch.jar file using jmxfetch.rb script with versions specified in release.json. It seems like jmxfetch version uses log4j version v2.12.2 even in the latest release.  IMO, until jmxfetch is updated with log4j v2.16 and Datadog agent uses it, this issue won't be addressed.
Is there any ETA on getting this fixed in jmxfetch and datadog-agent image with jmx?

They won't upgrade past 2.12.2 because anything past that doesn't support Java 7.
Where does the log4j jar get downloaded to? Can we just modify the jmxfetch / install script to run
zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class

It seems like jmxfetch version uses log4j version v2.12.2 even in the latest release. IMO, until jmxfetch is updated with log4j v2.16 and Datadog agent uses it, this issue won't be addressed.

@knowshan / @sudosoul I can't comment on the timelines but per the upstream page at https://logging.apache.org/log4j/2.x/, 2.12.2 is not vulnerable:

The Log4j team has been made aware of a security vulnerability, CVE-2021-45046, that has been addressed in Log4j 2.12.2 for Java 7 and 2.16.0 for Java 8 and up.
...
In version 2.12.2 Log4j disables access to JNDI by default. Usage of JNDI in configuration now need to be enabled explicitly. Calls to the JndiLookup will now return a constant string. Also, Log4j now limits the protocols by default to only java. The message lookups feature has been completely removed.

@sgnn7  You're  right, thank you - spoke too soon.
This issue can be closed then.
@jenle-chime

The question is how datadog pull plugin. If I download plugin 2 days ago (after 7.32.2 was release) and jmxfetch was bump 11 hours ago, should I reload datadaog or need any other action to refresh jmx plugin version?

I check that and agent  7.32.2 was installed on 11.Dec using 0.44.4 version of JMXFetch and. changes for using Log4j 2.12.2 are made in 0.44.5 https://github.com/DataDog/jmxfetch/releases/tag/0.44.5
Anyone can confirm that Datadog-agent still has Log4j issue ??

I see version used is 6.26.0 for our service. How to upgrade the version here and as @szibis  said 2.12.2 is still not added to 6.32.2 . So how to proceed and looks like adding only -Dlog4j2.formatMsgNoLookups=True will not work completely as per below link - https://github.com/advisories/GHSA-7rjr-3q55-vv33
Any suggestions here?

We are waiting for 7.32.3 and 6.32.3 https://github.com/DataDog/datadog-agent/pull/10240

We are currently working on releasing versions 6.32.3 and 7.32.3 of the Agent that include a patched version of the log4j library. We'll update this issue when the release is available.

https://github.com/DataDog/datadog-agent/releases/tag/7.32.3
@jenle-chime

We have released 7.32.3 and 6.32.3 (https://github.com/DataDog/datadog-agent/releases/tag/7.32.3). These Agent versions include log4j 2.12.2, which fixes both CVE-2021-44228 and CVE-2021-45046, as documented on the log4j security page.
Please refer to https://www.datadoghq.com/log4j-vulnerability/ for further information, thanks!

Will the chef/kitchen recipe be updated with this new package?
https://supermarket.chef.io/cookbooks/datadog#changelog

@nwaldropEXR To be clear, our official chef cookbook is not affected by these vulnerabilities. The agent versions that it installs can be affected depending on the agent versions you've configured the cookbook to install. You can pin the version of the Agent that the cookbook installs with the ["datadog"]["agent_version"] attribute. You'll find more details in the cookbook's README
Please reach out to our support team if you have further questions about the chef cookbook, thanks!

As mentioned in https://github.com/DataDog/datadog-agent/issues/10301 there is yet another CVE which is fixed in 2.12.3.
See: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-45105

Apache Log4j2 versions 2.0-alpha1 through 2.16.0 (excluding 2.12.3) did not protect from uncontrolled recursion from self-referential lookups. This allows an attacker with control over Thread Context Map data to cause a denial of service when a crafted string is interpreted. This issue was fixed in Log4j 2.17.0 and 2.12.3.

Could you reopen any of these issues to track it until finished?

Hey @JorgenG,
https://www.datadoghq.com/log4j-vulnerability/ has already been updated with the infromation about that CVE:
Versions 6.32.3 & 7.32.3:
...
Our analysis of CVE-2021-45105 indicates that the JMXfetch client is not affected. The implementation
has a hardcoded PatternLayout and does not use any context lookup. As a result, it does not rely on any
configuration that an attacker can control to execute the exploit.

With that said, we have just released agent versions 7.32.4 and 6.32.4 that completely remove log4j as a dependency from jmxfetch to mitigate any further concerns.


Create an issue

Dart infers type incorrectly with inheritance.
Dart type inference is not working properly with inheritance and field overriding.
If we don't explicitly write   final Dog pet = Dog(); (repetitive) the analyzer thinks that pet is Animal.  (which is not wrong but problematic, it should be Dog)
class Animal {}

class Dog extends Animal {
  void bark() {
    print("Daawg!");
  }
}

abstract class Owner {
  Animal get pet;
}

class DogOwner extends Owner {
  @override
  final pet = Dog();
}

void main() {
  final dog = Dog();
  dog.bark(); // pass

  final owner = DogOwner();
  owner.pet.bark();  // error. Why ??
}

However, DogOwner.pet is clearly of type Dog (at least for the human reader)
Is this a bug in Dart or intended ? I am curious to know why would this be intended.
Dart version 3.4.3

You need to declare that the type of the getter in DogOwner is Dog:
class DogOwner extends Owner {
  @override
  final Dog pet = Dog();
}

When you don't declare the return type of a getter (in this case: the implicitly induced getter of the instance variable DogOwner.pet), the step which is called override inference prevails over the kind of inference that provides the type of a variable based on its initializing expression. So DogOwner.pet has the type Animal rather than Dog because it overrides Owner.pet has type Animal.


Fix for StaticInt no longer subtyping Integer

Fixes https://github.com/JuliaLinearAlgebra/BLASBenchmarksCPU.jl/issues/79
Adds Static as a direct dependency (it was already an indirect dependency).
Tests pass locally with Static v0.7.8 and Static v0.8.8, hence the compat bounds of 0.7, 0.8.
Suggestions, comments and other feedback is welcome!
With this fix I was able to obtain these benchmark results:
julia> versioninfo()
Julia Version 1.9.4
Commit 8e5136fa297 (2023-11-14 08:46 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 64 × AMD Ryzen Threadripper PRO 3975WX 32-Cores
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, znver2)
  Threads: 64 on 64 virtual cores
Environment:
  JULIA_EDITOR = code

Comparing them to the ones on https://github.com/JuliaLinearAlgebra/Octavian.jl, they are pretty interesting. Unsurprisingly, MKL.jl does not do very well since this is on an AMD CPU. Surprisingly, the performance of BLIS drops massively from size 244 (median 783.96 GFLOPS) to size 249 (median 55.69 GFLOPS). No idea what is going on there. The performance of the other libraries seems to agree with the results in Octavian.jl.

That's an interesting CPU, you're welcome to add those results to the README or docs of either package.
Thanks for the fix!


Jenny idle CPU spikes

Hi, I have a question regarding Jenny.
Frequently when Jenny is idle it spikes to using a lot of CPU (ranging from 15% to 70%).
This can occur even when jenny has been idle for awhile, and I've never sent a gen request to jenny (so it had always been idle, never used to gen).
Why is this?
I have yet to probe for traffic concerning Jenny w/wireshark, or searched for the source code to review it; because I though it would be faster to just ask regarding this suspicious behavior.
My regards

Hi,
that's weird. When you start jenny server it opens a socket connection based on your properties (usually localhost or <IP_ADDRESS>) to receive commands like gen.
I just checked the code. The connection goes both ways, that means you can type when the server runs and the client will be notified. I think I can remove that code
while (true)
{
    server.Send(Encoding.UTF8.GetBytes(Console.ReadLine()));
}

I will take a closer look soon and keep you updated

The while keeps the server alive and prevents the program from early termination, but I can remove Console.ReadLine()
If I get this correctly, when jenny is idle it basically stops at Console.ReadLine()

A quick check on my machine resulted in 0.0% cpu usage when idle

A quick check on my machine resulted in 0.0% cpu usage when idle
On my machine it's usually at 0.0% CPU usage when idling between gen commands.
Although, that's when I'm intermittently sending gen commands (so consistent usage/gen commands sent, with small breaks ranging from 5 ~ 10min between sending gen commands).
However, after being idle for a period of time w/no gen requests (for around 15min ~ 1hr) it shoots up to high CPU usage pretty consistently.

I think I resolved the issue
It only occurred when I (mistakenly) had multiples instances of rider's external tool command open upon generating.
When I noticed this, and closed all instances except one; it always stayed at 0.0% idle cpu usage after letting it sit idle for the usual time it took to make it hop in cpu usage.
Thanks for helping

Ok, I'm glad you could resolve your issue


only test master branch

As recommended by @richafrank in #179

Short PR this time - ready for review


fix(spam): Removing debug spam

:tada: This PR is included in version 6.6.3 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket:


[GEOGRAPHIC FILTER] Rebase code for geographic filters

Make sure the geographic filters code is running rebased off the source branch

closing + backlog label -- to be revisited (maybe)


Tooltip mode doesn't seem work with custom HTML Tooltip

Hi,
I use a custom HTML tooltip.
But the mode doesn't seem to work. I want it to be 'point', but it looks like it uses 'index' when you use a custom HTML tooltip.
options: {
  tooltips: {
    enabled: false;
    mode: 'point',
    intersect: false,
    custom = //some code
  }
}

Is this a bug? Or it it supposed to work like this because it is a custom tooltip, and should I code myself a function to resize the right points on the chart?

This is a question and does not contain any interactive example to help us triage. Closing as it does not follow new issue guidelines

Ask questions and get support on StackOverflow: https://stackoverflow.com/questions/tagged/chart.js
DO NOT create issues for questions or support requests.
Issues are reserved for BUG reports and FEATURE requests.
Bug reports MUST be submitted with an interactive example (http://codepen.io/pen?template=JXVYzq).


[PREVIEW] teuthology: add support for graph walks of QA suites

This is a preview PR for the branch used for my talk at Cephalocon 2024:
https://ceph2024.sched.com/event/1ktVR/the-art-of-teuthology-patrick-donnelly-ibm-inc
TODO:

To maintain consistency with the --subset operator on matrices, we
should ensure that each fragment is included at least once in a subset
(if the subset is large enough to make that possible). The probable
way to do that is to generate the randomized set of children for each
node once and always pull the first child if a path should be
generated (then put that child at the end of the queue of children).
matrix implementation of rados creates 4455758 jobs whereas graph implementation creates 2792029 jobs. This may be a divergence caused by nested subsets but needs investigation.
print fragments not included in any path walk (job) as a warning.
docs
tests

I think this is really cool improvement. I have a possibly silly suggestion, based on the nice graphs in your cephalocon presentation, would it be reasonable to have a mode that generates graphvis output that can be used to generate graphical representations of the steps?  This kind of visualization could be handy to debug/understand the steps as they're assembled.

I think this is really cool improvement. I have a possibly silly suggestion, based on the nice graphs in your cephalocon presentation, would it be reasonable to have a mode that generates graphvis output that can be used to generate graphical representations of the steps? This kind of visualization could be handy to debug/understand the steps as they're assembled.

I already played around with a few options (including a terminal representation, which is ... fun to grok). The code is there. I would ultimately want to:

make a .svg for the path walk that produced a job, in the job dir
make a .svg for the entire suite graph in the run dir
make a .svg for a dry-run for inspection prior to scheduling

If you have tooling/library suggestions, please do share.


Fix nested storage lookup check not accounting for case insensitive filesystems

Resolves https://github.com/ppy/osu/issues/19996#issuecomment-1229428045.
Tests do not fail as it stands. I can't figure out the correct call to reproduce what the user was seeing. Help.

As for the question why the test isn't failing before the fix - I have no answer to that yet. I will note that I tried to reproduce on both a case-sensitive filesystem (ext4) and a case-insensitive one (fat32), but in both cases I was testing from linux, and the sentry reports are exclusive to windows. So I'm thinking there's possibly some windows specific nonsense that affects outputs of Path.GetFullPath() or similar? but I have no actual proof...

I've tested on windows though, and cannot repro. I was thinking I wasn't covering the precise calls in the order that osu! is.

I tried to reproduce using the scenario originally provided by the reporter in the issue this is supposing to fix, without the fix, just to see what's crashing, and I'm not getting the crash at all. Not sure what that's about, so I asked for more precise repro steps.

Is this fine to merge then?

I think so. None of us can repro the original issue, but it should fix it and hasn't seemed to break anything during testing.
Probably get this one in and revert is it ends up causing any issues.


Added: bitbucket pipelines example

Just added some docs for bitbucket. The example is a stripped-down version of what I'm using for a Laravel deployment, so it should be verbose enough for most starting points.

[ ] Bug fix #…?
[ ] New feature?
[ ] BC breaks?
[ ] Tests added?
[x] Docs added?

Nice! Thanks.

Great! Just a heads up that the image used in the example is a 7.x image.
Is that alright?
On Fri, Dec 13, 2024, 12:46 p.m. Anton Medvedev @.***>
wrote:

Merged #3979 https://github.com/deployphp/deployer/pull/3979 into
master.
—
Reply to this email directly, view it on GitHub
https://github.com/deployphp/deployer/pull/3979#event-15653405654, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAK37GGMR6GKOO6SMUM2LC32FNBUFAVCNFSM6AAAAABTSU2JCCVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJVGY2TGNBQGU3DKNA
.
You are receiving this because you authored the thread.Message ID:
@.***>

v7 is ok


Statement-ize Looping Forms

Restrict grammar of Rust language for 1.0 so that all looping syntactic forms (for, loop, while, and while let) are statements (instead of expressions). Forms like let d = loop { }; and (while foo() { }) all produce errors at parse time.
Rendered draft
Spawned off of discussion of RFC #352

So right now, is the following true?
{ loop { } - 3 } // ==> e
{ (loop { } - 3) } // type error
{ if false {} else {} - 3 } // ==> -3
{ (if false {} else {} - 3) } // type error

If so, would keep the existing symmetry and just mandate a final semicolon for a block that ends in a loop. Temporarily it's weird, but long run it is the most consistent.

@Ericson2314 I'm sorry, I cannot parse your sentence

If so, would keep the existing symmetry and just mandate a final semicolon for a block that ends in a loop.

(Did you mean to have an "I" in there, as in "I would"?)
I need a more concrete elaboration of what alternative you are proposing; I honestly cannot infer what it is from what you have written here.

@pnkfelix I am sorry, both for the typo and the lack of clarity. I did mean "I would" (edited for posterity). What I am proposing is disallowing ending a block with a loop, unless the loop is followed by a semicolon. This gives us:
    { loop { break } - 3 }; // ==> -3
    { (loop { break } - 3) }; // type error
    { loop { break } }; // syntax error for now
    { loop { break }; }; // ==> ()
    { if false {} else {} - 3 }; // ==> -3
    { (if false {} else {} - 3) }; // type error
    { if false { 3 } else { 3 } }; // ==> 3
    { if false { 3 } else { 3 }; }; // ==> ()
    { { } - 3 }; // ==> -3
    { ({ } - 3) }; // type error
    { { 3 } } // ==> 3
    { { 3 }; } // ==> ()

Which can then become backwards compatibly:
    { loop { break } - 3 }; // ==> -3
    { (loop { break } - 3) }; // type error
    { loop { break 3 } }; // ==> 3
    { loop { break 3 }; }; // ==> ()
    { if false {} else {} - 3 }; // ==> -3
    { (if false {} else {} - 3) }; // type error
    { if false { 3 } else { 3 } }; // ==> 3
    { if false { 3 } else { 3 }; }; // ==> ()
    { { } - 3 }; // ==> -3
    { ({ } - 3) }; // type error
    { { 3 } } // ==> 3
    { { 3 }; } // ==> ()

@Ericson2314 I will add that to alternatives.  I can see the appeal, yet  it would (IMO) make a lot of code a teensy bit uglier.

Thanks!

I am not keen on requiring more semicolons than we require now -- seems like we could add these extensions backwards compatbility even without this, though it would mean that while/for loops with a "break with value" would have type Option<T> vs () (asymmetric).
That said, I am unpersuaded as to the need for the original feature itself -- I agree it makes for elegant code in some cases to break with a value, but you can achieve many if not all of the same ends using iterators, filter_map, and a single call to next().

though it would mean that while/for loops with a "break with value" would have type Option vs () (asymmetric).

(To be clear, there are other options, such as a required else clause attached to such for/while-loops, as described in RFC #352.  But of course, those details are not really germane to this RFC discussion; this RFC is solely about answering the question about how much future proofing we wish to do today.)

This feels a bit odd to me. Without requiring semicolons, it feels inconsistent with the rest of the language. And requiring semicolons is ugly.
As near as I can tell, the only thing this accomplishes is allowing looping constructs to always evaluate to a non-() type (when not in statement position) in the future, without breaking code that expressly uses looping constructs as expressions and expects a () result (which is rare). But I'm not convinced that's a worthwhile change. I can see the merits of allowing for looping constructs to sometimes have non-() types (e.g. with a break <expr> coupled with an else block), but it would feel surprising to me to have looping constructs always have a non-() type, and even more-so if they only did this when used in a non-statement position.

I'm in favor of the sentiment as futureproofing.
The phrase "then we probably will require the use of parentheses around such forms when they appear as the tail expression of a block" seems worrying since putting the expression-loop in tail position would seem to be a common case for the feature.

@kballard The idea isn't that loops always have a non-() type in expression position, but that absent of a concrete plan it is only safe they do for compatibility. Also it is better to think of the loops in "statement position" as expressions with their value thrown away, rather than a "true statement" like a let binding. That way, the type of the loop itself is context-free.

Overall I still trend negative on this proposal, because I think the scheme we have is working pretty well and the need to change it is small. But I just re-read the RFC briefly and realized I was slightly confused about the distinction between the alternatives and main proposal. Unfortunately, now that I grok it better, I feel like there is a fundamental conflict between C-like and expression-like usage that makes me dislike the idea of while and for uniformly adopting Option type (see below).
In particular, (iiuc) the RFC is saying that a looping control-flow construct cannot be in the tail expression of a block. That is, in the following program:
fn foo() {
    while something { ...; break; ... }
}

the while loop here would be parsed today as a tail expression of type () but in this proposal would be parsed as a statement and there would be no tail expression.
This implies that in the future, if while loops uniformly had type Option<T> for some T, then the following program would not type check:
fn bar(i: i32) -> Option<i32> {
    while something(i) { if something_else(i) { break i; } ... }
}

Instead one would be required to write return while... or (while...) or something like that.
On the other hand, if we adopted the alternative that @Ericson2314 proposed], then we would find that the fallout would be much larger, because we would need a ;. I would go so far as to call it a non-starter for me, because I think that the original fn we saw (with unit return type) ought to be legal.
This conflict seems to me to be somewhat fundamental though, and seems to suggest to me that making while and for uniformly have Option type is misguided. That is, we can't accept both the foo and bar examples above. The only way to accept both would be if we made a while loop with a "value-less" break (as today) have () type, and a while loop with a value-carrying break have Option type. Which also means we don't need to change anything today to prepare for that possibility.
Is there a flaw in this reasoning? I guess the flaw might be that we want both foo and bar to continue parsing -- perhaps one might be willing to sacrifice foo or bar (for example, by requiring foo to have a ;).

In re-reading @brson's comment I realize he put my entire post into 1 sentence. ;)

Okay, so far it sounds like core team may be trending towards the second listed alternative:

Obvious 2: We could decide that non-unit looping expressions are only worth adopting if they can be added without requiring a change like this.

I'll keep this RFC up for a while to see if anyone comes in with strong arguments against that view.

For what it's worth (which, I acknowledge, probably isn't much), I am in favor of requiring semicolons on the end of looping expressions. I disagree with the aesthetic concern about requiring new semicolons: I actually consider it more elegant, and more revealing of programmer intention, to require semicolons after loops than to treat the braces from loops (and if statements) specially. And I think we are currently treating loops specially:

Where the language requires the result of evaluation to be (), a semicolon is usually required.
Where the language requires braces, the result of evaluation is usually allowed to be other than ().

Requiring a semicolon after a loop statement reveals programmer intention that the value returned by the looping expression is (currently, until loops can return non-unit values) meaningless. Regarding the appendix, it is surprising that those two if expressions behave differently. If a semicolon were required on the end of an if statement, it would resolve the ambiguity, and the surprise would disappear.

While I do prefer the semicolon route, Here's an alternative plan that is fully backwards compatible:
First some context, I have two guiding principles:

break; should mean the same thing as break ();. This preserves the symmetry with return, and more broadly (though subjectively) aligns with the other cases of syntactic support for eliding ().
loop { } is the most powerful looping construct---all others can be desugared to it. If we want to add new functionality too loops, it is most important that it be available with loop { } as the other types of loops can be manually desugared by the programmer to take advantage of the new functionality.

So lets add:
a : A
---------------------------------
loop { ... break a; ... } : A

All our existing loop { }s desugar to
loop { ... break (); ... } : ()

and thus keep their type from today.

@Ericson2314 I know you already said this new alternative plan is fully backwards compatible, but just to be 100% clear: This RFC was meant to address future-proofing concerns to accommodate #352  / #961 .
If I understand what you said properly, you are advocating (or at least outlining) a plan that requires no changes from Rust as it stands today; i.e. no future proofing, right?  I only ask to try to clarify whether you are suggesting that we can indeed close this RFC.

@pnkfelix Yes, exactly. That plan would replace #352 / #961 and close this.

For the record, a third plan is to make it so that the for- and while- loops without explicit breaks return () too. Since there is only one way to exit those loops, I don't see any useful information could be returned. But I'd still likefor... / while... { ... break (); ... } to return Option<()>, both for consistency with breaking an value of an arbitrary type, and pragmatically to distinguish between the two ways the loop can be exited. The only way this plan would be backwards compatible would be to make break; and break (); mean different things, but I consider that a non-starter.

For what it's worth, my preference is (and has been all along) for the approach where Option is not special cased, for and while are extended with an optional else clause (whatever we name it), the return type of the various looping constructs depend on the presence or absence of break and/or else, and break is short for break () (as @Ericson2314 also writes).
So:
x: T

loop { }: ! // for<T> T

loop { break }: ()

loop { break x }: T

while (foo) { }: ()

while (foo) { break }: ()

while (foo) { break x }: type error

while (foo) { break x } else { x }: T

with for..in the same as while. (As far as I'm aware this is backwards compatible.)

@glaebhoerl Ah, I initially thought of else rules as "functional break requires else", which breaks the break break () equivalence. But if it is instead thought of as ... == ... else { () }, the rules as the same as with if (if x { () } is a legal expression), and the break break () equivalence is preserved. I'm not wild about else as option gives the same expressive power without extra syntax, but it does help with compatibility and for that I might forgive it.
So in sum, if the core team changes their mind and decides semicolons are OK after all, then I like that and eventually arriving at:
given x:T,  y: U
loop { }: !
loop { break }: ()
loop { break x }: T
while foo { }: ()
while foo { break }: Option<()>
while foo { break x }: Option<T>
for _ in for { }: ()
for _ in for { break }: Option<()>
for _ in for { break x }: Option<T>
while let PAT = y { }: U // y if it doesn't match the pattern
while let PAT = y { break }: Result<(), U>
while let PAT = y { break x }: Result<T, U>

If semicolons are no-go, I like @glaebhoerl's plan (but what do you do with while let), or my "plan 2" (https://github.com/rust-lang/rfcs/pull/955#issuecomment-78314356) which is forwards compatible with @glaebhoerl's plan and backwards compatible with the status-quo and thus can be viewed as a stepping stone.

Requiring semicolons seems a non-starter since other C-like languages don't and it would be extremely surprising for programmers (most likely, expletives would be uttered upon discovering this rule).
Making "break" and "break ()" do different things instead seems ok, and might actually be good as it emphasizes the fact that the behavior of "break ()" is something new compared to C-like languages.

I like @glaebhoerl 's plan too, because it doesn't change the meaning of existing loops, but merely extends them in a consistent fashion. I also prefer the "else" syntax to using "Option", as it makes the common use-case clear and simple:
for x in xs {
   if x.foo == y { break x; }
} else {
    // Handle missing case
}

While of course the same thing can be achieve with "Option", it requires either an additional "if" or "match" statement, or a closure. Using "else" just feels leaner and more straightforward, even though it is techincally introducing new syntax.

@bill-myers Thank you for putting your case against semicolons so plainly. I disagree because brace-blocks mean something completely different in Rust than they do in other C-like languages: in other languages, they are used to group statements. In Rust, a brace-block is generally an expression. Expressions are generally terminated with semicolons. Requiring a semicolon makes this fundamental difference in brace-usage actually appear different in code, which I think is to the good.
If semicolons were required, then a programmer used to C who forgets a semicolon on the end of a while-loop would probably utter an expletive. But that would be followed by an "aha" moment when they understand what's going on, and the language's behavior and design would become more clear as a result. That is, the confusion is brief, and isolated to developers learning the language for the first time (and who should, as such, be expecting to learn new concepts as they try the language out).
On the other hand, if semicolons are not required to terminate statements, then when you run into situations like @pnkfelix described in the appendix to this RFC (showing an ambiguity between an if statement and an if expression), it would be confusing even to intermediate Rust developers. There is no initial confusion or expletive uttered when working with the language for the first time, but on the other hand, that allows a possibly mistaken interpretation of brace-expressions to become ingrained in the reader's mind, so that the expletive is uttered much later, more rarely, but with more force.
To make this more concrete, I'd advocate for a rule that all statements should be terminated with semicolons. Braces in most Rust code blocks denote expressions. Expressions and statements should be unambiguous.

(i am no longer a proponent of doing this, if i ever was ...; closing.)

To be clear, is just the stopped gap ruled out now / are the backwards compatible plans still on the table for post 1.0?

@Ericson2314 the hypothetical feature of allowing loops to return values is still a potential future feature, at least in the sense that #961 remains open.
The particular detail of one kind of future proofing proposed by this RFC #955, however, is no longer on the table.

Gotcha---didn't realize a postponement issue was made when the original RFC was closed.


Auto center and ground the model so it appears correct in most cases in AR

This does not have any affect in 2D displays.

A lot of warning signs for me here:

No issue referenced
Lacking PR details in description
No tests


class-api proposal (again)

This is an improved class api proposal for a PyDust class which resides in __init__.py
It provides a simple programmatic api such that you can do the following:
from pydust import PyDust

exts = [
    'fib._fib=fib/fib.zig',
    'fib/hello.zig',
    'fib/inner/classes.zig',
]

p = PyDust(exts)
p.build()

or
from pydust import PyDust

p = PyDust()

exts = [
    'fib._fib=fib/fib.zig',
    'fib/hello.zig',
    'fib/inner/classes.zig',
]

p.add_extensions(exts)
p.build()

or
from pydust import PyDust
from pydust import config

ext = config.ToolPydust(name='fib._fib', path='fib/fib.zig', limited_api=True)

p3 = PyDust()
p3.extensions.append(ext)

The commandline functionality is integrated as a classsmethod, but can be alternatively separated into a single function in __main__.py if that is preferable.

Would you be able to describe how you're using / planning to use Pydust? It would be useful to understand the motivation so we can figure out how what the ideal end state is and move towards it vs trying to review incremental changes to APIs.

Would you be able to describe how you're using / planning to use Pydust? It would be useful to understand the motivation so we can figure out how what the ideal end state is and move towards it vs trying to review incremental changes to APIs.

Sure. While the current documented way of using pydust is via poetry, if you want to use alternative build systems such as cmake one would need (1) to provide a command line interface or (2) be able to run scripts which contain build instructions.
The recent build subcmd addresses the former to some extent. For example, in  cmake on could 'naively' call pydust using the following:
execute_process(
    COMMAND python3 -m pydust build fib/fib.zig
    RESULTS_VARIABLE pydust_succeeded
    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
)

Alternatively, one could use the add_custom_target / add_custom_command pair in a similar fashion to only trigger execution in the event of a specified dependency.
The programmatic interface becomes immediately useful if you want to integrate pydust in a regular setup.py file or if you are using a custom python build system. It is also useful in a script (as per case 2 above) if there are many files to deal with and it is better run as a script than via a command line interface.

Let's leave this PR for now, it may be premature... so I'm closing it.