[BUG] Inconsistent behavior of snapshot list between v1 and v2 volume

Describe the bug
The behavior of snapshot list is inconsistent between v1 and v2 volume:
Scenario 1:

Create a v1 volume and take 2 snapshots.  And get snapshot list of the volume, there are only 2 items and no volume-head:

curl -u "${RANCHER_ACCESS_KEY}:${RANCHER_SECRET_KEY}" \
-X POST \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-d '{}' \
'http://<IP_ADDRESS>:30000/v1/volumes/test-3?action=snapshotList'

{

    "data": [ 2 items
        {
            "actions": { },
            "checksum": "",
            "children": {
                "be6d466e-a187-4f0c-bd34-8f1a7f5f9241": true,
            },
            "created": "2024-01-11T00:42:07Z",
            "id": "6ec4407c-93df-4b33-8821-6fccad570960",
            "labels": { },
            "links": {
                "self": "http://<IP_ADDRESS>:9500/v1/snapshots/6ec4407c-93df-4b33-8821-6fccad570960",
            },
            "name": "6ec4407c-93df-4b33-8821-6fccad570960",
            "parent": "",
            "removed": false,
            "size": "109051904",
            "type": "snapshot",
            "usercreated": true,
        },
        {
            "actions": { },
            "checksum": "",
            "children": {
                "volume-head": true,
            },
            "created": "2024-01-11T00:43:20Z",
            "id": "be6d466e-a187-4f0c-bd34-8f1a7f5f9241",
            "labels": { },
            "links": {
                "self": "http://<IP_ADDRESS>:9500/v1/snapshots/be6d466e-a187-4f0c-bd34-8f1a7f5f9241",
            },
            "name": "be6d466e-a187-4f0c-bd34-8f1a7f5f9241",
            "parent": "6ec4407c-93df-4b33-8821-6fccad570960",
            "removed": false,
            "size": "0",
            "type": "snapshot",
            "usercreated": true,
        },
    ],
    "links": {
        "self": "http://<IP_ADDRESS>:9500/v1/volumes/test-3",
    },
    "resourceType": "snapshot",
    "type": "collection",

}

Scenario 2:

Removed a snapshot of a v1 volume, it's just marked as removed=true, but it can still be found in the snapshot list:

{

    "data": [ 3 items
        {
            "actions": { },
            "checksum": "",
            "children": {
                "57c261f4-b026-498f-b554-e8001e5ef3de": true,
            },
            "created": "2024-01-11T00:38:57Z",
            "id": "523a8302-9103-4e1b-af16-04caac39c683",
            "labels": { },
            "links": {
                "self": "http://<IP_ADDRESS>:9500/v1/snapshots/523a8302-9103-4e1b-af16-04caac39c683",
            },
            "name": "523a8302-9103-4e1b-af16-04caac39c683",
            "parent": "",
            "removed": false,
            "size": "97677312",
            "type": "snapshot",
            "usercreated": true,
        },
        {
            "actions": { },
            "checksum": "",
            "children": {
                "volume-head": true,
            },
            "created": "2024-01-11T00:41:02Z",
            "id": "57c261f4-b026-498f-b554-e8001e5ef3de",
            "labels": { },
            "links": {
                "self": "http://<IP_ADDRESS>:9500/v1/snapshots/57c261f4-b026-498f-b554-e8001e5ef3de",
            },
            "name": "57c261f4-b026-498f-b554-e8001e5ef3de",
            "parent": "523a8302-9103-4e1b-af16-04caac39c683",
            "removed": true,
            "size": "4210688",
            "type": "snapshot",
            "usercreated": true,
        },
        {
            "actions": { },
            "checksum": "",
            "children": { },
            "created": "2024-01-11T00:41:02Z",
            "id": "volume-head",
            "labels": { },
            "links": {
                "self": "http://<IP_ADDRESS>:9500/v1/snapshots/volume-head",
            },
            "name": "volume-head",
            "parent": "57c261f4-b026-498f-b554-e8001e5ef3de",
            "removed": false,
            "size": "0",
            "type": "snapshot",
            "usercreated": false,
        },
    ],
    "links": {
        "self": "http://<IP_ADDRESS>:9500/v1/volumes/test-2",
    },
    "resourceType": "snapshot",
    "type": "collection",

}

Removed a snapshot of a v2 volume, it's removed from the snapshot list:

{

    "data": [
        {
            "actions": { },
            "checksum": "",
            "children": {
                "volume-head": true,
            },
            "created": "2024-01-11T00:42:07Z",
            "id": "6ec4407c-93df-4b33-8821-6fccad570960",
            "labels": { },
            "links": {
                "self": "http://<IP_ADDRESS>:9500/v1/snapshots/6ec4407c-93df-4b33-8821-6fccad570960",
            },
            "name": "6ec4407c-93df-4b33-8821-6fccad570960",
            "parent": "",
            "removed": false,
            "size": "109051904",
            "type": "snapshot",
            "usercreated": true,
        },
    ],
    "links": {
        "self": "http://<IP_ADDRESS>:9500/v1/volumes/test-3",
    },
    "resourceType": "snapshot",
    "type": "collection",

}

Need to confirm if it's the expected behavior, if so, we will need to modify the automated test cases and have different implementations for v1 and v2 volume.
To Reproduce

Expected behavior

Support bundle for troubleshooting

Environment

Longhorn version:
Impacted volume (PV):
Installation method (e.g. Rancher Catalog App/Helm/Kubectl):
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version:

Number of control plane nodes in the cluster:
Number of worker nodes in the cluster:

Node config

OS type and version:
Kernel version:
CPU per node:
Memory per node:
Disk type (e.g. SSD/NVMe/HDD):
Network bandwidth between the nodes (Gbps):

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal):
Number of Longhorn volumes in the cluster:

Additional context
Related to test cases test_snapshot, test_snapshot_prune_and_coalesce_simultaneously and test_restore_inc.

For scenario 2, the result of v2 volume is expected. The snapshot is actually deleted unlike v1 volume.

Created https://github.com/longhorn/longhorn/issues/7624 for document.
And created https://github.com/longhorn/longhorn/issues/7623 for automated test cases modification.

Will this impact the presentation at UI? Moving to 1.7.0 first, but we need to clarify the diff or make it consistent.

Will this impact the presentation at UI?

No.

Verified passed on master-head (longhorn-spdk-engine 64886f5). Taking snapshots of v2 volume and getting its snapshot list, volume-head can be found in the snapshot list.


[vpj][dvc][server][controller][test] Use a single object as producer callback and produce result future

Use a single object as producer callback and produce result future
This PR combines Future returned by PubSubProducerAdapter::sendMessage and
PubSubProducerCallback is one of the arguments for the sendMessage API.  The
goal is to avoid the additional object allocation for Future  that happens during the
conversion as it is only used at a few places for synchronous/blocking calls. Also, we
would like to leverage PubSubProducerCallback to get a result of send action as we
already use callback on every sendMessage call.
All methods in VeniceWriter::{put|update|delete} now return void. Callers will callback
passed to these VW APIs for async/blocking calls.
Follow up PR w.r.t. the following comment by @FelixGV

I took a look at the code and noticed that we almost never use the future returned from sendMessage. We use it in some control message scenario, and in a few tests, but I believe those should be not too hard to refactor into using callbacks. Can we do this, so that this function and all those that call it can return void instead? I think it would simplify our code, and would avoid the object allocation of this wrapper future which is almost always useless.

How was this PR tested?

GHCI and InternalCI
Does this PR introduce any user-facing changes?

[x] No. You can skip the rest of this section.
[ ] Yes. Make sure to explain your proposed changes and call out the behavior change.

Closing this PR for now.l


fix 'status_iterator' duplicate package

One day, I failed sphinx build which version is 4.4.0 with following error.
Extension error:
Could not import extension sphinxcontrib.youtube (exception: No module named 'sphinx.util.display')

It seems 'status iterator' class had been duplicated in 2 packages and was deleted it 'sphinx.util.display'.
https://www.sphinx-doc.org/en/master/extdev/deprecated.html

You've got this backwards. sphinx.util.status_iterator has been deprecated since Sphinx 6.1 and will be removed in Sphinx 8.0. You should preferably upgrade to a recent Sphinx. If you can't do that, then I think that downgrading to sphinxcontrib-youtube v1.3.0 will possibly work for you.
See also #63

Thank you for helpful information. As you said, build was succeeded after freeze version v1.3.0.


Changes required for a successful build that passes tests

This commit makes the necessary changes to get out a successful release that passes all tests.
Also added some scripts to help with release validation.
Maybe we can add this whole process to CI, so that we are always ready to release?

Can you please squash your changes then git commit --amend -s to sign off and pass the DCO check? Thanks.


InsecurePlatformWarning

When running build command, I get:
...
Step 12 : RUN pip install -r /opt/sensu-grid/requirements.txt
---> Running in bab5526aaa24
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
You are using pip version 7.1.0, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting Flask==0.12.2 (from -r /opt/sensu-grid/requirements.txt (line 1))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/77/32/e3597cb19ffffe724ad4bf0beca4153419918e7fa4ba6a34b04ee4da3371/Flask-0.12.2-py2.py3-none-any.whl (83kB)
Collecting PyYAML==3.12 (from -r /opt/sensu-grid/requirements.txt (line 2))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/4a/85/db5a2df477072b2902b0eb892feb37d88ac635d36245a72a6a69b23b383a/PyYAML-3.12.tar.gz (253kB)
Collecting argparse (from -r /opt/sensu-grid/requirements.txt (line 3))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/f2/94/3af39d34be01a24a6e65433d19e107099374224905f1e0cc6bbe1fd22a2f/argparse-1.4.0-py2.py3-none-any.whl
Collecting requests==2.18.4 (from -r /opt/sensu-grid/requirements.txt (line 4))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl (88kB)
Collecting gunicorn==19.7.1 (from -r /opt/sensu-grid/requirements.txt (line 5))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/64/32/becbd4089a4c06f0f9f538a76e9fe0b19a08f010bcb47dcdbfbc640cdf7d/gunicorn-19.7.1-py2.py3-none-any.whl (111kB)
Collecting six==1.11.0 (from -r /opt/sensu-grid/requirements.txt (line 6))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting futures==3.2.0 (from -r /opt/sensu-grid/requirements.txt (line 7))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl
Collecting diskcache==3.0.1 (from -r /opt/sensu-grid/requirements.txt (line 8))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/68/1e/99614279bb9502a2c0e59ac22ee5ab9ecaf0ce4e85a4edce842eeaf7b7db/diskcache-3.0.1-py2.py3-none-any.whl
Collecting click>=2.0 (from Flask==0.12.2->-r /opt/sensu-grid/requirements.txt (line 1))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl (81kB)
Collecting itsdangerous>=0.21 (from Flask==0.12.2->-r /opt/sensu-grid/requirements.txt (line 1))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl
Collecting Werkzeug>=0.7 (from Flask==0.12.2->-r /opt/sensu-grid/requirements.txt (line 1))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/20/c4/12e3e56473e52375aa29c4764e70d1b8f3efa6682bef8d0aae04fe335243/Werkzeug-0.14.1-py2.py3-none-any.whl (322kB)
Collecting Jinja2>=2.4 (from Flask==0.12.2->-r /opt/sensu-grid/requirements.txt (line 1))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl (126kB)
Collecting urllib3<1.23,>=1.21.1 (from requests==2.18.4->-r /opt/sensu-grid/requirements.txt (line 4))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl (132kB)
Collecting certifi>=2017.4.17 (from requests==2.18.4->-r /opt/sensu-grid/requirements.txt (line 4))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/56/9d/1d02dd80bc4cd955f98980f28c5ee2200e1209292d5f9e9cc8d030d18655/certifi-2018.10.15-py2.py3-none-any.whl (146kB)
Collecting chardet<3.1.0,>=3.0.2 (from requests==2.18.4->-r /opt/sensu-grid/requirements.txt (line 4))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
Collecting idna<2.7,>=2.5 (from requests==2.18.4->-r /opt/sensu-grid/requirements.txt (line 4))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl (56kB)
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->Flask==0.12.2->-r /opt/sensu-grid/requirements.txt (line 1))
/usr/lib/python2.6/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://files.pythonhosted.org/packages/4d/de/32d741db316d8fdb7680822dd37001ef7a448255de9699ab4bfcbdf4172b/MarkupSafe-1.0.tar.gz
Installing collected packages: click, itsdangerous, Werkzeug, MarkupSafe, Jinja2, Flask, PyYAML, argparse, urllib3, certifi, chardet, idna, requests, gunicorn, six, futures, diskcache
...
Need a later version of python, or get this working on CentOS 7, or something. This looks really cool, but is (no longer?) working out of the box for me on an ubuntu EC2 system.
The container runs, but when visiting the page I get:
File "/usr/lib/python2.6/site-packages/gunicorn/workers/base.py", line 135, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/lib/python2.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/lib/python2.6/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
return self.load_wsgiapp()
File "/usr/lib/python2.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/lib/python2.6/site-packages/gunicorn/util.py", line 352, in import_app
import(module)
File "/opt/sensu-grid/sensugrid.py", line 34, in
from diskcache import Cache
File "/usr/lib/python2.6/site-packages/diskcache/init.py", line 3, in
from .core import Cache, Disk, UnknownFileWarning, EmptyDirWarning, Timeout
Thanks.

The host OS is Ubuntu 14.04 and I was using FROM centos:centos6 in the Dockerfile. In a wild experiment (lol) I tried changing it to FROM centos:centos7.
The platform warning/error comes from within container, while it's running pip install -r requirements.txt.

oic, ok I'll see what I can do, might take a short while though

I'm assuming this "fixed" it. If not, please re-open and I'll check again. But the latest changes should work.


Diode Insertion fails

Description
With DIODE_INSERTION_STRATEGY=4 OpenLane flow fails.
Expected behavior
Flow should work all diode insertion strategy.
Environment

Kernel: Linux v3.10.0-1160.49.1.el7.x86_64
Distribution: centos 7
Python: v3.6.8 (OK)
Container Engine: docker v20.10.16 (OK)
OpenLane Git Version:<PHONE_NUMBER>2b67c490c96ded79b93aa2bb570a48
pip: INSTALLED
python-venv: INSTALLED
---
PDK Version Verification Status: OK
---
Git Log (Last 3 Commits)

8717388 2022-09-20T00:03:58+02:00 Point to `open_pdks` upstream (#1372) - Kareem Farid -  (HEAD -> master, tag: 2022.09.20, origin/master, origin/HEAD)
3ee8cf0 2022-09-19T20:38:28+02:00 Use OpenDB as the Primary Layout Format (#1244) - Mohamed Gaber -  ()
d375db9 2022-09-15T13:35:19+02:00 [BOT] Update magic (#1318) - Openlane Bot -  (tag: 2022.09.16)
---
Git Remotes

origin	https://github.com/The-OpenROAD-Project/OpenLane.git (fetch)
origin	https://github.com/The-OpenROAD-Project/OpenLane.git (push)

Reproduction Material

Update `design/spm/config.json` with `"DIODE_INSERTION_STRATEGY": 4,`
cd OpenLane
make mount
./flow.tcl -design spm

Flow fail with below log during diode insertion.
Logs

[STEP 16]
[INFO]: Running Diode Insertion (log: designs/spm/runs/diode_4/logs/routing/16-diodes.log)...
[ERROR]: during executing: "openroad -python /openlane/scripts/odbpy/diodes.py place --diode-cell sky130_fd_sc_hd__diode_2 --diode-pin DIODE --fake-diode-cell sky130_fd_sc_hd__diode_2 --input-lef /openlane/designs/spm/runs/diode_4/tmp/merged.nom.lef --output-def /openlane/designs/spm/runs/diode_4/tmp/routing/16-diodes.def --output /openlane/designs/spm/runs/diode_4/tmp/routing/16-diodes.odb /openlane/designs/spm/runs/diode_4/tmp/14-spm.buffers_removed.odb |& tee /dev/null /openlane/designs/spm/runs/diode_4/logs/routing/16-diodes.log"
[ERROR]: Exit code: 1
[ERROR]: Last 10 lines:
OpenROAD 4174c3ad802d2ac1d04d387d2c4b883903f6647e
This program is licensed under the BSD-3 license. See the LICENSE file for details.
Components of this program may be licensed under more restrictive licenses which must be honored.
Usage: diodes.py [OPTIONS] COMMAND [ARGS]...
Try 'diodes.py --help' for help.

Error: No such command 'place'.
child process exited abnormally

[INFO]: Saving current set of views in 'designs/spm/runs/diode_4/results/final'...
[INFO]: Generating final set of reports...
[INFO]: Created manufacturability report at 'designs/spm/runs/diode_4/reports/manufacturability.rpt'.
[INFO]: Created metrics report at 'designs/spm/runs/diode_4/reports/metrics.csv'.
[INFO]: Saving runtime environment...
[ERROR]: Flow failed.

Duplicate of https://github.com/The-OpenROAD-Project/OpenLane/issues/1379 - proposed fix in https://github.com/The-OpenROAD-Project/OpenLane/pull/1381


Jetpack Compose

Did you think to add Jetpack Compose to the project?
What do you think about it?

Did you think to add Jetpack Compose to the project?
What do you think about it?

I have planed to make Jetpack Compose and SwiftUI versions, stay tuned.

Did you think to add Jetpack Compose to the project?
What do you think about it?

I have planed to make Jetpack Compose and SwiftUI versions, stay tuned.

Cool, I would like to see it!


Errors due to BLINCUSUSER and other variables not getting replaced

Ive been getting some errors when trying to execute shells inside my blincus-managed incus containers because the ~/.config/blincus/templates didnt get customized with my user configurations upon install. Maybe the "personalize" function on initialize.sh should be uncommented? Or it should be moved to the installation script instead of the main blincus one

the next release moves the customization to JIT so it won't be accidentally skipped. thanks for the report.


Fix AMM missing XRP Asset Metadata

High Level Overview of Change
In the NewFields of a AMM object after a AMMCreate transaction, Asset would be omitted if it is XRP, which is incorrect. A new amendment fixAMMMetadata fixes this problem

Context of Change

Type of Change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Refactor (non-breaking change that only restructures code)
[ ] Performance (increase or change in throughput and/or latency)
[x] Tests (you added tests for code that already exists, or your new feature included in this PR)
[ ] Documentation update
[ ] Chore (no impact to binary, e.g. .gitignore, formatting, dropping support for older tooling)
[ ] Release

API Impact

[ ] Public API: New feature (new methods and/or new fields)
[ ] Public API: Breaking change (in general, breaking changes should only impact the next api_version)
[ ] libxrpl change (any change that may affect libxrpl or dependents of libxrpl)
[ ] Peer protocol change (must be backward compatible or bump the peer protocol version)

Codecov Report
Attention: Patch coverage is 50.00000% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 70.9%. Comparing base (5aa1106) to head (2c0572c).
Report is 1 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff           @@
##           develop   #5012   +/-   ##
=======================================
  Coverage     70.9%   70.9%
=======================================
  Files          796     796
  Lines        66792   66798    +6
  Branches     11002   10998    -4
=======================================
+ Hits         47379   47385    +6
  Misses       19413   19413

Files
Coverage Δ

src/ripple/protocol/Feature.h
100.0% <ø> (ø)

src/ripple/protocol/impl/Feature.cpp
93.9% <ø> (ø)

src/ripple/protocol/impl/STIssue.cpp
94.1% <100.0%> (+0.6%)
:arrow_up:

src/ripple/protocol/impl/STCurrency.cpp
73.5% <0.0%> (-7.1%)
:arrow_down:

... and 5 files with indirect coverage changes

Does it make sense to add another virtual method to STBase, say canInsertMetadata() with the default implementation returning isDefault() and STIssue/STCurrency returning false? canInsertMetadata() is only called in the metadata handling. This ensures that isDefault() semantic is not broken.

Does it make sense to add another virtual method to STBase, say canInsertMetadata() with the default implementation returning isDefault() and STIssue/STCurrency returning false? canInsertMetadata() is only called in the metadata handling. This ensures that isDefault() semantic is not broken.

I'm not sure if that would be needed. isDefault() itself is only used for metadata purposes(though would need to double check this), so I don't think we would need to introduce another method for this. And by Ed's definition, isDefault() is only supposed to return true if the value represents "nothing", so omitting XRP would clearly be wrong behavior.

Can you confirm it's only used for the metadata?
Does badCurrency() represent "nothing"? By the way, STAmount has isDefault() as return (mValue == 0) && mIsNative;; i.e. XRP(0), which is clearly not "nothing".

Does it make sense to add another virtual method to STBase, say canInsertMetadata() with the default implementation returning isDefault() and STIssue/STCurrency returning false? canInsertMetadata() is only called in the metadata handling. This ensures that isDefault() semantic is not broken.

I'm not sure if that would be needed. isDefault() itself is only used for metadata purposes(though would need to double check this), so I don't think we would need to introduce another method for this. And by Ed's definition, isDefault() is only supposed to return true if the value represents "nothing", so omitting XRP would clearly be wrong behavior.

Can you confirm it's only used for the metadata?
Does badCurrency() represent "nothing"? By the way, STAmount has isDefault() as return (mValue == 0) && mIsNative;; i.e. XRP(0), which is clearly not "nothing".

That is really a scary question for a Rippel employee to ask! 😨 The badCurrency is the encoding of the string  XRP as 160-bit asset code and used to prevent trust lines with that asset code from being created, probably as a safety to prevent people from being tricked. But attackers can still do this trick by using other character case or even a space. Very sloppy. I am surprsied nobody else has discussed this!

FWIW, I am uncomfortable with this change. Both STCurrency and STIssue are basic protocol building blocks, and because of this we cannot assume that isDefault() functions will be only ever used for metadata generation (that is, regardless of the answer to the question posed by @gregtatcam , things may change in the future). IMO we should either add a new function with the required semantics and change other bits (i.e. where metadata is generated) to use it, or do something else.

FWIW, I am uncomfortable with this change. Both STCurrency and STIssue are basic protocol building blocks, and because of this we cannot assume that isDefault() functions will be only ever used for metadata generation (that is, regardless of the answer to the question posed by @gregtatcam , things may change in the future). IMO we should either add a new function with the required semantics and change other bits (i.e. where metadata is generated) to use it, or do something else.

I agree with this.
Here is what we did:
bool const recordDefaultAmounts = to.rules().enabled(fixXahauV1);
...
bool const shouldRecord =
                    (obj.getSType() == STI_AMOUNT && recordDefaultAmounts) ||
                    !obj.isDefault();

FWIW, I am uncomfortable with this change. Both STCurrency and STIssue are basic protocol building blocks, and because of this we cannot assume that isDefault() functions will be only ever used for metadata generation (that is, regardless of the answer to the question posed by @gregtatcam , things may change in the future). IMO we should either add a new function with the required semantics and change other bits (i.e. where metadata is generated) to use it, or do something else.

I agree with this.
Here is what we did:
bool const recordDefaultAmounts = to.rules().enabled(fixXahauV1);
...
bool const shouldRecord =
                    (obj.getSType() == STI_AMOUNT && recordDefaultAmounts) ||
                    !obj.isDefault();

Having read https://github.com/XRPLF/rippled/issues/4965#issuecomment-2079688630 I think I was mistaken. The semantics of isDefault() is defined in STBase ("does the object hold any useful data ?"); neither STIssue nor STCurrency implement it correctly, as of today. Having said that, I also think that the proposed amendment is badly named, as this is not an AMM amendment. It is a currency/IOU metadata amendment.

FWIW, I am uncomfortable with this change. Both STCurrency and STIssue are basic protocol building blocks, and because of this we cannot assume that isDefault() functions will be only ever used for metadata generation (that is, regardless of the answer to the question posed by @gregtatcam , things may change in the future). IMO we should either add a new function with the required semantics and change other bits (i.e. where metadata is generated) to use it, or do something else.

I agree with this.
Here is what we did:
bool const recordDefaultAmounts = to.rules().enabled(fixXahauV1);
...
bool const shouldRecord =
                    (obj.getSType() == STI_AMOUNT && recordDefaultAmounts) ||
                    !obj.isDefault();

Having read #4965 (comment) I think I was mistaken. The semantics of isDefault() is defined in STBase ("does the object hold any useful data ?"); neither STIssue nor STCurrency implement it correctly, as of today. Having said that, I also think that the proposed amendment is badly named, as this is not an AMM amendment. It is a currency/IOU metadata amendment.

I disagree as STAmount clearly defines the isDefault() as return (mValue == 0) && mIsNative; so if that is incorrect then yeah it should just be false AND STAmount should be fixed as well, however if that is the correct default response then STIssue and STCurrency are correct in their response and this fix needs to be handled outside of the isDefault() behavior.
But alas I really don't have any firm position on how its handled in ripple(d).

return (mValue == 0) && mIsNative;

I think the condition that you are citing here just happens to match the definition of "no useful data here", in the context of STAmount.


Another attempt to get revoluteJoint

Hi, I noticed that 50% of the forks of your project add revolute joints.
I did too, and I tried to copy your style as much as possible. Do you think this can be merged?
Oh, and like #101, I also noticed the jointType member, and also went for separate class - it's just cleaner this way.

Sorry, had to use this branch for work.
I'll try again later.


Allow client objects to have resource methods

Spec Issue : https://github.com/ballerina-platform/ballerina-spec/issues/1080

[x] Open github discussion https://github.com/ballerina-platform/ballerina-lang/discussions/36265
[x] Add parser changes
[x] Add TypeChecker changes
[x] Add Desugar changes
[x] Add BIR changes
[x] Add test cases

Fixed with  #36959


Release

Coverage increased (+0.005%) to 90.274% when pulling 06590ff985dc0ca70e771fee4eb48fbb5fe12f5a on release into 08b2dd8560cb2ebc8d142892e79a83fa1fbf2d90 on preprod.


Whitelisting Blocks FileMatchContentsProcessorWithContext

Hey Luke
Please check the attachment, It looks like when white listing packages the file match processors aren't being taken into effect.
The directory structure is a bit more "realworld"
Whitelisting the "test" package also returns
"io.github.lukehutch.fastclasspathscanner.matchprocessor.FileMatchContentsProcessorWithContext" as a scanned class? :) Not an issue just wondering why :)
Thanks man! Great Great library!
servicelocatortest.zip

First off the bat -- as mentioned in my comment in the previous bug you filed, #162, you cannot whitelist wildcarded/globbed filenames like *.json. That's the purpose of the different types of FileMatchProcessor, such as one that matches file extensions.
Secondly, as also mentioned in #162, you need to add FastClasspathScanner#alwaysScanClasspathElementRoot() to your code before the scan. And you'll need to build the git master version to get that functionality, I haven't checked it in yet. Please verify if this fixes your problem.
Thirdly, the reason you're getting classes outside of a whitelisted package is that they are being referenced by classes within the whitelisted package. I call these "external classes". Yes, it's a bit weird, but the alternative can also give unwanted results. I may change the default on this in the future. For now, try enabling "strict mode":
https://github.com/lukehutch/fast-classpath-scanner/wiki/2.-Constructor#external-classes

Hey Luke
Not sure if you managed to take a look, I must have left those white lists in by mistake,
output.put("persistence.xml", new FileMatchContentsProcessorWithContext() {
			@Override
			public void processMatch(File classpathElt, String relativePath, byte[] fileContents) throws IOException {
				System.out.println("Matched persistence.xml : " + classpathElt.getCanonicalPath());
			}
		});

I'm matching on the path leaf "persistence.xml" and the others,
scan.matchFilenamePathLeaf(key, value);
The whitelisted package is "META-INF", I tried many variations of this to see if it would ever match.
		s.add("META-INF");
		s.add("dir:META-INF");

I've also whitelisted "WEB-INF" to see scanning in that directory when specified and this too doesn't get found.
Unfortunately even with the META-INF package white listed this file is never found.
The files have moved out of the root directory into their real-world locations, I can successfully read application.xml on the root path using the given fix. This one is related to searching for files in white listed packages with file match processors :)

@GedMarc you've got to look at the logs whenever something weird is happening... always take a look at the output of .verbose(). In this case you have:
2017-12-28T07:50:00.943-0700	FastClasspathScanner	Parsing scan spec
2017-12-28T07:50:00.944-0700	FastClasspathScanner	-- Only a leaf filename may be used with a "jar:" entry in the scan spec, got "lib/antlr-2.7.7.jar" -- ignoring
2017-12-28T07:50:00.944-0700	FastClasspathScanner	-- Ignoring extra text after "dir:" in scan spec entry: dir:META-INF
2017-12-28T07:50:00.944-0700	FastClasspathScanner	-- Ignoring scan spec entry with no effect: "dir:META-INF"
2017-12-28T07:50:00.944-0700	FastClasspathScanner	-- Ignoring extra text after "dir:" in scan spec entry: dir:lib
2017-12-28T07:50:00.944-0700	FastClasspathScanner	-- Ignoring scan spec entry with no effect: "dir:lib"
2017-12-28T07:50:00.944-0700	FastClasspathScanner	-- Ignoring extra text after "dir:" in scan spec entry: dir:resources
2017-12-28T07:50:00.944-0700	FastClasspathScanner	-- Ignoring scan spec entry with no effect: "dir:resources"
2017-12-28T07:50:00.944-0700	FastClasspathScanner	-- Only a leaf filename may be used with a "jar:" entry in the scan spec, got "lib/aop*.jar" -- ignoring
2017-12-28T07:50:00.944-0700	FastClasspathScanner	-- Whitelisted relative path prefixes:  [resources/, */xml/, */json/, */js/, lib/, test/]
2017-12-28T07:50:00.944-0700	FastClasspathScanner	-- Blacklisted relative path prefixes:  [sun/, javax/, java/]
2017-12-28T07:50:00.944-0700	FastClasspathScanner	-- Specifically-whitelisted classfiles: [META-INF.class]

So:

You need "jar:antlr-2.7.7.jar", not "jar:lib/antlr-2.7.7.jar"
"dir:" is a bit misleading, but it doesn't currently allow you to specify a directory. It just makes it so that directories are whitelisted but jars are not. (i.e. jars are not scanned).
META-INF is being recognized as a class name: Specifically-whitelisted classfiles: [META-INF.class]. Currently class names are recognized in the scan spec by an initial capital letter, since by Java convention, package names should all be lowercase. This is a very common exception to that, so I just checked that exception into the code.

In the next major version of FCS (probably 3.0), I'm going to do a major revamp of the API, and there will be separate calls for all of these things, e.g. .disableScanJars(), .whitelistPackage(...), .whitelistClass(...), etc.
Please try the git master version, and simply add "META-INF" to the scan spec as if it is a package.

META-INF can now be given as a package name for whitelisting in version 2.9.4. Please comment if it somehow doesn't work for you.


Restrict transaction witness malleability to only approved owners

Can we restrict tx that has possibility to be malleable, to only owners who we trust to do that.
John answer: It would require some fundamental changes to the transaction model to prevent third-party witness malleability in all cases, as it was designed to be malleated by third parties.

Lets bundle one more question toggether.
What about BlockProducer paying for additional cost because, he is the one that has control over it

In the rollup mode, the block producer will always pay for relaying blocks to L1. Those costs will far outweigh the costs to peers for storing the extra witness data.
In side-chain mode, this cost isn't currently captured at all. Perhaps we need some way to designate witnesses provided by the user vs added by third parties. That way if we wanted, we could implement schemes like deducting witness data created by the block producer from their coinbase award.


Updates for Calico v3.13.3

Description

Related issues/PRs

Todos

[ ] Tests
[ ] Documentation
[ ] Release note

Release Note

None required

Deploy preview for calico ready!
Built with commit f29b0dc29731b78834834cc0a3ae38bfb4bc7c15
https://deploy-preview-3453--calico.netlify.app


Create SECURITY.md

JATUPORN
Description:
Describe your changes.
Related issue:
Add link to the related issue.
Check list:

[x] Mark if documentation changes are required.
[x] Mark if tests were added or updated to cover the changes.

https://github.com/blockscout/blockscout/labels?sort=count-asc


[Feature request] Implement autocomplete for multiple tables without aliases

Problem
When using multiple joined tables, autocomplete for second and further tables is not available in main autocomplete suggestion list, only first table's fields are shown.
E.g.:
SELECT *
FROM table1 INNER JOIN table2 ON field1 = field2

When You try to use autocomplete instead of asterisk (not on asterisk, which expands asterisk to fields of table1) or inside ON clause (when You type f), fields are listed only for first table.
Solution
When using autocomplete, list of fields should show all available fields from all joined tables.
Duplication of fields should not be a concern, but it could be next feature to automatic insert table or alias for duplicated fields.
Alternatives
One way to use autocomplete is allowing to use relation between tables to create comparison for ON clause (using CTRL+SPACE after ON). It is not always sufficient when there are multiple relations.
Another way to use autocomplete for second and further tables is using aliases or full table with dot notation. It could be easier to use when implemented #7191.
E.g.:
SELECT *
FROM table1 INNER JOIN table2 ON field1 = table2.field2

Entering table2. then using autocomplete shows correctly fields for table2.

This issue is copy of https://github.com/dbeaver/dbeaver/issues/2230#issuecomment-333371082.

For me the autocomplete on JOIN statements used to work, but I have since started to work on Mac OSX and it's not working anymore. Version 6.3.4

will be fixed in #12335

Thanks a lot!


added binding for API endpoint:get messages read recipt

added a local function for getting a messages read recipt
reference https://zulip.com/api/get-read-receipts
proof of working:

Please fix the typos in the function name -- these should all match the URLs in the API documentation.
Check out the Zulip commit message guidelines for more details.

hi any updates on this @timabbott


ios device id changes each time, from 0.23

react-native run-ios

Every time run this command, the app install id changes. Version before 0.23 works well.

Can you provide some details? Is this a bug? What you mean by app install id? Do you mean device ID? Identifier for vendor?

Yes, device ID, iOS app well generate new one when reinstall.
It seems “react-native run-ios” will reinstall the app.
I'm using react-native-device-info to get the device ID, after upgrade rn, ID changes every time.

Checked the library and uniqueId is deviceIdentifierForVendor which will change every time you reinstall the app and I think that's the ID I think you might be using.
We could add a flag to run-ios to not install the app, just run, or check that if possible.
Is that the id you meant? Could you link to specific line in that library that changes for you?

That's the right ID I used.
Sorry, I don't known how to setup the script, node script is a difficulty.
I just work around by stored and restored the ID when app is starting to run.

Thanks - going to send a PR in a bit

I can't tell if there ever was a PR here. Since there's a workaround anyway and it's been idle for a while, I'm going to close this issue. I think it is fixed though. Feel free to reopen if someone can repro this bug from a new RN version.


add easybuild module

I also renamed the internal Python package and conda environment to avoid naming conflicts with mobie-utils-python.

I will merge this now, so the CI pipeline for the cluster points to the correct repository branch...


narrow: Use dict to map operator to by_* method in NarrowBuilder.

Updates the logic for identifying the method to use to extend the query for the given term from a narrow to use a dictionary that maps the operator string to the by_* method in the NarrowBuilder class.
Previously, the by_* method was determined by building a string based on the operator string and replacing dashes with underscores.
See relevant CZO conversation

Notes:

I went with adding the dict on the class instance in the __init__ method, so that the method is bound to self and any changes to the method names immediately are reflected in the mapped values.

An alternative would be to create the dict as a constant in the NarrowBuilder class with strings that match the by_* methods, and to use getattr in add_term to get the method. If the method name was changed, automated tests would catch the broken mapped string.

Self-review checklist

[ ] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[ ] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

This seems like a reasonable approach to making this explicit, merged, thanks @laurynmm!


docs(slider): a11y slider docs added to website

Reference #491
Make sure content matches: https://pages.github.ibm.com/IBMa/Carbon_Integration/Doc/Slider-doc.html

LGTM!