iOS: Building documentation (Docc) gives error coming from FirebaseCoreExtension while building with Xcode 16

Description
With latest Xcode Version 16.0 (16A242d), when building the project everything is fine but when I build the Documentation (Product > Build Documentation), I am getting this error via:

/.../Project/Pods/FirebaseCore/FirebaseCore/Sources/Public/FirebaseCore/FIRLoggerLevel.h:23:28: redefinition of 'FIRLoggerLevel'

Building the documentation of the same project with Xcode Version 15.4 works just fine, so the error must come from the new xcodebuild (Xcode 16.0 Build version 16A242d).
Here are the pods in the project:
pod 'Firebase/Crashlytics', '11.3.0'

Removing the Firebase/Crashlytics  pod from the project resolves the documentation build issue.
Reproducing the issue

Create Xcode project using Xcode 15.4 (do not use Xcode 16.0 as CocoaPods does not yet support it)
Close the Xcode project
Go to project directory and init and install pods: $ pod init
Open the Podfile and add pod 'Firebase/Crashlytics', '11.3.0'
Go to project directory and install the pods: $ pod install --repo-update
Open the newly created Xcode workspace
Make sure that the project target is selected
Build "docc" documentation by pressing "Product > Build Documentation" in the MacOS navigation bar.

You get errors from FirebaseCoreExtension during build.
Firebase SDK Version
11.3
Xcode Version
16.0
Installation Method
CocoaPods
Firebase Product(s)
Crashlytics
Targeted Platforms
iOS
Relevant Log Output
/Users/.../Project/Pods/FirebaseCore/FirebaseCore/Sources/Public/FirebaseCore/FIRLoggerLevel.h:23:28: redefinition of 'FIRLoggerLevel'

If using Swift Package Manager, the project's Package.resolved

Expand Package.resolved snippet

Replace this line with the contents of your Package.resolved.

If using CocoaPods, the project's Podfile.lock

Expand Podfile.lock snippet

PODS:
  - Firebase/CoreOnly (11.3.0):
    - FirebaseCore (= 11.3.0)
  - Firebase/Crashlytics (11.3.0):
    - Firebase/CoreOnly
    - FirebaseCrashlytics (~> 11.3.0)
  - FirebaseCore (11.3.0):
    - FirebaseCoreInternal (~> 11.0)
    - GoogleUtilities/Environment (~> 8.0)
    - GoogleUtilities/Logger (~> 8.0)
  - FirebaseCoreExtension (11.3.0):
    - FirebaseCore (~> 11.0)
  - FirebaseCoreInternal (11.3.0):
    - "GoogleUtilities/NSData+zlib (~> 8.0)"
  - FirebaseCrashlytics (11.3.0):
    - FirebaseCore (~> 11.0)
    - FirebaseInstallations (~> 11.0)
    - FirebaseRemoteConfigInterop (~> 11.0)
    - FirebaseSessions (~> 11.0)
    - GoogleDataTransport (~> 10.0)
    - GoogleUtilities/Environment (~> 8.0)
    - nanopb (~> 3.30910.0)
    - PromisesObjC (~> 2.4)
  - FirebaseInstallations (11.3.0):
    - FirebaseCore (~> 11.0)
    - GoogleUtilities/Environment (~> 8.0)
    - GoogleUtilities/UserDefaults (~> 8.0)
    - PromisesObjC (~> 2.4)
  - FirebaseRemoteConfigInterop (11.3.0)
  - FirebaseSessions (11.3.0):
    - FirebaseCore (~> 11.0)
    - FirebaseCoreExtension (~> 11.0)
    - FirebaseInstallations (~> 11.0)
    - GoogleDataTransport (~> 10.0)
    - GoogleUtilities/Environment (~> 8.0)
    - GoogleUtilities/UserDefaults (~> 8.0)
    - nanopb (~> 3.30910.0)
    - PromisesSwift (~> 2.1)
  - GoogleDataTransport (10.1.0):
    - nanopb (~> 3.30910.0)
    - PromisesObjC (~> 2.4)
  - GoogleUtilities/Environment (8.0.2):
    - GoogleUtilities/Privacy
  - GoogleUtilities/Logger (8.0.2):
    - GoogleUtilities/Environment
    - GoogleUtilities/Privacy
  - "GoogleUtilities/NSData+zlib (8.0.2)":
    - GoogleUtilities/Privacy
  - GoogleUtilities/Privacy (8.0.2)
  - GoogleUtilities/UserDefaults (8.0.2):
    - GoogleUtilities/Logger
    - GoogleUtilities/Privacy
  - nanopb (3.30910.0):
    - nanopb/decode (= 3.30910.0)
    - nanopb/encode (= 3.30910.0)
  - nanopb/decode (3.30910.0)
  - nanopb/encode (3.30910.0)
  - PromisesObjC (2.4.0)
  - PromisesSwift (2.4.0):
    - PromisesObjC (= 2.4.0)

DEPENDENCIES:
  - Firebase/Crashlytics (= 11.3.0)

SPEC REPOS:
  trunk:
    - Firebase
    - FirebaseCore
    - FirebaseCoreExtension
    - FirebaseCoreInternal
    - FirebaseCrashlytics
    - FirebaseInstallations
    - FirebaseRemoteConfigInterop
    - FirebaseSessions
    - GoogleDataTransport
    - GoogleUtilities
    - nanopb
    - PromisesObjC
    - PromisesSwift

SPEC CHECKSUMS:
  Firebase: 5c575140761e22324806f401e38c483d58db2dec
  FirebaseCore: 8542de610f35f86196ba26cdb2544565a5157c8e
  FirebaseCoreExtension: 30bb063476ef66cd46925243d64ad8b2c8ac3264
  FirebaseCoreInternal: ac26d09a70c730e497936430af4e60fb0c68ec4e
  FirebaseCrashlytics: ba7b6a55dc10393f6583d87d8600d0d3ab2671d8
  FirebaseInstallations: 58cf94dabf1e2bb2fa87725a9be5c2249171cda0
  FirebaseRemoteConfigInterop: c3a5c31b3c22079f41ba1dc645df889d9ce38cb9
  FirebaseSessions: 655ff17f3cc1a635cbdc2d69b953878001f9e25b
  GoogleDataTransport: aae35b7ea0c09004c3797d53c8c41f66f219d6a7
  GoogleUtilities: 26a3abef001b6533cf678d3eb38fd3f614b7872d
  nanopb: fad817b59e0457d11a5dfbde799381cd727c1275
  PromisesObjC: f5707f49cb48b9636751c5b2e7d227e43fba9f47
  PromisesSwift: 9d77319bbe72ebf6d872900551f7eeba9bce2851

PODFILE CHECKSUM: dd40a8909158eeaa7bf1342f0d7ee46699de9f64

COCOAPODS: 1.15.2

@VladimirAmiorkovFlutter Thanks for the report. We've reproduced and are working on a fix for the next release.

@paulb777
Is this fix included in the latest 11.4.0 release from 3 days ago?
I looked over the release notes and the change log and I do not find a reference to this issue ID, so it looks like it is not included event tho it was fixed over 3 weeks ago.
When can we expect this to be released, is there a way to track an issue's release?
Thank you.

@VladimirAmiorkovFlutter Yes this fix was included in 11.4.0.
It's shown more clearly in the Release Notes at https://firebase.google.com/support/release-notes/ios. The links you shared are lower level release notes from a PR by PR perspective in which you can see the PR that closed this issue.

Hi @paulb777 ,
Thank you for that.
In that case can we reopen this issue as it is not really fixed. It can be reproduced with the original steps.

Hmm, what Xcode version are you using? I'm not seeing those errors with 16.1

I am using the latest official Version 16.0 (16A242d).

Your previous issue was reproduced (and fixed) with a slightly different repro (using pod gen). When I follow exactly the repro steps, I see the ongoing issue with 11.4.0 I'll investigate.

Confirmed that #13993 fixes this. The plan is to include in the upcoming 11.5.0.

@paulb777 Thank you for the update, looking forward to 11.5.0


Chroma (duckdb+parquet) DB isn't saving the parquet files for persisted DB until application is killed

System Info
Chroma v0.2.36, python 3.9
Who can help?
No response
Information

[ ] The official example notebooks/scripts
[ ] My own modified scripts

Related Components

[ ] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[X] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction
`
settings = Settings(
chroma_db_impl='duckdb+parquet',
persist_directory="db",
anonymized_telemetry=False
)
        pages = self._load_single_document(file_path=file_path)
        docs = text_splitter.split_documents(pages)
        db = Chroma.from_documents(docs, embedding_function, client_settings=settings)

`
Expected behavior
All files for the database should be created in the db directory. However, the parquet files are not being created when db.from_documents() is called.
However, this is not happening. All the files in the index directory are created at this time.
Later, when the application is killed (flask application, so in between requests the DB is torn down and thus should be persisited), then the parquet files show up.

It is a good practice to explicitly persist
# Explicitl persist
db.persist()

Thank you. I added an explicit persist. Here's what I have debugging through the code:
db = Chroma.from_documents(...)   # After this line, I see the index folder created, but no parquet files.
db.persist() gives the following error: ValueError: You must specify a persist_directory oncreation to persist the collection.
But everything is being added to my persist directory, 'db'.
That seems like a bug, definitely not expected behaviour

Okay, seemingly, I need to call out a persist directory regardless if it is in the client Settings object or not. The following does NOT produce the error, and works as expected:
`
settings = Settings(
chroma_db_impl='duckdb+parquet',
persist_directory=os.path.join(directory, 'db'),
anonymized_telemetry=False
)
db = Chroma(embedding_function=embedding_function, client_settings=settings, persist_directory='db')
Sounds like a bug with Chroma, I'll post over on that project. Closing this

On second look, it appears the issue is in the Chroma wrapper. The following code in langchain/vectorstores/chroma.py seems to be the problem:
    def __init__(
        self,
        collection_name: str = _LANGCHAIN_DEFAULT_COLLECTION_NAME,
        embedding_function: Optional[Embeddings] = None,
        persist_directory: Optional[str] = None,
        client_settings: Optional[chromadb.config.Settings] = None,
        collection_metadata: Optional[Dict] = None,
        client: Optional[chromadb.Client] = None,
    ) -> None:
        """Initialize with Chroma client."""
        try:
            import chromadb
            import chromadb.config
        except ImportError:
            raise ValueError(
                "Could not import chromadb python package. "
                "Please install it with `pip install chromadb`."
            )

        if client is not None:
            self._client = client
        else:
            if client_settings:
                self._client_settings = client_settings
            else:
                self._client_settings = chromadb.config.Settings()
                if persist_directory is not None:
                    self._client_settings = chromadb.config.Settings(
                        chroma_db_impl="duckdb+parquet",
                        persist_directory=persist_directory,
                    )
            self._client = chromadb.Client(self._client_settings)

        self._embedding_function = embedding_function
        **self._persist_directory = persist_directory**
        self._collection = self._client.get_or_create_collection(
            name=collection_name,
            embedding_function=self._embedding_function.embed_documents
            if self._embedding_function is not None
            else None,
            metadata=collection_metadata,
        )

The bolded line indicates that the persist directory is only being set if that is in fact passed in, where as if it is used in the client.Settings object, I'd expect it to get unpacked and used. Is this expected behaviour?

@JackLeick
I did some investigation around this and feel your issue is valid. Following is the summary. Do let me know if it addresses your problem statement.
Issue
the persist_directory is not being set when it's passed in the client_settings object. This is likely because the persist_directory is only being set if it's passed as a separate argument to the Chroma constructor, and not when it's included in the client_settings object.
Code Fix
def __init__(
    self,
    collection_name: str = _LANGCHAIN_DEFAULT_COLLECTION_NAME,
    embedding_function: Optional[Embeddings] = None,
    persist_directory: Optional[str] = None,
    client_settings: Optional[chromadb.config.Settings] = None,
    collection_metadata: Optional[Dict] = None,
    client: Optional[chromadb.Client] = None,
) -> None:
    """Initialize with Chroma client."""
    try:
        import chromadb
        import chromadb.config
    except ImportError:
        raise ValueError(
            "Could not import chromadb python package. "
            "Please install it with `pip install chromadb`."
        )

    if client is not None:
        self._client = client
    else:
        if client_settings:
            self._client_settings = client_settings
            self._persist_directory = client_settings.persist_directory if client_settings.persist_directory else persist_directory
			#updated to set _persist_directory either from client_settings.persist_directory or #persist_directory(inp args). if persist_directory(inp args) as per "def persist()" must specify a #persist_directory which still holds good as per the original design
        else:
            self._client_settings = chromadb.config.Settings()
            if persist_directory is not None:
                self._client_settings = chromadb.config.Settings(
                    chroma_db_impl="duckdb+parquet",
                    persist_directory=persist_directory,
                )
                self._persist_directory = persist_directory
        self._client = chromadb.Client(self._client_settings)

    self._embedding_function = embedding_function
    self._collection = self._client.get_or_create_collection(
        name=collection_name,
        embedding_function=self._embedding_function.embed_documents
        if self._embedding_function is not None
        else None,
        metadata=collection_metadata,
    )

Change Summary:
The init method of the Chroma class has been updated to correctly handle the persist_directory parameter. Previously, persist_directory was only being set if it was passed as a separate argument to the Chroma constructor, and not when it was included in the client_settings object. The updated code now correctly sets persist_directory in all scenarios.
Addressed Scenarios:

persist_directory is provided in client_settings: If client_settings is provided and it includes persist_directory, then self._persist_directory is set to client_settings.persist_directory.

persist_directory is not provided in client_settings but is passed as an argument: If client_settings is provided but it does not include persist_directory, and persist_directory is passed as a separate argument, then self._persist_directory is set to the persist_directory argument.

client_settings is not provided, but persist_directory is passed as an argument: If client_settings is not provided and persist_directory is passed as a separate argument, then self._persist_directory is set to the persist_directory argument.

Neither client_settings nor persist_directory is provided: If neither client_settings nor persist_directory is provided, then self._persist_directory remains unset.

Thank you! This looks like a great fix. That will address my issue.

Issue has been addressed via commit : https://github.com/hwchase17/langchain/commit/a2830e3056e4e616160b150bf5ea212a97df2dc4


Error "message":"Invalid credential","error_cd":"AUT4034"

Hi,
Are you able to save the data? We are getting invalid credentials error.
Thanks
Kumar

same here. can you please suggest what might be causing this issue ?

The error code is changed now :
{
    "status_cd": "0",
    "error": {
        "message": "Invalid credential",
        "error_cd": "AUT4038"
    }
}

Can you please suggest

We are waiting for v0.3 api docs, v0.2 that we have implemented has become invalid

@gnalinikumar : did you find a resolution to the issue? Can you please help just in case


[OpenAI GPT3] ...

Extension
https://www.raycast.com/simicvm/openai-gpt3
Description
The extension won't pop back to root search even if I set it to 5 seconds in the settings. Other extensions work just fine, I think it's specific to this plugin.
Steps To Reproduce
No response
Current Behaviour
No response
Expected Behaviour
No response

Unfortunately I am unable to reproduce this issue, and without additional details I am not able to help.

Here’s a recording of the issue. I’ve set my raycast ‘popping back to root search’ time to immediately.
From: Marko Simic @.>
Date: Thursday, March 30, 2023 at 10:53 AM
To: raycast/extensions @.>
Cc: LeoLee53 @.>, Author @.>
Subject: Re: [raycast/extensions] [OpenAI GPT3] ... (Issue #5482)
Unfortunately I am unable to reproduce this issue, and without additional details I am not able to help.
—
Reply to this email directly, view it on GitHubhttps://github.com/raycast/extensions/issues/5482#issuecomment-1489609151, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A6QVUXSL4IOYGPZUZ3QK7K3W6TYSFANCNFSM6AAAAAAWCEHYDQ.
You are receiving this because you authored the thread.Message ID: @.***>

By the way, what further detail do you need? I’ll see if I can provide to help you solve the issue

发件人: Marko Simic @.>
发送时间: Thursday, March 30, 2023 10:53:22 AM
收件人: raycast/extensions @.>
抄送: LeoLee53 @.>; Author @.>
主题: Re: [raycast/extensions] [OpenAI GPT3] ... (Issue #5482)
Unfortunately I am unable to reproduce this issue, and without additional details I am not able to help.
―
Reply to this email directly, view it on GitHubhttps://github.com/raycast/extensions/issues/5482#issuecomment-1489609151, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A6QVUXSL4IOYGPZUZ3QK7K3W6TYSFANCNFSM6AAAAAAWCEHYDQ.
You are receiving this because you authored the thread.Message ID: @.***>


Rework example and recategorize methods

example reflects intended usage, categories are listed to aid understanding

Codecov Report
Patch and project coverage have no change.

Comparison is base (19dc125) 99.60% compared to head (b71f85d) 99.60%.

:exclamation: Current head b71f85d differs from pull request most recent head 09f79f7. Consider uploading reports for the commit 09f79f7 to get more accurate results

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@           Coverage Diff           @@
##           master     #245   +/-   ##
=======================================
  Coverage   99.60%   99.60%
=======================================
  Files         224      224
  Lines        3266     3266
=======================================
  Hits         3253     3253
  Misses         13       13

Impacted Files
Coverage Δ

rubico.js
100.00% <ø> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Too many scrolls is scrolls are visible in the browser

Double vertical scroll on the left
Horizontal scroll on the bottom

Chrome, Linux

@a-chabin https://github.com/browserslist/browserl.ist/pull/359 fixed only Horizontal scroll on the bottom, but we still have second but (just render any query and then reduce browser window height)


Deis Router does not allow for X-Forwarded-Proto to be "wss"

The deis router does not allow for a currently existing X-Forwarded-Proto to be "wss" for websockets. It instead rewrites the proto to be "http"

An example of this is using a whoami container to test:
`curl --verbose --header 'Host:whoami.example.com' --header 'X-Forwarded-Proto: wss' <IP_ADDRESS>

Rebuilt URL to: <IP_ADDRESS>/
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 80 (#0)

GET / HTTP/1.1
Host:whoami.example.com
User-Agent: curl/7.43.0
Accept: /
X-Forwarded-Proto: wss

< HTTP/1.1 200 OK
< Server: nginx/1.10.1
< Date: Tue, 30 Aug 2016 20:08:05 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 273
< Connection: keep-alive
< Vary: Accept-Encoding
<
Hostname: whoami-v2-cmd-7smgh
IP: <IP_ADDRESS>
IP: ::1
IP: <IP_ADDRESS>
IP: fe80::42:aff:fe02:340c
GET / HTTP/1.1
Host: whoami.example.com
User-Agent: curl/7.43.0
Connection: close
Accept: /
Connection: close
X-Forwarded-For: <IP_ADDRESS>
X-Forwarded-Proto: http
`


Re-order import so @grpc/proto-loader works

In this post, @mfateev told me to use this project with Node grpc to call the service.
I followed the tutorial but found this strange problem.
I added this project using git submodule to a folder called api and used the following code to load package definition. It fails out of the box but works with the changes in the pull request, which did nothing but re-order the import
const grpc = require('@grpc/grpc-js')
const protoLoader = require('@grpc/proto-loader');

const PROTO_PATH = './api/temporal/api/workflowservice/v1/service.proto'

const packageDefinition = protoLoader.loadSync(
    PROTO_PATH,
    {
        keepCase: true,
        longs: String,
        enums: String,
        defaults: true,
        oneofs: true,
        includeDirs: ['./api']
    }
)
const protoDescriptor = grpc.loadPackageDefinition(packageDefinition).temporal.api.workflowservice.v1

@sunshineo could you provide more context, how did you come up with the re-ordering and fixing the issue this way?
In Temporal Web we take a similar approach, except it uses an older grpc node module. I wonder why the newer @grpc/grpc-js only works after re-ordering

@sunshineo could you provide more context, how did you come up with the re-ordering and fixing the issue this way?
In Temporal Web we take a similar approach, except it uses an older grpc node module. I wonder why the newer @grpc/grpc-js only works after re-ordering

@feedmeapples It was error out saying cannot import, so I start to copy paste everything into one file piece by piece. And through pure luck I noticed the order matters. I agree it must be the new grpc library doing something weird. I'm still stuck after 2 hard days. Please help https://community.temporal.io/t/how-to-do-operations-tctl-can-do-from-any-programming-language/1256/4


Fix cairo-quartz context translate call

Fixes the issue reported in #105

This breaks the demo application for me.  Before this change, everything displays fine, after this, nothing is displayed.


Add user styles (corrected)

Allows user to add CSS scripts to a userstyles folder. Files will be automatically loaded as CSS, allowing users to update specific features of Fastmail via CSS

@joelekstrom LGTM? :)

Hi @mdbraber I'm sorry I totally forgot about this. Will try it out within a day or so!

@mdbraber Code looks great!
I have two minor change requests though:

I'd prefer the "User scripts..." and "User styles..." button to be centered in the settings view
The example in the README.txt doesn't change anything visibly for me, would be nice to have an example that has a visible impact


[BUG] PV using v2 engine cannot attach

Describe the bug (🐛 if you encounter this issue)
Hello. Trying out longhorn again. Immediately having an error with the v2 engine.
Error starting pvc-e8fa1df2-3a7c-4083-8211-bdabe0702228-e-05f2e230: failed to create instance: rpc error: code = Unknown desc = failed to start SPDK engine: rpc error: code = Unknown desc = error sending message, id 218381, method nvmf_create_subsystem, params {nqn.2023-01.io.longhorn.spdk:pvc-e8fa1df2-3a7c-4083-8211-bdabe0702228-e-05f2e230 true false 0 0 0}: {"code": -32603,"message": "Unable to create subsystem nqn.2023-01.io.longhorn.spdk:pvc-e8fa1df2-3a7c-4083-8211-bdabe0702228-e-05f2e230"}

[longhorn-instance-manager] time="2023-09-26T18:36:10Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-e8fa1df2-3a7c-4083-8211-bdabe0702228-e-05f2e230" error="unexpected output: [{HostNQN:nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-d05099fd7887 HostID:c0c39aba-c2db-4d29-81d8-50e57cd47ea1 Subsystems:[{Name:nvme-subsys1 NQN:nqn.2023-01.io.longhorn.spdk:pvc-e8fa1df2-3a7c-4083-8211-bdabe0702228-e-05f2e230 Paths:[{Name:nvme1 Transport:tcp Address:traddr=<IP_ADDRESS>,trsvcid=20001,src_addr=<IP_ADDRESS> State:live}]}]} {HostNQN:nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-d05099fd7887 HostID: Subsystems:[{Name:nvme-subsys0 NQN:nqn.2014.08.org.nvmexpress:19871987TPBF2105210040100089T-FORCE TM8FPZ001T Paths:[{Name:nvme0 Transport:pcie Address:0000:01:00.0 State:live}]}]}]"
[longhorn-instance-manager] time="2023-09-26T18:36:11Z" level=warning msg="failed to get subsystem for nvme device /dev/nvme1n1" error="unexpected output: [{HostNQN:nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-d05099fd7887 HostID:c0c39aba-c2db-4d29-81d8-50e57cd47ea1 Subsystems:[{Name:nvme-subsys1 NQN:nqn.2023-01.io.longhorn.spdk:pvc-e8fa1df2-3a7c-4083-8211-bdabe0702228-e-05f2e230 Paths:[{Name:nvme1 Transport:tcp Address:traddr=<IP_ADDRESS>,trsvcid=20001,src_addr=<IP_ADDRESS> State:live}]}]} {HostNQN:nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-d05099fd7887 HostID: Subsystems:[]}]"
[longhorn-instance-manager] time="2023-09-26T18:36:11Z" level=warning msg="failed to get subsystem for nvme device /dev/nvme0n1" error="unexpected output: [{HostNQN:nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-d05099fd7887 HostID:c0c39aba-c2db-4d29-81d8-50e57cd47ea1 Subsystems:[]} {HostNQN:nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-d05099fd7887 HostID: Subsystems:[{Name:nvme-subsys0 NQN:nqn.2014.08.org.nvmexpress:19871987TPBF2105210040100089T-FORCE TM8FPZ001T Paths:[{Name:nvme0 Transport:pcie Address:0000:01:00.0 State:live}]}]}]"

To Reproduce

Setup Longhorn V2
Add storage block devices
Create PVC

Expected behavior
PVC attaches
Support bundle for troubleshooting

Environment

Longhorn version: 1.15.1
Installation method (e.g. Rancher Catalog App/Helm/Kubectl): kubectl
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: Kubeadm

Number of management node in the cluster: 1
Number of worker node in the cluster: 5

Node config

OS type and version: Arch Linux
Kernel version: 6.1.54-1-lts
CPU per node: ~20
Memory per node: ~128
Disk type(e.g. SSD/NVMe/HDD): NVMe
Network bandwidth between the nodes: 4gbps (LACP)

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): Baremetal
Number of Longhorn volumes in the cluster: 1
Impacted Longhorn resources:

Volume names:

supportbundle_733adbbc-a668-4738-bbb3-87b5326d9b17_2023-09-26T18-57-23Z.zip

cc @derekbit @shuo-wu

The output of nvme list-subsys -o json looks weird in this case.
One has hostnqn nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-d05099fd7887 and HostID c0c39aba-c2db-4d29-81d8-50e57cd47ea1. The other one has same hostnqn but empty HostID.
What's the nvme cli version on your longhorn node?
cc @shuo-wu @DamiaSan

Testing with https://archive.archlinux.org/packages/n/nvme-cli/nvme-cli-1.16-2-x86_64.pkg.tar.zst when I'm back at my desk, thanks!

Works :+1:

@sfxworks
If you are available, can you help to go to one of the instance-manager pods, and then build nvme-cli v2.5
zypper install -y python3 meson python3-pip
git clone -b v2.5 https://github.com/linux-nvme/nvme-cli.git
cd nvme-cli
meson setup --force-fallback-for=libnvme .build
meson compile -C .build
meson install -C .build

Show the list-subsys output
nvme list-subsys -o json

I would like to check what the output is when using the nvme-cli in the container.
Thank you.

I think to have the same issue testing the engine V2.
[longhorn-manager-m4k4n] time="2023-10-17T12:51:00Z" level=error msg="Failed to sync Longhorn VolumeAttachment longhorn-system/pvc-380cb239-1678-4315-9013-32bf973867b6" controller=longhorn-volume-attachment error="longhorn-volume-attachment: failed to sync VolumeAttachment longhorn-system/pvc-380cb239-1678-4315-9013-32bf973867b6: Operation cannot be fulfilled on volumeattachments.longhorn.io \"pvc-380cb239-1678-4315-9013-32bf973867b6\": the object has been modified; please apply your changes to the latest version and try again" node=sole-metal111
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:01Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:01Z" level=warning msg="Failed to get devices for address <IP_ADDRESS>:20225 and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:02Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:03Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:04Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:05Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:06Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:07Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:08Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:09Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:10Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:11Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:12Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:13Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:14Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:15Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:16Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:17Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:18Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:19Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:20Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:21Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:22Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:23Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:24Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:25Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:26Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:27Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:28Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:29Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:30Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:31Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:32Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:33Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:34Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:35Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:36Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:37Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:38Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:39Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:40Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:41Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:42Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:43Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:44Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:45Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:46Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:47Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:48Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:49Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:50Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:51Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:52Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:53Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:54Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:55Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:56Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:57Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:58Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:51:59Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:52:00Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [longhorn-instance-manager] time="2023-10-17T12:52:01Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[longhorn-manager-pxr85] time="2023-10-17T12:52:02Z" level=error msg="Error syncing Longhorn engine" controller=longhorn-engine engine=longhorn-system/pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4 error="failed to sync engine for longhorn-system/pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4: failed to create instance: rpc error: code = Unknown desc = failed to start SPDK engine: rpc error: code = Unknown desc = failed to load device info after starting NVMe initiator pvc-380cb239-1678-4315-9013-32bf973867b6: json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32" node=sole-metal107
[longhorn-manager-pxr85] time="2023-10-17T12:52:02Z" level=info msg="Event(v1.ObjectReference{Kind:\"Engine\", Namespace:\"longhorn-system\", Name:\"pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4\", UID:\"e23cc650-ef82-4621-9366-3073fe7219cf\", APIVersion:\"longhorn.io/v1beta2\", ResourceVersion:\"9538895\", FieldPath:\"\"}): type: 'Warning' reason: 'FailedStarting' Error starting pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4: failed to create instance: rpc error: code = Unknown desc = failed to start SPDK engine: rpc error: code = Unknown desc = failed to load device info after starting NVMe initiator pvc-380cb239-1678-4315-9013-32bf973867b6: json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"
[longhorn-manager-pxr85] time="2023-10-17T12:52:02Z" level=warning msg="Instance pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4 is state error, error message: "
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [2023-10-17 12:52:06.607022] subsystem.c: 228:spdk_nvmf_subsystem_create: *ERROR*: Subsystem NQN 'nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4' already exists
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [2023-10-17 12:52:06.607067] nvmf_rpc.c: 416:rpc_nvmf_create_subsystem: *ERROR*: Unable to create subsystem nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4
[longhorn-manager-pxr85] time="2023-10-17T12:52:06Z" level=error msg="Error syncing Longhorn engine" controller=longhorn-engine engine=longhorn-system/pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4 error="failed to sync engine for longhorn-system/pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4: failed to create instance: rpc error: code = Unknown desc = failed to start SPDK engine: rpc error: code = Unknown desc = error sending message, id 1100698, method nvmf_create_subsystem, params {nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4    true false 0 0 0}: {\"code\": -32603,\"message\": \"Unable to create subsystem nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4\"}" node=sole-metal107
[longhorn-manager-pxr85] time="2023-10-17T12:52:06Z" level=info msg="Event(v1.ObjectReference{Kind:\"Engine\", Namespace:\"longhorn-system\", Name:\"pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4\", UID:\"e23cc650-ef82-4621-9366-3073fe7219cf\", APIVersion:\"longhorn.io/v1beta2\", ResourceVersion:\"9539698\", FieldPath:\"\"}): type: 'Warning' reason: 'FailedStarting' Error starting pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4: failed to create instance: rpc error: code = Unknown desc = failed to start SPDK engine: rpc error: code = Unknown desc = error sending message, id 1100698, method nvmf_create_subsystem, params {nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4    true false 0 0 0}: {\"code\": -32603,\"message\": \"Unable to create subsystem nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4\"}"
[longhorn-manager-pxr85] time="2023-10-17T12:52:07Z" level=warning msg="Instance pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4 is state error, error message: "
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [2023-10-17 12:52:10.616850] subsystem.c: 228:spdk_nvmf_subsystem_create: *ERROR*: Subsystem NQN 'nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4' already exists
[instance-manager-414c632ae0900aaddb697e7f2fcc1eab] [2023-10-17 12:52:10.616892] nvmf_rpc.c: 416:rpc_nvmf_create_subsystem: *ERROR*: Unable to create subsystem nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4
[longhorn-manager-pxr85] time="2023-10-17T12:52:10Z" level=error msg="Error syncing Longhorn engine" controller=longhorn-engine engine=longhorn-system/pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4 error="failed to sync engine for longhorn-system/pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4: failed to create instance: rpc error: code = Unknown desc = failed to start SPDK engine: rpc error: code = Unknown desc = error sending message, id 1100712, method nvmf_create_subsystem, params {nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4    true false 0 0 0}: {\"code\": -32603,\"message\": \"Unable to create subsystem nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4\"}" node=sole-metal107

# nvme list-subsys -o json
{
  "Subsystems" : [
    {
      "Name" : "nvme-subsys0",
      "NQN" : "nqn.1994-11.com.samsung:nvme:PM9A3:2.5-inch:S6CKNN0TB15862      ",
      "Paths" : [
        {
          "Name" : "nvme0",
          "Transport" : "pcie",
          "Address" : "0000:c1:00.0",
          "State" : "live"
        }
      ]
    }
  ]
}

~# lsblk
NAME              MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
loop0               7:0    0  63.3M  1 loop /snap/core20/1852
loop1               7:1    0 111.9M  1 loop /snap/lxd/24322
loop2               7:2    0  49.8M  1 loop
loop3               7:3    0  40.8M  1 loop /snap/snapd/20092
loop4               7:4    0  63.5M  1 loop /snap/core20/2015
loop5               7:5    0  40.9M  1 loop /snap/snapd/20290
sda                 8:0    0 446.1G  0 disk
├─sda1              8:1    0   512M  0 part /boot/efi
└─sda2              8:2    0 445.6G  0 part
  └─vgroot-lvroot 253:0    0 445.6G  0 lvm  /
nvme0n1           259:0    0     7T  0 disk

Environment
Longhorn version: 1.15.1
Installation method (e.g. Rancher Catalog App/Helm/Kubectl): Helm
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: RKE2
    Number of management node in the cluster: 3
    Number of worker node in the cluster: 20
Node config
    OS type and version: Ubuntu 22.04.3
    Kernel version: 5.15.0-86-generic
    CPU per node: 64
    Memory per node: 256G
    Disk type(e.g. SSD/NVMe/HDD): NVMe
    Network bandwidth between the nodes: 100G (LACP)
Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): Baremetal
Number of Longhorn volumes in the cluster: 1

I think to have the same issue testing the engine V2.

What version of nvme cli do you have? Have you tried to use nvme v1.16?

@DamiaSan yes exactly.
# apt list | grep nvme
libnvme-dev/jammy-updates 1.3-1~ubuntu22.04 amd64
libnvme1/jammy-updates 1.3-1~ubuntu22.04 amd64
nvme-cli/jammy-updates,now 1.16-3ubuntu0.1 amd64 [installed]
nvme-stas/jammy-updates 2.2.1-1~ubuntu22.04 all
python3-nvme/jammy-updates 1.3-1~ubuntu22.04 amd64

Based on the error log:
time="2023-10-17T12:51:52Z" level=warning msg="Failed to get devices for address : and nqn nqn.2023-01.io.longhorn.spdk:pvc-380cb239-1678-4315-9013-32bf973867b6-e-cf77d2a4" error="json: cannot unmarshal number<PHONE_NUMBER>8 into Go struct field NvmeDevice.MaximumLBA of type int32"

It seems that this bug is triggered


Allow background styling

When generating the pdf-document, the PDFOptions do not specify the "PrintBackground" option. By default this is set to false.
This setting prevents us to apply background-styling on headers. Is is possible to make it configurable?

Sure, could you please contribute a pull request?

@MaxMelcher, see #92.

Thanks @bvermolen!


fix: skip indirect go.mod deps

With Go 1.17, go.mod has way too many indirect dependencies and we don't
need to push them to the release notes.
Signed-off-by: Andrey Smirnov<EMAIL_ADDRESS>

/approve

/m --ff


Adding the Ocetrac example page

The Ocetrac is a very useful tool for tracking the any signal (marine heatwave) in this case. A notebook that is focusing on this would be helpful. And a further development of the Marine heatwave portal based on this might be of possibility.

Ocetrac https://ocetrac.readthedocs.io/en/latest/index.html and GitHub repo


Simplify adding widget to a pane

To add a widget to a pane we need to pass a reference to the pane's page as a parameter to the widget's builder.

Why?
Suggestion:  Simply pass a reference to the pane, let the widget's builder call the get() method
Instead of
lecui::widgets::label_builder label(pane_one.get(), "my_label");
allow
lecui::widgets::label_builder label(pane_one, "my_label");

Why?

The reason is that we didn't want builders knowing anything about each other. In the interface, containers don't know about widgets, and widgets don't know specific containers, they only know about a page. All containers as different as they are have a page whose reference we then fetch using .get()).
While it is certainly possible to allow
lecui::widgets::label_builder label(pane_one, "my_label");
this would mean we'd have to allow the same for all other containers since pane_one in this code is actually a builder. The result is that all widgets would know about all container builders and everything gets entangled and we thought this would lead to a more complicated app.
So while it is certainly annoying to call .get() we thought it's a smaller price to pay than to have everything coupled.
Then again, the example code could use some improvement, because calling the builder pane_one is a bit dubious. Perhaps we can improve the example code to something like this:
// add a pane to the "home page"
lecui::containers::pane_builder pane_one_builder(home, "pane_one");
pane_one_builder().rect().width(300.f).height(300.f);

auto& pane_one = pane_one_builder.get();

// add label to "pane_one"
auto& label = lecui::widgets::label_builder(pane_one, "my_label")();
label
    .text("This is a simple label and it's in a pane and we're going to make it long deliberately")
    .rect()
    .width(pane_one.size().get_width())
    .height(20.f);

What do you think?

Thank you for the suggestion. We are working on hiding the builders entirely and this will simplify the process as you suggested.

Suggestion: Simply pass a reference to the pane

This has since been implemented here 2544ae0d29f57a48a81d5ae53b0a05e873d34883. Thank you for the suggestion.
For example, the following code:
auto& home = _page_man.add("home");

auto& pane = containers::pane::add(home);
pane.rect().left(10.f).top(10.f).width(home.size().get_width() - 20.f).height(150.f);

auto& slider = widgets::slider::add(pane);
slider
    .value(68.f)
    .events().slider = [this](float value) { message("Value set to " + std::to_string(value)); };

auto& label = widgets::label::add(pane);
label
    .text("Slider widget example")
    .rect().height(30.f).snap_to(slider.rect(), rect::snap_type::bottom, 10.f);

Produces this:

The lines of interest here are the following:
auto& pane = containers::pane::add(home);
...
auto& slider = widgets::slider::add(pane);
auto& label = widgets::label::add(pane);


[forge-expansion-panel] add an option to support not rendering the expanded content until expanded

Describe the feature:
The forge-expansion-panel should provide an option to not render the expanded content until the expansion panel is actually expanded. In many scenarios, the default behavior may be fine, but in other scenarios where a lot of content might be getting shown in here, it becomes problematic.
Describe any alternatives you've considered:
In Angular, we've gotten around this by providing an *ngIf on nested context and "caching" whether or not it's been expanded once or before to persist it in the DOM. There are some qwirks to this though if you want animations to work, since *ngIf could result in animations not working at all. To get around this, we had to *ngIf it in and force change detection in the beforeOpen like so:
public beforeOpen = (index: number) => {
    this.expandedCache[index] = true;
    this._changeDetectorRef.detectChanges();
};

This ensured that the content to expand was in the DOM before the expansion animation happened.
Additional context:
In our specific scenario, we are rendering HTML and working with a data set where customers could have hundreds of expansion panels, each of which could have A LOT of text to parse as HTML.

@conner-fallone I don't believe this is something that the Forge expansion panel can be responsible for unfortunately because it doesn't own/control the DOM you are providing. The same issue exists with the native <details> element because it is just controlling when the browser renders your content visibly, but the actual elements are still active in the DOM when collapsed.
Your workaround is something I would expect you to need to do unfortunately, and I understand it is a pain. I've thrown around the idea in the past to create an Angular-specific version of the Forge expansion panel so that it can tie into the lifecycle of Angular to not insert the content into the DOM until just before it is expanded. As you've noted, the content needs to be in the DOM before the animations are triggered since it uses JavaScript to compute the height of the content to expand to dynamically.
Unless I'm misunderstanding, or if you have any suggestions how this may be implemented in a framework-agnostic way, then our hands are tied on this one and we may need to close this issue.

@conner-fallone I don't believe this is something that the Forge expansion panel can be responsible for unfortunately because it doesn't own/control the DOM you are providing. The same issue exists with the native <details> element because it is just controlling when the browser renders your content visibly, but the actual elements are still active in the DOM when collapsed.
Your workaround is something I would expect you to need to do unfortunately, and I understand it is a pain. I've thrown around the idea in the past to create an Angular-specific version of the Forge expansion panel so that it can tie into the lifecycle of Angular to not insert the content into the DOM until just before it is expanded. As you've noted, the content needs to be in the DOM before the animations are triggered since it uses JavaScript to compute the height of the content to expand to dynamically.
Unless I'm misunderstanding, or if you have any suggestions how this may be implemented in a framework-agnostic way, then our hands are tied on this one and we may need to close this issue.

Agreed that there is probably not much we can do at this level, as I currently do not have any suggestions. Should i open it on the forge-angular repo for further discussion? I do think having this flexibility would be valuable.


Apply PEP8 Guide Style

Code formatting for more pythonic code 🐍 according to PEP8

The double spacing between functions is unnecessary and making it difficult for me to see what you're actually changing because the entire code base is being highlighted.
I'll merge the minor refactoring and source code encoding though.

The double spacing between functions is good practice, take a look to PEP8.
I only apply code refactorization with advance IDE like Pycharm. All the lines are affected because the good practices recommend use 4 spaces in order to tabs.
The code formating doesn't change anything and the behaviour is the same as before. Please merge all the changes.
Thanks in advance.

I need to do this because I have un mind to add code reviewer like Codacy and add testing with Travis CI.
Codacy is static analyzer checking security, code style, cyclometric measurements, unused code...


ENH: Add tool to lint diffs

Reference issue
None
What does this implement/fix?
As discussed on the mailing list:
https://mail.python.org/pipermail/scipy-dev/2020-January/023927.html
we can move towards better code style without causing disruption
by adding extra lint checks for diffs. This adds a script that
lints the diff between the tip of the current branch and where it
split off of master.
Additional information
Currently only one new check is enabled (whitespace after ,).
Do we want others?

It would be cool to activate this as part of runtests.py. Does it run on code that is changed but not checked in yet (git diff)?

Hooked it into runtests with the --pep8 flag. It does run on unstaged/staged but not committed changes.

I think this is ready for review. I imagine that we can bikeshed on which rules to enable indefinitely, so upon further reflection I suggest that we continue that discussion elsewhere. Once we can lint diffs it's very easy to enable new checks anyway.

Hooked it into runtests with the --pep8 flag. It does run on unstaged/staged but not committed changes.

So what's the recommended workflow then? If previously it were something like
... hack hack hack ...
$ python runtests.py
... tests pass
$ git commit
$ git push

when is the time for $ python runtests.py --pep8 and on what branch etc?

On Feb 2, 2020, at 10:59 AM, Paul van Mulbregt<EMAIL_ADDRESS>wrote:

@pvanmulbregt commented on this pull request.
In tools/lint_diff.py:

branch_commits = rev_list('HEAD', 100)

master_commits = set(rev_list('master', 100))
IIUC this is looking back 100 commits on the branch. Do we have stats on the number of commits per PR? My expectation is that 100 is reasonable, but maybe there have been some PRs with a lot of commits??
And also looking back on 100 commits on master. I wonder how many PRs branched more than 100 commits ago, and yet don't need a rebase to avoid conflicts? This actually does seem possible given the elapsed time to review and merge PRs.

I’ve had pr’s a couple hundred and in one case a couple thousand behind master. The latter case is less common I’d guess but can happen.
I was thinking that if the plan is to stitch into runtests.py then adding an argument for the number of commits to look back with whatever as the sensible default. It sounds like it is used by pep8 flag so what I was thinking is:
Python runtests.py -v pep8 -n 100
For going back 100 (example).
Python runtests.py -v pep8
As example using whatever is the default.
But if we dont want to have that in the options then we will use whatever the default is always, often it will be too long and occasionally it will be too short.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

Hooked it into runtests with the --pep8 flag. It does run on unstaged/staged but not committed changes.

So what's the recommended workflow then? If previously it were something like
... hack hack hack ...
$ python runtests.py
... tests pass
$ git commit
$ git push

when is the time for $ python runtests.py --pep8 and on what branch etc?

IIUC the workflow would be something

... hack hack hack ...
$ python runtests.py
... tests pass
$ python runtests.py --pep8
... linter passes
$ git commit
$ git push
I'm not sure though. If the two `runtest.py` calls could be a single call (running tests and linting) would be even better, I'm always forgetting to lint.

It looks like the capture_output argument was added to subprocess.run after python 3.6 in 3.7
Compare
https://docs.python.org/3.6/library/subprocess.html
With
https://docs.python.org/3.7/library/subprocess.html
Are we still supporting 3.6?
-Lucas Roberts

On Feb 3, 2020, at 8:24 AM<EMAIL_ADDRESS>wrote:

On Feb 2, 2020, at 10:59 AM, Paul van Mulbregt<EMAIL_ADDRESS>wrote:

@pvanmulbregt commented on this pull request.
In tools/lint_diff.py:

branch_commits = rev_list('HEAD', 100)

master_commits = set(rev_list('master', 100))
IIUC this is looking back 100 commits on the branch. Do we have stats on the number of commits per PR? My expectation is that 100 is reasonable, but maybe there have been some PRs with a lot of commits??
And also looking back on 100 commits on master. I wonder how many PRs branched more than 100 commits ago, and yet don't need a rebase to avoid conflicts? This actually does seem possible given the elapsed time to review and merge PRs.

I’ve had pr’s a couple hundred and in one case a couple thousand behind master. The latter case is less common I’d guess but can happen.
I was thinking that if the plan is to stitch into runtests.py then adding an argument for the number of commits to look back with whatever as the sensible default. It sounds like it is used by pep8 flag so what I was thinking is:
Python runtests.py -v pep8 -n 100
For going back 100 (example).
Python runtests.py -v pep8
As example using whatever is the default.
But if we dont want to have that in the options then we will use whatever the default is always, often it will be too long and occasionally it will be too short.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

Are we still supporting 3.6?

Yes, SciPy 1.5.0 will be Python 3.6+.

Ok, CI is green (Python 3.6 compat issues fixed). Re the workflow question, yes it is
$ python runtests.py
...
# python runtests.py --pep8
...

but I believe it has always been that way. Perhaps it makes sense to include --pep8 as part of the default invocation, but that is I believe orthogonal to the present change.
If the number of commits that we search back looking for the branch point turns out to be too small, then we can simply bump it. Note that including it as an option does not really help because you won't be able to bump it in CI without a PR doing the bumping anyway.

Ok, CI is green (Python 3.6 compat issues fixed). Re the workflow question, yes it is
$ python runtests.py
...
$ python runtests.py --pep8
...

but I believe it has always been that way. Perhaps it makes sense to include --pep8 as part of the default invocation, but that is I believe orthogonal to the present change.

Agreed.

If the number of commits that we search back looking for the branch point turns out to be too small, then we can simply bump it. Note that including it as an option does not really help because you won't be able to bump it in CI without a PR doing the bumping anyway.

I understand now. I had previously understood the intent was for developers. What you say definitely makes sense for CI tool-thanks @person142
Left two comments, both non-blocking and marked as approved.

This got left unmerged, looks good to go - rerunning CI to check if it still passes.

Rebased... let's see if that helps.

Let's give this a go! Thanks @person142, all

🤞 let's hope it works as expected.


Ability to disable a conformity check

Coverage increased (+0.05%) to 96.635% when pulling 55af995cdf1017bdbf1b65b75fd1dc3d1ea16aaf on disable_conformity into 84a85ae6498350b08b17df07f2d9206e230044e9 on main.


Not able to see the blue button in this step: Create custom integration

Describe the bug
This is not a product bug; rather a lack of recent doc.
The instructions for creating a custom integration seem to have changed.
How do I create a bot for workplace?
I am not able to see the blue button in this step: Create custom integration
https://docs.microsoft.com/en-us/azure/bot-service/bot-service-channel-connect-facebook?view=azure-bot-service-4.0#create-a-custom-integration
I only see this:

Also, as a general feedback, I find it useful to have Workplace in a separate page from the Facebook messenger (for easier reading).
Thank you!

Hi, I created a new Workplace trial and I have visible the admin panel now.
The UI is changed a bot, but I see the button "Create custom integration" [all good].

I still find it useful to separate the Facebook messenger integration from the Facebook workplace integration, for better following the steps. Thank you.

Following up with a dev. Sorry for the long silence on this issue.

The corrected article will be available in the next documentation update.


WIP: fix memory leaks

fixes #33

So, SCIP.jl just never called the freeSCIP, and SCIP's memory was accumulating. After this is fixed now, there's still memory usage accumulating, even after manual gc() calls. But the memory usage is reduced at least partially.
We used SCIP's internal memory debugging tools (BMSgetMemoryUsed) and found that, none of the memory tracked by SCIP/BMS is left behind, after a single SCIP instance is created and the Julia wrapper gc'ed. When creating multiple SCIP instances, they share a "memory shell" but are gc'ed individually. Therefore, the reported memory usage is reduced step-by-step, but in the end, completely.
However, according to top, not all of the memory is freed properly. Also, valgrind found lots of lost/reachable memory:
==7748== LEAK SUMMARY:
==7748==    definitely lost: 40 bytes in 1 blocks
==7748==    indirectly lost: 0 bytes in 0 blocks
==7748==      possibly lost: 6,368,351 bytes in 66,227 blocks
==7748==    still reachable: 44,119,928 bytes in 93,188 blocks
==7748==         suppressed: 0 bytes in 0 blocks
==7748== Reachable blocks (those to which a pointer was found) are not shown.
==7748== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==7748==
==7748== For counts of detected and suppressed errors, rerun with: -v
==7748== Use --track-origins=yes to see where uninitialised values come from
==7748== ERROR SUMMARY: 3976 errors from 2472 contexts (suppressed: 0 from 0)

So far, we didn't see any valgrind warnings that point to SCIP.jl code.

@mlubin: how can we go about debugging this properly?

You can use memory profiling to look at the places where Julia allocates memory, this might point in the right direction.

For what it's worth, I tried using the memory profiling. It seems to track only the lines in the top-level source file, so the output is not so helpful in our case (see gist:
        - function leaky(solver, d)
        -     m = Model(solver=solver)
        0     @variable(m, x[1:d] >= 0)
        0     solve(m)
        - end

It seems to track only the lines in the top-level source file

It creates .mem files for all julia code used. Look inside SCIP.jl/src.

 I suspected that more files would be created, but the git status of SCIP.jl was clean, so I assumed that no extra files are there. Of course, *.jl.mem is in .gitignore :confused:

I still find it difficult to learn from the .mem output.
I ran another experiment, printing the output of SCIP's memory shell.
It shows the same behavior as discussed above: If several SCIP instances are allocated, and some are freed, the SCIP memory shell still shows the memory of the remaining ones. What is interesting, is that the actual calls to the finalizer seems to be somewhat delayed.
If SCIPs are created (and solved and go out of scope?) in a loop, none of them are freed until after the loop is finished and I force gc(). Consequently, the memory usage accumulates:
run 3x
force garbage collection.
-- calling CSIPfreeModel -- 0 mem used
-- calling CSIPfreeModel -- 19043000 mem used
-- calling CSIPfreeModel -- 9521500 mem used

But if I call gc() during the loop, no memory accumulation takes place:
run 3x with gc() after every iteration
-- calling CSIPfreeModel -- 0 mem used
-- calling CSIPfreeModel -- 0 mem used
-- calling CSIPfreeModel -- 0 mem used
force garbage collection.

So maybe there is not even a bug, and Julia's garbage collection is just too lazy? Could it be made aware of the indirect memory usage through the C calls, in order to trigger more frequent GCs?

There are no guarantees on when garbage collection will run. Even a call to gc() is just a suggestion, I think.

Could it be made aware of the indirect memory usage through the C calls, in order to trigger more frequent GCs?

I'm not aware of a good solution to this, but it's worth posing the question on Discourse.

Note freemodel! was designed to give users more control over releasing solver resources.


handle some fuzz edge cases

Codecov Report
Patch and project coverage have no change.

Comparison is base (a9379da) 100.00% compared to head (064c05d) 100.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@             Coverage Diff             @@
##               1.5     #1073     +/-   ##
===========================================
  Coverage   100.00%   100.00%
===========================================
  Files           46        25     -21
  Lines         2652      1184   -1468
  Branches       396       134    -262
===========================================
- Hits          2652      1184   -1468

Flag
Coverage Δ

foundry
?

production
100.00% <ø> (ø)

reference
?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 40 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


feature: add a keyboard shortcut to toggle checkboxes

Is your feature request related to a problem? Please describe.
When working witch TODO lists with checkboxes it is convenient to toggle them on and off with a single keyboard shortcut without manually editing the source file.
Describe the solution you'd like
Introduce the shortcut that would toggle a checkbox on and off while the cursor is positioned anywhere in the line that contains this checkbox. An good example of the desired behavior is Markdown All-in-one extension for VS Code: https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one#keyboard-shortcuts
Describe alternatives you've considered
The current alternative is to simply edit the document itself which is not as efficient.
Additional information
No response

I get the idea, but this is primarily meant to be a UI plugin, as opposed to text objects / motions.
What you're looking for fits really well as a feature in: https://github.com/tadmccorkle/markdown.nvim (if it's not already supported).

OK, sorry for misunderstanding - I'm very much new to NeoVIM. You are right indeed and markdown.nvim contains the desired ability: https://github.com/tadmccorkle/markdown.nvim?tab=readme-ov-file#toggle-tasks. Thanks for pointing at it!

No worries at all, is a reasonable ask! Happy to help!