Wrong mountpoint generated on secondary storage for ISO on live migration

ISSUE TYPE

Bug Report

COMPONENT NAME
Secondary Storage

CLOUDSTACK VERSION
4.18.1

CONFIGURATION
KVM using 2x secondary storage and many primary storage mounts
OS / ENVIRONMENT
Ubuntu 22.04
SUMMARY
When attempting to live migrate a VM Cloudstack throws an error from libvirt with 'Invocation exception, caused by: com.cloud.utils.exception.CloudRuntimeException: Exception during migrate: org.libvirt.LibvirtException: Cannot access storage file '/mnt/667dc116-bc57-3e1f-b21a-96babdbcebd4/2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb.iso': No such file or directory'
This file exists on the secondary storage, and the agent successfully creates a new storage pool using the ISO, however it generates it under a new directory structure.
STEPS TO REPRODUCE
This seems to only happen with long-standing VM's, newly VM's seem to migrate fine. It's possible this only occurs when a storage pool needs to be created.
Here is the excerpt from the migration job from the management server;
2024-04-11 19:46:29,547 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Preparing to migrate [1] volumes for VM [VM instance {"id":25995,"instanceName":"i-2-25995-VM","type":"User","uuid":"ed56cab5-6657-407c-8def-0a7b0a50b568"}].
2024-04-11 19:46:29,629 DEBUG [o.a.c.s.i.TemplateDataFactoryImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) template 2695 is already in store:6, type:Image
2024-04-11 19:46:29,682 DEBUG [c.c.a.m.ClusteredAgentAttache] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Seq<PHONE_NUMBER>819743874648: Forwarding Seq<PHONE_NUMBER>819743874648:  { Cmd , MgmtId:<PHONE_NUMBER>2, via: 343(node181), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.PrepareForMigrationCommand":{"vm":{"id":"25995","name":"i-2-25995-VM","state":"Running","type":"User","cpus":"2","minSpeed":"2000","maxSpeed":"2000","minRam":"(4.00 GB)<PHONE_NUMBER>","maxRam":"(4.00 GB)<PHONE_NUMBER>","arch":"x86_64","os":"Other PV Virtio-SCSI (64-bit)","platformEmulator":"Other PV Virtio-SCSI (64-bit)","bootArgs":"","enableHA":"true","limitCpuUse":"false","enableDynamicallyScaleVm":"false","params":{"kvm.vnc.port":"5903","rootdisksize":"5","kvm.vnc.address":"<IP_ADDRESS>","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"ed56cab5-6657-407c-8def-0a7b0a50b568","enterHardwareSetup":"false","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"02de7a0f-bd65-4d7c-a56f-d70b233c7c1b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"a6d82da8-fdc3-38d0-8a3f-2798f270bbf8","name":"Primary2-Netapp-18","id":"59","poolType":"NetworkFilesystem","host":"<IP_ADDRESS>","path":"/csrp_primary2_ssd18","port":"2049","url":"NetworkFilesystem://<IP_ADDRESS>/csrp_primary2_ssd18/?ROLE=Primary&STOREUUID=a6d82da8-fdc3-38d0-8a3f-2798f270bbf8","isManaged":"false"}},"name":"ROOT-25995","size":"(5.00 GB)<PHONE_NUMBER>","path":"02de7a0f-bd65-4d7c-a56f-d70b233c7c1b","volumeId":"41742","vmName":"i-2-25995-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"59","id":"41742","deviceId":"0","bytesReadRate":"(0 bytes) 0","bytesWriteRate":"(0 bytes) 0","iopsReadRate":"(0 bytes) 0","iopsWriteRate":"(0 bytes) 0","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"diskSeq":"0","path":"02de7a0f-bd65-4d7c-a56f-d70b233c7c1b","type":"ROOT","_details":{"storageHost":"<IP_ADDRESS>","managed":"false","storagePort":"2049","storage.pool.disk.wait":"60","volumeSize":"(5.00 GB)<PHONE_NUMBER>"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"template/tmpl/2/2695/2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb.iso","origUrl":"http://url/grml64-full_2018.12.iso","uuid":"9159f256-d399-4b0f-8a0f-5cacff1c39f7","id":"2695","format":"ISO","accountId":"2","checksum":"237972093d31e89b35d92ab1dabe06ed","hvm":"true","displayText":"GRML 2018.12 rescue CD","imageDataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://<IP_ADDRESS>/csrp_secondary_ssd2","_role":"Image"}},"name":"2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb","guestOsType":"Linux/BSD 64-bit","size":"(672.25 MB) 704905216","hypervisorType":"None","bootable":"false","uniqueName":"2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb","directDownload":"false","deployAsIs":"false"}},"diskSeq":"3","type":"ISO"}],"nics":[{"deviceId":"0","networkRateMbps":"1000","defaultNic":"true","pxeDisable":"false","nicUuid":"96572d76-6eb8-47ae-a98c-ef3d9920fefc","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false","MacLearning":"false"},"dpdkEnabled":"false","uuid":"5942a519-a626-4f49-8e58-d6cbf9052131","ip":"<IP_ADDRESS>","netmask":"<IP_ADDRESS>","gateway":"<IP_ADDRESS>","mac":"02:00:45:06:00:18","dns1":"<IP_ADDRESS>","dns2":"<IP_ADDRESS>","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://1966","isolationUri":"vlan://1966","isSecurityGroupEnabled":"false","name":"cloudbr0","ip6Dns1":"","ip6Dns2":""}],"vcpuMaxLimit":"2","configDriveLabel":"config","configDriveLocation":"SECONDARY","guestOsDetails":{},"extraConfig":{}},"rollback":"false","wait":"0","bypassHostMaintenance":"false"}}] } to<PHONE_NUMBER>0
2024-04-11 19:46:30,164 DEBUG [c.c.a.t.Request] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Seq<PHONE_NUMBER>819743874648: Received:  { Ans: , MgmtId:<PHONE_NUMBER>2, via: 343(node181), Ver: v1, Flags: 110, { PrepareForMigrationAnswer } }
2024-04-11 19:46:30,185 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) VM instance {"id":25995,"instanceName":"i-2-25995-VM","type":"User","uuid":"ed56cab5-6657-407c-8def-0a7b0a50b568"} state transited from [Running] to [Migrating] with event [MigrationRequested]. VM's original host: Host {"id":488,"name":"node108","type":"Routing","uuid":"01132381-6797-4bb1-8aab-e9e990029c64"}, new host: Host {"id":343,"name":"node181","type":"Routing","uuid":"9def292d-3a1d-4fe2-b9ac-d4730944c59c"}, host before state transition: Host {"id":488,"name":"node108","type":"Routing","uuid":"01132381-6797-4bb1-8aab-e9e990029c64"}
2024-04-11 19:46:30,200 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Hosts's actual total CPU: 105600 and CPU after applying overprovisioning: 105600
2024-04-11 19:46:30,200 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) We are allocating VM, increasing the used capacity of this host:343
2024-04-11 19:46:30,200 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Current Used CPU: 70000 , Free CPU:35600 ,Requested CPU: 4000
2024-04-11 19:46:30,200 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Current Used RAM: (102.00 GB)<PHONE_NUMBER>48 , Free RAM:(13.78 GB)<PHONE_NUMBER>6 ,Requested RAM: (4.00 GB)<PHONE_NUMBER>
2024-04-11 19:46:30,200 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) CPU STATS after allocation: for host: 343, old used: 70000, old reserved: 0, actual total: 105600, total with overprovisioning: 105600; new used:74000, reserved:0; requested cpu:4000,alloc_from_last:false
2024-04-11 19:46:30,201 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) RAM STATS after allocation: for host: 343, old used: (102.00 GB)<PHONE_NUMBER>48, old reserved: (0 bytes) 0, total: (115.78 GB)<PHONE_NUMBER>24; new used: (106.00 GB)<PHONE_NUMBER>44, reserved: (0 bytes) 0; requested mem: (4.00 GB)<PHONE_NUMBER>,alloc_from_last:false
2024-04-11 19:46:30,204 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Host: 343 has cpu capability (cpu:48, speed:2200) to support requested CPU: 2 and requested speed: 2000
2024-04-11 19:46:30,204 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Checking if host: 343 has enough capacity for requested CPU: 4000 and requested RAM: (4.00 GB)<PHONE_NUMBER> , cpuOverprovisioningFactor: 1.0
2024-04-11 19:46:30,207 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Hosts's actual total CPU: 105600 and CPU after applying overprovisioning: 105600
2024-04-11 19:46:30,207 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) We need to allocate to the last host again, so checking if there is enough reserved capacity
2024-04-11 19:46:30,207 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Reserved CPU: 0 , Requested CPU: 4000
2024-04-11 19:46:30,207 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Reserved RAM: (0 bytes) 0 , Requested RAM: (4.00 GB)<PHONE_NUMBER>
2024-04-11 19:46:30,207 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) STATS: Failed to alloc resource from host: 343 reservedCpu: 0, requested cpu: 4000, reservedMem: (0 bytes) 0, requested mem: (4.00 GB)<PHONE_NUMBER>
2024-04-11 19:46:30,207 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Host does not have enough reserved CPU available, cannot allocate to this host.
2024-04-11 19:46:30,207 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Checking if host: 343 has enough capacity for requested CPU: 4000 and requested RAM: (4.00 GB)<PHONE_NUMBER> , cpuOverprovisioningFactor: 1.0
2024-04-11 19:46:30,209 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Hosts's actual total CPU: 105600 and CPU after applying overprovisioning: 105600
2024-04-11 19:46:30,209 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Free CPU: 35600 , Requested CPU: 4000
2024-04-11 19:46:30,209 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Free RAM: (13.78 GB)<PHONE_NUMBER>6 , Requested RAM: (4.00 GB)<PHONE_NUMBER>
2024-04-11 19:46:30,209 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Host has enough CPU and RAM available
2024-04-11 19:46:30,209 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) STATS: Can alloc CPU from host: 343, used: 70000, reserved: 0, actual total: 105600, total with overprovisioning: 105600; requested cpu:4000,alloc_from_last_host?:false ,considerReservedCapacity?: true
2024-04-11 19:46:30,209 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) STATS: Can alloc MEM from host: 343, used: (102.00 GB)<PHONE_NUMBER>48, reserved: (0 bytes) 0, total: (115.78 GB)<PHONE_NUMBER>24; requested mem: (4.00 GB)<PHONE_NUMBER>, alloc_from_last_host?: false , considerReservedCapacity?: true
2024-04-11 19:46:30,232 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Setting VLAN persistence to [{"1966":true}] as part of migrate command for VM [VM {id: "25995", name: "i-2-25995-VM", uuid: "ed56cab5-6657-407c-8def-0a7b0a50b568", type: "User"}].
2024-04-11 19:46:30,234 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Setting CPU shares to [379] as part of migrate command for VM [VM {id: "25995", name: "i-2-25995-VM", uuid: "ed56cab5-6657-407c-8def-0a7b0a50b568", type: "User"}].
2024-04-11 19:46:30,248 DEBUG [c.c.a.m.ClusteredAgentAttache] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Seq<PHONE_NUMBER>455906704593: Forwarding Seq<PHONE_NUMBER>455906704593:  { Cmd , MgmtId:<PHONE_NUMBER>2, via: 488(node108), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.MigrateCommand":{"vmName":"i-2-25995-VM","destIp":"<IP_ADDRESS>","migrateStorageManaged":"false","migrateNonSharedInc":"false","autoConvergence":"true","hostGuid":"46a313b7-5b71-3530-b361-f9717f16e75a-LibvirtComputingResource","isWindows":"false","vmTO":{"id":"25995","name":"i-2-25995-VM","state":"Running","type":"User","cpus":"2","minSpeed":"2000","maxSpeed":"2000","minRam":"(4.00 GB)<PHONE_NUMBER>","maxRam":"(4.00 GB)<PHONE_NUMBER>","arch":"x86_64","os":"Other PV Virtio-SCSI (64-bit)","platformEmulator":"Other PV Virtio-SCSI (64-bit)","bootArgs":"","enableHA":"true","limitCpuUse":"false","enableDynamicallyScaleVm":"false","params":{"kvm.vnc.port":"5903","rootdisksize":"5","kvm.vnc.address":"<IP_ADDRESS>","Message.ReservedCapacityFreed.Flag":"false"},"uuid":"ed56cab5-6657-407c-8def-0a7b0a50b568","enterHardwareSetup":"false","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"02de7a0f-bd65-4d7c-a56f-d70b233c7c1b","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"a6d82da8-fdc3-38d0-8a3f-2798f270bbf8","name":"Primary2-Netapp-18","id":"59","poolType":"NetworkFilesystem","host":"<IP_ADDRESS>","path":"/csrp_primary2_ssd18","port":"2049","url":"NetworkFilesystem://<IP_ADDRESS>/csrp_primary2_ssd18/?ROLE=Primary&STOREUUID=a6d82da8-fdc3-38d0-8a3f-2798f270bbf8","isManaged":"false"}},"name":"ROOT-25995","size":"(5.00 GB)<PHONE_NUMBER>","path":"02de7a0f-bd65-4d7c-a56f-d70b233c7c1b","volumeId":"41742","vmName":"i-2-25995-VM","accountId":"2","format":"QCOW2","provisioningType":"THIN","poolId":"59","id":"41742","deviceId":"0","bytesReadRate":"(0 bytes) 0","bytesWriteRate":"(0 bytes) 0","iopsReadRate":"(0 bytes) 0","iopsWriteRate":"(0 bytes) 0","hypervisorType":"KVM","directDownload":"false","deployAsIs":"false"}},"diskSeq":"0","path":"02de7a0f-bd65-4d7c-a56f-d70b233c7c1b","type":"ROOT","_details":{"storageHost":"<IP_ADDRESS>","managed":"false","storagePort":"2049","storage.pool.disk.wait":"60","volumeSize":"(5.00 GB)<PHONE_NUMBER>"}},{"data":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"template/tmpl/2/2695/2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb.iso","origUrl":"http://url/grml64-full_2018.12.iso","uuid":"9159f256-d399-4b0f-8a0f-5cacff1c39f7","id":"2695","format":"ISO","accountId":"2","checksum":"237972093d31e89b35d92ab1dabe06ed","hvm":"true","displayText":"GRML 2018.12 rescue CD","imageDataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://<IP_ADDRESS>/csrp_secondary_ssd2","_role":"Image"}},"name":"2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb","guestOsType":"Linux/BSD 64-bit","size":"(672.25 MB) 704905216","hypervisorType":"None","bootable":"false","uniqueName":"2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb","directDownload":"false","deployAsIs":"false"}},"diskSeq":"3","type":"ISO"}],"nics":[{"deviceId":"0","networkRateMbps":"1000","defaultNic":"true","pxeDisable":"false","nicUuid":"96572d76-6eb8-47ae-a98c-ef3d9920fefc","details":{"MacAddressChanges":"true","ForgedTransmits":"true","PromiscuousMode":"false","MacLearning":"false"},"dpdkEnabled":"false","uuid":"5942a519-a626-4f49-8e58-d6cbf9052131","ip":"<IP_ADDRESS>","netmask":"<IP_ADDRESS>","gateway":"<IP_ADDRESS>","mac":"02:00:45:06:00:18","dns1":"<IP_ADDRESS>","dns2":"<IP_ADDRESS>","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://1966","isolationUri":"vlan://1966","isSecurityGroupEnabled":"false","name":"cloudbr0","ip6Dns1":"","ip6Dns2":""}],"vcpuMaxLimit":"2","configDriveLabel":"config","configDriveLocation":"SECONDARY","guestOsDetails":{},"extraConfig":{}},"executeInSequence":"false","migrateDiskInfoList":[],"dpdkInterfaceMapping":{},"newVmCpuShares":"379","vlanToPersistenceMap":{"1966":"true"},"wait":"0","bypassHostMaintenance":"false"}}] } to<PHONE_NUMBER>4
2024-04-11 19:46:30,658 DEBUG [c.c.a.t.Request] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Seq<PHONE_NUMBER>455906704593: Received:  { Ans: , MgmtId:<PHONE_NUMBER>2, via: 488(node108), Ver: v1, Flags: 10, { MigrateAnswer } }
2024-04-11 19:46:30,658 INFO  [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Migration was unsuccessful.  Cleaning up: VM instance {"id":25995,"instanceName":"i-2-25995-VM","type":"User","uuid":"ed56cab5-6657-407c-8def-0a7b0a50b568"}
2024-04-11 19:46:30,680 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Releasing [1] volumes for VM [VM instance {"id":25995,"instanceName":"i-2-25995-VM","type":"User","uuid":"ed56cab5-6657-407c-8def-0a7b0a50b568"}] from host [Host {"id":343,"name":"node181","type":"Routing","uuid":"9def292d-3a1d-4fe2-b9ac-d4730944c59c"}].
2024-04-11 19:46:30,689 WARN  [c.c.a.AlertManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) alertType=[17] dataCenterId=[1] podId=[1] clusterId=[null] message=[Unable to migrate vm i-2-25995-VM from host node108 in zone CSRP01 and pod CSRP01-1].
2024-04-11 19:46:30,710 DEBUG [c.c.a.AlertManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Have already sent: 1 emails for alert type '17' -- skipping send email
2024-04-11 19:46:30,711 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) VM instance {"id":25995,"instanceName":"i-2-25995-VM","type":"User","uuid":"ed56cab5-6657-407c-8def-0a7b0a50b568"} is a [User], returning null for control Nic IP.
2024-04-11 19:46:30,730 DEBUG [c.c.a.m.ClusteredAgentAttache] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Seq<PHONE_NUMBER>819743874649: Forwarding Seq<PHONE_NUMBER>819743874649:  { Cmd , MgmtId:<PHONE_NUMBER>2, via: 343(node181), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","forceStop":"false","vlanToPersistenceMap":{"1966":"true"},"volumesToDisconnect":[],"vmName":"i-2-25995-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] } to<PHONE_NUMBER>0
2024-04-11 19:46:30,745 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) VM instance {"id":25995,"instanceName":"i-2-25995-VM","type":"User","uuid":"ed56cab5-6657-407c-8def-0a7b0a50b568"} state transited from [Migrating] to [Running] with event [OperationFailed]. VM's original host: Host {"id":488,"name":"node108","type":"Routing","uuid":"01132381-6797-4bb1-8aab-e9e990029c64"}, new host: Host {"id":488,"name":"node108","type":"Routing","uuid":"01132381-6797-4bb1-8aab-e9e990029c64"}, host before state transition: Host {"id":343,"name":"node181","type":"Routing","uuid":"9def292d-3a1d-4fe2-b9ac-d4730944c59c"}
2024-04-11 19:46:30,758 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Hosts's actual total CPU: 105600 and CPU after applying overprovisioning: 105600
2024-04-11 19:46:30,759 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Hosts's actual total RAM: (115.78 GB)<PHONE_NUMBER>20 and RAM after applying overprovisioning: (115.78 GB)<PHONE_NUMBER>24
2024-04-11 19:46:30,759 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) release cpu from host: 343, old used: 74000,reserved: 0, actual total: 105600, total with overprovisioning: 105600; new used: 70000,reserved:0; movedfromreserved: false,moveToReserveredfalse
2024-04-11 19:46:30,759 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) release mem from host: 343, old used: (106.00 GB)<PHONE_NUMBER>44,reserved: (0 bytes) 0, total: (115.78 GB)<PHONE_NUMBER>24; new used: (102.00 GB)<PHONE_NUMBER>48,reserved:(0 bytes) 0; movedfromreserved: false,moveToReserveredfalse
2024-04-11 19:46:30,766 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Invocation exception, caused by: com.cloud.utils.exception.CloudRuntimeException: Exception during migrate: org.libvirt.LibvirtException: Cannot access storage file '/mnt/667dc116-bc57-3e1f-b21a-96babdbcebd4/2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb.iso': No such file or directory
2024-04-11 19:46:30,766 INFO  [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-151:ctx-6a3ae7c2 job-3041976/job-3041979 ctx-5442e019) (logid:c631290f) Rethrow exception com.cloud.utils.exception.CloudRuntimeException: Exception during migrate: org.libvirt.LibvirtException: Cannot access storage file '/mnt/667dc116-bc57-3e1f-b21a-96babdbcebd4/2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb.iso': No such file or directory

on the agent I can see the pool being searched for, then created since it cannot find it and finally a stop command from the management server.
2024-04-11 19:46:29,732 DEBUG [resource.wrapper.LibvirtPrepareForMigrationCommandWrapper] (agentRequest-Handler-2:null) Preparing host for migrating VM {id: "25995", name: "i-2-25995-VM", uuid: "ed56cab5-6657-407c-8def-0a7b0a50b568", type: "User"}
2024-04-11 19:46:29,732 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) Looking for libvirtd connection at: qemu:///system
2024-04-11 19:46:29,751 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) Can not find KVM connection for Instance: i-2-25995-VM, continuing.
2024-04-11 19:46:29,751 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) Looking for libvirtd connection at: lxc:///
2024-04-11 19:46:29,751 INFO  [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) No existing libvirtd connection found. Opening a new one
2024-04-11 19:46:29,752 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) Can not find LXC connection for Instance: i-2-25995-VM, continuing.
2024-04-11 19:46:29,752 WARN  [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) Can not find a connection for Instance i-2-25995-VM. Assuming the default connection.
2024-04-11 19:46:29,752 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) Looking for libvirtd connection at: qemu:///system
2024-04-11 19:46:29,771 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-2:null) nic=[Nic:Guest-<IP_ADDRESS>-vlan://1966]
2024-04-11 19:46:29,771 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-2:null) creating a vNet dev and bridge for guest traffic per traffic label cloudbr0
2024-04-11 19:46:29,771 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-2:null) Executing: /usr/share/cloudstack-common/scripts/vm/network/vnet/modifyvlan.sh -v 1966 -p bond0 -b brbond0-1966 -o add
2024-04-11 19:46:29,772 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-2:null) Executing while with timeout : 1800000
2024-04-11 19:46:29,808 DEBUG [kvm.resource.BridgeVifDriver] (agentRequest-Handler-2:null) Execution is successful.
2024-04-11 19:46:29,808 DEBUG [resource.wrapper.LibvirtPrepareForMigrationCommandWrapper] (agentRequest-Handler-2:null) disk volumeTO[uuid=02de7a0f-bd65-4d7c-a56f-d70b233c7c1b|path=02de7a0f-bd65-4d7c-a56f-d70b233c7c1b|datastore=PrimaryDataStoreTO[uuid=a6d82da8-fdc3-38d0-8a3f-2798f270bbf8|name=Primary2-Netapp-18|id=59|pooltype=NetworkFilesystem]] has no passphrase or encryption
2024-04-11 19:46:29,809 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) Attempting to create storage pool 49d099eb-456c-3262-867a-3dd38b623a04 (NetworkFilesystem) in libvirt
2024-04-11 19:46:29,809 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) Looking for libvirtd connection at: qemu:///system
2024-04-11 19:46:29,827 WARN  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) Storage pool 49d099eb-456c-3262-867a-3dd38b623a04 was not found running in libvirt. Need to create it.
2024-04-11 19:46:29,827 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) Didn't find an existing storage pool 49d099eb-456c-3262-867a-3dd38b623a04 by UUID, checking for pools with duplicate paths
...
2024-04-11 19:46:29,931 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) Attempting to create storage pool 49d099eb-456c-3262-867a-3dd38b623a04
2024-04-11 19:46:29,931 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) <pool type='netfs'>
2024-04-11 19:46:29,931 DEBUG [utils.script.Script] (agentRequest-Handler-2:null) Executing: /bin/bash -c mountpoint -q /mnt/49d099eb-456c-3262-867a-3dd38b623a04
2024-04-11 19:46:29,932 DEBUG [utils.script.Script] (agentRequest-Handler-2:null) Executing while with timeout : 3600000
2024-04-11 19:46:29,937 DEBUG [utils.script.Script] (agentRequest-Handler-2:null) Exit value is 32
2024-04-11 19:46:29,937 DEBUG [utils.script.Script] (agentRequest-Handler-2:null) 32
2024-04-11 19:46:30,123 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) Trying to fetch storage pool 49d099eb-456c-3262-867a-3dd38b623a04 from libvirt
2024-04-11 19:46:30,123 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) Looking for libvirtd connection at: qemu:///system
2024-04-11 19:46:30,146 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) Successfully refreshed pool 49d099eb-456c-3262-867a-3dd38b623a04 Capacity: (9.0931 TB)<PHONE_NUMBER>320 Used: (7.2693 TB)<PHONE_NUMBER>312 Available: (1.8238 TB)<PHONE_NUMBER>008
2024-04-11 19:46:30,149 INFO  [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) Trying to fetch storage pool a6d82da8-fdc3-38d0-8a3f-2798f270bbf8 from libvirt
2024-04-11 19:46:30,149 DEBUG [kvm.resource.LibvirtConnection] (agentRequest-Handler-2:null) Looking for libvirtd connection at: qemu:///system
2024-04-11 19:46:30,151 DEBUG [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) Successfully refreshed pool a6d82da8-fdc3-38d0-8a3f-2798f270bbf8 Capacity: (5.0000 TB)<PHONE_NUMBER>880 Used: (2.0016 TB)<PHONE_NUMBER>600 Available: (2.9984 TB)<PHONE_NUMBER>280
2024-04-11 19:46:30,151 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-2:null) This host utilizes cgroupv2 (as the max shares value is [105600]), thus, the VM requested shares of [4000] will be converted to consider the host limits; the new CPU shares value is [379].
2024-04-11 19:46:30,151 DEBUG [resource.wrapper.LibvirtPrepareForMigrationCommandWrapper] (agentRequest-Handler-2:null) Setting CPU shares to [379] for the migration of VM [VM {id: "25995", name: "i-2-25995-VM", uuid: "ed56cab5-6657-407c-8def-0a7b0a50b568", type: "User"}].
2024-04-11 19:46:30,152 DEBUG [cloud.agent.Agent] (agentRequest-Handler-2:null) Seq<PHONE_NUMBER>819743874648:  { Ans: , MgmtId:<PHONE_NUMBER>2, via: 343, Ver: v1, Flags: 110, [{"com.cloud.agent.api.PrepareForMigrationAnswer":{"dpdkInterfaceMapping":{},"newVmCpuShares":"379","result":"true","wait":"0","bypassHostMaintenance":"false"}}] }
2024-04-11 19:46:30,721 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) Request:Seq<PHONE_NUMBER>819743874649:  { Cmd , MgmtId:<PHONE_NUMBER>2, via: 343, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":"false","checkBeforeCleanup":"false","forceStop":"false","vlanToPersistenceMap":{"1966":"true"},"volumesToDisconnect":[],"vmName":"i-2-25995-VM","executeInSequence":"false","wait":"0","bypassHostMaintenance":"false"}}] }
2024-04-11 19:46:30,721 DEBUG [cloud.agent.Agent] (agentRequest-Handler-1:null) Processing command: com.cloud.agent.api.StopCommand

It seems that Cloudstack created '/mnt/49d099eb-456c-3262-867a-3dd38b623a04'  however when libvirt migrated the XML from the other host it still references the old mount point at '/mnt/667dc116-bc57-3e1f-b21a-96babdbcebd4' which doesnt exist on the destination host and libvirt throws an error to Cloudstack.
EXPECTED RESULTS
Live migration of ISO working, with Cloudstack creating the correct mount that the libvirt XML is using.

ACTUAL RESULTS
Libvirt file not found error

@weizhouapache any idea of any further details I should include for this bug? I think 2x secondary storage VMs is largely untested and the root cause of this.

@weizhouapache any idea of any further details I should include for this bug? I think 2x secondary storage VMs is largely untested and the root cause of this.

@loth Is the iso stored on both secondary storages ? Maybe db says it is but actually it isn't.

I checked both storage mounts, ISO exists:
root@s-27167-VM:/mnt/SecStorage/49e5c7d7-bef0-32c9-8be4-505b3b0b51be# find . -name 2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb.iso
./template/tmpl/2/2695/2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb.iso
root@s-27167-VM:/mnt/SecStorage/49e5c7d7-bef0-32c9-8be4-505b3b0b51be# cd ..
root@s-27167-VM:/mnt/SecStorage# cd cadb1c27-afcb-314d-a866-e039feae8492/
root@s-27167-VM:/mnt/SecStorage/cadb1c27-afcb-314d-a866-e039feae8492# find . -name 2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb.iso
./template/tmpl/2/2695/2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb.iso

Furthermore I think this is more of a bug within cloudstack and how it is generating the location of the ISO. We can see in the output of libvirtd errorlog "/mnt/667dc116-bc57-3e1f-b21a-96babdbcebd4/2695-2-bec1fada-4469-3072-ab2c-799b0163f1eb.iso" is in the domain XML however cloudstack created /mnt/49d099eb-456c-3262-867a-3dd38b623a04 for the VM to mount the ISO from (and the ISO does exist at this mount point) but never updated the XML to use this new mount point.  The migration completes but the VM fails to start due to this mount point not existing.
So either 2 things needed to happen for this not to error in cloudstack

Use the old mount from the XML during migration when spinning up a new nfs mount at the migration destination
Update the XML to use the new mount point on the destination (not sure if this is possible)

@loth there are some PRs out for similar issues, #8952 and for the 4.19 branch #8909. Will you have the chance to test with those patches?

@loth there are some PRs out for similar issues, #8952. Will you have the chance to test with that patch?

Hello,
I tried the mentioned patch and it diddnt affect the issue. Here is a rundown on what I did to reproduce it:
setup new 4.18 env with patch
upload iso (grml-small in my case)
add 2nd secondary storage mount
wait for templates to populate
redownload/upload template if "secondary storage bypassed" shows
create test VM
attach uploaded iso
attempt live migration
Exception during migrate: org.libvirt.LibvirtException: Cannot access storage file '/mnt/7d79fb97-cdae-3a13-a0ac-29f3b1f37a1e/203-2-76328563-4e13-3db1-87be-e81f46e9394a.iso': No such file or directory
ISO exists on both secondary storage mounts:
root@mgt01-tyler-nfs-test:/data/secondary2# ls -laR  | grep 203-2-76328563-4e13-3db1-87be-e81f46e9394a
-rw-rw-rw- 1 root root 532938752 Apr 29 21:35 203-2-76328563-4e13-3db1-87be-e81f46e9394a.iso
root@mgt01-tyler-nfs-test:/data/secondary2# cd ..
root@mgt01-tyler-nfs-test:/data# cd secondary1
root@mgt01-tyler-nfs-test:/data/secondary1# ls -laR  | grep 203-2-76328563-4e13-3db1-87be-e81f46e9394a
-rw-rw-rw- 1 root root 532938752 Apr 29 21:29 203-2-76328563-4e13-3db1-87be-e81f46e9394a.iso
root@mgt01-tyler-nfs-test:/data/secondary1#

On node01 (destination of live migration) mount for secondary storage was created as '/mnt/1919082e-71af-3ce0-b6d7-df2bf6249f3b'
root@node01-tyler-nfs-test:~# mount | grep secondary
<IP_ADDRESS>:/data/secondary1/template/tmpl/2/203 on /mnt/1919082e-71af-3ce0-b6d7-df2bf6249f3b type nfs (rw,nosuid,nodev,noexec,relatime,vers=3,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=<IP_ADDRESS>,mountvers=3,mountport=37418,mountproto=udp,local_lock=none,addr=<IP_ADDRESS>)

however libvirt is looking for the ISO at '/mnt/7d79fb97-cdae-3a13-a0ac-29f3b1f37a1e' thus we get an error on migration
Can confirm the ISO does exist at the new directory created on node01, just with the wrong mount UUID string:
root@mgt01-tyler-nfs-test:/data/secondary1# ls -la /data/secondary1/template/tmpl/2/203
total 520464
drwxrwxrwx 2 root root      4096 Apr 29 21:29 .
drwxrwxrwx 4 root root      4096 Apr 29 21:42 ..
-rw-rw-rw- 1 root root 532938752 Apr 29 21:29 203-2-76328563-4e13-3db1-87be-e81f46e9394a.iso
-rw-rw-rw- 1 root root       344 Apr 29 21:29 template.properties

thanks @loth
I think it is a bug
ACS only stores which ISO is attached to a vm, but does not store the image store id of the ISO.

I investigated this with a 4.19 branch env and cannot reproduce the error during the live migration.
I've following setup to start with

2x Ubuntu 22.04 hosts in the same cluster
1 secondary store
What I did for my 2 different attempts:
1st -
Deployed a VM
Registered an ISO
Attached the ISO to the VM
Added another secondary store
Live-migrated VM
Waited for the ISO to be available on both the secondary store

2nd -

Registered an ISO
Added second secondary store
Waited for the ISO to be available on both the secondary store
Deployed a VM
Attached the ISO to the VM
Live-migrated VM

In both cases, migration worked without an error.
@loth @weizhouapache am I missing anything?

I investigated this with a 4.19 branch env and cannot reproduce the error during the live migration. I've the following setup to start with

2x Ubuntu 22.04 hosts in the same cluster
1 secondary store

What I did for my 2 different attempts: 1st -

Deployed a VM
Registered an ISO
Attached the ISO to the VM
Added another secondary store
Live-migrated VM
Waited for the ISO to be available on both the secondary store

2nd -

Registered an ISO
Added second secondary store
Waited for the ISO to be available on both the secondary store
Deployed a VM
Attached the ISO to the VM
Live-migrated VM

In both cases, migration worked without an error. @loth @weizhouapache am I missing anything?

thanks @shwstppr
I will have a look

@loth @shwstppr
I am able to reproduce the issue in 4.20
the migration worked sometimes, but not all the time
Exception during migrate: org.libvirt.LibvirtException: Cannot access storage file '/mnt/ec8dfdd8-f341-3b0a-988f-cfbc93e46fc4/251-2-2b2071a4-21c7-340e-a861-1bd30fb5cbed.iso': No such file or directory

fixed by #9212


Create 0703-kth-largest-element-in-a-stream.ts

File(s) Modified: typescript/0703-kth-largest-element-in-a-stream.ts
Language(s) Used: typescript
Submission URL:  https://leetcode.com/problems/kth-largest-element-in-a-stream/solutions/4971837/typescript-using-minheap-beats-99/

Important
Please make sure the file name is lowercase and a duplicate file does not already exist before merging.

@arkadiusz-wrzawinski Thanks for your contribution 👍


llvm_11: Correct path for TLI-musl patch

Motivation for this change
The path for the the TLI-musl.patch is incorrect in the llvm_11 derivation (it should be one more directory down like all the other llvm derivations)
~/nixpkgs (master)> nix build -f . pkgsMusl.llvm_11
error: getting attributes of path '/home/tyson/nixpkgs/pkgs/development/compilers/llvm/11/TLI-musl.patch': No such file or directory
~/nixpkgs (master)>

After fix the patch path
~/nixpkgs (llvm_11-musl)> nix build -f . pkgsMusl.llvm_11
~/nixpkgs (llvm_11-musl)> ./result/bin/llc --version
LLVM (http://llvm.org/):
  LLVM version 11.1.0
...

Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[X] Ensured that relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

Can you patch this for all other LLVM versions, too?


Unable to integrate with Nest.js [CommonJS support]

Guys, are you planning to add the CommonJS support?

@R-iskey having the same issues over here -- I know this isn't ideal, but this can be resolved by using awaited dynamic imports. You can use this pattern with classes that's working good for our team:
import type { UnipileClient, UnipileClient as UnipileClientType } from 'unipile-node-sdk';
import { AccountOwnerProfileApiResponse } from 'unipile-node-sdk/dist/users/account-owner-profile';

class UnipileClientManager {
  protected static instance: UnipileClientManager;

  protected client?: UnipileClientType;

  private UnipileClient?: new (dsn: string, apiKey: string) => UnipileClientType;

  private constructor() {
    console.log('Constructor called - initializing client');
    this.initializeClient();
  }

  private async initializeClient() {
    try {
      // Dynamic import of the SDK
      const unipileModule = await import('unipile-node-sdk');
      this.UnipileClient = unipileModule.UnipileClient;

      // Initialize with API credentials
      if (process.env['UNIPILE_DSN_URL'] && process.env['UNIPILE_API_KEY']) {
        this.client = new this.UnipileClient(
          process.env['UNIPILE_DSN_URL'],
          process.env['UNIPILE_API_KEY'],
        );
        console.log('Client successfully initialized');
      } else {
        console.error('Missing API Key or DSN URL for Unipile');
      }
    } catch (error) {
      console.error('Failed to initialize Unipile client:', error);
    }
  }

  private checkClient(): asserts this is { client: UnipileClient } {
    if (!this.client) {
      throw new Error('Unipile client is not initialized');
    }
  }

  public static async getInstance(): Promise<UnipileClientManager> {
    if (!UnipileClientManager.instance) {
      console.log('Creating new instance');
      UnipileClientManager.instance = new UnipileClientManager();
      await UnipileClientManager.instance.initializeClient();
    } else {
      console.log('Returning existing instance');
    }
    return UnipileClientManager.instance;
  }

  public getClient(): UnipileClientType {
    if (!this.client) {
      throw new Error('Unipile client is not initialized');
    }
    return this.client;
  }

  public async getUserProfile(account_id: string): Promise<AccountOwnerProfileApiResponse> {
    this.checkClient();
    return this.client.users.getOwnProfile(account_id);
  }
}

export async function getUnipileClient() {
  return UnipileClientManager.getInstance();
}

// Usage example
async function getUserProfile() {
  const unipileClient = await getUnipileClient();
  const userProfile = await unipileClient.getUserProfile('123');
  console.log(userProfile);
}

await getUserProfile();

Or, if you don't want to use classes:
import type { UnipileClient } from 'unipile-node-sdk';

let client: UnipileClient | null = null;

async function createClient(dsn: string, apiKey: string) {
  try {
    // Dynamic import of the SDK
    const { UnipileClient } = await import('unipile-node-sdk');
    return new UnipileClient(dsn, apiKey);
  } catch (error) {
    console.error('Failed to initialize client:', error);
    throw error;
  }
}

// Simple factory function that ensures we only create one client
async function getClient() {
  if (!client) {
    client = await createClient(
      process.env.UNIPILE_DSN_URL!,
      process.env.UNIPILE_API_KEY!
    );
  }
  return client;
}

// Usage example
async function example() {
  const client = await getClient();
  const profile = await client.users.getOwnProfile('123');
  console.log(profile);
}

@R-iskey hey again! So turns out that did not work for us :melting_face: as soon as webpack bundled, we got tons of errors. So I actually just published a fork with full CJS support!
https://www.npmjs.com/package/unipile-node-sdk-with-cjs
https://github.com/arimgibson/unipile-node-sdk/tree/ari/cjs

Thanks @arimgibson, I had a look and I'm merging this ! 👍

@pozorfluo lemme do a little clean up and will PR officially! There's also a catch which not sure if relevant -- I removed node-fetch for compatibility reasons as it also only supports ESM in future versions. This means Node 18.x+, which is current recommended in your docs, would be a requirement. It's a breaking release

you're correct about node 18, this would become required.

@pozorfluo I believe this can be closed now, yes?


Make entire admin boxes clickable on division pages

https://rd2l.gg/divisions/HJSzO_xoM

Closed by 5e6dac6


dns templates do not properly implement deletion

Unable to remove records or DM resource. Even when feeding an empty set of resources dm tries to POST and complains about missing entity.change parameter. When looking at the API reference it appears DM uses wrong method and should use DELETE:
  location: /deployments/myproject/resources/myrecord-resource
  message: "{\"ResourceType\":\"gcp-types/dns-v1:resourceRecordSets\",\"ResourceErrorCode\"\
    :\"400\",\"ResourceErrorMessage\":{\"code\":400,\"errors\":[{\"domain\":\"global\"\
    ,\"message\":\"The 'entity.change' parameter is required but was missing.\",\"\
    reason\":\"required\"}],\"message\":\"The 'entity.change' parameter is required\
    \ but was missing.\",\"statusMessage\":\"Bad Request\",\"requestPath\":\"https://dns.googleapis.com/dns/v1/projects/myproject/managedZones/myrecord-resource/changes\"\
    ,\"httpMethod\":\"POST\"}}"```

This looks like an issue with gcp-types/dns-v1:resourceRecordSets.
Please follow https://github.com/GoogleCloudPlatform/deploymentmanager-samples/issues/509 for tracking. I escalate this with the DM team.


Live migration fail with "Error collecting checkpoint log file"

Required information

Distribution: Ubuntu 16.04
Kernel version: 4.4.0-38-generic
LXC version: 2.0.4
LXD version: 2.0.4
Storage backend in use: zfs

Issue description
Live migration end up with the following error:
error: Error transferring container data: migration failed 1
Log file /var/log/lxd/lxd.log shows :
t=2016-10-06T17:35:17+0100 lvl=eror msg="Error collecting checkpoint log file" err="open /tmp/lxd_checkpoint_986842051/dump.log: no such file or directory"
/var/log/lxd/lxd.log: https://gist.github.com/fluboi/e2a84fda21c64f524fc0a7c30d2aa8a3
/var/log/lxd/ct1/lxc.log: https://gist.github.com/fluboi/aa0d72c824f8646327e9729ce3c42448
Steps to reproduce
Basic fresh install with 2 Ubuntu 16.04. "Normal" migration is working properly. (by stopping CT before)

Sounds like criu failed before it even got to write to any log files. @tych0

What's your criu version? (criu --version should suffice) I thought this usually meant criu didn't understand the arguments, but somehow the arguments aren't listed in your log, which leads me to suspect something is very wrong :(

I'm using criu Version: 2.0. (Ubuntu 16.04, repository package)

Ok, and I just looked and 2.0.4 should emit the flags it passes to criu, so now I'm confused. Is the above the paste of the whole log?

Yes, I've paste the whole logs created between the lxc movecommand, and the fail  migration failed 1

On the remote host (the one that should receive the CT), I also have that lxd log : lvl=eror msg="Error during migration sink" err="migration failed 1\n" ... but I'm not sure it helps a lot..

I guess I'm not sure what to tell you, other than perhaps try to upgrade to the latest CRIU? I don't see where in the code criu or the liblxc child would exit 2, and I'm not sure why you're not getting the flags that were actually passed to criu.
What's the output of cat /proc/$lxd_pid/maps? Do you have a 1.1 liblxc loaded or something?

 sudo cat /proc/2429/maps | grep liblxc
7fe854067000-7fe8540f1000 r-xp 00000000 fc:00 394234                     /usr/lib/x86_64-linux-gnu/liblxc.so.1.2.0
7fe8540f1000-7fe8542f1000 ---p 0008a000 fc:00 394234                     /usr/lib/x86_64-linux-gnu/liblxc.so.1.2.0
7fe8542f1000-7fe8542f3000 r--p 0008a000 fc:00 394234                     /usr/lib/x86_64-linux-gnu/liblxc.so.1.2.0
7fe8542f3000-7fe8542f4000 rw-p 0008c000 fc:00 394234                     /usr/lib/x86_64-linux-gnu/liblxc.so.1.2.0

Don't know how to read that. It should not be 2.0.X ? Or liblxc version is not linked to LXC version ?

No, that's right. hm.

Do you have some other executable named criu in your path somewhere maybe? which -a criu?

I've upgrade criu to 2.6. And... it still not working, but I have a totally different error message. Should I go back to 2.0 version, open a new issue for that :
CLI:
error: Error transferring container data: migration dump failed
(00.922683) Error (criu/action-scripts.c:116): One of more action scripts failed
(00.922715) Error (criu/cr-dump.c:1654): Pre dump script failed with 32512!
(00.923072) Error (criu/ptrace.c:54): Unable to detach from 2726: No such process
(01.049008) Error (criu/stats.c:73): BUG at criu/stats.c:73

lxc.log:
            lxc<PHONE_NUMBER>4726.466 INFO     lxc_criu - criu.c:exec_criu:419 - execing: /usr/sbin/criu dump --tcp-established --file-locks --link-remap --manage-cgroups=full --ext-mount-map auto --enable-external-sharing --enable-external-masters --enable-fs hugetlbfs --enable-fs tracefs -D /tmp/lxd_checkpoint_974840257 -o /tmp/lxd_checkpoint_974840257/dump.log -vvvvvv --action-script � -t 2726 --freeze-cgroup /sys/fs/cgroup/freezer///lxc/ct1 --ext-mount-map /dev/console:console --external tty[8801:e] --force-irmap --ghost-limit<PHONE_NUMBER>44
            lxc<PHONE_NUMBER>4726.469 DEBUG    lxc_commands - commands.c:lxc_cmd_handler:893 - peer has disconnected
            lxc<PHONE_NUMBER>4729.320 ERROR    lxc_criu - criu.c:do_dump:920 - dump signaled with 6

Maybe the way I've install criu 2.6 is not corect (downloading the .deb package, and installing it using dpkg -i)
Currently which -a criu only return /usr/sbin/criu. (I haven't test before the upgrade)

Heh, this bug should be fixed too, actually. It could be one of two things, one is fixed in 2.0.4 and another is fixed in 2.0.5. How did you get your liblxc and LXD? From the archive?

Exactly what I'v done to upgrade criu:
wget https://launchpad.net/ubuntu/+source/criu/2.6-1ubuntu1/+build/10991540/+files/criu_2.6-1ubuntu1_amd64.deb
dpkg -i criu_2.6-1ubuntu1_amd64.deb

(Also rebooted after that)
My initial install was just as simple as :
sudo apt install lxd  zfsutils-linux zfs-dkms bridge-utils criu in a clean (just installed) Ubuntu server 16.04
I'll try (tomorrow or this week-end) to re-do a fresh install, but I'm pretty sure I'll be facing the same bug as there is nothing eccentric in my current install ! :)

But I'm currently still in LXC & LXD Version 2.0.4 accordind to lxc info. I've just upgraded criu.

Right, I think this should be fixed in 2.0.5, which should hit the archive soon. You can use the stable PPA in the meantime, though https://launchpad.net/~ubuntu-lxc/+archive/ubuntu/stable
Can you try and upgrade liblxc, do a restart of LXD, and then do a checkpoint of a container?

Hmm, be very careful when using that PPA. For LXD, it will upgrade you from 2.0.4 to 2.4.1. Once that's done, you cannot go back as we don't support database downgrades!
So if you don't mind running the latest feature release and get all the new features every month or so, go for it. But if you want to stick to the production grade release, you should wait for 2.0.5 to make its way through updates rather than upgrade to 2.4.1.

So if I understand well, "stable release" is not really ready for production environments ? Sounds not natural for me as a Debian guy !
When you say :

2.0.5, which should hit the archive soon
What means "soon" ? days ? weeks ? months ? I need to setup a production environment... soon !

I will test the PPA "stable" in my testing lab, but for the production I would like something... stable ! It would be great if live migration works in the production environment, but if not (and effectively corrected in a near future), it's not dramatic.
Thanks a lot for your help.

So for all the LXC products (LXC, LXCFS and LXD), we publish normal "stable" releases at various intervals, which introduce new features and may change command line arguments and user experience. We also publish LTS releases of those 3 projects every two years, those LTS releases then get bugfix and security updates but no new feature or user experience change.
So both LXD 2.0.5 and LXD 2.4 are stable. It's just that support for LXD 2.4 will only last a few months before we tell people to move to LXD 2.5 or whatever's current while support for LXD 2.0.x will last 5 years.
As for the PPAs, we have ppa:ubuntu-lxc/stable which is the latest release (as opposed to pre-release or git snapshots) of all products and then we have ppa:ubuntu-lxc/lts which is the latest LTS release of all products.
Anyway, I was travelling this week (still am, sending this from over the atlantic) and only had time to push the most urgent releases to the Ubuntu archive (there was a deadline for Ubuntu 16.10 on Thursday) so that was LXC 2.0.5, LXCFS 2.0.4 and LXD 2.4.1.
Next up is pushing LXC 2.0.5, LXCFS 2.0.4 and LXD 2.0.5 to xenial-proposed which I'll most likely be doing on Monday, after review by the SRU team, the packages will go in proposed for 7 days of testing, if all goes well, they'll then hit xenial-updates. So the most optimistic ETA for 2.0.5 in the Ubuntu 16.04 archive is the 17th of October.
Once those hit xenial-updates, we'll then update the backports we have in trusty-backports for those still on Ubuntu 14.04.

Ok thanks for your clear answer.
And about criu versions ? Is it totally independent ? Is it possible to do criu downgrades ? (seems yes, but if you can confirm...)

Yeah, CRIU is independent. LXC just calls out to the criu binary, so as long as they don't remove options in a new release you'll be fine. We have detection code for the criu version to drop some arguments when running older releases.

Ok. Thanks.
I've tried to reproduce the original bug (about checkpoint log file), from clean install.
You can find a trace of what I've done here.
In short: I'm not able to reproduce the same bug. But I'm facing something that look like more the one already underline in that previous message.
By then adding ppa:ubuntu-lxc/stable, update/upgrade/restart LXD, I confirm that the bug disappear and that I'm able to live migrate containers.
Packages upgraded with ppa add:
liblxc1/xenial 2.0.5-0ubuntu1~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.0.4-0ubuntu1~ubuntu16.04.2]
libseccomp2/xenial 2.3.1-2ubuntu2~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.2.3-3ubuntu3]
lxc-common/xenial 2.0.5-0ubuntu1~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.0.4-0ubuntu1~ubuntu16.04.2]
lxcfs/xenial 2.0.4-0ubuntu1~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.0.3-0ubuntu1~ubuntu16.04.1]
lxd/xenial 2.4.1-0ubuntu1~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.0.4-0ubuntu1~ubuntu16.04.1]
lxd-client/xenial 2.4.1-0ubuntu1~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.0.4-0ubuntu1~ubuntu16.04.1]

Do you confirm that this specific bug is corrected in the coming LXD 2.0.5 ? :)

All fixes related to migration from master were included, so yeah, the commit is in 2.0.5

Cool, so sounds like the newer liblxc and lxd does fix the issue for you, so closing this issue as it's fixed upstream. I'm hoping we won't get delayed in pushing LXC and LXD 2.0.5 to Ubuntu 16.04 so you've got this fixed soon!


How to use?

How do I use this after installing it?

Meeee toooo

How to use Khan answers for answers


feat: paper applications - new fields

This PR addresses #(750)

[x] Addresses the issue in full
[ ] Addresses only certain aspects of the issue

Description
Adding ReceivedAt and ReceivedBy fields to application table. Present on the Partners Portal when creating/updating/viewing paper applications; ReceivedAt follows logic for SubmissionDate. Appears in the application export csv.
How Can This Be Tested/Reviewed?
Login to Partners Portal as an admin or a partner.
Navigate to a listing's application tab.
Add an application, fill out Date Received At, Time Received At and Received By fields.
Submit application, verify data appears correctly in details view.
Edit application, verify data appears correctly, edit data, submit and verify change.
Go back to the application tab.
Export application csv and verify new fields are present and contain correct data.
Author Checklist:

[x] Added QA notes to the issue with applicable URLs
[x] Reviewed in a desktop view
[ ] Reviewed in a mobile view
[ ] Reviewed considering accessibility
[x] Added tests covering the changes
[ ] Made corresponding changes to the documentation
[x] Ran yarn generate:client and/or created a migration when required

Review Process:

Read and understand the issue
Ensure the author has added QA notes
Review the code itself from a style point of view
Pull the changes down locally and test that the acceptance criteria is met
Either (1) explicitly ask a clarifying question, (2) request changes, or (3) approve the PR, even if there are very small remaining changes, if you don't need to re-review after the updates

@mcgarrye I'm curious if there was a convo about adding this to Core as well? Seems like it would be universally useful.

@mcgarrye I'm curious if there was a convo about adding this to Core as well? Seems like it would be universally useful.

We did discuss and product decided against adding it to core


Codespaces - cannot install the extension

It seems this extensions is tagged as strict ui kind and hence it cannot be installed in web.
"extensionKind": [
		"ui"
]

Please change it to
"extensionKind": [
		"ui",
                "workspace"
]

or remove the entry completely so that VS Code can infer it smartly.

I actually already set this:
https://github.com/misolori/vscode-fluent-icons/blob/b70187d287d60b3e217e10f59d6abc1e5f79f474/package.json#L20-L22
Is there a reason why it's not picked up by the web?

It seems the extension was not published with the change - I see that latest version (that got published one hour ago from this time) has the change and can be installed in codespaces.
So feel free to close this.

Ah, I think I must've forgot to rebuild when I added that. Thanks!


Fully initialize JAAS before launching jsgui (in tests)

helps, but doesn't solve it entirely.  new PR coming with more fixes, and capturing outstanding issues.


Tests

Should have tests to ensure it's working and we don't introduce regressions.
Resources

https://github.com/sindresorhus/atom-set-text/blob/master/spec/main-spec.js
https://github.com/ben-eb/atom-cssnano/blob/master/spec/cssnano-spec.js

We then should probably also include support for Travis and Appveyour.

I think Travis is enough. Even Atom doesn't yet test on AppVeyor and AppVeyor is just too unstable and bad. See: https://github.com/sindresorhus/atom-set-text/blob/master/.travis.yml

Good point -- i just thougt it could be a good idea testing it on windows since there might be differences due to the handling of line-endings.

@florianb Yup. Can consider it in the future when Atom has battle-tested it.

The tests should include fixtures for:

[ ] empty projects (#67)

FYI: Atom doesn't test on AppVeyor because it's near impossible to build on Windows properly :stuck_out_tongue:
There is a recommended CI configuration for AppVeyor that works quite well though, which can be found here: https://github.com/atom/ci

Hmmm - the build-script for OSX seems to apply atom into the project-directory letting xo complain about the style: https://travis-ci.org/florianb/atom-editorconfig/jobs/120570253
:bulb: Exclude the ./atom/-dir from the xo-searchpath?
:bulb: Change the buildscript?

Exclude the ./atom/-dir from the xo-searchpath?

:+1:

Thanks everybody for support! :beers:


Locust deployment doesn't use the proper FRONTEND_ADDR

Describe the bug
In the step to deploy the load generator, the current locust-deployment.yaml uses the status.hostIP as the URL for the FRONTEND_ADDR:
        env:
        - name: NODE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.hostIP
        - name: FRONTEND_ADDR
          value: "$(NODE_IP):8082"

The nginx-loadgenerator Service is defined as a LoadBalancer which, on K8s deployments with true LoadBalancers configured, pull EXTERNAL-IP addresses from a different IP range than the host that the pod is deployed on:
kubectl get pods -o wide
NAME                                                          READY   STATUS    RESTARTS   AGE    IP              NODE                NOMINATED NODE   READINESS GATES
nginx-7b95fb6b6b-26gvm                                        1/1     Running   0          17m    <IP_ADDRESS>     wpc1                <none>           <none>
nginx-7b95fb6b6b-99jr6                                        1/1     Running   0          17m    <IP_ADDRESS>     wpc1                <none>           <none>
nginx-7b95fb6b6b-bd5gj                                        1/1     Running   0          17m    <IP_ADDRESS>     wpc1                <none>           <none>
nginx-7b95fb6b6b-c6dp8                                        1/1     Running   0          17m    <IP_ADDRESS>     wpc1                <none>           <none>
nginx-loadgenerator-758668dc4c-lhdwb                          1/1     Running   0          8m     <IP_ADDRESS>     wpc1                <none>           <none>
splunk-otel-collector-agent-2qctd                             1/1     Running   0          3h3m   <IP_ADDRESS>   k8s-worker-intel2   <none>           <none>
splunk-otel-collector-agent-bhjsx                             1/1     Running   0          3h3m   <IP_ADDRESS>   k8s-worker-intel3   <none>           <none>
splunk-otel-collector-agent-pm6rb                             1/1     Running   0          3h3m   <IP_ADDRESS>   k8s-master-intel    <none>           <none>
splunk-otel-collector-agent-rr8cb                             1/1     Running   1          3h3m   <IP_ADDRESS>   k8s-worker-intel1   <none>           <none>
splunk-otel-collector-agent-t4ccr                             1/1     Running   0          3h3m   <IP_ADDRESS>   k8s-worker-intel4   <none>           <none>
splunk-otel-collector-agent-zhvh2                             1/1     Running   0          176m   <IP_ADDRESS>   wpc1                <none>           <none>
splunk-otel-collector-k8s-cluster-receiver-58d879c9f7-5qczm   1/1     Running   0          3h3m   <IP_ADDRESS>      wpc1                <none>           <none>

kubectl get services
NAME                  TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)          AGE
kubernetes            ClusterIP      <IP_ADDRESS>        <none>          443/TCP          4h57m
nginx                 LoadBalancer   <IP_ADDRESS>    <IP_ADDRESS>   8082:31075/TCP   5m40s
nginx-loadgenerator   LoadBalancer   <IP_ADDRESS>   <IP_ADDRESS>   8080:30011/TCP   5m7s

Consequently, the $(NODE_IP):8082 address doesn't match the LoadBalancer address and the pod deployment repeatedly fails to deploy.
Resolution
The simple fix is just to use the nginx service name nginx in the URL instead of the NODE_IP.
In locust-deployment.yaml, change:
        env:
        - name: NODE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.hostIP
        - name: FRONTEND_ADDR
          value: "$(NODE_IP):8082"

to
        env:
        - name: FRONTEND_ADDR
          value: "nginx:8082"

To Reproduce
Deploy locust-deployment.yaml in a K8s cluster with a dedicated load balancer which has it's own address pool (eg., metallb).  The load balanced IPs will be pulled from a different pool than node hosts and the deployment will fail.
Expected behavior
The hostname or IP address of the LoadBalancer for the nginx service is used instead of the host IP the pod is running on.
Environment (please complete the following information):
kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:50:19Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.6", GitCommit:"8a62859e515889f07e3e3be6a1080413f17cf2c3", GitTreeState:"clean", BuildDate:"2021-04-15T03:19:55Z", GoVersion:"go1.15.10", Compiler:"gc", Platform:"linux/amd64"}

Additional context

@billmeyer Awesome, thank you! Fix applied to repo :-)


Per variable TM order

The example in Sec 3 of https://home.cs.colorado.edu/~xich8622/manual/manual-2.0.0.pdf allows to pass min/max order per variable as in adaptive orders { min { Ii :4 , G :4} , max { Ii :6 , G :6} } but I think this is currently not supported.

I was able to abstract AdaptiveTMOrder for a flexible API for state-dependent TM orders
See test for various usages
@test Flowstar._order_string(AdaptiveTMOrder( ("x"=>2, "y" =>3, "z" => 4), ("x"=>20, "y" =>30, "z" => 40))) ==
        Flowstar._order_string(AdaptiveTMOrder( ["x"=>2, "y" =>3, "z" => 4], ["x"=>20, "y" =>30, "z" => 40])) ==
        Flowstar._order_string(AdaptiveTMOrder( ("x" => (2,20), "y" => (3,30), "z" => (4,40)) )) ==
        Flowstar._order_string(AdaptiveTMOrder( ["x" => (2,20), "y" => (3,30), "z" => (4,40)] )) ==
        Flowstar._order_string(AdaptiveTMOrder( [:x => (2,20), :y => (3,30), :z => (4,40)] ))
        "adaptive orders { min {x :2 , y :3 , z :4} , max {x :20 , y :30 , z :40}}"


Dos -issue

When i try to bypass 403 it send lots of requests so low bandwidth server can't those request

You can put a delay between the requests using -d flag or reduce the number of goroutines running at the same time using -m.


[DOC][MINOR] Removed repeated keyword causing confusion

Removed "instead of a JAR"

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

@manubatham20 can you fill the PR description? What confusion this PR fixes?

It's trivial but seems OK to me.

Jenkins test this please

Test build #129512 has started for PR 29956 at commit e10e4a7.

Test build #129512 has finished for PR 29956 at commit e10e4a7.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129512/
Test PASSed.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34117/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34117/

It's fine by me. It would be nicer if we fill the PR description correctly though. @manubatham20 can you update PR description?

Merged to master


[2.x] Avatar url in left bottom is wrong

Package
filament/filament
Package Version
2.0
Laravel Version
8.75.0
Livewire Version
2.8
PHP Version
8.0.11
Bug description
Avatar url in left bottom is wrong

/storage/ missed
Steps to reproduce
public function getFilamentAvatarUrl(): ?string
{
    return $this->avatar_url;
}

Relevant log output
No response

@webparadoxer You probably want to do a Storage::url() or something.

but in table all ok, and left corner going from admin, not me

hmm, and second problem - if clear avatar_url showing ok from ui-avatars.com
but nothing in table

https://filamentadmin.com/docs/2.x/admin/users#setting-up-avatars

You need to wrap your URL in Storage::url() within getFilamentAvatarUrl().

You need to wrap your URL in Storage::url() within getFilamentAvatarUrl().

You need to wrap your URL in Storage::url() within getFilamentAvatarUrl().

but how can show avatars from  ui-avatars in table?

@webparadoxer The ui-avatars are automatically generated.


Write missing docstrings

To get interrogate up to 100%, we were missing a handful of docstrings. These were mainly module strings, but they are important for the API documentation.

Hi @TomJamesW 👋
Would you mind looking through these docstrings when you get the chance? The only reasonably lengthy one is in the Evaluator class but I want to make sure they aren't misleading.


Retire email address field

I'm working on a third-party registry and it confused me that docker login will prompt for an email address which isn't even used for authentication.
This change will drop the unused email address while keeping support for reading configuration files that include the email address (plenty of the current tests assert that).

The experimental build failure seems to be unrelated although I'm not sure if this is expected.

Distribution is +1 on this.

Fine by me if distribution is ok for it.

@tt do you please mind squashing your commits into 1?
@jlhawn ping for AuthConfig impact!

Does this mean we're also retiring the Registration feature of docker login? I'm okay with not storing it in the config file (it's never used again anyway). I'm even more okay with not having a user signup feature in the CLI but there are higher powers involved that have kept that feature alive when there have been several attempts in the past to remove it.

If we decide to retire registration from the CLI, a note needs to be added to https://github.com/docker/docker/blob/master/docs/misc/deprecated.md

fwiw, I'm +1 on retiring it, it has come up several times in the past :-)

+1 for retiring the email

There are/where also instructions on how to create a new account on hub using docker login that would need to be removed - see /docs/userguide/dockerrepos.md

See also https://github.com/docker/distribution/issues/308

:+1: for the idea. Automated registration is always confusing for users. I said most of my opinion on this in https://github.com/docker/distribution/issues/308#issuecomment-88244735.

Oh, think we can add: closes https://github.com/docker/docker/issues/12991

@aaronlehmann @icecrime is this ready for docs review? I see approvals, but no LGTM's yet

+1 for change.
Most of the test failures look unrelated but this one (the last failure jenkins lists) looks like it may be related, just because it fails on a login attempt and I don't see any updates to the integration tests.

[d64837959] waiting for daemon to start
[d64837959] waiting for daemon to start
[d64837959] waiting for daemon to start
[d64837959] waiting for daemon to start
[d64837959] waiting for daemon to start
[d64837959] waiting for daemon to start
[d64837959] waiting for daemon to start
[d64837959] daemon started
github.com/docker/docker/integration-cli/_test/_obj_test/docker_cli_v2_only.go:175:
    ...open github.com/docker/docker/integration-cli/_test/_obj_test/docker_cli_v2_only.go: no such file or directory
... Error: Expected v1 login attempt

github.com/docker/docker/integration-cli/_test/_obj_test/docker_cli_v2_only.go:184:
    ...open github.com/docker/docker/integration-cli/_test/_obj_test/docker_cli_v2_only.go: no such file or directory
... Error: Not all endpoints contacted after push2 0

[d64837959] exiting daemon```

moved to docs review, can you please squash the commits

@jfrazelle given that this is an API change, should a "versioned" struct be added for older API versions?
See https://github.com/docker/docker/pull/16564.
Also when searching for docs that need an update, I noticed this registry/auth.go#L72, and wasn't sure if that's still relevant?
Thanks @tt! Some pointers for docs and other changes that I think are needed:
I noticed this also needs updates in the completion scripts;

Bash: https://github.com/docker/docker/blob/master/contrib/completion/bash/docker#L915-L927
zsh: https://github.com/docker/docker/blob/master/contrib/completion/zsh/_docker#L578-L584
fish: https://github.com/docker/docker/blob/master/contrib/completion/fish/docker.fish#L226

(we can do that as a follow-up PR if you want, then I'll ping the maintainers for those scripts)
Looked up docs that mention this, and (I hope I have them all) came up with:

We need a mention in Deprecated Features
This contains a change in the API, so needs a mention in the v1.21 API changes
/man/docker-login.1.md (mentions the -e/--email option and the description contains "Register or login")
same for /docs/reference/commandline/login.md
api/docker_remote_api_v1.21.md mentions "Get the default username and email", and shows the e-mail in the output

Oh, almost forgot the document @SvenDowideit mentioned; https://github.com/docker/docker/blob/master/docs/userguide/dockerrepos.md

ping @tt could you address the documentation changes?

@tt Don't forget to squash your commits too :wink:

ping @kencochrane, I just discussed this with Tibor and Michael, and we may need the input of the Hub team on this change, to check if you agree with making this change

I don't understand why this is in docs review, there are no code LGTM.

Had a quick chat with @kencochran. There's still a discussion going on internally for Docker Hub; overall they're not against this, but there are changes needed on their side to facilitate this, also for backward compatibility with older clients. Since 1.10 is still roughly two months away, that may be time enough for them, but I need to check further with some other people and will leave a comment if I know more

This seems to also be a problem for Google: https://cloud.google.com/container-registry/docs/auth#using_an_access_token
Email	Unused, must be a well-formed email address (e.g 1234@5678.com).

can you squash your commits

ping @bfirsh I've assigned you to this issue, following the e-mail discussion that you'll be coordinating this with Docker Hub team :smile:

And apologies to @tt that this is taking so long, thanks for being patient with us! :heart:

This seems to also be a problem for Google: https://cloud.google.com/container-registry/docs/auth#using_an_access_token
Email   Unused, must be a well-formed email address (e.g 1234@5678.com).

@mattmorr Any way to get this relaxed?

@stevvooe We only have that because it's required by docker login.  Not sure what you think needs to be relaxed?

@mattmoor The wording there makes it sound like if the Email field is missing, the login will be refused. If we remove the field from the docker client, will one still be able to login to GCS?

@stevvooe Absolutely, we only put garbage strings in here today.  We can qualify this as important for certain versions of the client if you plan to remove it, thanks for the heads up.

Ping @bfirsh

Currently putting together a proposal for what we can do here – will give an update in a day or two.

@bfirsh bump

Still working with Hub team to figure out something workable – will have something soon!

I've written up an improved design here with impact to both Engine and Hub: https://github.com/docker/docker/issues/18668
Comments welcome!

@bfirsh when can we resume work on this PR?

@tiborvass This is blocked on the Hub not requiring email addresses, which is being worked on at the moment.

Should we close it and re-open/carry once it's done on the Hub side ? :wink:

that sounds like a good idea.

Ping @bfirsh any updates?

@tiborvass Afraid not – Hub have a lot on their plate. :(
Will let you know when I have an update from them.

@bfirsh Let's get an estimate together. No reason to keep pushing this back.

@jfrazelle Is it really a good practice to close issues just because they are blocked? Most everywhere else, closing an issue means it's resolved, usually "fixed" or "won't fix".

@JonathonReinhart first it's a pull request, not an issue ; the related issue is there and is still open :wink:. This pull-request has been closed because it stalled and we are waiting for changes from upstream :stuck_out_tongue_closed_eyes:.

@JonathonReinhart this is a -pull request-, not an issue. The pull request cannot be worked on pending the dependency on hub changes, so there's no need to keep it open for now (related issue is https://github.com/docker/docker/issues/6400)

@jfrazelle @vdemeester @thaJeztah Sorry guys, too early in the morning :-)  I'll remove my comments  in a few minutes as to not clutter up the conversation here.


[QUESTION] Elements disappear(ed)

I am replicating the map view of the example app in my own project.
It seems there is a layout issue, which I am not sure of by what it is caused.
Basically all elements created by react are appearing for a very short moment and then they get animated to disappear
https://user-images.githubusercontent.com/1685307/136293706-99df3f44-1bc4-4f90-ba8a-4188e0233a3c.mov
Is this a known issue? I am on IOS 14.5

Ok, I think I misunderstood something. Gonna close the issue


Drop f0d749baaed896dfa02dcb6399f5febcca93931a.patch

Drop f0d749baaed896dfa02dcb6399f5febcca93931a.patch in patches/mainline and patches/tip when https://github.com/ClangBuiltLinux/linux/issues/1677 is addressed on the Linux side.

https://github.com/ClangBuiltLinux/continuous-integration2/commit/7061bdcd18ebf9343ff218a8bad5f3681de19219


Simplify unstructured decoding process

Many functions were relying on manifests to be of type []map[string]interface{} and then processing these manifests by converting them to unstructured pointers and then assigning the return value to the Object of this pointer, which was of type map[string]interface{}.
Instead, we now convert all manifests to unstructured.Unstructured at the start of a deployManifest request. This makes other functions easier to work with, and they are no longer converting the manifests to unstructured then back again!
Additionally, we are now working without pointers unless functions "require" them.
In short, this PR does:

convert all manifests ([]map[string]interface{}) at the start of a deployManifest operation to unstructured.Unstructured
simplifies our mergeManifests function in doing so
simplifies our kubernetes.SortManifests(...) func to no longer return an error

I'd recommend reading the comment on the kubernetes.ToUnstructured(...) func as well, went down an interesting rabbit hole with what happens when we just assume the user has passed in a valid Kubernetes manifest and do no decoding.

I forked this off of #35, so we can merge this first or not... and deal with the consequences later.


[BUG] Native library do not contain software renderer

Description
Browser panel doesn't render anything if chromium switches to software rendering(SwANGLE). This happens mostly if the application is started in some kind of Terminal Server / Citrix environment missing GPU acceleration or if the GPU drivers are a bit older.
I did some research and found out that jcefmaven native artifacts are missing the files vk_swiftshader.dll, vk_swiftshader_icd.json and vulkan-1.dll (or their Mac/Linux equivalents) which are mandatory for software rendering. Re-adding them fixes this issue for me.
Is there a special reason why the official CEF packages contains these files but jcefmaven does not?
(https://cef-builds.spotifycdn.com/cef_binary_105.3.36+g88e0038+chromium-105.0.5195.102_windows64.tar.bz2 vs. https://github.com/jcefmaven/jcefmaven/releases/download/105.3.36/jcef-natives-windows-amd64-jcef-0cdd84c+cef-105.3.36+g88e0038+chromium-105.0.5195.102.jar)
To Reproduce
I can reproduce this behaviour by forcing vulkan software or hardware renderer with the startup flags: "--use-gl=angle --use-angle=swiftshader" or "--use-gl=angle --use-angle=vulkan" on my AMD GPU, OSR is disabled. Just opening google.de in a simple JFrame.
See also https://chromium.googlesource.com/chromium/src/+/6b68d1e9b4fec5a7ded58c4ba0fd08078c740c70/docs/gpu/swiftshader.md
Environment

OS: Windows
Architecture: amd64
Version: JCEF with at least Chromium 100 onwards

For some reason swiftshader got lost in jcef when updating to chromium 105. I might be able to add ghe libraries back once I have time for it. Most likely in the upcoming week.
See: https://bitbucket.org/chromiumembedded/java-cef/commits/e611861ddbf0322c77d9de04e8806347539e3a17

Hey @HugoBoffin, sorry I did not have the time previously to look further into this. I built the most up2date version from jcef a while ago and it still does not contain the swiftshader native libraries. While there was a folder called "swiftshader" previously, it only contained another copy of gles. If the distribution scripts from jcef do not contain the required libraries, there is nothing my scripts and deployment can do about it. Would you like to file a bug report in the official jcef repo on bitbucket?

@HugoBoffin I created a pull request on the jcef repository adding them back. Once the PR is accepted, I will update jcefmaven to upstream :)
https://bitbucket.org/chromiumembedded/java-cef/pull-requests/96/add-vulkan-and-swiftshader-libraries-on

Will be included in build <IP_ADDRESS>.


[feature request] make the autosuggest__ CSS prefix configurable

All CSS classes have a autosuggest__ prefix. Could you make this prefix configurable via props?

Didn't get this in for 2.0.0, but can implement it without a breaking change. Help still wanted.


Matched Instance column doesn't work for CSV export

Description
When running Export Data from the Optimization Candidates screen, the output does not open correctly. this is because "matchedInstance" is a JSON object so the commas throw off the CSV format.
Steps to Reproduce
Click "Show Optimization Candidates"
Click "Export Data"
Open export.csv in Excel
See that the columns stop lining up after "matchedInstance".
Expected Behaviour
Columns should line up with the header.
Relevant Logs / Console output
See how the columns go wrong after "region"

Your Environment

NR1 CLI version used: datanerd/nr1/1.2.2 win32-x64 node-v10.16.3
Browser name and version: Chrome 79.0.3945.88
Operating System and version: Windows 10 v1903

Additional context
Not sure what data is supposed to be displayed there but I suggest it would just show the "type" value - will submit a PR for this accordingly, but feel free to change it if you want different information in there

:tada: This issue has been resolved in version 1.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Fixed #28058 -- Restored empty BoundFields evaluating to True.

https://code.djangoproject.com/ticket/28058

Cheers!
linking https://github.com/kmike/django-widget-tweaks/issues/44


Add guide with list of whitelisted domains

See this for now inside email-sent.js:
    'https://github.com/',
    'https://bitbucket.org/',
    'https://gitlab.com/',
    'https://phabricator.com/',
    'https://gitea.io/',
    'https://gogs.io/'

Should be clear for every user when filling the form.
https://github.com/italia/onboarding.developers.italia.it/blob/master/src/email-sent.js#L131

Fixed in https://github.com/italia/developers-italia-onboarding/pull/41. Closing.


Fixed hardcode translation

Fixed hardcode translation

@OlegLialin Japanese and Chinese are machine translations, right?

@OlegMakarenko yes, it's Japanese and Chinese are machine translations


gulp-ava error: process.send is not a function

When using gulp-ava get error:
[14:54:30] Using gulpfile ~/work/ctx-ava/gulpfile.js
[14:54:30] Starting 'test-ava'...
[14:54:31] 'test-ava' errored after 875 ms
[14:54:31] Error in plugin 'gulp-ava'
Message:
Command failed: /home/feofanov/work/ctx-ava/node_modules/gulp-ava/node_modules/.bin/ava /home/feofanov/work/ctx-ava/test/test/ava/test.js --color
TypeError: process.send is not a function
From CLI ava works fine

What gulp-ava version are you using?

0.2.0

Just tried<EMAIL_ADDRESS>with an official example (https://github.com/sindresorhus/gulp-ava) and it works. I assume you linked some future version of AVA into your directory.

I use ava 0.3.0

I just realized we are in ava repository, not gulp-ava. So, this is unclear why it's failing now. Could you please prepare a set of steps to reproduce your error and open this issue over at https://github.com/sindresorhus/gulp-ava? Thanks.


Move 'babelify' settings into grunt file to isolate from outside world

Move 'babelify' settings into grunt file to isolate from outside world
It is reported that putting babelify settings into package.json can 'contaminate' other NPM projects, so the settings are moved inline into the grunt config.

Tested this change briefly, and on my local setup it seems to "change nothing" - Would appreciate a double check from elsewhere.
Hope the format of the pull request is clean. It seemed to want to take my merge along with it for some reason.

OK, looks like I was beaten to it, so ignore/reject this :)


SLF4J-579 Export both org.slf4j and org.slf4j.helper in version 1.8

additionally to the default 2.x versions
Enable baselining against latest 1.7.36 version to enforce correct semantic versioning

Relates to #331

Can you tell where this happened? I somehow fail to read that from the print-out.

Baselining just emits warnings (no errors). Those are emitted at the end of the report:
[WARNING] org.slf4j: Excessive version increase; detected 2.0.10, suggested 1.8.0
[WARNING] org.slf4j.event: Excessive version increase; detected 2.0.10, suggested 2.0.0
[WARNING] org.slf4j.helpers: Excessive version increase; detected 2.0.10, suggested 2.0.0
[WARNING] org.slf4j.spi: Excessive version increase; detected 2.0.10, suggested 2.0.0
[INFO] Baseline analysis complete, 0 error(s), 4 warning(s)

This is due to the fact that by default maven-bundle-plugin exports each package with the bundle version (if no explicit package version annotation is found). SLF4J isn't following Semantic Versioning according to OSGi (and this should be fine here). I don't foresee any issues with that unless there is a major bundle version update because most consumers should have an import version range covering everything from the current major version till (excluding) the next one. What matters in this context would only be baselining errors caused by the manual 1.8.x version.

@ceki Any feedback?

@kwin I reviewed https://jira.qos.ch/browse/SLF4J-579 where @ceki mentions that there were many changes in org.slf4j.helpers and the classes in that package might not be compatible between slf4j 2.x and 1.x versions.
IIUC the baseline tests that you introduce check that compatibility at the API level, but how about implementation? Isn't there a chance that some of the org.slf4j.helpers classes behave differently, even if their API is compatible?
If that's the case, you might need "pinned" unit tests to verify that the behavior of the relevant classes remains stable going forward. Maybe that's just MessageFormatter, out of the many classes in the helpers package?

Baselining only checks method signatures but I consider this enough here. Semantically incompatible changes for the same method signature are very unlikely, but once there is an actual issue we can fix it then. Hopefully @ceki can decide if there already is an issue, but I understood from JIRA comments that only new stuff has been added in those packages.

Thank you for this report.
Please let me know if commit 02b36a269a9b13da is satisfactory.

This PR contains baselining which I consider crucial to maintain backwards compatibility. Any reason you don't want to integrate that @ceki?


Public API review

Review public API prior to RC.

Can't review the public API if you want have one...

Pardon? By review I mean just going through all the new public types, e.g. IEndpointFeature, Endpoint, etc, and make sure we are happy with the names, namespaces, etc.

It was a joke, I suppose we'll have to review those too

Prior to preview1 I'll review the public API myself.
Prior to RC I'll organize a meeting to review everything together.

Let's also discuss whether IEndpointSelectorPolicy should be async.

Reviewed API prior to preview1 - https://github.com/aspnet/Routing/issues/676
Will have another API review in preview2 milestone.

Do we need another review session for 2.2?

I don't think so. The only we didn't get through was the https://github.com/aspnet/Routing/blob/release/2.2/src/Microsoft.AspNetCore.Routing/Matching/INodeBuilderPolicy.cs and it's pretty esoteric.


WIP: MEEG data

Now opened into master.
related to https://github.com/scitran/reaper/pull/12

Okay, I finally got a view on my data after the reaper uploads it, but the fields are not being filled properly:

I suspect that I am doing something stupid, but I expected populating the nims_ properties would make them get used by the sorting mechanisms. I'll look into it a bit early next week to see if I can figure it out, but in the meantime if anyone has ideas, let me know.

I'm glad to see you got something going. You are correct in suspecting that setting the nims_ properties should get things populated.
Please set your API to insecure mode in config.toml and then hit the API directly with your browser at /api. Type in your email address, click the /projects link, and then use the debug links to drill down into your data. Please describe or post what you see.
JSONView is a helpful Chrome plugin.

Will do on Monday. Is that procedure currently documented somewhere? If not I'll find a README to put it into.

Okay, I investigated a bit, and now I have a few questions.

First, I see where one issue is coming from in the api module here:
https://github.com/scitran/api/blob/master/util.py#L132
Reproducing here:
        existing_group_ids = [g['_id'] for g in db.groups.find(None, ['_id'])]
        group_id_matches = difflib.get_close_matches(datainfo['group_id'], existing_group_ids, cutoff=0.8)
        if len(group_id_matches) == 1:
            group_id = group_id_matches[0]
            project_name = datainfo['project_name'] or 'untitled'
        else:
            group_id = 'unknown'
            project_name = datainfo['group_id'] + ('/' + datainfo['project_name'] if datainfo['project_name'] else '')

The group_id I want to use for this dataset is neuromag, but it doesn't exist, so it uses the group_id = 'unknown' and a new created project_name that is a combination of my group_id='neuromag' and my original project='eric_non_space'. Is this the desired behavior? I would have expected that it would add a group if it didn't exist. Do I need to do it manually first or something?

Second, when looking at what actually goes in the database using the suggested insecure mode, here is the demo data under projects:
{
files: [ ],
group: "SciTran",
name: "Testdata",
timestamp: "2013-04-19T09:15:29+00:00",
attachment_count: 0,
debug: {
group<EMAIL_ADDRESS>details<EMAIL_ADDRESS>sessions<EMAIL_ADDRESS>},
timezone: "UTC",
_id: "55db1b993e7413eb5f5a084b",
permissions: [
{
access: "admin",
_id<EMAIL_ADDRESS>}
]
},

And here is the new data:
{
files: [ ],
group: "unknown",
name: "neuromag/eric_non_space",
timestamp: "2015-08-24T13:40:24.923705+00:00",
attachment_count: 0,
debug: {
group<EMAIL_ADDRESS>details<EMAIL_ADDRESS>sessions<EMAIL_ADDRESS>},
timezone: "UTC",
_id: "55db1ea008315b196124a333"
}

In other words, there are no access permissions by default. I'm a bit surprised the admin account doesn't get access to every dataset by default, if they exist. So I'm wondering if 1) admin should be added as having perms on every dataset, and 2) how to modify add/modify permissions and groups.

Third, for Neuromag M/EEG data, right now there is no identifier in the data for which data belongs to which group when it appears on the scanner, so everything will belong to neuromag to start. We probably need to change this somehow, maybe by modifying the data after it appears in the database to start. How can this be done? This isn't an issue for getting this proof-of-concept working, but it will be an issue once we actually start trying to use scitran with real groups and real data.

Fourth, a small side note -- I see that my user has both wheel=True and root=True, do these really mean the same thing / are they redundant?
{
wheel: true,
preferences: { },
firstname: "Eric",
created: {
$date:<PHONE_NUMBER>935
},
lastname: "Larson",
modified: {
$date:<PHONE_NUMBER>935
},
avatar: "https://gravatar.com/avatar/bfd1a4884c9621f28ee42e38f652db99?s=512&d=mm",
_id<EMAIL_ADDRESS>root: true,
email<EMAIL_ADDRESS>}

Forcing the nims_group_id to be one that already exists (SciTran) I get something that I can access, but now only the project field gets populated automatically:

Resetting the database, changing more of the fields to match the sample data (e.g., nims_project, nims_session_id, nims_session_label, nims_session_subject), and re-reaping the file did not populate the other fields.
So from this it looks like group_id (neuromag) will not be automatically added, but if I use an existing name (SciTran) the project (eric_non_space') will be added, but session_*` will not. Is this the expected behavior?

Yes, groups have to exist for data privacy reasons. Fuzzy matching on group names is in place, though, because people can't be trusted to correctly type their own name. We have a UI for creating new groups. It is behind the "gear" icon in the SDM UI.
Site admins have access to all date regardless of permissions. Permissions for an implicitly created project are inherited from the roles of that group. To modify permissions of a project through the UI, click on the project name and update the permissions. Adding permission to the project in 'unknown' does not make much sense, though. You should rather move that session to a new project in the proper group. That's under "Actions" in the UI.
wheel means that you can become a superuser, root means that you are currently acting as a superuser.
I don't know what's going on with your session and acquisition data. You need to drill further down in to the raw API.

Phew! That was a bit of a pain for me to figure out, but in the process I've made some improvements to the api code (https://github.com/scitran/api/pull/36) to make debugging a little bit easier. With second-to-last commit to this PR, I got here:

Turns out I had to have nims_metadata_status return something non-None. I don't think that parameter was described previously, so I added a short description in the code here, hopefully it's reasonable.
After that commit, my session, acquisition, and description were correct, and I just needed to get the "Data Type" field populated properly, which sat at (unknown). I couldn't figure out how to do it in /data, turns out I needed to change something in /api. I've put that in as part of https://github.com/scitran/api/pull/36, but it's easy to separate if need be.

With that, I think this is ready for review/merge. It only depends on https://github.com/scitran/api/pull/36 to get the filetype to appear correctly, so this could safely be merged before that, I think.

One remaining thing I'm not sure about is if I need to do anything with this file, though:
https://github.com/scitran/data/blob/master/scitran/data/modules.json

Congratulations! And thanks for staying the course.
I'm glad you found the hack spot in api/util.py. It is definitely high on the list of things that need to be fixed as part of the refactoring I proposed in scitran/scitran#2.
I don't know if you have to do anything with modules.json. Pinging @kevlarkevin for advice.
Overall, :+1:

@gsfr I think I was able to figure out the modules.json from the code, it maps a domain.kind to a Python module, in this case meeg.FIF to the meeg module. This json gets read in and used by module_by_type(...) to import the proper module.
Now I need to make Travis happy again...

All better @gsfr, now it passes the tests


Enforce Limited Emoji Set Backend

As of now, setting a limited emoji set just means users only see specific emojis to react with. This means that they can still send requests for any emoji (that return successful), even if it is not on the limited list.

Closed because new Reactions plugin fixes this issue.


Oracle migration script issue with Callback URL value as NULL

Description:
In oracle empty strings '' are considered same as null. Therefore, when executing following query,
UPDATE IDN_OAUTH_CONSUMER_APPS SET CALLBACK_URL='' WHERE CALLBACK_URL IS NULL /
It doesn't change anything, the value remains null. Therefore, users can't update the keys in application keys page since callback value is "null"  still in migrated apps.
ref: https://stackoverflow.com/questions/49678159/insert-empty-string-in-oracle

@shilmyhasan Could you please close the issue if this is solved ?

@HiranyaKavishani This particular query is available only when migrating from 3.1.0. I will check on this during 3.1.0 to 4.1.0 migration.

Hi Team,
The issue is resolved.
Thanks


Terra demographics banner

Summary
This is the demographics banner for Terra.
Additional Details
This is a barebones demographics banner without tags or link support. These components will be added in the future.
Thanks for contributing to Terra.
@cerner/terra

How is content supposed to wrap for this component? Should we enforce some seperation from the left and right columns?
@mmalaker

I had to make some changes to get rid of the I18n stuff, so I opened up a new pull request. https://github.com/cerner/terra-core/pull/204


improve ability to remove nulls from MutableLists

I think we should make at least one of the following changes to MutableList:

Make shared formal Integer remove(Element&Object element) accept nulls,
Make shared formal void prune() return the number of affected entries

Currently, in a generic context, I'm removing an item from a list with:
    Boolean remove(Item item) {
            if (exists item) {
                return list.remove(item) != 0;
            }
            value oldSize = list.size;
            list.prune();
            return oldSize != list.size;
    }

The code above is a performance bug, too, for lists with slow size operations.

Let's make prune() return the count then.
Sent from my iPhone

On 08 Apr 2016, at 21:40, John Vasileff<EMAIL_ADDRESS>wrote:
I think we should make at least one of the following changes to MutableList:
Make shared formal Integer remove(Element&Object element) accept nulls,
Make shared formal void prune() return the number of affected entries
Currently, in a generic context, I'm removing an item from a list with:
Boolean remove(Item item) {
        if (exists item) {
            return list.remove(item) != 0;
        }
        value oldSize = list.size;
        list.prune();
        return oldSize != list.size;
}

The code above is a performance bug, too, for lists with slow size operations.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

Fixed in https://github.com/ceylon/ceylon-sdk/pull/610


[i2c] Add multi-controller and broadcast command support

Add support for...

Bus arbitration for both the controller (multi-controller environments) and the target (broadcast commands)
Consolidated transaction boundary checking in a bus monitor

Start / Stop detection
Bus idle timeouts (SCL and SDA continuously high)
Bus active timeouts (SCL continuously low)
Bus free status signal
Adjust controller and target FSMs to respond to bus monitor's outputs.

Clock synchronization
Simultaneously active host and target modules
SMBus's clock extension timeout

Accumulate count over an entire transaction; don't reset per instance

Change fault reporting (SDA interference, SDA instability, SCL interference) to harmonize with multi-controller support.

"SDA interference" no longer triggers from SDA being 0 at the first controller enablement. Just read the RX Oversample CSR instead.
"SCL interference" should be ignored in multi-controller environments, as this is simply clock synchronization. Leave the interrupt masked.

Rework the functionality meant to prevent stale data in the TX FIFO from transmitting from the target. Avoid races with software via an optional mode that simply halts the FSM and stretches the clock on the first bit for every read transfer.
Fix up address matching to not trigger if the mask is completely clear.
Fix up "fault" injection timing so pulses do not get deleted by CDC instrumentation and so they don't trigger assertions.

Whoops, forgot that the bus monitor can't be active right away at chip-level -- The pads aren't necessarily connected to the IP's pins immediately, and on earlgrey, those input muxes start with SCL and SDA low!


function id

use a function identification technology to recognize functions and associate with their name and other metadata (like source library).
let rules match on these names/metadata.
consider filtering out rule matches in well known library functions to reduce FPs (by default).
discussion: https://github.com/fireeye/capa/discussions/414

#431 using capa features to identify functions

#434 using vivisect vamp backend to identify functions

#435 using FLIRT to identify functions

having function id leads to changes in the following places:

an initial pass across all functions to match signatures.
a.  add symbol name to function (and any local names)
b. attach metadata to address indicating signature hit
during feature extraction, skip functions that have metadata attached to address and indicating library function
during feature extraction, inspect call targets, and if they're to a matched function, emit API feature with recovered name

question: do we assume all signature matches are library functions?
initial take: seems simplest for now. some technologies (e.g. FLIRT) don't necessarily have associated library/category/metadata.

do underlying backends provide function id on their own? or do they provide the APIs that capa needs to do its own form of function id? i think its this second one. though maybe we should support both, so IDA can continue to mark lib functions?
to support underlying function id:

has_function_id(va: int) -> bool
get_function_name(va: int) -> str

to support FLIRT matching by capa:

get_bytes(va: int, size: int) -> bytes

how do we provide function names/symbols to the backends as they're extracting features?

+1 on supporting both to leverage backend's function ID plus capa's own ID on top.
We could change the extractor get_functions signature to support modifiers, e.g. skip_libs, like the current IDA helper function does.

this is closed in #446