HMIP-USB not working: no GPIO/USB connected RF-hardware found

Hello,
I am trying desperately to make this work on a RPI4 system with OMV5 on it.
So far I managed to get the TTYUSB0 device attached to the docker like this:
Host dev:
crw-rw---- 1 root dialout 188,   0 Aug 29 20:47 ttyUSB0
Container dev:
crw-rw----    1 root     20        188,   0 Aug 29 21:32 ttyUSB0
However the startup log file always tells me no RF device has been found:

Starting /etc/init.d/S11InitRFHardware
Identifying Homematic RF-Hardware: HmIP-RFUSB (1b1f:c020) USB stick is connected -> ensure you do the following on the host:

modprobe cp210x
echo 1b1f c020 >/sys/bus/usb-serial/drivers/cp210x/new_id
BidCos-RF: none, HmIP: none, OK

Starting /etc/init.d/S12UpdateRFHardware
Updating Homematic RF-Hardware: no GPIO/USB connected RF-hardware found

What could be the issue? Anything I tried from other similar reported errors doesn´t help.
I tried starting the container from the portainer ui as well as from by direct docker command:
docker run --device /dev/ttyUSB0 --name CCU angelnu/ccu:latest
Log:

Starting CCU services
Starting /etc/init.d/S00InstallAddon
Starting /etc/init.d/S00watchdog
S00watchdog - skipping
Starting /etc/init.d/S01InitHost
S01InitHost - defaults
Checking device
PiVCCU kernel module is not installed and did not recognize HW Hardware : BCM2711
-> Homematic PCB adapter will not work!!!
Starting /etc/init.d/S02InitRTC
S02InitRTC - skipping
Starting /etc/init.d/S03InitURandom
S03InitURandom - skipping
Starting /etc/init.d/S04CheckFactoryReset
Checking for Factory Reset: not required
Starting /etc/init.d/S04CheckResizeLocalFS
S04CheckResizeLocal - skipping
Starting /etc/init.d/S05CheckBackupRestore
Checking for Backup Restore: not required
Starting /etc/init.d/S05avahi-setup.sh
Starting /etc/init.d/S06InitSystem
Initializing System: mkdir: can't create directory '/var/lib/dbus': No such file or directory
chmod: /var/lib/dbus: No such file or directory
OK
Starting /etc/init.d/S07DisableHdmi
/entrypoint.sh: line 21: /etc/init.d/S07DisableHdmi: Permission denied
Starting /etc/init.d/S07logging
Starting logging: OK
Starting /etc/init.d/S10udev
S10udev - skipping
Starting /etc/init.d/S11InitRFHardware
Identifying Homematic RF-Hardware: HmIP-RFUSB (1b1f:c020) USB stick is connected -> ensure you do the following on the host:

modprobe cp210x
echo 1b1f c020 >/sys/bus/usb-serial/drivers/cp210x/new_id
BidCos-RF: none, HmIP: none, OK

Starting /etc/init.d/S12UpdateRFHardware
Updating Homematic RF-Hardware: no GPIO/USB connected RF-hardware found
Starting /etc/init.d/S13irqbalance
S13irqbalance - skipping
Starting /etc/init.d/S21rngd
S21rngd - skipping
Starting /etc/init.d/S30dbus
Starting system message bus: done
Starting /etc/init.d/S31bluetooth
S31bluetooth - skipping
Starting /etc/init.d/S40network
S40network - skipping
Starting /etc/init.d/S45ifplugd
S45ifplugd - skipping
Starting /etc/init.d/S48MigrateSecuritySettings
Starting : OK
Starting /etc/init.d/S48ntp
Starting ntpd: OK
Starting /etc/init.d/S49hs485d
Preparing start of hs485d: no Hm-Wired hardware found
Starting /etc/init.d/S49xinetd
Starting xinetd: OK
Starting /etc/init.d/S50eq3configd
Starting eq3configd: OK
Starting /etc/init.d/S50lighttpd
Starting lighttpd: creating new SSL cert... OK
Starting /etc/init.d/S50ssdpd
Starting ssdpd: OK
Starting /etc/init.d/S50sshd
Starting /etc/init.d/S55InitAddons
Initializing Third-Party Addons: OK
Starting /etc/init.d/S58LGWFirmwareUpdate
Starting LGWFirmwareUpdate: ...OK
Starting /etc/init.d/S59SetLGWKey
Setting LAN Gateway keys: OK
Starting /etc/init.d/S59snmpd
cat: can't open '/etc/config/snmp/snmpd.conf': No such file or directory
Starting /etc/init.d/S60hs485d
Starting hs485d: no Hm-Wired hardware found
Starting /etc/init.d/S60multimacd
Starting multimacd: SKIPPING (missing HM_HOST_GPIO_UART)
Starting /etc/init.d/S60openvpn
Starting /etc/init.d/S61rfd
Starting rfd: ........................................ERROR
Starting /etc/init.d/S62HMServer
Starting HMIPServer: (no HMIP dev) ....OK
Starting /etc/init.d/S69UDSServices
Starting eq3-uds-services: OK
Starting /etc/init.d/S70ReGaHss
Starting ReGaHss: .OK
Starting /etc/init.d/S97CloudMatic
Starting CloudMatic: OK
Starting /etc/init.d/S97NeoServer
Starting NeoServer: OK
Starting /etc/init.d/S98StartAddons
Starting Third-Party Addons: OK
Starting /etc/init.d/S98crond
Starting crond: OK
Starting /etc/init.d/S99SetupLEDs
Setup onboard LEDs: booted, OK
Done starting CCU services
Register trap for SIGTERM

HM_mode seems all empty:
sudo docker exec CCU cat /var/hm_mode

HM_HMIP_DEV=''
HM_HMIP_DEVNODE=''
HM_HMRF_DEV=''
HM_HMRF_DEVNODE=''
HM_HOST='Unknown'
HM_HOST_GPIO_RESET=''
HM_HOST_GPIO_UART=''
HM_LED_GREEN=''
HM_LED_GREEN_MODE1='none'
HM_LED_GREEN_MODE2='none'
HM_LED_RED=''
HM_LED_RED_MODE1='none'
HM_LED_RED_MODE2='none'
HM_LED_YELLOW=''
HM_LED_YELLOW_MODE1='none'
HM_LED_YELLOW_MODE2='none'
HM_MODE='NORMAL'
HM_RTC=''

Did you try running the commands mentioned by the log?
    modprobe cp210x
    echo 1b1f c020 >/sys/bus/usb-serial/drivers/cp210x/new_id

Another thing that would be useful is that you post the output from deploy.sh. This script takes care of preparing the host to run the container.

Yes I ran the modprobe and echo commands which make the ttyUSB0 available in the first place.
Without these commands the ttyUSB0 doesnt show up at all in /dev/
I watched the dmesg output and it sees the HMIP-USB being connected/disconnected.
How do I find the output of deploy.sh?
I could find the container inspection:

{
"AppArmorProfile": "",
"Args": [],
"Config": {
"AttachStderr": false,
"AttachStdin": false,
"AttachStdout": false,
"Cmd": [
"/entrypoint.sh"
],
"Domainname": "",
"Entrypoint": null,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"ExposedPorts": {
"2001/tcp": {},
"22/tcp": {},
"80/tcp": {},
"8181/tcp": {}
},
"Hostname": "35d08af519be",
"Image": "angelnu/ccu:latest",
"Labels": {},
"OnBuild": null,
"OpenStdin": false,
"StdinOnce": false,
"Tty": false,
"User": "",
"Volumes": {
"/usr/local": {}
},
"WorkingDir": ""
},
"Created": "2020-08-29T19:32:23.221485484Z",
"Driver": "overlay2",
"ExecIDs": [
"efb9c2e2e0010ab4284185b2ea682af5684ce2f645f406f252a4fdecdf40a2ff"
],
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/32905d85f111b9c61262b6a44185ba001fff8d0f0a971410d5803175755c5149-init/diff:/var/lib/docker/overlay2/cfee9866d1518b3ffb970c98898ed9c2fc7d998d552db1e7cfa4b91e5ac1c9c0/diff:/var/lib/docker/overlay2/0af05178644a68e6d6ad6bea01a152a446e95864854dab422146d1309cfe3f7b/diff",
"MergedDir": "/var/lib/docker/overlay2/32905d85f111b9c61262b6a44185ba001fff8d0f0a971410d5803175755c5149/merged",
"UpperDir": "/var/lib/docker/overlay2/32905d85f111b9c61262b6a44185ba001fff8d0f0a971410d5803175755c5149/diff",
"WorkDir": "/var/lib/docker/overlay2/32905d85f111b9c61262b6a44185ba001fff8d0f0a971410d5803175755c5149/work"
},
"Name": "overlay2"
},
"HostConfig": {
"AutoRemove": false,
"Binds": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceWriteIOps": null,
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"CapAdd": [
"AUDIT_WRITE",
"CHOWN",
"DAC_OVERRIDE",
"FOWNER",
"FSETID",
"KILL",
"MKNOD",
"NET_BIND_SERVICE",
"NET_RAW",
"SETFCAP",
"SETGID",
"SETPCAP",
"SETUID",
"SYS_CHROOT"
],
"CapDrop": [
"AUDIT_CONTROL",
"BLOCK_SUSPEND",
"DAC_READ_SEARCH",
"IPC_LOCK",
"IPC_OWNER",
"LEASE",
"LINUX_IMMUTABLE",
"MAC_ADMIN",
"MAC_OVERRIDE",
"NET_ADMIN",
"NET_BROADCAST",
"SYSLOG",
"SYS_ADMIN",
"SYS_BOOT",
"SYS_MODULE",
"SYS_NICE",
"SYS_PACCT",
"SYS_PTRACE",
"SYS_RAWIO",
"SYS_RESOURCE",
"SYS_TIME",
"SYS_TTY_CONFIG",
"WAKE_ALARM"
],
"Capabilities": null,
"Cgroup": "",
"CgroupParent": "",
"ConsoleSize": [
0,
0
],
"ContainerIDFile": "",
"CpuCount": 0,
"CpuPercent": 0,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpuShares": 0,
"CpusetCpus": "",
"CpusetMems": "",
"DeviceCgroupRules": null,
"DeviceRequests": null,
"Devices": [
{
"CgroupPermissions": "rwm",
"PathInContainer": "/dev/ttyUSB0",
"PathOnHost": "/dev/ttyUSB0"
}
],
"Dns": [],
"DnsOptions": null,
"DnsSearch": null,
"ExtraHosts": [],
"GroupAdd": null,
"IOMaximumBandwidth": 0,
"IOMaximumIOps": 0,
"Init": false,
"IpcMode": "private",
"Isolation": "",
"KernelMemory": 0,
"KernelMemoryTCP": 0,
"Links": null,
"LogConfig": {
"Config": {},
"Type": "json-file"
},
"MaskedPaths": null,
"Memory": 0,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": null,
"NanoCpus": 0,
"NetworkMode": "bridge",
"OomKillDisable": null,
"OomScoreAdj": 0,
"PidMode": "",
"PidsLimit": null,
"PortBindings": {},
"Privileged": true,
"PublishAllPorts": true,
"ReadonlyPaths": null,
"ReadonlyRootfs": false,
"RestartPolicy": {
"MaximumRetryCount": 0,
"Name": "no"
},
"Runtime": "runc",
"SecurityOpt": [
"label=disable"
],
"ShmSize": 67108864,
"UTSMode": "",
"Ulimits": null,
"UsernsMode": "",
"VolumeDriver": "",
"VolumesFrom": null
},
"HostnamePath": "/var/lib/docker/containers/35d08af519be9d646037c5a9e80364c5e8273d7ddf1da999ee3888caafc758a2/hostname",
"HostsPath": "/var/lib/docker/containers/35d08af519be9d646037c5a9e80364c5e8273d7ddf1da999ee3888caafc758a2/hosts",
"Id": "35d08af519be9d646037c5a9e80364c5e8273d7ddf1da999ee3888caafc758a2",
"Image": "sha256:c6db9288b43f89a37398caaa23abb309b109cd715ec672e15a8cac6527829724",
"LogPath": "/var/lib/docker/containers/35d08af519be9d646037c5a9e80364c5e8273d7ddf1da999ee3888caafc758a2/35d08af519be9d646037c5a9e80364c5e8273d7ddf1da999ee3888caafc758a2-json.log",
"MountLabel": "",
"Mounts": [
{
"Destination": "/usr/local",
"Driver": "local",
"Mode": "",
"Name": "01c0a2d351ddd22e1bab956f6db96ab31058942cc3af41ffc2ae2923cf6f4b15",
"Propagation": "",
"RW": true,
"Source": "/var/lib/docker/volumes/01c0a2d351ddd22e1bab956f6db96ab31058942cc3af41ffc2ae2923cf6f4b15/_data",
"Type": "volume"
}
],
"Name": "/CCU",
"NetworkSettings": {
"Bridge": "",
"EndpointID": "22041c401e95b9f9f5374ae3b88c5a11c88e7477fd902701763c8c229a0be9c0",
"Gateway": "<IP_ADDRESS>",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"HairpinMode": false,
"IPAddress": "<IP_ADDRESS>",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:11:00:03",
"Networks": {
"bridge": {
"Aliases": null,
"DriverOpts": null,
"EndpointID": "22041c401e95b9f9f5374ae3b88c5a11c88e7477fd902701763c8c229a0be9c0",
"Gateway": "<IP_ADDRESS>",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAMConfig": {},
"IPAddress": "<IP_ADDRESS>",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"Links": null,
"MacAddress": "02:42:ac:11:00:03",
"NetworkID": "9d04939007466f190b61e7cc1e7aa0565b4976540ef9334ed80ac007463618e1"
}
},
"Ports": {
"2001/tcp": [
{
"HostIp": "<IP_ADDRESS>",
"HostPort": "32773"
}
],
"22/tcp": [
{
"HostIp": "<IP_ADDRESS>",
"HostPort": "32775"
}
],
"80/tcp": [
{
"HostIp": "<IP_ADDRESS>",
"HostPort": "32774"
}
],
"8181/tcp": [
{
"HostIp": "<IP_ADDRESS>",
"HostPort": "32772"
}
]
},
"SandboxID": "131d06bb031c4aac0f1bd8bf0a5a3a51c1cb7f2aff502c424f22fdc75bda0038",
"SandboxKey": "/var/run/docker/netns/131d06bb031c",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null
},
"Path": "/entrypoint.sh",
"Platform": "linux",
"Portainer": {
"ResourceControl": {
"Id": 13,
"ResourceId": "35d08af519be9d646037c5a9e80364c5e8273d7ddf1da999ee3888caafc758a2",
"SubResourceIds": [],
"Type": 1,
"UserAccesses": [],
"TeamAccesses": [],
"Public": false,
"AdministratorsOnly": true,
"System": false
}
},
"ProcessLabel": "",
"ResolvConfPath": "/var/lib/docker/containers/35d08af519be9d646037c5a9e80364c5e8273d7ddf1da999ee3888caafc758a2/resolv.conf",
"RestartCount": 0,
"State": {
"Dead": false,
"Error": "",
"ExitCode": 0,
"FinishedAt": "0001-01-01T00:00:00Z",
"OOMKilled": false,
"Paused": false,
"Pid": 11218,
"Restarting": false,
"Running": true,
"StartedAt": "2020-08-29T19:32:24.47103835Z",
"Status": "running"
}
}

I found the issue!
The HMIP-RFUSB doesn´t like any sort of USB extension, not even a quality USB hub with 10 cm cable.
Whats strange is that it finds the module using lsusb etc. but it won´t be registered by the CCU software, probably since only that software tries to talk to the daughter board inside the stick.
I guess the power supply to the RF board might be at its limit on my unit.

BidCos-RF: none, HmIP: HMIP-RFUSB, OK
Starting /etc/init.d/S12UpdateRFHardware
Updating Homematic RF-Hardware: HMIP-RFUSB: 2.8.6, OK

Just a shame my USB ports are normally blocked for SSDs/HDDs and the chunky HMIp-RFUSB doesn´t fit but thats another issue...

I had missed your comment that you were using an own serial2usb adapter x-) Glad you got it working.
Exactly: homematic uses a custom serial device driver - this is what limits so much the HW. In the meanwhile the author of pivccu wrote its own device driver for several adapters so I use his in deploy.sh. So in other words: this docker container supports the HW supported by pivccu.
Any reason why you do not connect the PCB to the Raspberry GPIOs directly?

I am using the official HMIP-RFUSB module.
I might look at other USB adapters, good to know there´s more supported through pivccu.
The flexibility of USB is what I prefer, there´s only one GPIO connector but plenty of USB (if adapters work with the devices XD).
Still planning to use the GPIO for something else (current monitor).
In any case thanks for your work on making this docker!


[SuperEditor][SuperReader][Android] Fix long-press firing after scrolling and releasing the pointer (Resolves #1535)

[SuperEditor][SuperReader][Android] Fix long-press firing after scrolling and releasing the pointer. Resolves #1535
When the editor or reader is inside a CustomScrollview, trying to drag is opening the software keyboard and changing the selection. This was partially solved in https://github.com/superlistapp/super_editor/pull/1536. However one issue remained:
Starting a scroll gestures and the releasing the pointer is still causing the issue. The cause is that, when the timer fires, we aren't scrolling anymore.
To fix that, this PR cancels the long-press timer as soon as the user starts the scrolling gesture.
I kept the existing test, which simulates the user dragging and holding down the pointer, and added another one simulating the user starting a drag gesture and then releasing the pointer.

@Jethro87 could you please try this PR?

@angelosilvestre This looks good. I am not experiencing the issue on Android anymore.

I cherry-picked this to unblock some other work: https://github.com/superlistapp/super_editor/pull/1553


IIIF: handle DecompressionBombError

We sometimes get notifications about internal server errors from the IIIF endpoint, caused by an unhandled PIL.Image.DecompressionBombError: Image size<PHONE_NUMBER> pixels) exceeds limit of 178956970 pixels, could be decompression bomb DOS attack.
Exception on /iiif/record:zvvmh-nan78:first_flood_detection.tif/full/!800,800/0/default.jpg [GET]
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/flask/app.py", line 2529, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/flask_resources/resources.py", line 65, in view
    return view_meth()
           ^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/flask_resources/content_negotiation.py", line 116, in inner_content_negotiation
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/flask_cors/decorator.py", line 130, in wrapped_function
    resp = make_response(f(*args, **kwargs))
                         ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/flask_resources/parsers/decorators.py", line 51, in inner
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/flask_resources/parsers/decorators.py", line 51, in inner
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/flask_resources/parsers/decorators.py", line 51, in inner
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/invenio_rdm_records/resources/iiif.py", line 131, in _wrapper
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/invenio_rdm_records/resources/iiif.py", line 241, in image_api
    to_serve = self.service.image_api(
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/invenio_rdm_records/services/iiif/service.py", line 167, in image_api
    image = IIIFImageAPIWrapper.open_image(data)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/flask_iiif/api.py", line 581, in open_image
    image = Image.open(source)
            ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/PIL/Image.py", line 3482, in open
    im = _open_core(
         ^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/PIL/Image.py", line 3466, in _open_core
    _decompression_bomb_check(im.size)
  File "/usr/lib/python3.12/site-packages/PIL/Image.py", line 3365, in _decompression_bomb_check
    raise DecompressionBombError(msg)
PIL.Image.DecompressionBombError: Image size<PHONE_NUMBER> pixels) exceeds limit of 178956970 pixels, could be decompression bomb DOS attack.

I was unsure about which HTTP status code to use here and was almost leaning towards a 5XX, but in this StackOverflow question the code 403 was recommended (for valid reasons, IMO): https://stackoverflow.com/questions/15192477/http-status-code-when-single-request-asks-for-too-large-resource-or-too-many-of

Update: This error gets triggered by the default previewer for TIFF images, e.g. on the landing page for: https://researchdata.tuwien.ac.at/records/zvvmh-nan78
Neither with nor without this PR does the previewer display anything meaningful for the TIFF image, but with this PR it's at least not a 500 response anymore.


Hardy state

Finished Hardy State Task for Superposition Kata

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: melmatt7 sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Pulling from the wrong feed

When I set up my username and ClientID it pulls photos from the wrong feed, for which I don't even have access (it pulls from helaspamexico, but instead it should be pulling from helaspa -> I am logged in to this and have generated ClientID for this)... Has anyone experienced something similar?
Best Regards

This is likely a bug in the plugin. In order to fetch media from a particular user, you need to provide the user's ID, so the first step of the plugin is to fetch an array of users that match the username string (i.e. "helaspa"). The plugin assumes that the first result is the actual user, however, in your case, it's apparently not. When I have some time, I'll fix the plugin to be more strict about which user is selected after the first request.

Thanks for an answer.. Temporarily I solved this by adding $ behind the username, but I think it's not a pefect solution..


unable reprocess the data with old schema through Apache hudi ,Caused by: HoodieUpsertException: Error upserting bucketType UPDATE for partition :34

Hi All,
I am unable to re-process the data with old schema through apache Hudi.
Getting below exception while running the job.
*****ERROR
Job aborted due to stage failure. Caused by: HoodieUpsertException: Error upserting bucketType UPDATE for partition :34 Caused by: HoodieException: org.apache.hudi.exception.HoodieException: java.util.concurrent.ExecutionException: org.apache.hudi.exception.HoodieException: operation has failed Caused by: HoodieException: java.util.concurrent.ExecutionException: org.apache.hudi.exception.HoodieException: operation has failed Caused by: ExecutionException: org.apache.hudi.exception.HoodieException: operation has failed Caused by: HoodieException: operation has failed Caused by: InvalidRecordException: Parquet/Avro schema mismatch: Avro field 'EVAR106' not found
at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.handleUpsertPartition(BaseSparkCommitActionExecutor.java:288)
at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.lambda$execute$ecf5068c$1(BaseSparkCommitActionExecutor.java:139)
at org.apache.spark.api.java.JavaRDDLike.$anonfun$mapPartitionsWithIndex$1(JavaRDDLike.scala:105)
at org.apache.spark.api.java.JavaRDDLike.$anonfun$mapPartitionsWithIndex$1$adapted(JavaRDDLike.scala:105)
at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsWithIndex$2(RDD.scala:920)
at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsWithIndex$2$adapted(RDD.scala:920)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:60)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:380)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:344)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:60)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:380)
at org.apache.spark.rdd.RDD.$anonfun$getOrCompute$1(RDD.scala:393)
at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1445)
at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1355)
at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1419)
at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1238)
at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:391)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:342)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:60)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:380)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:344)
at org.apache.spark.scheduler.ResultTask.$anonfun$runTask$3(ResultTask.scala:75)
at com.databricks.spark.util.ExecutorFrameProfiler$.record(ExecutorFrameProfiler.scala:110)
at org.apache.spark.scheduler.ResultTask.$anonfun$runTask$1(ResultTask.scala:75)
at com.databricks.spark.util.ExecutorFrameProfiler$.record(ExecutorFrameProfiler.scala:110)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:55)
at org.apache.spark.scheduler.Task.doRunTask(Task.scala:150)
at org.apache.spark.scheduler.Task.$anonfun$run$1(Task.scala:119)
at com.databricks.spark.util.ExecutorFrameProfiler$.record(ExecutorFrameProfiler.scala:110)
at org.apache.spark.scheduler.Task.run(Task.scala:91)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$13(Executor.scala:812)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1643)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:815)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at com.databricks.spark.util.ExecutorFrameProfiler$.record(ExecutorFrameProfiler.scala:110)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:671)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hudi.exception.HoodieException: org.apache.hudi.exception.HoodieException: java.util.concurrent.ExecutionException: org.apache.hudi.exception.HoodieException: operation has failed
at org.apache.hudi.table.action.commit.SparkMergeHelper.runMerge(SparkMergeHelper.java:102)
at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.handleUpdateInternal(BaseSparkCommitActionExecutor.java:317)
at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.handleUpdate(BaseSparkCommitActionExecutor.java:308)
at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.handleUpsertPartition(BaseSparkCommitActionExecutor.java:281)
... 38 more
Caused by: org.apache.hudi.exception.HoodieException: java.util.concurrent.ExecutionException: org.apache.hudi.exception.HoodieException: operation has failed
at org.apache.hudi.common.util.queue.BoundedInMemoryExecutor.execute(BoundedInMemoryExecutor.java:143)
at org.apache.hudi.table.action.commit.SparkMergeHelper.runMerge(SparkMergeHelper.java:100)
... 41 more
Caused by: java.util.concurrent.ExecutionException: org.apache.hudi.exception.HoodieException: operation has failed
i have tried with  the hoodie.datasource.write.reconcile.schema=true property ,but didn't work for me .
Please let me know the resolution  for the issue.

The schema evolutino needs to follow https://hudi.apache.org/docs/schema_evolution.
For these old schema data, it is recommended that you manually convert to the new schema data before merging.

the expectation w/ reconcile schema is.
lets say initial schema is schemaA
and then table got upgraded to schemaB (by adding new columns to the end).
after this upgrade, if you try to ingest records w/ schemaA, by default it will fail. but if you set the reconcile schema config, it will succeed.
but do remember that schemaB should be backwards compataible w/ schemaA.

if you can provide us w/ table schema and schema of incoming batch, we can see whats the issue.

@nleena123 : any updates to follow up on here. if you got it resolved, do close the github issue.

closing due to inactivity. highly recommend to re-open if you are still having issues.


move metric initialisation upfront connection creation to avoid potential npe

We have a setup where we regularly create a new Memcached client to verify memcached is still running. Now what happens if you totally cut down your network (no LAN & WLAN). You'll get a NPE within MemcachedConnection as the first connection try fails, and then the failure handling tries again trying to increment a metric, which at this time isn't init'd.
java.lang.NullPointerException
    at net.spy.memcached.MemcachedConnection.queueReconnect(MemcachedConnection.java:714)
    at net.spy.memcached.MemcachedConnection.createConnections(MemcachedConnection.java:241)
    at net.spy.memcached.MemcachedConnection.<init>(MemcachedConnection.java:174)
    at net.spy.memcached.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:196)
    at net.spy.memcached.MemcachedClient.<init>(MemcachedClient.java:207)

This alone wouldn't be too bad if during init a Selector was opened already, which then isn't closed any more. So on mac each time the Selector is opened 2 PIPEs and 1 KQUEUE are created, this leads eventually to a create AF_UNIX socket: Too many open files in system error making the whole system unresponsive.
  public MemcachedConnection(final int bufSize, final ConnectionFactory f,
      final List<InetSocketAddress> a, final Collection<ConnectionObserver> obs,
      final FailureMode fm, final OperationFactory opfactory) throws IOException {
    //...
    selector = Selector.open();

I've fixed the NPE by moving the metric init to an earlier place, but maybe some try catch around the whole init method would be better to close the all potentially opened handles before bubbling the exception.

@daschl any plans to merge this into the main branch?

@philnate yeah I think we can merge something like this, but we should also see how we can address the NPE, right?
Did you get a chance to sign the CLA yet? See https://developer.couchbase.com/open-source-projects on "how to contribute code". Thanks!

Sounds good. Well my understanding was that the ordering of the commands where wrong. So the proposed solution would be to make sure the metrics are initialized before the connection is tried to be establised. What else should be addressed regarding the NPE, that looked like the main reason to me?

Is there no plans to fix this?


Add gif to show the modal in action

Description
Add a gif and a line of text for better documentation
Visual changes

Thank you for contributing, @degodfather!
Currently we use an automated script to take screenshots for all docs so it's easy to update them whenever designs change.
To maintain consistency, I'm not going to merge this PR for now.


[SPARK-10764][ML] Add optional caching to Pipelines

What changes were proposed in this pull request?
This PR is to allow users to persist the output of a particular PipelineStage in the Pipeline. It also allows users to persist only a subset of output that is actually needed for the next stage instead of the whole big data frame.
For example, while running LDA a pipeline may look something like Tokenizer -> StopWordRemover -> CountVectorizer -> LDA. Here a user might want to persist the data frame coming out of CountVectorizer so that the LDA iterations work on this persisted data. Also, we need the flexibility to persist only the subset of columns (say docId, and features) that are needed for LDA rather than all extra columns that may have been added by the previous stages.
Also, another issue is that once the pipeline is fit on a data, for the transform on the same data it has to pass through all the previous stages again. In many cases this can be optimized if we already had access to data cached in the intermediate stage. In the example above, it will be useful if the data previously persisted in CountVectorizer can be reused to get topicDistributions from the LDA on the train data set itself. Otherwise, the data needs to again pass through all the stages.
For this, the PipelineStage exposes following methods

persist(level: StorageLevel, colExprs: String*) - where the user can specify storage level and subset of the output dataframe of the stage to be persisted.
persist(colExprs: String*) - uses default storage level.
getPersistedDf(): Option[DataFrame] - to retrieve the cached data frame for a stage. This will be useful when we only need to run the pipeline from a specific stage. For instance, pipelineModel.transform(countVectorizer.getPersistedDf.get) should only run the final LDA stage of the pipeline. The persisted dataframe s only available once the Pipeline.fit has been called.

Changes in Pipeline

The fit method checks whether a particular stage is marked to persist its output. If yes, it marks the output data frame as persistent according to the storage level and column subset. For Executors in the stage, it associates the persistant dataframe with both the Executor as well as the resultant Transformer.
The transformSchema methods is changes so that the column pruning as result of specifying subset columns for persisting is taken into account.

Changes in PipelineModel

transform(dataset) method now checks if the dataset argument was persisted in any of the stages during Pipeline.fit. If yes, it ensures only the stages after that stage are run (else all the stages are run). These two cases are handled for transformSchema.

How was this patch tested?

Existing tests
Added new test in PipelineSuite to test above scenarios.

Can one of the admins verify this patch?

cc @jkbradley Please take a look.

cc @mengxr @jkbradley -- Hi, any thoughts on this? Thanks!

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?


[Suggestion] Polish language

Hello, I've made polish translations to this plugin, if you want, you can add it.
messages.yml
https://paste.gg/p/anonymous/4b072718ea874092ab1a94ff603fdd0f
config.yml
https://paste.gg/p/anonymous/18768ee0959c4e8ab6fca6cd9bb1e692

Thanks!

Lang/Config file will be changing


[bug][exploit] Issue with fake stake as reported by PIVX team needs to be tested in ION wallets

We have reports from PIVX team that there is a Staking bug that allows a wallet to "fake stake"
We need to test the current test wallet to ensure the "Fake Stake" issue is currently present in our testing wallets. The likelyhood of it actively being there is extemely high but we still need a positive test of the exploit.
https://github.com/initc3/i-cant-believe-its-not-stake/tree/master/stratiX/test/functional
@ckti  can you please fillout the rest of the desription as you learn these details.

have tested this using the stratisX code and can find no evidence of fake stake


Parameterize chart templates

Please describe the Improvement and/or Feature Request

Use templating and parameterization with values.yaml to make it easier to modify chart in the future.
Scope (please mark with X where applicable)

New Functionality          [ ]
Install                    [ ]
SMI Traffic Access Policy  [ ]
SMI Traffic Specs Policy   [ ]
SMI Traffic Split Policy   [ ]
Permissive Traffic Policy  [ ]
Ingress                    [ ]
Egress                     [ ]
Envoy Control Plane        [ ]
CLI Tool                   [ ]
Metrics                    [ ]
Certificate Management     [ ]
Sidecar Injection          [ ]
Logging                    [ ]
Debugging                  [ ]
Tests                      [ ]
CI System                  [ ]
Project Release            [ ]

Possible use cases

Closing this as it is complete. @nshankar13 - please add details when you can. Thank you.

Parameterized fields to make it easier for values-override in templates. Ex: fluent-bit output, webhook-config-name-prefix, etc.


data_processing: allow map_to_scans.py to use torch and cuda

I see a pretty significant speed up by running the sparse quantize operation on the GPU (sequence 00 goes from about 40 minutes to 90 seconds).
Thank you for creating the project, I hope this contribution is helpful.

Thanks for the pull request! :)


'targetPlatforms' missing when nixos-rebuild with nixpkgs >= nixos-unstable in flake

Describe the bug
nixos-rebuild [test|switch] fails with the error of attribute 'targetPlatforms' missing for my machine, whose configurations include rust-based apps and nixpkgs input in flake.nix set to github:nixos/nixpkgs/nixos-unstable or github:nixos/nixpkgs
Steps To Reproduce

set up flake.nix with inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
sudo nixos-rebuild test

Additional context
this error did not occur when nixpkgs.url = "github:nixos/nixpkgs/nixos-24.05";
log from nixos-rebuild test --show-trace
Notify maintainers
@alyssais

Metadata
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.6.54, NixOS, 24.11 (Vicuna), 24.11.20241009.5633bcf`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.24.8`
 - nixpkgs: `/nix/store/60sn02zhawl3kwn0r515zff3h6hg6ydz-source`

Add a :+1: reaction to issues you find important.

Are you using a rustc from somewhere other than Nixpkgs?  I don't think this is a problem inside Nixpkgs, but I could be wrong!

I got the same problem. Seems to be related to COSMIC. If I disable it (services.desktopManager.cosmic.enable = false;) the error goes away.

Does that mean you're using the cosmic overlay, which uses some non-Nixpkgs rustc?

Yeah, exactly. If I understand correctly, they build a custom rust platform in the flake:
https://github.com/lilyinstarlight/nixos-cosmic/blob/a1e3c25b1d6fb0a88a5bbb61cc4bd502439a68ff/flake.nix#L20-L25

seems like that's the case for me as well, as i'm trying to use an overlay with custom rust too:
https://github.com/oppiliappan/statix/blob/d324490e45bcda1b664f28b6f88da8580deda76b/flake.nix#L39

Please report to wherever your rustc comes from.  They should provide targetPlatforms and badTargetPlatforms attributes listing which platform doubles their rustc can build for.

I'm also hitting this, using fenix with:
pkgs = nixpkgs.legacyPackages.${system}.extend fenix.overlays.default;

rustToolchain = pkgs.fenix.combine (with pkgs.fenix; [
  stable.toolchain
  targets.wasm32-unknown-unknown.stable.rust-std
]);

buildRustPackage =
  (pkgs.makeRustPlatform {
    cargo = rustToolchain;
    rustc = rustToolchain;
  })
  .buildRustPackage;

This broke as of https://github.com/NixOS/nixpkgs/commit/0852f8eb8481cfa310a131017a0e73f2333c8bba.

In my case I had been including wezterm in my flake, which uses nixos-unstable while wezterm uses nixpkgs-unstable. And removing the inputs.nixpkgs.follows statement fixed the error:
--- a/flake.nix
+++ b/flake.nix
@@ -2,10 +2,7 @@
   description = "My personal NUR repository";
   inputs = {
     nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
-    wezterm = {
-      url = "github:wez/wezterm?dir=nix";
-      inputs.nixpkgs.follows = "nixpkgs";
-    };
+    wezterm.url = "github:wez/wezterm?dir=nix";
   };
   outputs = { self, nixpkgs, ... } @ inputs:
     let

Yeah COSMIC uses rustc from rust-overlay while the rustc in NixOS stable is still too old. I'll take a look at what can fix that downstream

Please report to wherever your rustc comes from. They should provide targetPlatforms and badTargetPlatforms attributes listing which platform doubles their rustc can build for.

per the maintainer's response, i will close this issue. if anyone else faces the same error, check if any of the flakes has rustPlatform*, and report the issue to the rust input of that flake.


Move all text editor view state into a presenter object

This PR changes our approach to text editor rendering by centralizing all state in a TextEditorPresenter object that sits between the view and the model.

The presenter manages a mutable state object that is synchronously updated whenever there are changes from the model or changes to measurements from the view. The view consumes this state object when updating the DOM asynchronously so we don't need to recompute it before every render like we do now.
This reconfiguration paves the way to perform all DOM updates manually rather than relying on React's virtual DOM reconciliation system. The view will maintain a copy of the presenter state, and it will perform a comparison of the presenter's state against the copy when updating the DOM.
The net result will be a design very similar to React's reconciliation mechanism, but my hope is that a lower-level implementation targeting our specific problem domain will make this design easier to profile and open up specific optimizations that wouldn't be possible at a higher level of abstraction. We already perform DOM updates to lines and line numbers manually to optimize around reconciliation overhead; this would just complete that process.
Remaining work
Optimizations

[x] Perform a single decorations query when the visible row range changes to cache the visible decorations, then read decoration classes from that when building lines, line numbers, etc rather than querying multiple times on every change.
[x] When decorations are added / removed, update only the decoration classes on lines and line numbers rather than doing a full pass on their content.

Fixes

[x] Fix velocity scrolling upward
[x] Fix TextEditorComponent specs
[x] Compute row id correctly when the first visible line is soft-wrapped.
[x] Eliminate TextEditorPresenter::getHeight
[x] Audit handling of scrollTop/scrollLeft
[ ] Enforce maximum scrollTop/Left based on height/width and content. This should hopefully address some flakiness in the random test.

Is there a problem modifying the DOM directly at the same time?
On Wed, Jan 28, 2015 at 9:44 AM, Nathan Sobo<EMAIL_ADDRESS>wrote:

This PR changes our approach to text editor rendering by centralizing all
state in a TextEditorPresenter object that sits between the view and the
model.
[image: presenter diagram]
https://cloud.githubusercontent.com/assets/1789/5943048/d3ef770a-a6d8-11e4-9ed7-aa4ce2a650f2.png
The presenter manages a mutable state object that is synchronously updated
whenever there are changes from the model or changes to measurements from
the view. The view consumes this state object when updating the DOM
asynchronously so we don't need to recompute it before every render like we
do now.
This reconfiguration paves the way to perform all DOM updates manually
rather than relying on React's virtual DOM reconciliation system. The view
will maintain a copy of the presenter state, and it will perform a
comparison of the presenter's state against the copy when updating the DOM.
The net result will be a design very similar to React's reconciliation
mechanism, but my hope is that a lower-level implementation targeting our
specific problem domain will make this design easier to profile and open up
specific optimizations that wouldn't be possible at a higher level of
abstraction. We already perform DOM updates to lines and line numbers
manually to optimize around reconciliation overhead; this would just
complete that process.
You can view, comment on, or merge this pull request online at:
https://github.com/atom/atom/pull/5293
Commit Summary

Start on TextEditorPresenter with lines state
Handle changing ::scrollTop in TextEditorPresenter
Handle changing ::clientHeight in TextEditorPresenter
Handle changing ::lineHeight in TextEditorPresenter
Update TextEditorPresenter when the editor’s content changes
Add width to lines state based on the computed scrollWidth
Instantiate presenter in TextEditorComponent and update measurements
Include line text in presenter state
Add 1 to the last row to ensure it’s visible
:art:
Account for overdrawMargin of startRow when computing the endRow
Add 1 pixel to scrollWidth to account for cursor if not soft-wrapped
Make all lines visible if no external client height is assigned
Use getters in TextEditorPresenter internally for consistency
Include endOfLineInvisibles in presenter state
Fix endRow calculation
Include more fields in line state
Disable spec until presenter approach stabilizes
Start using TextEditorPresenter in LinesComponent
Handle scoped character widths in TextEditorPresenter
Update TextEditorPresenter with scoped character widths in component
Add top-level .content object to presenter state
Remove unused local vars
Add content.indentGuidesVisible to TextEditorPresenter::state
Update .content.indentGuidesVisible when editor’s grammar changes
Use TextEditorPresenter::state.content.indentGuidesVisible
Remove unused argument
Read scrollWidth from the presenter state when rendering
Add .decorationClasses to line state on initial render
Add lineStateForScreenRow helper
Simplify assertions
Instantiate presenter with minimal parameters in specs
Handle updates to line decorations in TextEditorPresenter
Honor the ‘onlyEmpty’ and ‘onlyNonEmpty’ line decoration options
Don’t apply line decorations to last line if it ends at column 0
Don’t apply line decorations to mini editors
Show/hide line decorations when TextEditor::mini changes
Remove unused code
Fix spec
Reorganize specs on TextEditorPresenter to mirror structure of state
Add ::state.content.scrollHeight to TextEditorPresenter
Use presenter to supply scrollHeight to lines component
Add ::state.content.scrollTop/Left to TextEditorPresenter
Use presenter’s scrollTop/scrollLeft in LinesComponent
:art:
Add TextEditorPresenter::state.content.cursors
Use presenter state in CursorsComponent
Fix 0-width cursors in presenter instead of view
Add TextEditorPresenter::state.content.blinkCursorsOff
Add TextEditorPresenter::onDidUpdateState
Blink cursors based on presenter state
Remove shouldComponentUpdate hook
Remove unused props
Fix spec organization
Add highlights state to TextEditorPresenter
Use presenter to render flashes
Use presenter to render highlights
Start on TextEditorPresenter::state.lineNumbers
Reflect changes to line number decorations in presenter state
Deprecate TextEditor::getGutterDecorations
Add ‘foldable’ to line number presenter state
Move vertical scroll state to root of presenter state object
Move .lineNumbers onto .gutter property of presenter state
Update presenter scrollHeight when clientHeight changes
Use presenter for gutter scrollHeight and scrollTop
Key line numbers by buffer row and soft-wrap count
Build line numbers based on presenter state
Don’t decorate soft-wrapped lines/line numbers unless spanned by
marker
:art:
Honor the ‘onlyHead’ option for line and line-number decorations
Remove unused properties in EditorComponent and children
Add overlay decorations to TextEditorPresenter::state
Use presenter for rendering overlay decorations
Remove unused React props and methods that build them
Add TextEditorPresenter::state.content.backgroundColor
Use backgroundColor from presenter in LinesComponent
Add TextEditorPresenter::state.scrollingVertically
Use presenter’s scrollingVertically property to defer line
measurement
Set content.indentGuidesVisible in presenter if editor is mini
Remove unused local var
Use presenter state for placeholder text
Use presenter state for gutter background color
Add maxLineNumberDigits to presenter state
Wait for required measurements before building some presenter state
Construct TextEditorPresenter before component mounts
Only store backgroundColor and gutterBackgroundColor on presenter
Use maxLineNumberDigits from presenter state in GutterComponent
Add TextEditorPresenter::state.mousewheelScreenRow

File Changes

M spec/text-editor-component-spec.coffee
https://github.com/atom/atom/pull/5293/files#diff-0 (24)
A spec/text-editor-presenter-spec.coffee
https://github.com/atom/atom/pull/5293/files#diff-1 (1417)
M src/cursor-component.coffee
https://github.com/atom/atom/pull/5293/files#diff-2 (6)
M src/cursors-component.coffee
https://github.com/atom/atom/pull/5293/files#diff-3 (51)
M src/display-buffer.coffee
https://github.com/atom/atom/pull/5293/files#diff-4 (2)
M src/gutter-component.coffee
https://github.com/atom/atom/pull/5293/files#diff-5 (195)
M src/highlight-component.coffee
https://github.com/atom/atom/pull/5293/files#diff-6 (106)
M src/highlights-component.coffee
https://github.com/atom/atom/pull/5293/files#diff-7 (14)
M src/lines-component.coffee
https://github.com/atom/atom/pull/5293/files#diff-8 (256)
M src/overlay-manager.coffee
https://github.com/atom/atom/pull/5293/files#diff-9 (52)
M src/text-editor-component.coffee
https://github.com/atom/atom/pull/5293/files#diff-10 (195)
A src/text-editor-presenter.coffee
https://github.com/atom/atom/pull/5293/files#diff-11 (601)
M src/text-editor-view.coffee
https://github.com/atom/atom/pull/5293/files#diff-12 (2)
M src/text-editor.coffee
https://github.com/atom/atom/pull/5293/files#diff-13 (13)

Patch Links:

https://github.com/atom/atom/pull/5293.patch
https://github.com/atom/atom/pull/5293.diff

—
Reply to this email directly or view it on GitHub
https://github.com/atom/atom/pull/5293.

Is there a problem modifying the DOM directly at the same time?

I might consider it, but I'm trying to break this up into pieces at the request of the team since it's already a big change as is.

I'm sorry.  I meant could a package modifying the DOM directly conflict
with this presenter modification?
Now that I think about it the answer is probably no.
On Wed, Jan 28, 2015 at 12:47 PM, Nathan Sobo<EMAIL_ADDRESS>wrote:

Is there a problem modifying the DOM directly at the same time?
I might consider it, but I'm trying to break this up into pieces at the
request of the team since it's already a big change as is.
—
Reply to this email directly or view it on GitHub
https://github.com/atom/atom/pull/5293#issuecomment-71912557.

@mark-hahn Yeah it shouldn't interfere. Although one thing I'm hoping to solve is to provide a hook for appropriate times to modify the DOM yourself to prevent reflows. That's fairly fuzzy at the moment though and a ways off.

appropriate times to modify the DOM yourself to prevent reflows.

That would be cool.  Can a package post changes to the presenter and let it
handle everything?
On Wed, Jan 28, 2015 at 1:22 PM, Nathan Sobo<EMAIL_ADDRESS>wrote:

@mark-hahn https://github.com/mark-hahn Yeah it shouldn't interfere.
Although one thing I'm hoping to solve is to provide a hook for appropriate
times to modify the DOM yourself to prevent reflows. That's fairly
fuzzy at the moment though and a ways off.
—
Reply to this email directly or view it on GitHub
https://github.com/atom/atom/pull/5293#issuecomment-71918792.

The basic structure is in place here, so now it's time to optimize. It breaks down to about 6ms updating the model and presenter state and 4ms updating the DOM for a single character insert. I have reason to believe I can improve on this substantially.

There's a ton of low hanging fruit on the presenter update because I did everything in the most naive way, so that's the work I'll do on this PR and save DOM update optimizations for a subsequent PR. To ensure optimizations don't introduce regressions, I'm going to build a spec that randomly mutates an editor and compares the state of a brand new presenter to that of a preexisting presenter to be sure updates were performed correctly.
After that, some of the presenter optimizations I have in mind:

[ ] Perform a single decorations query when the visible row range changes to cache the visible decorations, then read decoration classes from that when building lines, line numbers, etc rather than querying multiple times on every change.
[ ] When decorations are added / removed, update only the decoration classes on lines and line numbers rather than doing a full pass on their content.
[ ] Cache foldability status on tokenized lines so we don't have to recompute it when updating line numbers.

I have more ideas, but I'd like to try these things first and re-evaluate.
I also want to re-introduce shouldComponentUpdate methods on all the React components which should improve reconciliation performance.

We're going to try this out for a couple days before merging, but this should be ready to go. Next steps:

Cache "foldable" status of lines so it's cheaper to update the line numbers in the presenter
Switch to manual DOM manipulation
Try tiled rendering

Randomized spec is failing. Need to get that resolved before merging.

I'm going to go ahead and cache foldable in the tokenized buffer before merging since we're recomputing the gutter more frequently with this design. I'm thinking that's the last step on this unless any other performance issues crop up.

Finished caching "foldable" status on tokenized lines. The sync presenter update is down to about 3.4ms, and only about 1ms is spent in the presenter. Still room for optimization there, but I think we have other :fish: to :egg: before we worry about it. Want to spend a day using this in its final form before merging.

:sparkles: hoorah! :beers:

@maxbrunsfeld Give it a build and see if it feels more responsive to you, if you weren't already testing that branch.


Note admonition in task description now renders, and renders incorrectly

https://percy.io/e5d9f90c/guides/builds/15507671/changed/876102490?browser=edge&browser_ids=18%2C19%2C20%2C21&subcategories=unreviewed%2Cchanges_requested&viewLayout=side-by-side&viewMode=new&width=375&widths=375
https://github.com/redhat-developer/app-services-guides/pull/391

Fixed see screenshot


[HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

Changes -

Added a wrapper over HoodieDeltaStreamer for adding support for executing incremental sync from multiple topics in case of big organisations.
Added object representing topic level overrides for multi topic execution
Added concerned test cases.

@pratyakshsharma : Let me know once you addressed the comments.
Thanks,
Balaji.V

@pratyakshsharma  lets revive this ? :)

Working on it, @vinothchandar

@pratyakshsharma : Let me know if you need any clarification.

@pratyakshsharma : Let us know if you need any help on getting this through :) ?

@pratyakshsharma : Let us know if you need any help on getting this through :) ?

@bvaradar The fixes in this PR depend on https://github.com/apache/incubator-hudi/pull/1395. Let us try to close it as soon as possible to get this through. :)

@bvaradar got chance to take a pass?

@pratyakshsharma : Per your previous comment, was waiting for you to give a green signal :)  I will look at it sometime today. There are conflicts in this PR. If you can resolve them, that will be great.

@bvaradar done. Please have a look.

@pratyakshsharma Can you update the wiki for HoodieMultiDeltaStreamer similar to https://hudi.incubator.apache.org/docs/writing_data.html#deltastreamer?
Can you advise in which release this tool will be available?

Codecov Report

Merging #1150 into master will increase coverage by 0.19%.
The diff coverage is 74.21%.

@@             Coverage Diff              @@
##             master    #1150      +/-   ##
============================================
+ Coverage     67.48%   67.67%   +0.19%
- Complexity      261      290      +29
============================================
  Files           343      350       +7
  Lines         16565    16825     +260
  Branches       1693     1707      +14
============================================
+ Hits          11179    11387     +208
- Misses         4648     4687      +39
- Partials        738      751      +13

Impacted Files
Coverage Δ
Complexity Δ

...apache/hudi/utilities/deltastreamer/DeltaSync.java
72.13% <ø> (-0.14%)
38.00 <0.00> (ø)

...utilities/deltastreamer/TableExecutionContext.java
65.00% <65.00%> (ø)
9.00 <9.00> (?)

...s/deltastreamer/HoodieMultiTableDeltaStreamer.java
73.01% <73.01%> (ø)
18.00 <18.00> (?)

...main/scala/org/apache/hudi/DataSourceOptions.scala
93.25% <100.00%> (+0.07%)
0.00 <0.00> (ø)

...i/utilities/deltastreamer/HoodieDeltaStreamer.java
81.06% <100.00%> (+0.86%)
10.00 <2.00> (+2.00)

...rg/apache/hudi/io/storage/HoodieParquetWriter.java
100.00% <0.00%> (ø)
0.00% <0.00%> (ø%)

...he/hudi/io/storage/HoodieStorageWriterFactory.java
93.75% <0.00%> (ø)
0.00% <0.00%> (ø%)

.../hudi/execution/MergeOnReadLazyInsertIterable.java
64.70% <0.00%> (ø)
0.00% <0.00%> (ø%)

...ache/hudi/common/inline/fs/InMemoryFileSystem.java
89.65% <0.00%> (ø)
0.00% <0.00%> (?%)

...hudi/common/inline/fs/InLineFsDataInputStream.java
53.84% <0.00%> (ø)
0.00% <0.00%> (?%)

... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 04449f3...e0eb241. Read the comment docs.

@bvaradar Please take a pass.

@bvaradar Have addressed the comments. Please take a pass.


[iOS] Make course menu items more bright

Right now course menu items appear as if they are inactive (as if they cannot be clicked), need to make course menu items more bright

@moiz994 unselected menu items look a little 'disabled'. Should we use some other grey color for it? Or confirm it's ok and we will close this issue.
Thank you

@rnr true, they do look disabled. Is there any other appropriate color in the color palette? If not, let's do some quick hit and trial.


Help with plugin authoring

I'm looking for some direction with regards to how to determine the rbenv root from within a plugin.
The wiki states that the $RBENV_ROOT variable will be set as part of the environment when the plugin runs. This rather clearly implies that $RBENV_ROOT is safe to use.
But further along in the wiki, authors are instructed to use rbenv commands for inspecting the environment instead of constructing it themselves. This seems to imply that invoking $(rbenv-root) would be preferred.
Normally, it wouldn't matter, but I've encountered a weird bug (#715) with rbenv-update where $RBENV_ROOT isn't set in the environment as it should be.
Which is the recommended use? $RBENV_ROOT or $(rbenv-root)?

The docs are correct: RBENV_ROOT will always be available. Closing in favor of further discussion in #715


Add maintianer Rick Ducott

Signed-off-by: Lachlan Evenson<EMAIL_ADDRESS>cc @slack @michelleN

So happy to have you on board @rickducott!

Much appreciated!


feat: pin prisma versions to latest

Using the dataproxy engine type in templates-node causes a search and replace for a specific prisma version. Since Renovate keeps bumping this version in templates repo, the check in templates-node for the previous package version does not work, causing tests to fail. Instead of manually updating templates-node each week to search and replace the correct version we can just search for the static tag dev

If the version find and replace for the dataproxy check is failing, then we should improve that. I'm guessing it is a regex, so we can change it.
Pinning to an NPM tag is almost never a good idea, even if Prisma follows semver. The reason we use specific versions is because we know that version works with the template. If something breaks in Prisma 3.3, we would never know, and we might clone this broken version for someone.
Moreover, if someone clones this template and runs yarn, one day they'll get v3.2.1, and then next week they might get 3.3.0, without changing a single line of code. These kinds of "automatic" upgrades is precisely why NPM tags are not a good idea. Worse still, NPM tags don't support semver, so one of these days they'll start getting Prisma 4.0, all without changing a single line of code.
I think we should close this PR, and create a new one improving the findand-replace regex


Use the native XMLHttpRequest instead of a SERVICE_MSG_REQUEST_FLAG flag mechanism

@inikulin wrote:

We have a SERVICE_MSG_REQUEST_FLAG in https://github.com/churkin/testcafe->hammerhead/blob/es6/src/client/sandbox/xhr/index.js . It was used to skip tracking of the transport >messages. I'm wondering, since we use native methods now, do we still need this flag?

I'm working on this


Fix missing slash in admin endpoints

As discussed with @severo, this PR fixes the missing slash in some admin endpoints, e.g.:

https://datasets-server.huggingface.co/admin/force-refreshdataset-config-names

After this PR, it will be:

https://datasets-server.huggingface.co/admin/force-refresh/dataset-config-names

Related to:

#1246

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

Codecov Report
Patch coverage: 80.00% and project coverage change: -4.60 :warning:

Comparison is base (24e8344) 90.44% compared to head (6612fb5) 85.85%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1264      +/-   ##
==========================================
- Coverage   90.44%   85.85%   -4.60%
==========================================
  Files         181       20     -161
  Lines       10565      707    -9858
==========================================
- Hits         9556      607    -8949
+ Misses       1009      100     -909

Flag
Coverage Δ

jobs_cache_maintenance
?

jobs_mongodb_migration
?

libs_libcommon
?

services_admin
85.85% <80.00%> (ø)

services_api
?

services_worker
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

services/admin/src/admin/app.py
88.00% <ø> (ø)

services/admin/src/admin/routes/cancel_jobs.py
52.63% <0.00%> (ø)

services/admin/src/admin/routes/force_refresh.py
63.15% <100.00%> (ø)

services/admin/tests/test_app.py
100.00% <100.00%> (ø)

services/admin/tests/test_app_real.py
94.11% <100.00%> (ø)

... and 161 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


gitAndTools.gh: 1.3.0 -> 1.3.1

https://github.com/cli/cli/releases/tag/v1.3.1

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 105833 run on x86_64-linux 1

1 package built:

github-cli (gitAndTools.gh)

Result of nixpkgs-review pr 105833 1

1 package built:

github-cli (gitAndTools.gh)


Bug in code sample

Code sample in step 4 in 'Using the change feed processor library section' declares a variable called feedOptions and sets StartFromBeginning property but the variable is not used at all.

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fdcc897a-d6df-3305-e9cb-01889a3c5d4f
Version Independent ID: 739a9fc7-3782-a860-ac6d-4194fd1ce2e8
Content: Working with the change feed support in Azure Cosmos DB
Content Source: articles/cosmos-db/change-feed.md
Service: cosmos-db
GitHub Login: @rafats
Microsoft Alias: rafats

@jkonecki Thanks for the feedback! I have assigned the issue to the content author to evaluate and update as appropriate.

@jkonecki the ChangeFeedOptions object is passed as a parameter to the CreateDocumentChangeFeedQuery method that is available within the change feed processor SDK. It's not a bug. Please let us know if you have additional questions.

Hello @SnehaGunda,
I think we are talking about different pieces of code. Sample code in Step 4 doesn't call CreateDocumentChangeFeedQuery at all. There are no more code samples on this page as Step 4 is the last one.

@jkonecki got you! I was looking at the previous section. yes I see the bug, the code block should instead be feedProcessorOptions.StartFromBeginning = true; In future, if you see any issues, feel free to click on the edit button on the page and submit your change, We would love to take community contributions !
I fixed it the following PR. We will now proceed with closing the issue.

#please-close


ECDSA

Any new updates on ecdsa? If anyone has fixed it please drop a link of that.

Yes, same for me. Any hints how to fix the broken ECDSA?
spachner


Log device when sending/receiving a message fails

As Operations Engineer,
I want the logging to contain the device when sending or receiving a message to it fails,
so I can easier detect the cause of the problem.
Besides the device, also mention the current retry count for the message in the logging.
Preferably show the device identification in the logging. If it's much easier to show the device UID, then show the device UID.
The current logging just shows "Unable to connect" or "Failed to receive ...". Operations needs to know to which device we could not send the message.
Adapt this in both R2 and R3. R2 doesn't have a retry count, so only add the retry count for R3.

Retry count is already present in the log.

com.alliander.osgp.platform.domain.exceptions.PlatformException: Exception occurred: Failed to receive response within timelimit 30000 ms
at com.alliander.osgp.platform.infra.jms.publiclighting.PublicLightingResponseMessageFinder.findMessage(PublicLightingResponseMessageFinder.java:63) ~[PublicLightingResponseMessageFinder.class:na]
at com.alliander.osgp.platform.application.services.AdHocManagementService.dequeueSetTransitionResponse(AdHocManagementService.java:392) ~[AdHocManagementService.class:na]
at com.alliander.osgp.platform.application.services.AdHocManagementService$$FastClassBySpringCGLIB$$251f75f2.invoke() ~[spring-core-3.2.10.RELEASE.jar:na]

SOLVED.
R2
[flexadmin@flexovl-v2-test tomcat7]$ grep -i timelimit  catalina.out
14:39:16.369 [publicLightingRequestMessageListenerContainer-3] WARN  (OslpChannelHandlerClient.java:40) c.a.o.p.s.OslpChannelHandlerClient - Failed to receive response from SSLD-HRL-19 within timelimit 30000 ms
java.io.IOException: Failed to receive response within timelimit 30000 ms
java.io.IOException: Failed to receive response within timelimit 30000 ms
R3:
t@osgp-tst-02 tomcat7]# grep -i timelimit catalina.out
12:55:20.993 [webdevicesimulator-autonomous-lightswitching-on-1] WARN  (OslpChannelHandler.java:112) c.a.o.w.service.OslpChannelHandler - Failed to receive response from device SSLD_11-88-AHM-014 within timelimit 20000 ms
java.io.IOException: Failed to receive response within timelimit 20000 ms
java.io.IOException: Failed to receive response within timelimit 20000 ms
13:25:20.163 [webdevicesimulator-autonomous-lightswitching-on-1] WARN  (OslpChannelHandler.java:112) c.a.o.w.service.OslpChannelHandler - Failed to receive response from device SSLD_11-88-AHM-014 within timelimit 20000 ms
java.io.IOException: Failed to receive response within timelimit 20000 ms
java.io.IOException: Failed to receive response within timelimit 20000 ms


Android support

Are you planning to support translateLocally on android in the future?

Yes, it is planned! There's a non public alpha where the QT project is compiled to android, but the process is very ad-hoc for now. We plan to release beta android/iphone support using the direct QT compilation.


Releases

What about releases?
We can host git archives on github by adding tags  and relink to latest release on the main site.
git tag v1.3.5
git push --tags

It's good idea.

This is a very good idea.
Using the official API to create a release:
https://developer.github.com/v3/repos/releases/#create-a-release
you'll get a downloadable zip or tar.gz file created automatically.


How to refer to UNIX-Python-executable in "select interpreter" in VS Code instead of .exe?

I was forced to use WSL on Windows 10 to install the Python library Cartopy on my Windows machine correctly. It is way easier in a Linux-distribution, so I chose Ubuntu 20.04 WSL2.
As a consequence, I created a virtual Python environment via WSL in my project folder via python -m venv linux-venv.
The problem is now that all binaries/executables were compiled for the WSL-distribution and are thus not selectable from VS Code started from within Windows 10.
Consequently, I installed VS Code on WSL and started it from there.
My hope was that I can then manually select the right interpreter path linux-venv/bin/python from my project's root directory.
Unfortunately, it does not work either, even though I'm doing this from VS Code running on WSL:

Clicking on my desired interpreter linux-venv/bin/python, the following error message is displayed:

If I try to "browse to my desired executable" instead, they are not displayed since solely ".exe" - file extensions are allowed:

Next, I can solely browse ".exe" - files, wherefore I cannot select the UNIX-compiled executables.

Conclusion:
It does not seem possible at the moment to select a UNIX-compiled interpreter in VS Code properly on a Windows 10 - machine, not even using WSL.
Since the aforementioned Python library Cartopy solely works well on UNIX-systems, I need to install my virtual environment from WSL.
In order to continue with my project, I would need to be able to select the UNIX-compiled interpreter, but I cannot.

We do not support selecting WSL binaries on Python extension running on Windows. Please use the Remote - WSL extension: https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl. It installs a copy of Python extension on WSL so you can select those binaries. Let me know if that helps.


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-yml-archive-non-versioning-archive/pull/445 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-yml-archive-non-versioning-archive.git

Docs Build status updates of commit cac7f90:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

docs/TOC.yml
:white_check_mark:Succeeded
View

docs/index.yml
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/article3-1-1.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/article3-1-2.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/article3-1-3.yml
:white_check_mark:Succeeded
View

docs/to-be-archive/level1/sql-database-engine.png
:white_check_mark:Succeeded

docs/to-be-archive/level2/article3-2-1.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level2/article3-2-2.md
:white_check_mark:Succeeded
View

docs/to-be-archive/level2/article3-2-3.yml
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Circular orbits for r0 < 4M

KerrGeoOrbit gives 1/0's for circular orbits in Schwarzschild with r0 < 4M (and more generally for orbits below the IBCO in Kerr)

This has been fixed and Scott Hughes has confirmed the calculation of the flux very close to the light-ring using results from Gremlin. Perhaps we should add a unit test for this case? For now I'll close the issue.


Add Search Results

Add UI Modals for Number search and Delete request

@alexboateng I would like to pick up this issue


File Credentials Support

Intent
Full support for credentials persistence via a file ~/.connect-credentials.
Following up the work done at #2308, this PR completes the feature by:

Having a main credentials service constructor NewCredentialsService
NewCredentialsService is called by endpoints that do credentials operations
NewCredentialsService tries to:

set up the default system's keyring service, NewKeringCredentialsService
falling back to file based credentials when there is no keyring available. NewFileCredentialsService

This allows for different funtionality on credentials persistence to share the same interface for credentials operations List(), Get(), Set(), Delete()

Screenshots of Workbench using the file backed credentials service

Type of Change

[ ] Bug Fix

[x] New Feature

[ ] Breaking Change

[ ] Documentation

[ ] Refactor

[ ] Tooling

Automated Tests
Unit tests created and updated
Directions for Reviewers
To manually verify this changes:

Start a new workbench session,
Install the compatible packaged VSIX file, generated by CI on this PR's job
Reload session if needed, from the command pallette > Developer: Reload Window
Create and delete credentials as desired
Remove workbench sessions, see how credentials are persisted next time you start a new session

Checklist

[ ] I have updated CHANGELOG.md to cover notable changes.

if it should be a little more "name-spaced"... Something like .posit-connect-credentials

I've had this same question. I think it is fine to get this merged in and have a separate discussion around it since that is distinct from the implementation here.

My favorite is probably .posit-connect

To mimic our .posit directory in projects we could do .posit/.connect-credentials, but like I said I think we should have this as a follow-up discussion.


Display feature information in documentation

In the public API for regorus in docs.rs, display that certain API is available only if a specific feature is available.
For example, https://docs.rs/regorus/latest/regorus/struct.Engine.html#method.get_coverage_report does not list that coverage feature is necessary for this API.
There has been an attempt to make this work: https://github.com/microsoft/regorus/blob/56bd8841d09120366f5aebd58a12ad84602210d7/Cargo.toml#L165
@mkulke What is the right way to do this?

So, for the time being this relies on nightly and requires additional annotations, not sure if it will work reliably in all cases. stabilization for this doc feature is [ongoing. However, it looks like in the current main the annotation is a bit off, when adjusting it, doc renders ok for that method:


Support Multi Token Rewards

Original meta issue: https://github.com/ubiquity/ubiquibot/issues/753
Right now we support payouts only in a single token (permit URL example) per a single network (DAI for mainnet + WXDAI for gnosis). We should add support for payouts in multiple tokens so that the following example scenario would be supported:

Bounty hunter solves an issue
Bounty hunter gets rewards in DAI and UBQ on mainnet

What should be done:

Refactor the bot's config so that each reward asset had a unique incentive and payout configuration. Example for DAI and UBQ on mainnet:

{
  "payouts": [
     {
       "evm-network-id": 1,
       "token-address": "0x6b175474e89094c44da98b954eedeac495271d0f", // DAI
       "price-multiplier": 1,
       "issue-creator-multiplier": 2,
       "payment-permit-max-price":<PHONE_NUMBER>740991,
       "assistive-pricing": false,
       "comment-incentives": false,
       "incentives": {
         "comment": {
           "elements": {},
           "totals": {
             "word": 0
           }
         }
       }
     },
    {
       "evm-network-id": 1,
       "token-address": "0x4e38D89362f7e5db0096CE44ebD021c3962aA9a0", // UBQ
       "price-multiplier": 0.1,
       "issue-creator-multiplier": 0.2,
       "payment-permit-max-price":<PHONE_NUMBER>740991,
       "assistive-pricing": false,
       "comment-incentives": false,
       "incentives": {
         "comment": {
           "elements": {},
           "totals": {
             "word": 0
           }
         }
       }
     }
  ]
}

So when there are multiple reward tokens set then multiple price labels should be generated. For example, for DAI and UBQ in the config above and Priority: 1 (Normal) with Time: 1 Day the following price labels should be generated:
a) Price: 100 DAI
b) Price: 10 UBQ

Refactor the permit URL query params (example) and pass base64 encoded array of permits instead of a single one (this is how it works right now)

/start

Closing it until the new specs are ready https://github.com/ubiquity/ubiquibot/issues/753#issuecomment-1997149917


Ft develop chr3 viewall offers #158626657

What does this PR do?
allows registered users to login and view all ride requests posted.
all test are passing
Description of Task to be completed
Increase code coverage
Endpoints created and tested?

Endpoints
Functionality

`GET /api/v1/rides/
View detail of a ride offer

What PT story
The Link to PT story is here

Pull Request Test Coverage Report for Build 91

0 of 0   changed or added relevant lines in 0 files are covered.
11 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.1%) to 89.868%

Files with Coverage Reduction
New Missed Lines
%

app/rides/views.py
1
96.0%

app/auth/views.py
2
96.67%

app/models.py
8
85.19%

Totals

Change from base Build 61:
-0.1%

Covered Lines:
204

Relevant Lines:
227

💛  - Coveralls


Hello, I'm having some problems. RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

Training model (TrainedModels/pantheon/2021_02_22_15_28_21_generation_train_depth_3_lr_scale_0.1_act_lrelu_0.05)
Training model with the following parameters:
number of stages: 6
number of concurrently trained stages: 3
learning rate scaling: 0.1
non-linearity: lrelu
Training on image pyramid: [torch.Size([1, 3, 26, 42]), torch.Size([1, 3, 31, 51]), torch.Size([1, 3, 40, 66]), torch.Size([1, 3, 57, 94]), torch.Size([1, 3, 106, 175]), torch.Size([1, 3, 152, 250])]
stage [0/5]::   0%|                                                                                | 0/1000 [00:00<?, ?it/s]T
raceback (most recent call last):
File "main_train.py", line 118, in
train(opt)
File "G:\ConSinGAN\ConSinGAN\training_generation.py", line 48, in train
fixed_noise, noise_amp, generator, d_curr = train_single_scale(d_curr, generator, reals, fixed_noise, noise_amp, opt, scale_num, writer)
File "G:\ConSinGAN\ConSinGAN\training_generation.py", line 156, in train_single_scale
gradient_penalty = functions.calc_gradient_penalty(netD, real, fake, opt.lambda_grad, opt.device)
File "G:\ConSinGAN\ConSinGAN\functions.py", line 122, in calc_gradient_penalty
create_graph=True, retain_graph=True, only_inputs=True)[0]
File "D:\Anaconda3\envs\ConSinGAN\lib\site-packages\torch\autograd_init_.py", line 149, in grad
inputs, allow_unused)
RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

Hi, that looks more like a problem with your Pytorch installation. Are you sure you have the correct CUDA and CUDNN version installed for your graphic card and Pytorch version?

嗨，这看起来更像是您的Pytorch安装问题。您确定为图形卡和Pytorch版本安装了正确的CUDA和CUDNN版本吗？

Hi, does the current ConsinGAN environment support Pytorch 1.7?

I haven't tested it with Pytorch 1.7 but in general it should work (I assume at least it would give you a different error message from the one above). The error is thrown at the torch.autograd.grad() function which is why I believe it's a problem with your environment and not with the code itself.
I would suggest running the code on CPU (use flag --not_cuda to see if it works on CPU or if you get a more informative error message. I haven't tested it on CPU myself so you might have to add to(torch.device('cpu') at some points if Pytorch raises errors about GPU/CPU mismatch.

I haven't tested it with Pytorch 1.7 but in general it should work (I assume at least it would give you a different error message from the one above). The error is thrown at the torch.autograd.grad() function which is why I believe it's a problem with your environment and not with the code itself.
I would suggest running the code on CPU (use flag --not_cuda) to see if it works on CPU or if you get a more informative error message. I haven't tested it on CPU myself so you might have to add .to(torch.device('cpu')) at some points if Pytorch raises errors about GPU/CPU mismatch.

Thank you very much. Use Flag -- Not CUDA can run.There's another question I'd like to ask you.If I want to input a single channel grayscale image for training, how should I modify the network?

Just set --nc_im 1 and represent your image as shape (H x W x 1), i.e. 1 channel instead of 3 for RGB

Just set --nc_im 1 and represent your image as shape (H x W x 1), i.e. 1 channel instead of 3 for RG
I've got --nc_im 1, but I'm running into the following problem.
Training model (TrainedModels/07/2021_02_24_22_00_10_generation_train_depth_3_lr_scale_0.1_act_lrelu_0.05)
Training model with the following parameters:
number of stages: 6
number of concurrently trained stages: 3
learning rate scaling: 0.1
non-linearity: lrelu
Traceback (most recent call last):
File "main_train.py", line 118, in
train(opt)
File "G:\ConSinGAN\ConSinGAN\training_generation.py", line 23, in train
real = functions.adjust_scales2image(real, opt)
File "G:\ConSinGAN\ConSinGAN\functions.py", line 185, in adjust_scales2image
real = imresize(real_, opt.scale1, opt)
File "G:\ConSinGAN\ConSinGAN\imresize.py", line 52, in imresize
im = np2torch(im,opt)
File "G:\ConSinGAN\ConSinGAN\imresize.py", line 26, in np2torch
x = color.rgb2gray(x)
File "D:\Anaconda3\envs\ConSinGAN\lib\site-packages\skimage\color\colorconv.py", line 799, in rgb2gray
rgb = _prepare_colorarray(rgb[..., :3])
File "D:\Anaconda3\envs\ConSinGAN\lib\site-packages\skimage\color\colorconv.py", line 152, in _prepare_colorarray
raise ValueError(msg)
ValueError: the input array must be have a shape == (.., ..,[ ..,] 3)), got (164, 250, 1)

You will have to change the code slightly then to adapt to this.
Another easy work-around is to just convert your gray-scale image to a "color image" with 3 channels, e.g. with OpenCV cv2.cvtColor(gray_img, cv.CV_GRAY2RGB)

You will have to change the code slightly then to adapt to this.
Another easy work-around is to just convert your gray-scale image to a "color image" with 3 channels, e.g. with OpenCV cv2.cvtColor(gray_img, cv.CV_GRAY2RGB)

There are some problems when I change the code. Can you give me some advice?

What are the problems?

I had the same problem 3 days ago, and I used conda install pytorch==1.4.0 torchvision==0.5.0 cudatoolkit=10.1 -c pytorch to unexpectedly ran it. This vision of torch is the same as SinGAN, maybe you can try it. :  )

Training model (TrainedModels/pantheon/2021_02_22_15_28_21_generation_train_depth_3_lr_scale_0.1_act_lrelu_0.05) Training model with the following parameters: number of stages: 6 number of concurrently trained stages: 3 learning rate scaling: 0.1 non-linearity: lrelu Training on image pyramid: [torch.Size([1, 3, 26, 42]), torch.Size([1, 3, 31, 51]), torch.Size([1, 3, 40, 66]), torch.Size([1, 3, 57, 94]), torch.Size([1, 3, 106, 175]), torch.Size([1, 3, 152, 250])]
stage [0/5]:: 0%| | 0/1000 [00:00<?, ?it/s]T raceback (most recent call last): File "main_train.py", line 118, in train(opt) File "G:\ConSinGAN\ConSinGAN\training_generation.py", line 48, in train fixed_noise, noise_amp, generator, d_curr = train_single_scale(d_curr, generator, reals, fixed_noise, noise_amp, opt, scale_num, writer) File "G:\ConSinGAN\ConSinGAN\training_generation.py", line 156, in train_single_scale gradient_penalty = functions.calc_gradient_penalty(netD, real, fake, opt.lambda_grad, opt.device) File "G:\ConSinGAN\ConSinGAN\functions.py", line 122, in calc_gradient_penalty create_graph=True, retain_graph=True, only_inputs=True)[0] File "D:\Anaconda3\envs\ConSinGAN\lib\site-packages\torch\autograd__init__.py", line 149, in grad inputs, allow_unused) RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

I have met the same question. At last, how can you resolve the problem? Thank you very much for your sharing and guidance.


Fix Meter hierarchy

Meter is not a subclass of Electrical_System but devices for a collection of points related to metering. Moving it as a top class of Equipment.

@jbkoh do you have cycles to address this or should I fix it up?

Let me update this afternoon. Thanks for the heads up.
With regards,
Jason Koh
On Mon, Feb 3, 2020 at 7:21 AM Gabe Fierro<EMAIL_ADDRESS>wrote:

@jbkoh https://github.com/jbkoh do you have cycles to address this or
should I fix it up?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/BrickSchema/Brick/pull/99?email_source=notifications&email_token=AAL76E4GBJP2PITPAQVXPA3RBAZARA5CNFSM4KMR5MEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKUHK5A#issuecomment-581465460,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAL76EZZIZTEHHB3IAZGDU3RBAZARANCNFSM4KMR5MEA
.

The Meter is an important concept because it is used for tracking usage such as for billing services. I think if we fully eliminate Meters, then people will be confused. This motivates keeping Power_Meter and adding Energy_Meter
Building_Power_Meter and Building_Energy_Meter are useful concepts to refer to as the meter for a full building. We could definitely model these concepts with a hasPoint relationship to an instance of Building. While that's the direction to go, its maybe worth keeping these around until we've touched up the Location class structure

Then, why not Building_Thermal_Power_Meter, Building_Electrical_Power_Meter, etc.? We will have a lot of redundant classes (co-existing Power_Meter and Power_Sensor, which will confuse the users more in my opinion.

Power_Meter and Power_Sensor are different things they aren't redundant. One is equipment one is a point.
We probably should have Building-flavored meter classes as well! Good point

They are redundant as they are modeling the same concepts in two different places, i.e., measuring some quantities. The difference is whether it's a collection of such measurements or a single point of measurement. People would be confused about what to choose for their modeling regardless of Equipment vs Point. Basically, we should follow the principle of Occum's razor.
Still, I don't mind putting quantity-specific meters if they suffice either of the two things:

Do they exist?
Are they useful?

Note that we are "introducing" these classes, which requires reasoning.

Why are we bringing in "reasoning" into this context? This was always the case and is true independent of the structure of the classes. Its also dubious how relevant Occam's razor is here: the goal is to model what exists in the world, not the simplest possible reduction of it, which overlooks the fine semantic distinctions made by real practitioners.
Recall that equipment and points are different: meters are equipment and sensors are points. Meters are physical pieces of hardware and it will become important to model them as such as we expand into modeling the electrical hierarchy. The choice to make is whether or not it is worth pushing the descriptions of the substances being measured into the name of the equipment that is doing that measurement. How do you feel about an initial step which is the Building_Meter?

Why are we bringing in "reasoning" into this context? This was always the case and is true independent of the structure of the classes. Its also dubious how relevant Occam's razor is here: the goal is to model what exists in the world, not the simplest possible reduction of it, which overlooks the fine semantic distinctions made by real practitioners.

Yeah, it's exactly what I am saying. We need to model what exists in the world and care about real practitioners. My points are

Do Power_Meter exist if we have Power_Sensor? In other words, is there equipment dedicated to measuring multiple powers "only"?

If such Power_Meter exists in the real world, I have no objection.

If it does not exist, why is it useful to have both Power_Sensor and Power_Meter, whereas they will confuse real practitioners?

Recall that equipment and points are different: meters are equipment and sensors are points. Meters are physical pieces of hardware and it will become important to model them as such as we expand into modeling the electrical hierarchy. The choice to make is whether or not it is worth pushing the descriptions of the substances being measured into the name of the equipment that is doing that measurement.

I am not sure what you are trying to bring this here. If you are saying Power_Meter is the equipment corresponding to Power_Sensor, we need to create all types of equipment for all the Point classes. Moreover, we never limited the definition of Point to virtual points only.

How do you feel about an initial step which is the Building_Meter?

There are instances of meter equipment measuring different types of points for the entire building. So I'd say Building_Meter is a collection of different metering points dedicated to building, as I saw in our building.

Power meters are definitely real (e.g. Smart Meters). They are the physical equipment. Points are _by _definition the digital representation of the sensors, setpoints, etc in the building.  For this reason, a meter may have multiple sensors/points (power sensor, gas usage sensor, etc). I think we could use some better documentation to point out that the points should be attached to a meter where appropriate, to be sure. This doesn't "slippery slope" into making Equipment for all the points, either. This is a very concrete case where we are drawing a distinction the physical piece of metering equipment and the sensor streams it produces.

Smart Meters are an electronic device that records consumption of electric energy and communicates the information to the electricity supplier for monitoring and billing.
The above def is not for "power"-specific devices so smart meters are not a "Power_Meter" but may have "Power_Sensor". I still can't see a real-world example of Power_Meter. I can see a device like this dedicated to electrical energy, which we should interpret as a sensor in Brick.
On the other hand, Electricity_Meter should be there as there are devices measuring different points of electrical properties like smart meters in your link.
I looked up power meter here and there

this: measuring power AND energy.
that: measuring voltage, power, energy, etc.

Again, I haven't seen any examples where a device is dedicated to only power but multiple instances of power. Instead, people commonly refer to "electrical meters" for measuring different electrical properties.

Using concepts in [square brackets], I would rephrase this as "drawing a distinction the [physical piece of metering equipment] the [features of interest] that the equipment measures, the [sensors] that measure the feature, and the streams of [observations] that some [measurement procedure] executed in the equipment produces."  This more detailed distinction uses SSN/SOSA terms as I understand them.  Not all of those [things] are necessarily [brick things].

The FSGIM only talks about electric meters which provide both power and energy measurements, and emissions meters that provide emission measurements (CH4, CO, CO2, CO2e, Hg).  At Cornell we also have chilled water meters (cooling energy) and potable water meters (total quantity and rate).
Curiously, the ASHRAE glossary defines metering as a process, "collection of energy data using a measuring device or group of measuring devices", and metered data as "energy end use data collected over time using a measuring device or group of measuring devices."  But doesn't define a meter as a measuring device.
The SEAS Smart Meter Ontology defines lots of meters, but leaves out chilled water which I guess is a subclass of Thermal Meter.

The most useful distinction for us is between "the [physical piece of metering equipment] the [features of interest] that the equipment measures". Having the equipment brought forward explicitly means that it can be positioned in terms of a meter hierarchy or downstream/upstream of other electric-related entities.
As a concrete proposal: how about Meter (generic) and Building_Meter as equipment. Then the different sensors may be associated with these equipment.
(going to avoid a digression where a "Power Meter" is a meter that only measures power, which is feasible and real, albeit not as helpful as one which measures other electrical characteristics)

Decided on the call:

Meter with substance-related subclasses: Water_Meter, Electric_Meter, etc
Add Building_Meter with the same substance-related subclasses above (until Location hierarchy is fleshed out)


HSEARCH-4045 Infer tools.jar location (for Byteman) correctly even when a different JVM is used for Maven and Surefire/Failsafe

https://hibernate.atlassian.net/browse/HSEARCH-4045
@fax4ever Your problems should be gone after you rebase your work on this branch. I tested locally and it did fix the problems.

Merged, thanks


Suggestion: being able to give a name to a saved snippet

When signed in, it would be great to be able to name a snippet. Especially when sharing an url in twitter (https://twitter.com/carbon_app/status/1235069621188751361), the twitter card could have a different title than simply "Carbon Snippet". Same for description.

What do you think ?

This is a great idea, and someone else also shared in it this issue https://github.com/carbon-app/carbon/issues/870#issue-498103778
Do you have suggestions for where the naming should go? I was thinking about adding an input to the bottom toolbar to the far left, opposite the copy/delete buttons. What do you think?

Is it possible to get this screen working locally by bypassing api or something ? I'd love to help.
Concerning UX, ss snippets are created automatically from the home page, name (and description) wouldn't be mandatory, so unset by default.

I was thinking about adding an input to the bottom toolbar to the far left, opposite the copy/delete buttons

yes I think bottom bar could work nice. But don't forget you'll need to show validation messages so maybe a modal or something could also works great.

@smknstd I'll look into that for you! I'm going to try and get to this feature next weekend.
In the meantime, if you want to start by adding a borderless text input of the bottom left of the Snippet Toolbar, that would be really helpful! You won't need the backend to work on that part of this feature 👍


feat(perp): Query that expresses when the next funding payment is

The funding_rate_interval of "30 min" given by nibid q perp params query tells you how to query the funding payments.
All of the info on funding payment timing is inside the epoch epoch-infos query.
Ex.
nibid q epochs epoch-infos | jq -r '.epochs[1]'

Returns
{
  "identifier": "30 min",
  "start_time": "2022-10-28T10:18:00.818642936Z",
  "duration": "1800s",
  "current_epoch": "1510",
  "current_epoch_start_time": "2022-11-28T20:48:00.818642936Z",
  "epoch_counting_started": true,
  "current_epoch_start_height": "743968"
}


Add "select as" support

Problem
In special cases I need select field with different name
In SQL, this is possible using the keyword AS, but Prisma don't support this feature
Suggested solution
I suggest add next syntax for support "select as" in Prisma
model User {
  id   Int
  name String
}

// Prisma will generate type like this for results with "select as"
type UserWithFullName = {
  fullName: string;
};

const users: UserWithFullName[] = prisma.user.findMany({
  select: {
    name: {
      as: 'fullName',
    },
  },
});

console.log(user[0]); // { fullName: 'Bran' }

Alternatives
Right now I need to use Array.map for transform field name in the results
// Prisma will generate type like this for result
type UserWithFullName = {
  fullName: string;
};

const users = prisma.user.findMany({
  select: {
    name: true,
  },
});

const usersWithFullName: UserWithFullName[] = users.map((user) => ({
  ...user,
  fullName: user.name,
}));

This can be difficult in the case of nested objects
Additional context
Implementation can transform field name both in the SQL query (with native AS) and when processing the results

Hi @yeswell, there are currently two ways to obtain a SELECT AS-like feature with Prisma.

Use @map when you need want to set an alias for every query involving a specific field. You just need to update your Prisma schema with:

model User {
  id   Int @id @default(autoincrement())
  fullName String @map("name")
// ^^^^^^^ `fullName` is the field name in the Prisma Client, while `name` is the column name as stored in your  database
}

which you can use in TypeScript as:
import { PrismaClient } from '@prisma/client'

const prisma = new PrismaClient()
await prisma.user.create({ data: { id: 1, fullName: 'Foo Bar' } })
const users = await prisma.user.findMany({ select: { fullName: true } })
console.log(users)

//^ [{ fullName: 'Foo Bar' }]

Side note: each model requires at least 1 primary key, hence the @id usage for the id field.

Use Prisma Client extensions when you want to set an alias in certain cases only - for instance, only when retrieving the result of a query. Here's one such example:

model User {
  id   Int @id @default(autoincrement())
  name String
}

which you can use in TypeScript as:
import { PrismaClient } from '@prisma/client'

const prisma = new PrismaClient().$extends({
  name: 'addFullNameToUser',
  result: {
    user: {
      fullName: {
        needs: { name: true },
        compute(user) {
          return user.name;
        },
      },
    },
  },
})

await prisma.user.create({ data: { id: 1, name: 'Foo Bar' } })
const users = await prisma.user.findMany({ select: { fullName: true } })
console.log(users)

//^ [{ fullName: 'Foo Bar' }]

Do you think these available options are enough for your use cases? Thanks for your feedback!

Hi, @jkomyno!
Thanks for the additional tips on the possibility of renaming, which I did not mention in my issue!
I know about the @map and .$extends(), but I think these are not completely suitable methods and I will try to explain why:

the @map is needed for permanent renaming, I think. For example, if the database column names use snake_case and the code uses camelCase
the .$extends() is closer, but it globally adds a new field to the type and calculates it with each request

In the case of special complex queries with large nesting (for example, reports), these are not very suitable options. And for such cases, the "select as" syntax would be very good:

it allows to rename the field locally
doesn't add unnecessary calculations for other queries

Here is an example of a complex query that I'm talking about
const report = await prisma.company.findMany({
  select: {
    id: {
      as: 'companyId',
    },
    name: true,
    goods: {
      select: {
        id: {
          as: 'goodId',
        },
        parameters: true,
      },
    },
  },
  where: {
    goods: {
      some: {
        createdAt: {
          gte: lastMonth,
        },
      },
    },
  },
});

Hi @yeswell, I'm struggeling to see what this could be useful for.
In your example, if the feature was implemented, the query would return something like this:
[
    {
        companyId: 1,
        goods: [
            { goodId: 1, parameters: { ... } },
            { goodId: 2, parameters: { ... } }
        ]
    }
]

However, I don't see how this is useful. You already know what you are querying (a company), and the name of the goods array already shows that it contains good-objects, so the goodId seems unnecessarily specific.
Maybe I am missing something, but I do not know when this feature would be useful, and could not be replaced by a @map. It makes sense in your first user-example, but there it would be better to you a @map, I believe. Do you maybe have another example with large nesting where this "select as" would be required? Or would you want a query with "select as" option to return something else? Thanks!


Add ability to get the rotation rate from the IMU

Adds a getRate function that returns an Optional of the rotation rate in degrees per sec. This was supported by all the IMU types, so the Optional may be overkill. I'm mainly asking for this because our code had a need for it and I wanted to only have the gyro defined in one place. This also makes the YAGSL interface to the IMUs more complete. Let me know if there's any changes I should make.

We were using it to make adjustments to the rotation of the robot. I'm not sure I understand it myself, this looks like something a student added last year. It's possible it's to adjust for gyro drift when turning fast.
double gyroRate = Units.degreesToRadians(swerveDrive.getGyroRate()) * 0.01;
Rotation2d correctedRotation = swerveDrive.getHeading().minus(new Rotation2d(gyroRate));
chassisSpeeds =
ChassisSpeeds.fromFieldRelativeSpeeds(
alliance == Alliance.Red ? -xSpeed : xSpeed,
alliance == Alliance.Red ? -ySpeed : ySpeed,
turnSpeed,
correctedRotation);

I also cleaned up the classes that used gyro instead of imu. Made it easier to change things.