updatehub with openthread

Hello
I am doing some changes to the updatehub code with the openthread demo. and successfully connect with the updatehub local server.
here are my code logs.

create a package  version using this
** ../../../../bootloader/mcuboot/scripts/imgtool.py sign --key ../../../../bootloader/mcuboot/root-ec-p256.pem --header-size 0x200 --align 8 --version 2.0.0 --slot-size 0x67000 --pad-header build/zephyr/zephyr.bin build/zephyr/zephyr-2.0.0.bin
**
linux config

my problem is code not successfully update the image.

Hi @nehalrohit,
There are two commands to sign image:

with --pad option (used to upload image using west flash) This can be BIN or HEX
imgtool.py sign --key <dir>/root-rsa-2048.pem --header-size 0x200 --align 8 --version 1.0.0 --slot-size 0x67000 --pad build/zephyr/zephyr.hex build/zephyr/zephyr-1.0.0.hex

without --pad/--pad-header options (used to create UHU package) this can be ONLY BIN
imgtool.py sign --key <dir>/root-rsa-2048.pem --header-size 0x200 --align 8 --version 2.0.0 --slot-size 0x67000 build/zephyr/zephyr.bin build/zephyr/zephyr-2.0.0.bin

The *.pem file must be the same that you used to compile MCUboot. On my case, I've been using the default root-rsa-2048.pem. My suggestion is, keep as simple possible at first and validate the full procedure.
CC: @otavio

Hi @nandojve
with --pad option (used to upload image using west flash) This can be BIN or HEX
imgtool.py sign --key /root-rsa-2048.pem --header-size 0x200 --align 8 --version 1.0.0 --slot-size 0x67000 --pad build/zephyr/zephyr.hex build/zephyr/zephyr-1.0.0.hex
with only --pad does not work for me. it gives me an error.

if I am using with pad-header and use root-rsa-2048.pem this key, successfully generate hex files but it gives me an error

So I used  ../../../../bootloader/mcuboot/scripts/imgtool.py sign --key ../../../../bootloader/mcuboot/root-ec-p256.pem --header-size 0x200 --align 8 --version 1.0.0 --slot-size 0x67000 --pad-header build/zephyr/zephyr.hex build/zephyr/zephyr-1.0.0.hex
that Is work for me.
and create a package with this
../../../../bootloader/mcuboot/scripts/imgtool.py sign --key ../../../../bootloader/mcuboot/root-ec-p256.pem --header-size 0x200 --align 8 --version 2.0.0 --slot-size 0x67000 --pad-header build/zephyr/zephyr.bin build/zephyr/zephyr-2.0.0.bin
still, give me an error , please give me some proper solution about it because it's very important for me my entire project stuck just because of this

Hi @nandojve
I am doing a successful image upload but it takes too much time to update the new image about 20min.

but after reboot, the new image not update.

Hi @nehalrohit ,
Let's try figure out!

why you are using root-ec-256.pem instead root-rsa-2048.pem?
Remember, you must use same key that MCUboot have been built. The default is root-rsa-2048.pem. Otherwise, the image will not be validated.

The MCUboot that you are using have only 1 slot. You need enable dual slot because you must have a second place to download the wholly image on the device before trigger swap.

Let's use this procedure for now to build MCUboot version.
This is how I build Nordic MCUboot to proper work with UpdateHub:
cd <your zephyr dir>
git checkout master
git  fetch upstream or origin
git rebase upstream/master
west update
cd ../bootloader/mcuboot/boot/zephyr
rm -rf build-nrf52840dk_nrf52840
mkdir build-nrf52840dk_nrf52840 && cd build-nrf52840dk_nrf52840
cmake -GNinja -DBOARD=nrf52840dk_nrf52840 ..

# note below PEM file selected!
MCUBoot bootloader key file: <your zephyr dir>/bootloader/mcuboot/root-rsa-2048.pem

ninja
west update

Note that there are two slots now (primary and secondary images)
*** Booting Zephyr OS build zephyr-v2.4.0-1305-g2a462de260c6  ***
[00:00:00.383,575] <inf> mcuboot: Starting bootloader
[00:00:00.389,984] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.400,329] <inf> mcuboot: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.410,797] <inf> mcuboot: Boot source: none
[00:00:00.416,290] <wrn> mcuboot: Failed reading image headers; Image=0
[00:00:00.423,889] <err> mcuboot: Unable to find bootable image

If you upgrade a hello world sample without network support, UpdateHub will never send any information about what happened after upgrade to the server and server will wait "forever". Please, package the same image that you have running on the device that access UpdateHub-CE with a different version:

Follow this procedure:

build the image that you have access to UpdateHub CE

sign with --pad option
imgtool.py sign --key <dir>/root-rsa-2048.pem --header-size 0x200 --align 8 --version 1.0.0 --slot-size 0x67000 --pad build/zephyr/zephyr.hex build/zephyr/zephyr-1.0.0.hex

Upload the image to board
west flash --hex-file build/zephyr/zephyr-1.0.0.hex

sign without --pad/--pad-header options (now must select the bin file)
imgtool.py sign --key <dir>/root-rsa-2048.pem --header-size 0x200 --align 8 --version 2.0.0 --slot-size 0x67000 build/zephyr/zephyr.bin build/zephyr/zephyr-2.0.0.bin

Create UHU package
https://docs.zephyrproject.org/latest/samples/subsys/mgmt/updatehub/README.html#step-8-create-a-package-with-updatehub-utilities-uhu

Notes:

UpdateHub is small and don't increase the firmware size. It selects all pieces and put it together to have a full functional OTA.
OpenThread is a very large network layer and probably may be responsible for half image size.
Speed, It is difficult say something without proper configuration.

My initial metrics are:

less than 10min for IEEE 802.15.4
less than 30min for BLE IPSP

I'm looking forward to hear from you the result.

hello @nandojve
sign without --pad/--pad-header options (now must select the bin file)
imgtool.py sign --key /root-rsa-2048.pem --header-size 0x200 --align 8 --version 2.0.0 --slot-size 0x67000 build/zephyr/zephyr.bin build/zephyr/zephyr-2.0.0.bin
when I create a sign image for ota it gives error.
Error: Header padding was not requested and image does not start with zeros

Hi @nandojve
Thank you very much I did it successfully.
after adding  sysreboot function. I did successfully update the image.

it takes around 23 min (updatehub+openthread bin file 386kB) is it any other configuration to reduce update time.
it is prj.conf file
CONFIG_NCS_SAMPLES_DEFAULTS=n
CONFIG_NEWLIB_LIBC=n
Generic networking options
CONFIG_NETWORKING=n
CONFIG_NET_CONNECTION_MANAGER=n
Enable multiprotocol service layer
CONFIG_MPSL=n
Kernel options
CONFIG_MAIN_STACK_SIZE=2048
CONFIG_INIT_STACKS=y
Increase set for threads with meta-irq priority
CONFIG_NUM_METAIRQ_PRIORITIES=1
Logging
CONFIG_NET_LOG=n
CONFIG_NET_STATISTICS=n
Disable certain parts of Zephyr IPv6 stack
CONFIG_NET_IPV6_NBR_CACHE=n
CONFIG_NET_IPV6_MLD=n
Stack sizes configuration
CONFIG_NET_TX_STACK_SIZE=1200
CONFIG_NET_RX_STACK_SIZE=1500
Network buffers
CONFIG_NET_PKT_RX_COUNT=10
CONFIG_NET_PKT_TX_COUNT=10
CONFIG_NET_BUF_RX_COUNT=16
CONFIG_NET_BUF_TX_COUNT=16
L2 OpenThread enabling
CONFIG_NET_L2_OPENTHREAD=y
CONFIG_OPENTHREAD_DEBUG=n
CONFIG_OPENTHREAD_L2_DEBUG=n
CONFIG_OPENTHREAD_L2_LOG_LEVEL_INF=n
Disable vanilla mbedtls in favor of custom backends
CONFIG_OPENTHREAD_MBEDTLS=n
Disable unneeded mbedtls modules
CONFIG_MBEDTLS_CIPHER_MODE_CBC=n
CONFIG_MBEDTLS_CIPHER_MODE_CTR=n
CONFIG_MBEDTLS_CIPHER_MODE_CFB=n
CONFIG_MBEDTLS_CIPHER_MODE_OFB=n
CONFIG_MBEDTLS_CIPHER_MODE_XTS=n
CONFIG_MBEDTLS_GCM_C=n
CONFIG_MBEDTLS_CHACHA20_C=n
CONFIG_MBEDTLS_POLY1305_C=n
CONFIG_MBEDTLS_CHACHAPOLY_C=n
CONFIG_MBEDTLS_DHM_C=n
CONFIG_MBEDTLS_RSA_C=n
CONFIG_MBEDTLS_SHA1_C=n
CONFIG_MBEDTLS_SHA512_C=n
CONFIG_MBEDTLS_HEAP_SIZE=10240

Copyright (c) 2020 Nordic Semiconductor

SPDX-License-Identifier: LicenseRef-BSD-5-Clause-Nordic

nRF board library
CONFIG_DK_LIBRARY=y
Enable OpenThread CoAP support API
CONFIG_OPENTHREAD_COAP=y
Configure sample logging setting
CONFIG_COAP_SERVER_LOG_LEVEL_DBG=n
CONFIG_OT_COAP_UTILS_LOG_LEVEL_DBG=n
Network shell
CONFIG_NET_SHELL=y
CONFIG_SHELL_ARGC_MAX=26
CONFIG_SHELL_CMD_BUFF_SIZE=416
Network sockets
CONFIG_NET_SOCKETS=y
CONFIG_NET_SOCKETS_POSIX_NAMES=y
CONFIG_NET_SOCKETS_POLL_MAX=4
OpenThread manual start
CONFIG_OPENTHREAD_MANUAL_START=y
CONFIG_OPENTHREAD_DNS_CLIENT=y
Enable OpenThread features set
CONFIG_OPENTHREAD_NORDIC_LIBRARY_MASTER=y
CONFIG_OPENTHREAD_MTD=y
CONFIG_OPENTHREAD_MTD_SED=y
CONFIG_OPENTHREAD_POLL_PERIOD=3000
CONFIG_RAM_POWER_DOWN_LIBRARY=y
CONFIG_DEVICE_POWER_MANAGEMENT=y
Turn on the updatehub
CONFIG_UPDATEHUB=y
Minimal Heap mem pool size for the updatehub working
CONFIG_HEAP_MEM_POOL_SIZE=8192
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048
CONFIG_SHELL_STACK_SIZE=8192
Network management events
CONFIG_NET_CONNECTION_MANAGER=y
CONFIG_NET_IPV4=n
CONFIG_NET_CONFIG_NEED_IPV4=n
CONFIG_NET_IPV6=y
CONFIG_NET_DHCPV4=n
Optional if you would like test on the your server
CONFIG_SHELL=y
CONFIG_NET_SHELL=y
CONFIG_UPDATEHUB_SHELL=y
CONFIG_SHELL_STACK_SIZE=6144
Debug helpers
#CONFIG_LOG=y
#CONFIG_UPDATEHUB_LOG_LEVEL_DBG=y
#CONFIG_COAP_LOG_LEVEL_DBG=y
LOG_IMMEDIATE could infer problems on WIFI reception and is recommended keep disabled
Use with caution!!
#CONFIG_LOG_IMMEDIATE=y
CONFIG_UPDATEHUB_PRODUCT_UID="e4d37cfe6ec48a2d069cc0bbb8b078677e9a0d8df3a027c4d8ea131130c4265f"
#CONFIG_UPDATEHUB_POLL_INTERVAL=1
#CONFIG_UPDATEHUB_MANUAL=1
CONFIG_UPDATEHUB_CE=y
CONFIG_COAP_INIT_ACK_TIMEOUT_MS=10000
Add Google <IP_ADDRESS> DNS Server
CONFIG_DNS_SERVER_IP_ADDRESSES=y
CONFIG_DNS_SERVER1="fdaa:bb:1::2"
Add UpdateHub Cloud
CONFIG_UPDATEHUB_SERVER="fd11:1111:1122:0:35f5:e92d:7629:b4bd"
#CONFIG_UPDATEHUB_COAP_BLOCK_SIZE_EXP=4
#CONFIG_STDOUT_CONSOLE=y
#CONFIG_I2C=y
#CONFIG_SENSOR=y
#CONFIG_SHT3XD=y
CONFIG_REBOOT=y

Hi @nehalrohit ,
Below is my setup
IEEE 802.15.4:
Host: PC Linux > 4.9 + RADVD > 2.7 + ATUSB dongle or WPANUSB (can be nRF/rf2xx running Zephyr)
Remote Node:  nRF/rf2xx
OpenThread:
Host: PC Linux > 4.9 + RADVD > 2.7 + nRF52 NCP OpenThread 1.1.1, original code (not Zephyr's)
Remote Node:  nRF52
I would like know how is your setup?
How many nodes you have at same time?
Note:
#CONFIG_UPDATEHUB_COAP_BLOCK_SIZE_EXP=4
Above Kconfig define the CoAP block size. The default value is 6 which means 1024 bytes payload.
CoAP will request IPv6 data fragments to complete the 1024 payload before pass that data to UpdateHub.
You can decrease expoente to 4 and 2 to check best performance. I think it will be 2 for OpenThread.
CONFIG_UPDATEHUB_COAP_BLOCK_SIZE_EXP=4
and
CONFIG_UPDATEHUB_COAP_BLOCK_SIZE_EXP=2

Hi @nandojve
I have some questions.
1.- how did you setup RADVD in your host Linux?
2 - total time to update the image from your openthread setup?
3 - node and Linux system are in the same network(PC Linux --> wifi --> border router ) is it possible to ping Linux system from the node.
currently for testing, my setup same as your PC Linux -> NCP dongle(NCP OpenThread 1.1.1)  and 1 node
I  change CONFIG_UPDATEHUB_COAP_BLOCK_SIZE_EXP=2 or 4  but it takes more time as compared to  CONFIG_UPDATEHUB_COAP_BLOCK_SIZE_EXP=6
Nehal........

1.- how did you setup RADVD in your host Linux?

It is not necessary to run UpdateHub and have internet connectivity.

2 - total time to update the image from your openthread setup?

The below scenario uses OT 1.1 with NCP on a Linux PC as border router. It uses Tayga for NAT64 and OT example default configs. This is based on PR #30460. The UpdateHub-CE is a docker hosted at an US service provider at NY. I'm running at South America and ping is around 138 ms from OT node to US server.
Application un-signed
[513/518] Linking CXX executable zephyr/zephyr_prebuilt.elf
Memory region         Used Size  Region Size  %age Used
           FLASH:      339648 B       412 KB     80.51%
            SRAM:      115631 B       256 KB     44.11%
        IDT_LIST:         120 B         2 KB      5.86%
[518/518] Linking CXX executable zephyr/zephyr.elf

Download: ~ 16m 30s
Swap: ~30s
Confirm: ~1 min
*** Booting Zephyr OS build zephyr-v2.4.0-2049-gdd5f0994f9d2  ***
[00:00:00.393,005] <inf> mcuboot: Starting bootloader
[00:00:00.399,475] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x1
[00:00:00.409,942] <inf> mcuboot: Secondary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x3
[00:00:00.420,501] <inf> mcuboot: Boot source: none
[00:00:00.426,208] <inf> mcuboot: Swap type: none
[00:00:00.878,601] <inf> mcuboot: Bootloader chainload address offset: 0xc000
[00:00:00.886,535] <inf> mcuboot: Jumping to the first image slot

uart:~$ *** Booting Zephyr OS build zephyr-v2.4.0-2239-gacd86316efd9  ***
[00:00:00.004,272] <inf> ieee802154_nrf5: nRF5 802154 radio initialized
[00:00:00.016,815] <inf> fs_nvs: 8 Sectors of 4096 bytes
[00:00:00.016,815] <inf> fs_nvs: alloc wra: 0, f40
[00:00:00.016,815] <inf> fs_nvs: data wra: 0, 178
[00:00:00.030,334] <inf> main: UpdateHub sample app started
[00:00:00.030,334] <inf> main: Confirming the boot image
[00:00:00.030,395] <inf> main: Network connected
[00:00:00.030,395] <inf> main: Starting UpdateHub polling mode
[00:00:00.030,395] <inf> updatehub: SHA-256 verification on download and from flash
[00:00:02.031,982] <err> updatehub: Could not receive data
uart:~$ updatehub info
Unique device id: d7eed6b80dd18d94
Firmware Version: 1.0.0
Product uid: e4d37cfe6ec48a2d069cc0bbb8b078677e9a0d8df3a027c4d8ea131130c4265f
UpdateHub Server: XXXXXXX
[00:00:16.942,413] <inf> updatehub: No update available
[00:01:02.200,653] <inf> updatehub: No update available
[00:02:02.393,066] <inf> updatehub: Probe metadata received
[00:18:31.403,961] <inf> updatehub: Firmware download complete
�*** Booting Zephyr OS build zephyr-v2.4.0-2049-gdd5f0994f9d2  ***u can reboot now
[00:00:00.255,889] <inf> mcuboot: Starting bootloader
[00:00:00.262,390] <inf> mcuboot: Primary image: magic=unset, swap_type=0x1, copy_done=0x3, image_ok=0x1
[00:00:00.272,827] <inf> mcuboot: Secondary image: magic=good, swap_type=0x2, copy_done=0x3, image_ok=0x3
[00:00:00.283,294] <inf> mcuboot: Boot source: none
[00:00:00.289,001] <inf> mcuboot: Swap type: test
[00:00:33.645,111] <inf> mcuboot: Bootloader chainload address offset: 0xc000
[00:00:33.653,045] <inf> mcuboot: Jumping to the first image slot

uart:~$ *** Booting Zephyr OS build zephyr-v2.4.0-2239-gacd86316efd9  ***
[00:00:00.004,272] <inf> ieee802154_nrf5: nRF5 802154 radio initialized
[00:00:00.016,662] <inf> fs_nvs: 8 Sectors of 4096 bytes
[00:00:00.016,662] <inf> fs_nvs: alloc wra: 0, c60
[00:00:00.016,662] <inf> fs_nvs: data wra: 0, 680
[00:00:00.045,623] <inf> main: UpdateHub sample app started
[00:00:00.045,623] <inf> main: Confirming the boot image
[00:00:00.045,745] <inf> main: Network connected
[00:00:00.045,745] <inf> main: Starting UpdateHub polling mode
[00:00:00.045,745] <inf> updatehub: SHA-256 verification on download and from flash
[00:00:02.047,363] <err> updatehub: Could not receive data
[00:01:02.214,233] <inf> updatehub: No update available
uart:~$ updatehub info
Unique device id: d7eed6b80dd18d94
Firmware Version: 2.0.0
Product uid: e4d37cfe6ec48a2d069cc0bbb8b078677e9a0d8df3a027c4d8ea131130c4265f
UpdateHub Server: XXXXXX
uart:~$ net ping XXXXXX
PING XXXXXX
8 bytes from XXXXXX to fd11:22::6a58:2c2c:1425:7900: icmp_seq=0 ttl=48 rssi=0 time=139 ms
8 bytes from XXXXXX to fd11:22::6a58:2c2c:1425:7900: icmp_seq=1 ttl=48 rssi=0 time=136 ms
8 bytes from XXXXXX to fd11:22::6a58:2c2c:1425:7900: icmp_seq=2 ttl=48 rssi=0 time=138 ms
uart:~$

3 - Any reason why it takes too much time to ota update( when I did ota with using thread for Taxsas CC1352R1 MCU it takes less than 10 sec)?

This is interesting, are you sure you can download ~400k byte data from a server, with NAT, store data on flash than update in less than 10s? Maybe I can achieve using 100M bit Ethernet on Zephyr. I think Taxsas numbers that you shared seems unrealistic knowing that you can have 128 byte max air frame at 250k bit rate with compressed IPv6.

4 - node and Linux system are in the same network(PC Linux --> wifi --> border router ) is it possible to ping the Linux system from the node.

Yes, you can add a manual IP on node and Linux or use RADVD.
If you ping from node net ping ff02::1 you will see Linux IPv6 unicast slaac address.

Hi @nandojve
I am following your guide above to do OTA too.
In the last step, I encountered this error:<err> updatehub: Could not receive data
uart:~$ updatehub info
Unique device id: 4969258d739710de
Firmware Version: 1.0.0
Product uid: e4d37cfe6ec48a2d069cc0bbb8b078677e9a0d8df3a027c4d8ea131130c4265f
UpdateHub Server: 64:ff9b::6a0d:d7b7
[00:01:02.171,905] <inf> updatehub: Probe metadata received
[00:01:30.928,161] <err> updatehub: Could not receive data
[00:01:32.928,253] <err> updatehub: Could not receive data
[00:01:34.928,344] <err> updatehub: Could not receive data
[00:01:36.928,436] <err> updatehub: Could not receive data
[00:01:38.928,527] <err> updatehub: Could not receive data
[00:01:49.753,234] <err> updatehub: Could not receive data
[00:01:51.753,356] <err> updatehub: Could not receive data
[00:01:53.753,448] <err> updatehub: Could not receive data
[00:01:55.753,540] <err> updatehub: Could not receive data
[00:01:57.753,631] <err> updatehub: Could not receive data
[00:02:57.824,279] <err> updatehub: Could not receive data
[00:02:59.824,371] <err> updatehub: Could not receive data
uart:~$ net ping 64:ff9b::<IP_ADDRESS>
PING 64:ff9b::<IP_ADDRESS>
8 bytes from 64:ff9b::808:808 to fd11:22::dfbd:cd1:d6f0:d1d4: icmp_seq=0 ttl=111 rssi=0 time=48 ms
8 bytes from 64:ff9b::808:808 to fd11:22::dfbd:cd1:d6f0:d1d4: icmp_seq=1 ttl=111 rssi=0 time=58 ms
8 bytes from 64:ff9b::808:808 to fd11:22::dfbd:cd1:d6f0:d1d4: icmp_seq=2 ttl=111 rssi=0 time=64 ms
uart:~$

Could you please help me to solve this problem? Thanks in advance!

Hi @IHIDCHAOS ,
UpdateHub is expecting an UpdateHub-CE running at 64:ff9b::6a0d:d7b7. Are you sure that UpdateHub-CE is running? Are there routes, no firewall blocking etc?
https://docs.zephyrproject.org/latest/samples/subsys/mgmt/updatehub/README.html#step-2-1-updatehub-ce-community-edition

Hi @IHIDCHAOS ,
UpdateHub is expecting an UpdateHub-CE running at 64:ff9b::6a0d:d7b7. Are you sure that UpdateHub-CE is running? Are there routes, no firewall blocking etc?
https://docs.zephyrproject.org/latest/samples/subsys/mgmt/updatehub/README.html#step-2-1-updatehub-ce-community-edition

Hi @nandojve ,
I am sure that my UpdateHub-CE is running like this (old picture):

And I think the output [00:01:02.171,905] <inf> updatehub: Probe metadata received can also prove this.
I have tried to capture the packet from Wireshark on My OTBR wpan0, It seems that it's always sending the same packet.
I will put some packets later. Thank you!

Hi @IHIDCHAOS ,
Yes, you are right! It seems something in the UpdateHub-CE side, at first. Are you using latest version or an old one?
Any data that you can share may help to find the problem. Share your diff configs from current master too.

Yes I am using the latest Version: commit 4ca7942411f70f0caee6cd872ab14200f88b221b (HEAD -> master, origin/master)
I modified:
ihid@ubuntu:~/zephyrproject/zephyr$ git diff
diff --git a/samples/subsys/mgmt/updatehub/overlay-ot.conf b/samples/subsys/mgmt/updatehub/overlay-ot.conf
index d425f950a3..9f5ec77eae 100644
--- a/samples/subsys/mgmt/updatehub/overlay-ot.conf
+++ b/samples/subsys/mgmt/updatehub/overlay-ot.conf
@@ -28,10 +28,11 @@ CONFIG_OPENTHREAD_PANID=4660
 CONFIG_OPENTHREAD_CHANNEL=15
 CONFIG_OPENTHREAD_NETWORK_NAME="OpenThreadDemo"
 CONFIG_OPENTHREAD_XPANID="0x1111111122222222"
-CONFIG_OPENTHREAD_JOINER=y
-CONFIG_OPENTHREAD_JOINER_AUTOSTART=y
-CONFIG_OPENTHREAD_JOINER_PSKD="J01NU5"
+#CONFIG_OPENTHREAD_JOINER=y
+#CONFIG_OPENTHREAD_JOINER_AUTOSTART=y
+#CONFIG_OPENTHREAD_JOINER_PSKD="J01NME"
 CONFIG_OPENTHREAD_SLAAC=y
+CONFIG_NET_IF_UNICAST_IPV6_ADDR_COUNT=6

 CONFIG_NET_IPV6_NBR_CACHE=n
 CONFIG_NET_IPV6_MLD=n
@@ -42,7 +43,6 @@ CONFIG_MBEDTLS_SSL_MAX_CONTENT_LEN=768
 # UpdateHub
 CONFIG_MAIN_STACK_SIZE=8192
 CONFIG_UPDATEHUB_COAP_BLOCK_SIZE_EXP=2
-
 # Only tested with 6to4
 #
 #######################
@@ -54,4 +54,4 @@ CONFIG_DNS_SERVER_IP_ADDRESSES=y
 # Add prefix        64:ff9b::
 CONFIG_DNS_SERVER1="64:ff9b::808:808"
 # Repeat the above step with your server IPv4 address translated to IPv6
-CONFIG_UPDATEHUB_SERVER="<your server ip address>"
+CONFIG_UPDATEHUB_SERVER="64:ff9b::6a0d:d7b7"
diff --git a/samples/subsys/mgmt/updatehub/src/main.c b/samples/subsys/mgmt/updatehub/src/main.c
index a2d82323ae..efde6c34db 100644
--- a/samples/subsys/mgmt/updatehub/src/main.c
+++ b/samples/subsys/mgmt/updatehub/src/main.c
@@ -11,6 +11,7 @@
 #include <net/net_conn_mgr.h>
 #include <net/wifi_mgmt.h>
 #include <dfu/mcuboot.h>
+#include <power/reboot.h>

 #if defined(CONFIG_UPDATEHUB_DTLS)
 #include <net/tls_credentials.h>
@@ -27,6 +28,7 @@ static struct net_mgmt_event_callback mgmt_cb;

 void start_updatehub(void)
 {
+       int ret;
 #if defined(CONFIG_UPDATEHUB_SAMPLE_POLLING)
        LOG_INF("Starting UpdateHub polling mode");
        updatehub_autohandler();

And add: zephyrproject/zephyr/samples/subsys/mgmt/updatehub/overlay-prj.conf
#
# Copyright (c) 2018 O.S.Systems
#
# SPDX-License-Identifier: Apache-2.0
#
CONFIG_SHELL=y
CONFIG_OPENTHREAD_SHELL=y
CONFIG_UPDATEHUB_SHELL=y
#Example of product_uid, you must set you product uid.
CONFIG_UPDATEHUB_PRODUCT_UID="e4d37cfe6ec48a2d069cc0bbb8b078677e9a0d8df3a027c4d8ea131130c4265f"

#If you use the autorandler, need set UPDATEHUB_POLL_INTERVAL.
#This example, the updatehub will poll on the server
#request for updates one to one minute.
#(min: 0 minute, máx: 30 days)

CONFIG_UPDATEHUB_SAMPLE_MANUAL=y
#CONFIG_UPDATEHUB_POLL_INTERVAL=1

CONFIG_UPDATEHUB_CE=y

#You need set the updatehub-ce ip Address.
#This is a example of ip address
CONFIG_UPDATEHUB_SERVER="64:ff9b::6a0d:d7b7"

UpdateHub-CE runs the latest version too and the port are opened.:
docker run -it -d -p 8080:8080 -p 5683:5683/udp --rm updatehub/updatehub-ce:latest


Add @netlify/bots as owner on crates.io

So we can publish to crates.io from a machine account (e. g. netlify-jenkins user).
I cannot do this on my own because I'm not an owner of the library on crates.io.
Instructions here: https://doc.rust-lang.org/cargo/reference/publishing.html#cargo-owner

% cargo owner --add github:netlify:bots :(
    Updating crates.io index
error: failed to invite owners to crate netlify_toml: api errors (status 200 OK): could not find the github team netlify/bots

Ugh, is it perhaps because it's private somehow?

Alternatively can you add netlify-jenkins as owner? That is probably already enough.

i think it needs a crates account?
error: failed to invite owners to crate netlify_toml: api errors (status 200 OK): could not find user with login `netlify-jenkins`

Going to cr(e)ate that really quick for netlify-jenkins.

✅ let's try again!

Owner user netlify-jenkins has been invited to be an owner of crate netlify_toml


Add a monitor to watch for census county changes

RMR staffers who update county-level mortgage data need to know when a
county changes name or FIPS code, as sometimes happens (and cropped up in 2017). This monitor
compares the posted census change log against a local base change log.
If a change is detected, it sends emails with the change and updates the
base change log so the monitor can watch for the next change.
The monitor can be called with the management command oah_check_county_changes [--email]
The --email option will accept one or more email recipients,
separated by spaces, who will be notified about changes.
Review
@mthibos @amymok
Testing
The monitor can be run locally or in Jenkins, where a test is running daily.

Coverage decreased (-3.2%) to 96.056% when pulling 03f25f83091d323dc69a1038264f2c374bd45fcf on data-automation into c72513b675d270613d98c9129f6b8f0654e05e48 on master.

Coverage decreased (-3.2%) to 96.056% when pulling 03f25f83091d323dc69a1038264f2c374bd45fcf on data-automation into c72513b675d270613d98c9129f6b8f0654e05e48 on master.

Coverage increased (+0.2%) to 99.425% when pulling 96ed34f46eff0c14b899d8db688ef513290d7f5b on data-automation into c72513b675d270613d98c9129f6b8f0654e05e48 on master.

Coverage increased (+0.2%) to 99.425% when pulling 75b64126c5c6bd39b52537d5fb9594536a5884df on data-automation into c72513b675d270613d98c9129f6b8f0654e05e48 on master.

Coverage increased (+0.2%) to 99.425% when pulling ad57996d659cdd9668efee8ba767623ae31af6d6 on data-automation into c72513b675d270613d98c9129f6b8f0654e05e48 on master.

This is great~


HEIF image rotation metadata not respected

Some HEIC/HEIF images from iOS have rotation metadata (in "irot" and/or EXIF?) that we're parsing but not respecting when generating JPEGs (thumbnails & full size) for the web UI.
@mpl, I sent you an example on Telegram.
Part of #969

Running go4.org/media/heif/dumpheiff on the image in question,
primary item: 49
spatial extents: 4032 x 3024
properties:
        "ispe": &bmff.ImageSpatialExtentsProperty{FullBox:bmff.FullBox{box:(*bmff.box)(0xc000087720), Version:0x0, Flags:0x0}
, ImageWidth:0xfc0, ImageHeight:0xbd0}
        "irot": &bmff.ImageRotation{box:(*bmff.box)(0xc000087770), Angle:0x3}
        "pixi": &bmff.box{size:16, boxType:bmff.BoxType{0x70, 0x69, 0x78, 0x69}, body:(*io.LimitedReader)(0xc0000d4de0), pars
ed:bmff.Box(nil), slurp:[]uint8{0x0, 0x0, 0x0, 0x0, 0x3, 0x8, 0x8, 0x8}}
EXIF dump:
        SensingMethod = 2
        Orientation = 6
        Software = "11.3"
        ExposureBiasValue = "0/1"
        ExposureTime = "1/30"
        LensModel = "iPhone 7 back camera 3.99mm f/1.8"
        ...

Notably: "irot" is 0x3 (units of 90 degrees counter-clockwise) and the EXIF Orientation is 6. See https://www.daveperrett.com/articles/2012/07/28/exif-orientation-handling-is-a-ghetto/ for pictures and http://sylvana.net/jpegcrop/exif_orientation.html a table.
We can probably just use the irot value.
I'll add an accessor to https://godoc.org/go4.org/media/heif

@mpl, see just-pushed go4.org commit (referenced above)

I think we'll want to patch heiftojpeg with something like:
diff --git a/Srcs/heiftojpeg/heiftojpeg.cpp b/Srcs/heiftojpeg/heiftojpeg.cpp
index d49f94d..5696cba 100644
--- a/Srcs/heiftojpeg/heiftojpeg.cpp
+++ b/Srcs/heiftojpeg/heiftojpeg.cpp
@@ -111,7 +111,7 @@ static void addExif(ImageFileReaderInterface::DataVector exifData, string fileNa
     }
 }

-static void processFile(char *filename, char *outputFileName)
+static void processFile(char *filename, char *outputFileName, int rotate)
 {
     HevcImageFileReader reader;
     reader.initialize(filename);
@@ -197,6 +197,7 @@ static void processFile(char *filename, char *outputFileName)
     Magick::Image image = montage.front();
     image.magick("JPEG");
     image.crop(Magick::Geometry(gridItem.outputWidth, gridItem.outputHeight));
+    image.rotate(double(rotate));
     image.quality(92);

     string timingName = "magick montage+crop";
@@ -225,7 +226,7 @@ static void processFile(char *filename, char *outputFileName)

 int usage()
 {
-    cerr << "Usage: heiftojpeg [-v] [-s <max_dimension>] <input.heic> <output.jpg>\n";
+    cerr << "Usage: heiftojpeg [-v] [-s <max_dimension>] [-r <rotate>] <input.heic> <output.jpg>\n";
     return 1;
 }

@@ -237,6 +238,7 @@ int main(int argc, char *argv[])
     char *outputFileName = NULL;
     int index;
     int c;
+    int rotate = 0;

     opterr = 0;
     while ((c = getopt (argc, argv, "vs:")) != -1) {
@@ -247,6 +249,8 @@ int main(int argc, char *argv[])
             case 's':
                 MAX_SIZE = atoi(optarg);
                 break;
+            case 'r':
+                rotate = atoi(optarg);
             case '?':
                 return usage();
             default:
@@ -270,7 +274,7 @@ int main(int argc, char *argv[])

     try {
         chrono::steady_clock::time_point begin = chrono::steady_clock::now();
-        processFile(inputFileName, outputFileName);
+        processFile(inputFileName, outputFileName, rotate);
         chrono::steady_clock::time_point end = chrono::steady_clock::now();
         if (VERBOSE) {
             cout << "[timing] processFile " << chrono::duration_cast<chrono::milliseconds>(end - begin).count() << "ms" << endl;

... so we can explicitly tell heiftojpeg what to do (since we've already read the HEIF file's irot section), via https://godoc.org/go4.org/media/heif#Item.Rotations.
Then we continue to do the /bin/true thing for exiftool in the container and generate JPEGs with no EXIF (notably: no Orientation field, since the pixels will be correctly encoded without metadata).
I verified that in 2018, Chrome and Telegram Desktop still get JPEG EXIF rotation wrong.
But if upstream heiftojpeg wanted to fix this, they could just do the right thing without the flag.
They'd then need to remove the Orientation field from the output JPEG's metadata, though.

Dunno if there's existing Go code to do rotation. probably?

Yes. And it's also just a for loop.
In any case, I fixed the rotation with https://github.com/perkeep/heif/commit/e83468424c031af9734f5f382cf0bfb525b63cf6 to heiftojpeg, but it's incredibly slow for me. It takes 1.7 seconds on a single image. It's invoking ffmpeg for each 500x500 pixel "tile" of the image.
This'll work for now, but we should move to something faster.
We can write a little C program around libheif to do what we want.


Validation fails even though debug api is clearly present.

** Description **
I setup the debugger in my plsql instance and enabled the extension, but on the IDEA side it think's it's not there.
To Reproduce
Install the debugger in pl/sql delivered by home-brew
Try and debug something in IntelliJ.
Expected behavior
I can debug in intellij
Screenshots

Desktop (please complete the following information):

OS: macOS Sonoma
Intellj: 2024.1.5
postgresql 14
debugger plugin: 241.0.1

Hi @epakala,
To resolve the issue, please ensure that the debugger is installed as a shared library on your PostgreSQL server. You can achieve this by adding the appropriate pldbgapi extension to the shared_preload_libraries in your postgresql.conf file. This setup is necessary for the plugin to function correctly.
For your convenience, I provide a pre-configured Docker image that includes the debugger extension. You can easily use this image with the following command:
docker run -p 5515:5432 --name PG15-debug -e POSTGRES_PASSWORD=postgres -d galien0xffffff/postgres-debugger:15

This image is based on the official PostgreSQL image and supports versions 11 to 16. More detailed instructions on how to set this up can be found in the README of the repository.
Let me know if you need further assistance!
Best regards,
Galien


'reportUnusedDisableDirectives' must be any of "error", "warn", "off", and null

When I update my eslint installation, VSCode linting stops working with the below error. It works fine from the command line, so something in vscode is inserting it.
I don't set reportUnusedDisableDirectives myself, and this happens even when I disable everything in my .eslintrc.
I added logging in the eslint module and I see that reportUnusedDisableDirectives is set to true.
(node:1345199) UnhandledPromiseRejectionWarning: Error: Invalid Options:
- 'reportUnusedDisableDirectives' must be any of "error", "warn", "off", and null.
    at processOptions (/home/wmertens/Projects/Cap/node_modules/.pnpm/eslint@8.6.0/node_modules/eslint/lib/eslint/eslint.js:293:15)
    at new ESLint (/home/wmertens/Projects/Cap/node_modules/.pnpm/eslint@8.6.0/node_modules/eslint/lib/eslint/eslint.js:440:34)
    at Object.e.newESLintClass (/home/wmertens/.vscode/extensions/dbaeumer.vscode-eslint-2.2.2/server/out/eslintServer.js:1:167243)
    at Ce (/home/wmertens/.vscode/extensions/dbaeumer.vscode-eslint-2.2.2/server/out/eslintServer.js:1:186299)
    at h.then.o.validate (/home/wmertens/.vscode/extensions/dbaeumer.vscode-eslint-2.2.2/server/out/eslintServer.js:1:176323)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
(node:1345199) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 11230)
[Info  - 7:28:35 AM] Invalid Options: - 'reportUnusedDisableDirectives' must be any of "error", "warn", "off", and null.

I was able to work around this by setting reportUnusedDisableDirectives: true in my eslintrc. I tried setting it to "error" first but that gave an error, in the eslintrc it has to be a boolean.

@wmertens can you please provide me with a Github repository I can clone that demos that. Otherwise it is very hard to tell why this fails.

@dbaeumer thanks for checking! I have it with current master of https://github.com/StratoKit/strato-db.
Actually the workaround doesn't work, I forgot I patched eslint to ignore it.
The options that it uses are just the below, not from my eslintrc.
[Info  - 1:36:20 PM] Invalid Options: - 'reportUnusedDisableDirectives' true must be any of "error", "warn", "off", and null.
[Object: null prototype] {
  reportUnusedDisableDirectives: true,
  cwd: '/home/wmertens/Projects/strato-db'
}

I think they're the command line options somehow, when I run npx eslint . in the terminal the options it uses are
{
  allowInlineConfig: true,
  cache: false,
  cacheLocation: '.eslintcache',
  cacheStrategy: 'metadata',
  errorOnUnmatchedPattern: true,
  extensions: undefined,
  fix: undefined,
  fixTypes: undefined,
  ignore: true,
  ignorePath: undefined,
  overrideConfig: {
    env: undefined,
    globals: undefined,
    ignorePatterns: undefined,
    parser: undefined,
    parserOptions: undefined,
    plugins: undefined,
    rules: undefined
  },
  overrideConfigFile: undefined,
  reportUnusedDisableDirectives: undefined,
  resolvePluginsRelativeTo: undefined,
  rulePaths: undefined,
  useEslintrc: true
}

If you have special options on the command line you can use the eslint.options VS Code setting to let ESLint know about this. Can you give that a try?

Aha! That's why - somehow that configuration was in my eslint.options. Maybe I added it in there a few years ago? I removed it and everything's working again. Thanks!


OLS-641: Azure OpenAI-specific config

Description

Azure OpenAI-specific config
more unit tests
updated model-loading code

Type of change

[ ] Refactor
[x] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[ ] Bump-up dependent library

Related Tickets & Documents

Related Issue #OLS-641

Checklist before requesting a review

[x] I have performed a self-review of my code.
[x] PR has passed all pre-merge test jobs.
[ ] If it is a core feature, I have added thorough tests.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.09%. Comparing base (0867837) to head (2dca6a6).
Report is 33 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #964      +/-   ##
==========================================
- Coverage   95.17%   95.09%   -0.08%
==========================================
  Files          54       54
  Lines        2030     2122      +92
==========================================
+ Hits         1932     2018      +86
- Misses         98      104       +6

Files
Coverage Δ

ols/app/models/config.py
95.70% <100.00%> (-0.25%)
:arrow_down:

... and 1 file with indirect coverage changes

/lgtm

/approve


small fix to create the hab group, this will make the hab_install res…

…ource compatible with SLES 12
Signed-off-by: devoptimist<EMAIL_ADDRESS>Description
full context https://github.com/chef-cookbooks/habitat/issues/107
This change creates the hab group in the install resource. before the hab user is created.
This is done because not all Linux OS's use the default behaviour of the useradd command.
Issues Resolved
https://github.com/chef-cookbooks/habitat/issues/107
Check List

[ ] All tests pass. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/TESTING.MD
[ ] New functionality includes testing.
[ ] New functionality has been documented in the README if applicable
[ ] All commits have been signed for the Developer Certificate of Origin. See https://github.com/chef-cookbooks/community_cookbook_documentation/blob/master/CONTRIBUTING.MD

Sorry for the late response on the merge. Thanks for the change


Setting fillColor of Checkbox component but color does not change

I am trying set the fillColor prop and the component still using Indigo color.
 <MKCheckbox checked={true} fillColor="#f00" />

I tried setTheme too like the doc suggest but the result was the same.
I think that could be something with TickView widget but its just a guess.

I get this one too, only on iOS.

By the way, i'm using android.

Please update to RN 0.20.0 (the final release https://github.com/facebook/react-native/commit/6496feb61e9205baa3f25cf448c4fce912bd6250).
Android should work, iOS do have the problem, maybe caused by some change in RN 0.20.0

I'm having the same issue on iOS.  I did some quick digging and it seems the settings inside TickView.m are never being called.  This is true both for fillColor and insets.  I can see the conversion happen in RCTConvert for the color value I'm passing in, but then it's never set on the returned view.  I'm thinking this is most likely a bug related to 0.20 as it doesn't seem to happen on another project I have with 0.18.

I've got the same issue (with RN 0.20.0) with IOS, the fill colour seems to be getting ignored.

+1

:+1:

+1

Same issue on RN 0.21, on iOs only also.
I suspect it comes from the re-implementation of processColor but I can't seem to find where the breaking change is....

Removing the parseColor function seems to correct the bug for me (see Pull Request #121)
Anybody knows if it makes it break on older versions of React-Native ?

Still have this issue on v0.21

Hi! Any info on this? It seems really strange :8ball:

+1

+1

Sorry for the delay.
The fix is included in v0.3.1, just pushed to NPM, or you can try the latest build on master branch.


[SPARK-44705][PYTHON] Make PythonRunner single-threaded

What changes were proposed in this pull request?
PythonRunner, a utility that executes Python UDFs in Spark, uses two threads in a producer-consumer model today. This multi-threading model is problematic and confusing as Spark's execution model within a task is commonly understood to be single-threaded.
More importantly, this departure of a double-threaded execution resulted in a series of customer issues involving race conditions and deadlocks between threads as the code was hard to reason about. There have been multiple attempts to reign in these issues, viz., fix 1, fix 2, fix 3. Moreover, the fixes have made the code base somewhat abstruse by introducing multiple daemon monitor threads to detect deadlocks. This PR makes PythonRunner single-threaded making it easier to reason about and improving code health.
Current Execution Model in Spark for Python UDFs
For queries containing Python UDFs, the main Java task thread spins up a new writer thread to pipe data from the child Spark plan into the Python worker evaluating the UDF. The writer thread runs in a tight loop: evaluates the child Spark plan, and feeds the resulting output to the Python worker. The main task thread simultaneously consumes the Python UDF’s output and evaluates the parent Spark plan to produce the final result.
The I/O to/from the Python worker uses blocking Java Sockets necessitating the use of two threads, one responsible for input to the Python worker and the other for output. Without two threads, it is easy to run into a deadlock. For example, the task can block forever waiting for the output from the Python worker. The output will never arrive until the input is supplied to the Python worker, which is not possible as the task thread is blocked while waiting on output.
Proposed Fix
The proposed fix is to move to the standard single-threaded execution model within a task, i.e., to do away with the writer thread. In addition to mitigating the crashes, the fix reduces the complexity of the existing code by doing away with many safety checks in place to track deadlocks in the double-threaded execution model.
In the new model, the main task thread alternates between consuming/feeding data to the Python worker using asynchronous I/O through Java’s SocketChannel. See the read() method in the code below for approximately how this is achieved.
case class PythonUDFRunner {

  private var nextRow: Row = _
  private var endOfStream = false
  private var childHasNext = true
  private var buffer: ByteBuffer = _

  def hasNext(): Boolean = nextRow != null || {
     if (!endOfStream) {
       read(buffer)
       nextRow = deserialize(buffer)
       hasNext
     } else {
       false
     }
  }

  def next(): Row = {
     if (hasNext) {
       val outputRow = nextRow
       nextRow = null
       outputRow
     } else {
       null
     }
  }

  def read(buf: Array[Byte]): Row = {
    var n = 0
    while (n == 0) {
    // Alternate between reading/writing to the Python worker using async I/O
    if (pythonWorker.isReadable) {
      n = pythonWorker.read(buf)
    }
    if (pythonWorker.isWritable) {
      consumeChildPlanAndWriteDataToPythonWorker()
    }
  }

  def consumeChildPlanAndWriteDataToPythonWorker(): Unit = {
      // Tracks whether the connection to the Python worker can be written to.
      var socketAcceptsInput = true
      while (socketAcceptsInput && (childHasNext || buffer.hasRemaining)) {
        if (!buffer.hasRemaining && childHasNext) {
          // Consume data from the child and buffer it.
          writeToBuffer(childPlan.next(), buffer)
          childHasNext = childPlan.hasNext()
          if (!childHasNext) {
            // Exhausted child plan’s output. Write a keyword to the Python worker signaling the end of data input.
            writeToBuffer(endOfStream)
          }
        }
        // Try to write as much buffered data as possible to the Python worker.
        while (buffer.hasRemaining && socketAcceptsInput) {
          val n = writeToPythonWorker(buffer)
          // `writeToPythonWorker()` returns 0 when the socket cannot accept more data right now.
          socketAcceptsInput = n > 0
        }
      }
    }
}

Why are the changes needed?
This PR makes PythonRunner single-threaded making it easier to reason about and improving code health.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Existing tests.

cc @HyukjinKwon

Merged to master.

This PR caused the failure of the Scala 2.13 mima check. https://github.com/apache/spark/pull/42479

@utkarsh39
I found that this PR may caused some PySpark test cases to fail in the Java 17 daily tests(pyspark-sql and pyspark-connect module)：

https://github.com/apache/spark/actions/runs/5837423492
https://github.com/apache/spark/actions/runs/5843658110
https://github.com/apache/spark/actions/runs/5849761680

To verify this , I conducted some local testing using Java 17
java -version
openjdk version "17.0.8" 2023-07-18 LTS
OpenJDK Runtime Environment Zulu17.44+15-CA (build 17.0.8+7-LTS)
OpenJDK 64-Bit Server VM Zulu17.44+15-CA (build 17.0.8+7-LTS, mixed mode, sharing)

Revert to the previous PR before SPARK-44705 and run the following commands:

// [SPARK-44765][CONNECT] Simplify retries of ReleaseExecute
git reset --hard 9bde882fcb39e9fedced0df9702df2a36c1a84e6
export SKIP_UNIDOC=true
export SKIP_MIMA=true
export SKIP_PACKAGING=true
./dev/run-tests --parallelism 1 --modules "pyspark-sql"

Finished test(python3.9): pyspark.sql.tests.test_udtf (57s) ... 2 tests were skipped
Tests passed in 59 seconds

The tests in pyspark.sql.tests.test_udtf passed.

Revert to SPARK-44705 and run the following commands:

// [SPARK-44705][PYTHON] Make PythonRunner single-threaded
git reset --hard 8aaff55839493e80e3ce376f928c04aa8f31d18c
export SKIP_UNIDOC=true
export SKIP_MIMA=true
export SKIP_PACKAGING=true
./dev/run-tests --parallelism 1 --modules "pyspark-sql"

======================================================================
FAIL: test_udtf_with_analyze_table_argument_adding_columns (pyspark.sql.tests.test_udtf.UDTFTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/sql/tests/test_udtf.py", line 1340, in test_udtf_with_analyze_table_argument_adding_columns
    assertSchemaEqual(
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/testing/utils.py", line 356, in assertSchemaEqual
    raise PySparkAssertionError(
pyspark.errors.exceptions.base.PySparkAssertionError: [DIFFERENT_SCHEMA] Schemas do not match.
--- actual
+++ expected
- StructType([StructField('a', LongType(), True)])
+ StructType([StructField('id', LongType(), False), StructField('is_even', BooleanType(), True)])

======================================================================
FAIL: test_udtf_with_analyze_table_argument_repeating_rows (pyspark.sql.tests.test_udtf.UDTFTests) (query_no=0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/sql/tests/test_udtf.py", line 1394, in test_udtf_with_analyze_table_argument_repeating_rows
    assertSchemaEqual(df.schema, expected_schema)
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/testing/utils.py", line 356, in assertSchemaEqual
    raise PySparkAssertionError(
pyspark.errors.exceptions.base.PySparkAssertionError: [DIFFERENT_SCHEMA] Schemas do not match.
--- actual
+++ expected
- StructType([StructField('id', LongType(), False), StructField('is_even', BooleanType(), True)])
+ StructType([StructField('id', LongType(), False)])

======================================================================
FAIL: test_udtf_with_analyze_table_argument_repeating_rows (pyspark.sql.tests.test_udtf.UDTFTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/sql/tests/test_udtf.py", line 1400, in test_udtf_with_analyze_table_argument_repeating_rows
    self.spark.sql(
AssertionError: AnalysisException not raised

======================================================================
FAIL: test_udtf_with_analyze_using_accumulator (pyspark.sql.tests.test_udtf.UDTFTests) (query_no=0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/sql/tests/test_udtf.py", line 1625, in test_udtf_with_analyze_using_accumulator
    assertSchemaEqual(df.schema, StructType().add("col1", IntegerType()))
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/testing/utils.py", line 356, in assertSchemaEqual
    raise PySparkAssertionError(
pyspark.errors.exceptions.base.PySparkAssertionError: [DIFFERENT_SCHEMA] Schemas do not match.
--- actual
+++ expected
- StructType([StructField('a', IntegerType(), True), StructField('b', IntegerType(), True)])
+ StructType([StructField('col1', IntegerType(), True)])

======================================================================
FAIL: test_udtf_with_analyze_using_accumulator (pyspark.sql.tests.test_udtf.UDTFTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/sql/tests/test_udtf.py", line 1628, in test_udtf_with_analyze_using_accumulator
    self.assertEqual(test_accum.value, 222)
AssertionError: 111 != 222

----------------------------------------------------------------------
Ran 174 tests in 54.619s

FAILED (failures=34, errors=6, skipped=2)

There are 34 test failures after this one merged.
@utkarsh39 Do you have time to fix these test cases?  For this, I have created SPARK-44797.
Or should we revert this PR to restore the Java 17 daily tests first? @HyukjinKwon @ueshin @dongjoon-hyun

@utkarsh39
I found that this PR may caused some PySpark test cases to fail in the Java 17 daily tests(pyspark-sql and pyspark-connect module)：

https://github.com/apache/spark/actions/runs/5837423492
https://github.com/apache/spark/actions/runs/5843658110
https://github.com/apache/spark/actions/runs/5849761680

To verify this , I conducted some local testing using Java 17
java -version
openjdk version "17.0.8" 2023-07-18 LTS
OpenJDK Runtime Environment Zulu17.44+15-CA (build 17.0.8+7-LTS)
OpenJDK 64-Bit Server VM Zulu17.44+15-CA (build 17.0.8+7-LTS, mixed mode, sharing)

Revert to the previous PR before SPARK-44705 and run the following commands:

// [SPARK-44765][CONNECT] Simplify retries of ReleaseExecute
git reset --hard 9bde882fcb39e9fedced0df9702df2a36c1a84e6
export SKIP_UNIDOC=true
export SKIP_MIMA=true
export SKIP_PACKAGING=true
./dev/run-tests --parallelism 1 --modules "pyspark-sql"

Finished test(python3.9): pyspark.sql.tests.test_udtf (57s) ... 2 tests were skipped

The tests in pyspark.sql.tests.test_udtf passed.

Revert to SPARK-44705 and run the following commands:

// [SPARK-44705][PYTHON] Make PythonRunner single-threaded
git reset --hard 8aaff55839493e80e3ce376f928c04aa8f31d18c
export SKIP_UNIDOC=true
export SKIP_MIMA=true
export SKIP_PACKAGING=true
./dev/run-tests --parallelism 1 --modules "pyspark-sql"

======================================================================
FAIL: test_udtf_with_analyze_table_argument_adding_columns (pyspark.sql.tests.test_udtf.UDTFTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/sql/tests/test_udtf.py", line 1340, in test_udtf_with_analyze_table_argument_adding_columns
    assertSchemaEqual(
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/testing/utils.py", line 356, in assertSchemaEqual
    raise PySparkAssertionError(
pyspark.errors.exceptions.base.PySparkAssertionError: [DIFFERENT_SCHEMA] Schemas do not match.
--- actual
+++ expected
- StructType([StructField('a', LongType(), True)])
+ StructType([StructField('id', LongType(), False), StructField('is_even', BooleanType(), True)])

======================================================================
FAIL: test_udtf_with_analyze_table_argument_repeating_rows (pyspark.sql.tests.test_udtf.UDTFTests) (query_no=0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/sql/tests/test_udtf.py", line 1394, in test_udtf_with_analyze_table_argument_repeating_rows
    assertSchemaEqual(df.schema, expected_schema)
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/testing/utils.py", line 356, in assertSchemaEqual
    raise PySparkAssertionError(
pyspark.errors.exceptions.base.PySparkAssertionError: [DIFFERENT_SCHEMA] Schemas do not match.
--- actual
+++ expected
- StructType([StructField('id', LongType(), False), StructField('is_even', BooleanType(), True)])
+ StructType([StructField('id', LongType(), False)])

======================================================================
FAIL: test_udtf_with_analyze_table_argument_repeating_rows (pyspark.sql.tests.test_udtf.UDTFTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/sql/tests/test_udtf.py", line 1400, in test_udtf_with_analyze_table_argument_repeating_rows
    self.spark.sql(
AssertionError: AnalysisException not raised

======================================================================
FAIL: test_udtf_with_analyze_using_accumulator (pyspark.sql.tests.test_udtf.UDTFTests) (query_no=0)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/sql/tests/test_udtf.py", line 1625, in test_udtf_with_analyze_using_accumulator
    assertSchemaEqual(df.schema, StructType().add("col1", IntegerType()))
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/testing/utils.py", line 356, in assertSchemaEqual
    raise PySparkAssertionError(
pyspark.errors.exceptions.base.PySparkAssertionError: [DIFFERENT_SCHEMA] Schemas do not match.
--- actual
+++ expected
- StructType([StructField('a', IntegerType(), True), StructField('b', IntegerType(), True)])
+ StructType([StructField('col1', IntegerType(), True)])

======================================================================
FAIL: test_udtf_with_analyze_using_accumulator (pyspark.sql.tests.test_udtf.UDTFTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/yangjie01/SourceCode/git/spark-mine-sbt/python/pyspark/sql/tests/test_udtf.py", line 1628, in test_udtf_with_analyze_using_accumulator
    self.assertEqual(test_accum.value, 222)
AssertionError: 111 != 222

----------------------------------------------------------------------
Ran 174 tests in 54.619s

FAILED (failures=34, errors=6, skipped=2)

There are 34 test failures after this one merged.
@utkarsh39 Do you have time to fix these test cases? For this, I have created SPARK-44797.
Or should we revert this PR to restore the Java 17 daily tests first? @HyukjinKwon @ueshin @dongjoon-hyun

I will try to get these tests fixed ASAP

I think #42422 includes the fix. Could you take a look?

I merged #42422. Let's see the next daily tests. Thanks.

I merged #42422. Let's see the next daily tests. Thanks.

Thanks ~

I merged #42422. Let's see the next daily tests. Thanks.

https://github.com/apache/spark/actions/runs/5861115482/job/15890643041

Still some failed, can't determine the reason for now, further investigation is needed.

@LuciferYang The error is different from the previous one that seems to be fixed.
Is it possible to rerun the test? I guess it's just flaky.

@LuciferYang The error is different from the previous one that seems to be fixed. Is it possible to rerun the test? I guess it's just flaky.

re-run the failed ones, there are 26 [Errno 111] Connection refused, not sure this is flaky

@LuciferYang The error is different from the previous one that seems to be fixed. Is it possible to rerun the test? I guess it's just flaky.

The test passed after retrying, thanks for your work ~ @ueshin


chore: updating msal-node axios dependency due to security risk

Resolving possible security issue for Axios dependency on msal-node.
Issue (component governance):
Exposure of Sensitive Information to an Unauthorized Actor in GitHub repository axios/axios prior to 0.26.
Solution: update Axios dependency in msal-node package to stable version.

We will moving off of axios completely in #4744

And for future reference, please note we use npm, not yarn, for this repository.

Thanks for letting me know!
Karin Blanford | Software Engineer
From: Jason Nutter @.>
Date: Wednesday, May 4, 2022 at 12:38 PM
To: AzureAD/microsoft-authentication-library-for-js @.>
Cc: Karin Blanford @.>, Author @.>
Subject: Re: [AzureAD/microsoft-authentication-library-for-js] chore: updating msal-node axios dependency due to security risk (PR #4785)
And for future reference, please note we use npm, not yarn, for this repository.
—
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzureAD%2Fmicrosoft-authentication-library-for-js%2Fpull%2F4785%23issuecomment-1117763543&data=05|01|Karin.Blanford%40microsoft.com|7c9ef345ab2745d1ee0b08da2e05a960|72f988bf86f141af91ab2d7cd011db47|1|0|637872899112496213|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=o4prBSFly%2BKA%2FTjGIMqgB38QHrVzx%2BwZ2DTNVgTzw1w%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPYS6KRTCCJBQBCFQRPVP5LVILG3HANCNFSM5VC53H3A&data=05|01|Karin.Blanford%40microsoft.com|7c9ef345ab2745d1ee0b08da2e05a960|72f988bf86f141af91ab2d7cd011db47|1|0|637872899112496213|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=9adasgoykDnPnPFeRhLYmik%2BQcXWL5KdcUQLtW%2BBoE4%3D&reserved=0.
You are receiving this because you authored the thread.Message ID: @.***>


About Ordering module

I see there is a new Ordering module that defines the return type for ordering functions. I'm wondering if we would get more flexibility by defining Ordering.t as the comparison function (not just the return value) similar to the code below and as seen in https://github.com/gcanti/fp-ts/blob/master/src/Ord.ts
type t<'a> = ('a, 'a) => int // (or float)

let eval = (t, x, y) => t(x, y)
let map = (f, t, x, y) => eval(t, f(x), f(y))
let fromFloat = (cmp, x, y) => cmp(x, y)->Belt.Int.fromFloat
let reverse = (t, x, y) => eval(t, y, x)

let eq = (t, x, y) => eval(t, x, y) === 0
let neq = (t, x, y) => eval(t, x, y) !== 0

let lt = (t, x, y) => eval(t, x, y) < 0
let lte = (t, x, y) => eval(t, x, y) <= 0
let gt = (t, x, y) => eval(t, x, y) > 0
let gte = (t, x, y) => eval(t, x, y) >= 0

let min = (t, x, y) => lte(t, x, y) ? x : y
let max = (t, x, y) => lte(t, x, y) ? y : x

let clamp = (t, low, high) => v => min(t, high, max(t, low, v))

The primary consumers of these comparison functions are (1) Array.sort, and (2) constructing a Belt Set and Map. Here's how you could use it. I think this approach makes it easier to sort/compare objects by specific properties because it is easier to construct a comparison function via map. Once you have a Ordering.t function, it becomes trivial to modify it with reverse or use it with pairs of items with min, max, gte, neq. The Belt data structures need an int returning function and we can provide a converter in this module so functions that return a float can be used there as well.
module Person = {
  type person = {
    name: string,
    age: int,
  }

  let getAge = i => i.age
}

// Ordering.t is a function
let youngestFirst = Ordering.map(Person.getAge, Int32.compare)
let oldestFirst = Ordering.map(Person.getAge, Int32.compare)->Ordering.reverse
let sortPeopleOldestFirst = people => people->Js.Array2.sortInPlaceWith(oldestFirst)
let oldestOfTwo = (a, b) => Ordering.max(oldestFirst, a, b)

// Other way
let youngestFirst2 = (a, b) => Int32.compare(a->Person.getAge, b->Person.getAge)
let sortPeopleYoungestFirst = people => people->Js.Array2.sortInPlaceWith(youngestFirst2)
let youngestOfTwo = (a, b) => Int32.compare(a->Person.getAge, b->Person.getAge) <= 0 ? a : b

Summary of benefits of this approach

We provide various built-in comparison functions like String.compare, Date.compare, and Float.compare. But the moment you want to sort in reverse, you need to write your own function. It is simpler to do something like String.compare->Ordering.reverse to tweak/modify the existing function. This gives you a function you can easily plug into Array.sort and other places.
There are libraries that require a comparison function that returns an int, like when building a Belt Set or Map. It is easier to tweak/modify the existing function to make it work in these contexts; just call Date.compare->Ordering.toInt
It is extremely common to want to sort an object based on a projection/key like in my example above for sorting people by age. It is convenient to call a function like Ordering.map to construct a new comparison function. This is supported by the TypeScript fp-ts library and is similar to the Rust maxByKey function - https://doc.rust-lang.org/std/cmp/trait.Ord.html.
Once you have a comparison function, there are many useful things to do with it that are not supported by the current Ordering module. Things like min, max, clamp, gt, gte, eq, etc.

See https://doc.rust-lang.org/std/cmp/index.html for the ordering module in Rust
See docs for the TypeScript version in fp-ts
https://gcanti.github.io/fp-ts/modules/Ord.ts.html

@zth what do you think?

Will add my thoughts soon. Meanwhile, pinging in @cknitt and @glennsl who I think both has a better idea of the Ordering module than me.

The proposal here doesn't seem meaningfully different from the last time this was discussed (#77), and my opinion hasn't really changed either. To briefly reiterate, I think the small bit of convenience it adds is dwarfed by the additional complexity and high barrier to learning,. It's better suited for an opinionated third-party library.

Nothing is stopping programmers from avoiding the “complexity” of my proposal and writing their own comparison functions. I’m just saying that if we’re going to have an Ordering module we might as well make it a bit more useful. Otherwise we’re guaranteed that people who want to use Belt Set or Map, or want to sort in reverse, or sort an object by key, etc. MUST write their own comparison function. What is so complex about my proposal here? Where is the "high barrier to learning"? I'm an ok programmer and am pretty sure I could easily grok all this, especially if there was a single example in the API help. Not only would I grok it, I'd be thankful to the ReScript and Core team for making it so much easier to do various forms of sorting, max, min, etc.
I think your Ordering module might make things more confusing for Javascript programmers. Before if they wanted to sort an array they knew they needed to provide a function that returns a number, with the sign indicating relative order. Now they see a function that returns an Ordering.t and have to inspect what that is. You wrote somewhere that the idiomatic compare function is subtraction and this will certainly NOT yield a value like Ordering.less. So now I wonder whether I can use subtraction at all. There is no API documentation help. Is eliminating magic numbers like -1.0 enough of a benefit to justify a whole new module?
You have a strong opinion about what should go in Core. I think we should ALSO give serious consideration to the smart teams of people who built Rust, fp-ts and others. My proposal was never seriously “discussed”. There were a handful of comments in a PR from a very small number of people. This is what a thoughtful discussion looks like for the C# language: https://github.com/dotnet/csharplang/blob/main/meetings/2022/LDM-2022-01-12.md I have raised important issues and opportunities that deserve more than a quick "too opinionated" dismissal. You have declined to address ANY of the specific concerns and ignored ALL the proposed benefits. Your primary objection seemed to be that it wouldn't get used that much and would be an extra thing for people to "wade through". But now that you've created such a module I think it is fair and productive to question the scope of that module.
I assume some of the push back is something like "Rescript is just a better Typescript". But it is also, and based on its ancestry, a functional programming language. From this perspective, an Ordering module that converts functions into other functions fits perfectly well, which is why you see this kind of thing in Haskell and fp-ts. Another argument is that Core is just a 1-1 wrapper around JavaScript. If this is true we should remove Result, remove Option and use Nullable instead, make Array.push return an int, etc. I prefer to embrace the fact that ReScript and Core provides a practical balance between lightweight functional programming concepts and JavaScript and figure out how to elegantly combine them.

I agree with @glennsl. I do appreciate the effort to simplify, but I'm not confident it'll achieve its goal. Therefore I also think this fits better in an opinionated 3rd party lib for starters. We can revisit when that lib has been used for a while and there's a clear view of what the users of that lib thinks of it, whether it improves the situation or not.
I feel like we've discussed the rest at length in other PRs and issues. This includes us not having the same view of what to consider for Core, how functional it should be, and so on.
I think continuing adding these types of things to rescript-extras is a good approach. That way you can shape your "Core extension" just as you'd like, and you're free to incorporate any opinionated FP direction you'd like, using fp-ts and all of the other projects you've mentioned.


Incorrect expires_at value, with Square authorization

When I send this request:

POST /oauth2/token
by doing
client.class #> OAuth2::Client
client.auth_code.get_token(params[:code], token_params || {})
I get
expires_at=2018
expires_at.class #> Integer
So it is incorrect datatime value.
Processing Square response data is going in initialize method of OAuth2::AccessToken class.
Especially in row 50:
@expires_at &&= @expires_at.to_i
Square response contain expires_at in STRING format, so when doing:
"2018-09-08T09:25:17Z".to_i     #> 2018
U will get some integer value which is can not to be converted to valid time data

link for Square docs https://docs.connect.squareup.com/api/oauth#post-token.
Just check is expires_at string convert it in Time class and then in Integer.

Sorry, but I am not able to test last master version. But as I see that row of code what I talk about is not changed. So, pleaseeeeeeee include code for this case (it is easy, just one if) into your nearest release. I hope it will happen before my product release.

If you could write a failing spec it would help us address the issue.  Please submit a PR.  Without that it is unlikely to be released on any specific timetable.

Hey, I want to make push but do not have access rights. Can you fix it?

You should fork this repo and make the patch there, and then submit a pull request.  Looking forward to it!

Hello @pboling, I have done fork and submit a pull request. But forgot to set you as reviever and it is impossible to edit this, also I did not find how to remove pull request.
So here is the link: https://github.com/oauth-xx/oauth2/pull/408


Nested list setting

Closes #50.

I've renamed NestedSetting to NestedDictSetting and left NestedSetting as an alias.
I've created NestedListSetting.

There are a few things that should be addressed before merging this:

Should we deprecate NestedSetting? I could modify the __init__ method to emit a deprecation warning.
I've only created NestedListSetting operating with lists. I could also create separate NestedIterableSetting and NestedTupleSetting. However, @ziima proposes to get rid of these types altogether and just transform any iterable setting to tuple in order to prevent any in-place manipulation. I'm not sure about this, but I'd agree that one type of nested iterables (and most likely even unnested ones) should be enough. In that case, NestedListSetting probably shouldn't check the type of the setting to be list.
The NestedListSetting can't envelop NestedDictSetting right now. I'd say that it's mostly NestedDictSetting's fault. Other than that, NestedListSetting can envelop any setting type, even another NestedListSetting. I've documented this behavior. Are we content with this situation for the time being, or should I try to modify NestedDictSetting in a way that it can be inserted into NestedListSetting?

There is at least one Django setting, that could be represented by NestedDictSetting inside NestedListSetting – TEMPLATES (see docs). But, maybe we could implement this in another pull request, because it will require non-trivial reconstruction of NestedDictSetting.

Yes, let's keep it simple for now. If you or ziima really need nested dicts in nested lists we can add support later.
OK for the deprecation warning in NestedSetting.
About the other iterables, indeed it does not seem necessary to check the type of the setting (list, tuple, etc.). It's true it would be better to also make it immutable, but let's just hope for now that users won't try to modify the setting outside of settings and tests.

Oh, and thank you very much for this PR!

I'll continue with the updates on Monday. Have a nice weekend :wink:.

I think it's good for me. Thank you very much @stinovlas for this PR, and thank you @ziima for the code review! I'll merge once you have resolved the conversations.

Thank you, it was a pleasure =o). Would you consider releasing a new version to PyPI?

Of course, I'm currently pulling/checking/testing and then I'll release a new version :slightly_smiling_face:

Done! :tada:


How to groovy script with JFXComboBox

if jfxcombobox editable false only selection gender available male and female how to using groovy script file

I am sorry but I cannot understand the question. Could you please give more details about what you need to do?


Clean Data Overview Workflow

Well done Josh!!
Fonti's contribution:

Wrote a function for plotting barplot of species by family and used map for this
Moved collect() calls to the start
Removed unnecessary filter(!is.na(species)) calls.

@JoshNitschke can you take a look at this then we can merge into master.
This is more for your benefit of seeing what things are changed and the style of coding etc.
Let me know if you have any questions!


Testing Build: Error: Activating extension 'meltylabs.melty' failed

Error: Activating extension 'meltylabs.melty' failed: Cannot find module '/workspaces/melty/extensions/spectacular/out/extension.js' Require stack: - /workspaces/melty/out/vs/loader.js - /workspaces/melty/out/bootstrap-amd.js - /workspaces/melty/out/bootstrap-fork.js.
Console logs:
[16073:0910/200307.076212:INFO:CONSOLE(29)] "Error: Activating extension 'meltylabs.melty' failed: Cannot find module '/workspaces/melty/extensions/spectacular/out/extension.js'
Require stack:

/workspaces/melty/out/vs/loader.js
/workspaces/melty/out/bootstrap-amd.js
/workspaces/melty/out/bootstrap-fork.js.", source: vscode-file://vscode-app/workspaces/melty/out/vs/workbench/browser/parts/notifications/notificationsAlerts.js (29)

Reproduction:
run ./scripts/code.sh and wait for the window to load.
Attached screenshot:

Did you build the extension first? See contributing.md for local setup.. seems like you didn't run melty:extension-dev nom script

Ok I will try that.


SEC-3.1: Authentication : Adding hostname configuration and encoding …

SEC-3.1: Authentication :

Adding hostname configuration
motd-banner configuration required encoding type for JSON_IETF Added encoding type JSON_IETF for gnmi GET

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 57.219%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1181

Relevant Lines:
2064

💛  - Coveralls

Please rebase this PR forward to pickup the latests changes to the test, including the removal of the ssh_ip flag.

I did a rebase myself and I am get unexpected error for user "alice": rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp <IP_ADDRESS>:6030: connect: no route to host" for every gNMI call that uses the modified context.

I did a rebase myself and I am get unexpected error for user "alice": rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp <IP_ADDRESS>:6030: connect: no route to host" for every gNMI call that uses the modified context.

Please rebase this PR forward to pickup the latests changes to the test, including the removal of the ssh_ip flag.

Rebase is done and also updated the code changes to configure hostname alone, gnmi.get Encoding: gpb.Encoding_JSON_IETF, is already part of latest production script

I did a rebase myself and I am get unexpected error for user "alice": rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp <IP_ADDRESS>:6030: connect: no route to host" for every gNMI call that uses the modified context.

i am not seeing this issue in my environment.

/gcbrun

I'm getting errors on all gNMI requests:
gnmi.Set unexpected error for user "alice": rpc error: code = Unavailable desc = error on request {
        replace:  {
          path:  {
            elem:  {
              name:  "system"
            }
            elem:  {
              name:  "config"
            }
            elem:  {
              name:  "hostname"
            }
          }
          val:  {
            json_ietf_val:  "\"ondatraDUT\""
          }
        }
        }: JGrpcServer: Error parsing metadata for RPC /gnmi.gNMI/Set
        ```

@avaneesh90, what cptx image ID are you using?


License

Hi, you project looks promising, but it lacks a license, which generally means (quoting GitHub FAQ) that the default copyright laws apply [...]; you retain all rights to your source code and nobody else may reproduce, distribute, or create derivative works from your work, so it's legally hard to actually do anything with your project.
Is this really your intent?
Thanks in advance :)

Hi Morwenn,
Thanks for pointing this out. I've put it under MIT license so you don't need to worry about a civil case from me.
What are you hoping to use ol-react for? Cheers,
Richard

Hi, thanks for taking the time to update the license.
We had a new project at work, and decided at some point to use React and OpenLayers 3 among other libraries, without really knowing how to bind them and your library looked like an interesting base to start working. Well, long story short, since there was no license we didn't use it, but I guess it could help us to solve some problems in a near future.
Anyway, thanks again! :)


Added an event for default rule "Launch Package Management Process In Container"

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind documentation

/kind tests

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area commands

/area pkg

/area events

What this PR does / why we need it:
Added an event for default rule "Launch Package Management Process In Container" as mentioned here
Which issue(s) this PR fixes:

Fixes #178
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: GLVSKiriti
Once this PR has been reviewed and has the lgtm label, please assign fededp for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: ce47eb860d3a0f5bf0bb158faaf9c9e6f904b58f

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, GLVSKiriti
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


feat(TFD-3568): Flatten Kafka datastore to triggers.

What is the current behavior? (You can also link to an open issue here)
Activating a trigger on a sub properties result into a crash of the serialization/deserialization of the JSONSchema API.
What is the new behavior?
Flatten Kafka Properties.
Please check if the PR fulfills these requirements

[X] The commit message follows Talend standard
[X] Tests for the changes have been added (for bug fixes / features)
[X] Docs have been added / updated (for bug fixes / features) ?
[X] The code coverage on new code >75%
[X] The new code does not introduce new technical issues (sonar / eslint)

What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build / CI related changes
[ ] Other... Please describe:

Does this PR introduce a breaking change?

[ ] Yes
[ ] No

If this PR contains a breaking change, please describe the impact and migration path for existing applications: ...
Other information:

Pull request test result: success :star:
Sonar results

To be checked with component-runtime, but if I remember correctly, the future of component properties is going to look a lot flatter than the hierarchical promise of daikon Properties (but still retain many of the advantages).
In the meantime, it's probably not worth investing in a fix in the current components API.  I don't like the idea of widespread copy/pasting like this but it's likely to be addressed when we migrate to 1.0

Checked with component-runtime, the persisted properties are flattened and reshaped into data objects.  The challenge of migration from components is likely to be simplified by flattening objects now.

Chen Bin > This is require only when there is a trigger on your inner-property.
I did not see the PR with the required, I will integrate these changes. Thank you!


Clarification for SingleSignOn -> Scope

Hello,
I'm implementing the "Keep Me Signed In" as is shown in this example https://docs.microsoft.com/en-us/azure/active-directory-b2c/custom-policy-keep-me-signed-in and they use the SingleSignOn Scope = "Tenant". So I used like the example and my application works fine. However, when I was look into the technical details provided in this page, I decided that I don't want my Single Sign On to be applied to my entire Tenant, only for a specific application.
However, when I changed the scope to "Application" I noticed that when I signed In and Signed out I was never prompted to provide the Login credentials anymore, as if I have never signed out.
I reverted back to Tenant and it worked, I changed to "Policy" and it worked. But I don't understand why it wouldn't work for "Tenant".
Is there any technical detail that I might be missing?
Thank you

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1e3ce49f-746e-9326-9846-32d569558b65
Version Independent ID: 4978d164-555d-b3cc-2989-5246b0f44313
Content: RelyingParty - Azure Active Directory B2C
Content Source: articles/active-directory-b2c/relyingparty.md
Service: active-directory
Sub-service: b2c
GitHub Login: @msmimart
Microsoft Alias: mimart

@henriquebelotto
Thanks for the Question! We are currently investigating and will get back on this.

#reassign:JasSuri

Its because with application scoped SSO, you must pass the client_id of the app you want to sign out of, otherwise the logout isnt really complete as you described.
https://docs.microsoft.com/en-us/azure/active-directory-b2c/openid-connect#send-a-sign-out-request

client_id
No*
The application ID that the Azure portal assigned to your application.*This is required when using Application isolation SSO configuration and Require ID Token in logout request is set to No.

@henriquebelotto - Thank you for your feedback, hopefully you were able to resolve the issue or these replies help. I'll go ahead and close this for now, but let us know if you have further feedback on the docs. #please-close


Add standard eventbridge fields to the example payloads

The example payloads for eventbridge events only showed fields that buildkite varies. As raised in https://github.com/buildkite/docs/pull/2935, copy and pasting this into the eventbridge sandbox as-is results in an error message that the sample event is not valid.
Putting the full payload in the examples will make it easier for users to test patterns using the eventbridge sandbox.

yes, it does @gilesgas


Speed support Bluetooth Mesh device dwdz.switch.sw0a01

I am buy a ble-mesh-switch.
Use XiaomiGateway3, must open ble&zigbee-date he is show, but is Can not Control!
Use Xiaomi Miot Auto, must open Cloud model he is show and Can to control, but has the state delay!
I think Use XiaomiGateway3, to Control. LAN Control is no delay !!
So.. I want Speed support Bluetooth Mesh device.
This is ble-mesh-switch document.
device : https://home.miot-spec.com/s/4252
control functions: https://home.miot-spec.com/spec?type=urn%3Amiot-spec-v2%3Adevice%3Aswitch%3A0000A003%3Adwdz-sw0a01%3A1%3A0000C808
Expect Support!!!   Thanks!!

2022-02-10 23:44:58  DEBUG    base          <IP_ADDRESS> [MQTT] log/miio b'\x1b[0;32m2022:02:10:23:44:58.134 [D] miio_client_func: ot_agent_recv_handler_one(): fd:9, msg:{"method":"_async.ble_event","params":{"dev":{"did":"blt.3.19clqod2c5c01","mac":"A4:C1:38:B8:0C:35","pdid":1371},"evt":[{"eid":4100,"edata":"4d00"}],"frmCnt":101,"gwts":1644507898},"id":876} length:190 bytes\x1b[0m'
2022-02-10 23:45:00  DEBUG    base          <IP_ADDRESS> [MQTT] log/miio b'\x1b[0;32m2022:02:10:23:45:00.064 [D] miio_client_func: ot_agent_recv_handler_one(): fd:9, msg:{"method":"properties_changed","params":[{"did":"1046242546","siid":2,"piid":1,"value":false}],"id":877} length:104 bytes\x1b[0m'
2022-02-10 23:45:06  DEBUG    base          <IP_ADDRESS> [MQTT] gw/B4E3F9FFFEB70BAE/heartbeat b'{"networkUp":true,"networkPanId":"0x06CF","radioTxPower":7,"radioChannel":15}'
2022-02-10 23:45:32  DEBUG    base          <IP_ADDRESS> [MQTT] log/miio b'\x1b[0;32m2022:02:10:23:45:32.414 [D] miio_client_func: ot_agent_recv_handler_one(): fd:9, msg:{"method":"properties_changed","params":[{"did":"1046242546","siid":2,"piid":1,"value":true}],"id":883} length:103 bytes\x1b[0m'
2022-02-10 23:45:37  DEBUG    base          <IP_ADDRESS> [MQTT] log/miio b'\x1b[0;32m2022:02:10:23:45:37.364 [D] miio_client_func: ot_agent_recv_handler_one(): fd:9, msg:{"method":"properties_changed","params":[{"did":"1046242546","siid":2,"piid":1,"value":false}],"id":886} length:104 bytes\x1b[0m'
2022-02-10 23:45:57  DEBUG    base          <IP_ADDRESS> [MQTT] log/miio b'\x1b[0;32m2022:02:10:23:45:58.274 [D] miio_client_func: ot_agent_recv_handler_one(): fd:9, msg:{"method":"properties_changed","params":[{"did":"1046242546","siid":2,"piid":1,"value":true}],"id":889} length:103 bytes\x1b[0m'
2022-02-10 23:45:59  DEBUG    base          <IP_ADDRESS> [MQTT] log/miio b'\x1b[0;32m2022:02:10:23:45:59.374 [D] miio_client_func: ot_agent_recv_handler_one(): fd:9, msg:{"method":"properties_changed","params":[{"did":"1046242546","siid":2,"piid":1,"value":false}],"id":890} length:104 bytes\x1b[0m'
2022-02-10 23:45:59  DEBUG    base          <IP_ADDRESS> [MQTT] log/miio b'\x1b[0;32m2022:02:10:23:45:59.924 [D] miio_client_func: ot_agent_recv_handler_one(): fd:9, msg:{"method":"properties_changed","params":[{"did":"1046242546","siid":2,"piid":1,"value":true}],"id":891} length:103 bytes\x1b[0m'
2022-02-10 23:46:00  DEBUG    base          <IP_ADDRESS> [MQTT] log/miio b'\x1b[0;32m2022:02:10:23:46:00.024 [D] miio_client_func: ot_agent_recv_handler_one(): fd:9, msg:{"method":"properties_changed","params":[{"did":"1046242546","siid":2,"piid":1,"value":false}],"id":892} length:104 bytes\x1b[0m'
2022-02-10 23:46:06  DEBUG    base          <IP_ADDRESS> [MQTT] gw/B4E3F9FFFEB70BAE/heartbeat b'{"networkUp":true,"networkPanId":"0x06CF","radioTxPower":7,"radioChannel":15}'

The MQTT Logs ( Open & Close Switch )
2022-02-10 23:45:32  DEBUG    base          <IP_ADDRESS> [MQTT] log/miio b'\x1b[0;32m2022:02:10:23:45:32.414 [D] miio_client_func: ot_agent_recv_handler_one(): fd:9, msg:{"method":"properties_changed","params":[{"did":"1046242546","siid":2,"piid":1,"value":true}],"id":883} length:103 bytes\x1b[0m'
2022-02-10 23:45:37  DEBUG    base          <IP_ADDRESS> [MQTT] log/miio b'\x1b[0;32m2022:02:10:23:45:37.364 [D] miio_client_func: ot_agent_recv_handler_one(): fd:9, msg:{"method":"properties_changed","params":[{"did":"1046242546","siid":2,"piid":1,"value":false}],"id":886} length:104 bytes\x1b[0m'

How it display when you using Gateway3?
Got enity?
By the way try not to use translation software like this one.
It's terrible.

How it display when you using Gateway3? Got enity? By the way try not to use translation software like this one. It's terrible.

Not have enity!
But I open Zigbee 和 BLE 效能数据, then he has enity.
This is he info:
enity:
Name：Unknown MESH Mesh
enityID: sensor.ec4d3eb236c7_mesh
device:
Device: Unknown MESH
Mdel：mesh(4252)
Manufacturers：Unknown
He was not identified
（ Sorry.., My English is so bad... ）

This model already supported (4252) in master version. Will be in next release #629


valueNotifier.value doesn't change

Hi @siddhesh-tamhanekar
I have created an event listener like below to capture index change when playing a playlist. But whenever the event listener fires it only throws an initial value it never gives an updated value. Below is my function.
void initAudio() async {

    player = MediaPlayerPlugin.create(isBackground: true, showNotification: true);
    await player.initialize();

    player.valueNotifier.addListener(() {

      this.currentPlaylistIndex = player.valueNotifier.value.currentIndex;
      print("state changed:"+player.valueNotifier.value.duration.toString());

      print("state changed:"+player.valueNotifier.value.currentIndex.toString());

      setState(()
      {

      });
    });

    await player.setPlaylist(Playlist(playlist));
    player.play();

    setState(() {
       audioState = 1;
    });
  }
}

Hi @ajsatam  I have used the media player at multiple applications I didn't come across such error
You can look at the sample application https://github.com/siddhesh-tamhanekar/podcastApp
which demonstrates building duration widgets and playlist etc.
Your code looks okay at first glance but needs to debug about what is going wrong can you share more details such as your device, android version and useful debugging information.
meanwhile, you can try to install the podcast app given above and try if it works fine on the device.
Hopefully, This will help you to solve your problem
All the best.

Hi @siddhesh-tamhanekar I checked the project. Downloaded the apk on One Plus six and One plus 7 pro and it's the same. While the podcast changes but the name of the podcast and selected podcast in the list view below remains the same. It doesn't change.
Below is the device information:
Name: One Plus six, One plus 7 Pro
Android version : 9
OxygenOS version : 9.0.7
Please let me know if you need any more information.

HI @ajsatam  I have checked for android version:9 (pie) on the emulator and it works perfectly maybe there is a problem with OxygenOS
Unfortunately, I didn't have Oneplus device so couldn't able to debug.
will you please provide debug log message which could reveal the information

Also checked on the below device but the same issue
Name : Lenovo K50
Android version : 5.0

@ajsatam  I don't have either 5.0 version device.
You could share logs here to get some clue.

I'm a bit new to Android studio but it's the same which you get in the Console window right? Here it is. Please let me know otherwise.
Lenovo log.zip


Ability to load aliases from pastebin or similar, to make aliases longer than IRC message limits possible

Limited to bot admins, obviously.

Superseded by #100.


Training Code

Good Day..
This is not an issue..
but I'm asking regarding the availability of model training code.
best Regards

Do you get the training code ?


Setup VQE Minimize Backend to persist the AcceleratorBuffer periodically

... in order to be able to restart the calculation if it ends unexpectedly.

For now @zpparks314 is adding this to vqe.py in benchmarks/


Add oceandatr

Insert URLs to the project here:
https://github.com/emlab-ucsb/oceandatr

[x] The projects is active, documented, open source licensed, shows usage from external parties and is directly targeting environmental sustainability. Find more details in the Contribution Guide.

All issues labeled as 'Good First Issue' of the project listed on OpenSustain.tech will be visible on ClimateTriage.com. This is a great way to welcome new community members to your project.

@jflowernet @jaseeverett


nrf52840 can't mount formatting

[00:00:00.003,601]  littlefs: LittleFS version 2.1, disk version 2.0
[00:00:00.003,814]  littlefs: FS at NRF_FLASH_DRV_NAME:0xf8000 is 8 0x1000-byte blocks with 512 cycle
[00:00:00.003,845]  littlefs: sizes: rd 16 ; pr 16 ; ca 64 ; la 32
[00:00:00.003,967]  littlefs: Corrupted dir pair at 0 1
[00:00:00.003,967]  littlefs: can't mount (LFS -84); formatting
[00:00:00.176,269]  littlefs: /lfs mounted
[00:00:00.178,070]  bt_hci_core: HW Platform: Nordic Semiconductor (0x0002)
[00:00:00.178,070]  bt_hci_core: HW Variant: nRF52x (0x0002)
[00:00:00.178,100]  bt_hci_core: Firmware: Standard Bluetooth controller (0x00) Versionwrhello log print hello,56,30.760000
iam getting above error and warning can some help me

I'm not aware of any configurations of MCUboot that use littlefs. What RTOS are you using? Does this message occur each time, or only on the first boot? Given how Littlefs works, I would expect this message on the first boot after flash is cleared.
If it is being corrupted each time, one possibility is that the partition layout of the flash has overlaps, and the littlefs is getting overwritten by something else.


Deprecate text metrics

What does this PR do?

Deprecate RougeMetric and BleuScore in favour of their counterparts introduced in version 0.5 of torchmetrics.
Before submitting

[ ] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
[ ] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure your PR does only one thing, instead of bundling different changes together?
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests? [not needed for typos/docs]
[ ] Did you verify new and existing tests pass locally with your changes?
[ ] If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

[ ] Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃

Hello @SkafteNicki! Thanks for opening this PR.

In the file flash/text/seq2seq/core/metrics.py:

Line 26:26: W291 trailing whitespace

Do see the Hitchhiker's guide to code style


UPDATE readme remove example usage for now

Current coverage is 100.00%

Branch #10 has no coverage reports uploaded yet.

No diff could be generated. No reports for master found.

Powered by Codecov. Updated on successful CI builds.


Typo in README

There is a typo in the selected word in your README.
It should be Software.

By the way what does IMO mean in your ReadMe?

Hi👋! I actually can't believe people actually read my profile.
I'm very glad you caught that typo (mobile editing, y'know? 😅).
"IMO" means "In my opinion" (maybe I should add that in parenthesis)

So it’s fixed! Thanks 😁 for helping me out!


feat(network): dial peer in network run function

Pull Request type
Please check the type of change your PR introduces:

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build-related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

Issue Number: N/A
What is the new behavior?

Does this introduce a breaking change?

[ ] Yes
[ ] No

Other information

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Current dependencies on/for this PR:

#1739  👈
#1738
#1729
#1704
#1670
#1715
main

This stack of pull requests is managed by Graphite.


Implement Adobe Analytics instrumentation

This is currently blocked, waiting for instrumentation details from the Delta team

See discussion in https://adobe-dx-support.slack.com/archives/C04JGUPQAPM/p1680656369861309


changes for password encryption issue

Complete changes for custom directive for deletion confirmation and password encryption.

@tabladrum LGTM