Raspberry Pi High Quality Camera on AstroArch

Hello everyone,
I have successfully installed AstroArch on my Raspberry Pi 5 and everything is working well so far. Now I would like to use the Raspberry Pi High Quality Camera. Unfortunately this is not recognized in Kstars/EKOS. Also the command
libcamera-hello --list-cameras
does not show any camera. Does anyone have an idea how to activate the camera?
Tanks very much, Oliver

hi @Olli781 I am not sure about the problem here, I don't have a pi camera myself but I am willing to help you troubleshoot.
The package on astroarch is pretty old and I would start from compiling the latest version, after that you can upgrade and we can check if it was just that, if not we will check further.
In the meanwhile, can you report what v4l2-ctl --list-devices command says?

Hello Mattia,
First of all, thank you very much for your help. As suggested, I updated all packages. Unfortunately the camera is still not working. Attached is the result of the command:
v4l2-ctl --list-devices
astronaut@astroarch
pispbe (platform:1000880000.pisp_be):
/dev/video20
/dev/video21
/dev/video22
/dev/video23
/dev/video24
/dev/video25
/dev/video26
/dev/video27
/dev/video28
/dev/video29
/dev/video30
/dev/video31
/dev/video32
/dev/video33
/dev/video34
/dev/video35
/dev/video36
/dev/video37
/dev/media1
/dev/media2
rp1-cfe (platform:1f00128000.csi):
/dev/video0
/dev/video1
/dev/video2
/dev/video3
/dev/video4
/dev/video5
/dev/video6
/dev/video7
/dev/media0
It seems that the camera is recognized by Raspberry PI on the CSI port. Unfortunately, the "libcamera-hello" command doesn't work either.
Attached is an excerpt from my config.txt. I have added the passage in bold:

#For more options and information see:
#http://rptl.io/configtxt
initramfs initramfs-linux.img followkernel
#Uncomment some or all of these to enable the optional hardware interfaces
#dtparam=i2c_arm=on
#dtparam=i2s=on
#dtparam=spi=on
#Additional overlays and parameters are documented
#/boot/firmware/overlays/README
#Automatically load overlays for detected cameras
camera_auto_detect=1
#Automatically load overlays for detected DSI displays
display_auto_detect=1
#Enable DRM VC4 V3D driver
dtoverlay=vc4-fkms-v3d
max_framebuffers=2
framebuffer_depth=24
#Uncomment if hdmi display is not detected and composite is being output
#hdmi_force_hotplug=1
#Uncomment if you want to disable wifi or bluetooth respectively
#dtoverlay=disable-wifi
#dtoverlay=disable-bt
#Uncomment this to enable infrared communication.
#dtoverlay=gpio-ir,gpio_pin=17
#dtoverlay=gpio-ir-tx,gpio_pin=18
#Run as fast as firmware / board allows
arm_boost=1
[cm4]
#Enable host mode on the 2711 built-in XHCI USB controller.
#This line should be removed if the legacy DWC2 controller is required
#(e.g. for USB device mode) or if USB support is not required.
otg_mode=1
[all]
dtparam=i2c_arm=on
dtparam=audio=on
disable_overscan=1
gpu_mem=256
start_file=start_x.elf
fixup_file=fixup_x.dat
initramfs initramfs-linux.img followkernel
cma_lwm= cma_hwm= cma_offline_start=
disable_splash=1
3dtparam=krnbt=on
hdmi_drive=2
dtoverlay=i2c-rtc
[pi5]
dtparam=rtc_bbat_vchg=3000000

Best regards, Oliver

hi again @Olli781
I cracked the compilation and it seems my trick works, can you try to run the following commands and report back if the camera is recognized? Please reboot the pi after before calling the camera
wget http://astromatto.com:9000/aarch64/libcamera-0.2.0-2-aarch64.pkg.tar.xz
wget http://astromatto.com:9000/aarch64/libcamera-docs-0.2.0-2-aarch64.pkg.tar.xz
wget http://astromatto.com:9000/aarch64/libcamera-ipa-0.2.0-2-aarch64.pkg.tar.xz
wget http://astromatto.com:9000/aarch64/libcamera-tools-0.2.0-2-aarch64.pkg.tar.xz
wget http://astromatto.com:9000/aarch64/rpicam-apps-1.4.3-1-aarch64.pkg.tar.xz
wget http://astromatto.com:9000/aarch64/gst-plugin-libcamera-0.2.0-2-aarch64.pkg.tar.xz
sudo pacman -U gst-plugin-libcamera-0.2.0-2-aarch64.pkg.tar.xz libcamera-0.2.0-2-aarch64.pkg.tar.xz libcamera-docs-0.2.0-2-aarch64.pkg.tar.xz libcamera-ipa-0.2.0-2-aarch64.pkg.tar.xz libcamera-tools-0.2.0-2-aarch64.pkg.tar.xz rpicam-apps-1.4.3-1-aarch64.pkg.tar.xz

Please note the the package rpicams-apps provides both libcamera-* and rpicam-* as valid binaries

Oh, seems there are some duplicated files around, please as a quick test remove phd2 and retry that command, to remove it sudo pacman -R phd2 if you don't use it at all you can live without it.
Will try to fix the libcamera packages to avoid conflicts in the next days
Let me know if that works @Olli781

Hello Matt, we are a small step further. The libcamera-hello command works, but a library is still missing:
~ » libcamera-hello
libcamera-hello: error while loading shared libraries: libpisp.so.1: cannot open shared object file: No such file or directory
Any idea how you can reinstall this?

@Olli781 forgive me, I installed that yesterday but forgot to add it to the commands, to install it
sudo pacman -Sy && sudo pacman -S libpisp

Hi Matt, first of all I have to apologize for the late reply. The command "libcamera-hello" works:
Mode selection for 1640:1232:12:P
SRGGB10_CSI2P,640x480/0 - Score: 4504.81
SRGGB10_CSI2P,1640x1232/0 - Score: 1000
SRGGB10_CSI2P,1920x1080/0 - Score: 1541.48
SRGGB10_CSI2P,3280x2464/0 - Score: 1718
SRGGB8,640x480/0 - Score: 5504.81
SRGGB8,1640x1232/0 - Score: 2000
SRGGB8,1920x1080/0 - Score: 2541.48
SRGGB8,3280x2464/0 - Score: 2718
Stream configuration adjusted
[0:12:24.939976720] [1267]  INFO Camera camera.cpp:1183 configuring streams: (0)
1640x1232-YUV420 (1) 1640x1232-BGGR16_PISP_COMP1
[0:12:24.940103590] [1268]  INFO RPI pisp.cpp:1405 Sensor: /base/axi/pcie@120000
/rp1/i2c@80000/imx219@10 - Selected sensor format: 1640x1232-SBGGR10_1X10 - Sele  cted CFE format: 1640x1232-PC1B
dmaHeap allocation failure for rpicam-apps0
ERROR: *** failed to allocate capture buffers for stream ***
Only the preview doesn't work, but that's not a bad thing. The camera is not yet recognized under kstars. I think for this you still have to install the "indi pylibcamera - driver":
_https://github.com/scriptorron/indi_pylibcamera_
Is it possible under astroarch?

Hi @Olli781 great success! Indi pylibcamera is already available, you can try to install and have a go with sudo pacman -S indi_pylibcamera
I am not sure it works, this was compiled a long time ago, if you can test is tho and report back it is not working I will recompile a more recent version tomorrow 🙂

hi @Olli781 if you installed the package indi_pylibcamera I suggest to remove it using sudo pacman -R indi_pylibcamera and install the newest version with sudo pacman -Sy indi-pylibcamera
I recompiled a few packages and I added also the dev rule to allow video users to use the pi allocator (without sudo I couldn't get indi pylibcamera to work). indi pylibcamera seems to work(ish) although I am using a normal camera and it's failing with some weird error, maybe it will work with a pi camera?

Hey Matt,
it partially works. If I use the command "indiserver -v indi_pylibcamera" the indi driver starts:
_2024-03-22T18:59:58: startup: indiserver -v indi_pylibcamera
2024-03-22T18:59:58: Driver indi_pylibcamera: pid=1395 rfd=6 wfd=6 efd=7
2024-03-22T18:59:58: listening to port 7624 on fd 5
2024-03-22T18:59:58: Local server: listening on local domain at: @/tmp/indiserve                                  r
2024-03-22T18:59:59: Driver indi_pylibcamera: [0:03:54.891807630] [1395] ERROR I                      PAModule ipa_module.cpp:172 Symbol ipaModuleInfo not found
2024-03-22T18:59:59: Driver indi_pylibcamera: [0:03:54.891831316] [1395] ERROR I                     PAModule ipa_module.cpp:292 v4l2-compat.so: IPA module has no valid info
2024-03-22T18:59:59: Driver indi_pylibcamera: [0:03:54.891859318] [1395]  INFO C                               amera camera_manager.cpp:284 libcamera v0.2.0
2024-03-22T18:59:59: Driver indi_pylibcamera: [0:03:54.902340682] [1397]  INFO R                       PI pisp.cpp:662 libpisp version v1.0.4 2e1ad45c8f0e-dirty 19-03-2024 (12:49:28)
2024-03-22T18:59:59: Driver indi_pylibcamera: [0:03:54.914271682] [1397]  INFO R                                PI pisp.cpp:1121 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx219@10                      to CFE device /dev/media2 and ISP device /dev/media0 using PiSP variant BCM2712                       C0
2024-03-22T18:59:59: Driver indi_pylibcamera: indi_pylibcamera.indidevice-INFO-                      found cameras: [{'Model': 'imx219', 'Location': 2, 'Rotation': 180, 'Id': '/base                        /axi/pcie@120000/rp1/i2c@80000/imx219@10', 'Num': 0}]
2024-03-22T18:59:59: Driver indi_pylibcamera: indi_pylibcamera.indidevice-INFO-selected logging level: LOGGING_INFO
Unter KSTARS the pylibcamera is also available. But when I chose the pylibcamera, KSTARS crashed.
I'm trying to completely reinstall Astroach and see if it works. We are definitely getting closer to our goal :-)

I believe is the same error I am seeing then. Can you kindly turn on CCD logging and paste me the error?
For me it was python crashing because a prop INDI_RAW doesn't exist

Hi Matt,
I tried another way: I started the Indi driver on the PI5 and then connected to the PI5 via my Windows computer and KSTARS.
A connection is also established there:
2024-03-23T08:51:40: [INFO] configuration /home/astronaut/.indi_pylibcamera/CONFIG1.json does not exist
2024-03-23T08:51:39: [INFO] camera properties: {'Model': 'imx219', 'UnitCellSize': (1120, 1120), 'ColorFilterArrangement': 0, 'Location': 2, 'Rotation': 180, 'PixelArraySize': (3280, 2464), 'PixelArrayActiveAreas': [(8, 8, 3280, 2464)], 'ScalerCropMaximum': (0, 0, 0, 0), 'SystemDevices': (20738, 20740, 20742, 20744, 20746, 20748, 20750, 20741, 20743, 20745, 20747, 20749)}
2024-03-23T08:51:39: [INFO] opening camera
2024-03-23T08:51:39: [INFO] closing camera
2024-03-23T08:51:39: [INFO] connecting to camera /base/axi/pcie@120000/rp1/i2c@80000/imx219@10
But when I want to take a picture, it doesn't work. Here is an excerpt from PUTTY:
_~ » indiserver -v indi_pylibcamera                          astronaut@astroarch
2024-03-23T08:44:17: startup: indiserver -v indi_pylibcamera
2024-03-23T08:44:17: Driver indi_pylibcamera: pid=1182 rfd=6 wfd=6 efd=7
2024-03-23T08:44:17: listening to port 7624 on fd 5
2024-03-23T08:44:17: Local server: listening on local domain at: @/tmp/indiserve                                                    r
2024-03-23T08:44:19: Driver indi_pylibcamera: [0:00:59.297232535] [1182] ERROR I                                                    PAModule ipa_module.cpp:172 Symbol ipaModuleInfo not found
2024-03-23T08:44:19: Driver indi_pylibcamera: [0:00:59.297265554] [1182] ERROR I                                                    PAModule ipa_module.cpp:292 v4l2-compat.so: IPA module has no valid info
2024-03-23T08:44:19: Driver indi_pylibcamera: [0:00:59.297304721] [1182]  INFO C                                                    amera camera_manager.cpp:284 libcamera v0.2.0
2024-03-23T08:44:19: Driver indi_pylibcamera: [0:00:59.314660219] [1184]  INFO R                                                    PI pisp.cpp:662 libpisp version v1.0.4 2e1ad45c8f0e-dirty 19-03-2024 (12:49:28)
2024-03-23T08:44:19: Driver indi_pylibcamera: [0:00:59.343032495] [1184]  INFO R                                                    PI pisp.cpp:1121 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx219@10                                                     to CFE device /dev/media0 and ISP device /dev/media1 using PiSP variant BCM2712                                                    _C0
2024-03-23T08:44:19: Driver indi_pylibcamera: indi_pylibcamera.indidevice-INFO-                                                     found cameras: [{'Model': 'imx219', 'Location': 2, 'Rotation': 180, 'Id': '/base                                                    /axi/pcie@120000/rp1/i2c@80000/imx219@10', 'Num': 0}]
2024-03-23T08:44:19: Driver indi_pylibcamera: indi_pylibcamera.indidevice-INFO-                                                     selected logging level: LOGGING_INFO
2024-03-23T08:44:53: Client 9: new arrival from <IP_ADDRESS>:52396 - welcome!
2024-03-23T08:44:53: Driver indi_pylibcamera: indi_pylibcamera.indidevice-INFO-                                                     connecting to camera /base/axi/pcie@120000/rp1/i2c@80000/imx219@10
2024-03-23T08:44:53: Driver indi_pylibcamera: indi_pylibcamera.indidevice-INFO-                                                     closing camera
2024-03-23T08:44:53: Driver indi_pylibcamera: indi_pylibcamera.indidevice-INFO-                                                     opening camera
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.027540807] [1182] ERROR I                                                    PAModule ipa_module.cpp:172 Symbol ipaModuleInfo not found
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.027561955] [1182] ERROR I                                                    PAModule ipa_module.cpp:292 v4l2-compat.so: IPA module has no valid info
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.027591622] [1182]  INFO C                                                    amera camera_manager.cpp:284 libcamera v0.2.0
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.042728139] [1188]  INFO R                                                    PI pisp.cpp:662 libpisp version v1.0.4 2e1ad45c8f0e-dirty 19-03-2024 (12:49:28)
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.055101805] [1188]  INFO R                                                    PI pisp.cpp:1121 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx219@10                                                     to CFE device /dev/media0 and ISP device /dev/media1 using PiSP variant BCM2712                                                    _C0
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.056552082] [1182] ERROR I                                                    PAModule ipa_module.cpp:172 Symbol ipaModuleInfo not found
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.056572082] [1182] ERROR I                                                    PAModule ipa_module.cpp:292 v4l2-compat.so: IPA module has no valid info
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.056597675] [1182]  INFO C                                                    amera camera_manager.cpp:284 libcamera v0.2.0
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.068840600] [1191]  INFO R                                                    PI pisp.cpp:662 libpisp version v1.0.4 2e1ad45c8f0e-dirty 19-03-2024 (12:49:28)
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.081000969] [1191]  INFO R                                                    PI pisp.cpp:1121 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx219@10                                                     to CFE device /dev/media0 and ISP device /dev/media1 using PiSP variant BCM2712                                                    C0
2024-03-23T08:44:53: Driver indi_pylibcamera: indi_pylibcamera.indidevice-INFO-                                                     camera properties: {'Model': 'imx219', 'UnitCellSize': (1120, 1120), 'ColorFilte                                                    rArrangement': 0, 'Location': 2, 'Rotation': 180, 'PixelArraySize': (3280, 2464)                                                    , 'PixelArrayActiveAreas': [(8, 8, 3280, 2464)], 'ScalerCropMaximum': (0, 0, 0,                                                     0), 'SystemDevices': (20738, 20740, 20742, 20744, 20746, 20748, 20750, 20741, 20                                                    743, 20745, 20747, 20749)}
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.085830636] [1182]  INFO C                                                    amera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 640x480-BGGR                                                    16_PISP_COMP1
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.085949302] [1191]  INFO R                                                    PI pisp.cpp:1405 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx219@10 - Selecte                                                    d sensor format: 640x480-SBGGR10_1X10 - Selected CFE format: 640x480-PC1B
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.090366321] [1182]  INFO C                                                    amera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 1640x1232-BG                                                    GR16_PISP_COMP1
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.090462117] [1191]  INFO R                                                    PI pisp.cpp:1405 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx219@10 - Selecte                                                    d sensor format: 1640x1232-SBGGR10_1X10 - Selected CFE format: 1640x1232-PC1B
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.094644468] [1182]  INFO C                                                    amera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 1920x1080-BG                                                    GR16_PISP_COMP1
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.094747153] [1191]  INFO R                                                    PI pisp.cpp:1405 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx219@10 - Selecte                                                    d sensor format: 1920x1080-SBGGR10_1X10 - Selected CFE format: 1920x1080-PC1B
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.098900190] [1182]  INFO C                                                    amera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 3280x2464-BG                                                    GR16_PISP_COMP1
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.098981135] [1191]  INFO R                                                    PI pisp.cpp:1405 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx219@10 - Selecte                                                    d sensor format: 3280x2464-SBGGR10_1X10 - Selected CFE format: 3280x2464-PC1B
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.106393245] [1182]  INFO C                                                    amera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 640x480-SBGG                                                    R16
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.106512967] [1191]  INFO R                                                    PI pisp.cpp:1405 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx219@10 - Selecte                                                    d sensor format: 640x480-SBGGR8_1X8 - Selected CFE format: 640x480-BYR2
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.111695523] [1182]  INFO C                                                    amera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 1640x1232-SB                                                    GGR16
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.111802467] [1191]  INFO R                                                    PI pisp.cpp:1405 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx219@10 - Selecte                                                    d sensor format: 1640x1232-SBGGR8_1X8 - Selected CFE format: 1640x1232-BYR2
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.117874467] [1182]  INFO C                                                    amera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 1920x1080-SB                                                    GGR16
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.117972763] [1191]  INFO R                                                    PI pisp.cpp:1405 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx219@10 - Selecte                                                    d sensor format: 1920x1080-SBGGR8_1X8 - Selected CFE format: 1920x1080-BYR2
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.124528392] [1182]  INFO C                                                    amera camera.cpp:1183 configuring streams: (0) 640x480-XBGR8888 (1) 3280x2464-SB                                                    GGR16
2024-03-23T08:44:53: Driver indi_pylibcamera: [0:01:33.124620855] [1191]  INFO R                                                    PI pisp.cpp:1405 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx219@10 - Selecte                                                    d sensor format: 3280x2464-SBGGR8_1X8 - Selected CFE format: 3280x2464-BYR2
2024-03-23T08:44:53: Client 10: new arrival from <IP_ADDRESS>:52397 - welcome!
2024-03-23T08:44:53: Driver indi_pylibcamera: indi_pylibcamera.indidevice-INFO-                                                     configuration /home/astronaut/.indi_pylibcamera/CONFIG1.json does not exist
2024-03-23T08:45:48: Driver indi_pylibcamera: indi_pylibcamera.indidevice-INFO-                                                     exposure settings: CameraSettings: FastExposure=False, DoRaw=True, ProcSize=(328                                                    0, 2464), RawMode={'label': '3280x2464 RGGB 10bit', 'size': (3280, 2464), 'true                                                    size': (3280, 2464), 'camera_format': 'SRGGB10', 'bit_depth': 10, 'binning': (1,                                                     1)}, CameraControls={'ExposureTime': 1000000, 'AnalogueGain': 10.66666698455810                                                    5, 'AeEnable': False, 'AeConstraintMode': <AeConstraintModeEnum.Normal: 0>, 'AeE                                                    xposureMode': <AeExposureModeEnum.Normal: 0>, 'AeMeteringMode': <AeMeteringModeE                                                    num.CentreWeighted: 0>, 'AwbEnable': False, 'AwbMode': <AwbModeEnum.Auto: 0>, 'B                                                    rightness': 0.0, 'ColourGains': (2.0, 2.0), 'Contrast': 1.0, 'ExposureValue': 0.                                                    0, 'NoiseReductionMode': <NoiseReductionModeEnum.Off: 0>, 'Saturation': 1.0, 'Sh                                                    arpness': 0.0}
2024-03-23T08:45:48: Driver indi_pylibcamera: indi_pylibcamera.indidevice-INFO-                                                     reconfiguring camera
2024-03-23T08:45:48: Driver indi_pylibcamera: [0:02:28.400229446] [1195]  INFO C                                                    amera camera.cpp:1183 configuring streams: (0) 224x190-BGR888 (1) 3280x2464-SBGG                                                    R16
2024-03-23T08:45:48: Driver indi_pylibcamera: [0:02:28.400415594] [1191]  INFO R                                                    PI pisp.cpp:1405 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx219@10 - Selecte                                                    d sensor format: 3280x2464-SBGGR10_1X10 - Selected CFE format: 3280x2464-BYR2
2024-03-23T08:45:49: Driver indi_pylibcamera: indi_pylibcamera.indidevice-INFO-                                                     got exposed frame
2024-03-23T08:45:55: Driver indi_pylibcamera: WARNING: IERSWarning: malformed IE                                                    RS table from https://datacenter.iers.org/data/9/finals2000A.all: [Errno 2] No s                                                    uch file or directory: '/usr/lib/python3.11/site-packages/astropy_iers_data/data                                                    /ReadMe.finals2000A' [astropy.utils.iers.iers]
2024-03-23T08:46:04: Driver indi_pylibcamera: WARNING: IERSWarning: failed to do                                                    wnload https://maia.usno.navy.mil/ser7/finals2000A.all: <urlopen error [Errno -5                                                    ] No address associated with hostname> [astropy.utils.iers.iers]
2024-03-23T08:46:04: Driver indi_pylibcamera: WARNING: IERSWarning: unable to do                                                    wnload valid IERS file, using local IERS-B [astropy.utils.iers.iers]
2024-03-23T08:46:04: Driver indi_pylibcamera: Exception in thread Thread-2 (__Ex                                                    posureLoop):
2024-03-23T08:46:04: Driver indi_pylibcamera: Traceback (most recent call last):
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/thread                                                    ing.py", line 1045, in _bootstrap_inner
2024-03-23T08:46:04: Driver indi_pylibcamera:     self.run()
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/thread                                                    ing.py", line 982, in run
2024-03-23T08:46:04: Driver indi_pylibcamera:     self._target(*self._args, **se                                                    lf._kwargs)
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/indi_pylibcamera/CameraControl.py", line 886, in __ExposureLoop
2024-03-23T08:46:04: Driver indi_pylibcamera:     hdul = self.createRawFits(arra                                                    y=array, metadata=metadata)
2024-03-23T08:46:04: Driver indi_pylibcamera:            ^^^^^^^^^^^^^^^^^^^^^^^                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/indi_pylibcamera/CameraControl.py", line 572, in createRawFits
2024-03-23T08:46:04: Driver indi_pylibcamera:     **self.snooped_FitsHeader(),
2024-03-23T08:46:04: Driver indi_pylibcamera:       ^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/indi_pylibcamera/CameraControl.py", line 486, in snooped_FitsHeader
2024-03-23T08:46:04: Driver indi_pylibcamera:     cAltAz = c.transform_to(astrop                                                    y.coordinates.AltAz(obstime=astropy.time.Time.now(), location=ObsLoc))
2024-03-23T08:46:04: Driver indi_pylibcamera:              ^^^^^^^^^^^^^^^^^^^^^                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/coordinates/sky_coordinate.py", line 692, in transform_to
2024-03-23T08:46:04: Driver indi_pylibcamera:     new_coord = trans(self.frame,                                                     generic_frame)
2024-03-23T08:46:04: Driver indi_pylibcamera:                 ^^^^^^^^^^^^^^^^^^                                                    ^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/coordinates/transformations.py", line 1537, in call
2024-03-23T08:46:04: Driver indi_pylibcamera:     curr_coord = t(curr_coord, cur                                                    r_toframe)
2024-03-23T08:46:04: Driver indi_pylibcamera:                  ^^^^^^^^^^^^^^^^^                                                    ^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/coordinates/transformations.py", line 1027, in call
2024-03-23T08:46:04: Driver indi_pylibcamera:     return supcall(fromcoord, tofr                                                    ame)
2024-03-23T08:46:04: Driver indi_pylibcamera:            ^^^^^^^^^^^^^^^^^^^^^^^                                                    ^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/coordinates/builtin_frames/icrs_observed_transforms.py", line 33                                                    , in icrs_to_observed
2024-03-23T08:46:04: Driver indi_pylibcamera:     astrom = erfa_astrom.get().apc                                                    o(observed_frame)
2024-03-23T08:46:04: Driver indi_pylibcamera:              ^^^^^^^^^^^^^^^^^^^^^                                                    ^^^^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/coordinates/erfa_astrom.py", line 53, in apco
2024-03-23T08:46:04: Driver indi_pylibcamera:     xp, yp = get_polar_motion(obst                                                    ime)
2024-03-23T08:46:04: Driver indi_pylibcamera:              ^^^^^^^^^^^^^^^^^^^^^                                                    ^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/coordinates/builtin_frames/utils.py", line 45, in get_polar_moti                                                    on
2024-03-23T08:46:04: Driver indi_pylibcamera:     iers_table = iers.earth_orient                                                    ation_table.get()
2024-03-23T08:46:04: Driver indi_pylibcamera:                  ^^^^^^^^^^^^^^^^^                                                    ^^^^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/utils/state.py", line 58, in get
2024-03-23T08:46:04: Driver indi_pylibcamera:     return cls.validate(cls._value                                                    )
2024-03-23T08:46:04: Driver indi_pylibcamera:            ^^^^^^^^^^^^^^^^^^^^^^^                                                    ^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/utils/iers/iers.py", line 1001, in validate
2024-03-23T08:46:04: Driver indi_pylibcamera:     value = IERS_Auto.open()
2024-03-23T08:46:04: Driver indi_pylibcamera:             ^^^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/utils/iers/iers.py", line 822, in open
2024-03-23T08:46:04: Driver indi_pylibcamera:     cls.iers_table = IERS_B.open()
2024-03-23T08:46:04: Driver indi_pylibcamera:                      ^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/utils/iers/iers.py", line 274, in open
2024-03-23T08:46:04: Driver indi_pylibcamera:     cls.iers_table = cls.read(**kw                                                    args)
2024-03-23T08:46:04: Driver indi_pylibcamera:                      ^^^^^^^^^^^^^                                                    ^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/utils/iers/iers.py", line 739, in read
2024-03-23T08:46:04: Driver indi_pylibcamera:     table = super().read(file, for                                                    mat="cds", readme=readme, data_start=data_start)
2024-03-23T08:46:04: Driver indi_pylibcamera:             ^^^^^^^^^^^^^^^^^^^^^^                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/table/connect.py", line 62, in call
2024-03-23T08:46:04: Driver indi_pylibcamera:     out = self.registry.read(cls,                                                     *args, **kwargs)
2024-03-23T08:46:04: Driver indi_pylibcamera:           ^^^^^^^^^^^^^^^^^^^^^^^^                                                    ^^^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/io/registry/core.py", line 221, in read
2024-03-23T08:46:04: Driver indi_pylibcamera:     data = reader(*args, **kwargs)
2024-03-23T08:46:04: Driver indi_pylibcamera:            ^^^^^^^^^^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/io/ascii/connect.py", line 19, in io_read
2024-03-23T08:46:04: Driver indi_pylibcamera:     return read(filename, **kwargs                                                    )
2024-03-23T08:46:04: Driver indi_pylibcamera:            ^^^^^^^^^^^^^^^^^^^^^^^                                                    ^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/utils/decorators.py", line 604, in wrapper
2024-03-23T08:46:04: Driver indi_pylibcamera:     return function(*args, **kwarg                                                    s)
2024-03-23T08:46:04: Driver indi_pylibcamera:            ^^^^^^^^^^^^^^^^^^^^^^^                                                    ^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/utils/decorators.py", line 604, in wrapper
2024-03-23T08:46:04: Driver indi_pylibcamera:     return function(*args, **kwarg                                                    s)
2024-03-23T08:46:04: Driver indi_pylibcamera:            ^^^^^^^^^^^^^^^^^^^^^^^                                                    ^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/utils/decorators.py", line 604, in wrapper
2024-03-23T08:46:04: Driver indi_pylibcamera:     return function(*args, **kwarg                                                    s)
2024-03-23T08:46:04: Driver indi_pylibcamera:            ^^^^^^^^^^^^^^^^^^^^^^^                                                    ^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   [Previous line repeated 2 more t                                                    imes]
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/io/ascii/ui.py", line 463, in read
2024-03-23T08:46:04: Driver indi_pylibcamera:     dat = reader.read(table)
2024-03-23T08:46:04: Driver indi_pylibcamera:           ^^^^^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/io/ascii/cds.py", line 357, in read
2024-03-23T08:46:04: Driver indi_pylibcamera:     return super().read(table)
2024-03-23T08:46:04: Driver indi_pylibcamera:            ^^^^^^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/io/ascii/core.py", line 1414, in read
2024-03-23T08:46:04: Driver indi_pylibcamera:     self.lines = self.inputter.get                                                    lines(table, newline=newline)
2024-03-23T08:46:04: Driver indi_pylibcamera:                  ^^^^^^^^^^^^^^^^^                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/io/ascii/core.py", line 335, in get_lines
2024-03-23T08:46:04: Driver indi_pylibcamera:     with get_readable_fileobj(tabl                                                    e, encoding=self.encoding) as fileobj:
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/contex                                                    tlib.py", line 137, in enter
2024-03-23T08:46:04: Driver indi_pylibcamera:     return next(self.gen)
2024-03-23T08:46:04: Driver indi_pylibcamera:            ^^^^^^^^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera:   File "/usr/lib/python3.11/site-p                                                    ackages/astropy/utils/data.py", line 364, in get_readable_fileobj
2024-03-23T08:46:04: Driver indi_pylibcamera:     fileobj = io.FileIO(name_or_ob                                                    j, "r")
2024-03-23T08:46:04: Driver indi_pylibcamera:               ^^^^^^^^^^^^^^^^^^^^                                                    ^^^^^^^
2024-03-23T08:46:04: Driver indi_pylibcamera: FileNotFoundError: [Errno 2] No su                                                    ch file or directory: '/usr/lib/python3.11/site-packages/astropy_iers_data/data/                                                    eopc04.1962-now'
What exactly do you mean by "CCD logging"? If you briefly describe what you mean, I can send you the error report.
By the way, thank you for your great support!

hi @Olli781! Thank you very much for trying again, I believe yesterday crashes were due to the udev rules not triggering when you installed pylibcamera (my fault :| ) a reboot likely fixed it.
When you start ekos, there is a series of icons (where also the indi one is) and there should be a button called "Logs", if you click there you can enable/disable logging and the logging level for specific components (and decide if you want to log to file), if you choose verbose logging to file and mark the CCD, the camera logs will flow to the logfile Kstars makes (Kstars logs files are found under ~/.local/share/kstars/logs).
Anyway, it seems starting indiserver by hand also logs all python errors, so we are fine here :)
It seems to me that astropy does some magic stuff and is trying to download some files but it's not able to do so, it seems those URLs are working, is your Pi connected to the internet?

or maybe packaged astropy is missing some files, I will check that

ah yes, this seems indeed to be an error directly related to the astropy-iers-data package, give a while to figure it out

@Olli781 I packaged a corrected version of astropy iers data, if you update the system with update-astroarch it should pull the newest version which should fix your issue. Let me know!

Hi Matt,
it works!!!
The camera is recognized correctly and the driver runs stable. Here is the log file:
[2024-03-24T22:17:36.844 UTC INFO ][   org.kde.kstars.ekos.capture] - "Capture exposure = 1 sec, type = Light, filter = , upload mode = 0, batch mode = false, seq prefix = %t%T%F_%t_%T_%F_Light, format = RAW, encoding = , binning = 1x1, Cannot subframe"
[2024-03-24T22:17:36.847 UTC INFO ][   org.kde.kstars.ekos.capture] - "Capturing 1.000-second  image..."
[2024-03-24T22:17:36.892 UTC INFO ][           org.kde.kstars.indi] - indi_pylibcamera :  "[INFO] exposure settings: CameraSettings: FastExposure=False, DoRaw=True, ProcSize=(3280, 2464), RawMode={'label': '3280x2464 RGGB 10bit', 'size': (3280, 2464), 'true_size': (3280, 2464), 'camera_format': 'SRGGB10', 'bit_depth': 10, 'binning': (1, 1)}, CameraControls={'ExposureTime': 1000000, 'AnalogueGain': 10.666666984558105, 'AeEnable': False, 'AeConstraintMode': <AeConstraintModeEnum.Normal: 0>, 'AeExposureMode': <AeExposureModeEnum.Normal: 0>, 'AeMeteringMode': <AeMeteringModeEnum.CentreWeighted: 0>, 'AwbEnable': False, 'AwbMode': <AwbModeEnum.Auto: 0>, 'Brightness': 0.0, 'ColourGains': (2.0, 2.0), 'Contrast': 1.0, 'ExposureValue': 0.0, 'NoiseReductionMode': <NoiseReductionModeEnum.Off: 0>, 'Saturation': 1.0, 'Sharpness': 0.0} "
[2024-03-24T22:17:36.893 UTC INFO ][           org.kde.kstars.indi] - indi_pylibcamera :  "[INFO] reconfiguring camera "
[2024-03-24T22:17:38.087 UTC INFO ][           org.kde.kstars.indi] - indi_pylibcamera :  "[INFO] got exposed frame "
[2024-03-24T22:17:39.396 UTC INFO ][           org.kde.kstars.indi] - indi_pylibcamera :  "[INFO] preparing frame as BLOB: 16171200 bytes "
[2024-03-24T22:17:39.396 UTC INFO ][           org.kde.kstars.indi] - indi_pylibcamera :  "[INFO] sending BLOB "
[2024-03-24T22:17:40.364 UTC INFO ][   org.kde.kstars.ekos.capture] - "Captured /tmp/kstars.fQWnNX"
[2024-03-24T22:17:40.367 UTC INFO ][   org.kde.kstars.ekos.capture] - "Download Time: 0.60 s, New Download Time Estimate: 0.60 s."
[2024-03-24T22:17:41.060 UTC WARN ][              kf.notifications] - Failed to play sound with canberra: File or data not found_
Only one warning appears when updating:
warning: gst-plugin-libcamera: local (0.2.0-2) is newer than extra (0.2.0-1)
warning: libcamera: local (0.2.0-2) is newer than extra (0.2.0-1)
warning: libcamera-docs: local (0.2.0-2) is newer than extra (0.2.0-1)
warning: libcamera-ipa: local (0.2.0-2) is newer than extra (0.2.0-1)
warning: libcamera-tools: local (0.2.0-2) is newer than extra (0.2.0-1)
there is nothing to do
Next I will try to install astroarch on an SSD. That should bring a performance gain.
Thank you again for your great support. Astroarch is a really great project!

@Olli781 I am happy it finally works :D it's been fun digging into libcamera and other stuff.
The warning are fine for now, those packages are unstable - please ignore them and remember, if you make a new installation on a SSD you will need to pull the libcamera custom packages with
wget http://astromatto.com:9000/aarch64/libcamera-0.2.0-2-aarch64.pkg.tar.xz
wget http://astromatto.com:9000/aarch64/libcamera-docs-0.2.0-2-aarch64.pkg.tar.xz
wget http://astromatto.com:9000/aarch64/libcamera-ipa-0.2.0-2-aarch64.pkg.tar.xz
wget http://astromatto.com:9000/aarch64/libcamera-tools-0.2.0-2-aarch64.pkg.tar.xz
wget http://astromatto.com:9000/aarch64/rpicam-apps-1.4.3-1-aarch64.pkg.tar.xz
wget http://astromatto.com:9000/aarch64/gst-plugin-libcamera-0.2.0-2-aarch64.pkg.tar.xz
sudo pacman -U gst-plugin-libcamera-0.2.0-2-aarch64.pkg.tar.xz libcamera-0.2.0-2-aarch64.pkg.tar.xz libcamera-docs-0.2.0-2-aarch64.pkg.tar.xz libcamera-ipa-0.2.0-2-aarch64.pkg.tar.xz libcamera-tools-0.2.0-2-aarch64.pkg.tar.xz rpicam-apps-1.4.3-1-aarch64.pkg.tar.xz && sudo pacman -S libpisp
```

This week I will work on automating this step in a similar fashion I did for bleeding packages so it will be easy to install them and possibly they will be the official ones for raspberris in one of the next release.

I will let you know once this shortcut to install the rpi libcamera packages will be ready!

and last but not least, thanks for your support! :heart: great to hear people like this project

this should be done, in theory now, if one runs sudo pacman -S indi-pylibcamera it should pull libcamera for the raspberry directly

hi @ripkrizbi thanks for the report, I noticed I may have packaged a wrong version of python-v4l2 which is indeed missing that attribute. Will have a look soon

hi @ripkrizbi can you try to run update-astroarch and tell me if that is fixed? I shipped the newest version (0.3.4) of python-v4l2 now which bring in the failed attr V4L2_CID_MPEG_VIDEO_H264_PROFILE

happy it works!

think it's time to close this one as fixed :)
feel free to re-open if I missed something


Add support  HuggingFace Inference Endpoint for embeddings

Feature request
https://huggingface.co/inference-endpoints
Motivation
We do not have support for HuggingFace Inference endpoints for tasks like embeddings and must be easy to implement inheriting the class from Embeddings base class
InferenceEndpointHuggingFaceEmbeddings(Embeddings): with only
self.endpoint_name = endpoint_name         self.api_token = api_token
Your contribution
I could submit a PR to this issue since I've created a class for this

Not really. HuggingFaceEmbeddings as you said only support sentence-transformer (basically running in local the embedding model) but the idea is trying to replicate the class of HuggingFaceendpoint (That does not support embedding)
https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/llms/huggingface_endpoint.py
I have used HF Endpoints for embeddings to be used as an API for multiple services

Yes, I think we are talking about two different things. While you are referring to HuggingFaceEmbeddings, I was talking about HuggingFaceHubEmbeddings . This Hub class does provide the possibility to use Huggingface Inference as Embeddings, just only the sentence-transformer models. They dont run lucally than but via the HG API as imported as following:
from huggingface_hub.inference_api import InferenceApi
If you are saying though that you want to replicate the HuggingFaceEndpoint then indeed my pointing to the Hub class was not correct, as the Endpoint class is using Huggingface Endpoints instead of inference:
from huggingface_hub.hf_api import HfApi
That indeed is missing.

will this be implemented?

There is a HuggingFaceInferenceAPIEmbeddings class for using publicly available hugging face endpoints. It contains hardcoded _api_url property with public endpoints url. To support protected huggingface endpoints we need to replace this url with custom one.
I suggest to make an extra BaseModel field api_url: Optional[str] = None and in embed_documents function instead of posting to self._api_url post to self.api_url or self._api_url. That way owners of protected endpoint will be able to use it using the same class.
I would also rename self._api_url -> self._default_api_url to avoid confusion, but I am not sure, is it is ok to change protected field name.


Add support for optional dist backend in param scoring

Currently, PruningParamsGradScorer handles gradient communication between processes in DDP runs by creating a gloo backend. This was initially done this way to overcome limitations of the nccl backend when passing around gradients as cpu-tensors. Some pruners (for example oBERT) need to communicate only one gpu-tensor, and thus can use the default distributed backend that comes with DDP (i.e. nccl). This implementation should allow creating groups with any distributed backend, and should also take care about cases when additional group is not need (i.e. in the oBERT implementation, with dist_backend=None).
Remaining ToDos if this gets accepted: pruning modifiers that are using this scorer should specify which dist. backend to use.
@KSGulin I assume we will need to update only MFAC modifiers, which should initialize scorer with dist_backend='gloo'

hi @eldarkurtic, code looks good overall, any reason why the default is currently set to None rather than 'gloo' to keep existing functionality the default?

@bfineran the gloo backend is a special case used currently only by MFAC pruners to pass around huge cpu-tensors. I think it's not something a user would expect to have as a default behaviour. The default proposed in the PR (with None) is going to use whatever the default distributed backend is already initialized by the pytorch DDP launcher, and won't create additional groups unless explicitly specified with the dist_backend argument.


Elastic tables and Synapse Link for Dataverse record deletion / TTL

Hi,
When I add a Elastic  table (using overwrite mode) to the Synapse Link for Dataverse, the data is properly exported to the data lake. Performing creates, updates and deletes work as expected.
However, when I set the time to live for a record in the elastic table, the record that expires and is deleted from the database, the record is not removed from the data lake.
This is not what I expected. I expected the same behavior as when a record is deleted from the UI. Could you have a look at this to see if this behavior is as designed or not. If this is by design, could you make sure this is clear from the documentation?
Kind Regards,
Pieter Hilkemeijer

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 0bb9fd94-483d-1d6a-6c60-7b79c8a45d26
Version Independent ID: 0bb9fd94-483d-1d6a-6c60-7b79c8a45d26
Content: Elastic tables for developers (preview) - Power Apps
Content Source: powerapps-docs/developer/data-platform/elastic-tables.md
Service: powerapps
Sub-service: dataverse-developer
GitHub Login: @pnghub
Microsoft Alias: gned

Thanks for your comment, @pieterh. This issue has been assigned to the doc author for review.

@sabinn-msft Can you also look at this question?
This seems more like a question for What is Azure Synapse Link for Dataverse? or possibly Create and edit elastic tables (preview) since the scope of this article is to use Dataverse APIs to work with elastic tables.

Hi @pieterh - Thanks for reporting this.
This is a known gap that requires to be fixed (ETA TBD) - "When time to live (TTL) is used on a row, the row will get deleted from the elastic table when TTL has expired. If it's synchronized to a data lake using Synapse Link for Dataverse before TTL expiry, it won't be deleted from the data lake"
It has documented now in the public doc.
Best, girish

thank you


Increase postgres resources 1->1.5Gi

Last time (2f07b2318) we tried to lower the value of shared buffers but it hasn't helped much so it's probably time to request more memory. The shared buffers value is 15% of the request/limit.

After cleaning more old SRPM logs the memory consumption has lowered to 0.8GiB so this might not be needed after all. (or not now)


Could this detector be used in the Web Worker context?

Thanks a lot for putting this library together! Amazing stuff!
I have one question. It seems that this polyfill fails for me when I'm trying to use it from a Web Worker. On the other hand MSDN states (second NOTE there) that this interface should be accessible from Web Worker.
The method which fails is isImageBitmapSource which references types like HTMLImageElement and they are not present in the Web Worker global scope.
Could you please tell what do you think?

Thanks @paluh for pointing this out!
Fixing it should not be hard, but could you please provide an example of where this occurs? I would like to add something like that to my tests.

Hi @undecaf, thanks for your quick response.
The problem which I mentioned is caused by this piece from isImageBitmapSource: https://github.com/undecaf/barcode-detector-polyfill/blob/master/src/BarcodeDetectorPolyfill.ts#L233 which is called from detect so I'm not able to call really use this method in my Web Worker (I hope I did not mess up anything ;-))
I can try to put together a PR this weekend as I will back hacking on the project and I have the code with Web Worker on a branch.

Hi @paluh, I implemented what I think will fix the problem in release 0.9.21.
In JSDelivr, it may take some time until the latest release becomes the default download; so for now please specify the release number explicitly in the dependency URL, e.g. https://cdn.jsdelivr.net/npm/<EMAIL_ADDRESS>

Thank you very much @undecaf! Works like a charm!
P.S. I feel a bit bad - I should provide the PR myself but I'm in the middle of my work week...

@paluh No need to feel bad -- glad I could help!


Remove Unused Autoscaler Cloud Providers

Issue #, if available:
https://github.com/aws/eks-anywhere-build-tooling/issues/2084
https://github.com/aws/eks-anywhere-build-tooling/issues/2092
Description of changes:
This PR strips out unused cloud providers from cluster autoscaler.
The immediate benefit is that https://github.com/aws/eks-anywhere-build-tooling/issues/2084 is resolved.
Longer term, this should reduce our surface area for future CVEs.
This PR also includes fixes for https://github.com/aws/eks-anywhere-build-tooling/issues/2092 in 1-22 and 1-24 autoscaler images. I can separate this into a different PR if we want.

/lgtm

/lgtm

/approve


Car_Ownership_Prediction added

#1050

@invigorzz313 i've updated pls check it

@ashis2004 these all PRs you are making , please go through the repo first after then make a PR or create issues accordingly


Local development

Hi @Mansari and @vinay-google
Thanks for this great example! , was very useful.
I have everything working but now I'm not sure how my team and I can work locally on the project.
What I did was to tunnel server.js with ngrok and put the public url on deployment.json and default.json, then rebuilding . It works, I can see my local changes but now if anyone wants to work it will have to re-deploy with their own ngrok url blocking the rest of the team.
Is there another way to work locally?  I could only find this which shows there is live reload: https://developers.googleblog.com/2021/03/evolving-google-workspace-add-ons-with.html but it doesnt explain how
Thanks!

Hi @llermaly,
Thank you for your feedback! I love what you shared about building a local test environment, that would be a great blog post to publish if you are interested, or feel free to add it as an addition to the READ.ME.
Now for co-developing an add-on in a team, there are a few ways I can think of setting this up:
Option 1 : One add-on that everyone works on it
In this case, you will deploy ONE copy of the add-on, publish it internally your domain, and then either install it for everyone as an admin (using admin install) or have those developing install it from the Google Workspace Marketplace.
This copy of the add-on will be pointing to a published HTTPS endpoint that the team controls (e.g. linked to a dev/staging copy of the add-on backend code), and as the team continues to push their updates to this branch/codebase, you either manually update the deployment or have a CI/CD pipeline that does that. You can configure a CI/CD (or a deployment script) to pick any differences in the deployment.json for example, and use the [Google Workspace Add-ons APIs] (https://developers.google.com/workspace/add-ons/reference/rest) to create/update deployments, and refresh your hosting server with the latest checked-in backend code. Everyone will now see the latest version of the add-on and will be able to collaborate / QA one version.
Option 2 : Every developer has their own copy of the add-on
In this case, everyone will do exactly what you did. They will have their own add-on that is linked to their local codebase and any refresh will show up in their own version.
There is nothing that prevents you from doing both. You can have an individual copy of the add-on for each developer linked to their local code, and you can have one (or many) shared copies - staging, QA, prod, that are shared amongst the team members.
These ideas are just off the top of my head before I head to sleep, so forgive me if I missed anything, and feel free to comment if you have any additional questions.

Thanks for your quick answer! . Option 1 will not allow live testing, right?

It all depends on where the code is running - if you point this version of the add-on to the code running on your machine, then you can debug it live on yours. If you change the endpoints to point to someone else's machine, then it can be debug there.
You can indeed run Cloud Run (which is used in this sample) locally . If you wanted to refactor the code/deployment to use Cloud Functions instead, they can also be run locally.
Does this help? If you can let me know how you plan to setup the dev environment for your team, or what is your ideal setup for debugging/testing, I am sure @vinay-google may have some more guidance to share as well.

@Mansari Maybe I'm missing something, because to run it locally what I did was to tunnel localhost to a public address that I had to add to the config, then to test the UI I went to the actual addon.
Doing this if another member wants to work and do the same , will override the URL I created and I will not be able to work on my changes.
Is this accurate? Or there is a way to set the URL to localhost and see live changes in the UI without affecting others ?
If we use Cloud Run locally we should only replace the URL in the json config? or tunneling is also needed to be reachable from the addon?
Sorry for so many questions, I'm not very familiar to this workflow. On similar situations (e.g: atlassian apps) the repo comes with a way to run the server locally and set the app to developer mode.

@llermaly You are correct. If each member wants to test on their local machine, then they would have to do the tunneling + updating the deployment manifest to point to their URL, thus overriding the other's changes.
if you are looking for a collaborative coding environment where everyone have access to the same codebase and the add-on is linked to that codebase, I would then suggest using something like Repl.it (not an endorsement as I have not personally used it) or similar shared IDE / hosted code solutions may be the answer. In that setup, you can host one endpoint that you can point the add-on to, but you + your colleagues will have access to the code that runs it to monitor/modify it together.
Does that help? I asked my colleague @PierrickVoulet who also might be able to add more thoughts on this.

@llermaly BTW a new guide on how to debug Google Workspace Add-ons has just been published. It features a similar approach using ngrok, but might be helpful to review if it gives you any additional guidance.

Thanks @llermaly ! Don't hesitate to reach out if anything else comes up. For now, I'll mark this issue as resolved.


Activity Indicator and EmptyDataset

Hey ya'll!
Currently, my problem is that the EmptyDataset view appears while my tableview is fetching data. Then it disappears while the tableview gets data. Ideally, I'd like an activity indicator to appear while the data is being fetched and to have the EmptyDataset view appear only if there's no data to present.
I could use GCD with a time interval to give my tableview time to load data while presenting the activity indicator view. However, I feel that's kind of hacky. Is there a delegate method that I'm missing? Thanks in advance!

You can override customViewForEmptyDataSet: and return a UIActivityIndicatorView while your controller is fetching data.
- (UIView *)customViewForEmptyDataSet:(UIScrollView *)scrollView {
    if (self.isLoading) {
        UIActivityIndicatorView *indicator = [[UIActivityIndicatorView alloc] initWithActivityIndicatorStyle:UIActivityIndicatorViewStyleGray];
        [indicator startAnimating];
        return indicator;
    }
    return nil;
}

Awesome! That works! However, when I have an empty array for my tableview the indicator spins forever.
What's the best way to disable the custom UIActivityIndicatorView and display my EmptyDataSet when there is no data?

Not sure what you mean, you can just make sure isLoading is false and reload your tableView with the empty array.
Any time customViewForEmptyDataSet: returns nil, the standard empty state will be shown.

Ok, I'm coding in Swift. Is there an equivalent to isLoading?

You have to implement isLoading yourself regardless if you use Swift or Objc.
It may be better to post a question on StackOverflow if you can't get this to work, because this isn't an issue with DZNEmptyDataSet.

Good call. Thanks for the help @bartvandendriessche !

@bartvandendriessche what is a good place to implement isLoading?


Report dataset Metrics Summary on Temporal

Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA

[ ] My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"

https://issues.apache.org/jira/browse/GOBBLIN-XXX

Description

[ ] Here are some details about my PR, including screenshots (if applicable):

Tests

[ ] My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

[ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Codecov Report
Attention: Patch coverage is 0% with 54 lines in your changes are missing coverage. Please review.

Project coverage is 41.21%. Comparing base (fb85f07) to head (f392a96).

Files
Patch %
Lines

...temporal/ddm/activity/impl/CommitActivityImpl.java
0.00%
37 Missing :warning:

.../ddm/workflow/impl/ExecuteGobblinWorkflowImpl.java
0.00%
15 Missing :warning:

...che/gobblin/temporal/ddm/work/assistance/Help.java
0.00%
2 Missing :warning:

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3912      +/-   ##
============================================
- Coverage     46.62%   41.21%   -5.41%
+ Complexity    11222     2194    -9028
============================================
  Files          2244      475    -1769
  Lines         88372    20216   -68156
  Branches       9680     2344    -7336
============================================
- Hits          41202     8332   -32870
+ Misses        43455    11000   -32455
+ Partials       3715      884    -2831

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


dino: 0.3.1 -> 0.4.0

https://github.com/dino/dino/releases/tag/v0.4.0
Switched to gtk4, cleaned up dependencies and tested in a MUC.
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Thanks for the detailed investigations!

Cannot nix-instantiate `dino' because:
error: opening file '/nix/store/ryn8lxprr6f4g3dl1nbjxs58bwafs9fi-pkg-config-wrapper-0.29.2.drv': No such file or directory

o_O
@OfBorg build dino

I just fixed the version number in the about dialog and added more codecs for video calls.


Run image-pr.yaml pipeline on PRs to dev branch as well

Resolved in commit https://github.com/Quansight/qhub-cloud/commit/5856e8e1afde009731edb44d02275cfe4d3660be through PR https://github.com/Quansight/qhub-cloud/pull/351. This PR also fixed the issue with CI not building jupyterlab image properly

Awesome, thanks!


Llama in modelling value head tests

Llama is now fully supported. Follows #261

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


Added dark theme for photoeditor

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


PPO expert policy changes obs_shape of env

After this update, I found PPO expert will change the obs_shape of env from 259 to 275, which lead to reset() and step() return different observation size.
In previous versions, it seemed that the observation would only be read with a size of 275 without changing the env configuration.
Besides, load network with GPU might be more efficient. It is recommended to add a switch to select GPU or CPU policy.

here, I fix this bug with re-update config and use torch expert. Running efficiency has been improved by about 3 times.
import torch
import numpy as np
import os.path as osp
from metadrive.engine.engine_utils import get_global_config
from metadrive.obs.state_obs import LidarStateObservation

ckpt_path = osp.join(osp.dirname(__file__), "expert_weights.npz")
_expert_weights = None
_expert_observation = None

def obs_correction(obs):
    obs[15] = 1 - obs[15]
    obs[10] = 1 - obs[10]
    return obs

def numpy_to_torch(weights, device):
    """
    Convert numpy weights to torch tensors and move them to the specified device.
    """
    torch_weights = {}
    for k in weights.keys():
        torch_weights[k] = torch.from_numpy(weights[k]).to(device)
    return torch_weights

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

def torch_expert(vehicle, deterministic=False, need_obs=False):
    global _expert_weights
    global _expert_observation
    expert_obs_cfg = dict(
        lidar=dict(num_lasers=240, distance=50, num_others=4, gaussian_noise=0.0, dropout_prob=0.0),
        random_agent_model=False
    )
    origin_obs_cfg = dict(
        lidar=dict(num_lasers=240, distance=50, num_others=0, gaussian_noise=0.0, dropout_prob=0.0),
        random_agent_model=False
    )
    with torch.no_grad():  # Disable gradient computation
        if _expert_weights is None:
            _expert_weights = numpy_to_torch(np.load(ckpt_path), device)
            config = get_global_config().copy()
            config["vehicle_config"].update(expert_obs_cfg)
            _expert_observation = LidarStateObservation(config)
            assert _expert_observation.observation_space.shape[0] == 275, "Observation not match"

        vehicle.config.update(expert_obs_cfg)
        obs = _expert_observation.observe(vehicle)
        vehicle.config.update(origin_obs_cfg)
        obs = obs_correction(obs)
        obs = torch.from_numpy(obs).float().unsqueeze(0).to(device)  # Convert to tensor and move to device
        weights = _expert_weights
        x = torch.matmul(obs, weights["default_policy/fc_1/kernel"]) + weights["default_policy/fc_1/bias"]
        x = torch.tanh(x)
        x = torch.matmul(x, weights["default_policy/fc_2/kernel"]) + weights["default_policy/fc_2/bias"]
        x = torch.tanh(x)
        x = torch.matmul(x, weights["default_policy/fc_out/kernel"]) + weights["default_policy/fc_out/bias"]
        x = x.squeeze(0).cpu()  # Move back to CPU and remove batch dimension
        mean, log_std = torch.split(x, 2, dim=-1)
        if deterministic:
            return (mean.numpy(), obs.cpu().numpy()) if need_obs else mean.numpy()
        std = torch.exp(log_std)
        action = torch.normal(mean, std).cpu()  # Move back to CPU
        return (action.numpy(), obs.cpu().numpy()) if need_obs else action.numpy()

def torch_value(obs, weights):
    with torch.no_grad():  # Disable gradient computation
        obs = torch.from_numpy(obs).float().unsqueeze(0).to(device)  # Convert to tensor and move to device
        weights = _expert_weights
        x = torch.matmul(obs, weights["default_policy/fc_value_1/kernel"]) + weights["default_policy/fc_value_1/bias"]
        x = torch.tanh(x)
        x = torch.matmul(x, weights["default_policy/fc_value_2/kernel"]) + weights["default_policy/fc_value_2/bias"]
        x = torch.tanh(x)
        x = torch.matmul(x, weights["default_policy/value_out/kernel"]) + weights["default_policy/value_out/bias"]
        return x.squeeze(0).cpu().numpy()  # Move back to CPU and remove batch dimension

ok, I just file a PR.


BAEL-2995 - possible lossy conversion

test this please

(just a Jenkins/CI comment to kick off a new build)


Add Inversify-based dependency injection to infrastructure components

By using the @injectable & @inject decorators present within the inversify framework, the storage backing aggregate repositories, messaging, etc. can be implemented in this library while leaving wiring up of dependencies to consumers via appropriate set up of Inversify container instances at application startup.

Completed in #7


RFC: color prop API

Currently our type declarations contain the following definition for color props:
type Color = 'inherit' | 'primary' | 'secondary' | 'default';

indicating that there is a library wide understanding what these color represent and that every component that has a color prop should implement each variant.
However only primary and secondary are implemented for every component with a color prop. inherit and default are not implemented in every component. default doesn't even have a consistent style.
Implementation overview

Component
primary
secondary
inherit
default

AppBar
x
x
x
x

Badge
x
x

x

Button
x
x
x
x

Chip
x
x

x

Icon
x
x
x

IconButton
x
x
x
x

SvgIcon
x
x
x

Typography
x
x
x
x

default variant
Implementation

Component
color
background-color

AppBar
theme.palette.getContrastText(backgroundColorDefault)
theme.palette.type === 'light'  ? theme.palette.grey[100 ] : theme.palette.grey[900 ]

Badge
theme.palette.textColor (which is undefined)
theme.palette.color (also undefined)

Button
theme.typography.button
global stylesheet

Chip
theme.palette.getContrastText(backgroundColor)
theme.palette.type === 'light'  ? theme.palette.grey[300 ] : theme.palette.grey[700 ]

IconButton
theme.palette.action.active
fade(theme.palette.action.active, theme.palette.action.hoverOpacity) if :hover

Typography
global stylesheet
global stylesheet

Proposal
Remove it because:

not even the actual default value for the components
not mentioned in the material spec
broken for Badge with no report (I was not able to determine when this actually broke but I guess this happened a few months ago; Edit: passed undefined even in 1.0.0-alpha.2)

People can always set the color prop to undefined which will result in no applied css rules concerning color which is a proper default in my opinion.
inherit variant
Implementation

Component
color
backgroundColor

AppBar
global stylesheet
global stylesheet

Button
inherit
global stylesheet

Icon
global stylesheet
global stylesheet

IconButton
inherit
global stylesheet

SvgIcon
global stylesheet
global stylesheet

Typography
inherit
global stylesheet

Funny enough in Icon fontSize="inherit" color="inherit" causes font-size: inherit; but no defined color in css.
Also the default for fontSize in those components is default and applies always no css rules but the default for color is inherit which applies sometimes no css rules. This might as well be removed. There is no value in a named default value in my opinion but this is should be discussed separately.
Proposal
No strong opinion about that one. Either repurpose this as a default replacement which means color and background-color are not set or actually set inherit which is the most obvious. AppBar for example does not do anything with inherit which might be confusing.
/cc @mui-org/core-contributors

default doesn't even have a consistent style

The default color should be whatever is consistent with the spec if no alternative color enum is supplied, but as you suggest, there doesn't need to be a default variant for that.
Removing it would be a breaking change though, so deprecation warnings required.

Removing it would be a breaking change though, so deprecation warnings required.

This will only affect the AppBar. For the other components "default" would just be an alias for undefined.
Badges are currently in a weird sport because "default" is broken anyway (or maybe not since I don't know how they looked). Maybe that's a good thing because nobody would mourn about a breaking change.
The versioning would look like this:

minor: deprecate "default" for AppBar; make it a no-op for the other components
major: remove "default"

default variant
The default variant is the value that matches the specification. It's the color value you would expect by default. I don't see why it should be removed. What do we win? How do you solve the problem instead? Regarding the Badge, yes it's definitely broken. For this component, I don't think that a default variant makes sense. What we have now is an inherit variant. I think that we can simply rename default -> inherit in a minor release.

People can always set the color prop to undefined which will result in no applied css rules concerning color which is a proper default in my opinion.

So it's how you are suggesting to solve the problem instead. Isn't 'default' more explicit than undefined?
inherit variant

Funny enough in Icon fontSize="inherit" color="inherit" causes font-size: inherit; but no defined color in css.

Yes, it's expected, we rely on the default browser value. Isn't that correct?

To sum-up, I think that we can fix the Badge implementation and we should be good.

There is no value in a named default value in my opinion but this is should be discussed separately.

I don't understand this point.

The AppBar is an intereseting case. The specification changed, having a default value here doesn't make sense anymore.

Yes, it's expected, we rely on the default browser value. Isn't that correct?

@oliviertassinari
It's inconsistent. Sometimes it sets the css property to inherit sometimes it doesn't set anything. Those are two different things. I would expect that setting "inherit" in a prop to mean the same thing across components.

So it's how you are suggesting to solve the problem instead. Isn't 'default' more explicit than undefined? Let say I set color={undefined} Wouldn't you expect the color to inherit? Well, it won't.
By using 'inherit' and 'default', we are making the behavior deterministic.

It doesn't matter what I expect. The runtime already falls back to whatever is defined as a default value if you pass undefined as an argument to a function. If I'm used to that behavior when using javascript I'm used to that behavior in whatever is written in javascript.
TL;DR: Remove "default" from AppBar, set the default of Badge to inherit followed by some rant about stringly typing a default value.
I have never seen a component API that used a string literal to instruct that the component should use its default behavior. My main issue is with facebook's understanding of what an optional type is. Using PropTypes.oneOf(["default", ...colors]) will allow "default" | undefined | null | typeof colors. An explicit undefined | "default" is equivalent. Setting null causes a runtime error without a prop-types warning.
What I don't understand is why the current API is easier to understand than ProTypes.oneOf(colors) and setting the color className if color != null. Setting an argument to undefined is saying to the runtime that it should fallback to the default value. That is true for javascript as a language so I think we should expect from users to know that.
Or would you design the signature of a function so that users can write
function multiply(a, b = "default") {
  const c = b === "default" ? 1 : b;
  return a * c;
}

I think this is really confusing if we set the default value of most components to the literal "default" and on AppBar to "primary". So for some components the default is "default" and for some it's "primary" and then for Badge it's "inherit". Naming a default value if there is no correspondence to an actual named color variant like primary, secondary, action etc. just seems arbitrary.
It's also bad because "primary" means the same for the color across components. That is not true for "default". Depending on the component it's either some shade of grey or black with some transparency.

I would expect that setting "inherit" in a prop to mean the same thing across components.

It does mean the same thing: As a user, you can set a color on the parent and see it being applied to the children. Same output, no matter the implementation. What changed, is that in some cases, we have to add some extra CSS, in some other cases we don't need it. It's an implementation detail.

It doesn't matter what I expect.

It's about having the default behavior documented in the API. Sometimes, the color inherit: inherit, sometimes it's a specific value defined by the specification: default. I find it wrong to set a color value in the CSS while having color = undefined, if we set a value, it should have some meaningful value, maybe it should be renamed spec.

It's an implementation detail.

Fair enough.

I find it wrong to set a color value in the CSS while having color = undefined.

That is the current behavior though. undefined and "default" are equivalent.

It's about having the default behavior documented in the API.

That is not the case currently. I don't know what color is applied when I use "default". I know what "primary", "secondary" and "error" means if I know the material design palette. I know what action means which is an addition in this implementation. "default" however is not defined. "background" and "surface" might make sense for some components but those are currently not implemented.
For example https://material-ui.com/api/icon-button/ has separate classes for each color category except "default". It does not say which class is applied for the "default" category. So I guess putting them in root is fine. But then those rules are also applied if I use a different category. This is a special behavior of the string literal "default".

"The color of the component. It supports those theme colors that make sense for this component."

"default" is not a theme color so where is this taken from? The same argument could be made for "inherit" but I'm ok with the tradeoff here since it's related to css inherit.

That is not the case currently. I don't know what color is applied when I use "default"

@eps1lon In this case, what about using an explicit value and removing the default value? I really like this approach too.

Let's take https://material.io/design/color/the-color-system.html#color-theme-creation as the baseline. We have primary, secondary, surface and background and error and call them categories.
primary and secondary have light, main and dark variants as well as emphasis variants that are used for buttons. high defaults to dark, normal to main and low would be what is currently used for "default" (see https://material.io/design/components/buttons.html#hierarchy-placement)
Each category + variant combination has a contrast color which is named as On + category.
I believe that the API is simply <Component color="primary" variant="dark" /> and the implementation just™ has to pick from the palette. Everything that is currently used from the grey palette is just™ On* with opacity. For buttons one would use emphasis instead of variant.
I would hope that there is no need for helpers like getContrastText or fade other than in createPalette.
This is at least how I interpret the current spec.

The light color type is needed for the Typography component.

Please don't remove the inherit color property as we make extensive use for custom theming of our app. Thanks!

@skirunman inherit is a must-have, there is no plan in losing this capability. I think that the concern is about how we harmonize it.


Fix parallel creating commit status bug with tests (#21911)

backport #21911

Should we really backport a test that introduces a data race we haven't been able to fix yet?

I'm waiting #21998

Should we really backport a test that introduces a data race we haven't been able to fix yet?

Looks like the race detection cannot be reproducible every time.


[ListItem] add support for 3-line list items in v1

This seems quite simple to implement, but wanted to check if ListItem should have a prop for number of secondary text lines like 0.x version does secondaryTextLines: PropTypes.oneOf([1, 2]) to support 3-line list items as per Material Design spec.

[x] I have searched the issues of this repository and believe that this is not a duplicate.

how to add secondaryTextLines with version v1 in material ui

It would be nice to have.


Got collisions for v1() when generating many IDs

Hi,
I have been using node-uuid for a while, and it has in total generated about 500k IDs that I use. I now ran a check and found, to my surprise, 2 collisions among those IDs.
The collisions were generated in a sequence where 5-10 IDs are created synchronously and stored in the same database document. For one of the collisions it happened directly following the original entry. For the other collision there were 2 entries between them.
One of the collisions was:
e7a69040-b412-11e2-ad2c-57e3dd379c2d
Should this really be possible? Perhaps there is some flaw in the implementation.
I was using Node 0.8 at the time, if that matters.

Closing due to inactivity. Please reopen if this is still a demonstrable bug.
From skimming the comments it sounds like this may have been an related to the accidental inclusion of Math.random()?

What was the take away from this thread? Is this the expected behavior?
[1, 2, 3].forEach( function(i) {
  console.log(i);
  console.log(uuid.v1())
});

outputs:
1
d8982ec0-275e-11e6-aaac-a53c3533f1ce
2
d89855d0-275e-11e6-aaac-a53c3533f1ce
3
d89855d1-275e-11e6-aaac-a53c3533f1ce

@FreakTheMighty The key seems to be different. Take a deep look at it :)


Enable Prow for PR merging

We should enable prow to review PRs .
It looks like the issue is that the ci-bots don't have permission on the repo

Hello!
It looks like Prow is trying to merge things, but there isn't an OWNERS file in this repository. As a result, the approve plugin can't determine who to assign for approval.
Information about OWNERS files and how to create them can be found at https://go.k8s.io/owners. A simple YAML file, such as the example below, can do for now.
See https://github.com/kubeflow/blog/pull/31 for an example of this; the link to OWNERS is currently dead.
# See the OWNERS docs at https://go.k8s.io/owners

reviewers:
- some_github_name
- another_github_name


feat: update to be compliant with other generators + cleanup

Updated generator to be compliant with the generator generator-ui5-ts-app

newest versioin of tooling
updated scripts
updated some of the syntax
cleanup of unused code

Looks good to me overall, only the getSource version must be fixed. Plus two other comments.


Update overlay hover popovers

When hovering on the overlays users see this:

But designs have since been updated eg:

See all designs here

is it weird i kinda like the first one better?

Oh CC @franknoirot
I know you said "hard to put into words" so I'm guessing you just mean the vibe of it and I guess the composure?
 And I think the container could be feeling a bit cramped. Here's what I tweaked in this version:

Remove the "[Function Name] ·" prefix, because it adds a lot of that larger font size. I'll have a think about how to communicate what line function is being used.
moved the Constrained/Unconstrained tag up to the top, bumped down its padding. I think it sharing a row with that description text feels a little more cramped too.
Double the padding around the whole card

This feels less cramped.
In regards to 1. we are still highlighting in the editor

But is the thinking about when the editor is closed?

ooo i like those better


Bug fixes

Fixes #184
Fixes #186
Fixes #185
Fixes #188

Flexbox fixes weren't half as bad as they could have been

Did a deploy on prod. All of these changes are good to go 🎉


Sprite/Image/Texture Editor/Creator

So as an artist who also uses godot to work on games, i found it rather time consuming having to back and forth between a sprite editor and have to export all the time and such, So i came up with a rather rough mockup for a sprite editor, i dont know how to Code or anything as im mainly an artist, so any feedback on my idea would be helpful

Might be doable as a plugin, unless some of the core devs wants it in core :smile:

I would love to give it a shot to write one, but I am not even sure what node to use in godot in order to do pixel painting.
There was a user interested in this a while ago on facebook - he wanted to make an animation editor in godot

if you save a png file to the disk, it will be reloaded automatically, it
shouldn't be time consuming..  maybe something is wrong in the workflow?
On Wed, Sep 14, 2016 at 6:19 AM, Todor Imreorov<EMAIL_ADDRESS>wrote:

I would love to give it a shot to write one, but I am not even sure what
node to use in godot in order to do pixel painting.
There was a user interested in this a while ago on facebook - he wanted to
make an animation editor in godot
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/6477#issuecomment-246954516,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF-Z2xQZfMo-gbtIxq-9WYtN6SrnIBkQks5qp7wtgaJpZM4J8G2m
.

The workflow is good,
However when you rapidly prototype games and don't have any assets to play with, having the ability to create a simple placeholder sprite inside the game engine - be it a square of some color - is a nice thing to have.
Also if you have it built into the editor- it makes you feel like you are working in a full fledged game studio - not just a game engine with an editor.
We could take example from Game maker Studio's success, and other engines that have a built in sprite editor- clickteam fusion and construct2.
That said, there is an interesting plugin for simple pixel editing being developer here:
https://github.com/cwkx/godot-pixel-painter

Seems like a waste of time.  Just set your graphics app to use the godot project as it's default save position and then any updated save will auto update in your project.  That's not tedious at all and perhaps better because you then have the full power of your graphics app and not something stripped down that's been integrated.

Yes I agree that this would be out of scope of a game engine. There are great tools for vector and raster graphics, many of which are free and open source, and do way more things than we could even dream of implementing, if it weren't completely tangential to our work as game engine developers. Our purpose is to use assets, not create them - asset creation is a domain in its own right and we shouldn't spread our resources thin when there is no practical gain for the user.
If there's a use case for a trivial sprite editor à la MSPaint, it could easily be implemented as a plugin, but not in the main engine.


Uncaught ReferenceError Fix: wrap Drift account ID in quotes

Fixes issue with https://github.com/railslove/rack-tracker/pull/139
Thank you in advance!

damn, guess I should have done a better review.
thanks for the fix. can you maybe adjust a test for that?

Sure, is this sufficient? Screenshot shows the test case failing on master and passing on this branch with the change.
Happy to extend further if needed

will prepare the release this evening

@sassela please check the new release 1.11.1 :)

thanks @DonSchado
1.11.1 is the best release! :shipit:

Thanks to you both! Much appreciated


Minor JS tweaks found with ESLint.

BTW, I still believe the current JSHint settings are weird, but I guess the final call is for @alanshaw to make :)

What about semicolons?

Not my choice; that's what is already being used.
On Jan 4, 2015 6:47 PM, "Ilan Biala"<EMAIL_ADDRESS>wrote:

What about semicolons?
—
Reply to this email directly or view it on GitHub
https://github.com/alanshaw/david-www/pull/164#issuecomment-68639383.

@alanshaw can semicolons be added in?

@alanshaw: bump


ServiceCollector Failing

Describe the bug
When running the ServiceCollector on Windows 8, Windows Server 2012, (maybe more, but not Win10)
To Reproduce
Steps to reproduce the behavior:

Run Asa.exe collect -s --verbose
Observe (almost) every object has an error

Expected behavior
Not have an error for every service (it seems that is fails for every service)
Screenshots
Muted Areas (not the Command line part) are the name of the machine

System Configuration (please complete the following information):

OS: Windows
OS Version:  Windows 8.1 x64, Windows Server 2012, could be more. (not Win10)
Application Version: 2.2.40+0f97498320
CLI or GUI: CLI

Thanks for all your reports @a-gazin and for your patience.
I've identified the issue here.
We're pulling out all the fields in the WMI objects (based on Windows 10 documentation), without a try around each property fetch.  It looks like (at least) one of those properties isn't supported on Windows 8, throwing the error here. (https://github.com/microsoft/AttackSurfaceAnalyzer/blob/244012b9643f326097ea462955772e730a89f8cc/Lib/Collectors/ServiceCollector.cs#L62-L64)
I have a fix ready in #460 that I'll roll out tonight.

Thank you for the fix! and also, Thanks for having such a quick response time and also fix time!!


Updated to net8 and updated all the packages to latest

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (7a7052c) 92.01% compared to head (6fd4088) 91.98%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #89      +/-   ##
==========================================
- Coverage   92.01%   91.98%   -0.04%
==========================================
  Files          12       12
  Lines         664      661       -3
  Branches       46       46
==========================================
- Hits          611      608       -3
  Misses         33       33
  Partials       20       20

Flag
Coverage Δ

dotnet
91.98% <ø> (-0.04%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes


Move review profanity text

Description
This PR is related to Issue #229. The profanity text was moved from the title section to the input section.
Screenshots

Before

After (updated: 2/24)

Steps to verify/test this change:
Open the review modal by clicking the "+ Add Review" button on a course information page.
Final Checks:

[ ] Verify successful deployment
[ ] Delete branch

Moved the text below the input box as helper text.


[DO NOT MERGE] HSEARCH-5010 Upgrade to Lucene 9.9-SNAPSHOT

https://hibernate.atlassian.net/browse/HSEARCH-5010
Yeah, it was a change in the signature, now the skipping has more control:
/** Controls {@link LeafFieldComparator} how to skip documents */
public enum Pruning {
  /** Not allowed to skip documents. */
  NONE,
  /**
   * Allowed to skip documents that compare strictly better than the top value, or strictly worse
   * than the bottom value.
   */
  GREATER_THAN,
  /**
   * Allowed to skip documents that compare better than the top value, or worse than or equal to the
   * bottom value.
   */
  GREATER_THAN_OR_EQUAL_TO
}

Should I just push this branch to the main repo and create a similar one for lucene10 ?

Should I just push this branch to the main repo and create a similar one for lucene10 ?

To wip/* branches, you mean? Yes, please do.

And thanks :)

Lucene 10 has this additional change:
https://github.com/hibernate/hibernate-search/commit/714507e0c3f0d120967d4fe05f937bf4f4ac40fd#diff-6a2145db3f85bebf0c4b7c350cc349abc8e5b1acd4a80fd9da4fefde38210fe6
pushed both wip/* branches to the main repo so closing this PR 😃

@marko-bekhta Looks like there's still one failing test: https://ci.hibernate.org/job/hibernate-search-dependency-update/job/main/170/#showFailuresLink
Would you mind having a look, please?


cpu profile

Hi,
My GO application is running inside (one core dedicated to) docker container.
cpu profile is enabled. After the I see outfile.prof. There is nothing inside file.
Why is this behavior?
When I add two dedicated to core containers, outfile.prof has contains.
Thanks for your help
Best regards
sothy

Go has it's own profiler (but uses upstream pprof). Both are distinct from gperftools. Report your problem elsewhere.
If I'd guess what your problem is, I'd guess that profile isn't accumulated fast enough for profile file to be dumped before you check. If you give it enough time, I'm sure you'll see it. Also make sure your program exits normally. Not sure about Go, but gperftools do dump profile at end.


6.5.4 is missing from CHANGELOG.md

Documentation Related To Component:
CHANGELOG.MD
Please check those that apply

[ ] typo
[x] documentation doesn't exist
[ ] documentation needs clarification
[ ] error(s) in example
[ ] needs example

Description Of The Issue
The changelog entry for the 6.5.4 release appears to be missing.

It's in the 6.x branch: https://github.com/ReactiveX/rxjs/blob/6.x/CHANGELOG.md

Oh I missed that... Thank you! Should it be on master though?

Looks like it's resolved.

Looks like it's resolved.


Disable filtering for widgets

Story details: https://app.shortcut.com/cartoteam/story/193004
Added property filterable in every widget with filtering capabilities.
When filterable is false, widget's click event is disabled.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.01%) to 70.314%

Totals

Change from base Build<PHONE_NUMBER>:
0.01%

Covered Lines:
1142

Relevant Lines:
1529

💛  - Coveralls

@VictorVelarde totally agree. But I cannot tackle that right now since I don't have specific dedication for C4R, I'm sorry. I can make it in a future, maybe for the final release.

In my opinion, the carto-react-template example is not mandatory in this case, but updating the documentation (adding the property to the reference) is a must.

I've created this task for doc (https://app.shortcut.com/cartoteam/story/202069/widgets-document-new-filterable-prop-for-widgets), but I think we can already land this one


Deadline submit attr concurent tasks

We are not supporting settings of concurent tasks at the moment but those would be beneficial as sometimes rendering of nuke or other apps with only one thread is not efficiently using hardware resources.
In nuke write node at Deadline Tab we will add Amount of Concurent Tasks knob which will be set by default to 1. Settings will be also defined. In case settings will be defined to 10 and user will not change render node knob to anything, settings will be used. If user change nuke write knob to anything other then 1 then value for submission will be taken from here instead of openpype settings.

Could this not use the same workflow as the other Deadline settings which can be validated with the knobs validator?


[Feature Request]:  How to build agents automatically from User's request?

Is your feature request related to a problem? Please describe.
in autogen studio, is that possible it create necessary agents from user's request by autogen.Autobuilder?
Describe the solution you'd like
no
Additional context
no

@taozhiyuai can you give some examples to what that would look like?  It sounds like you are basically asking for a "llm to agent" code generator? I suspect the surface area for error will grow dramatically with this feature, but if the problem is constrained it might be easier to build.

@taozhiyuai can you give some examples to what that would look like?  It sounds like you are basically asking for a "llm to agent" code generator? I suspect the surface area for error will grow dramatically with this feature, but if the problem is constrained it might be easier to build.

in autogen api,there is autobuilder.what i need is this function.i type "plot 7 days ibm stock price graph.",for example. it will do the job


push active caches on a stack instead of overwriting them

This would save all the hassle of storing a result, executing a query, and restoring the previous result.
mysql_tquery(..., "MyCallback");

public MyCallback()
{
    new Cache:res = cache_save();

    for(new i; i != cache_num_rows(); i++)
    {
        FunctionThatCallsMysqlQuery();
        cache_set_active(res);
    }

    cache_delete(res);
}

FunctionThatCallsMysqlQuery()
{
	new Cache:res = mysql_query(...);

	cache_delete(res);
}

Basically restore the previous active cache after the cache_delete call in FunctionThatCallsMysqlQuery.
I have to make sure to not break any existing code though, so gotta think this through.

Yup!


Saved Template Wrong Keyboard Shortcut

Bug Description
When a saved template is in focus, pressing on Space key does multiple jobs, it opens the delete modal and fires scroll event at the same time. It should do one job at a time (either it should scroll or handle the delete modal).
IMO pressing on Delete/Backspace makes more sense to open the delete modal since the entire template gets focused and pressing on Space should handle the scroll.
Expected Behaviour
Pressing on Space button shouldn't fire multiple events at a time.
Steps to Reproduce

Go to Saved templates
Create 10+ templates
Focus on a template using the keyboard.
Press on Space key

Screenshots
https://user-images.githubusercontent.com/17360543/144630866-132a9292-0d19-4eaa-bf29-595a8cbfd04d.mov
Additional Context

Plugin Version: 1.16.0-alpha.0
WordPress Version: 5.8.2
Operating System: macOS Monterey (M1)
Browser: Chrome

Moving back to backlog since no active work has occurred on it since early dec.