feat: Control Net support

I implemented a Control Net here; it's somewhat similar to the encoder phase of stable diffusion, but with some convolutional layers. The implementation should be complete and should work (currently only on CPU). In the meantime, it seems that I am getting NaNs in the execution of ControlNet. I will be conducting some tests.
The resulting tensors yield very large values; it is likely that they are released while traversing the computation graph and reading residual memory.

For me it fails in a different way:
make -j && ./bin/sd --mode txt2img -p "a person" -m ../models/sd-v1-4.ckpt

[INFO]  stable-diffusion.cpp:6105 - loading model from '../models/sd-v1-4.ckpt'
[INFO]  model.cpp:641  - load ../models/sd-v1-4.ckpt using checkpoint format
[INFO]  stable-diffusion.cpp:6131 - Stable Diffusion 1.x
[INFO]  stable-diffusion.cpp:6137 - Stable Diffusion weight type: f32
[INFO]  stable-diffusion.cpp:6296 - total memory buffer size = 2731.37MB (clip 470.66MB, unet 2165.24MB, vae 95.47MB)
[INFO]  stable-diffusion.cpp:6298 - loading model from '../models/sd-v1-4.ckpt' completed, taking 9.89s
[INFO]  stable-diffusion.cpp:6312 - running in eps-prediction mode
[INFO]  stable-diffusion.cpp:5334 - loading control net from 'models/control_openpose-fp16.safetensors'
[INFO]  model.cpp:638  - load models/control_openpose-fp16.safetensors using safetensors format
ggml_tallocr_alloc: not enough space in the buffer (needed<PHONE_NUMBER>560, largest block available<PHONE_NUMBER>76)
GGML_ASSERT: /Users/ggerganov/development/github/stable-diffusion.cpp/ggml/src/ggml-alloc.c:116: !"not enough space in the buffer"
Abort trap: 6

Try specifying --type f16

Now it completes, but output.png is black

Option:
    n_threads:         16
    mode:              txt2img
    model_path:        ../models/sd-v1-4.ckpt
    wtype:             f16
    vae_path:
    taesd_path:
    esrgan_path:
    output_path:       output.png
    init_img:
    prompt:            a person
    negative_prompt:
    cfg_scale:         7.00
    clip_skip:         -1
    width:             512
    height:            512
    sample_method:     euler_a
    schedule:          default
    sample_steps:      20
    strength(img2img): 0.75
    rng:               cuda
    seed:              42
    batch_count:       1
    vae_tiling:        false
System Info:
    BLAS = 1
    SSE3 = 0
    AVX = 0
    AVX2 = 0
    AVX512 = 0
    AVX512_VBMI = 0
    AVX512_VNNI = 0
    FMA = 0
    NEON = 1
    ARM_FMA = 1
    F16C = 0
    FP16_VA = 1
    WASM_SIMD = 0
    VSX = 0
[DEBUG] stable-diffusion.cpp:6095 - Using CPU backend
[INFO]  stable-diffusion.cpp:6105 - loading model from '../models/sd-v1-4.ckpt'
[INFO]  model.cpp:641  - load ../models/sd-v1-4.ckpt using checkpoint format
[DEBUG] model.cpp:1142 - init from '../models/sd-v1-4.ckpt'
[INFO]  stable-diffusion.cpp:6131 - Stable Diffusion 1.x
[INFO]  stable-diffusion.cpp:6137 - Stable Diffusion weight type: f16
[DEBUG] stable-diffusion.cpp:6139 - loading vocab
[DEBUG] stable-diffusion.cpp:739  - vocab size: 49408
[DEBUG] stable-diffusion.cpp:6149 - ggml tensor size = 384 bytes
[DEBUG] stable-diffusion.cpp:1510 - clip params backend buffer size =  236.18 MB (449 tensors)
[DEBUG] stable-diffusion.cpp:2527 - unet params backend buffer size =  1650.16 MB (720 tensors)
[DEBUG] stable-diffusion.cpp:3586 - vae params backend buffer size =  95.47 MB (164 tensors)
[DEBUG] stable-diffusion.cpp:6166 - preparing memory for the weights
[DEBUG] stable-diffusion.cpp:6198 - loading weights
[DEBUG] model.cpp:1224 - loading tensors from ../models/sd-v1-4.ckpt
[DEBUG] stable-diffusion.cpp:6286 - model size = 1969.67MB
[INFO]  stable-diffusion.cpp:6296 - total memory buffer size = 1981.80MB (clip 236.18MB, unet 1650.16MB, vae 95.47MB)
[INFO]  stable-diffusion.cpp:6298 - loading model from '../models/sd-v1-4.ckpt' completed, taking 9.97s
[INFO]  stable-diffusion.cpp:6312 - running in eps-prediction mode
[DEBUG] stable-diffusion.cpp:6339 - finished loaded file
[INFO]  stable-diffusion.cpp:5334 - loading control net from 'models/control_openpose-fp16.safetensors'
[INFO]  model.cpp:638  - load models/control_openpose-fp16.safetensors using safetensors format
[DEBUG] model.cpp:704  - init from 'models/control_openpose-fp16.safetensors'
[DEBUG] stable-diffusion.cpp:5263 - control net params backend buffer size =  693.43 MB (347 tensors)
[DEBUG] model.cpp:1224 - loading tensors from models/control_openpose-fp16.safetensors
[INFO]  stable-diffusion.cpp:5402 - control net model loaded
[DEBUG] stable-diffusion.cpp:7218 - prompt after extract and remove lora: "a person"
[INFO]  stable-diffusion.cpp:7223 - apply_loras completed, taking 0.00s
[DEBUG] stable-diffusion.cpp:1457 - parse 'a person' to [['a person', 1], ]
[DEBUG] stable-diffusion.cpp:889  - split prompt "a person" to tokens ["a</w>", "person</w>", ]
[DEBUG] stable-diffusion.cpp:1624 - learned condition compute buffer size: 2.58 MB
[DEBUG] stable-diffusion.cpp:6462 - computing condition graph completed, taking 40 ms
[DEBUG] stable-diffusion.cpp:1457 - parse '' to [['', 1], ]
[DEBUG] stable-diffusion.cpp:889  - split prompt "" to tokens []
[DEBUG] stable-diffusion.cpp:1624 - learned condition compute buffer size: 2.58 MB
[DEBUG] stable-diffusion.cpp:6462 - computing condition graph completed, taking 38 ms
[INFO]  stable-diffusion.cpp:7262 - get_learned_condition completed, taking 80 ms
[INFO]  stable-diffusion.cpp:7278 - sampling using Euler A method
[INFO]  stable-diffusion.cpp:7282 - generating image: 1/1 - seed 42
[DEBUG] stable-diffusion.cpp:5616 - controlnet compute buffer size: 547.79 MB -> 13 control output tensors
[DEBUG] stable-diffusion.cpp:2894 - diffusion compute buffer size: 559.71 MB
  |>                                                 | 0/20 - 0.00it/s[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |==>                                               | 1/20 - 4.14s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |=====>                                            | 2/20 - 3.96s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |=======>                                          | 3/20 - 3.95s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |==========>                                       | 4/20 - 4.00s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |============>                                     | 5/20 - 3.96s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |===============>                                  | 6/20 - 4.04s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |=================>                                | 7/20 - 4.00s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |====================>                             | 8/20 - 3.86s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |======================>                           | 9/20 - 4.01s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |=========================>                        | 10/20 - 3.96s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |===========================>                      | 11/20 - 4.00s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |==============================>                   | 12/20 - 3.96s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |================================>                 | 13/20 - 3.90s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |===================================>              | 14/20 - 3.92s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |=====================================>            | 15/20 - 3.94s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |========================================>         | 16/20 - 4.02s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |==========================================>       | 17/20 - 4.00s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |=============================================>    | 18/20 - 3.87s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |===============================================>  | 19/20 - 3.95s/it[DEBUG] stable-diffusion.cpp:6606 - control net start
[DEBUG] stable-diffusion.cpp:6608 - control net end
  |==================================================| 20/20 - 4.00s/it
[INFO]  stable-diffusion.cpp:7294 - sampling completed, taking 79.55s
[INFO]  stable-diffusion.cpp:7302 - generating 1 latent images completed, taking 79.57s
[INFO]  stable-diffusion.cpp:7304 - decoding 1 latents
[DEBUG] stable-diffusion.cpp:3717 - vae compute buffer size: 1664.00 MB
[DEBUG] stable-diffusion.cpp:7113 - computing vae [mode: DECODE] graph completed, taking 5.24s
[INFO]  stable-diffusion.cpp:7314 - latent 1 decoded, taking 5.24s
[INFO]  stable-diffusion.cpp:7318 - decode_first_stage completed, taking 5.24s
[INFO]  stable-diffusion.cpp:7335 - txt2img completed in 84.90s
[INFO]  main.cpp:541  - save result image to 'output.png'

Try running it in debug mode.

@ggerganov  It was a variable that was causing an overflow. With debug mode, I managed to detect it, although NaNs (Not a Number) are still being generated somewhere in the computation graph, anyway thank you for your time. I am going to check.

@ggerganov Is what I'm doing correct when extracting data from collected tensors into a vector? I suspect I might be reading a memory address with residual data.

@leejet can you test this?

@leejet can you test this?

@leejet can you test this?

@FSSRepo Where did you download the ControlNet model from? When I used the official provided weights from https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main, the tensor names didn't match.

@leejet this is controlnet v1.0 f16 models, link

@leejet those pth models have the prefix control_model. that's why they have invalid names.

@leejet Control Net for stable diffusion 1.5:
Control Net v1 Control Net v1.1

It looks like ControlNet is in effect.
.\bin\Release\sd.exe -m ..\..\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned-emaonly.safetensors -p "a person" --control-net  ..\models\control_openpose-fp16.safetensors  --control-image ..\assets\control.png -v

omg, u guys are bringing everything to sd cpp! thank u for all ur hard work, can't wait for inpainting and outpainting too be a part of controlnet

@leejet Ready, the official pth models should also be working now. I should emphasize that they also include other t2i models; those are a different architecture. They make changes to the input blocks, while ControlNet makes changes to the output blocks and the middle block.

Great job, and thank you for your contributions. I'll find some time to review and merge this PR.

@leejet All working correctly

@FSSRepo you can remove the async+sync cuda hack when https://github.com/ggerganov/ggml/pull/679 gets merged.

@Green-Sky Does it also solve the VAE tiling problem?

@Green-Sky Could you try this pull request if it works correctly for you? Also, if you try removing this #ifdef.
__STATIC_INLINE__ void ggml_backend_tensor_get_and_sync(ggml_backend_t backend, const struct ggml_tensor* tensor, void* data, size_t offset, size_t size) {
  // delete this to use just ggml_backend_tensor_get
#ifdef SD_USE_CUBLAS
    if(!ggml_backend_is_cpu(backend)) {
        ggml_backend_tensor_get_async(backend, tensor, data, offset, size);
        ggml_backend_synchronize(backend);
    } else {
        ggml_backend_tensor_get(tensor, data, offset, size);
    }
#else
    ggml_backend_tensor_get(tensor, data, offset, size);
#endif
}

with and without vae tiling

@FSSRepo control net works very well for the first image after the model was loaded. Is it possible to make it work for more than one image?

@Green-Sky Does it also solve the VAE tiling problem?

which one? the neony artifacts or the "some tiles look lower resolution" ?
I ran the branch rn and it appears the tiling is not executed? despite it being set.
    vae_tiling:        true

[DEBUG] ggml_extend.hpp:603  - clip compute buffer size: 2.58 MB
[DEBUG] stable-diffusion.cpp:460  - computing condition graph completed, taking 8 ms
[INFO ] stable-diffusion.cpp:1266 - get_learned_condition completed, taking 28 ms
[INFO ] stable-diffusion.cpp:1282 - sampling using Euler A method
[INFO ] stable-diffusion.cpp:1286 - generating image: 1/1 - seed 42
[DEBUG] ggml_extend.hpp:603  - unet compute buffer size: 2137.25 MB
  |==================================================| 20/20 - 1.11it/s
[INFO ] stable-diffusion.cpp:1298 - sampling completed, taking 18.37s
[INFO ] stable-diffusion.cpp:1306 - generating 1 latent images completed, taking 18.37s
[INFO ] stable-diffusion.cpp:1308 - decoding 1 latents
[DEBUG] ggml_extend.hpp:603  - vae compute buffer size: 3329.00 MB
[DEBUG] stable-diffusion.cpp:1111 - computing vae [mode: DECODE] graph completed, taking 1.49s
[INFO ] stable-diffusion.cpp:1318 - latent 1 decoded, taking 1.49s
[INFO ] stable-diffusion.cpp:1322 - decode_first_stage completed, taking 1.49s
[INFO ] stable-diffusion.cpp:1339 - txt2img completed in 19.89s

... if however I am wrong and the tiling is executed, then it's perfect now. Even the seams would be gone.

@Green-Sky you can try again

@Green-Sky you can try again

toggling this section does not make a difference:
#ifdef SD_USE_CUBLAS
    if(!ggml_backend_is_cpu(backend)) {
        ggml_backend_tensor_get_async(backend, tensor, data, offset, size);
        ggml_backend_synchronize(backend);
    } else {
        ggml_backend_tensor_get(tensor, data, offset, size);
    }
#else
    ggml_backend_tensor_get(tensor, data, offset, size);
#endif

as is:

without:

at least it looks like it deterministic. (besides github deciding randomly how large an image is displayed/processed)

also, if the tiles are overlapped, no steams should be visible.
(weak src: https://github.com/madebyollin/taesd/issues/8#issuecomment-1675992525 )

When conducting tests, I am also obtaining results that seem to be of low resolution. I'm not sure what the cause might be.

Segmentation with control_seg-fp16 is also working very well.

The missing part is to create the preprocessors. But that is cumbersome and somewhat complicated, as it requires implementing other models (image2depth and openpose) and algorithms to perform Canny (edge detection).

Not sure, if this is of any help. But I tried to do canny edge detection with openCv and it works quite well.
This is the code that I use for Open Frameworks:
						image.load(result.getPath());
						int lowThreshold = 50;
						const int max_lowThreshold = 100;
						const int ratio = 3;
						const int kernel_size = 3;
						cv::Mat src, src_gray;
						cv::Mat detected_edges;
						src = ofxCv::toCv(image);
						cv::cvtColor(src, src_gray, cv::COLOR_BGR2GRAY);
						cv::blur(src_gray, detected_edges, cv::Size(3, 3));
						cv::Canny(detected_edges, detected_edges, lowThreshold, lowThreshold* ratio, kernel_size);
						ofxCv::toOf(detected_edges, image);
						image.update();

And here a general example: https://docs.opencv.org/4.x/da/d5c/tutorial_canny_detector.html

That implies using very heavy libraries that are not really necessary for the functioning of this project itself; it's better to have it integrated from scratch in ggml.

That implies using very heavy libraries that are not really necessary for the functioning of this project itself; it's better to have it integrated from scratch in ggml.

True (at least sam.cpp works with ggml, but SamAutomaticMaskGenerator, which is needed for segmenting, is not part of sam.cpp yet).

In the past few weeks, I have been busy with pull requests related to svd, so I didn't have a chance to merge this PR. Now that I've addressed some issues, I'm ready to merge this PR. Thank you for your contribution.

@leejet Thank you for continuing to maintain the project despite having other jobs. I'll see if in the next few days, when I have time to implement flash attention in llama.cpp, I can adapt the web UI that I already have created to the project.

Would controlnet with SDXL work with this update?

I ran the above command and got the following error message, can anyone help me?
root@c-66fe0199-152fc2e0-851f6bf93291:/tmp# ./sd --mode txt2img -p "a person" -m /mnt/auto/sd-v1-4.ckpt --control-net /mnt/auto/sd/control_openpose-fp16.safetensors  --control-image /mnt/auto/sd/openposeimg.png  --control-net-cpu
[INFO ] stable-diffusion.cpp:195  - loading model from '/mnt/auto/sd-v1-4.ckpt'
[INFO ] model.cpp:796  - load /mnt/auto/sd-v1-4.ckpt using checkpoint format
[INFO ] stable-diffusion.cpp:235  - Version: SD 1.x
[INFO ] stable-diffusion.cpp:266  - Weight type:                 f32
[INFO ] stable-diffusion.cpp:267  - Conditioner weight type:     f32
[INFO ] stable-diffusion.cpp:268  - Diffusion model weight type: f32
[INFO ] stable-diffusion.cpp:269  - VAE weight type:             f32
[INFO ] control.hpp:434  - loading control net from '/mnt/auto/sd/control_openpose-fp16.safetensors'
[INFO ] model.cpp:793  - load /mnt/auto/sd/control_openpose-fp16.safetensors using safetensors format
[INFO ] control.hpp:453  - control net model loaded
[INFO ] stable-diffusion.cpp:482  - total params memory size = 3656.84MB (VRAM 937.60MB, RAM 2719.24MB): clip 469.44MB(RAM), unet 2155.33MB(RAM), vae 94.47MB(RAM), controlnet 937.60MB(VRAM), pmid 0.00MB(RAM)
[INFO ] stable-diffusion.cpp:501  - loading model from '/mnt/auto/sd-v1-4.ckpt' completed, taking 15.44s
[INFO ] stable-diffusion.cpp:528  - running in eps-prediction mode
[INFO ] stable-diffusion.cpp:655  - Attempting to apply 0 LoRAs
[INFO ] stable-diffusion.cpp:1132 - apply_loras completed, taking 0.00s
[INFO ] stable-diffusion.cpp:1256 - get_learned_condition completed, taking 448 ms
[INFO ] stable-diffusion.cpp:1279 - sampling using Euler A method
[INFO ] stable-diffusion.cpp:1283 - generating image: 1/1 - seed 42
/root/software/stable-diffusion.cpp-build/build/_deps/sd-src/ggml/src/ggml.c:4508: GGML_ASSERT(ggml_can_repeat(b, a)) failed
./sd(+0x1597db)[0x5570d32957db]
./sd(+0x15b821)[0x5570d3297821]
./sd(+0x181258)[0x5570d32bd258]
./sd(+0xea1e2)[0x5570d32261e2]
./sd(+0xec054)[0x5570d3228054]
./sd(+0xacbe5)[0x5570d31e8be5]
./sd(+0xb78e5)[0x5570d31f38e5]
./sd(+0xc21d0)[0x5570d31fe1d0]
./sd(+0xa15ea)[0x5570d31dd5ea]
./sd(+0xa3368)[0x5570d31df368]
./sd(+0x1aed7)[0x5570d3156ed7]
/usr/lib/x86_64-linux-gnu/libc.so.6(+0x2dfd0)[0x7f92772aafd0]
/usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x7d)[0x7f92772ab07d]
./sd(+0x22f55)[0x5570d315ef55]
Aborted (core dumped)
root@c-66fe0199-152fc2e0-851f6bf93291:/tmp# ./sd --mode txt2img -p "a person" -m /mnt/auto/sd-v1-4.ckpt --control-net /mnt/auto/sd/control_openpose-fp16.safetensors  --control-image /mnt/auto/sd/openposeimg.png  --control-net-cpu --type f16 -v
Option:
n_threads:         4
mode:              txt2img
model_path:        /mnt/auto/sd-v1-4.ckpt
wtype:             f16
clip_l_path:
t5xxl_path:
diffusion_model_path:
vae_path:
taesd_path:
esrgan_path:
controlnet_path:   /mnt/auto/sd/control_openpose-fp16.safetensors
embeddings_path:
stacked_id_embeddings_path:
input_id_images_path:
style ratio:       20.00
normalize input image :  false
output_path:       output.png
init_img:
control_image:     /mnt/auto/sd/openposeimg.png
clip on cpu:       false
controlnet cpu:    true
vae decoder on cpu:false
strength(control): 0.90
prompt:            a person
negative_prompt:
min_cfg:           1.00
cfg_scale:         7.00
guidance:          3.50
clip_skip:         -1
width:             512
height:            512
sample_method:     euler_a
schedule:          default
sample_steps:      20
strength(img2img): 0.75
rng:               cuda
seed:              42
batch_count:       1
vae_tiling:        false
upscale_repeats:   1
System Info:
BLAS = 0
SSE3 = 1
AVX = 1
AVX2 = 1
AVX512 = 1
AVX512_VBMI = 0
AVX512_VNNI = 0
FMA = 1
NEON = 0
ARM_FMA = 0
F16C = 1
FP16_VA = 0
WASM_SIMD = 0
VSX = 0
[DEBUG] stable-diffusion.cpp:180  - Using CPU backend
[INFO ] stable-diffusion.cpp:195  - loading model from '/mnt/auto/sd-v1-4.ckpt'
[INFO ] model.cpp:796  - load /mnt/auto/sd-v1-4.ckpt using checkpoint format
[DEBUG] model.cpp:1320 - init from '/mnt/auto/sd-v1-4.ckpt'
[INFO ] stable-diffusion.cpp:235  - Version: SD 1.x
[INFO ] stable-diffusion.cpp:266  - Weight type:                 f16
[INFO ] stable-diffusion.cpp:267  - Conditioner weight type:     f16
[INFO ] stable-diffusion.cpp:268  - Diffusion model weight type: f16
[INFO ] stable-diffusion.cpp:269  - VAE weight type:             f16
[DEBUG] stable-diffusion.cpp:271  - ggml tensor size = 400 bytes
[DEBUG] clip.hpp:171  - vocab size: 49408
[DEBUG] clip.hpp:182  -  trigger word img already in vocab
[DEBUG] ggml_extend.hpp:1046 - clip params backend buffer size =  235.06 MB(RAM) (196 tensors)
[DEBUG] ggml_extend.hpp:1046 - unet params backend buffer size =  1640.25 MB(RAM) (686 tensors)
[DEBUG] ggml_extend.hpp:1046 - vae params backend buffer size =  94.47 MB(RAM) (140 tensors)
[DEBUG] stable-diffusion.cpp:398  - loading weights
[DEBUG] model.cpp:1530 - loading tensors from /mnt/auto/sd-v1-4.ckpt
[INFO ] control.hpp:434  - loading control net from '/mnt/auto/sd/control_openpose-fp16.safetensors'
[DEBUG] ggml_extend.hpp:1046 - control_net params backend buffer size =  689.48 MB(RAM) (340 tensors)
[INFO ] model.cpp:793  - load /mnt/auto/sd/control_openpose-fp16.safetensors using safetensors format
[DEBUG] model.cpp:861  - init from '/mnt/auto/sd/control_openpose-fp16.safetensors'
[DEBUG] model.cpp:1530 - loading tensors from /mnt/auto/sd/control_openpose-fp16.safetensors
[INFO ] control.hpp:453  - control net model loaded
[INFO ] stable-diffusion.cpp:482  - total params memory size = 2659.26MB (VRAM 689.48MB, RAM 1969.78MB): clip 235.06MB(RAM), unet 1640.25MB(RAM), vae 94.47MB(RAM), controlnet 689.48MB(VRAM), pmid 0.00MB(RAM)
[INFO ] stable-diffusion.cpp:501  - loading model from '/mnt/auto/sd-v1-4.ckpt' completed, taking 15.38s
[INFO ] stable-diffusion.cpp:528  - running in eps-prediction mode
[DEBUG] stable-diffusion.cpp:572  - finished loaded file
[DEBUG] stable-diffusion.cpp:1378 - txt2img 512x619
[DEBUG] stable-diffusion.cpp:1127 - prompt after extract and remove lora: "a person"
[INFO ] stable-diffusion.cpp:655  - Attempting to apply 0 LoRAs
[INFO ] stable-diffusion.cpp:1132 - apply_loras completed, taking 0.00s
[DEBUG] conditioner.hpp:325  - parse 'a person' to [['a person', 1], ]
[DEBUG] clip.hpp:311  - token length: 77
[DEBUG] ggml_extend.hpp:998  - clip compute buffer size: 1.40 MB(RAM)
[DEBUG] conditioner.hpp:453  - computing condition graph completed, taking 148 ms
[DEBUG] conditioner.hpp:325  - parse '' to [['', 1], ]
[DEBUG] clip.hpp:311  - token length: 77
[DEBUG] ggml_extend.hpp:998  - clip compute buffer size: 1.40 MB(RAM)
[DEBUG] conditioner.hpp:453  - computing condition graph completed, taking 145 ms
[INFO ] stable-diffusion.cpp:1256 - get_learned_condition completed, taking 295 ms
[INFO ] stable-diffusion.cpp:1279 - sampling using Euler A method
[INFO ] stable-diffusion.cpp:1283 - generating image: 1/1 - seed 42
/root/software/stable-diffusion.cpp-build/build/_deps/sd-src/ggml/src/ggml.c:4508: GGML_ASSERT(ggml_can_repeat(b, a)) failed
./sd(+0x1597db)[0x5565d2f8d7db]
./sd(+0x15b821)[0x5565d2f8f821]
./sd(+0x181258)[0x5565d2fb5258]
./sd(+0xea1e2)[0x5565d2f1e1e2]
./sd(+0xec054)[0x5565d2f20054]
./sd(+0xacbe5)[0x5565d2ee0be5]
./sd(+0xb78e5)[0x5565d2eeb8e5]
./sd(+0xc21d0)[0x5565d2ef61d0]
./sd(+0xa15ea)[0x5565d2ed55ea]
./sd(+0xa3368)[0x5565d2ed7368]
./sd(+0x1aed7)[0x5565d2e4eed7]
/usr/lib/x86_64-linux-gnu/libc.so.6(+0x2dfd0)[0x7f146d7d0fd0]
/usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x7d)[0x7f146d7d107d]
./sd(+0x22f55)[0x5565d2e56f55]
Aborted (core dumped)


Fix "first Friday of next month" returning Friday of next week

This commit only fixes bug "first Friday of next month" returning Friday of next week, JS is corrected accordingly. Some basic tests are included in Specs, and have passed.


[stable/RabbitMQ-HA] Restarting all nodes of a cluster doesn't result in a healthy cluster

I was trying to restart my RabbitMQ cluster after deploying a new version of the chart. I deleted all pods to let Kubernets create new pods of the new version. RabbitMQ expects that the last node to go down, will be the first that will come up. Unfortunately, Kubernetes started another node first. That node couldn't connect to the cluster and went down again. Rinse en repeat.
I found the setting for a statefullset to start all pods at once and thus could restart my cluster.

Use a rollingupdate to only bring down one at a time and allow RabbitMQ to maintain a quorum.


[FLOC-1490, FLOC-1494] Initial IBlockDeviceAPI and an initial Loopback implementation for creating, attaching and listing block devices

Fixes https://clusterhq.atlassian.net/browse/FLOC-1490
Fixes https://clusterhq.atlassian.net/browse/FLOC-1494
This introduces an (incomplete) interface for block-device backends.  It also implements that interface using a local-only loopback block device provider.  This was split out of a larger branch which also includes new IDeployer implementation that knows how to use this new interface to provide block-device-based storage for a datasets convergence agent (that code to be up for review shortly, you can see it in https://github.com/ClusterHQ/flocker/tree/iblockdevice-deployer-FLOC-1502 (currently contains all of this code and some more code besides)).
The new interface and implementation also covers the "list existing block devices" functionality that's required but there's no separate JIRA issue for that work.

Looking forward to seeing the whole thing! Please address and resubmit.

Thanks @itamarst
I addressed or answered all your comments, I think.
Ready for another review.


Support for pixi v5

I was wondering, because of migration issues, if there will be support for pixi v5 when it's released?

As soon as pixi-v5 beta.1 gets released. soon.

Btw, if you use only zIndex inside single container, v5 has it in the package.

Thanks for the fast and helpful response.


Shouldn't we 400 if the JSON is invalid?

Shouldn't we 400 if the JSON is invalid? Right now, json_decode will just return a null on this line:
https://github.com/zendframework/zend-expressive-helpers/blob/master/src/BodyParams/JsonStrategy.php#L32

@rodmcnew Thank you, but I'm closing this in favor of #12, which reports the same issue and was opened first. If you have any input on the proposed solution or any interest in working on it, please comment there.


Disabling Validation Inference

There should be a way to disable the validation parts between each epoch in the colab. For massive datasets, it can take up to 20 minutes to generate those samples. Being able to disable it would save a lot of time.

Hey, sorry for the delay.  Thanks for bringing this up, will look into it.

Just merged a fix!


Add an option to run gradio web demo with very low vram

What
Add an option to very low VRAM usage.
Description
Add an option to use the GPU in a sequential offload mode so we  are able to run "THUDM/CogVideoX-5b"
on my RTX 3060 12GB.
I saw it taking less than 7GB of VRAM.
Message to the developers
This project is great. The best local video generation I saw so far.

I'm closing due:
https://github.com/THUDM/CogVideo/pull/206
I saw after I created the code


Allow disabling strict checking of Workflow operator outputs vs schema

This allows disabling strict checking of operator outputs against operator output schemas, which may allow some Workflows to run in cases where they otherwise wouldn't. This TypeError was introduced in order to catch cases where operators were fibbing about their own output, but we haven't fully been able to resolve all of those cases, so this provides an escape hatch that may help in some cases. Setting strict=False here is an acknowledgement that downstream code (e.g. in Merlin Models) may not run as expected in all cases (for example due to dtype discrepancies), but I suspect we may still be okay in many cases, so this gives another option to try.
Related to #1580 (though not an actual fix for the issue reported therein)

I appreciate the help.  Thank you.

Fundamentally, this change is awesome 🙂 I really like this way of thinking where we are saying "okay, feel free to use the underlying machinery at your own risk. We provide you all this functionality, but if you want to step off the beaten track, feel free to do so". We are dealing with the fact that NVTabular is built on top of these other libraries very elegantly here.
So I would like to see this change merged.
But if the #1589 would get merged at some point (which I also think is important, we are significantly expanding the set of aggregations that we support and cast floats64 to floats32 which makes a lot of sense, it's easy to eat up a lot of memory with these aggregations really quickly), so if #1589 gets merged, the unit test here will stop raising an error and will fail.
ATM I don't see a good way to test the functionality in this PR (unless we find some other operator or aggregation that we currently don't support and don't plan to support in near future, or can play a trick on the operator and amend the schema mid-flight). In fact, same goes for #1589 -- I think the functionality is good there, it is just the additional test that is problematic.

Maybe even with #1589 getting merged this could be tested via doing a list aggregation of floats32 on the CPU? 🤔 This would be a genuine scenario that I am aware of which wouldn't run without strict=False.

@radekosmulski Thanks for the comments, I think you're entirely on target there! Re: PR etiquette etc, I'm really digging your thorough and well-thought out comments. The only thing I'd mention is that for small changes, you can create suggestions from the comment box with the button that has the "piece of paper plus/minus" icon, which lets you directly propose code changes the author can incorporate with a button click. I used it to make the change you proposed by creating a suggestion on my own PR, but we could skip a step. 😺

Thank you very much @karlhigley, great to hear! 🙂 I also really appreciate the bouncing of ideas back and forth and the discussion working on this together 🙂
Good to know about the ability to add changes that can immediately be committed from the web GUI! Could I also please ask you one more question? On #1580 I squashed my commits when I thought I was done, but I saw this kills the earlier discussion a little bit (some earlier code comments disappear, etc, which I don't think happens if you just keep pushing ). Aaaah and now I see the web GUI gives you an option to squash and commit at the end?
So if I am reading this right, the workflow would be to just keep piling commits on top of each other and then squashing and merging via the web GUI?
Thank you very much! 🙂

@radekosmulski I am also normally inclined to clean up my commits, but since all PRs get squashed before merging, I've mostly stopped. Piling commits on top of each other seems to be the preferred workflow.

Thank you for your answer @karlhigley! Good to know! 🙂

@karlhigley , please link this to an initiative

This PR is obsolete after the extraction of executors into Core


Make client_secret optional for password grant type

Add tests and flag
fix up test name (clientId > clientSecret)

Completion of https://github.com/oauthjs/node-oauth2-server/pull/312

Updated!

Updated and rebased!

LGTM

@mjsalinger this is incredibly useful, can we get this rolled out as 3.0.0-b3.2 soon?


Add URL share button

This button should use the fa icon fa-share-alt, and it should be present both in city-view and in city-comparison components.

Which button?

@christaylor19 there is no button yet. There should be some kind of button (or link), with the fa-share-alt icon (without text), somewhere in the city-view and in the city-comparison components.
For example, a suitable place may be next to the compare link (to the right), here:

It should be something like <i className="fas fa-share-alt">, wrapped by an anchor tag (or button). There might be already in npm a component for sharing in the social media. So the link/button should open this component (maybe a popup?) and from there the user should be able to share the current URL.


sonos --actions pipe-able

Greetings, pwt!
Is it possible to make sonos --actions pipe-able?
For example, it would be nice to be able to do:
sonos --actions | grep "speaker" or sonos --actions | tee actions.txt
For me, these commands produce the following error:
Traceback (most recent call last):   File "/usr/local/bin/sonos", line 8, in <module>     sys.exit(main())   File "/usr/local/lib/python3.9/site-packages/soco_cli/sonos.py", line 151, in main     list_actions()   File "/usr/local/lib/python3.9/site-packages/soco_cli/action_processor.py", line 2589, in list_actions     items_per_line = get_terminal_size().columns // item_spacing OSError: [Errno 25] Inappropriate ioctl for device
I am guessing this is related to sub-processes created by the api and the inability to do i/o to/from the sub-process.
Session Info:
I am operating on an M1 macbook pro, but have installed soco-cli under an x86_64 brew installation.
soco-cli version:   0.4.46 soco version:       0.26.3 python version:     3.9.10 command path:       /usr/local/bin/sonos
As always, many thanks.

Thanks for reporting this.
It's not a subprocess issue. What's breaking is the get_terminal_size() call, which is used to determine the width of the terminal window, in order to print the actions neatly in columns. That doesn't work when piped.
I've fixed this now (it will print a simple vertical list), e.g.:
% sonos --actions | grep vol
volume
vol
relative_volume
rel_vol
ramp_to_volume
group_volume_equalize
group_volume_equalise
group_volume
group_vol
group_relative_volume
group_rel_vol
fixed_volume

I have some other pending changes I want to add, so I'll hold this fix back until the release of v0.4.48.

Fantastic. Thanks so much!
On Mon, Mar 14, 2022 at 11:09 AM pwt @.***> wrote:

Thanks for reporting this.
It's not a subprocess issue. What's breaking is the get_terminal_size()
call, which is used to determine the width of the terminal window, in order
to print the actions neatly in columns. That doesn't work when piped.
I've fixed this now (it will print a simple vertical list), e.g.:
% sonos --actions | grep vol
volume
vol
relative_volume
rel_vol
ramp_to_volume
group_volume_equalize
group_volume_equalise
group_volume
group_vol
group_relative_volume
group_rel_vol
fixed_volume
I have some other pending changes I want to add, so I'll hold this fix
back until the release of v0.4.48.
—
Reply to this email directly, view it on GitHub
https://github.com/avantrec/soco-cli/issues/32#issuecomment-1066918976,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAXHV2Z2VJ3CPS5PG3F5DMDU75JB3ANCNFSM5QVMDZRQ
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
@.*** | w:617-513-2509 | c:908-873-8195


OAuth 2.0 documents

Hi, UPS team
Where is the OAuth 2.0 documents? I can see the alert on UPS official site which remind me to finish auth migration, but I can't find the documents.
https://wwwcie.ups.com/security/v1/oauth/token is the correct url for OAuth 2.0?

Hi, thank you for your comment. Here are some of the UPS Developer Portal resources for OAuth 2.0: https://developer.ups.com/get-started & https://developer.ups.com/oauth-developer-guide.
Here are the URLs, also available in the OAuth OpenAPI specs:
Customer Integration Environment URL:  https://wwwcie.ups.com/security/v1/oauth/token
Production URL: https://onlinetools.ups.com/security/v1/oauth/token


Deutsch-Josa algorithm content to qsharp

adding deutsch-josa content to new branch

Closing temporarily to have a good view of PRs in progress. We can reopen when it is ready for review.


Testing, please ignore. (yxmzev)

This is a bug bounty test.. Please DO NOT APPROVE THIS! (yxmzev)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.yxmzev nkdio seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Added basic crypt checkin url

Added a checkin url to spoof a crypt server. This will respond with a 200 to stop the crypt client from attempting to escrow every 2 minutes.

Should be accessible at munkireporturl/index.php?/module/filevault_escrow/checkin
one would use ``munkireporturl/index.php?/module/filevault_escrow/ormunkireporturl/index.php?/module/filevault_escrow` as the serverurl for crypt

May want to update the README as well

🤦♂ I'll get on that.


<EMAIL_ADDRESS>- cannot find module @azure/identity

Environment

<EMAIL_ADDRESS>in npm
macOS 12.5.1
nodejs 14.19.2

Describe the bug
<EMAIL_ADDRESS>(the latest), when require'd, has a Cannot find module error for @azure/identity.
Error: Cannot find module '@azure/identity'
Require stack:
-<EMAIL_ADDRESS>
@azure/identity is only a devDependency: https://github.com/Azure/azure-sdk-for-js/blob/57387dcc64a57d7e1d524291752c6c576af28ef7/sdk/cosmosdb/cosmos/package.json#L110
To Reproduce
Steps to reproduce the behavior:

mkdir test
cd test
npm init -y
npm i<EMAIL_ADDRESS>echo 'require("@azure/cosmos")' > index.js
node index.js

Expected behavior
No errors when importing/requiring the @azure/cosmos module.
Actual behavior
Cannot find module @azure/identity error encountered when importing/requiring the @azure/cosmos module.
Workaround
Pin to<EMAIL_ADDRESS>

@jay-most

Looks like this was introduced in https://github.com/Azure/azure-sdk-for-js/commit/cc267fe571644752785c3b562cbe1abae2bf3e2b
For some reason the logger from identity is being used: https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/cosmosdb/cosmos/src/request/request.ts#L3
instead of the cosmos logger: https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/cosmosdb/cosmos/src/common/logger.ts
perhaps VS Code autocompleted the import in a silly way?

/cc @sajeetharan

@jay-most Looks like a blocking issue, can you please check asap?

I can confirm that with the repro steps above with v3.17.1, the issue is resolved.

@shazron thanks for confirming!


Dismiss needs a completion block

Since ASDepthModal changes the rootViewController of the navigationController, we need some type of call back after calling "dismiss" to let us know when the original rootViewController is restored.

PR welcome!


[Documentation] Disclaimer Texts

In finalizing the application, we need legal texts to show to the user. These texts include,

Login text (The first screen the user sees)
Consent text (One-time-seen legal documentation)
Sensor usage approval text (One-time-seen for each sensor after their selection)

The texts can be found in the document,
IDP - Disclaimer Text
@damidfm Could you please update this document so that we can put them into the app before releasing the beta versions for testing?

Texts added, thanks!


added new contributor 0031

Added new contributor 0031 Alexandra Seibicke

Thanks, I'll take care of merging.


connectivity: Add "short" suite

Fix https://github.com/cilium/cilium-cli/issues/1735.

I wonder if the desired effect could be achieved by using the --test flag to filter in/out what you want to test in your specific test runs:
    --test strings    Run tests that match one of the given regular expressions, skip tests by starting the expression with '!', target Scenarios with e.g. '/pod-to-cidr'

I wonder if the desired effect could be achieved by using the --test flag to filter in/out what you want to test in your specific test runs:

Yep, good idea. I'm giving a try in https://github.com/cilium/cilium/pull/29073.


libc: bump to 0.2.43

What have you changed? (mandatory)
Bump libc to 0.2.43
What are the type of the changes? (mandatory)
Improvement
How has this PR been tested? (mandatory)
Not required
Does this PR affect documentation (docs/docs-cn) update? (mandatory)
No
Does this PR affect tidb-ansible update? (mandatory)
No

is that for some purpose?

I agree with @solotzg that this version bump should be carried when rust-rocksdb is upgraded. Seems that we can just close it for now?

Never mind. If you feel like we should close it, just close it.


Memory leak when using Nokogiri::XML::Builder with XML namespaces in nested elements

What problems are you experiencing?
Maybe related or similar to #1771. Memory leak when using Nokogiri::XML::Builder and namespaces. Only namespace definitions beginning with xmlns: cause the leak (because Nokogiri::XML::Document#create_element is implemented that way), and nesting seems to be important too. I guess  Nokogiri::XML::Node#add_namespace_definition doesn't free that definition when the node is garbage collected.
What's the output from nokogiri -v?
# Nokogiri (1.8.5)
    ---
    warnings: []
    nokogiri: 1.8.5
    ruby:
      version: 2.5.1
      platform: x86_64-linux
      description: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
      engine: ruby
    libxml:
      binding: extension
      source: packaged
      libxml2_path: "/home/p/.gem/ruby/2.5.1/gems/nokogiri-1.8.5/ports/x86_64-pc-linux-gnu/libxml2/2.9.8"
      libxslt_path: "/home/p/.gem/ruby/2.5.1/gems/nokogiri-1.8.5/ports/x86_64-pc-linux-gnu/libxslt/1.1.32"
      libxml2_patches:
      - 0001-Revert-Do-not-URI-escape-in-server-side-includes.patch
      - 0002-Fix-nullptr-deref-with-XPath-logic-ops.patch
      - 0003-Fix-infinite-loop-in-LZMA-decompression.patch
      libxslt_patches: []
      compiled: 2.9.8
      loaded: 2.9.8

Can you provide a self-contained script that reproduces what you're seeing?
require 'nokogiri'

puts "Nokogiri version: #{Nokogiri::VERSION}"
puts "PID: #{$$}"

NS = {
  'xmlns:env': 'http://schemas.xmlsoap.org/soap/envelope/'
}

100.times do |i|
  puts "##{i}"
  5_000.times do
    Nokogiri::XML::Builder.new do |xml|
      #xml.send 'env:Envelope'
      xml.send 'Envelope', nil, NS do |xml|
        xml.send 'Foobar', nil, NS # no leak without this inner call
      end
    end
  end
  GC.start
end

When I run this script, its RSS grows continually from ~12 MB up to 60 MB before it exits.

The script above is flawed, as the elements don't actually reference a namespace.  Will post the results as soon as it's finished.

Here are the few largest definitely lost sections related to Nokogiri (largest first):
==4581== 20,995,506 bytes in 499,893 blocks are definitely lost in loss record 8,434 of 8,434
==4581==    at 0x40307FF: malloc (in /home/p/.linuxbrew/Cellar/valgrind/3.14.0/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4581==    by 0x4AED402: objspace_xmalloc0 (gc.c:7921)
==4581==    by 0x4AED667: ruby_xmalloc0 (gc.c:7989)
==4581==    by 0x4AED696: ruby_xmalloc (gc.c:7998)
==4581==    by 0x903AEF8: xmlStrndup (xmlstring.c:45)
==4581==    by 0x903AF98: xmlStrdup (xmlstring.c:71)
==4581==    by 0x8FBA419: xmlNewNs (tree.c:766)
==4581==    by 0x8F6AB59: add_namespace_definition (xml_node.c:1327)
==4581==    by 0x4C3D4AE: call_cfunc_2 (vm_insnhelper.c:1741)
==4581==    by 0x4C3DF1C: vm_call_cfunc_with_frame (vm_insnhelper.c:1918)
==4581==    by 0x4C3E06B: vm_call_cfunc (vm_insnhelper.c:1934)
==4581==    by 0x4C45B70: vm_exec_core (insns.def:915)
==4581==
==4581== 33,332 (40 direct, 33,292 indirect) bytes in 1 blocks are definitely lost in loss record 8,382 of 8,434
==4581==    at 0x40307FF: malloc (in /home/p/.linuxbrew/Cellar/valgrind/3.14.0/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4581==    by 0x4AED402: objspace_xmalloc0 (gc.c:7921)
==4581==    by 0x4AED667: ruby_xmalloc0 (gc.c:7989)
==4581==    by 0x4AED696: ruby_xmalloc (gc.c:7998)
==4581==    by 0x8F88AA7: xmlFindCharEncodingHandler (encoding.c:1700)
==4581==    by 0x8F692DE: get (xml_encoding_handler.c:12)
==4581==    by 0x4C3D471: call_cfunc_1 (vm_insnhelper.c:1735)
==4581==    by 0x4C3DF1C: vm_call_cfunc_with_frame (vm_insnhelper.c:1918)
==4581==    by 0x4C3E06B: vm_call_cfunc (vm_insnhelper.c:1934)
==4581==    by 0x4C3F2AD: vm_call_method_each_type (vm_insnhelper.c:2232)
==4581==    by 0x4C3F93A: vm_call_method (vm_insnhelper.c:2355)
==4581==    by 0x4C3FB10: vm_call_general (vm_insnhelper.c:2398)
==4581==
==4581== 32,640 bytes in 1 blocks are indirectly lost in loss record 8,375 of 8,434
==4581==    at 0x40307FF: malloc (in /home/p/.linuxbrew/Cellar/valgrind/3.14.0/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4581==    by 0x4D8944B: __gconv_open (gconv_open.c:164)
==4581==    by 0x4D88E29: iconv_open (iconv_open.c:71)
==4581==    by 0x8F88A0E: xmlFindCharEncodingHandler (encoding.c:1691)
==4581==    by 0x8F692DE: get (xml_encoding_handler.c:12)
==4581==    by 0x4C3D471: call_cfunc_1 (vm_insnhelper.c:1735)
==4581==    by 0x4C3DF1C: vm_call_cfunc_with_frame (vm_insnhelper.c:1918)
==4581==    by 0x4C3E06B: vm_call_cfunc (vm_insnhelper.c:1934)
==4581==    by 0x4C3F2AD: vm_call_method_each_type (vm_insnhelper.c:2232)
==4581==    by 0x4C3F93A: vm_call_method (vm_insnhelper.c:2355)
==4581==    by 0x4C3FB10: vm_call_general (vm_insnhelper.c:2398)
==4581==    by 0x4C47BCD: vm_exec_core (insns.def:1441)
==4581== 32,640 bytes in 1 blocks are possibly lost in loss record 8,374 of 8,434
==4581==    at 0x40307FF: malloc (in /home/p/.linuxbrew/Cellar/valgrind/3.14.0/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4581==    by 0x4D8944B: __gconv_open (gconv_open.c:164)
==4581==    by 0x4D88E29: iconv_open (iconv_open.c:71)
==4581==    by 0x8F88A2B: xmlFindCharEncodingHandler (encoding.c:1692)
==4581==    by 0x8F692DE: get (xml_encoding_handler.c:12)
==4581==    by 0x4C3D471: call_cfunc_1 (vm_insnhelper.c:1735)
==4581==    by 0x4C3DF1C: vm_call_cfunc_with_frame (vm_insnhelper.c:1918)
==4581==    by 0x4C3E06B: vm_call_cfunc (vm_insnhelper.c:1934)
==4581==    by 0x4C3F2AD: vm_call_method_each_type (vm_insnhelper.c:2232)
==4581==    by 0x4C3F93A: vm_call_method (vm_insnhelper.c:2355)
==4581==    by 0x4C3FB10: vm_call_general (vm_insnhelper.c:2398)
==4581==    by 0x4C47BCD: vm_exec_core (insns.def:1441)
==4581==
==4581== 2,268 bytes in 54 blocks are possibly lost in loss record 8,064 of 8,434
==4581==    at 0x40307FF: malloc (in /home/p/.linuxbrew/Cellar/valgrind/3.14.0/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4581==    by 0x4AED402: objspace_xmalloc0 (gc.c:7921)
==4581==    by 0x4AED667: ruby_xmalloc0 (gc.c:7989)
==4581==    by 0x4AED696: ruby_xmalloc (gc.c:7998)
==4581==    by 0x903AEF8: xmlStrndup (xmlstring.c:45)
==4581==    by 0x903AF98: xmlStrdup (xmlstring.c:71)
==4581==    by 0x8FBA419: xmlNewNs (tree.c:766)
==4581==    by 0x8F6AB59: add_namespace_definition (xml_node.c:1327)
==4581==    by 0x4C3D4AE: call_cfunc_2 (vm_insnhelper.c:1741)
==4581==    by 0x4C3DF1C: vm_call_cfunc_with_frame (vm_insnhelper.c:1918)
==4581==    by 0x4C3E06B: vm_call_cfunc (vm_insnhelper.c:1934)
==4581==    by 0x4C45B70: vm_exec_core (insns.def:915)
==4581==
==4581== 1,638 bytes in 39 blocks are indirectly lost in loss record 7,975 of 8,434
==4581==    at 0x40307FF: malloc (in /home/p/.linuxbrew/Cellar/valgrind/3.14.0/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==4581==    by 0x4AED402: objspace_xmalloc0 (gc.c:7921)
==4581==    by 0x4AED667: ruby_xmalloc0 (gc.c:7989)
==4581==    by 0x4AED696: ruby_xmalloc (gc.c:7998)
==4581==    by 0x903AEF8: xmlStrndup (xmlstring.c:45)
==4581==    by 0x903AF98: xmlStrdup (xmlstring.c:71)
==4581==    by 0x8FBA419: xmlNewNs (tree.c:766)
==4581==    by 0x8F6AB59: add_namespace_definition (xml_node.c:1327)
==4581==    by 0x4C3D4AE: call_cfunc_2 (vm_insnhelper.c:1741)
==4581==    by 0x4C3DF1C: vm_call_cfunc_with_frame (vm_insnhelper.c:1918)
==4581==    by 0x4C3E06B: vm_call_cfunc (vm_insnhelper.c:1934)
==4581==    by 0x4C45B70: vm_exec_core (insns.def:915)

It looks like memory is being leaked at two locations.

Hi, thanks for reporting this, and apologies for the delay in replying. I've been getting false positives from valgrind in nokogiri's CI pipeline and needed to figure out how to suppress those before digging into this report.
I'll take a look this weekend.

Also - thank you for using the issue-reporting template, and for providing such clear information about the leak. You've helped a lot!

No worries.  It seems add_namespace_definition() fails to find the already defined namespace.

The many many defined namespaces aren't freed when the document is garbage collected. In Nokogiri_wrap_xml_namespace(), there are three calls to Ruby's Data_Wrap_Struct(VALUE klass, void (*mark)(), void (*free)(), void *sval), but only one of them actually passes a pointer to the deallocator function.

OK, I think I've nailed the root cause down. Running your script before the fix:
# v1.8.5
#0	VSZ/RSS:	 66064 14548
...
#9	VSZ/RSS:	 69136 17832
...
#19	VSZ/RSS:	 72608 21264
...
#29	VSZ/RSS:	 76064 24696
...
#39	VSZ/RSS:	 79540 28128
...
#49	VSZ/RSS:	 83012 31824

and with a patch applied
# PATCHED
#0	VSZ/RSS:	 65644 14304
...
#9	VSZ/RSS:	 65644 14688
...
#19	VSZ/RSS:	 65644 14688
...
#29	VSZ/RSS:	 65596 14664
...
#39	VSZ/RSS:	 65596 14664
...
#49	VSZ/RSS:	 65596 14664

and valgrind no longer dumps out any lost memory associated with allocations having xmlNewNs in their stack.
Going to clean the code up a bit before committing, should be able to cut a release this weekend.

Just checked and saw the commits. Thanks for the effort!
Does this also fix the problem of excessively defined namespaces due to the inability to find existing definitions? Or was that a non-issue maybe?

@paddor the "excessively defined namespaces" is an artifact of how the builder is implemented: each node is created (with namespaces) before it is added to the document tree, so there's no way to know (without a code design change) where to search in the document for a relevant existing namespace definition. As a result, the Builder (actually it's Document#create_element) is forced to create a new ns definition on each (temporarily) orphan node.
I'm open to a pull request that addresses the unnecessary creation of namespace definitions, but at this point it's an optimization and is thus unlikely to get my attention anytime soon.
One further note: once the node is added to a document, Nokogiri will search the ancestors for a relevant pre-existing nsDef and if one exists will remove the duplicate from the reparented node. (More exactly, this node is parked in an "unlinked nodes" hash owned by the Document rather than being immediately freed because it's possible for references to exist. (This is pre-existing code.)) The changeset in #1815 introduces better behavior, to make sure that all references to the removed nsDef are repointed to the correct nsDef hanging off the ancestor element. I may play around with whether this means it's not possible to have any hanging references to it, in which case we may be able to free the nsDef up immediately. Need to think about this and try to break it.

@paddor Node are always created with an associated document; but they're created with an associated parent. Because there's no parent, there's no where to look for an existing matching namespace.
I explored freeing this nsDef but determined it's not safe, and added a test demonstrating why it's not safe in 38a28fe.

The fix in #1815 has been merged into master. Will be in the next release, hopefully in the next few days.

Thanks! Looking forward to the next release.

Any plans when the next release will be released?

Release is coming together now. Ideally soon after libxml 2.9.9 drops next week. Watch this milestone for updates: https://github.com/sparklemotion/nokogiri/milestone/16


Updating the rest of src/foundry/client/ui

Fixes #2086, fixes #2087, fixes #2068, fixes #2085, touches #2092 and #2089

Just as a side note here: Please don't use force pushes when you are resolving review comments. I now have to look at everything again instead of just the changes that resolve my comments.


Graduate SSA

Graduate SSA to GA. Please let me know if you think SSA is not yet ready/ working as intended.
I think it is important we graduate SSA, so we can remove the flag in future versions and simplify the code.
Pull Request Motivation

Kind
/kind feature
Release Note
Graduated SSA

I think we first want to graduate to beta https://cert-manager.io/docs/installation/featureflags/.
To move this forward would be good to write up what the current state is (to what extent it's implemented in controller etc).

If SSA is disabled by default at the moment ("alpha" as we call it), then I'd suggest that we move it to beta now, which will enable the feature for everyone.
@inteon Could you change "GA" to "Beta" and update the SSA design document to mention the change, and also add some provisional dates on when we plan on graduating it to GA?

If SSA is disabled by default at the moment ("alpha" as we call it), then I'd suggest that we move it to beta now, which will enable the feature for everyone.

I think we first need to actually fully implement it across all components which is currently not the case (cainjector does not have it and afaik there might be more bits in controller code as well)
Also beta is off by default https://cert-manager.io/docs/installation/featureflags/

I'm closing this PR & will create new PRs in the future that promote the feature gates.


HW4: Problem 2 - Negative Word Count

@bskowron23 I think this is a typo. I get 805 for the negative word count.  Also you used 805 in the s and p equations:
https://github.com/jasonwalker80/cse427s/blob/22e9722b52d3a7fcf59e96b0fc849cf8791696ab/hw4/HW4.tex#L77

Yep will correct
On Sep 28, 2017 11:12, "Jason Walker"<EMAIL_ADDRESS>wrote:

@bskowron23 https://github.com/bskowron23 I think this is a typo. I get
805 for the negative word count. Also you used 805 in the s and p equations:
https://github.com/jasonwalker80/cse427s/blob/
22e9722b52d3a7fcf59e96b0fc849cf8791696ab/hw4/HW4.tex#L77
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jasonwalker80/cse427s/issues/5, or mute the thread
https://github.com/notifications/unsubscribe-auth/ASmLVI9a6iCl771RHolBZ2ZxXQQOZzQvks5sm8VSgaJpZM4PnhUr
.


Partial update to ASP.NET Core RC2

Usage of MemoryConfigurationSource and shifting to Microsoft.Extensions.Configuration and Microsoft.Extensions.Configuration.Memory as MemoryConfigurationProvider needs an MemoryConfigurationSource.
https://github.com/aspnet/Configuration/blob/dev/src/Microsoft.Extensions.Configuration/MemoryConfigurationProvider.cs

Hi @kdaveid, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, DNFBOT;

Didnt' see https://github.com/aspnet/Docs/pull/1231/commits/c30416142ccd4a3cc7bd1d3ae77dc812c94745f3 so I drop this PR.


Msal node alpha 8 release

msal-node alpha.7 had some issues with the /dist uploaded to npm. Releasing alpha.8 fixed the issue. We think it is an npm glitch.

Coverage decreased (-0.08%) to 83.416% when pulling d2aae4012f30c31f0b6e01587b883d16dc7c0453 on msal-node-alpha-8-release into 8dcb135b56b8ec6cb787583208e68e73cc71d7af on dev.


feature request: Edges going around nodes

Hi, do you have any plans to add edges that could go around their source and target nodes if those are in the way of the edge path. Like this kind of behaviour here. That would be greatly appreciated. Without that feature we have to have several attach points on nodes depending on the direction the user wants to link to and where he moves his nodes after linking.
Thank you, let me know

For everyone who is checking out this issue, we have an overview of the current edge behaviours in the edge-types example.
We are not planning to refactor/change this part in the near future. PRs are very welcome!

Hi @MatthieuJnon, could you help me with how to implement the attach points as you mentioned for the workaround? Examples would be fine.

I created a POC of this with the help of react-flow-chart and pathfinding, as I mentioned here.
As this stands right now, it is really not optimal, but it shows that it's relatively easy to do. Ideally the edges would share the matrix, instead of generating it on each render. This current iteration, makes it so it's extremely unresponsive, but hey - the proof of concept works.
The matrix should also account for scaling, with help from the snapGrid option.
Oh, and the creation of the matrix is extremely inefficient atm. But I just hacked it togeheter.
@moklick Would it be possible to store arbitrary stuff in the store? It would be neat not to have to use a new context just to share the matrix.

import React from 'react';
import {
  getMarkerEnd,
  useStoreState,
  Node,
} from 'react-flow-renderer';
import * as PF from 'pathfinding';

const finder = PF.JumpPointFinder({
  heuristic: PF.Heuristic.manhattan,
  diagonalMovement: PF.DiagonalMovement.Never,
});

function createMatrix(nodes: Node[]) {
  let maxX: number = 0;
  let maxY: number = 0;

  nodes.forEach(({ __rf: { width, height, position: { x, y } } }) => {
    const newX = x + width;
    const newY = y + height;

    if (newX > maxX) {
      maxX = newX;
    }

    if (newY > maxY) {
      maxY = newY;
    }
  });

  const matrix = new Array(maxY)
    .fill(0)
    .map(() => new Array(maxX).fill(0));

  // Go through the nodes again, and fill the matrix with the occupied spaces
  nodes.forEach(({ __rf: { width, height, position: { x, y } } }) => {
    // Create the array that we are going to insert for every row in the other array
    const row = new Array(width).fill(1);
    const stopY = y + height;

    for (let currentY = y; currentY <= stopY; currentY += 1) {
      if (matrix[currentY]) {
        matrix[currentY].splice(x, width, ...row);
      }
    }
  });

  return matrix;
}

function createSmartPath(matrix: number[][], sourceX, sourceY, targetX, targetY) {
  const grid = new PF.Grid(matrix);

  const path = PF.Util.compressPath(
    finder.findPath(
      sourceX,
      sourceY,
      targetX,
      targetY,
      grid,
    ),
  );

  if (path.length === 0) {
    return undefined;
  }

  const [first, ...rest] = path;
  let d = `M${first[0]} ${first[1]}`;
  rest.forEach(([x, y]) => {
    d += ` L${x} ${y}`;
  });
  return d;
}

export function CustomEdge({
  id,
  sourceX,
  sourceY,
  targetX,
  targetY,
  style = {},
  arrowHeadType,
  markerEndId,
}) {
  const nodes = useStoreState((state) => state.nodes);

  const matrix = createMatrix(nodes);
  const path = createSmartPath(matrix, sourceX, sourceY, targetX, targetY);

  const markerEnd = getMarkerEnd(arrowHeadType, markerEndId);

  return (
    <>
      <path
        id={id}
        style={style}
        className="react-flow__edge-path"
        d={path}
        markerEnd={markerEnd}
      />
    </>
  );
}

We will not add a pathfinding logic to this package. If you need a special edge logic you need to implement a custom edge by yourself.
docs: https://reactflow.dev/docs/api/edge-types/
example: https://reactflow.dev/examples/edges/

@moklick  The decision to not add it actually hurts the project.  Currently, representing a circular reference by connecting a node's output handle to it's own input handle makes the edge invisible.  Your current edge types demonstrate a concern about aesthetics, but your lack of desire to support edge routing leaves the assumption of a "too difficult thus I don't care" mentality.


TST: test that Geometry subclasses properly return NotImplemented in comparison with non-geometry

Closes https://github.com/shapely/shapely/issues/1056
This was already implicitly fixed by https://github.com/shapely/shapely/pull/1239 by removing the python __eq__ implementation and only keeping the C-level implementation (which already correctly returned NotImplemented when needed). But this PR is still adding an explicit test for it.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 81.098%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2068

Relevant Lines:
2550

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+2.8%) to 83.945%

Totals

Change from base Build<PHONE_NUMBER>:
2.8%

Covered Lines:
2196

Relevant Lines:
2616

💛  - Coveralls


add edit alarm fragment ui

the page should have section to select time, weekdays, and ringtone.

I want to work on this.


LazyMergeKit - Tensor model.final_layernorm.weight required but not present in model ...

Hi there I'm trying to merge Phi-2 models using the following config:
`MODEL_NAME = "..."
yaml_config = """
models:

model: microsoft/phi-2
no parameters necessary for base model

model: rhysjones/phi-2-orange
parameters:
density: 0.5
weight: 0.5
model: cognitivecomputations/dolphin-2_6-phi-2
parameters:
density: 0.5
weight: 0.3
merge_method: ties
base_model: microsoft/phi-2
parameters:
normalize: true
dtype: float16

"""`
but i get the following error:
RuntimeError: Tensor model.final_layernorm.weight required but not present in model rhysjones/phi-2-orange
I tried with lxuechen/phi-2-dpo before insead of phi-2-orange but got the same error.
I'm executinhg on Google Collab with CPU Runtime with Remote_Code set to true.
Can someone help and tell me if I'm doing something wrong or if it just oesnt work with Phi?
Here is the full log:
mergekit-yaml config.yaml merge --copy-tokenizer --allow-crimes --out-shard-size 1B --lazy-unpickle --trust-remote-code Warmup loader cache:   0% 0/3 [00:00<?, ?it/s] Fetching 10 files: 100% 10/10 [00:00<00:00, 9925.00it/s] Warmup loader cache:  33% 1/3 [00:00<00:00,  5.18it/s] Fetching 11 files: 100% 11/11 [00:00<00:00, 71977.14it/s] Warmup loader cache:  67% 2/3 [00:00<00:00,  5.58it/s] Fetching 10 files: 100% 10/10 [00:00<00:00, 31583.61it/s] Warmup loader cache: 100% 3/3 [00:00<00:00,  5.69it/s]   0% 1/2720 [00:00<00:02, 1276.42it/s] Traceback (most recent call last):   File "/usr/local/bin/mergekit-yaml", line 8, in <module>     sys.exit(main())   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__     return self.main(*args, **kwargs)   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main     rv = self.invoke(ctx)   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke     return ctx.invoke(self.callback, **ctx.params)   File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke     return __callback(*args, **kwargs)   File "/content/mergekit/mergekit/options.py", line 76, in wrapper     f(*args, **kwargs)   File "/content/mergekit/mergekit/scripts/run_yaml.py", line 47, in main     run_merge(   File "/content/mergekit/mergekit/merge.py", line 90, in run_merge     for _task, value in exec.run():   File "/content/mergekit/mergekit/graph.py", line 191, in run     res = task.execute(**arguments)   File "/content/mergekit/mergekit/io/tasks.py", line 73, in execute     raise RuntimeError( RuntimeError: Tensor model.final_layernorm.weight required but not present in model rhysjones/phi-2-orange

This problem might come from the fact that Microsoft changed the architecture after phi-2's release. The models that were fine-tuned still use the old one. It might work if you find a copy of the old base model. See the difference in mergekit:

New: https://github.com/arcee-ai/mergekit/blob/dbb2eebf4ff0c21bda6069cf6de0e3e3b249f82e/mergekit/_data/architectures/phi2.json
Old: https://github.com/arcee-ai/mergekit/blob/dbb2eebf4ff0c21bda6069cf6de0e3e3b249f82e/mergekit/_data/architectures/phi2-old.json

Thanks, I'll try it with the old version. But I somehow got the same error when attempting a pass-through merge between phi-2 and deepseek, but I got the error for the deepseek model. Is it not possible to merge llms with different architectures using pass-through in general? Is there like a Blogpost where you go into this already that I haven't seen?

Looks like you still don't have the same tensors in all of your models. You can quickly check the names of your layers on the model card by clicking on the arrow next to "safetensors".

Looks like you still don't have the same tensors in all of your models. You can quickly check the names of your layers on the model card by clicking on the arrow next to "safetensors".

Thanks for the hint, I'll check that on my next attempt :)

Looks like you still don't have the same tensors in all of your models. You can quickly check the names of your layers on the model card by clicking on the arrow next to "safetensors".

I got it to work with another Phi-2 Model. I think you were right. My merge models had different names in tensors (transformer...) than Microsoft Phi-2(model....).
I got it to merge, now I'm trying to create gguf files using your notebook.
But now I get the following error:
Filtering content: 100% (3/3), 5.17 GiB | 59.74 MiB/s, done. Loading model file Phiter/model-00001-of-00003.safetensors Loading model file Phiter/model-00001-of-00003.safetensors Loading model file Phiter/model-00002-of-00003.safetensors Loading model file Phiter/model-00003-of-00003.safetensors Traceback (most recent call last):   File "/content/llama.cpp/convert.py", line 1483, in <module>     main()   File "/content/llama.cpp/convert.py", line 1419, in main     model_plus = load_some_model(args.model)   File "/content/llama.cpp/convert.py", line 1280, in load_some_model     model_plus = merge_multifile_models(models_plus)   File "/content/llama.cpp/convert.py", line 730, in merge_multifile_models     model = merge_sharded([mp.model for mp in models_plus])   File "/content/llama.cpp/convert.py", line 709, in merge_sharded     return {name: convert(name) for name in names}   File "/content/llama.cpp/convert.py", line 709, in <dictcomp>     return {name: convert(name) for name in names}   File "/content/llama.cpp/convert.py", line 684, in convert     lazy_tensors: list[LazyTensor] = [model[name] for model in models]   File "/content/llama.cpp/convert.py", line 684, in <listcomp>     lazy_tensors: list[LazyTensor] = [model[name] for model in models] KeyError: 'transformer.h.17.mlp.fc1.bias' ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   no ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes ggml_init_cublas: found 1 CUDA devices:   Device 0: Tesla T4, compute capability 7.5, VMM: yes main: build = 2270 (4804215c) main: built with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu main: quantizing 'Phiter/phiter.fp16.bin' to 'Phiter/phiter.Q4_K_M.gguf' as Q4_K_M llama_model_quantize: failed to quantize: failed to open Phiter/phiter.fp16.bin: No such file or directory main: failed to quantize model from 'Phiter/phiter.fp16.bin' ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   no ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes ggml_init_cublas: found 1 CUDA devices:   Device 0: Tesla T4, compute capability 7.5, VMM: yes main: build = 2270 (4804215c) main: built with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu main: quantizing 'Phiter/phiter.fp16.bin' to 'Phiter/phiter.Q5_K_M.gguf' as Q5_K_M llama_model_quantize: failed to quantize: failed to open Phiter/phiter.fp16.bin: No such file or directory main: failed to quantize model from 'Phiter/phiter.fp16.bin'
Do you see whats the problem here?
The fp16.bin couldn't be created, but why?
I used the collab from this blog artile of yours:
https://mlabonne.github.io/blog/posts/Quantize_Llama_2_models_using_ggml.html

Cool! You should be able to make GGUF versions of the model. Once again, maybe a problem with the old architecture? I can't really help you with that, unfortunately.

I got it working now thanks to some help from another friendly Huggingface User. I had to use an older Version of llama.cpp and use convertHFtogguf.py first and then quanitizing it with quantize (https://huggingface.co/brittlewis12/phi-2-orange-GGUF/discussions/1)
Thanks again for all your help. I finally have my first working merge now thanks to you :)
Feel free to check out my little Phiter https://huggingface.co/Venkman42/Phiter
I gave it a testrun and so far im quite satisfied wih the results. At least it doesnt seem to perform worse than the base models.
Would you kindly do me the honor and run an eval on it for YALL?

Haha well done! Sure, running the eval now :)

Thank you :) I'm curious how it will score
Btw, how long do these evals usually take for smaller model? And what hardware do you run them on?

Congrats new SOTA in terms of phi-2 fine-tune: https://huggingface.co/spaces/mlabonne/Yet_Another_LLM_Leaderboard 🎉
So I just use LLM AutoEval. It took 2 hours and 18 minutes to evaluate Phiter on a RTX 3090.

Damn, I had a feeling it was good, but I didn't think it would outsmart both base models on all benchmarks and even outperform the phixtral models.
Oh okay, so it doesn't take that much compute. Maybe I'll try running it myself sometime.
Thanks again for taking the time 😊
I'm gonna close this issue now 😁

I credited you on my model card for helping me troubleshoot, I hope that's okay :)


feat: mysql2 instrumentation

Which problem is this PR solving?
Adds autoinstrumentation for mysql2.
Short description of the changes
This is pretty much the same as opentelemetry-instrumentation-mysql2, but I simplified it by only wrapping Connection.prototype. This exact code might actually work for mysql as well.

If patched module are close, i think we should just add support for mysql2 inside mysql instrumentation, WDYT ?

Yes... It would be totally fine, unless... Types.
Could still be done, but I'd take it on outside of this PR.

Yes... It would be totally fine, unless... Types.

Not sure to understand why types are a problem ?

We don't have a good precedent of 2-package instrumentation - I mainly don't want to start that discussion here.
To answer your question: Because those do not match between the two packages. Not saying it's not doable, it absolutely is with unions and whatnot, but transforming the code to work with mysql2 was already a task and I don't see a good reason to push more into a PR.

FYI we have both http (that handle https too) and grpc (that handle grpc-js and grpc c++) that have this behavior and we don't have any issue with it.
I totally think that since you already written as a separate package it doesnt matter but i was more asking if you tried and there was issue.

Thanks for pointing this out, @vmarchaud. I'll look into that at some point! :)

cc @open-telemetry/javascript-approvers would appreciate some review so we can ship this with the next release :) thanks


Add executeSchedule methods to RScheduledExecutorService

executeSchedule methods return void method instead of ScheduledFuture this allow to avoid memory consumption by each schedule.

@mrniko Thank you for having the request.

Fixed! @yjain3326 Solved with WeakReference technique to track non used futures and release them.

@mrniko Great! I just tested it and it looks great.
Thank you for  fixing this.
Can you please update here when this is release? we would like to use this asap.

@yjain3326 During a week

@mrniko does 3.5.6 has this changes?

@yjain3326 No, it doesn't. Please wait for 3.5.7


feat(forge): stack traces grab contracts from etherscan when forking

When forking, we really should grab the etherscan contracts if the user has provided an etherscan api key
potential logic:

execute test
check if forked evm
identify known contracts for trace
check unidentified contract addresses on etherscan
compile contracts from etherscan
update traces with new known contracts

If nobody has started this I can. I'm a big user of forked tests and this would be a huge UX improvement.
I'm thinking the best way to tackle this would be to change the definition of CallTraceArea's update_identified function to take a flag to specify if this is a fork test (which we can get by the presence of self.evm_opts.fork_url in ContractRunner). From there we just need to update update_identified to fetch the ABI from etherscan if it doesn't match the code of any known contracts. I'll also have to think about the best way to figure out what Chain to pass the ethers-etherscan Client, since we only have access to the chain id at this point (which is as far as I know cannot easily be converted to the Chain type). I could write a little lookup table but this would not be fully sufficient as it wouldn't differentiate between chains that use the same chain id as ethereum mainnet such as ETC.
Does this sound like a good plan?

Yes that sounds like a good plan

Closed in #1284


Upgrade all AWS Registry deployments to use OpenSearch

💡 Description

Per #15 , upgrade all registries to OpenSearch

opensearch is deployed on en-delta.


Add Format String Keyword

Adds the Format String keyword.
Closes #2822

add acceptance tests.
@pekkaklarck  or @bkhouri can you review it?

I stumbled on this PR looking for something similar before implementing it myself, glad you did it against the standard library.
One major comment here - the current implementation accepts as positional argument either a source string to be formatted, or a file which content will be used for that. It distinguishes between the two by a call to os.path.isfile(), but this approach has two major drawbacks:

mostly, performance hit - every keyword call will start off with file system access ( os.path.isfile() is a wrapper over os.stat()), which is order of magnitudes slower than the actual work of string formatting (the data for it is already in-memory),
in the corner case where the user wants to pass a string, but there's a file with the same name in the cwd,  this will lead to an unexpected result.

Thus I'd suggest to split the keyword in two, Format String and Format String From File (or similar name).

It was probably me who proposed adding the ability to read the template from a file. XML library has similar functionality and it has worked great.
I doubt the performance hit from checking does the file exist is big and it is also rather unlikely that a normal template string would match a path to an existing file. Both of these worries can be mostly mitigated by mandating that the path must be absolute and checking that with os.path.isabs before using isfile.

@pekkaklarck there is an existing issue for this addition - #2822, but I don't think it's currently targeted for 3.1; it would be good if it gets there.

Added #2822 to 3.1 scope.

If the possible problem with a template string matching an exist file with an absolute path turns out to actually cause problems, we can consider adding an optional argument to control is the template a string or path. I considered that so unlikely that I wouldn't do it now. The isabs check I mentioned earlier would probably be a good idea. The requirement for the path to be absolute would then need to be mentioned also in the docs.

If the possible problem with a template string matching an exist file with an absolute path turns out to actually cause problems, we can consider adding an optional argument to control is the template a string or path.

I also think we're spending too many cycles on a small issue - hitting a file name collision with a templated string is really a corner case. I fully agree os.path.abs will weed out the majority of the values usages, with no perf hit, leaving only a very small percentage getting down to the isfile() check; let's leave it at that.
Thanks for adding it to the scope 3.1 👍

@tminakov thanks for your review, I'm changing the code as you suggest tonight.
I think that absolute path is not easy to archive in some projects. For example, in my team I use Linux but the people who make our ATDD use Windows.
Another point is that relative path could be trick sometimes due the CWD changes as the tests are executed from another folder.

As I commented in my review, I think we should require the template path to be absolute. In addition to the benefits discussed earlier, absolute paths aren't affected by changes to the current working directory. It's easy to create such paths by using, for example, ${CURDIR}.
Anyway, the changes look good in general and I'm sure we can get this into RF 3.1. I'm trying to get beta 1 out this week before leaving for a one week holiday, but there's going to be beta 2 (or rc 1) if that deadline is missed.

Looks good to me now. There are few things that can still be cleaned up, but I'll merge this now and do that myself.


Stories in Docs view are sorted alphabetically

Stories are shown in the sidebar in the correct order, but they are ordered in the docs view alphabetically.
I've confirmed this is the case when using the latest storybook-builder-vite (0.1.15) and storybook 6.4.18.

This is going to be a long description 😆
This is because the docs page show the stories in export order which is always alphabetical. It was mentioned in this comment that the DocsPage Primary was being replaced.
This is because Primary.tsx calls componentStories() defined here: https://github.com/storybookjs/storybook/blob/64c31900964aa15420e6ec4e1d12f0b88961a457/lib/preview-web/src/PreviewWeb.tsx#L502 which is passed the csfFile from a couple lines above which is fetched using loadCSFFileByStoryId() which calls the importFn and passes the result to processCSFFileWithCache() which seems to intentionally not respect __namedExportsOrder judging by this test: https://github.com/storybookjs/storybook/blob/64c31900964aa15420e6ec4e1d12f0b88961a457/lib/store/src/csf/processCSFFile.test.ts#L51-L71.
Now that explanation is out of the way @shilman and @tmeasday (since you both were on the PR 🙂) do you have any thoughts on what we should do here?

OK, looks like this comment was a mistake @shilman: https://github.com/storybookjs/storybook/pull/16626#pullrequestreview-799609582
I think maybe we should revert that part of the change, the sorting of exports in processCSFFile() is not useless after all!

@tmeasday @shilman should we open an issue in the storybook repo?  Or is this being worked on already?

I got the same problem

I think that'd be great (the PR) but maybe @shilman can confirm, just in case?

Thanks guys for looking into this. Just switched from Webpack to this builder, and this took me by surprise rather immediately.

@sebastienbarre it looks like @shilman is taking a look at how to fix this (https://github.com/storybookjs/storybook/pull/17793#issuecomment-1078835804), let's see if we can't take care of it before adding it to that list of issues.

@joshwooding sorry for not making the connection here until now, but I think we fixed the problem you describe with componentStories() not using the story index order in this PR (on the 9th of March), which was released in alpha.47.
Are you sure this issue is still active? It sounds like @sebastienbarre is saying it is, but maybe they aren't on the latest alpha?

@tmeasday yes, sorry, I'm not on the alpha, I'm using v6.4.19. Will this land in the 6.x? Thank you.

Oh awesome, I just checked our examples, which are on 6.5.0-alpha.49, and the story order is working correctly now.  @sebastienbarre, this will land in 6.5.0 stable.
Closing this issue as fixed by https://github.com/storybookjs/storybook/pull/17669.  Thanks @tmeasday!

Thanks all! No need to apologies @sebastienbarre just trying to clear up my own confusion :)

@joshwooding sorry for not making the connection here until now, but I think we fixed the problem you describe with componentStories() not using the story index order in this PR (on the 9th of March), which was released in alpha.47.
Are you sure this issue is still active? It sounds like @sebastienbarre is saying it is, but maybe they aren't on the latest alpha?

Ah awesome 😄


ARROW-12729: [R] Fix length method for Table, RecordBatch

Oh, actually one small requested change please: both Table and RecordBatch inherit the virtual class ArrowTabular, so whenever we want to define a method for both Table and RecordBatch and the implementation of it is exactly the same, we should instead define it on ArrowTabular. Could you please change this to use that? The tests can stay the same.

Oh, actually one small requested change please: both Table and RecordBatch inherit the virtual class ArrowTabular, so whenever we want to define a method for both Table and RecordBatch and the implementation of it is exactly the same, we should instead define it on ArrowTabular. Could you please change this to use that? The tests can stay the same.

Good point, completely missed that, now done!


issue with directory of the same name/architecture

Hi Mike
First of all thank you very much for this great extension, second there seems to be an issue when generating a barrel file for a specific folder with a name that also exist in the test directory.
- lib
   └ models
           └ model.dart
- test
   └ models
           └ model_test.dart

The generated file will also export the test directory imports.
can you please fix this issue.

If I understand you, you choose to generate a barrel file in the lib/models folder, and it includes the modules_test.dart file, am I right? Does it happen when you choose to generate the barrel file for the current folder, or for the current folder and its subfolders? Or both?

no, when i generate a barrel file in the lib/models that contain 1,2 file, the generated file will contain exports of 1,2,3,4 files.
3,4 file are existed in the test directory with the same folder name test/models.

Alright! :smile:
I will try to recreate the issue as soon as possible (currently I'm a bit busy)! However, feel free to clone and contribute to the extension 🥇

Alright thank you for your quick response, I wish that i could contribute to this great extension, but sadly my programing skills are only limited to flutter.
Anyway take your time, and have a good day.

I will give you feedback as soon as possible

Error was fix! Updating the extension asap!

Thank you very much.

Hi mike, i hope you are doing well.
i was wondering when do you planning to release the new update on the VSCode store?

My most sincere apologies @DarkHeros09! For some reason I completely forgot to publish the update! It is being published as I answer you!
I am sorry 😅

My most sincere apologies @DarkHeros09! For some reason I completely forgot to publish the update! It is being published as I answer you!
I am sorry 😅

Great, happy to hear that.
Also nothing to apologize for, you are doing a great job.

I would appreciate you gave me feedback of the extension, to check that the problem has been solved as you expected 😄

I would appreciate you gave me feedback of the extension, to check that the problem has been solved as you expected 😄

sadly the problem still persist, i uninstall then installed the extension again just to make sure.😢

Have you made sure the version you were using it is the 1.2.1?

Yes i did, i even deleted the extension manually from the hidden folder, then updated to v1.2.1.

Yes, you are right, the issues it is not solved. I will revise everything as soon as I can.

I've tried in Linux and it seems to work. What OS are you using @DarkHeros09?

Win10 v20H2.

I will inspect on Windows, then

Hello @DarkHeros09 😄! I'm currently really busy with work and uni. I've had no time no check why the error happens. As soon as I can I will try to solve it. Just give me a wekk!

Hi @mikededo
Man don't you worry. take a month if you have, life is priorities, and uni is definitely number 1 here so take your time.
Good luck 👍.

Hello @DarkHeros09!
The wait is over! 😄. I have published an update which solves the issues in Windows OS machines. I've been testing it out and it works as expected.
Thank you for your patience, feel free to contact me when needed! 💯


class static variables and functions access with self

Suggestion
If I have static class variables, then I need to use class_name.variable_name
But:

I want to change the class name, then it can be problematic
Maybe my class name is quite long, then it is uncomfortable

Can TypeScript add a special word "self" that can use with static variables and functions? Conversion to JavaScript should be simple.
🔍 Search Terms
class
static
self
✅ Viability Checklist
My suggestion meets these guidelines:

[ ] This wouldn't be a breaking change in existing TypeScript/JavaScript code - NO
[ ] This wouldn't change the runtime behavior of existing JavaScript code - NO
[ ] This could be implemented without emitting different JS based on the types of the expressions - NO
[ ] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.) - NO
[ ] This feature would agree with the rest of TypeScript's Design Goals. - YES

⭐ Suggestion
TypeScript can replace "self" with the name of a class
📃 Motivating Example
class MinuKlassOnIkkaKirjeldavEtOleksHea{
	static muutuja: string;

	static init(nimi: string){
		MinuKlassOnIkkaKirjeldavEtOleksHea.muutuja = nimi + ".";
		//The new syntax:
		//self.muutuja = nimi + ".";
	}
}

💻 Use Cases
Shorter TypeScript code and no problems on changing the name of the class

You can use this

I want to change the class name, then it can be problematic

I don't see how this would be problematic, given the tooling support and refactoring capabilities nowadays.

Maybe my class name is quite long, then it is uncomfortable

Then create an alias.
const self = MinuKlassOnIkkaKirjeldavEtOleksHea;
self.muutuja = nimi + ".";

This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)

We don't add suggestions that don't meet this criterion.

Thank you for "this".
This is a better example need for "self":
class MinuKlassOnIkkaKirjeldavEtOleksHea{
	static muutuja: string;

	static arvuta(){
		//Do something there
	}
	static init_DOM(){
		//Some DOM manipulating
	}

	static init(nimi: string){
		$("#teeSeda").on("click", function(){
			MinuKlassOnIkkaKirjeldavEtOleksHea.arvuta();	//can't use this
			//The new syntax:
			//self.arvuta();
		});

		$(function(){
			MinuKlassOnIkkaKirjeldavEtOleksHea.init_DOM();	//can't use "this"
			//The new syntax:
			//self.init_DOM();
		}
	}
}

$("#teeSeda").on("click", function(){
			MinuKlassOnIkkaKirjeldavEtOleksHea.arvuta();	//can't use this
			//The new syntax:
			//self.arvuta();
		});

A better practice is to use arrow function.
$("#teeSeda").on("click", () => {
  this.arvuta();
});


User defined thread tracing functions aren't called

sys_trace_thread_switched_in_user and sys_trace_thread_switched_in_user are not called if CONFIG_UART_CONSOLE=y
Looking at the void sys_trace_k_thread_switched_in(void) implementation I feel like there's an oddity with the code block that disables calling user functions.
/* FIXME: Limitation of the current x86 EFI cosnole implementation. */
#if !defined(CONFIG_X86_EFI_CONSOLE) && !defined(CONFIG_UART_CONSOLE)

I got the impression the author (@enjiamai)  wanted to ! (X86_EFI_CONSOLE && UART_CONSOLE) logically but mistakenly pushed the ! into the condition which was supposed to result in !X86_EFI_CONSOLE || !UART_CONSOLE
I got this impression because I'm failing to see why UART_CONSOLE should disable calling user defined trace functions.
Please let me know if I'm wrong.
Please also mention any information which could help others to understand
the problem you're facing:
west build -p -b qemu_riscv32 -T samples/subsys/tracing/sample.tracing.user
p.s. this also won't work due to incorrect prj_user.conf file (see #60297). But it's an obvious+trivial fix.

What have you tried to diagnose or workaround this issue?

- #if !defined(CONFIG_X86_EFI_CONSOLE) && !defined(CONFIG_UART_CONSOLE)
+ #if !defined(CONFIG_X86_EFI_CONSOLE) || !defined(CONFIG_UART_CONSOLE)

Is this a regression? If yes, have you been able to "git bisect" it to a
specific commit?
Yes. #46918

To Reproduce
west build -p -b qemu_riscv32 -T samples/subsys/tracing/sample.tracing.user
Expected behavior
sys_trace_thread_switched_in_user and sys_trace_thread_switched_in_user would be called and print.
Impact
A local showstopper for thread tracing.
Logs and console output
Lack of prints isn't that interesting.
Environment (please complete the following information):
Linux, SDK 0.16.1
Additional context
If the fence condition is correct, then I would argue that just because x86 EFI console has limitations, the kernel shouldn't blindly disable calling the trace functions. I think this fence should've been somewhere else.

I had another look at the tracing_user related change in #46918, and find that change to be weird, the trancing_user APIs are supposed to be implemented by the user, it just so happens that its sample is using printk. If the other changes in #46918 is causing that sample to fail, I think the better way should be to filter out that testcase?
cc @andyross @enjiamai @jhedberg @cfriedt

If the other changes in https://github.com/zephyrproject-rtos/zephyr/pull/46918 is causing that sample to fail, I think the better way should be to filter out that testcase?

That was my conclusion too. Thanks for the PR btw 💯

Closed by #60666


preserve casing when printing help

When verb methods/parameters are composed of multiple words (usually with camel/pascal casing), it's hard to read the generated help due to the names being lower-cased.
I modified the code to preserve casing, and to properly compare names while ignoring case.
All tests are passing.

Thanks. Appreciated.


Rebase feature-frontchannel-logout to jaunifioren/master

Note: I beleive the most "clean" and safe course of action would be to delete the current torre-labs:feature-frontchannel-logout branch and push agurodriguez:feature-frontchannel-logout directly instead of merging them.

I agree @agurodriguez. Let's reset this branch to yours.
Please test that the migrations work correctly. I've had some issues in the past when updating from upstream.


Trying to use a wildcard (*) in Include / Exclude folder list doesn't work and produces an error

I think a fairly understandable use-case would be to include specific folder names throughout your vault, not just at specific paths, like this:

However, this does not work and also produces an error in the console:

Alternatively, instead of supporting the wildcard in the Include / Exclude folder list, another idea would be to provide an option for only checking inside the attachment folder(s) that is configured in the Obsidian settings:

Since the excluded folders lines are part of a regular expressions, you'd have to use .*.
This is also why the error says Nothing to repeat, the * quantifier requires something to repeat (in this case ., meaning any character).
Example with description from regex101:
Note the parentheses are not required, just added here for show the grouping of the match

I don't remember exactly why I decided on it being regular expressions so I might tag this as a bug.

Ah, it does work, thank you! Please consider adding a little text to the description of the Include / Exclude folders setting box that explains this, it's very useful.

I agree, I don't remember exactly why I went for regular expressions but it is way more powerful than regular globbing (using * as wildcards).
I'll leave the issue open as a reminder to add explanation to the description.


Migrate tests to netcoreapp2.1

.net core 2.0 is out-of-support and we should run tests on .net core 2.1 that's LTS
@weshaggard

Agreed. See https://github.com/dotnet/core/blob/master/microsoft-support.md for the support matrix.

Fixed with https://github.com/Azure/azure-sdk-for-net/pull/6247


Dependency upgrades

Notes:

This includes upgrading bullmq, which we have been holding due to https://github.com/taskforcesh/bullmq/issues/1437 - tested and verified on https://app.staging.medplum.com
I'm not sure when it happened, but something changed in <ErrorBoundary> which caused page re-renders.  I tweaked that a bit, and removed some batch calls for better caching.  This improves render performance.  I'd like to research more to understand what happened.

Coverage increased (+0.01%) to 93.99% when pulling 86ed86c157ee47dbe52d0468a142ed7a0f12032d on cody-dep-upgrades-2022-12-17 into 412431f24afff117a5b02b3027045fe5edc1d9a6 on main.


Serious changes in index.html

The main translation logic has been changed and the styling has been updated.
Everything works fine though.

Check and test the deploy preview before accepting.

good to go


Remove Model class attributes that are shadowed

What change does this PR introduce and why?
Model has class attributes defined for name, metadata, tags, infer,
and _id which are shadowed as instance attributes
during initialization. This change removes these class attributes since
they belong as instance attributes.
Please check if the PR fulfills these requirements

[x] Relevant tests for the changes have been added
[x] Relevant docs have been added / updated

Is there any functional motivation for this change, or just cleanliness? Initially preferred to define attributes at the class level for readability.

Is there any functional motivation for this change, or just cleanliness? Initially preferred to define attributes at the class level for readability.

A bit of both. Models inherit from Frozen whose __eq__ compares __dict__. These differ when loading models since the Model created with __init__ shadows the class attribute workflow while the loaded model does not. An example of the issue in this comment https://github.com/kolenaIO/kolena/pull/315#discussion_r1397890819.
Another potential fix is to shadow the workflow when loading the model, but I prefer getting rid of the shadowing altogether since it is confusing/bad practice.


Add build steps to allow x64 jdk8 cross compilation on M1 mac

Changes need to be made to allow x64 JDK8 to be built on our M1 machines.
https://github.com/adoptium/infrastructure/issues/2536#issuecomment-1708716478 includes steps taken to build x64 JDK8 in an ssh environment on our M1 mac machines. These steps need to be added to the build process

Two of the main additions will be
xcode-select -switch 'path to Xcode11.7'
and using Rosetta to initiate the build via the arch -x86_64 command
arch -x86_64 ./makejdk-any-platform.sh --clean-git-repo --jdk-boot-dir 'path to x64 jdk8 mac binary'/Contents/Home --configure-args '--with-toolchain-type=clang --openjdk-target=x86_64-apple-darwin --with-cups=/opt/homebrew/opt/cups/' --target-file-name jdk8_x64.tar.gz --build-variant temurin jdk8u

Out of interest I built a macos x64 jdk8 using xcode11.7 on macos12, I believe we currently use xcode10 on macos10 machines to build x64 jdk8.
It built successfully and am now running it through the aqa test pipeline job on a macos10.15 machine https://ci.adoptium.net/job/AQA_Test_Pipeline/180/console

Weve got this to a stage where we are able to cross compile x64 mac binaries on our arm64 machines. https://github.com/adoptium/temurin-build/issues/3509 remains a problem, but this issue can be closed


update darknet to latest version

Which triplets are supported/not supported? Have you updated the CI baseline?
All, Yes

Does your PR follow the maintainer guide?

Yes

Can you check if darknet's opencv* dependencies really need the default features of opencv*?
ATM darknet[opencv*] prevents opting out from default features of opencv. Cf. https://github.com/microsoft/vcpkg/issues/20471#issuecomment-1086560192.

@cenit ,Thanks for your pr, When I run the comand "./vcpkg install darknet[*]:x64-windows" tested the features locally, It failed :
CMake Error at ports/opencv2/portfile.cmake:6 (message):
  OpenCV 4 is installed, please uninstall and try again:

      vcpkg remove opencv4
Call Stack (most recent call first):
  scripts/ports.cmake:145 (include)

error: building opencv2:x64-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `git pull` and `.\vcpkg update`.
Then check for known issues at:

Then I ran the command "./vcpkg remove opencv4:x64-windows" to remove the opencv4 port and tried the command "./vcpkg install darknet[*]:x64-windows" again, but still got this error

yes, darknet is one of the very few ports that do not obey to new rules, meaning that there are features mutually exclusive and not additive.
In fact, if you use opencv-base, you cannot ask for opencv2-base or opencv3-base, because the opencv version on which they depend are incompatible between themselves

Tested fearture [cuda,cudnn,full,opencv-base,opencv-cuda] successfully in the  x64-windows triplet

Thanks!