Weird tensorflow lite message

Hello everyone.
I have a himax board and I am trying to deploy my own classification model. I am having some problems, hope someone can help me.
My model is a simpler version of a mobilenet_v2 for classification.
When invoking the model for inference, I am receiving a weird message from tensorflow lite.
Does anyone knows what it means?
tensorflow/lite/micro/kernels/arc_mli/depthwise_conv.cc:574 in_slice.Done() was not true.

I am receiving the models output after invoking the model, but don't know yet if the outputs are correct. I still have to use the himax camera at my domain environment.
I am just a little worried with this message, any help is welcome.
Thank you so much for the attention.

@PCPJ, hi
We used to face the problem at deploying another model on conv.cc.
We try to figure out the problem and finally find out that the problem is because the input tensor size at conv node will cause the max_out_lines_for_input value at function (arc_scratch_buffer_calc_slice_size_io at scratch_buf_mgr.cc) not be divisible.
Hence, we will face "in_slice.Done() was not true" warning. In our case, we change code at "tensorflow/lite/micro/kernels/arc_mli/scratch_buf_mgr.cc line 323 and 324" to following code.
      // max_out_lines_for_input =
      //     (max_lines_in - kernel_height + 1) / stride_height;
      max_out_lines_for_input = (max_lines_in - kernel_height + 1)%stride_height !=0?
          ((max_lines_in - kernel_height + 1) / stride_height)+1:(max_lines_in - kernel_height + 1) / stride_height;

And the warning is gone, and the answer is correct.
Thanks,
Kris

Hello @kris-himax.
Sorry for the late reply.
I tried to modify the lines as you suggested, but I am still receiving the same messages.
Also, I am testing the model in another way now, and the model isn't outputting the same compared to running using python tflite in my local machine. Do you think is has something to do with these messages?
I stored an rgb image into a static variable inside the code I am flashing into the board. Instead of sending images from the camera to the model, I am sending always this static image. I expected to receive the same output compared with running the same model, with same input, in my local machine. Instead I am receiving -127 as output.
Please allow me to show how I am doing it.

void loop() {
  hx_drv_uart_print("Preparing input!\n");
  if (input->type == TfLiteType::kTfLiteInt8) {
    void* quanti_param = input->quantization.params;
    assert(input->quantization.type == TfLiteQuantizationType::kTfLiteAffineQuantization);
    TfLiteAffineQuantization* quanti_affine_param = static_cast<TfLiteAffineQuantization*>(quanti_param);
    for (uint i = 0; i < (kNumCols*kNumRows); i+=3) {
      float r = ((float) my_img[i+0]);
      float g = ((float) my_img[i+1]);
      float b = ((float) my_img[i+2]);
      // ImageNet Normalization
      r = ((r)/255.0f -0.406f) / 0.225f;
      g = ((g)/255.0f -0.456f) / 0.224f;
      b = ((b)/255.0f -0.485f) / 0.229f;
      // Quantization normalization
      r = (r / quanti_affine_param->scale->data[0]) + quanti_affine_param->zero_point->data[0];
      g = (g / quanti_affine_param->scale->data[1]) + quanti_affine_param->zero_point->data[1];
      b = (b / quanti_affine_param->scale->data[2]) + quanti_affine_param->zero_point->data[2];
      input->data.int8[i+0] = (int8_t)b;
      input->data.int8[i+1] = (int8_t)g;
      input->data.int8[i+2] = (int8_t)r;
    }
  } else {
    hx_drv_uart_print("Input type isn't int [%d]!\n", input->type);
    assert(false);
  }

  hx_drv_uart_print("Invoking model!\n");
  // Run the model on this input and make sure it succeeds.
  if (kTfLiteOk != interpreter->Invoke()) {
    TF_LITE_REPORT_ERROR(error_reporter, "Invoke failed.");
  }

  hx_drv_uart_print("Getting output!\n");
  TfLiteTensor* output = interpreter->output(0);

  // Process the inference results.
  int8_t class1_score = output->data.int8[0];
  int8_t class2_score = output->data.int8[1];
  if (input->type == TfLiteType::kTfLiteInt8) {
    void* quanti_param = input->quantization.params;
    assert(input->quantization.type == TfLiteQuantizationType::kTfLiteAffineQuantization);
    TfLiteAffineQuantization* quanti_affine_param = static_cast<TfLiteAffineQuantization*>(quanti_param);
    float float_class1_score = ((float)class1_score - quanti_affine_param->zero_point->data[0] ) * quanti_affine_param->scale->data[0];
    float float_class2_score = ((float)class2_score - quanti_affine_param->zero_point->data[0] ) * quanti_affine_param->scale->data[0];
    hx_drv_uart_print("Class1 [%g] - Class2 [%g]\n", (double)float_class1_score, (double)float_class2_score);
    hx_drv_uart_print("Class1 [%d] - Class2 [%d]\n", class1_score, class2_score);
  }
}

The variable my_img has the raw data of the input RGB image.

Hi @PCPJ ,
In my opinion, I think that your "Preparing input" is wrong.
I saw that your model input type is kTfLiteInt8 at your code which means that you have already done quantization while you converting your model.
Did you do ImageNet Normalization while you do quantization to INT8 model?
If you have done ImageNet Normalization to calibrate your data, you don't need to do it again.
You only need to convert your input image from uint8 to int8, and do not need to do dequantization.
 TfLiteAffineQuantization* quanti_affine_param = static_cast<TfLiteAffineQuantization*>(quanti_param);
    for (uint i = 0; i < (kNumCols*kNumRows); i+=3) {
      float r = ((float) my_img[i+0]);
      float g = ((float) my_img[i+1]);
      float b = ((float) my_img[i+2]);
      // ImageNet Normalization
      r = ((r)/255.0f -0.406f) / 0.225f;
      g = ((g)/255.0f -0.456f) / 0.224f;
      b = ((b)/255.0f -0.485f) / 0.229f;
      // Quantization normalization
      r = (r / quanti_affine_param->scale->data[0]) + quanti_affine_param->zero_point->data[0];
      g = (g / quanti_affine_param->scale->data[1]) + quanti_affine_param->zero_point->data[1];
      b = (b / quanti_affine_param->scale->data[2]) + quanti_affine_param->zero_point->data[2];
      input->data.int8[i+0] = (int8_t)b;
      input->data.int8[i+1] = (int8_t)g;
      input->data.int8[i+2] = (int8_t)r;

And our Himax HM0360 camera can only get grayscale images. Maybe your model should change the input channel size.
Thanks,
Kris

Hi @kris-himax. Thank you so much for the reply.
I trained my model with images normalized with the ImageNet numbers. I performed the quantization this way:
converter = tf.compat.v1.lite.TFLiteConverter.from_saved_model(out_tf_path)

def representative_dataset():
    dataset = MyDataset()
    for image, target in dataset:
        image = image.numpy()
        yield ([image]) # THIS IMAGE IS NORMALIZED WITH IMAGENET NUMBERS

# Set the optimization flag.
converter.optimizations = [tf.lite.Optimize.DEFAULT]

# Enforce integer only quantization
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8

# Provide a representative dataset to ensure we quantize correctly.
converter.representative_dataset = representative_dataset
converter.allow_custom_ops = False

print("Converting to TF Lite quantized model.")
tflite_model = converter.convert()

I understand that you said I don't need to perform the ImageNet normalization again, but I didn't understand what you mean with:
You only need to convert your input image from uint8 to int8, and do not need to do dequantization.
I only need to subtract 128 from the input pixels? Like this:
    TfLiteAffineQuantization* quanti_affine_param = static_cast<TfLiteAffineQuantization*>(quanti_param);
    for (uint i = 0; i < (kNumCols*kNumRows); i+=3) {
      int r =  ((int) my_img[i+0]) - 128;
      int g = ((int) my_img[i+1]) - 128;
      int b = ((int) my_img[i+2]) - 128;
      input->data.int8[i+0] = (int8_t)b;
      input->data.int8[i+1] = (int8_t)g;
      input->data.int8[i+2] = (int8_t)r;

Also, I am using this function to retrieve the image from the camera:
TfLiteStatus GetImage(tflite::ErrorReporter* error_reporter, int image_width,
                      int image_height, int channels, int8_t* image_data) {
  static bool is_initialized = false;
  if (!is_initialized) {
    if (hx_drv_sensor_initial(&g_pimg_config) != HX_DRV_LIB_PASS) {
      return kTfLiteError;
    }
    is_initialized = true;
  }
  hx_drv_sensor_capture(&g_pimg_config);
  hx_drv_image_rescale((uint8_t*)g_pimg_config.raw_address,
                       g_pimg_config.img_width, g_pimg_config.img_height,
                       image_data, image_width, image_height);
  return kTfLiteOk;
}

My image data vector has size height*width*3channels. Is this function filling the first third of the vector, and letting the rest untouched? Any way, I will have to retrain the model to receive grayscale images.
Thank you so much again.

Hi @PCPJ ,
I understand that you said I don't need to perform the ImageNet normalization again, but I didn't understand what you mean with: You only need to convert your input image from uint8 to int8, and do not need to do dequantization. I only need to subtract 128 from the input pixels? Like this:
    TfLiteAffineQuantization* quanti_affine_param = static_cast<TfLiteAffineQuantization*>(quanti_param);
    for (uint i = 0; i < (kNumCols*kNumRows); i+=3) {
      int r =  ((int) my_img[i+0]) - 128;
      int g = ((int) my_img[i+1]) - 128;
      int b = ((int) my_img[i+2]) - 128;
      input->data.int8[i+0] = (int8_t)b;
      input->data.int8[i+1] = (int8_t)g;
      input->data.int8[i+2] = (int8_t)r;

Ans: Yes, that is what I mean.
The GetImage function was the template function at this google person detection example (https://github.com/tensorflow/tflite-micro/blob/a37043ff7a42e9d46345b4a6b33abda2fd1081f5/tensorflow/lite/micro/examples/person_detection/image_provider.cc), and we modified it to let it can run on our camera sensor.
The channels parameter at the GetImage function is unused because our sensor can only receive grayscale images.
If you use the GetImage function, the image_data pointer will be the output which is receive grayscale images from HIMAX HM0360 camera and had already been converted to be (heightwidth1channels) INT8 type.
Thanks,
Kris

Hi @kris-himax & @PCPJ, I have trained a classification model on 96X96X1 grayscale images and converted it to a .tflite model. The .tflite model size is 706 Kb.
The max Tensor Arena Size I found by trial and error is-
constexpr int kTensorArenaSize = 204 * 1024;
But when I am trying to flash the .img to himax board I am getting this error-
`Arena size is too small for all buffers. Needed 212064 but only 191120 was avai.
AllocateTensors() failed
MicroAllocator: Model allocation started before finishing previously allocated l
Failed starting model allocation.
Invoke failed. `
And if I try to increase the Tensor Arena Size it is not able to build .elf model
Do you have any idea what's the issue here?
Let me know if you need any additional info. Thanks!

Hello @chinya07. Which architecture are you using? The cropped mobilenet I am using have only 60 KB. When I convert the .tflite file to .cc, the size becomes 364 KB.  I think it is bigger than 1.5 MB, right? Try to use an architecture that ends up with less than 1.5 MB in size, because the person detection example model has 1.5 MB in size.

Hello @kris-himax. Thank you so much for the help, and sorry for the late reply.
I tried to feed the image the way you suggested and it seems to be working. The output isn't -128 anymore.
I still have to compare the output in the himax with the output using the .tflite file in my notebook, using the TFLite in python.

Can I test these stuffs and later close this issue?

Hi @PCPJ, my .cc model size is 706200 and my cropped mobilenet architecture has 660,578 trainable parameters in total.

@chinya07 hi,
And if I try to increase the Tensor Arena Size it is not able to build .elf model

What is the error message about building .elf file?
I think that maybe your tflite model is too big to put at SRAM or you can try to put .tflite file at flash.
Reference the flash api and image gen tool command
https://github.com/HimaxWiseEyePlus/bsp_tflu/tree/master/HIMAX_WE1_EVB_SDK#flash-api

Hi @kris-himax, sorry for the late reply.
Yes that is correct my model size was too big and it got even bigger while building the image. I cropped few more layers and made my model smaller and it worked! Thanks for your support and I will look into the flash-api as well.


Introduced two version support feature

Now that Vald can support multiple versions, clients should also be able to support multiple versions.
In this PR, I implemented a workflow to support multiple versions.
The main logic is implemented as a common workflow in vald-client-ci, so I implemented a call to it.

Summary by CodeRabbit

New Features

Updated release workflow to include licensing information, trigger configurations, and specific client settings.
Introduced a new workflow for syncing Vald using GitHub Actions with enhanced security measures and specific configurations.

Chores

Updated .gitignore to exclude specific directories and files.
Enhanced the Makefile with new variables and targets to improve build and deployment processes.

@coderabbitai review


linker: align CREATE_OBJ_LEVEL and zephyr_linker_section_obj_level

Use the same wildcard pattern for sorting. Note that the range wildcard, e.g. [0-9], can't be used as not all linkers support it.

Note: compliance issues are false positives.

@galak ping, related to https://github.com/zephyrproject-rtos/zephyr/pull/57282#discussion_r1185872653


When use Alias convertors not work

When select field projections with alias, converters not work.
Seems like problem in GenericMapping::read.
Because alias create new AliasedExpression, in GenericMapping::read should corresponding unwrapped expression.
Its very disappointed, I select different fields from different tables dynamically using generated attributes and so need to use alias, but in this way converters not work(
Maybe someone can suggested workaround?

Currently, for me this work:
class AliasAttributeDelegate<T, V>(attribute: QueryAttribute<T, V>, val aliasName: String) :
QueryAttribute<T, V> by attribute {
override fun getAlias(): String {
return aliasName
}
}


Add elm-slicex as lib for download stats

Thanks a lot @Wauplin ! We changed to a wildcard.
It would be helpful if we can get this approved as soon as possible as we are planning to push out the models today. Your help is much appreciated.

Hello @Wauplin! Thanks for merging our pull request. However, model download counts doesn't seem to be updating at all even after a week, despite independent cloning by other users for testing purposes. Our analytics for the demo in spaces appear to be functioning correctly.
Note - I've already attempted placing <model_folder> along with slicex_elm_config.json in both locations - root/<model_folder> and root/models/<model_folder>, to see if wildcards activate differently, but unfortunately, it didn't resolve the issue.
Could you please help us debug?

Hi @chinnadhuraisankar, sorry about that. Do you have an example repo I could check?

Example --> https://huggingface.co/slicexai/elm-v0.1_news_classification/tree/main

Hi @Wauplin! Any leads on what's causing the 'download stats' issue?

Hello @Wauplin! Checking in to see if you had a chance to look at the download_stats issue?


feat: Fresh plugin

This PR adds a basic Fresh plugin. I'll update the Fresh + Deno KV OAuth demo to use the plugin and refer to it again in a follow-up PR.
Prerequisite #169
Prerequisite #175
Feel free to review, @deer

deno_kv_oauth's simplicity really pays off, really nice to see how straight forward the plugin is!
well done! 👏 👏

It's still a question of whether this should be added to Fresh as an included plugin. But I'll merge this now as it's better for the feature to be tried and tested sooner. Thanks everyone for your help! 👍🏾


NCCL reports error when at the end of training epoch in multi-node PPO with Ray.

The training fails at the end of training epoch for updating vLLM model weights in PPO with Ray. Is there any advice for fixing the issue?. Thanks in advance!
Log:
Train epoch [1/1]:  99%|█████████▉| 127/128 [23:32<00:06,  6.63s/it, pg=-0.0683, rm=-0.5, ret=-0.5, glen=7.28e+3, tlen=7.4e+3, kl=0, act_lr=4.95e-7]
(ActorModelRayActor pid=496948)
Train epoch [1/1]: 100%|██████████| 128/128 [23:32<00:00, 26.11s/it, pg=-0.0683, rm=-0.5, ret=-0.5, glen=7.28e+3, tlen=7.4e+3, kl=0, act_lr=4.95e-7]
Train epoch [1/1]: 100%|██████████| 128/128 [23:32<00:00, 11.04s/it, pg=-0.0683, rm=-0.5, ret=-0.5, glen=7.28e+3, tlen=7.4e+3, kl=0, act_lr=4.95e-7]
(LLMRayActor pid=90621, ip=<IP_ADDRESS>) update weight: model.embed_tokens.weight, dtype: torch.bfloat16, shape: torch.Size([152064, 5120])
(LLMRayActor pid=82505, ip=<IP_ADDRESS>) update weight: model.layers.0.self_attn.q_proj.weight, dtype: torch.bfloat16, shape: torch.Size([5120, 5120])
(LLMRayActor pid=82508, ip=<IP_ADDRESS>) update weight: model.embed_tokens.weight, dtype: torch.bfloat16, shape: torch.Size([152064, 5120]) [repeated 7x across cluster]
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) mlx5: host_unknown: got completion with error:
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) mlx5: host_unknown: got completion with error:
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) [rank0]:[E1224 13:23:05.471175635 ProcessGroupNCCL.cpp:542] [Rank 11] Collective WorkNCCL(SeqNum=1, OpType=BROADCAST, NumelIn=778567680, NumelOut=778567680, Timeout(ms)=1800000) raised the following async exception: NCCL error: remote process exited or there was a network error, NCCL version 2.21.5
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) ncclRemoteError: A call failed possibly due to a network error or a remote process exiting prematurely.
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) Last error:
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) Attempt to use communicator before the previous operation returned ncclSuccess
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) Exception raised from checkForNCCLErrorsInternal at ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:2027 (most recent call first):
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x96 (0x7f57881f7446 in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libc10.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #1: c10d::ProcessGroupNCCL::checkForNCCLErrorsInternal(std::shared_ptr<c10d::NCCLComm>&) + 0x220 (0x7f542b16cf80 in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #2: c10d::ProcessGroupNCCL::WorkNCCL::checkAndSetException() + 0x7c (0x7f542b16d1cc in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #3: c10d::ProcessGroupNCCL::watchdogHandler() + 0x213 (0x7f542b174b93 in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #4: c10d::ProcessGroupNCCL::ncclCommWatchdog() + 0x14d (0x7f542b17661d in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #5: <unknown function> + 0xd6df4 (0x7f5841ad1df4 in /usr/lib/x86_64-linux-gnu/libstdc++.so.6)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #6: <unknown function> + 0x8609 (0x7f5844119609 in /usr/lib/x86_64-linux-gnu/libpthread.so.0)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #7: clone + 0x43 (0x7f5843ee4353 in /usr/lib/x86_64-linux-gnu/libc.so.6)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) [rank0]:[E1224 13:23:05.472808472 ProcessGroupNCCL.cpp:1785] [PG ID 4 PG GUID openrlhf Rank 11] Exception (either an error or timeout) detected by watchdog at work: 1, last enqueued NCCL work: 1, last completed NCCL work: -1.
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) [rank0]:[E1224 13:23:05.472816987 ProcessGroupNCCL.cpp:630] [Rank 11] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data.
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) [rank0]:[E1224 13:23:05.472820970 ProcessGroupNCCL.cpp:636] [Rank 11] To avoid data inconsistency, we are taking the entire process down.
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) [rank0]:[E1224 13:23:05.472853083 ProcessGroupNCCL.cpp:1595] [PG ID 4 PG GUID openrlhf Rank 11] Process group watchdog thread terminated with exception: NCCL error: remote process exited or there was a network error, NCCL version 2.21.5
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) ncclRemoteError: A call failed possibly due to a network error or a remote process exiting prematurely.
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) Last error:
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) Attempt to use communicator before the previous operation returned ncclSuccess
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) Exception raised from checkForNCCLErrorsInternal at ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:2027 (most recent call first):
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x96 (0x7f57881f7446 in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libc10.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #1: c10d::ProcessGroupNCCL::checkForNCCLErrorsInternal(std::shared_ptr<c10d::NCCLComm>&) + 0x220 (0x7f542b16cf80 in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #2: c10d::ProcessGroupNCCL::WorkNCCL::checkAndSetException() + 0x7c (0x7f542b16d1cc in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #3: c10d::ProcessGroupNCCL::watchdogHandler() + 0x213 (0x7f542b174b93 in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #4: c10d::ProcessGroupNCCL::ncclCommWatchdog() + 0x14d (0x7f542b17661d in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #5: <unknown function> + 0xd6df4 (0x7f5841ad1df4 in /usr/lib/x86_64-linux-gnu/libstdc++.so.6)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #6: <unknown function> + 0x8609 (0x7f5844119609 in /usr/lib/x86_64-linux-gnu/libpthread.so.0)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #7: clone + 0x43 (0x7f5843ee4353 in /usr/lib/x86_64-linux-gnu/libc.so.6)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,791 E 90738 94321] logging.cc:101: Unhandled exception: N3c1016DistBackendErrorE. what(): [PG ID 4 PG GUID openrlhf Rank 11] Process group watchdog thread terminated with exception: NCCL error: remote process exited or there was a network error, NCCL version 2.21.5
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) ncclRemoteError: A call failed possibly due to a network error or a remote process exiting prematurely.
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) Last error:
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) Attempt to use communicator before the previous operation returned ncclSuccess
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) Exception raised from checkForNCCLErrorsInternal at ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:2027 (most recent call first):
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x96 (0x7f57881f7446 in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libc10.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #1: c10d::ProcessGroupNCCL::checkForNCCLErrorsInternal(std::shared_ptr<c10d::NCCLComm>&) + 0x220 (0x7f542b16cf80 in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #2: c10d::ProcessGroupNCCL::WorkNCCL::checkAndSetException() + 0x7c (0x7f542b16d1cc in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #3: c10d::ProcessGroupNCCL::watchdogHandler() + 0x213 (0x7f542b174b93 in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #4: c10d::ProcessGroupNCCL::ncclCommWatchdog() + 0x14d (0x7f542b17661d in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #5: <unknown function> + 0xd6df4 (0x7f5841ad1df4 in /usr/lib/x86_64-linux-gnu/libstdc++.so.6)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #6: <unknown function> + 0x8609 (0x7f5844119609 in /usr/lib/x86_64-linux-gnu/libpthread.so.0)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #7: clone + 0x43 (0x7f5843ee4353 in /usr/lib/x86_64-linux-gnu/libc.so.6)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) Exception raised from ncclCommWatchdog at ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:1601 (most recent call first):
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x96 (0x7f57881f7446 in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libc10.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #1: <unknown function> + 0xe4271b (0x7f542ade371b in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #2: <unknown function> + 0xd6df4 (0x7f5841ad1df4 in /usr/lib/x86_64-linux-gnu/libstdc++.so.6)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #3: <unknown function> + 0x8609 (0x7f5844119609 in /usr/lib/x86_64-linux-gnu/libpthread.so.0)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) frame #4: clone + 0x43 (0x7f5843ee4353 in /usr/lib/x86_64-linux-gnu/libc.so.6)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,801 E 90738 94321] logging.cc:108: Stack trace:
(LLMRayActor pid=90738, ip=<IP_ADDRESS>)  /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/ray/_raylet.so(+0xfe573a) [0x7f5842bcc73a] ray::operator<<()
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/ray/_raylet.so(+0xfe81f8) [0x7f5842bcf1f8] ray::TerminateHandler()
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) /usr/lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f5844119609] start_thread
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) /usr/lib/x86_64-linux-gnu/libc.so.6(clone+0x43) [0x7f5843ee4353] __clone
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) *** SIGABRT received at time=1735017785 on cpu 60 ***
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) PC: @     0x7f5843e0800b  (unknown)  raise
(LLMRayActor pid=90738, ip=<IP_ADDRESS>)     @     0x7f5844125420       4176  (unknown)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>)     @     0x7f5841aa537c  (unknown)  (unknown)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>)     @     0x7f5841aa5090  (unknown)  (unknown)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,802 E 90738 94321] logging.cc:365: *** SIGABRT received at time=1735017785 on cpu 60 ***
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,802 E 90738 94321] logging.cc:365: PC: @     0x7f5843e0800b  (unknown)  raise
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,802 E 90738 94321] logging.cc:365:     @     0x7f5844125420       4176  (unknown)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,802 E 90738 94321] logging.cc:365:     @     0x7f5841aa537c  (unknown)  (unknown)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,802 E 90738 94321] logging.cc:365:     @     0x7f5841aa5090  (unknown)  (unknown)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) Fatal Python error: Aborted
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) Extension modules: msgpack._cmsgpack, google._upb._message, psutil._psutil_linux, psutil._psutil_posix, setproctitle, yaml._yaml, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, uvloop.loop, ray._raylet, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, pyarrow.lib, pyarrow._json, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, markupsafe._speedups, PIL._imaging, msgspec._core, PIL._imagingft, sentencepiece._sentencepiece, regex._regex, multidict._multidict, yarl._quoting_c, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, frozenlist._frozenlist, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pyarrow._compute, pandas._libs.ops, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, pyarrow._parquet, pyarrow._fs, pyarrow._azurefs, pyarrow._hdfs, pyarrow._gcsfs, pyarrow._s3fs, xxhash._xxhash, pyarrow._acero, pyarrow._csv, pyarrow._dataset, pyarrow._dataset_orc, pyarrow._parquet_encryption, pyarrow._dataset_parquet_encryption, pyarrow._dataset_parquet, zmq.backend.cython._zmq, cuda_utils, __triton_launcher (total: 109)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>)  [repeated 20x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) Extension modules: msgpack._cmsgpack, google._upb._message, psutil._psutil_linux, psutil._psutil_posix, setproctitle, yaml._yaml, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, uvloop.loop, ray._raylet, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, markupsafe._speedups, PIL._imaging, msgspec._core, PIL._imagingft, sentencepiece._sentencepiece, regex._regex, multidict._multidict, yarl._quoting_c, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, frozenlist._frozenlist, pyarrow.lib, pyarrow._json, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pyarrow._compute, pandas._libs.ops, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, pyarrow._parquet, pyarrow._fs, pyarrow._azurefs, pyarrow._hdfs, pyarrow._gcsfs, pyarrow._s3fs, xxhash._xxhash, pyarrow._acero, pyarrow._csv, pyarrow._dataset, pyarrow._dataset_orc, pyarrow._parquet_encryption, pyarrow._dataset_parquet_encryption, pyarrow._dataset_parquet, zmq.backend.cython._zmq, cuda_utils, __triton_launcher (total: 109)
(LLMRayActor pid=90738, ip=<IP_ADDRESS>) /opt/aps/python/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
(LLMRayActor pid=90738, ip=<IP_ADDRESS>)   warnings.warn('resource_tracker: There appear to be %d '
(raylet) A worker died or was killed while executing a task by an unexpected system error. To troubleshoot the problem, check the logs for the dead worker. RayTask ID: ffffffffffffffff40d332d228068defd963d9e311000000 Worker ID: 28203740bf646465a43f38b5c5ceb9f883cadeab124e26a78bd7f040 Node ID: 8440b94a47cd6aec9329c5f1a1ac0cc7ea3b7fe6586ac18245731034 Worker IP address: <IP_ADDRESS> Worker port: 10194 Worker PID: 92234 Worker exit type: SYSTEM_ERROR Worker exit detail: Worker unexpectedly exits with a connection error code 2. End of file. There are some potential root causes. (1) The process is killed by SIGKILL by OOM killer due to high memory usage. (2) ray stop --force is called. (3) The worker is crashed unexpectedly due to SIGSEGV or other unexpected errors.
Traceback (most recent call last):
  File "/opt/aps/python/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/aps/python/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/openrlhf/cli/train_ppo_ray.py", line 393, in <module>
    train(args)
  File "/opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/openrlhf/cli/train_ppo_ray.py", line 167, in train
    ray.get(refs)
  File "/opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
    return fn(*args, **kwargs)
  File "/opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
    return func(*args, **kwargs)
  File "/opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/ray/_private/worker.py", line 2623, in get
    values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
  File "/opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/ray/_private/worker.py", line 861, in get_objects
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ActorDiedError): ray::ActorModelRayActor.fit() (pid=496948, ip=<IP_ADDRESS>, actor_id=19916c52fbd6339467e1a91711000000, repr=<openrlhf.trainer.ray.ppo_actor.ActorModelRayActor object at 0x7fe460346bf0>)
  File "/opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/openrlhf/trainer/ray/ppo_actor.py", line 390, in fit
    trainer.fit(
  File "/opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/openrlhf/trainer/ppo_trainer.py", line 237, in fit
    status = self.ppo_train(steps)
  File "/opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/openrlhf/trainer/ray/ppo_actor.py", line 128, in ppo_train
    self._broadcast_to_vllm()
  File "/opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/openrlhf/trainer/ray/ppo_actor.py", line 162, in _broadcast_to_vllm
    ray.get(refs)
ray.exceptions.ActorDiedError: The actor died unexpectedly before finishing this task.
	class_name: LLMRayActor
	actor_id: f274c61d686e9942cf058dcc11000000
	pid: 90738
	namespace: 94838029-010a-46a8-b07f-e6d69f3331f5
	ip: <IP_ADDRESS>
The actor is dead because its worker process has died. Worker exit type: SYSTEM_ERROR Worker exit detail: Worker unexpectedly exits with a connection error code 2. End of file. There are some potential root causes. (1) The process is killed by SIGKILL by OOM killer due to high memory usage. (2) ray stop --force is called. (3) The worker is crashed unexpectedly due to SIGSEGV or other unexpected errors.
(LLMRayActor pid=90739, ip=<IP_ADDRESS>) update weight: model.layers.0.self_attn.q_proj.weight, dtype: torch.bfloat16, shape: torch.Size([5120, 5120]) [repeated 7x across cluster]
(raylet) A worker died or was killed while executing a task by an unexpected system error. To troubleshoot the problem, check the logs for the dead worker. RayTask ID: fffffffffffffffff274c61d686e9942cf058dcc11000000 Worker ID: d893621a96e2f86610a449412008b6be271a3f20fb114a86dc99951c Node ID: 8440b94a47cd6aec9329c5f1a1ac0cc7ea3b7fe6586ac18245731034 Worker IP address: <IP_ADDRESS> Worker port: 10185 Worker PID: 90738 Worker exit type: SYSTEM_ERROR Worker exit detail: Worker unexpectedly exits with a connection error code 2. End of file. There are some potential root causes. (1) The process is killed by SIGKILL by OOM killer due to high memory usage. (2) ray stop --force is called. (3) The worker is crashed unexpectedly due to SIGSEGV or other unexpected errors.
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) mlx5: host_unknown: got completion with error: [repeated 2x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [rank1]:[E1224 13:23:05.483840160 ProcessGroupNCCL.cpp:542] [Rank 12] Collective WorkNCCL(SeqNum=1, OpType=BROADCAST, NumelIn=778567680, NumelOut=778567680, Timeout(ms)=1800000) raised the following async exception: NCCL error: remote process exited or there was a network error, NCCL version 2.21.5
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) ncclRemoteError: A call failed possibly due to a network error or a remote process exiting prematurely. [repeated 3x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) Last error: [repeated 3x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) Attempt to use communicator before the previous operation returned ncclSuccess [repeated 3x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) Exception raised from checkForNCCLErrorsInternal at ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:2027 (most recent call first): [repeated 3x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x96 (0x7f0b20097446 in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libc10.so) [repeated 4x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) frame #4: c10d::ProcessGroupNCCL::ncclCommWatchdog() + 0x14d (0x7f07bf5d361d in /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/torch/lib/libtorch_cuda.so) [repeated 12x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) frame #3: <unknown function> + 0x8609 (0x7f0b359d5609 in /usr/lib/x86_64-linux-gnu/libpthread.so.0) [repeated 9x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) frame #4: clone + 0x43 (0x7f0b357a0353 in /usr/lib/x86_64-linux-gnu/libc.so.6) [repeated 4x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [rank1]:[E1224 13:23:05.485460463 ProcessGroupNCCL.cpp:1785] [PG ID 4 PG GUID openrlhf Rank 12] Exception (either an error or timeout) detected by watchdog at work: 1, last enqueued NCCL work: 1, last completed NCCL work: -1.
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [rank1]:[E1224 13:23:05.485470086 ProcessGroupNCCL.cpp:630] [Rank 12] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data.
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [rank1]:[E1224 13:23:05.485474150 ProcessGroupNCCL.cpp:636] [Rank 12] To avoid data inconsistency, we are taking the entire process down.
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [rank1]:[E1224 13:23:05.485506164 ProcessGroupNCCL.cpp:1595] [PG ID 4 PG GUID openrlhf Rank 12] Process group watchdog thread terminated with exception: NCCL error: remote process exited or there was a network error, NCCL version 2.21.5
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,803 E 92234 94325] logging.cc:101: Unhandled exception: N3c1016DistBackendErrorE. what(): [PG ID 4 PG GUID openrlhf Rank 12] Process group watchdog thread terminated with exception: NCCL error: remote process exited or there was a network error, NCCL version 2.21.5
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) Exception raised from ncclCommWatchdog at ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:1601 (most recent call first):
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,813 E 92234 94325] logging.cc:108: Stack trace:
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>)  /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/ray/_raylet.so(+0xfe573a) [0x7f0b3448873a] ray::operator<<()
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) /opt/aps/workdir/input/jiechen/.venv/lib/python3.10/site-packages/ray/_raylet.so(+0xfe81f8) [0x7f0b3448b1f8] ray::TerminateHandler()
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) /usr/lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7f0b359d5609] start_thread
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) /usr/lib/x86_64-linux-gnu/libc.so.6(clone+0x43) [0x7f0b357a0353] __clone
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) *** SIGABRT received at time=1735017785 on cpu 31 ***
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) PC: @     0x7f0b356c400b  (unknown)  raise
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>)     @     0x7f0b359e1420       4176  (unknown)
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>)     @     0x7f0b33361090  (unknown)  (unknown) [repeated 2x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,813 E 92234 94325] logging.cc:365: *** SIGABRT received at time=1735017785 on cpu 31 ***
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,813 E 92234 94325] logging.cc:365: PC: @     0x7f0b356c400b  (unknown)  raise
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,813 E 92234 94325] logging.cc:365:     @     0x7f0b359e1420       4176  (unknown)
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,814 E 92234 94325] logging.cc:365:     @     0x7f0b33361090  (unknown)  (unknown) [repeated 2x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) Fatal Python error: Aborted
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>)  [repeated 20x across cluster]
2024-12-24 13:23:11,705	ERR cli.py:68 -- ---------------------------------------
2024-12-24 13:23:11,705	ERR cli.py:69 -- Job 'raysubmit_nSWpvu9ifM1t1FEi' failed
2024-12-24 13:23:11,705	ERR cli.py:70 -- ---------------------------------------
2024-12-24 13:23:11,706	INFO cli.py:83 -- Status message: Job entrypoint command failed with exit code 1, last available logs (truncated to 20,000 chars):
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) *** SIGABRT received at time=1735017785 on cpu 31 ***
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) PC: @     0x7f0b356c400b  (unknown)  raise
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>)     @     0x7f0b359e1420       4176  (unknown)
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>)     @     0x7f0b33361090  (unknown)  (unknown) [repeated 2x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,813 E 92234 94325] logging.cc:365: *** SIGABRT received at time=1735017785 on cpu 31 ***
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,813 E 92234 94325] logging.cc:365: PC: @     0x7f0b356c400b  (unknown)  raise
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,813 E 92234 94325] logging.cc:365:     @     0x7f0b359e1420       4176  (unknown)
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) [2024-12-24 13:23:05,814 E 92234 94325] logging.cc:365:     @     0x7f0b33361090  (unknown)  (unknown) [repeated 2x across cluster]
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>) Fatal Python error: Aborted
(RayWorkerWrapper pid=92234, ip=<IP_ADDRESS>)  [repeated 20x across cluster]

Command:
nohup ray job submit --address="http://<IP_ADDRESS>:8265" \
   --runtime-env-json='{"working_dir": "/opt/aps/workdir/input/jiechen/scripts"}' \
   -- /opt/aps/workdir/input/jiechen/.venv/bin/python -m openrlhf.cli.train_ppo_ray \
   --ref_num_nodes 1 \
   --ref_num_gpus_per_node 8 \
   --critic_num_nodes 1 \
   --critic_num_gpus_per_node 8 \
   --actor_num_nodes 1 \
   --actor_num_gpus_per_node 8 \
   --vllm_num_engines 8 \
   --vllm_tensor_parallel_size 2 \
   --colocate_actor_ref \
   --pretrain $TOKENIZER_PATH \
   --remote_rm_url http://localhost:5001/get_reward \
   --save_path /opt/aps/workdir/input/zbc/results/$SAVE_MODEL_NAME \
   --ckpt_path /opt/aps/workdir/input/zbc/results/$SAVE_MODEL_NAME \
   --micro_train_batch_size 1 \
   --train_batch_size $BSZ \
   --micro_rollout_batch_size 1 \
   --rollout_batch_size 32 \
   --max_samples 100000 \
   --max_epochs 1 \
   --num_episodes 10 \
   --n_samples_per_prompt $N_SAMPLES \
   --prompt_max_len 512 \
   --generate_max_len 10000 \
   --zero_stage 3 \
   --bf16 \
   --actor_learning_rate $LR \
   --critic_learning_rate 9e-6 \
   --init_kl_coef $KL_COEF \
   --prompt_data $DATA_PATH \
   --input_key messages \
   --apply_chat_template \
   --packing_samples \
   --normalize_reward \
   --adam_offload \
   --flash_attn \
   --gradient_checkpointing \
   --save_steps 20 \
   --use_wandb ${wandb_token} \
   --wandb_run_name $SAVE_MODEL_NAME \
   --vllm_sync_backend nccl \

use  --vllm_sync_backend gloo for single node

same error


Head response with no body produces 500 Internal Server Error

I'm thinking there's an assumption here that there's a non-nil body in the response:
https://github.com/customink/lamby/blob/f69ac341a0dc0be54c74e30dade9ad9b89bf3ed4/lib/lamby/handler.rb#L20-L24
In a Rails action that uses head and no body in the response, the following error results:
Critical exception from handler
{
"errorMessage": "undefined method `empty?' for nil:NilClass",
"errorType": "Function<NoMethodError>",
"stackTrace": [
"/var/task/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/deflater.rb:99:in `block in each'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/actionpack-<IP_ADDRESS>/lib/action_dispatch/http/response.rb:145:in `each'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/actionpack-<IP_ADDRESS>/lib/action_dispatch/http/response.rb:145:in `each_chunk'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/actionpack-<IP_ADDRESS>/lib/action_dispatch/http/response.rb:126:in `each'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/actionpack-<IP_ADDRESS>/lib/action_dispatch/http/response.rb:74:in `each'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/actionpack-<IP_ADDRESS>/lib/action_dispatch/http/response.rb:473:in `each'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/body_proxy.rb:41:in `method_missing'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/body_proxy.rb:41:in `method_missing'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/body_proxy.rb:41:in `method_missing'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/rack-2.2.3/lib/rack/deflater.rb:96:in `each'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/lamby-3.1.1/lib/lamby/handler.rb:43:in `block in body'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/lamby-3.1.1/lib/lamby/handler.rb:42:in `tap'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/lamby-3.1.1/lib/lamby/handler.rb:42:in `body'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/lamby-3.1.1/lib/lamby/handler.rb:23:in `response'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/lamby-3.1.1/lib/lamby/handler.rb:7:in `call'",
"/var/task/vendor/bundle/ruby/2.7.0/gems/lamby-3.1.1/lib/lamby.rb:25:in `handler'",
"/var/task/app.rb:23:in `handler'"
]
}

Ack. Okay, it turns out Lamby wasn't the culprit here, but rather Rack::Deflater. Catalog is configured strangely for 4xx errors in its weird soup of Rack middleware, Style Bitz custom error pages, the Lamby handler, and Rails application configuration. I still haven't gotten to the bottom of it but needless to say I think we can close this issue.

Thanks Brian! We do have rack deflater tests in place just in case. @avinash-vllbh helped there. So if you think there is a way to express a use test case and Lamby is the obvious fix... let me know!!!


Add retrieve_multiple to backends

Add retrieve_multiple to backends, allow retrieve to use a list of record_identifiers and/or a list of result_identifiers

Closes #96

[ ] Docstrings still need to be updated


Fixes for argument mutations when calling view methods

I was experiencing a problem with Unicorn, so here are some fixes.
Given: a component view method with single argument, type-hinted as a Model.
Problem: consecutive calls to the method with the same arguments would result in successful first call, but exceptions in next calls. Behavior repeats with restart of a worker.
It would seem that parse_call_method_name() output in the lru_cache would later be mutated, resulting in incorrect caching.
P.S. And a few other fixes - I noticed some small problems on the way.

Thank you so much for digging into this bug and making this PR. I can't believe I never ran into it while testing. I really appreciate it and will merge it into main as soon as all the tests pass.

@all-contributors add @stat1c-void for code and tests

I just published 0.49.2 to https://pypi.org/project/django-unicorn/. Thanks again for the fixes. 🦄


IndexError: LightGBMLSS.train(valid_sets=[dataset_val]) calls set_valid_margin, which seems to expect both train+val

I call LightGBMLSS.train like this
early_stop = 20
dataset_params = {"feature_pre_filter": False}
dataset_train = lightgbm.Dataset(
    train_X,
    train_y["value"].ravel(),
    free_raw_data=False,
    params=dataset_params,
)
dataset_val = lightgbm.Dataset(
    val_X,
    val_y["value"].ravel(),
    free_raw_data=False,
    params=dataset_params,
)

distribution = lightgbmlss.model.LightGBMLSS(
    lightgbmlss.distributions.Gaussian.Gaussian(
        stabilization="None",
        response_fn="exp",
        loss_fn="nll",
    )
)

distribution.train(
    params=my_hyperparams,
    num_boost_round=9999999, # early_stopping should kick in before this
    train_set=dataset_train,
    valid_sets=[dataset_val],
    callbacks=[
        lightgbm.early_stopping(early_stop, verbose=False),
        lightgbm.log_evaluation(-1),
    ],
)

It results in this error
File ~/miniconda3/envs/package_name/lib/python3.9/site-packages/lightgbmlss/model.py:575, in LightGBMLSS.set_valid_margin(self, valid_sets, start_values)
    572 init_score_val1 = (np.ones(shape=(valid_sets1.get_label().shape[0], 1))) * start_values
    573 valid_sets1.set_init_score(init_score_val1.ravel(order="F"))
--> 575 valid_sets2 = valid_sets[1]
    576 init_score_val2 = (np.ones(shape=(valid_sets2.get_label().shape[0], 1))) * start_values
    577 valid_sets2.set_init_score(init_score_val2.ravel(order="F"))

IndexError: list index out of range

https://github.com/StatMixedML/LightGBMLSS/blob/master/lightgbmlss/model.py#L559-L579
It seems set_valid_margin expects both the training set and the validation set as parameters, but later when lgb.train is called, the same list is passed in as valid_sets, in addition to the training set via train_set as the first parameter:
https://github.com/StatMixedML/LightGBMLSS/blob/master/lightgbmlss/model.py#L171-L177
Have I misunderstood when I think this will cause the training set to be used as a validation set for early stopping (which may not be desirable) ?

It may also have unexpected behaviour if passed in several - for example 10 - validation sets, as it only processes the first two in the list.

Thanks for your interest in the project. I am on vacation until end of October, so please expect some delay in my reply.

@ninist Sorry for the very late response.
The initial code assumed that there are always two datasets in eval_sets. However, as you pointed out, it makes more sense to loop over the list so that it is also possible to have only one dataset in the list. I pushed the changes, using your suggested code
 for valid_set in valid_sets:
        self.set_init_score(valid_set)

@ninist Can I close this?

Sure, thanks - I assume there is no issue by calling set_init_score twice on the training set if it happens to be included in the list of validation sets too? (once at the start of train() and then again during set_valid_margins)

Sure, thanks - I assume there is no issue by calling set_init_score twice on the training set if it happens to be included in the list of validation sets too? (once at the start of train() and then again during set_valid_margins)

It shouldn't since it is only over-writing the existing values again.


Dynamic Naming of Actionsets using kanctl

Describe the solution you'd like
The kanctl Tool should be able to allow dynamic naming. At the moment 'kanctl' always sets the 'generateName: ' field in the metadata of the Actionset to the value of 'backup-'. This results in an Actionset having a generated name with backup-#####. Ideally, there should be a way to change the value of the generateName field for example by using a flag like --name {NAME}.
Describe alternatives you've considered

Dynamic Naming using the CLI
Generated Name using other information like name of application, time/date of backup, ... e.g. backup-postgres-2023-01-01-23-59

Also, it would be nice if one could add arbitrary labels. Currently we are creating the actionset the way we want by calling kanctl with --dry-run and manipulating the yaml with yq before applying it with kubectl.

@GeekYiwen I just created the PR that should fix the main problem.
@muffl0n should we create another issue for the labels? I can create one.


How to Manage TypeScripts?

A type script determines how the state of a UTXO evolves. For plain money, the states are just amounts and the type script specifies that a) sum of all inputs minus sum of all outputs minus fee is non-negative and b) all outputs are positive. User-defined tokens can either use the same evolution logic or invent their own.
Neptune-core needs to have a way to look up the type scripts based on their hash digests, because in order to generate a transaction, it needs to produce a proof that all type scripts involved halted gracefully. Except for the native coin type script, type scripts can be defined by anyone and do not need to be a part of the software. So how does neptune-core know about them and where does it store them?

So my first question would be:   how is neptune made aware of user-defined tokens and also type-scripts?     I would presume that it is through a transaction, meaning that both the token-type and type-script would be stored in the blockchain.
If not, then why not?    That would seem to be entirely out-of-band....
If stored in the blockchain, then perhaps the problem becomes one of efficient lookup.   When first encountered, we should store these items in a database.   Then we need to be diligent to update the database when rolling back blocks, for example.

how is neptune made aware of user-defined tokens and also type-scripts?

In principle, through out-of-band communication. Type scripts can be tailored to the particular participants. It might be prohibitively expensive to transmit type scripts over the blockchain for this use case.
That said, for some type scripts, such as e.g. those governing open-to-anyone user-defined tokens, the cost of transaction fees might be worth the benefit of wide dissemination. But even so, users who do not care about the token (or even other non-standard type scripts) can ignore them. To the extent that there is less cheap blockchain space, the type script users are paying for that in fees.
Users who want to use the blockchain for transmitting type scripts can use the public announcement field for that purpose.

Then we need to be diligent to update the database when rolling back blocks, for example.

I think the cost of keeping a type script database in sync with the blockchain does not justify the cost. Might as well store all type scripts and ignore rollbacks.


[coinbasepro] Cannot specify a size of base currency in a buy order

Hello,
don't know if I'm missing something here, but according the implementation, I cannot specify a size of the amount of coins I want to buy, in a buy order (market)
public static CoinbaseProPlaceMarketOrder adaptCoinbaseProPlaceMarketOrder(
      MarketOrder marketOrder) {
    return new CoinbaseProPlaceMarketOrder.Builder()
        .productId(adaptProductID(marketOrder.getCurrencyPair()))
        .clientOid(marketOrder.getUserReference())
        .type(CoinbaseProPlaceOrder.Type.market)
        .side(adaptSide(marketOrder.getType()))
        .funds(marketOrder.getType() == OrderType.BID ? marketOrder.getOriginalAmount() : null)
        .size(marketOrder.getType() == OrderType.ASK ? marketOrder.getOriginalAmount() : null)
        .build();
  }

If the order is a BUY, the order Type is BID...so this will result in size 0 and fund equals to the size I specified.
If my transaction is BTC-EUR, and i specify 0.1 in size (of course this means 0.1 BTC to buy), it will translate to 0.1 EUR to use to buy BTC.
Seems that i can specify an amount of coins only if I sell, but not if I buy.
Am I missing something here?

Hi, try to ping again on this, cause I cannot udnerstand the reason why it was implemented in this way.
Accordin to coinbase pro documentation:
MARKET ORDER PARAMETERS
-size | [optional]* Desired amount in base currency
-funds | [optional]* Desired amount of quote currency to use
One of size or funds is required.
But in the code:
 .funds(marketOrder.getType() == OrderType.BID ? marketOrder.getOriginalAmount() : null)
 .size(marketOrder.getType() == OrderType.ASK ? marketOrder.getOriginalAmount() : null)

There is no way to specify a size in a BID order, and there is no way to specify a price (fund) in a ASK order.
Any reason for this?

Hi, try to ping again on this, cause I cannot udnerstand the reason why it was implemented in this way.
Accordin to coinbase pro documentation:
MARKET ORDER PARAMETERS
-size | [optional]* Desired amount in base currency
-funds | [optional]* Desired amount of quote currency to use
One of size or funds is required.
But in the code:
 .funds(marketOrder.getType() == OrderType.BID ? marketOrder.getOriginalAmount() : null)
 .size(marketOrder.getType() == OrderType.ASK ? marketOrder.getOriginalAmount() : null)

There is no way to specify a size in a BID order, and there is no way to specify a price (fund) in a ASK order.
Any reason for this?

coinbases documentation states that you pass either size or funds
you don't pass both
in a BID we set funds, in ASK we set size - thing wrong here
in the original amount just set the amount of currency you want to buy or sell

Hello, thanks for replying.
I don't set both funds and size. I set only size. If you specifiy the size in the originalAmount, in a BUY, the code in xchange will use it as a FUNDS.
I can repeat the example, if my transaction is BTC-EUR, and I specify 0.1 in original amount (of course this means I want to buy 0.1 BTC), it will assign this value to the funds field. So when the BUY is performed, you will buy not 0.1 BTC, but only the BTC that you can buy with 0.1 EUR, and clearly this will return error cause 0.1 is not  enough to reach the limit for transactions in EUR.
I report again the code:
.funds(marketOrder.getType() == OrderType.BID ? marketOrder.getOriginalAmount() : null)
.size(marketOrder.getType() == OrderType.ASK ? marketOrder.getOriginalAmount() : null)
The size is 0 when you perform a BID type order.


Most of the links in roadmap don't work

Bug Report
Steps to Reproduce:
Click a link on any doc in the roadmap section. The link for the meetings don't work either.
Expected Result:
The documents in the link
Actual Result:
Error message
Environment:
...version and build of the project, OS and runtime versions, virtualised environment (if any), etc. ...
Additional Context:
...add any other context about the problem here. If applicable, add screenshots to help explain...

Thanks for reaching out @xevg! Can you provide an example of what type of link isn't working? I tried the various links under roadmap and wasn't seeing any issues. Example:
https://zenith.finos.org/docs/roadmap/roadmap-braintrust

I went through every link and checked whether docosaurus's link checker is working (it does), so, @xevg , can you please provide some more info?


allows you to specify dynamodb endpoint.

the original code only allows to use the local dynamodb, but now, you can specify the dynamodb endpoint so that you can use the AWS's online Dynamodb services.

support the aws credentials profile, so that you can use many  access keys and secret keys at the same time in a credentials file, the only thing you need to do is that specify the parameter --profile when starting confd process.

repair the package error.


Customising ClaimsPrincipal

I was wanting to add my own Role claims to the claims princpal during user login.
However it seems I cant override the claimsprincipal create method to do this.
What would be the suggested way to add claims at the login.
CognitoUserClaimsPrincipalFactory didnt persist the claims past the login page
replacing the HttpContext.User with a generated claims principal didnt keep past login either.

PR #166 should enable this feature. I will leave this pending PR review


fix(clerk-js): Add the render query param on the script

Description
This PR adds the ?render=explicit query param on the CF Turnstile script URL.
Currently, the script is working because the Turnstile script fallbacks to 'explicit' rendering when there is no cf-turnstile element in the DOM, and we use clerk-captcha and clerk-invisible-captcha for the DOM elements.
However, it's better to be explicit about the rendering strategy so that we can avoid accidental issues in the future.
Checklist

[ ] npm test runs as expected.
[ ] npm run build runs as expected.
[ ] (If applicable) JSDoc comments have been added or updated for any package exports
[ ] (If applicable) Documentation has been updated

Type of change

[ ] 🐛 Bug fix
[ ] 🌟 New feature
[ ] 🔨 Breaking change
[ ] 📖 Refactoring / dependency upgrade / documentation
[ ] other:

!snapshot

Hey @anagstef - the snapshot version command generated the following package versions:

Package
Version

@clerk/chrome-extension
1.3.20-snapshot.ve5fa754

@clerk/clerk-js
5.26.6-snapshot.ve5fa754

@clerk/clerk-expo
2.2.26-snapshot.ve5fa754

Tip: Use the snippet copy button below to quickly install the required packages.
@clerk/chrome-extension
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-js
npm i<EMAIL_ADDRESS>--save-exact

@clerk/clerk-expo
npm i<EMAIL_ADDRESS>--save-exact


Fix DI return code checks

The old code retried DI_Read calls until the return value was not negative. However, that never happens. DI_Read and most other DI functions have 1 as the success case, and positive numbers larger than 1 for errors. This means that without this PR, reads could sometimes fail (due to disc issues or whatever), and instead of noticing and retrying, Brainslug would just assume that the data was read correctly.

Addition: That also means that the di_has_partition variable in di.c is probably invalid / wrong, as it also doesn't correctly check the return code. That's not yet fixed with this PR, though.

If you're still actively working on this PR...
https://github.com/Chadderz121/brainslug-wii/blob/88a4069f5e3cb2d1583138e21d0c40ac86a575ab/src/apploader/apploader_get_ios.c#L116
Is this meant to still be ret < 0?


glTF not being rendered correctly. UV texture issue?

Bevy version
bevy = "0.8.1"
What you did
Created an image of a plane in blender with 100x100 m size. Gave a texture for each 1x1meter.

Bevy has no clue UV isn't the whole plane.
    let grid_plane = asset_server.load("models\\planewgrid.gltf#Scene0");

    // plane
    commands
        .spawn_bundle(SceneBundle {
            scene: grid_plane,
            ..Default::default()
        })
        .insert(Collider::cuboid(100.0, 0.1, 100.0));

What went wrong
This is what breaks in bevy:

Additional information
Here's the files:
http://gofile.me/6dlZG/cQxW3Sc6H

Could you try your gltf file in https://gltf-viewer.donmccurdy.com?
Textures done with nodes are often not exported by blender in the gltf format.

Looks fine

Could be missing KHR_texture_transform support, which is how blender exports scaled textures in glTF, I think.


Running workers creates multiple test runs

Howdy!
When we run npx codeceptjs run-workers 3 --suites, it ends up creating 3 test runs.  We are using suites and passing the suiteId in the config.  Do you have any recommendations for how to resolve this?
Thanks!
nak1

So, I'm guessing that means, we have to live the multiple test runs until CodeceptJS provides the events. haha!


release 0.4.2

Fix, see #91

Coverage remained the same at 99.247% when pulling 1b166dcfdd8bf3199f049949a8f97a14192dff78 on release-0.4.2 into 90918eb277f2c2c7c50efedcf534d6eb4d5a486c on master.


Create a key tracking superType for a11y

AccessibleSlider and KeyboardDragListener both have a lot of copied code in how they manage multiple keys and whether each is pressed. It could be nice to have a supertype that manages this all for you.
I am thinking that this type would mainly be made of a public data structure to access the state of the keyboard at any time, and also keydown/up (and potentially more) callbacks. I also see many helper methods like getShiftKeyDown.
I know that we have talked about doing this before, but I think our thought has always been as a "global" tracker, but we can't do that, because it would have to be added to the keydown/up DOM events for the individual PDOM element.
@jessegreenberg and I discussed it briefly on slack, and we think that this type of generic keyboard listener could be quite useful. @jessegreenberg do you have anything else to add?

@jessegreenberg and I discussed briefly the idea of having a global key tracker, but decided it was a bad way to go for the following reasons:

We are worried about how screen readers can effect the bubbling of keydown/up events, so it may be hard to maintain the global state.
There is no way to guarantee that it is in sync from a given listener, because it may have to use a listener attached to the same HTMLElement to update itself.

The alternative is to have this as a supertype, and a11y listeners that need access to the keyboard state will inherit from it.

We also recognize that we don't want to implement the timing stuff in the super type, because then all subtypes (like AccessibleSlider) would need to be stepped by the sim.

We have never (to my knowledge) created a type that extends a mixin and a regular type. I'm not really sure what this would look like, nor do I know how implement it. For now I'm going to hold off.
I also spent a bit more time to look into AccessibleSlider's implementation, and while it does share code with keyboardDragListener, it is not as similar as I thought it was.

We paired on this today during dev meeting. We created a KeyStateTracker, and some boilerplate for testing, that can be used as a way to track which keys are being used on the keyboard.
We decided that this tracker should be a type that can be instantiated multiple times, rather than a global keyboard tracker. One potential usage could be to track the keystate globally (maybe updated in Sim.js or Display.js), and then things like KeyboardDragListener, AccessibleSlider, FuzzBoard, and KeyboardFuzzer (https://github.com/phetsims/scenery/issues/871) can use this global state tracker. But there may be other usages too where we want to track a keystate independently from this global tracker.
The next steps for this are to write more tests for this, use in KeyboardFuzzer, re-implement KeyboardDragListener with this, use AccessibleSlider, and write more tests.

I made a few updates to this.

I updated the keyupUpdate function to delete keys from the object instead of setting them to null. It seemed more complete, and it made it easier to check if any keys were down. Let me know what you think.
I fixed a few bugs in the implementation, especially with the timer functionality and step method.
I added many tests to help exercise these new features.

Next I think we should try to add it to either the keyboard fuzzer, or to a global sim instance that can be used by the drag listener and the accessibleSlider mixin.

There are still TODOs marked for this issue, discovered during https://github.com/phetsims/chipper/issues/946

keyStateTracker tests have been working well without using KeyboardFuzzer.js in it. I removed the TODO for now, and we can come back to it if we need to build out the keyStateTracker tests more.


Visual Bug Where There are Two Hide Password Button Shown in the Login Screen

Description
There are two hide password button shown in the login screen.

Reproduction steps

Login at https://qa-anikultura-md.herokuapp.com/.
Input email and password.

Expected behavior
There should only be 1 button to hide and see password.
Relevant log output
No response
Affected browsers (if applicable)
No response

This bug is dependent on the browser and extensions (e.g. password managers) used. The bug does not appear on Google Chrome without password manager extensions. Users of LastPass Chrome extension observed that the hide password icon is glitched too.


feat: Filter namespaces per MetricPipeline

Description

Changes proposed in this pull request:

...

Related Issues and Documents

Closes:
Related issues:
#514
Traceability

[ ] The PR is linked to a GitHub Issue.
[ ] New features have a milestone label set.
[ ] New features have defined acceptance criteria in a corresponding GitHub Issue, and all criteria are satisfied with this PR.
[ ] The corresponding GitHub Issue has a respective area label.
[ ] The follow-up issues (if any) are linked in the Related Issues section.

Testability
The feature is unit-tested:

[ ] Yes.
[ ] No, because unit tests are not needed.
[ ] No, because of ...

The feature is e2e-tested:

[ ] Yes.
[ ] No, because e2e-tests are not needed.
[ ] No, because of ...

Tests conducted for the PR:

Codebase

[ ] My code follows the Effective Go style guidelines.
[ ] The code was planned and designed following the defined architecture and the separation of concerns.
[ ] The code has sufficient comments, particularly for all hard-to-understand areas.
[ ] This PR adds value and shows no feature creep.
[ ] I have augmented the test suite that proves my fix is effective or that my feature works.
[ ] Adjusted the documentation if the change is user-facing.

Is this not on any project board by purpose?

Is this not on any project board by purpose?

It is not yet ready for review, that is why it is not on the board. But it is already linked to the issue https://github.com/kyma-project/telemetry-manager/pull/567 and the issue is on the board.


use preferred interval value from sensor

I think the idea was to have a read-only (maybe const) register in the sensor


Always use Uri.encodeFull() in route path?

Just found out that I can not use chinese in the route path directly. I have to urlencode them before hand. Eg.,
context.vRouter.push('/页面1') does not work, but context.vRouter.push(Uri.encodeFull('/页面1')) does.
So a small question out of curiosity: Is it a best practice that we always wrap our route path in Uri.encodeFull()? since doing so will not change anything to a ascii string.
I have not seen such a reminder in the documents.

The path is a url and for url to be understood by web browser you must encode them.
You are right I should mention it in the doc !
Also you can use context.vRouter.pushSegments(['/页面1']) which will encode the url for you. This issue is that VRouter can NOT encode a url in push because / would be encoded as well. I could encode inbetween / but then someone might want it to be encoded. As you see there are no idea solution. But mentioning it in the docs is a good start :smile:
Let me know if this answers your question !

Got it and thanks again.
BTW, I guess by pushSegments(['/页面1']) you mean pushSegments(['页面1']), ie., a forward slash should be omitted in a segment since pushSegments will add it for you.

Oh yes you are right ! I updated my answer for any future person coming here, thanks !


Fix for NullReferenceException when deserializing a deleted DbRef

Hi,
I encountered a NullReferenceException during deserialization. The PR includes a test that reproduces the exact scenario. A quick summary of the steps:

create and insert an entity that has a list, which is a DbRef
create an instance of the referenced entity, add it to the first entity and store both
delete the referenced entity (directly, not by removing it from the list of the first entity)
load the first entity again => throws

The fix results in ignoring the deleted entity and just returning an empty list.

Thanks @andypelzer, I will merge in some days after I merge dev branch to master.

Thank you!


feat(tutorial): complete step 2

Closes #
{{short description}}
Changelog
New

{{new thing}}

Changed

{{change thing}}

Removed

{{removed thing}}

wooo number 2


Help ! Completely lost for theming with categories

Hi,
It's been a while but I'm trying to move from version 1 to the latest Pico's development.
But I can't make my theme working :-(
I have a three levels folder system:

root/supcategory/index.md

/*
Title: Mon parcours
Author: Brice Boucard
License: Creative Commons Attribution-ShareAlike 4.0 International License
category: true
Template: supcategory
*/

root/supcategory/category/index.md

/*
Title: Maîtrise Sciences du langage
Author: Brice Boucard
License: Creative Commons Attribution-ShareAlike 4.0 International License
category: true
Template: category
parent: Mon parcours
*/

root/supcategory/category/page.md

/*
Title: Discours : la vulgarisation scientifique
Description: Dossier sur le discours de vulgarisation scientifique, à partir d'un corpus de trois articles du Monde.
Author: Brice Boucard
Date: 2006/10/03
License: Creative Commons Attribution-ShareAlike 4.0 International License
Tags: études,linguistique,discours
parent: Maîtrise Sciences du langage
Template: index
*/

I've read https://github.com/picocms/Pico/issues/412 and https://github.com/picocms/Pico/issues/414 but I can't make anything work :-(
My old theme is here and https://github.com/bricebou/pico_momh
Here are some questions:

in my first level (supcategory), how can I list all the categories (separate folders with index.md within) ?
how can I list all pages in categories ?

Thanks in advance !

There were some breaking changes in Pico 2.0 towards theming. Please refer to the current WIP version of Pico's upgrade docs at http://phrozenbyte.github.io/Pico/in-depth/upgrade-pico-20/ to see what has changed and how to make your theme compatible with Pico 2.0.
Furthermore it seems that you're using the category meta wrong? In category.html there's L14 and L17, so you're comparing a page title (meta.title) with a boolean (category: true in your YAML Front Matter)?

It's because it's the previous version of the code but I've changed my yaml content header because  thought the "category" meta was from the core based on https://github.com/picocms/Pico/issues/414... :-/
Initially, here what a supcategory index.md:

/*
Title: Mon parcours
Author: Brice Boucard
License: Creative Commons Attribution-ShareAlike 4.0 International License
Template: supcategory
*/

And here is the category meta index.md:

/*
Title: Maîtrise Sciences du langage
Author: Brice Boucard
License: Creative Commons Attribution-ShareAlike 4.0 International License
Template: category
Category: Mon parcours
*/

My theme on v1 renders that : http://momh.fr/parcours/M1 But, with v2, it doesn't show anything but the title.

So, I'm trying to use in my category.twig template the current_page.tree_node.children mentioned in https://github.com/picocms/Pico/issues/414 but it seems it's "empty"...
{% for item in current_page.tree_node.children %}      {{ item.category }} - {{ item.title }}
 {{ meta["Category"] }}
                       {% endfor %}

You are not required to use Pico's new page tree system if you had a working solution before. Your old solution should work with Pico 2.0 without any changes necessary.
If you really want to use Pico's page tree system, refer to the current WIP version of Pico's page tree documentation. The tree_node.children array is not a list of pages, but a list of tree nodes. You're rather looking for item.page.meta.category - but note that item.page doesn't necessarily exist (see the docs).

The thing is: what worked doesn't work anymore...
I was using the AdvancedMeta plugin and declared some meta this way:
$config['custom_meta_values'] = array(
    //page category
    'category' => 'Category',
    //page status
    'license' => 'License',
    'thumbnail' => 'Thumbnail'
);

Unfortunately I can't help you if you just give me "it doesn't work anymore" :wink:
The plugin with this config shouldn't make any difference here.

Of course :-)
It was more kind of a question: do I have to keep with the version 2 of Pico the AdvancedMeta plugin, or is there a way to let it aside ?
I thought it was no longer available... and I thought you didn't have to declare custom meta...
But, of course, I wasn't clear --"
Thanks :-)))

There were some changes regarding the handling of meta headers that weren't registered by a plugin. See the 5th paragraph of http://phrozenbyte.github.io/Pico/in-depth/upgrade-pico-20/#amazing-new-features-for-theme-developers. You should access the meta value "Category" using meta["Category"] (note the uppercase C, i.e. case matters now).
The AdvancedMeta plugin never really was necessary. Its existance is a sign that Pico's handling of meta headers was a bit confusing in the past. That's one of the the reason why we no longer do unexpected things and simply pass unregistered meta headers through without change.
Theme developers don't have to register meta values. You just have to make sure that you use the exact same keys, i.e. if your theme uses {{ meta["category"] }}, use category: My category in your YAML Front Matter. If your theme rather uses {{ meta["Category"] }}, use Category: My category in your YAML Front Matter.

Ok...
But in my category.twig template, I have this:
{% for page in pages %}
    {% if page.meta["Category"] is sameas(cate) and page.template != 'category' %}

                <time class="vtl_date meta" datetime="{{ page.date }}" >{{ page.date|date("d/m/Y", "Europe/Paris")}}</time>

                    {{ page.title }}

    {% endif %}
{% endfor %}

Of course, it doesn't produce any result.
If I try to replace page.category with page.meta["Category"], it doesn't produce anything...
Any idea ? Thanks :-)

Try debugging which values the variables page.meta["Category"], page.meta["category"] and cate actually hold. Enalbe Twig's debug mode by adding twig_config: { debug: true } to your config/config.yml and add {{ dump(page.meta["Category"], page.meta["category"], cate) }} to your Twig template.

What does {{ dump(current_page, version) }} output? (outside the for loop)

Here it is:
array(11) { ["id"]=> string(17) "parcours/M1/index" ["url"]=> string(38) "http://dev.localhost/parcours/M1/index" ["title"]=> &string(29) "Maîtrise Sciences du langage" ["description"]=> &string(0) "" ["author"]=> &string(13) "Brice Boucard" ["time"]=> &string(0) "" ["date"]=> &string(0) "" ["date_formatted"]=> &string(0) "" ["raw_content"]=> &string(181) "/* Title: Maîtrise Sciences du langage Author: Brice Boucard License: Creative Commons Attribution-ShareAlike 4.0 International License Template: category Category: Mon parcours */" ["meta"]=> &array(10) { ["title"]=> &string(29) "Maîtrise Sciences du langage" ["author"]=> &string(13) "Brice Boucard" ["license"]=> string(65) "Creative Commons Attribution-ShareAlike 4.0 International License" ["template"]=> string(8) "category" ["category"]=> string(12) "Mon parcours" ["description"]=> &string(0) "" ["date"]=> &string(0) "" ["robots"]=> string(0) "" ["date_formatted"]=> &string(0) "" ["time"]=> &string(0) "" } ["content"]=> &string(0) "" } NULL

It seems that the version doesn't show anything, isn't it ?

Yes, that's because this is no Pico 2.0 installation :wink: You said that you were trying to "move from version 1 to the latest Pico's development", but you're still using Pico 1.0. You can download the latest beta of Pico 2.0 here: https://github.com/picocms/Pico/releases

Hum... I don't understand...
I've used git clone git@github.com:picocms/Pico.git and run composer install... what have I done wrong ?

You've cloned the master branch, that is always the current stable release branch. Until Pico 2.0 is released, Pico 2.0 is found in the pico-1.1 branch (development startet as Pico 1.1 and was later renamed to Pico 2.0). This is explained in Pico's CONTRIBUTING.md. Anyway, you shouldn't use Git branches for a productive website anyway, but rather the latest release.
You can download the latest beta of Pico 2.0 here: https://github.com/picocms/Pico/releases. Pico's pre-bundled release is basically just a Composer installation packed in a ZIP archive. There's virtually no difference between a pre-bundled release and a Composer installation, both use picocms/pico-composer as a basis (however, you can't use picocms/pico-composer directly without changing Composer's minimum-stability until Pico 2.0 is released, thus I'd recommend to simply use the pre-bundled release, composer works with pre-bundled releases, too).

Hum... --' Sorry...
So, I'm now running the latest beta of version 2.
But I still see a problem with no-case sensitive meta.
in my twig category template:
{{ page.template }} -- {{ page.meta["template"]}} -- {{ page.meta["Template"] }}

{{page.category }} -- {{ page.meta["category"]}} -- {{ page.meta["Category"] }}

And here the result:
-- index --
-- Master professionnel Édition -- Master professionnel Édition

And here is the meta header of my content file:
/*
Title: Histoire de l'édition : la censure en France...
Description: Dossier d'histoire de l'édition : la censure en France... de la censure « officielle » à la « censure invisible »
Author: Brice Boucard
Date: 2009/07/02
License: Creative Commons Attribution-ShareAlike 4.0 International License
Tags: études,édition
Category: Master professionnel Édition
Template: index
*/

I can't give you the result of the dump command you gave earlier because of a memory limit (small and slow computer and memory limit fixed in php.ini fixed at 256 MB)...

That's how it is supposed to work.
Template is a registered meta header that intentionally changes the meta header's key. That's the reason why it is only accessible using {{ page.meta["template"] }}. Just refer to Pico's onMetaHeaders event.
The reason why you can access the unregistered meta header Category using both {{ page.meta["category"] }} and {{ page.meta["Category"] }} is explained in http://phrozenbyte.github.io/Pico/in-depth/upgrade-pico-20/: {{ page.meta["Category"] }} is the correct one for Pico 2.0, but PicoDeprecated preserves BC by making the meta value also accessible using {{ page.meta["category"] }}. You can disable PicoDeprecated by adding PicoDeprecated.enabled: false to your config/config.yml.

Sorry ! too fast reading

Made it working ! ... --'
Thanks again :)


Electron: crash dump not produced for process.crash in renderer

Steps to Reproduce (get latest from master):

add a process.crash() somewhere into workbench code that executes after a timeout
run scripts/code.sh --crash-reporter-directory <path to empty folder>
wait for the crash
check the output of the crash folder

=> no dmp file exists (but other files that seem to indicate the crash directory is picked up)
This works fine when triggering a crash on the main side. So I wonder if we are doing something wrong with how we setup the crash reporter.

I actually see the same for forked process.

Here is the result of my OS testing:

Windows

main: success
renderer: success
child-process: success

macOS

main: success
renderer: fail
child-process: fail

Linux

main: success
renderer: success
child-process: fail

On windows and macOS where the underlying crash handler called Crashpad starts monitoring crashes in child processes (this includes render process, gpu process, child node process that are attached to the parent) from the moment its initialized in the main process. So any process that is created before the crash reporter is initialized in the main process will not be monitored. Currently we do the initialization from vs/workbench/electron-browser/window.ts which sort of covers this , but it is best to move the main process crash reporter initialization to src/main.js preferably before app.on(ready). This should fix the issue you are seeing on macOS and will also help with transition for upcoming refactor.
For Linux where the crash handler is Breakpad , it needs to be initialized in respective process and we are doing it correctly. The child process issue here is more of an upstream problem thats fixed in E9.

Awesome, thanks for clarifying. I do not think we can change this for the normal case because the crash reporter can be disabled via setting and we do not read settings so early. But I pushed a change to start the crash reporter right in the beginning when --crash-reporter-directory is provided, please check:
https://github.com/microsoft/vscode/blob/004282afb048d2d6a92e380ea6dbf9d7afd809bd/src/main.js#L60-L66

Change looks good 👍

I do not think we can change this for the normal case because the crash reporter can be disabled via setting and we do not read settings so early.

We have to deprecate this and only allow disabling crash reporter via argv.json , because with E9 crash reporter can only be initialized from the main process. If we allow users to configure via argv.json, we can restart the app and launch with --disable-crash-reporter flag and respect that.

Sounds good, let me rephrase this issue.

This is done :)


Discussion: Products

Ask: N-Tier, data-links, and multi-component services.
Use Case: Provision a server at AWS and add a firewall rule to an ODL router elsewhere so it can be accessed from the private network.
Products should be our place to shine. The core of Jellyfish should provide a powerful system that is well thought out and covers a lot of use cases.
The current vocabulary for defining products does not have the commands to put together cross-provider products. In addition adding new products has to come in the form of coded ProductTypes which require that a new module is installed and the application bounced. Every single new product requires custom development.
ProductTypes can contain custom business logic and that's a good step but Jellyfish should be providing a very powerful system to combine and manage your cloud needs. The architecture and development of a powerful products system should be the priority.
Suggestions

Build a more robust products system
Do away with provider lock-in for products so that they are built in a provider agnostic ways
Allow definitions to be imported into the system

Provide a library url for many definitions
Upload definitions

Provide components that can be combined to make a product.
Have an editor where the definitions can be created and edited by combining components
Develop a DSL to describe component needs so that nearly any cloud provider could be used

Have come across this: http://docs.oasis-open.org/tosca/TOSCA/v1.0/os/TOSCA-v1.0-os.html

Interesting, looks like Cloudify is built using Clojure. I'm wondering what language they are using to do the provisioning and if that ties into any of the native SDK's offered by AWS etc.
I'm still reading through TOSCA to get a sense of how this can be applied to JF. It looks like this would amount to another re-write of the module system which would be a pretty heavy, but perhaps necessary undertaking.
@stackus - were you thinking about changing out module system to be built around TOSCA or just pluck out useful pieces as relevant to JF?

@mafernando I haven't a clue how or with what they're doing the provisioning but the app uses the TOSCO standard so that way there is some separation from the various cloud and openstack endpoints and what your product is.
If you look at at like a modules job as to provide providers and products then yes. The definitions for products become data, and a module would provide translation of TOSCA to itself. Modules would be useful is expanding the vocabulary, for example ODL could add the language to host, depend on, and otherwise connect to OpenDaylight specific routers.
It's not all clear to me how it could come together just yet. But I think this might be something to look into to answer our issues with cross-provider products, extracting products from providers, and would give us an existing vocabulary to build on. TOSCA as a whole.


Add the ability to pull from a specific branch or git commit

While developing install-packer-module I realized, that, for dev purposes, I needed the ability to install a packer module that had not yet been published. The packer-module-in-progress was located either in a branch or was a specific commit.
This PR:

Enables fetch to pull from a specific git commit or branch
For a branch, fetch just pulls the latest commit on the branch
README update
Fixed a bug relating to handling an empty --tag constraint.

^^^ 'cc @brikis98

@brikis98 Merging now because this is a dependency on something else, but feedback welcome as always.

Looks like a really nice improvement. I made one comment about how to design your types. The basic idea to keep in mind is, as much as is reasonable in the language and use case, to make illegal states unrepresentable.


Configurable physics refresh rate

Add to config.json

Closed by #226

Hi - was just reading this from the devs, about using a secretsettings.txt file to override the default 60fps framerate. Is this something the VR mod already does? Given that article notes everything runs on that 60fps tick, was wondering if it's affecting smoothness in VR if it's stuck at 60?

@DanOlner yes, NomaiVR already sets the physics refresh rate of the game to tge hmd's refresh rate. This issue was about adding an option to override that value, but it was eventually removed. If you want to change the refresh of NomaiVR, you have to change your headset's refresh rate (in stramvr or oculus settings or whatever)


getting 404 error for /nodegraph path

getting 404 error for /nodegraph path

Hi @rahul70422
I don't have enough information to give you a specific answer.
I assume your problem could be connected with this one: https://github.com/k8spacket/k8spacket/issues/15

@rahul70422
This thread is inactive for 10 days.
I'm going to close this issue.


Update automanage-hotpatch.md

@saguiitay : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#label:"aq-pr-triaged"

Looks good; merging. Thanks @saguiitay.


operators dynatrace-operator (0.5.0)

Signed-off-by: Michael Rynkiewicz<EMAIL_ADDRESS>Thanks submitting your Operator. Please check below list before you create your Pull Request.
New Submissions

[ ] Are you familiar with our contribution guidelines?
[ ] Have you packaged and deployed your Operator for Operator Framework?

[ ] Have you tested your Operator in all supported installation modes?
[ ] Have you considered whether you want use semantic versioning order?
[ ] Is your submission signed?
[ ] Is operator icon set?

Updates to existing Operators

[x] Did you create a ci.yaml file according to the update instructions?
[x] Is your new CSV pointing to the previous version with the replaces property if you chose replaces-mode via the updateGraph property in ci.yaml?
[x] Is your new CSV referenced in the appropriate channel defined in the package.yaml or annotations.yaml ?
[x] Have you tested an update to your Operator when deployed via OLM?
[x] Is your submission signed?

Your submission should not

[x] Modify more than one operator
[x] Modify an Operator you don't own
[x] Rename an operator - please remove and add with a different name instead
[x] Modify any files outside the above mentioned folders
[x] Contain more than one commit. Please squash your commits.

Operator Description must contain (in order)

[x] Description about the managed Application and where to find more information
[x] Features and capabilities of your Operator and how to use it
[x] Any manual steps about potential pre-requisites for using your Operator

Operator Metadata should contain

[x] Human readable name and 1-liner description about your Operator
[x] Valid category name1
[x] One of the pre-defined capability levels2
[x] Links to the maintainer, source code and documentation
[x] Example templates for all Custom Resource Definitions intended to be used
[x] A quadratic logo

Remember that you can preview your CSV here.
--
1 If you feel your Operator does not fit any of the pre-defined categories, file an issue against this repo and explain your need
2 For more information see here

/retest

@J0zi @mvalarh, could one of you give this PR the "ok-to-test"?

@dt-team-kubernetes you have 2 options.

use replaces replaces: dynatrace-operator.v0.2.2
use brand new channel for 0.5.0 version

In case of option 2 you change package file and CSV remains the same.
Current state will fail on opm command

/merge possible

/merge possible


Implement Dijkstra's Algorithm in Java

Issue Description

Need the Java implementatio of Dijkstra's algorithm. For this issue, please provide the non-optimised O(V^2) implementation.
Corresponding post on website
Permitted languages - Java

Guidelines

Your code will be live on a tutorial website, so code readability and understandability will be given paramount importance.
Please follow coding conventions listed in CONVENTIONS.md
The directory structure of the project is pretty obvious, but take a moment to go through STRUCTURE.md if you are confused where to place your code.
Please add a few sample input output test case files too. Minimum 1. The more the merrier!

@VamsiSangam I will work on this.

Sure! Please proceed @ranesr :smile:


WindowsError: [Error 193] %1 is not a valid Win32 application

Getting this error when running Pokecli.py:
Encrypt.so directory: C:\Users\Win8\Downloads\PokemonGo-Bot-dev\PokemonGo-Bot-d
ev
2016-08-13 11:57:11,019 [PokemonGoBot] [INFO]
Traceback (most recent call last):
File "pokecli.py", line 578, in
main()
File "pokecli.py", line 145, in main
report_summary(bot)
File "pokecli.py", line 180, in report_summary
metrics.capture_stats()
File "C:\Users\Win8\Downloads\PokemonGo-Bot-dev\PokemonGo-Bot-dev\pokemongo_bo
t\metrics.py", line 84, in capture_stats
response_dict = request.call()
File "C:\Users\Win8\Downloads\PokemonGo-Bot-dev\PokemonGo-Bot-dev\pokemongo_bo
t\api_wrapper.py", line 106, in call
result = self._call()
File "C:\Users\Win8\Downloads\PokemonGo-Bot-dev\PokemonGo-Bot-dev\pokemongo_bo
t\api_wrapper.py", line 63, in _call
return PGoApiRequest.call(self)
File "c:\users\win8\downloads\pokemongo-bot-master\src\pgoapi\pgoapi\pgoapi.py
", line 210, in call
request.activate_signature(lib_path)
File "c:\users\win8\downloads\pokemongo-bot-master\src\pgoapi\pgoapi\rpc_api.p
y", line 85, in activate_signature
self.signature_lib = ctypes.cdll.LoadLibrary(lib_path)
File "C:\Python27\lib\ctypes_init.py", line 440, in LoadLibrary
return self.dlltype(name)
File "C:\Python27\lib\ctypes_init.py", line 362, in init
self._handle = _dlopen(self._name, mode)
WindowsError: [Error 193] %1 is not a valid Win32 application
Other Information
OS: Win 8
Branch: dev
Python Version: 2.7.12

Duplicate of #3810
You need to ensure you have the correct dll for you OS