How to convert cv::mat to rs::frame?

Required Info

Camera Model
{  D400 }

Firmware Version
(<IP_ADDRESS>)

Operating System & Version
{Win 10

Platform
PC

SDK Version
{2.13.0 }

Language
{C++/opencv }

Segment
VR/AR }

Issue Description
Hello,
I have already seen this question in the realsense community but didn't get the answer I want.
In rs_align, I have converted the rs2:: frame to cv:: Mat for using the joint bilateral filter provided by OpenCV to improve the quality of the depth map. The depth map of the processed mat type that is obtained cannot be used to call remove_background(rs2::video_frame& other, const rs::frame & depth_frame, float depth_scale, float clipping_dist).
So, I just want to know how to convert cv:: Mat to  rs2:: frame. If you can help me, I would be very grateful.
Thank you!

Hi @Liuyu010
Frame is not exactly equivalent to cv::Mat, as it carries a lot of camera specific information (link to the sensor, metadata, link to calibration, etc...) That's why there is no easy cv-->frame API.
It is possible, of course. I made a modified version of im-show example demonstrating how to go about it:
// License: Apache 2.0. See LICENSE file in root directory.
// Copyright(c) 2017 Intel Corporation. All Rights Reserved.

#include <librealsense2/rs.hpp> // Include RealSense Cross Platform API
#include <opencv2/opencv.hpp>   // Include OpenCV API

int main(int argc, char * argv[]) try
{
    // Declare depth colorizer for pretty visualization of depth data
    rs2::colorizer color_map;

    // Declare RealSense pipeline, encapsulating the actual device and sensors
    rs2::pipeline pipe;
    // Start streaming with default recommended configuration
    pipe.start();

    using namespace cv;
    const auto window_name = "Display Image";
    namedWindow(window_name, WINDOW_AUTOSIZE);

    // Create custom processing block and its output queue:
    rs2::frame_queue q;
    rs2::processing_block pb(
        [](rs2::frame f, rs2::frame_source& src)
    {
        // For each input frame f, do:

        const int w = f.as<rs2::video_frame>().get_width();
        const int h = f.as<rs2::video_frame>().get_height();

        // frame --> cv
        Mat image(Size(w, h), CV_16UC1, (void*)f.get_data(), Mat::AUTO_STEP);
        // do some (silly) processing
        cv::GaussianBlur(image, image, cv::Size(0, 0), 3);

        // Allocate new frame. Copy all missing data from f.
        // This assumes the output is same resolution and format
        // if not true, need to specify parameters explicitly
        auto res = src.allocate_video_frame(f.get_profile(), f);

        // copy from cv --> frame
        memcpy((void*)res.get_data(), image.data, w * h * 2);

        // Send the resulting frame to the output queue
        src.frame_ready(res);
    });
    pb.start(q); // Bind output of the processing block to be enqueued into the queue

    while (waitKey(1) < 0 && cvGetWindowHandle(window_name))
    {
        rs2::frameset data = pipe.wait_for_frames(); // Wait for next set of frames from the camera
        rs2::frame depth = data.get_depth_frame();

        // Send depth frame for processing
        pb.invoke(depth);
        // Wait for results
        depth = q.wait_for_frame();

        // Continue with the new frame
        depth = depth.apply_filter(color_map);

        // Query frame size (width and height)
        const int w = depth.as<rs2::video_frame>().get_width();
        const int h = depth.as<rs2::video_frame>().get_height();

        // Create OpenCV matrix of size (w,h) from the colorized depth data
        Mat image(Size(w, h), CV_8UC3, (void*)depth.get_data(), Mat::AUTO_STEP);

        // Update the window with new data
        imshow(window_name, image);
    }

    return EXIT_SUCCESS;
}
catch (const rs2::error & e)
{
    std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n    " << e.what() << std::endl;
    return EXIT_FAILURE;
}
catch (const std::exception& e)
{
    std::cerr << e.what() << std::endl;
    return EXIT_FAILURE;
}

Hi，dorodnic
Thank you for your reply! Your advice sure makes sense and I have tried to use it in my project. But there are still two problems that bother me.

In the processing_block pb(), not only do I want to pass depth_map into it but also include color_map. When I tried to modify it, I was often reminded that "this function will not be counted as accepting two parameters".How to transfer two parameters to processing_block pb()?
All my project are based on rs-align. So, I tried to use the method you provided in rs_align to process depth_map with OpenCV to see if it could be used as a way to improve background segmentation, but it doesn't work. Didi do something wrong?

Here is the wrong result.

Here is my code.
https://github.com/vivedu/Third-Person-VR/blob/JointBilateralFilter/segmentation.cpp

Finally, I would like to ask you whether there is a good background segmentation program or a good idea on the D400s realsense camera. Although I have tried all the filters you provided in rs_align(like deci_filter、spat_filter、temp_filter、hole_filter etc.), the result is still a serious situation of edge leakage. Looking forward to your reply.

Hi @Liuyu010
To pass two or more frames, you need to bundle them in a frameset. See an example here: rs-measure.cpp:191-233
Third-Person-VR seem to be a private repository, I couldn't access the code.
In general, I think that good background removal would leverage both color and depth data. In rs-grabcuts I tried to feed depth data as initial guess to purely RGB based graph-cuts background removal. This can show decent results, but performance is abysmal.
I wanted to try a hybrid approach for a while now, but couldn't make it work yet.
It is obvious that depth data helps in the problem of background removal, but in terms of depth sensing solutions, we are not yet in a place where you could just take the raw depth map and use it as a threshold. Main issue is that most depth cameras available today are composed from several sensors, each at slightly different location. This means that you will always going to get pixels that one sensor sees but the second does not, resulting in black pixels.
That said, I'm a software engineer, not a computer vision expert, so I'm sure there are people here who have better ideas.
P.S. I tried to fine-tune the example a bit for the D415 and came up with two suggestions:

You can fairly easily enable basic post-processing to get less holes:

rs2::temporal_filter temp;
rs2::hole_filling_filter hfill;

while (app) // Application still alive?
{
    // Using the align object, we block the application until a frameset is available
    rs2::frameset frameset = pipe.wait_for_frames().apply_filter(temp).apply_filter(hfill);

The D415 can deliver imperfect RGB from the left IR sensor (potentially reducing calibration artifacts)
You can try it and compare to the results from RGB sensor:

    rs2::pipeline pipe;

    rs2::config config;
    config.enable_stream(RS2_STREAM_DEPTH);
    config.enable_stream(RS2_STREAM_INFRARED, RS2_FORMAT_RGB8);
    rs2::pipeline_profile profile = pipe.start(config);

    auto sensor = profile.get_device().query_sensors().front();
    sensor.set_option(RS2_OPTION_VISUAL_PRESET, 6.0);
    sensor.set_option(RS2_OPTION_EXPOSURE, 50000.0);
    sensor.set_option(RS2_OPTION_LASER_POWER, sensor.get_option_range(RS2_OPTION_LASER_POWER).max);

[Realsense Customer Engineering Team Comment]
Hi @Liuyu010, If below answer have solved your question, we'd like to close this ticket. Thanks!

Hi，dorodnic
Sorry for reply so late!
I have tried what you advised last week, but it seems that I made the project more Chaotic and can't get the features I want(Maybe because of my unfamiliarity with developing on Intel RealSense).
In rs_align, all the features I want is that I can use the color_map and depth_map fetched from D415/D435 to add the Joint Bilateral Filter on the depth_map(This is a method I think can improve the edge effect of the map very well). But I am still stuck in unknowing how to use the depth_image processed by OpenCV to call  remove_background().
My current problem is the same as the two problems mentioned earlier. I have tried the pb() and changing the format of the input of the depth map of remove_background(), but all failed,
Here is my code. Sincerely hope to get your help!
// License: Apache 2.0. See LICENSE file in root directory.
// Copyright(c) 2017 Intel Corporation. All Rights Reserved.

#include <librealsense2/rs.hpp>
#include "window.hpp"
#include <imgui.h>
#include "imgui_impl_glfw.h"

#include <sstream>
#include <iostream>
#include <fstream>
#include <algorithm>
#include <cstring>

#include <opencv2/opencv.hpp>
#include <ximgproc.hpp>

void render_slider(rect location, float& clipping_dist);
void remove_background(rs2::video_frame& other_frame, const rs2::depth_frame& depth_frame, float depth_scale, float clipping_dist);
float get_depth_scale(rs2::device dev);
rs2_stream find_stream_to_align(const std::vector<rs2::stream_profile>& streams);
bool profile_changed(const std::vector<rs2::stream_profile>& current, const std::vector<rs2::stream_profile>& prev);

int main(int argc, char * argv[]) try
{
    // Create and initialize GUI related objects
    window app(1280, 720, "CPP - Align Example"); // Simple window handling
    ImGui_ImplGlfw_Init(app, false);      // ImGui library intializition
    rs2::colorizer c;                          // Helper to colorize depth images
    texture renderer;                     // Helper for renderig images

    // Create a pipeline to easily configure and start the camera
    rs2::pipeline pipe;
    //Calling pipeline's start() without any additional parameters will start the first device
    // with its default streams.
    //The start function returns the pipeline profile which the pipeline used to start the device
    rs2::pipeline_profile profile = pipe.start();

    // Each depth camera might have different units for depth pixels, so we get it here
    // Using the pipeline's profile, we can retrieve the device that the pipeline uses
    float depth_scale = get_depth_scale(profile.get_device());

    //Pipeline could choose a device that does not have a color stream
    //If there is no color stream, choose to align depth to another stream
    rs2_stream align_to = find_stream_to_align(profile.get_streams());

    // Create a rs2::align object.
    // rs2::align allows us to perform alignment of depth frames to others frames
    //The "align_to" is the stream type to which we plan to align depth frames.
    rs2::align align(align_to);

    // Define a variable for controlling the distance to clip
    float depth_clipping_distance = 1.f;

	// Create custom processing block and its output queue:
	rs2::frame_queue q;
	rs2::processing_block pb(
		[](rs2::frame f, rs2::frame_source& src)
		//[](rs2::frame f, rs2::frame color,rs2::frame_source& src)
	{

		// For each input frame f, do:
		const int w = f.as<rs2::video_frame>().get_width();
		const int h = f.as<rs2::video_frame>().get_height();

		// frame --> cv
		////Transform rs::color_frame to the 8UC3 type of Mat
		//cv::Mat color1(cv::Size(w, h), CV_8UC3, (void*)color.get_data(), cv::Mat::AUTO_STEP);
		//cv::imshow("Display image", color1);

		//Transform rs::depth_frame to the 8U3C type of Mat
		cv::Mat Mat_depth = cv::Mat(h, w, CV_16UC1, (void*)f.get_data());
		cv::GaussianBlur(Mat_depth, Mat_depth, cv::Size(0, 0), 3);
		//cvtColor(Mat_depth, Mat_depth, CV_BGR2GRAY);
		//imshow("Depth", Mat_depth);

		//// Apply Histogram Equalization
		//cv::equalizeHist(Mat_depth, Mat_depth);
		//cv::applyColorMap(Mat_depth, Mat_depth, cv::COLORMAP_JET);
		////cv::imshow("Display depth", depth);

		//ADD the Joint Bilateral Filter...
		//cv::Mat dst, dst1;
		//cv::ximgproc::jointBilateralFilter(color, Mat_depth, dst, -1, 3, 11);
		////cv::ximgproc::jointBilateralFilter(color, dst, dst1, -1, 3, 11);
		//cv::imshow("Display dst", dst);

		// Allocate new frame. Copy all missing data from f.
		// This assumes the output is same resolution and format
		// if not true, need to specify parameters explicitly
		auto res = src.allocate_video_frame(f.get_profile(), f);

		// copy from cv --> frame
		memcpy((void*)res.get_data(), Mat_depth.data, w * h * 2);

		// Send the resulting frame to the output queue
		src.frame_ready(res);
	});
	pb.start(q); // Bind output of the processing block to be enqueued into the queue

    while (app) // Application still alive?
    {
        // Using the align object, we block the application until a frameset is available
        rs2::frameset frameset = pipe.wait_for_frames();

        // rs2::pipeline::wait_for_frames() can replace the device it uses in case of device error or disconnection.
        // Since rs2::align is aligning depth to some other stream, we need to make sure that the stream was not changed
        //  after the call to wait_for_frames();
        if (profile_changed(pipe.get_active_profile().get_streams(), profile.get_streams()))
        {
            //If the profile was changed, update the align object, and also get the new device's depth scale
            profile = pipe.get_active_profile();
            align_to = find_stream_to_align(profile.get_streams());
            align = rs2::align(align_to);
            depth_scale = get_depth_scale(profile.get_device());
        }

        //Get processed aligned frame
        auto processed = align.process(frameset);

        // Trying to get both other and aligned depth frames
        rs2::video_frame other_frame = processed.first(align_to);
        rs2::frame aligned_depth_frame = processed.get_depth_frame();

		pb.invoke(aligned_depth_frame);
		// Wait for results
		aligned_depth_frame = q.wait_for_frame();

		// Query frame size (width and height)
		const int width = aligned_depth_frame.as<rs2::video_frame>().get_width();
		const int height = aligned_depth_frame.as<rs2::video_frame>().get_height();
		rs2::colorizer color;
		rs2::frame colorDepth = aligned_depth_frame.apply_filter(color);
		// Create OpenCV matrix of size (w,h) from the colorized depth data
		cv::Mat image(cv::Size(width, height), CV_8UC3, (void*)colorDepth.get_data(), cv::Mat::AUTO_STEP);

		// Update the window with new data
		imshow("CVshow", image);

        //If one of them is unavailable, continue iteration
        if (!aligned_depth_frame || !other_frame)
        {
            continue;
        }
        // Passing both frames to remove_background so it will "strip" the background
        // NOTE: in this example, we alter the buffer of the other frame, instead of copying it and altering the copy
        //       This behavior is not recommended in real application since the other frame could be used elsewhere
        remove_background(other_frame, aligned_depth_frame, depth_scale, depth_clipping_distance);

        // Taking dimensions of the window for rendering purposes
        float w = static_cast<float>(app.width());
        float h = static_cast<float>(app.height());

        // At this point, "other_frame" is an altered frame, stripped form its background
        // Calculating the position to place the frame in the window
        rect altered_other_frame_rect{ 0, 0, w, h };
        altered_other_frame_rect = altered_other_frame_rect.adjust_ratio({ static_cast<float>(other_frame.get_width()),static_cast<float>(other_frame.get_height()) });

        // Render aligned image
        renderer.render(other_frame, altered_other_frame_rect);

        // The example also renders the depth frame, as a picture-in-picture
        // Calculating the position to place the depth frame in the window
        rect pip_stream{ 0, 0, w / 5, h / 5 };
        pip_stream = pip_stream.adjust_ratio({ static_cast<float>(width),static_cast<float>(height) });
        pip_stream.x = altered_other_frame_rect.x + altered_other_frame_rect.w - pip_stream.w - (std::max(w, h) / 25);
        pip_stream.y = altered_other_frame_rect.y + altered_other_frame_rect.h - pip_stream.h - (std::max(w, h) / 25);

        // Render depth (as picture in pipcture)
        renderer.upload(c.process(aligned_depth_frame));
        renderer.show(pip_stream);

        // Using ImGui library to provide a slide controller to select the depth clipping distance
        ImGui_ImplGlfw_NewFrame(1);
        render_slider({ 5.f, 0, w, h }, depth_clipping_distance);
        ImGui::Render();

    }
    return EXIT_SUCCESS;
}
catch (const rs2::error & e)
{
    std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n    " << e.what() << std::endl;
    return EXIT_FAILURE;
}
catch (const std::exception & e)
{
    std::cerr << e.what() << std::endl;
    return EXIT_FAILURE;
}

float get_depth_scale(rs2::device dev)
{
    // Go over the device's sensors
    for (rs2::sensor& sensor : dev.query_sensors())
    {
        // Check if the sensor if a depth sensor
        if (rs2::depth_sensor dpt = sensor.as<rs2::depth_sensor>())
        {
            return dpt.get_depth_scale();
        }
    }
    throw std::runtime_error("Device does not have a depth sensor");
}

void render_slider(rect location, float& clipping_dist)
{
    // Some trickery to display the control nicely
    static const int flags = ImGuiWindowFlags_NoCollapse
        | ImGuiWindowFlags_NoScrollbar
        | ImGuiWindowFlags_NoSavedSettings
        | ImGuiWindowFlags_NoTitleBar
        | ImGuiWindowFlags_NoResize
        | ImGuiWindowFlags_NoMove;
    const int pixels_to_buttom_of_stream_text = 25;
    const float slider_window_width = 30;

    ImGui::SetNextWindowPos({ location.x, location.y + pixels_to_buttom_of_stream_text });
    ImGui::SetNextWindowSize({ slider_window_width + 20, location.h - (pixels_to_buttom_of_stream_text * 2) });

    //Render the vertical slider
    ImGui::Begin("slider", nullptr, flags);
    ImGui::PushStyleColor(ImGuiCol_FrameBg, ImColor(215.f / 255, 215.0f / 255, 215.0f / 255));
    ImGui::PushStyleColor(ImGuiCol_SliderGrab, ImColor(215.f / 255, 215.0f / 255, 215.0f / 255));
    ImGui::PushStyleColor(ImGuiCol_SliderGrabActive, ImColor(215.f / 255, 215.0f / 255, 215.0f / 255));
    auto slider_size = ImVec2(slider_window_width / 2, location.h - (pixels_to_buttom_of_stream_text * 2) - 20);
    ImGui::VSliderFloat("", slider_size, &clipping_dist, 0.0f, 6.0f, "", 1.0f, true);
    if (ImGui::IsItemHovered())
        ImGui::SetTooltip("Depth Clipping Distance: %.3f", clipping_dist);
    ImGui::PopStyleColor(3);

    //Display bars next to slider
    float bars_dist = (slider_size.y / 6.0f);
    for (int i = 0; i <= 6; i++)
    {
        ImGui::SetCursorPos({ slider_size.x, i * bars_dist });
        std::string bar_text = "- " + std::to_string(6-i) + "m";
        ImGui::Text("%s", bar_text.c_str());
    }
    ImGui::End();
}

void remove_background(rs2::video_frame& other_frame, const rs2::depth_frame& depth_frame, float depth_scale, float clipping_dist)
{
	const uint16_t* p_depth_frame = reinterpret_cast<const uint16_t*>(depth_frame.get_data());
	uint8_t* p_other_frame = reinterpret_cast<uint8_t*>(const_cast<void*>(other_frame.get_data()));

	int width = other_frame.get_width();
	int height = other_frame.get_height();
	int other_bpp = other_frame.get_bytes_per_pixel();

#pragma omp parallel for schedule(dynamic) //Using OpenMP to try to parallelise the loop
	for (int y = 0; y < height; y++)
	{
		auto depth_pixel_index = y * width;
		for (int x = 0; x < width; x++, ++depth_pixel_index)
		{
			// Get the depth value of the current pixel
			auto pixels_distance = depth_scale * p_depth_frame[depth_pixel_index];

			// Check if the depth value is invalid (<=0) or greater than the threashold
			if (pixels_distance <= 0.f || pixels_distance > clipping_dist)
			{
				// Calculate the offset in other frame's buffer to current pixel
				auto offset = depth_pixel_index * other_bpp;

				// Set pixel to "background" color (0x999999)
				std::memset(&p_other_frame[offset], 0x99, other_bpp);
			}
		}
	}
}
rs2_stream find_stream_to_align(const std::vector<rs2::stream_profile>& streams)
{
    //Given a vector of streams, we try to find a depth stream and another stream to align depth with.
    //We prioritize color streams to make the view look better.
    //If color is not available, we take another stream that (other than depth)
    rs2_stream align_to = RS2_STREAM_ANY;
    bool depth_stream_found = false;
    bool color_stream_found = false;
    for (rs2::stream_profile sp : streams)
    {
        rs2_stream profile_stream = sp.stream_type();
        if (profile_stream != RS2_STREAM_DEPTH)
        {
            if (!color_stream_found)         //Prefer color
                align_to = profile_stream;

            if (profile_stream == RS2_STREAM_COLOR)
            {
                color_stream_found = true;
            }
        }
        else
        {
            depth_stream_found = true;
        }
    }

    if(!depth_stream_found)
        throw std::runtime_error("No Depth stream available");

    if (align_to == RS2_STREAM_ANY)
        throw std::runtime_error("No stream found to align with Depth");

    return align_to;
}

bool profile_changed(const std::vector<rs2::stream_profile>& current, const std::vector<rs2::stream_profile>& prev)
{
    for (auto&& sp : prev)
    {
        //If previous profile is in current (maybe just added another)
        auto itr = std::find_if(std::begin(current), std::end(current), [&sp](const rs2::stream_profile& current_sp) { return sp.unique_id() == current_sp.unique_id(); });
        if (itr == std::end(current)) //If it previous stream wasn't found in current
        {
            return true;
        }
    }
    return false;
}

[Realsense Customer Engineering Team Comment]
Hi @Liuyu010 ,
from your posts, your question is "how to use the depth_image processed by OpenCV to call remove_background()",  could you reference following modified rs-align.cpp(librealsense/examples/align/rs-align.cpp):
// License: Apache 2.0. See LICENSE file in root directory.
// Copyright(c) 2017 Intel Corporation. All Rights Reserved.
#include <librealsense2/rs.hpp>
#include "../example.hpp"
#include <imgui.h>
#include "imgui_impl_glfw.h"
#include
#include
#include
#include
#include
#include <opencv2/opencv.hpp>  // Include OpenCV API
using namespace cv;
void render_slider(rect location, float& clipping_dist);
void remove_background(rs2::video_frame& other, const rs2::depth_frame& depth_frame, float depth_scale, float clipping_dist);
float get_depth_scale(rs2::device dev);
rs2_stream find_stream_to_align(const std::vectorrs2::stream_profile& streams);
bool profile_changed(const std::vectorrs2::stream_profile& current, const std::vectorrs2::stream_profile& prev);
int main(int argc, char * argv[]) try
{
// Create and initialize GUI related objects
window app(1280, 720, "CPP - Align Example"); // Simple window handling
ImGui_ImplGlfw_Init(app, false);      // ImGui library intializition
rs2::colorizer c;                          // Helper to colorize depth images
texture renderer;                    // Helper for renderig images
// Create a pipeline to easily configure and start the camera
rs2::pipeline pipe;
//Calling pipeline's start() without any additional parameters will start the first device
// with its default streams.
//The start function returns the pipeline profile which the pipeline used to start the device
rs2::pipeline_profile profile = pipe.start();

// Each depth camera might have different units for depth pixels, so we get it here
// Using the pipeline's profile, we can retrieve the device that the pipeline uses
float depth_scale = get_depth_scale(profile.get_device());

//Pipeline could choose a device that does not have a color stream
//If there is no color stream, choose to align depth to another stream
rs2_stream align_to = find_stream_to_align(profile.get_streams());

// Create a rs2::align object.
// rs2::align allows us to perform alignment of depth frames to others frames
//The "align_to" is the stream type to which we plan to align depth frames.
rs2::align align(align_to);

// Define a variable for controlling the distance to clip
float depth_clipping_distance = 1.f;

// Create custom processing block and its output queue:
rs2::frame_queue q;
rs2::processing_block pb(
[](rs2::frame f, rs2::frame_source& src)
{
// For each input frame f, do:
    const int w = f.as<rs2::video_frame>().get_width();
    const int h = f.as<rs2::video_frame>().get_height();

    // frame --> cv
    Mat image(Size(w, h), CV_16UC1, (void*)f.get_data(), Mat::AUTO_STEP);
    // do some (silly) processing
    cv::GaussianBlur(image, image, cv::Size(0, 0), 3);

    // Allocate new frame. Copy all missing data from f.
    // This assumes the output is same resolution and format
    // if not true, need to specify parameters explicitly
    auto res = src.allocate_video_frame(f.get_profile(), f);

    // copy from cv --> frame
    memcpy((void*)res.get_data(), image.data, w * h * 2);

    // Send the resulting frame to the output queue
    src.frame_ready(res);
});
pb.start(q); // Bind output of the processing block to be enqueued into the queue

while (app) // Application still alive?
{
    // Using the align object, we block the application until a frameset is available
    rs2::frameset frameset = pipe.wait_for_frames();

    // rs2::pipeline::wait_for_frames() can replace the device it uses in case of device error or disconnection.
    // Since rs2::align is aligning depth to some other stream, we need to make sure that the stream was not changed
    //  after the call to wait_for_frames();
    if (profile_changed(pipe.get_active_profile().get_streams(), profile.get_streams()))
    {
        //If the profile was changed, update the align object, and also get the new device's depth scale
        profile = pipe.get_active_profile();
        align_to = find_stream_to_align(profile.get_streams());
        align = rs2::align(align_to);
        depth_scale = get_depth_scale(profile.get_device());
    }

    //Get processed aligned frame
    auto processed = align.process(frameset);

    // Trying to get both other and aligned depth frames
    rs2::video_frame other_frame = processed.first(align_to);
    rs2::depth_frame aligned_depth_frame = processed.get_depth_frame();

    pb.invoke(aligned_depth_frame);
    aligned_depth_frame = q.wait_for_frame();

    //If one of them is unavailable, continue iteration
    if (!aligned_depth_frame || !other_frame)
    {
        continue;
    }
    // Passing both frames to remove_background so it will "strip" the background
    // NOTE: in this example, we alter the buffer of the other frame, instead of copying it and altering the copy
    //      This behavior is not recommended in real application since the other frame could be used elsewhere
    remove_background(other_frame, aligned_depth_frame, depth_scale, depth_clipping_distance);

    // Taking dimensions of the window for rendering purposes
    float w = static_cast<float>(app.width());
    float h = static_cast<float>(app.height());

    // At this point, "other_frame" is an altered frame, stripped form its background
    // Calculating the position to place the frame in the window
    rect altered_other_frame_rect{ 0, 0, w, h };
    altered_other_frame_rect = altered_other_frame_rect.adjust_ratio({ static_cast<float>(other_frame.get_width()),static_cast<float>(other_frame.get_height()) });

    // Render aligned image
    renderer.render(other_frame, altered_other_frame_rect);

    // The example also renders the depth frame, as a picture-in-picture
    // Calculating the position to place the depth frame in the window
    rect pip_stream{ 0, 0, w / 5, h / 5 };
    pip_stream = pip_stream.adjust_ratio({ static_cast<float>(aligned_depth_frame.get_width()),static_cast<float>(aligned_depth_frame.get_height()) });
    pip_stream.x = altered_other_frame_rect.x + altered_other_frame_rect.w - pip_stream.w - (std::max(w, h) / 25);
    pip_stream.y = altered_other_frame_rect.y + altered_other_frame_rect.h - pip_stream.h - (std::max(w, h) / 25);

    // Render depth (as picture in pipcture)
    renderer.upload(c.process(aligned_depth_frame));
    renderer.show(pip_stream);

    // Using ImGui library to provide a slide controller to select the depth clipping distance
    ImGui_ImplGlfw_NewFrame(1);
    render_slider({ 5.f, 0, w, h }, depth_clipping_distance);
    ImGui::Render();

}
return EXIT_SUCCESS;

}
catch (const rs2::error & e)
{
std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n    " << e.what() << std::endl;
return EXIT_FAILURE;
}
catch (const std::exception & e)
{
std::cerr << e.what() << std::endl;
return EXIT_FAILURE;
}
float get_depth_scale(rs2::device dev)
{
// Go over the device's sensors
for (rs2::sensor& sensor : dev.query_sensors())
{
// Check if the sensor if a depth sensor
if (rs2::depth_sensor dpt = sensor.asrs2::depth_sensor())
{
return dpt.get_depth_scale();
}
}
throw std::runtime_error("Device does not have a depth sensor");
}
void render_slider(rect location, float& clipping_dist)
{
// Some trickery to display the control nicely
static const int flags = ImGuiWindowFlags_NoCollapse
| ImGuiWindowFlags_NoScrollbar
| ImGuiWindowFlags_NoSavedSettings
| ImGuiWindowFlags_NoTitleBar
| ImGuiWindowFlags_NoResize
| ImGuiWindowFlags_NoMove;
const int pixels_to_buttom_of_stream_text = 25;
const float slider_window_width = 30;
ImGui::SetNextWindowPos({ location.x, location.y + pixels_to_buttom_of_stream_text });
ImGui::SetNextWindowSize({ slider_window_width + 20, location.h - (pixels_to_buttom_of_stream_text * 2) });

//Render the vertical slider
ImGui::Begin("slider", nullptr, flags);
ImGui::PushStyleColor(ImGuiCol_FrameBg, ImColor(215.f / 255, 215.0f / 255, 215.0f / 255));
ImGui::PushStyleColor(ImGuiCol_SliderGrab, ImColor(215.f / 255, 215.0f / 255, 215.0f / 255));
ImGui::PushStyleColor(ImGuiCol_SliderGrabActive, ImColor(215.f / 255, 215.0f / 255, 215.0f / 255));
auto slider_size = ImVec2(slider_window_width / 2, location.h - (pixels_to_buttom_of_stream_text * 2) - 20);
ImGui::VSliderFloat("", slider_size, &clipping_dist, 0.0f, 6.0f, "", 1.0f, true);
if (ImGui::IsItemHovered())
    ImGui::SetTooltip("Depth Clipping Distance: %.3f", clipping_dist);
ImGui::PopStyleColor(3);

//Display bars next to slider
float bars_dist = (slider_size.y / 6.0f);
for (int i = 0; i <= 6; i++)
{
    ImGui::SetCursorPos({ slider_size.x, i * bars_dist });
    std::string bar_text = "- " + std::to_string(6-i) + "m";
    ImGui::Text("%s", bar_text.c_str());
}
ImGui::End();

}
void remove_background(rs2::video_frame& other_frame, const rs2::depth_frame& depth_frame, float depth_scale, float clipping_dist)
{
const uint16_t* p_depth_frame = reinterpret_cast<const uint16_t*>(depth_frame.get_data());
uint8_t* p_other_frame = reinterpret_cast<uint8_t*>(const_cast<void*>(other_frame.get_data()));
int width = other_frame.get_width();
int height = other_frame.get_height();
int other_bpp = other_frame.get_bytes_per_pixel();

#pragma omp parallel for schedule(dynamic) //Using OpenMP to try to parallelise the loop
for (int y = 0; y < height; y++)
{
    auto depth_pixel_index = y * width;
    for (int x = 0; x < width; x++, ++depth_pixel_index)
    {
        // Get the depth value of the current pixel
        auto pixels_distance = depth_scale * p_depth_frame[depth_pixel_index];

        // Check if the depth value is invalid (<=0) or greater than the threashold
        if (pixels_distance <= 0.f || pixels_distance > clipping_dist)
        {
            // Calculate the offset in other frame's buffer to current pixel
            auto offset = depth_pixel_index * other_bpp;

            // Set pixel to "background" color (0x999999)
            std::memset(&p_other_frame[offset], 0x99, other_bpp);
        }
    }
}

}
rs2_stream find_stream_to_align(const std::vectorrs2::stream_profile& streams)
{
//Given a vector of streams, we try to find a depth stream and another stream to align depth with.
//We prioritize color streams to make the view look better.
//If color is not available, we take another stream that (other than depth)
rs2_stream align_to = RS2_STREAM_ANY;
bool depth_stream_found = false;
bool color_stream_found = false;
for (rs2::stream_profile sp : streams)
{
rs2_stream profile_stream = sp.stream_type();
if (profile_stream != RS2_STREAM_DEPTH)
{
if (!color_stream_found)        //Prefer color
align_to = profile_stream;
        if (profile_stream == RS2_STREAM_COLOR)
        {
            color_stream_found = true;
        }
    }
    else
    {
        depth_stream_found = true;
    }
}

if(!depth_stream_found)
    throw std::runtime_error("No Depth stream available");

if (align_to == RS2_STREAM_ANY)
    throw std::runtime_error("No stream found to align with Depth");

return align_to;

}
bool profile_changed(const std::vectorrs2::stream_profile& current, const std::vectorrs2::stream_profile& prev)
{
for (auto&& sp : prev)
{
//If previous profile is in current (maybe just added another)
auto itr = std::find_if(std::begin(current), std::end(current), [&sp](const rs2::stream_profile& current_sp) { return sp.unique_id() == current_sp.unique_id(); });
if (itr == std::end(current)) //If it previous stream wasn't found in current
{
return true;
}
}
return false;
}
the modified CMakeLists.txt(librealsense/examples/align/CMakeLists.txt) as following:
minimum required cmake version: 3.1.0
cmake_minimum_required(VERSION 3.1.0)
project(RealsenseExamplesAlign)
Save the command line compile commands in the build output
set(CMAKE_EXPORT_COMPILE_COMMANDS 1)
find_package(OpenCV REQUIRED)
message(STATUS "OpenCV library status:")
message(STATUS "    version: ${OpenCV_VERSION}")
message(STATUS "    libraries: ${OpenCV_LIBS}")
message(STATUS "    include path: ${OpenCV_INCLUDE_DIRS}")
include_directories(${OpenCV_INCLUDE_DIRS})
include(CheckCXXCompilerFlag)
CHECK_CXX_COMPILER_FLAG("-std=c++11" COMPILER_SUPPORTS_CXX11)
CHECK_CXX_COMPILER_FLAG("-std=c++0x" COMPILER_SUPPORTS_CXX0X)
if(COMPILER_SUPPORTS_CXX11)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=c11")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
elseif(COMPILER_SUPPORTS_CXX0X)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++0x")
endif()
if(BUILD_GRAPHICAL_EXAMPLES)
#align
add_executable(rs-align rs-align.cpp ../../third-party/imgui/imgui.cpp ../../third-party/imgui/imgui_draw.cpp ../../third-party/imgui/imgui_impl_glfw.cpp)
#    target_link_libraries(rs-align ${DEPENDENCIES})
target_link_libraries(rs-align ${DEPENDENCIES} ${OpenCV_LIBS})
include_directories(rs-align ../../common ../../third-party/imgui)
set_target_properties (rs-align PROPERTIES
FOLDER Examples
)
install(
    TARGETS

    rs-align

    RUNTIME DESTINATION
    ${CMAKE_INSTALL_PREFIX}/bin
)

endif()
we'd like to hear from you later.

Thank you for your patience.
I have tried what you suggested, but the output window is still dark.

After a step-by-step debugging, the preliminary judgment is aligned_depth_frame = q.wait_for_frame() cannot assign depth_frame from the queue to aligned_depth_frame.
Have you encountered any similar problems here? Looking forward to your reply.

[Realsense Customer Engineering Team Comment]
Hi @Liuyu010,
we need to dig more into this issue. Thanks for your investigation and patience.

Hi @RealSense-Customer-Engineering
According to the measure.cpp:191-233, when I tend to pass in two parameters(color_frame and depth_frame) at the same time to pb() , there are still a lot of syntax errors. Can you provide a more intuitive way?
In addition, your company recently released the D435i camera, will the camera have a greater effect on the rs_align? Is there any test results?
Thank you.

Hi @dorodnic ，
Can you help me about transferring two parameters(color_frame and depth_frame) to pb()?
I have tried much times, but still trapped. This is the last question, sincerely hope to get your help.

[Realsense Customer Engineering Team Comment]
Hi @Liuyu010 ，
hope the following new rs-align.cpp meet your requirement:
// License: Apache 2.0. See LICENSE file in root directory.
// Copyright(c) 2017 Intel Corporation. All Rights Reserved.
#include <librealsense2/rs.hpp>
#include "../example.hpp"
#include <imgui.h>
#include "imgui_impl_glfw.h"
#include
#include
#include
#include
#include
#include <opencv2/opencv.hpp>  // Include OpenCV API
using namespace cv;
void render_slider(rect location, float& clipping_dist);
void remove_background(rs2::video_frame& other, const rs2::depth_frame& depth_frame, float depth_scale, float clipping_dist);
float get_depth_scale(rs2::device dev);
rs2_stream find_stream_to_align(const std::vectorrs2::stream_profile& streams);
bool profile_changed(const std::vectorrs2::stream_profile& current, const std::vectorrs2::stream_profile& prev);
int main(int argc, char * argv[]) try
{
// Create and initialize GUI related objects
window app(1280, 720, "CPP - Align Example"); // Simple window handling
ImGui_ImplGlfw_Init(app, false);      // ImGui library intializition
rs2::colorizer c;                          // Helper to colorize depth images
texture renderer;                    // Helper for renderig images
// Create a pipeline to easily configure and start the camera
rs2::pipeline pipe;
//Calling pipeline's start() without any additional parameters will start the first device
// with its default streams.
//The start function returns the pipeline profile which the pipeline used to start the device
rs2::pipeline_profile profile = pipe.start();

// Each depth camera might have different units for depth pixels, so we get it here
// Using the pipeline's profile, we can retrieve the device that the pipeline uses
float depth_scale = get_depth_scale(profile.get_device());

//Pipeline could choose a device that does not have a color stream
//If there is no color stream, choose to align depth to another stream
rs2_stream align_to = find_stream_to_align(profile.get_streams());

// Create a rs2::align object.
// rs2::align allows us to perform alignment of depth frames to others frames
//The "align_to" is the stream type to which we plan to align depth frames.
rs2::align align(align_to);

// Define a variable for controlling the distance to clip
float depth_clipping_distance = 1.f;

// Create custom processing block and its output queue:
rs2::frame_queue q;
rs2::processing_block pb(
[&](rs2::frameset data, rs2::frame_source& src)
{
    rs2::frame f = data.get_depth_frame();
    // For each input frame f, do:

    const int w = f.as<rs2::video_frame>().get_width();
    const int h = f.as<rs2::video_frame>().get_height();

    // frame --> cv
    Mat image(Size(w, h), CV_16UC1, (void*)f.get_data(), Mat::AUTO_STEP);
    // do some (silly) processing
    cv::GaussianBlur(image, image, cv::Size(0, 0), 3);

    // Allocate new frame. Copy all missing data from f.
    // This assumes the output is same resolution and format
    // if not true, need to specify parameters explicitly
    auto res = src.allocate_video_frame(f.get_profile(), f);

    // copy from cv --> frame
    memcpy((void*)res.get_data(), image.data, w * h * 2);

    std::cout << "enqueue" << std::endl;
    q.enqueue(f);

    // Send the resulting frame to the output queue
    src.frame_ready(res);
});
//pb.start(q); // Bind output of the processing block to be enqueued into the queue

//pb >> q;

while (app) // Application still alive?
{
    // Using the align object, we block the application until a frameset is available
    rs2::frameset frameset = pipe.wait_for_frames();

    // rs2::pipeline::wait_for_frames() can replace the device it uses in case of device error or disconnection.
    // Since rs2::align is aligning depth to some other stream, we need to make sure that the stream was not changed
    //  after the call to wait_for_frames();
    if (profile_changed(pipe.get_active_profile().get_streams(), profile.get_streams()))
    {
        //If the profile was changed, update the align object, and also get the new device's depth scale
        profile = pipe.get_active_profile();
        align_to = find_stream_to_align(profile.get_streams());
        align = rs2::align(align_to);
        depth_scale = get_depth_scale(profile.get_device());
    }

    //Get processed aligned frame
    auto processed = align.process(frameset);

    // Trying to get both other and aligned depth frames
    rs2::video_frame other_frame = processed.first(align_to);
    //rs2::depth_frame aligned_depth_frame = processed.get_depth_frame();
    //rs2::depth_frame aligned_depth_frame;// = processed.get_depth_frame();

    pb.invoke(processed);

    rs2::frame depth = q.wait_for_frame();
    rs2::depth_frame aligned_depth_frame = depth.as<rs2::video_frame>();

    //If one of them is unavailable, continue iteration
    if (!aligned_depth_frame || !other_frame)
    {
        continue;
    }
    // Passing both frames to remove_background so it will "strip" the background
    // NOTE: in this example, we alter the buffer of the other frame, instead of copying it and altering the copy
    //      This behavior is not recommended in real application since the other frame could be used elsewhere
    remove_background(other_frame, aligned_depth_frame, depth_scale, depth_clipping_distance);

    // Taking dimensions of the window for rendering purposes
    float w = static_cast<float>(app.width());
    float h = static_cast<float>(app.height());

    // At this point, "other_frame" is an altered frame, stripped form its background
    // Calculating the position to place the frame in the window
    rect altered_other_frame_rect{ 0, 0, w, h };
    altered_other_frame_rect = altered_other_frame_rect.adjust_ratio({ static_cast<float>(other_frame.get_width()),static_cast<float>(other_frame.get_height()) });

    // Render aligned image
    renderer.render(other_frame, altered_other_frame_rect);

    // The example also renders the depth frame, as a picture-in-picture
    // Calculating the position to place the depth frame in the window
    rect pip_stream{ 0, 0, w / 5, h / 5 };
    pip_stream = pip_stream.adjust_ratio({ static_cast<float>(aligned_depth_frame.get_width()),static_cast<float>(aligned_depth_frame.get_height()) });
    pip_stream.x = altered_other_frame_rect.x + altered_other_frame_rect.w - pip_stream.w - (std::max(w, h) / 25);
    pip_stream.y = altered_other_frame_rect.y + altered_other_frame_rect.h - pip_stream.h - (std::max(w, h) / 25);

    // Render depth (as picture in pipcture)
    renderer.upload(c.process(aligned_depth_frame));
    renderer.show(pip_stream);

    // Using ImGui library to provide a slide controller to select the depth clipping distance
    ImGui_ImplGlfw_NewFrame(1);
    render_slider({ 5.f, 0, w, h }, depth_clipping_distance);
    ImGui::Render();

}
return EXIT_SUCCESS;

}
catch (const rs2::error & e)
{
std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n    " << e.what() << std::endl;
return EXIT_FAILURE;
}
catch (const std::exception & e)
{
std::cerr << e.what() << std::endl;
return EXIT_FAILURE;
}
float get_depth_scale(rs2::device dev)
{
// Go over the device's sensors
for (rs2::sensor& sensor : dev.query_sensors())
{
// Check if the sensor if a depth sensor
if (rs2::depth_sensor dpt = sensor.asrs2::depth_sensor())
{
return dpt.get_depth_scale();
}
}
throw std::runtime_error("Device does not have a depth sensor");
}
void render_slider(rect location, float& clipping_dist)
{
// Some trickery to display the control nicely
static const int flags = ImGuiWindowFlags_NoCollapse
| ImGuiWindowFlags_NoScrollbar
| ImGuiWindowFlags_NoSavedSettings
| ImGuiWindowFlags_NoTitleBar
| ImGuiWindowFlags_NoResize
| ImGuiWindowFlags_NoMove;
const int pixels_to_buttom_of_stream_text = 25;
const float slider_window_width = 30;
ImGui::SetNextWindowPos({ location.x, location.y + pixels_to_buttom_of_stream_text });
ImGui::SetNextWindowSize({ slider_window_width + 20, location.h - (pixels_to_buttom_of_stream_text * 2) });

//Render the vertical slider
ImGui::Begin("slider", nullptr, flags);
ImGui::PushStyleColor(ImGuiCol_FrameBg, ImColor(215.f / 255, 215.0f / 255, 215.0f / 255));
ImGui::PushStyleColor(ImGuiCol_SliderGrab, ImColor(215.f / 255, 215.0f / 255, 215.0f / 255));
ImGui::PushStyleColor(ImGuiCol_SliderGrabActive, ImColor(215.f / 255, 215.0f / 255, 215.0f / 255));
auto slider_size = ImVec2(slider_window_width / 2, location.h - (pixels_to_buttom_of_stream_text * 2) - 20);
ImGui::VSliderFloat("", slider_size, &clipping_dist, 0.0f, 6.0f, "", 1.0f, true);
if (ImGui::IsItemHovered())
    ImGui::SetTooltip("Depth Clipping Distance: %.3f", clipping_dist);
ImGui::PopStyleColor(3);

//Display bars next to slider
float bars_dist = (slider_size.y / 6.0f);
for (int i = 0; i <= 6; i++)
{
    ImGui::SetCursorPos({ slider_size.x, i * bars_dist });
    std::string bar_text = "- " + std::to_string(6-i) + "m";
    ImGui::Text("%s", bar_text.c_str());
}
ImGui::End();

}
void remove_background(rs2::video_frame& other_frame, const rs2::depth_frame& depth_frame, float depth_scale, float clipping_dist)
{
const uint16_t* p_depth_frame = reinterpret_cast<const uint16_t*>(depth_frame.get_data());
uint8_t* p_other_frame = reinterpret_cast<uint8_t*>(const_cast<void*>(other_frame.get_data()));
int width = other_frame.get_width();
int height = other_frame.get_height();
int other_bpp = other_frame.get_bytes_per_pixel();

#pragma omp parallel for schedule(dynamic) //Using OpenMP to try to parallelise the loop
for (int y = 0; y < height; y++)
{
    auto depth_pixel_index = y * width;
    for (int x = 0; x < width; x++, ++depth_pixel_index)
    {
        // Get the depth value of the current pixel
        auto pixels_distance = depth_scale * p_depth_frame[depth_pixel_index];

        // Check if the depth value is invalid (<=0) or greater than the threashold
        if (pixels_distance <= 0.f || pixels_distance > clipping_dist)
        {
            // Calculate the offset in other frame's buffer to current pixel
            auto offset = depth_pixel_index * other_bpp;

            // Set pixel to "background" color (0x999999)
            std::memset(&p_other_frame[offset], 0x99, other_bpp);
        }
    }
}

}
rs2_stream find_stream_to_align(const std::vectorrs2::stream_profile& streams)
{
//Given a vector of streams, we try to find a depth stream and another stream to align depth with.
//We prioritize color streams to make the view look better.
//If color is not available, we take another stream that (other than depth)
rs2_stream align_to = RS2_STREAM_ANY;
bool depth_stream_found = false;
bool color_stream_found = false;
for (rs2::stream_profile sp : streams)
{
rs2_stream profile_stream = sp.stream_type();
if (profile_stream != RS2_STREAM_DEPTH)
{
if (!color_stream_found)        //Prefer color
align_to = profile_stream;
        if (profile_stream == RS2_STREAM_COLOR)
        {
            color_stream_found = true;
        }
    }
    else
    {
        depth_stream_found = true;
    }
}

if(!depth_stream_found)
    throw std::runtime_error("No Depth stream available");

if (align_to == RS2_STREAM_ANY)
    throw std::runtime_error("No stream found to align with Depth");

return align_to;

}
bool profile_changed(const std::vectorrs2::stream_profile& current, const std::vectorrs2::stream_profile& prev)
{
for (auto&& sp : prev)
{
//If previous profile is in current (maybe just added another)
auto itr = std::find_if(std::begin(current), std::end(current), [&sp](const rs2::stream_profile& current_sp) { return sp.unique_id() == current_sp.unique_id(); });
if (itr == std::end(current)) //If it previous stream wasn't found in current
{
return true;
}
}
return false;
}
and our expert @dorodnic had been pointed out that you could pass frameset(contain color_frame and depth_frame and ir_frame...) to the pb.

[Realsense Customer Engineering Team Comment]
Hi @Liuyu010,
we had been validated the latest modified rs-align.cpp, and it works find as expected.
we'd like to close this ticket if it could solved your question.
we'd like to hear from you again.
Thanks!
Best regards

[Realsense Customer Engineering Team Comment]
@Liuyu010 Did you get any chance to try what we suggested to get this through? Please update. Thanks!

[Realsense Customer Engineering Team Comment]
Hi @Liuyu010,
any progress with this issue? we'd like to get a positive response from you.
Thanks.

Sorry for the later response.
We have successfully processed RGB images and depth images simultaneously to get a better depth map in pb(). And when we tend to use Background_romoval by this processed depth map, the background's depth_value of the map seems to be very confusing（As shown below.）. So, we are going to give up this plan, Thank you for your continued reply.

[Realsense Customer Engineering Team Comment]
@Liuyu010 Sorry I'm not very clear about what you mentioned about "the map seems to be very confusing". Could you please explain more about it?

Ummm...it means that the depth map's precision does not support the background removal very well. We are looking for other possible solutions.
Thank you for your sincere help.

[Realsense Customer Engineering Team Comment]
@liuyu010 Thanks for the feedback! We have one such example in the link https://github.com/IntelRealSense/librealsense/tree/master/wrappers/opencv/grabcuts  You can have a try. Thanks!

[Realsense Customer Engineering Team Comment]
@liuyu010 Thanks for your using Realsense camera! Close this ticket. If you still have other questions, please feel free to open new ticket.


Can I change the dots animation when I draw dynamic scatter plot?

I use setInterval() function to generate dynamic data for scatter plot chart and I set animation of the chart to be 'true'. but I found a problem in the dots refreshing animation styles. Once I use data.push() and data.shift() function in refreshing data (which means the value of dots in the scatter plot chart changed), the default animation styles go in this way: all the dots in one x-axis value animates themselves (these dots goes up and down and then to the end y-axis position).
I am wondering if I can adjust the animation style, and let the dots moving like left and right moving, so as let the dots changing looks like shifting from right to left, called panning effect (timeseries river? i don't know how to describe it exactly)?
Any suggestions about it? Thanks a lot!

Hi,
The issue queue is to be used for reporting defects and problems with C3.js, in addition to feature requests and ideas.
It is not a catch-all support forum.
For general support enquiries, please use the Google Group at https://groups.google.com/forum/#!forum/c3js.
All questions involving the interplay between C3.js and any other library (such as AngularJS) should be posted there first!
Thank you!


Update recipe for v2.2.5

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

@conda-forge-admin, please rerender

@ktlim I waited a few days but the bot never discovered the new release on GitHub so I decided to make the PR manually.

According to the status page, the bot found the release but encountered an error.

The bot finally triggered and automerged in #36.


Added NFT factory, erc721 and erc20 tests

Fixes #541

Translated NFT tests from Contracts NFT tests to python
Added required/remaining contract interfaces functions
Included bash install.sh script to copy contract ABI and address.json to artifacts lib

Pending conftest changes for fixtures, I will also approve this. Thank you @maldo-ctrl!

The fixtures have been merged (#554 ). I will approve it as well after the changes mentioned by @calina-c. Well done @maldo-ctrl!


request: Support message timestamps

Describe the solution you'd like
Discord has added message timestamps would be cool if that gets supported.
Describe alternatives you've considered
Display a normal date string.

This is already a feature through the discord-time component. This library won't be parsing unix timestamps for you just like it also doesn't convert markdown to html and the likes.

Oo, didn't saw that one, thanks!


add option to hide exit code

Add option to always hide the exit code.
Set AM_HIDE_EXIT_CODE to any value to enable.

I would prefer setting AM_HIDE_EXIT_CODE to 1 to hide exit code, as it goes with the convention of setting other variables.

The latest commit should fix it.


[CODE IMPROVEMENT] Import dataset from Hugging Face

🔧 Proposed code refactoring
Allow simple import of dataset from Hugging Face
from datasets import load_dataset
import pandas as pd

Load your dataset
dataset = load_dataset('fka/awesome-chatgpt-prompts')  # replace with the name of your desired dataset

 Convert to pandas DataFrame
df = pd.DataFrame(dataset['train'])

 Save DataFrame to csv
df.to_csv('my_dataset.csv', index=False)

Motivation
Raised by Maher on Discord (https://discord.com/channels/1097462770674438174/1100718594809147402/1136414136495001680)

@pascal-pfeiffer I would like to work on this issue, can you please give me more details that what exact changes i have to make ?


Bluetooth: controller: split: Fix NRF_CCM overrun

Fix missing assignment of NRF_CCM->MAXPACKETSIZE register
for PDU sizes smaller than 251 bytes. If there is CRC errors
causing PDU length fields to be higher than configured PDU
buffer sizes in the controller, without the MAXPACKETSIZE
register set to correct PDU size, CCM module could overrun
the PDU buffer and cause memory corruption. This fix is
applicable for all nRF52 Series SoCs except nRF52832 SoC.
Fixes #21107.
Signed-off-by: Vinayak Kariappa Chettimada<EMAIL_ADDRESS>

@LeBlue FYI, please approve if the issue gets fixed. I assume you will need to run the tests for few days.

@aescolar @cvinayak this introduces an issue on the simulator, since the new register is not mapped

Does it exist for a 52832? (that's what the nrf52_bsim tries to emulate)

No, actually that's true. The nRF52832 doesn't have this register. This must be a missing conditional compilation in the LL. @cvinayak ?

@carlescufi @aescolar it was an oversight with regards to the use of the SOC_NRF52832, replaced with the correct SOC_COMPATIBLE_NRF52832

@cvinayak

@LeBlue FYI, please approve if the issue gets fixed. I assume you will need to run the tests for few days.

I have modified the test setup and moved the peripherals on the edge of the radio reception range. Now the issue is observable (without the patch) after 10-20 mins (instead of several hours).
I am running the first version of this patch with 3 setups: the hci_uart sample, hci_usb sample and a custom hci_usb firmware for almost 20h with no issues. Since this is tested on the nrf52840 there should be no difference to the second patch version. I am quite sure that the issue #21107 is fixed with this patch.
If there is another iteration I can test it, too.


Log response body on 400 range

Zendesk API returns the response status 400 range and the error messages as its body when the request was not successful.
https://developer.zendesk.com/rest_api/docs/core/introduction#400-range
But the response status is only output to the log now.
So, I would like to suggest to log the response body also to understand why the request was failure.

RVM jruby problem fixes at #355

@grosser would you check this and #355 please?

Test passed! 😃

thx!

1.16.0

Thank you for merging!


Ike v1 or v2

I had issues using route based vpn gateway and ikev2 with my ASA. Tunnel would drop, not come up, all sorts of odd things. Finally read somewhere MS doesn't support that setup for the ASA. Using policy based vpn gateway and ikev1 I have had no issues.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 400bf897-2cdd-39e8-85df-34ad53ca7838
Version Independent ID: 606caa65-a95f-07b1-98a3-202479e6ce5e
Content
Content Source
Service: vpn-gateway

@akalbfell thanks for the feedback, I assigned the case to the author for further review.

Hello Gentlemen,
IKEv2 does work between Azure and FTD but not out the box. A bit of tweaking has to take place. I have implemented 4 of these IKEv2 Tunnels in the Field with no issues. Actually, one client in particular, was trying to get away from IKEv1 since they stated they needed IKEv2 for some sort of AD integration they were attempting through the tunnel. If FTD/ASA is on the FMC already, it takes about 6-8 hours to complete tunnel, with full configuration. Otherwise, it can take longer.

Also, if you want IKEv2, it has to be route-based on the Azure side, since IKEv2 is not supported on Policy-Based Azure.

Thanks for the response. I should have mentioned I didn't make any changes to the parameters of the phase 1 settings when usuing Ikev2 route based. My issues were with default settings.

Folks,
The specific doc described how to use the custom IPsec/IKE policy on an Azure route-based VPN gateway to connect to Cisco ASA with access-list-based Traffic Selectors. It was called out in the beginning of the doc and also in the "Notes" section. Please let us know if there are specific issues regarding the descriptions or specifications. IKEv2 was called out in the beginning of the page.
We can take it under review if the ask here is about adding a doc for Cisco ASA using IKEv1 and Azure policy-based VPN gateway. Please let us know. Otherwise, we can close or resolve this issue.
Thanks,
Yushun [MSFT]

#please-close


make dev-deps problem

administrator@ubuntu:~/work/src/github.com/TheThingsNetwork/ttn$ make dev-deps
package bytes: unrecognized import path "bytes" (import path does not begin with hostname)
package flag: unrecognized import path "flag" (import path does not begin with hostname)
package fmt: unrecognized import path "fmt" (import path does not begin with hostname)
package bufio: unrecognized import path "bufio" (import path does not begin with hostname)
package errors: unrecognized import path "errors" (import path does not begin with hostname)
package io: unrecognized import path "io" (import path does not begin with hostname)
package os: unrecognized import path "os" (import path does not begin with hostname)
package strings: unrecognized import path "strings" (import path does not begin with hostname)
package syscall: unrecognized import path "syscall" (import path does not begin with hostname)
package unicode/utf8: unrecognized import path "unicode/utf8" (import path does not begin with hostname)
package unsafe: unrecognized import path "unsafe" (import path does not begin with hostname)
package strconv: unrecognized import path "strconv" (import path does not begin with hostname)
package encoding/json: unrecognized import path "encoding/json" (import path does not begin with hostname)
package crypto/sha1: unrecognized import path "crypto/sha1" (import path does not begin with hostname)
package encoding/base64: unrecognized import path "encoding/base64" (import path does not begin with hostname)
package crypto/rand: unrecognized import path "crypto/rand" (import path does not begin with hostname)
package encoding/base32: unrecognized import path "encoding/base32" (import path does not begin with hostname)
package path/filepath: unrecognized import path "path/filepath" (import path does not begin with hostname)
package runtime: unrecognized import path "runtime" (import path does not begin with hostname)
package log: unrecognized import path "log" (import path does not begin with hostname)
package time: unrecognized import path "time" (import path does not begin with hostname)
package sort: unrecognized import path "sort" (import path does not begin with hostname)
package path: unrecognized import path "path" (import path does not begin with hostname)
package encoding/xml: unrecognized import path "encoding/xml" (import path does not begin with hostname)
package os/exec: unrecognized import path "os/exec" (import path does not begin with hostname)
package sync: unrecognized import path "sync" (import path does not begin with hostname)
package io/ioutil: unrecognized import path "io/ioutil" (import path does not begin with hostname)
package go/ast: unrecognized import path "go/ast" (import path does not begin with hostname)
package go/parser: unrecognized import path "go/parser" (import path does not begin with hostname)
package go/printer: unrecognized import path "go/printer" (import path does not begin with hostname)
package go/token: unrecognized import path "go/token" (import path does not begin with hostname)
package net/http: unrecognized import path "net/http" (import path does not begin with hostname)
package net/url: unrecognized import path "net/url" (import path does not begin with hostname)
package regexp: unrecognized import path "regexp" (import path does not begin with hostname)
package hash: unrecognized import path "hash" (import path does not begin with hostname)
package math: unrecognized import path "math" (import path does not begin with hostname)
package unicode: unrecognized import path "unicode" (import path does not begin with hostname)
package text/tabwriter: unrecognized import path "text/tabwriter" (import path does not begin with hostname)
package text/template: unrecognized import path "text/template" (import path does not begin with hostname)
make: *** [build-deps] Error 1
go version 1.8

Hi there !
Can you provide more info ? Look like it come from the go installation.

problem solved by Add govendor to deps #521

Did you installed govendor ?
go get -u github.com/kardianos/govendor

i changed i line in makefile, it didn't know the whole path of the command govendor

Don't forget to set $GOBIN and append it to your $PATH variable.
https://golang.org/doc/code.html#GOPATH


python312Packages.dask: 2024.8.0 -> 2024.8.1

Description of changes

python312Packages.dask: 2024.8.0 -> 2024.8.1 (https://docs.dask.org/en/latest/changelog.html)
python312Packages.dask-expr: 1.1.10 -> 1.1.11

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.


Fix g++ warnings

Fix for g++ being stricter than clang, also does not like memset on structs.

Codecov Report

Merging #123 (8dc5c44) into develop (edb686a) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff            @@
##           develop     #123   +/-   ##
========================================
  Coverage    63.59%   63.59%
========================================
  Files           60       60
  Lines        18010    18011    +1
========================================
+ Hits         11453    11454    +1
  Misses        6557     6557

Impacted Files
Coverage Δ

core/include/expressions/expression.h
92.59% <ø> (ø)

core/src/c_api/tiledb_utils.cc
76.82% <100.00%> (-0.08%)
:arrow_down:

core/src/expressions/expression.cc
94.00% <100.00%> (+0.04%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update edb686a...8dc5c44. Read the comment docs.


Optimized Type Checker performance in UnicodeScalarExtensions and SyntaxText

Description
This PR optimizes swift code to accommodate slowness of Type Checker.
Context
The following two methods are currently slow to compile in swift-syntax package:

UnicodeScalarExtensions.isValidIdentifierContinuationCodePoint
SyntaxText.isSlice(of:)

The optimization is about making the code a bit less "sweet" (as from "syntax sugar"), but a bit faster to get parsed.
Approach is simple: add type annotations, introduce local variables annotated with types.
Measurements
I have measured the performance of the following command:
xcodebuild -scheme swift-syntax-Package ONLY_ACTIVE_ARCH=YES -destination 'platform=macos' clean build > /dev/null

using time. The output of time was the following:

Before
After

2.10s user 0.53s system 20% cpu 12.784 total
2.08s user 0.51s system 20% cpu 12.629 total

2.11s user 0.50s system 20% cpu 12.896 total
2.09s user 0.50s system 20% cpu 12.802 total

2.12s user 0.50s system 20% cpu 12.720 total
2.07s user 0.52s system 20% cpu 12.793 total

2.10s user 0.51s system 20% cpu 12.905 total
2.08s user 0.53s system 20% cpu 12.768 total

2.05s user 0.47s system 20% cpu 12.638 total
2.07s user 0.53s system 20% cpu 12.710 total

2.12s user 0.52s system 20% cpu 12.569 total
2.05s user 0.50s system 19% cpu 12.846 total

2.14s user 0.53s system 20% cpu 12.792 total
2.07s user 0.50s system 20% cpu 12.514 total

More Details
I have used https://github.com/qonto/SwiftCompilationTimingParser to measure slow compiling code. The following were the top candidates for optimization:

location
symbol
ms

Sources/SwiftSyntax/generated/SyntaxVisitor.swift:3438:16
instance method visitationFunc(for:)
407.71

Sources/SwiftSyntax/SyntaxText.swift:97:15
instance method isSlice(of:)
299.29

Sources/SwiftSyntax/generated/ChildNameForKeyPath.swift:18:13
global function childName(_:)
293.93

Sources/SwiftSyntax/generated/SyntaxRewriter.swift:2113:16
instance method visitationFunc(for:)
269.33

Sources/SwiftParser/Lexer/UnicodeScalarExtensions.swift:14:52
getter isValidIdentifierContinuationCodePoint
123.67

Sources/SwiftSyntax/generated/TokenKind.swift:730:22
static method fromRaw(kind:text:)
85.13

Sources/SwiftSyntax/generated/SyntaxEnum.swift:306:8
instance method as(_:)
74.07

Out of these, there were only two which are not related to generated code.
After code optimization, the modified methods got the following timing:

location
symbol
ms
improvement

UnicodeScalarExtensions.swift:14:52
getter isValidIdentifierContinuationCodePoint
29.23
76.4%

SyntaxText.swift:97:15
instance method isSlice(of:)
7.1
97.7%

While the comparison might seem unreasonable to get this PR merged, in my personal opinion and experience, under certain circumstances this type of optimization of swift code for Type Checker might play a big role in overall compilation performance in the long run. What's important is not to introduce a regression, and so thorough measurements should be made.

👋🏻  @ahoppen ,

Thanks for investigating swift-syntax’s compile time.
In this version of the PR code is a lot less readable, especially because of the non-descriptive variable names first, second etc. Especially in isSlice(of:), I think we should be able to use a lot more descriptive variable names, like selfEnd and otherEnd, which should improve the code readability and the compile time at the same time.

thank you for your review. I am aware of the "ugliness" of optimized Swift code when it comes to fixing Type Checker performance issues. One of the worst swift code I have ever written was when optimizing Combine closures with explicit return types in a deeply nested context. Though, it reduced compilation duration by 50% in that project.

For the isValidIdentifierContinuationCodePoint, I briefly looked and couldn’t find any descriptive variable names that we could pick. But I think it would be interesting to see if adding as Bool could improve the compile time, while leaving the overall shape of the code the same.

I'll do my best and re-run tests with as Bool suggestion. I'd also consider adding meaning to the variables. Maybe LLM would be able to come up with such, if no other option would work 😅

Hey @ahoppen ,

For the isValidIdentifierContinuationCodePoint, I briefly looked and couldn’t find any descriptive variable names that we could pick. But I think it would be interesting to see if adding as Bool could improve the compile time, while leaving the overall shape of the code the same.

Indeed, I have applied as Bool and it works even faster, just 20.72 ms! Pushed my commits.

Thanks for the quick iteration time @art-divin.
Could you format the code using swift-format as described in https://github.com/apple/swift-syntax/blob/main/CONTRIBUTING.md#formatting and squash your commits?

Done ✅

For visibility, I am trying to optimize code in generated section of swift-syntax as well, by altering the way how code-generator works and adjusting templates.
My gut feeling is this: 1-2 seconds can be removed (out of 12), but I did not find yet way how to achieve this exactly ⏳

@swift-ci Please test

@swift-ci Please test Windows


Coder sets class comment to nil when changing class package or tag

Here changing the tag of the class sets its comment to nil.

Same when changing the package:

Fixed by patching RBCommentChange>>primitiveExecute in https://github.com/feenkcom/gtoolkit-maestro-rs/commit/9577b11f56abe1ad5abf39abfed63bb9d64c370e

We can remove our patch when https://github.com/pharo-project/pharo/issues/12105 is fixed.


Review the highlighting for runtime errors

As mentioned in #144, the following code is incorrectly highlighted (answer to where()) with Python 3.6 and 3.7:
nb = len(letter
        for letter in "word")

stack_data does proper highlighting of this code. It incorporates code I wrote (https://github.com/alexmojaki/stack_data/pull/20) which had been inspired by my own improvements - which obviously do not work correctly in this case.
I should review the highlighting code, most likely in a different branch created for that purpose - something I usually do not bother to do.

This has been taken care of by contributing to stack_data and using the new stack_data code instead of my own.


error comando /quests

What is your Quests version / build number (do not say "latest")?
5.0.3
Describe the problem and how to replicate it.
Español
He tenido un problema desde hace tiempo en el que el comando /quests no funciona. Intenté replicar todos mis complementos en otro servidor y todo funcionaba perfectamente. Sin embargo, en mi servidor, lo único que no funcionaba era /quests, mientras que /quest sí funcionaba.
Al poner los mismos complementos en otro servidor, el comando /quests funcionaba, así que no es una incompatibilidad.
Inglés
I have had an issue for some time where the /quests command does not work. I tried replicating all my plugins on another server and everything worked perfectly. However, on my server, the only thing that didn't work was /quests, while /quest did work.
When I put the same plugins on another server, the /quests command worked, so it's not an incompatibility.
Any console errors to share?
Not even a single one

If applicable, can you provide an example quest from quests.yml?
No response

I would need to see your startup log in order to try diagnosing this.
Unless your permissions are the issue, a plugin incompatibility is still most likely.

no the permissions are not the problem I have op
There are no cyst errors in the console although I will send you the logs

latest.log

Try updating WolfyUtilities; I see pretty significant errors which may affect commands in a roundabout way.
I also see you're using a dev build of MMOItems. I wouldn't recommend downgrading, but also try updating to a newer build from https://phoenixdevt.fr/devbuilds

actualice WolfyUtilities no funciono también lo elimine y tampoco

update WolfyUtilities doesn't work and /quests still doesn't work, also delete it and it doesn't work either

@superkaito Please let me know if the latest development build still has this issue:
https://ci.codemc.io/job/PikaMug/job/Quests/

I have found the cause of the problem. It was because the command was blocked by a permission in the default group. When I switched groups, everything was resolved.

[10:22:54 WARN]: me.pikamug.quests.exceptions.StageFormatException: Stage cannot be empty, see quest d stage 1
[10:22:54 WARN]:        at Quests-5.1.0.jar//me.pikamug.quests.storage.implementation.file.BukkitQuestYamlStorage.loadQuestStages(BukkitQuestYamlStorage.java:842)
[10:22:54 WARN]:        at Quests-5.1.0.jar//me.pikamug.quests.storage.implementation.file.BukkitQuestYamlStorage.init(BukkitQuestYamlStorage.java:110)
[10:22:54 WARN]:        at Quests-5.1.0.jar//me.pikamug.quests.BukkitQuestsPlugin.lambda$reload$4(BukkitQuestsPlugin.java:717)
[10:22:54 WARN]:        at org.bukkit.craftbukkit.scheduler.CraftTask.run(CraftTask.java:101)
[10:22:54 WARN]:        at org.bukkit.craftbukkit.scheduler.CraftAsyncTask.run(CraftAsyncTask.java:57)
[10:22:54 WARN]:        at com.destroystokyo.paper.ServerSchedulerReportingWrapper.run(ServerSchedulerReportingWrapper.java:22)
[10:22:54 WARN]:        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
[10:22:54 WARN]:        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
[10:22:54 WARN]:        at java.base/java.lang.Thread.run(Thread.java:1583)
[10:22:54 WARN]: [Quests] Unable to consider custom objectives because quest for 000001 was null
Although now I have another problem. When I create a mission, it gets canceled, and an error appears in the console.

I have found the cause of the problem. It was because the command was blocked by a permission in the default group. When I switched groups, everything was resolved.

Glad to hear it!
It looks like your quest 'd' has no stages, but please create a new issue if you need assistance with that.


Automated additions to output directory paths

Currently comparison wrappers like grid_stat, mode, mtd, etc. add additional information to the *_OUTPUT_DIR to place the output files. For example:
GRID_STAT_OUTPUT_DIR = /some/dir/path
grid_stat wrapper puts the output files in
/some/dir/path/201706010000/grid_stat/grid_stat_000000L_20170601_000000V.stat
This keeps output files organized. METplus adds the extra subdirs for each run. Do we want this to be automated, or do we want the user to specify these additional items if they want? The user could set:
GRID_STAT_OUTPUT_DIR = /some/dir/path/{CURRENT_VALID_TIME}/grid_stat
or
GRID_STAT_OUTPUT_DIR = /some/dir/path/{CURRENT_INIT_TIME}/grid_stat
and it would have the same behavior. The automated subdirectory functionality was added because the use cases used to develop the wrappers had this format.

These changes apply to GridStat, EnsembleStat, Mode, MTD, and PointStat output.
I added an output template config item for each of the above wrappers, i.e.
[filename_templates]
GRID_STAT_OUTPUT_TEMPLATE
If it is not set, METplus will add the previous init or valid YYYYMMDDHHMM/grid_stat to the output path. If it is set to empty, it will not append anything. Users can set this template to any valid filename template and it will add that information.


Fix issue torchmetrics F1 -> F1Score

From issue #47
From the commit in the 9th of January in pytorch lightning: https://github.com/PyTorchLightning/metrics/commit/c20860ae8924776281b6d36a781dcf458d62438d
I suggest to put the requirements to torchmetrics >= 0.8.1 :)
Would it make sense to you ?

From issue #47
From the commit in the 9th of January in pytorch lightning: PyTorchLightning/metrics@c20860a
I suggest to put the requirements to torchmetrics >= 0.8.1 :)
Would it make sense to you ?

Sure, makes sense to me!

@popcornell Do you agree to merge this ?


Game crashes seconds after world creation

Versions
Minecraft version: 1.20.1
Forge version: 47.3.0
GlitchCore version: <IP_ADDRESS>
Serene Seasons version: <IP_ADDRESS>
ImmersiveSnow version: 1.3.0
Steps to reproduce

Create a instance with only the mods above
Create a new world
Crash should occur in a few seconds

Log
full log: https://mclo.gs/vWElY79
Exception
[22:31:12] [Server thread/ERROR] [ne.mi.ev.EventBus/EVENTBUS]: Exception caught during firing event: generateSnowAndIce
	Index: 2
	Listeners:
		0: NORMAL
		1: ASM: class glitchcore.forge.handlers.TickEventHandler onLevelTick(Lnet/minecraftforge/event/TickEvent$LevelTickEvent;)V
		2: ASM: class straywave.minecraft.immersivesnow.forge.EventListener onWorldTick(Lnet/minecraftforge/event/TickEvent$LevelTickEvent;)V
java.lang.NoSuchFieldError: generateSnowAndIce
	at TRANSFORMER/immersivesnow@1.3.0/straywave.minecraft.immersivesnow.forge.hook.SereneSeasonsHook.shouldMelt(SereneSeasonsHook.java:28)
	at TRANSFORMER/immersivesnow@1.3.0/straywave.minecraft.immersivesnow.forge.ModHooksImpl.shouldMelt(ModHooksImpl.java:26)
	at TRANSFORMER/immersivesnow@1.3.0/straywave.minecraft.immersivesnow.ModHooks.shouldMelt(ModHooks.java)
	at TRANSFORMER/immersivesnow@1.3.0/straywave.minecraft.immersivesnow.Logic.shouldMelt(Logic.java:63)
	at TRANSFORMER/immersivesnow@1.3.0/straywave.minecraft.immersivesnow.Logic.checkAndUpdateBlock(Logic.java:52)
	at TRANSFORMER/immersivesnow@1.3.0/straywave.minecraft.immersivesnow.ImmersiveSnowEvents.onWorldTick(ImmersiveSnowEvents.java:51)
	at TRANSFORMER/immersivesnow@1.3.0/straywave.minecraft.immersivesnow.forge.EventListener.onWorldTick(EventListener.java:44)
	at TRANSFORMER/immersivesnow@1.3.0/straywave.minecraft.immersivesnow.forge.__EventListener_onWorldTick_LevelTickEvent.invoke(.dynamic)
	at MC-BOOTSTRAP/net.minecraftforge.eventbus/net.minecraftforge.eventbus.ASMEventHandler.invoke(ASMEventHandler.java:73)
	at MC-BOOTSTRAP/net.minecraftforge.eventbus/net.minecraftforge.eventbus.EventBus.post(EventBus.java:315)
	at MC-BOOTSTRAP/net.minecraftforge.eventbus/net.minecraftforge.eventbus.EventBus.post(EventBus.java:296)
	at TRANSFORMER/forge@47.3.0/net.minecraftforge.event.ForgeEventFactory.onPostLevelTick(ForgeEventFactory.java:930)
	at TRANSFORMER/minecraft@1.20.1/net.minecraft.server.MinecraftServer.m_5703_(MinecraftServer.java:899)
	at TRANSFORMER/minecraft@1.20.1/net.minecraft.server.MinecraftServer.m_5705_(MinecraftServer.java:814)
	at TRANSFORMER/minecraft@1.20.1/net.minecraft.client.server.IntegratedServer.m_5705_(IntegratedServer.java:89)
	at TRANSFORMER/minecraft@1.20.1/net.minecraft.server.MinecraftServer.m_130011_(MinecraftServer.java:661)
	at TRANSFORMER/minecraft@1.20.1/net.minecraft.server.MinecraftServer.m_206580_(MinecraftServer.java:251)
	at java.base/java.lang.Thread.run(Thread.java:840)

I'm also getting a crash on start.

Looks like Serene Seasons changed their config structure. It's fixed for newer Minecraft versions, alternatively you can downgrade Serene Seasons to the last version before the big overhaul. (Unless there is some big new feature/fix in Serene Seasons that was released, I don't see any reason to backport?)


dev/0.9 merge main

Please check if the PR fulfills these requirements

[x] The commit message follows our guidelines
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

Codecov Report

Merging #1023 (9bdf7a6) into dev/0.9 (71eb8cb) will decrease coverage by 0.06%.
The diff coverage is 26.48%.

@@             Coverage Diff             @@
##           dev/0.9    #1023      +/-   ##
===========================================
- Coverage    29.60%   29.53%   -0.07%
===========================================
  Files          329      331       +2
  Lines        14650    14710      +60
  Branches      2235     2252      +17
===========================================
+ Hits          4337     4345       +8
- Misses        9801     9855      +54
+ Partials       512      510       -2

Impacted Files
Coverage Δ

packages/core/src/2d/atlas/FontAtlas.ts
2.12% <0.00%> (-0.26%)
:arrow_down:

packages/core/src/2d/text/Font.ts
17.64% <0.00%> (ø)

packages/core/src/2d/text/TextUtils.ts
0.60% <ø> (ø)

packages/core/src/RenderPipeline/Basic2DBatcher.ts
20.00% <0.00%> (-1.12%)
:arrow_down:

...ges/core/src/RenderPipeline/BasicRenderPipeline.ts
11.48% <ø> (ø)

...kages/core/src/RenderPipeline/MeshRenderElement.ts
0.00% <0.00%> (ø)

packages/core/src/RenderPipeline/RenderElement.ts
100.00% <ø> (+80.00%)
:arrow_up:

packages/core/src/RenderPipeline/RenderQueue.ts
2.08% <0.00%> (ø)

packages/core/src/RenderPipeline/SpriteBatcher.ts
6.66% <0.00%> (-0.28%)
:arrow_down:

...kages/core/src/RenderPipeline/SpriteMaskManager.ts
8.10% <0.00%> (ø)

... and 22 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Stops performing jobs after few days

Hi,
Delayed_job stops performing background jobs after 3-4 days even if the system process looks normal.
Analysis:

system process looks running (normal)
ps ax ==> 14 ?        Rl   33255:19 delayed_job
review jobs (are not performed)
Delayed::Job.count ==> 1
Delayed::Job.count ==> 1 (still not processed after 1 hour)
To have working again, I need to kill the pod (restart system)

System
Hosting: Kubernetes
OS: Debian GNU/Linux 9
Ruby: 2.6
Rails: 6.0.2

Hi, Delayed_job stops performing background jobs after 3-4 days even if the system process looks normal.
Analysis:

system process looks running (normal)
ps ax ==> 14 ?        Rl   33255:19 delayed_job
review jobs (are not performed)
Delayed::Job.count ==> 1
Delayed::Job.count ==> 1 (still not processed after 1 hour)
To have working again, I need to kill the pod (restart system)

System
Hosting: Kubernetes OS: Debian GNU/Linux 9 Ruby: 2.6 Rails: 6.0.2

I'm experiencing totally the same situation MacOS. Ruby 3.0.2. On 2.7.5 everything works well. I figured out that it's related to daemons gem.

Hi, Delayed_job stops performing background jobs after 3-4 days even if the system process looks normal.
Analysis:

system process looks running (normal)
ps ax ==> 14 ?        Rl   33255:19 delayed_job
review jobs (are not performed)
Delayed::Job.count ==> 1
Delayed::Job.count ==> 1 (still not processed after 1 hour)
To have working again, I need to kill the pod (restart system)

System
Hosting: Kubernetes OS: Debian GNU/Linux 9 Ruby: 2.6 Rails: 6.0.2

I'm experiencing totally the same situation MacOS. Ruby 3.0.2. On 2.7.5 everything works well. I figured out that it's related to daemons gem.

Closing stale


CSRF protection

Refs #793

Needs unit tests.
More importantly: needs very, very careful consideration of how this plays with HTTP caching.

I don't want to set a cookie on a page response that is being cached.
Right now the ASGI middleware will be doing exactly that, which is bad.
But how do I get nature that when you load a page with a form that will be CSRF protected you have been served the cookie?
Maybe those pages should do something explicit to the request object indicating that the cookie is needed?
That works for Datasette (since it has mutable request objects) but I'm not sure how it would work in the asgi-csrf pure ASGI middleware context.

Django docs on CSRF and caching: https://docs.djangoproject.com/en/3.0/ref/csrf/#caching

I'm solving the compatibility with caching problem in this ticket: https://github.com/simonw/asgi-csrf/issues/7

Add unit tests illustrating the Vary: Cookie header and I'm done here.


WIP: Redo package tests in scala

In particular, remember the motivation is to help diagnose intermittent failures in parameterInheritance.
PPG 356

Deleted CLIPackage and remove redundant clauses from CLIBasic that are in WskBasic.  There are a few remaining tests that I am not sure are covered so I left them in place.
PPG 358.

Ready to go.  Either merge as is or confirm that the remaining tests in CLIBasic are also redundant and delete the suite.

Removed reaming tests which were covered by the corresponding suite in Scala and that left only one tests which belonged in CLIActionTests. Merged as this commit.


OpenBlas Multi-Threading on Windows

New to MxNet. I've set up and tested it on a 12-core windows machine, tried both the pre-build CPU-only package and building the package myself using CMake.
I tried the sample project under example/notebooks/cifar10-recipe.ipynb. I changed the python code to CPU-only mode, and modified the preprocess_threads and MXNET_CPU_WORKER_NTHREADS to 12 since I have 12 cores, but the training speed ( [12] in cifar10-recipe.ipynb) is still really slow, and according to the hardware monitor it only loads one core to 100%, other cores are still in idle.
INFO:root:Iter[0] Batch [50] Speed: 1.83 samples/sec
INFO:root:Iter[0] Batch [100] Speed: 1.77 samples/sec
INFO:root:Iter[0] Batch [150] Speed: 1.77 samples/sec
INFO:root:Iter[0] Batch [200] Speed: 1.78 samples/sec
INFO:root:Iter[0] Batch [250] Speed: 1.78 samples/sec
INFO:root:Iter[0] Batch [300] Speed: 1.78 samples/sec
INFO:root:Iter[0] Batch [350] Speed: 1.78 samples/sec

Are there any other settings related to multi-core utilization? Or is GPU based system the only way to go for large data set?
Thanks

If compiled with OpenBLAS, the blas kernel should be able to enable parallelization and it is not related to MXNET_CPU_WORKER_NTHREADS . Though I do not know the case for window's OpenBLAS

loop in @hjk41

Try to set OPENBLAS_NUM_THREADS
This should enable OpenBLAS to use multiple threads to do computation

@hjk41
I tried to set OPENBLAS_NUM_THREADS to 12, still didn't change anything.
In the OpenBLAS GitHub page I saw this in the documentation.
"If you compile this lib with USE_OPENMP=1, you should set OMP_NUM_THREADS environment variable. OpenBLAS ignores OPENBLAS_NUM_THREADS and GOTO_NUM_THREADS with USE_OPENMP=1."
Yes, I'm using OpenMP right now, so I changed OMP_NUM_THREADS to 12. But still, only one core is running at 100%.

I think this is a openblas issue. I wasn't able to get openblas running on multiple cores with octave either

Let me check with OpenBLAS
On Wed, Dec 16, 2015 at 11:27 AM, Eric Junyuan Xie<EMAIL_ADDRESS>
wrote:

I think this is a openblas issue. I wasn't able to get openblas running on
multiple cores with octave either
—
Reply to this email directly or view it on GitHub
https://github.com/dmlc/mxnet/issues/937#issuecomment-164978199.

--
HONG Chuntao
System Research Group
Microsoft Research Asia

Checked with OpenBLAS author, Windows binary distribution DOES support multi-threading. I also tried on my own machine, setting OPENBLAS_NUM_THREADS to 16 boosts the CPU utilization to 100% on my 16-core machine.
I think it depends on how the cifar example uses the library. If it does not use matrix multiplication much, you will not see much speedup when using multiple threads. I think for Cifar, most of the time should be spent on the convolution layers? Then it comes to how convolution is implemented on CPU. Can @antinucleon shed light on this?

mshadow now has OpenMP support enabled by default. I am closing this issue for now. If you still have the problem, please let us know.


Relax 'sub' claim requirement

Thank you for this well designed and rational library. My IdP setup, PingID + Azure AD,  issues access_tokens with the 'sub' claim. For example, I receive:
{
  "scope": [
    "openid",
    "email",
    "profile",
    "offline_access"
  ],
  "client_id": "my-client-id",
  "iss": "https://my-idp.domain.com",
  "aud": "my-idp.domain.com",
  "uid": "myupn",
  "mail"<EMAIL_ADDRESS>  "givenName": "Matt",
  "sn": "Mshell",
  "exp":<PHONE_NUMBER>
}

This results in the following failure to auth:
# jwt_validatory.py L113
if 'sub' not in payload:
    self.logger.debug("No subject in body, invalid token")
    return {"active": False}

The specs on this claim are at odds with one another.
RFC 7519

The “sub” (subject) claim identifies the principal that is the subject of the JWT. The claims in a JWT are normally statements about the subject. The subject value MUST either be scoped to be locally unique in the context of the issuer or be globally unique. The processing of this claim is generally application specific. The “sub” value is a case-sensitive string containing a StringOrURI value. Use of this claim is OPTIONAL.

OpenID Connect Core 1.0

REQUIRED. Subject Identifier. A locally unique and never reassigned identifier within the Issuer for the End-User, which is intended to be consumed by the Client, e.g., 24400320or AItOawmwtWwcT0k51BayewNvutrJUqsvl6qs7A4. It MUST NOT exceed 255 ASCII characters in length. The sub value is a case sensitive string.

Can this check be relaxed, or possibly the check made conditional via configuration?

Hi Matt, thanks for the comment.
It's true that we could relax the requirement and make the verification configurable. I will try to add such a feature shortly.
The two specifications that you quote are actually not at odds. The OIDC core spec defines the ID token, and there the sub claim is required. In other JWTs, as defined in RFC 7519, sub is not a required claim (e.g. JWT access tokens do not have to use that claim).


About calibration

Hello,
Thanks for your great work!
We have our own datasets. I wonder if the alignment between the lidar and the camera is not very good, for example, the intrinsics or extrinsics we calibrated are not very accurate, will it affect the performance of the BEVFusion network?
Thanks!

Hi @starnstar @VeeranjaneyuluToka,
Sorry for the delayed response. I would say yes, definitely calibration parameters could have some impact on the performance, but it is not that large. Previously I implemented a version of bevfusion which calculated the wrong cam2lidar and lidar2cam matrices, but it turns out that the final performance for the fusion model is only 0.1-0.2 mAP worse (than the correct implementation). So I guess the model is somehow robust to small misalignments in the calibration matrices.
Best,
Haotian

Hℹ @kentang-mit,
Thanks for your reply!

Hi,
I am trying to test the robustness against calibration. I applied 10° and 0.5m meters of de-calibration to cam2lidar and lidar2cam.  But I do not see any major change in the results.
I am also not able to figure out where the cam2lidar and lidar2cam transformations are applied. I added de-calibration while loading the nuscenes data set. Also which is the common coordinate frame used?
Thanks!


inkscape: 1.0.2 -> 1.1

Motivation for this change
Upstream update.
https://inkscape.org/news/2021/05/24/welcome-inkscape-11/
https://media.inkscape.org/media/doc/release_notes/1.1/Inkscape_1.1.html
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Added a release notes entry if the change is major or breaking
[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 124333 at a327d40b run on x86_64-linux 1

3 packages marked as broken and skipped:

gnome.gnome-documents
spring
springLobby

47 packages skipped due to time constraints:

adapta-gtk-theme
arc-theme
arx-libertatis
asciidoc-full
asciidoc-full-with-plugins
bibata-cursors
bibata-cursors-translucent
bibata-extra-cursors
btrbk
capitaine-cursors
...

1 package built successfully:

inkscape

1 suggestion:

warning: unused-argument
Unused argument: fetchpatch.
Near pkgs/applications/graphics/inkscape/default.nix:8:3:
  |
8 | , fetchpatch
  |   ^

Exporting as JPEG fails with:
Traceback (most recent call last):
  File "raster_output_jpg.py", line 23, in <module>
    JpegOutput().run()
  File "/nix/store/gqxcnl2k0v4bm0f0pbyr8a6nx2zv44bf-inkscape-1.1/share/inkscape/extensions/inkex/base.py", line 130, in run
    self.load_raw()
  File "/nix/store/gqxcnl2k0v4bm0f0pbyr8a6nx2zv44bf-inkscape-1.1/share/inkscape/extensions/inkex/base.py", line 143, in load_raw
    document = self.load(self.file_io)
  File "/nix/store/gqxcnl2k0v4bm0f0pbyr8a6nx2zv44bf-inkscape-1.1/share/inkscape/extensions/inkex/extensions.py", line 73, in load
    from PIL import Image
ModuleNotFoundError: No module named 'PIL'

Also lib2geom package can finally be used (needs to be updated first):
https://media.inkscape.org/media/doc/release_notes/1.1/Inkscape_1.1.html#Important_changes_for_packagers
Feel free to pick it from https://github.com/NixOS/nixpkgs/pull/50286

@jtojnar: Thanks for all the tips!
I removed gdl, added pillow for jpg export, and changed from gtkspell to gspell.
I tried to test spell check, but I didn't get the right dictionary installed yet, so it's not been verified.
I also pulled in your lib2geom patch, and updated it to the version used in inkscape 1.1. I tested that lib2geom built after this update. Unfortunately, when I tried to add it as a build dep for inkscape, I start seeing undefined references for double_conversion. Maybe you have a suggestion for this?

I guess it has something to do with the fact that it is built as static library, needs "-D2GEOM_BUILD_SHARED=ON"
.

Also something like this:
--- a/pkgs/applications/graphics/inkscape/default.nix
+++ b/pkgs/applications/graphics/inkscape/default.nix
@@ -3,7 +3,6 @@
 , boost
 , cairo
 , cmake
-, double-conversion
 , fetchurl
 , gettext
 , ghostscript
@@ -21,6 +20,7 @@
 , libexif
 , libpng
 , librevenge
+, lib2geom
 , librsvg
 , libsigcxx
 , libsoup
@@ -83,6 +83,10 @@ stdenv.mkDerivation rec {
       --replace "call('ps2pdf'" "call('${ghostscript}/bin/ps2pdf'"
     patchShebangs share/templates
     patchShebangs man/fix-roff-punct
+
+    # double-conversion is a dependency of 2geom
+    substituteInPlace CMakeScripts/DefineDependsandFlags.cmake \
+      --replace 'find_package(DoubleConversion REQUIRED)' ""
   '';

   nativeBuildInputs = [
@@ -101,7 +105,6 @@ stdenv.mkDerivation rec {
   buildInputs = [
     boehmgc
     boost
-    double-conversion
     gettext
     glib
     glib-networking
@@ -114,6 +117,7 @@ stdenv.mkDerivation rec {
     libexif
     libpng
     librevenge
+    lib2geom
     librsvg # for loading icons
     libsigcxx
     libsoup

@jtojnar: Yep, your suggested changes allow the lib2geom link to work. Thanks!
Hopefully I addressed all the other suggestions you made too.
I did have a slight concern about the lib2geom version, as I mentioned here: https://gitlab.com/inkscape/lib2geom/-/merge_requests/48#note_584835008
Nevertheless, I upgraded lib2geom to the 1.1 tagged version, even though it doesn't 100% match what the inkscape 1.1 used.

Great job. Thanks.

@jtojnar: Thanks for all the help with my first nixpkgs change. :)


spawn EACCES: Error: spawn EACCES

I tried your library in ubuntu
what could be the error :
var openurl = require('opn');
openurl("http://url.com");

It gives error as :
spawn EACCES: Error: spawn EACCES
    at exports._errnoException (util.js:1022:11)
    at ChildProcess.spawn (internal/child_process.js:313:11)
    at Object.exports.spawn (child_process.js:399:9)
    at module.exports (/home/lekhnath/.vscode/extensions/Shan.code-settings-sync-2.8.0/node_modules/opn/index.js:74:26)
    at Commons.<anonymous> (/home/lekhnath/.vscode/extensions/Shan.code-settings-sync-2.8.0/out/src/commons.js:189:25)
    at Generator.next (<anonymous>)
    at fulfilled (/home/lekhnath/.vscode/extensions/Shan.code-settings-sync-2.8.0/out/src/commons.js:4:58)
e.onUnexpectedError @ workbench.main.js:29
(anonymous) @ workbench.main.js:29
e.onUnexpectedError @ workbench.main.js:28
o @ workbench.main.js:28
t.onUnexpectedExtHostError @ workbench.main.js:28
e.handle @ workbench.main.js:28
s @ workbench.main.js:28
f @ workbench.main.js:28
_combinedTickCallback @ internal/process/next_tick.js:67
_tickCallback @ internal/process/next_tick.js:98

Does setting chmod +x /home/lekhnath/.vscode/extensions/Shan.code-settings-sync-2.8.0/node_modules/opn/xdg-open work? Might be related to https://github.com/sindresorhus/opn/issues/38.

Yes it works using setting that command. Why it need to set it, library should do it.

This module already does that. If you try installing opn standalone and use, it'll work. This is an issue with the Visual Code or the extension.


Hi 🥳

Just read the README.md with the Noticeable section.
Do not merge this. I had just a bit of time and was maybe missing work a bit. 😂
// With the attached Add func to the go struct it is now possible to call Add directly.

list := List{}
list.Add("Item");

Haha thank you for the help 😂
I was not aware of that notation, so thank you. After seeing the PR, I found it later on YT and some article as well, thanks.
I would actually like to merge it, I just have changed the README for it and removed the commented code.
Any objections?

🙌🏼 Sure you can merge it, no objections at all.
 Just attaching logic to your structs without really interfering with the struct itself. The struct stays quite lean 👍🏼

Agreed, also like it, keeps the data description inside the struct and not pollutes it that way. 👍


[Feature request] show current rank instead of highest ever reached

Would it be possible to show the current rank instead of the highest rank ever reached?

Hey,
Thanks for reaching out! The plugin always gets the current MMR values from bakkesmod's GetPlayerMMR function. This means that as long as bakkesmod updates the MMRs the plugin should update accordingly and display the current rank.
As I remember, something that can cause bakkesmod not to update the MMR is if you leave a match before it ends, eg. during the instant replay of the last goal. This is resolved once you play a match and wait for it to end and the winner be announced.
If you still encounter the issue please contact me with some more details.

Thanks for the reply! It was my mistake: it correctly shows the current rank when i change the selected playlist pressing F8. Strangely though it seems to not save my preferences when choosing a playlist from the plugin settings, i.e. if I choose Standard in the dropdown and i close and reopen the plugin settings, it will be back to Best

Do you have Auto Playlist enabled? It automatically sets the playlist to the one you're playing when you join a match, which in case of casual is Best.


Have options import at the top of proto files

We need to put the below line before other declarations in proto files.
import "spine/options.proto";

This way it would be clearer that (type_url_prefix) is imported from this options.proto file and users always have to import it.

It's done.


Update to 1.20.2 Please

I really wanna try this mod:)

sure, may be 2 hours later

actually, I don't have any PC with me. I will use gitpod to build a temp version for you

sorry about that I cant work without pc 🥸

Sounds good, take your time please 😊

Thanks already @RTAkland.

@sternschnaube Already updated to 1.20.2, go to release to download the latest version


Add AuthZ sample with some functional tests

Basically coverage of https://github.com/blowdart/AspNetAuthorizationWorkshop

Rolled into https://github.com/aspnet/AuthSamples/issues/2