Add a new joystick/gamepad to an interbotix arm PX100

Hi there!
I've tried to connect an Thrustmater T-wireless gamepad that it's compatible with PS3 & PS2. I've tried in MOveit, and I've realized
that some file had to be modified. After that, I've seen that in MOveit the joystick is used just for goals, not realtime movement...
So I've remember that in the interbotix github was a joystick control that never worked for my gamepad.
And I was in love when I've seen the package interbotix_joy_control ... again :)))
I've modified that cool file :+1: joy_control_node.cpp by adding a new class thrustM and setting it up.
In order to make it work, I've taken a part of code from PS3 & Xbox.
After some setting up the launch file with proper argument finally the robot was controlled in real-time. A very good point for Interbotix :)
I have some issues though:

Even if I change the designation of a command to another button, every time the command is issued from the first button.

Example, {"GRIPPER_PWM_DEC", 8}, so button '8' has to command the speed of the gripper, but button '0' does that.

The mappings aren't quite correct, example {"WAIST_CCW", 4}, in reality the command is set by button '6'.

It would be cool to add speed control of the robot joints. But all the mappings tested haven't been able to do that.

The code is here:
`
#include <ros/ros.h>
#include <sensor_msgs/Joy.h>
#include "interbotix_joy_control/ArmJoyControl.h"
// PS3 Controller button mappings
static const std::map<std::string, int> ps3 = {{"GRIPPER_PWM_DEC", 0}, // buttons start here
{"GRIPPER_OPEN", 1},
{"GRIPPER_PWM_INC", 2},
{"GRIPPER_CLOSE", 3},
{"EE_Y_INC", 4},
{"EE_Y_DEC", 5},
{"WAIST_CCW", 6},
{"WAIST_CW", 7},
{"SLEEP_POSE", 8},
{"HOME_POSE", 9},
{"FLIP_EE_X", 11},
{"FLIP_EE_ROLL", 12},
{"SPEED_INC", 13},
{"SPEED_DEC", 14},
{"SPEED_COURSE", 15},
{"SPEED_FINE", 16},
{"EE_X", 0},            // axes start here
{"EE_Z", 1},
{"EE_ROLL", 3},
{"EE_PITCH", 4}};
// PS4 Controller button mappings
static const std::map<std::string, int> ps4 = {{"GRIPPER_PWM_DEC", 0}, // buttons start here
{"GRIPPER_OPEN", 1},
{"GRIPPER_PWM_INC", 2},
{"GRIPPER_CLOSE", 3},
{"EE_Y_INC", 4},
{"EE_Y_DEC", 5},
{"WAIST_CCW", 6},
{"WAIST_CW", 7},
{"SLEEP_POSE", 8},
{"HOME_POSE", 9},
{"FLIP_EE_X", 11},
{"FLIP_EE_ROLL", 12},
{"EE_X", 0},            // axes start here
{"EE_Z", 1},
{"EE_ROLL", 3},
{"EE_PITCH", 4},
{"SPEED_TYPE", 6},
{"SPEED", 7}};
// Xbox 360 Controller button mappings
static const std::map<std::string, int> xbox360 = {{"GRIPPER_PWM_DEC", 0}, // buttons start here
{"GRIPPER_OPEN", 1},
{"GRIPPER_CLOSE", 2},
{"GRIPPER_PWM_INC", 3},
{"WAIST_CCW", 4},
{"WAIST_CW", 5},
{"SLEEP_POSE", 6},
{"HOME_POSE", 7},
{"FLIP_EE_X", 9},
{"FLIP_EE_ROLL", 10},
{"EE_X", 0},            // axes start here
{"EE_Z", 1},
{"EE_Y_INC", 2},
{"EE_ROLL", 3},
{"EE_PITCH", 4},
{"EE_Y_DEC", 5},
{"SPEED_TYPE", 6},
{"SPEED", 7}};
// ThrustMaster Controller button mappings !.!.o
static const std::map<std::string, int> thrustM = {{"GRIPPER_PWM_DEC", 8}, // buttons start here // ok
{"GRIPPER_OPEN", 1}, //realitate 1
{"GRIPPER_CLOSE", 2}, // realitate 3
{"GRIPPER_PWM_INC", 3}, // realitate 2
{"WAIST_CCW", 4}, // realitate 6
{"WAIST_CW", 5}, // // realitate 7
{"SLEEP_POSE", 6}, // realitate 8
{"HOME_POSE", 7}, // realitate 9
{"FLIP_EE_X", 9},
{"SPEED_TYPE", 0},
{"FLIP_EE_ROLL", 10},
{"EE_X", 0},            // axes start here
{"EE_Z", 1},
{"EE_ROLL", 2},
{"EE_PITCH", 5},
{"SPEED_TYPE", 4},
{"SPEED", 3}};
ros::Publisher pub_joy_cmd;                                 // ROS Publisher to publish ArmJoyControl messages
ros::Subscriber sub_joy_raw;                                // ROS Subscriber to get Joy messages from the 'joy_node'
interbotix_joy_control::ArmJoyControl prev_joy_cmd;         // Keep track of the previously commanded ArmJoyControl message so that only unique messages are published
std::map<std::string, int> cntlr;                           // Holds the controller button mappings
std::string controller_type;                                // Holds the name of the controller received from the ROS Parameter server
double threshold;                                           // Joystick sensitivity threshold
/// @brief Joystick callback to create custom ArmJoyControl messages to control the Arm
/// @param msg - raw sensor_msgs::Joy data
void joy_state_cb(const sensor_msgs::Joy &msg)
{
static bool flip_ee_roll_cmd = false;
static bool flip_ee_roll_cmd_last_state = false;
static bool flip_ee_x_cmd = false;
static bool flip_ee_x_cmd_last_state = false;
interbotix_joy_control::ArmJoyControl joy_cmd;
// Check if the ee_x_cmd should be flipped
if (msg.buttons.at(cntlr["FLIP_EE_X"]) == 1 && flip_ee_x_cmd_last_state == false)
flip_ee_x_cmd = true;
else if (msg.buttons.at(cntlr["FLIP_EE_X"]) == 1 && flip_ee_x_cmd_last_state == true)
flip_ee_x_cmd = false;
else if (msg.buttons.at(cntlr["FLIP_EE_X"]) == 0)
flip_ee_x_cmd_last_state = flip_ee_x_cmd;
// Check the ee_x_cmd
if (msg.axes.at(cntlr["EE_X"]) >= threshold && flip_ee_x_cmd == false)
joy_cmd.ee_x_cmd = interbotix_joy_control::ArmJoyControl::EE_X_INC;
else if (msg.axes.at(cntlr["EE_X"]) <= -threshold && flip_ee_x_cmd == false)
joy_cmd.ee_x_cmd = interbotix_joy_control::ArmJoyControl::EE_X_DEC;
else if (msg.axes.at(cntlr["EE_X"]) >= threshold && flip_ee_x_cmd == true)
joy_cmd.ee_x_cmd = interbotix_joy_control::ArmJoyControl::EE_X_DEC;
else if (msg.axes.at(cntlr["EE_X"]) <= -threshold && flip_ee_x_cmd == true)
joy_cmd.ee_x_cmd = interbotix_joy_control::ArmJoyControl::EE_X_INC;
// Check the ee_y_cmd
if (controller_type == "ps3" || controller_type == "ps4")
{
if (msg.buttons.at(cntlr["EE_Y_INC"]) == 1)
joy_cmd.ee_y_cmd = interbotix_joy_control::ArmJoyControl::EE_Y_INC;
else if (msg.buttons.at(cntlr["EE_Y_DEC"]) == 1)
joy_cmd.ee_y_cmd = interbotix_joy_control::ArmJoyControl::EE_Y_DEC;
}
else if (controller_type == "xbox360")
{
if (msg.axes.at(cntlr["EE_Y_INC"]) <= 1.0 - 2.0 * threshold)
joy_cmd.ee_y_cmd = interbotix_joy_control::ArmJoyControl::EE_Y_INC;
else if (msg.axes.at(cntlr["EE_Y_DEC"]) <= 1.0 - 2.0 * threshold)
joy_cmd.ee_y_cmd = interbotix_joy_control::ArmJoyControl::EE_Y_DEC;
}
// Check the ee_z_cmd
if (msg.axes.at(cntlr["EE_Z"]) >= threshold)
joy_cmd.ee_z_cmd = interbotix_joy_control::ArmJoyControl::EE_Z_INC;
else if (msg.axes.at(cntlr["EE_Z"]) <= -threshold)
joy_cmd.ee_z_cmd = interbotix_joy_control::ArmJoyControl::EE_Z_DEC;
// Check if the ee_roll_cmd should be flipped
if (msg.buttons.at(cntlr["FLIP_EE_ROLL"]) == 1 && flip_ee_roll_cmd_last_state == false)
flip_ee_roll_cmd = true;
else if (msg.buttons.at(cntlr["FLIP_EE_ROLL"]) == 1 && flip_ee_roll_cmd_last_state == true)
flip_ee_roll_cmd = false;
else if (msg.buttons.at(cntlr["FLIP_EE_ROLL"]) == 0)
flip_ee_roll_cmd_last_state = flip_ee_roll_cmd;
// Check the ee_roll_cmd
if (msg.axes.at(cntlr["EE_ROLL"]) >= threshold && flip_ee_roll_cmd == false)
joy_cmd.ee_roll_cmd = interbotix_joy_control::ArmJoyControl::EE_ROLL_CW;
else if (msg.axes.at(cntlr["EE_ROLL"]) <= -threshold && flip_ee_roll_cmd == false)
joy_cmd.ee_roll_cmd = interbotix_joy_control::ArmJoyControl::EE_ROLL_CCW;
else if (msg.axes.at(cntlr["EE_ROLL"]) >= threshold && flip_ee_roll_cmd == true)
joy_cmd.ee_roll_cmd = interbotix_joy_control::ArmJoyControl::EE_ROLL_CCW;
else if (msg.axes.at(cntlr["EE_ROLL"]) <= -threshold && flip_ee_roll_cmd == true)
joy_cmd.ee_roll_cmd = interbotix_joy_control::ArmJoyControl::EE_ROLL_CW;
// Check the ee_pitch_cmd
if (msg.axes.at(cntlr["EE_PITCH"]) >= threshold)
joy_cmd.ee_pitch_cmd = interbotix_joy_control::ArmJoyControl::EE_PITCH_UP;
else if (msg.axes.at(cntlr["EE_PITCH"]) <= -threshold)
joy_cmd.ee_pitch_cmd = interbotix_joy_control::ArmJoyControl::EE_PITCH_DOWN;
// Check the waist_cmd
if (msg.buttons.at(cntlr["WAIST_CCW"]) == 1)
joy_cmd.waist_cmd = interbotix_joy_control::ArmJoyControl::WAIST_CCW;
else if (msg.buttons.at(cntlr["WAIST_CW"]) == 1)
joy_cmd.waist_cmd = interbotix_joy_control::ArmJoyControl::WAIST_CW;
// Check the gripper_cmd
if (msg.buttons.at(cntlr["GRIPPER_CLOSE"]) == 1)
joy_cmd.gripper_cmd = interbotix_joy_control::ArmJoyControl::GRIPPER_CLOSE;
else if (msg.buttons.at(cntlr["GRIPPER_OPEN"]) == 1)
joy_cmd.gripper_cmd = interbotix_joy_control::ArmJoyControl::GRIPPER_OPEN;
// Check the robot_pose
if (msg.buttons.at(cntlr["HOME_POSE"]) == 1)
joy_cmd.robot_pose = interbotix_joy_control::ArmJoyControl::HOME_POSE;
else if (msg.buttons.at(cntlr["SLEEP_POSE"]) == 1)
joy_cmd.robot_pose = interbotix_joy_control::ArmJoyControl::SLEEP_POSE;
if (controller_type == "ps3")
{
// Check the speed_cmd
if (msg.buttons.at(cntlr["SPEED_INC"]) == 1)
joy_cmd.speed_cmd = interbotix_joy_control::ArmJoyControl::SPEED_INC;
else if (msg.buttons.at(cntlr["SPEED_DEC"]) == 1)
joy_cmd.speed_cmd = interbotix_joy_control::ArmJoyControl::SPEED_DEC;
// Check the toggle_speed_cmd
if (msg.buttons.at(cntlr["SPEED_COURSE"]) == 1)
  joy_cmd.toggle_speed_cmd = interbotix_joy_control::ArmJoyControl::SPEED_COURSE;
else if (msg.buttons.at(cntlr["SPEED_FINE"]) == 1)
  joy_cmd.toggle_speed_cmd = interbotix_joy_control::ArmJoyControl::SPEED_FINE;

}
else if (controller_type == "ps4" || controller_type == "xbox360")
{
// Check the speed_cmd
if (msg.axes.at(cntlr["SPEED"]) == 1)
joy_cmd.speed_cmd = interbotix_joy_control::ArmJoyControl::SPEED_INC;
else if (msg.axes.at(cntlr["SPEED"]) == -1)
joy_cmd.speed_cmd = interbotix_joy_control::ArmJoyControl::SPEED_DEC;
// Check the toggle_speed_cmd
if (msg.axes.at(cntlr["SPEED_TYPE"]) == 1)
  joy_cmd.toggle_speed_cmd = interbotix_joy_control::ArmJoyControl::SPEED_COURSE;
else if (msg.axes.at(cntlr["SPEED_TYPE"]) == -1)
  joy_cmd.toggle_speed_cmd = interbotix_joy_control::ArmJoyControl::SPEED_FINE;

}
// Check the gripper_pwm_cmd
if (msg.buttons.at(cntlr["GRIPPER_PWM_INC"]) == 1)
joy_cmd.gripper_pwm_cmd = interbotix_joy_control::ArmJoyControl::GRIPPER_PWM_INC;
else if (msg.buttons.at(cntlr["GRIPPER_PWM_DEC"]) == 1)
joy_cmd.gripper_pwm_cmd = interbotix_joy_control::ArmJoyControl::GRIPPER_PWM_DEC;
// Only publish a ArmJoyControl message if any of the following fields have changed.
if (!(prev_joy_cmd.ee_x_cmd == joy_cmd.ee_x_cmd &&
prev_joy_cmd.ee_y_cmd == joy_cmd.ee_y_cmd &&
prev_joy_cmd.ee_z_cmd == joy_cmd.ee_z_cmd &&
prev_joy_cmd.ee_roll_cmd == joy_cmd.ee_roll_cmd &&
prev_joy_cmd.ee_pitch_cmd == joy_cmd.ee_pitch_cmd &&
prev_joy_cmd.waist_cmd == joy_cmd.waist_cmd &&
prev_joy_cmd.gripper_cmd == joy_cmd.gripper_cmd &&
prev_joy_cmd.robot_pose == joy_cmd.robot_pose &&
prev_joy_cmd.speed_cmd == joy_cmd.speed_cmd &&
prev_joy_cmd.toggle_speed_cmd == joy_cmd.toggle_speed_cmd &&
prev_joy_cmd.gripper_pwm_cmd == joy_cmd.gripper_pwm_cmd))
pub_joy_cmd.publish(joy_cmd);
prev_joy_cmd = joy_cmd;
}
int main(int argc, char **argv)
{
ros::init(argc, argv, "interbotix_joy_control_node");
ros::NodeHandle n;
ros::param::get("~threshold", threshold);
ros::param::get("~controller", controller_type);
if (controller_type == "xbox360")
cntlr = xbox360;
else if (controller_type == "ps3")
cntlr = ps3;
else if (controller_type == "thrustM") //!.!.o mod
cntlr = thrustM;
else
cntlr = ps4;
sub_joy_raw = n.subscribe("joy", 100, joy_state_cb);
pub_joy_cmd = n.advertise<interbotix_joy_control::ArmJoyControl>("joy/commands", 100);
ros::spin();
return 0;
}
`

Hi @Ktysai,
Nice to see someone on their way to incorporating another joystick! About your issue - I've never used a thrust master controller before, but your first step would be to check that the joy node works on Linux with that type of controller. Essentially, do this (after paring the joystick with the Linux computer)

Open a terminal and type roscore
Open a second terminal and type rosrun joy joy_node
Open a third terminal and type rostopic echo /joy

Now, run through all the buttons and sticks on your thrust master controller one by one, and pay attention to the index by which they appear in the 'buttons' and 'axes' arrays. These are the mappings you want to use in the joy_node.cpp file.
Also note that you will have to edit some of the 'if' - 'else' conditions in the file to include your controller. For example, when checking the 'speed_cmd', you will have to include 'thrustM' in the condition as well, as by default, it's excluded.

OK, @swinterbotix!
I'll recheck the mappings in the joy rostopic, and with jstest -gtk. If only joy_node.cpp file has the control over all the mappings makes things easier.
The joystick works fine for controlling a differential two motor wheels robot, like turtlebot.

I've checked the joy topic and indeed the buttons are observed correctly. Tried diffident mappings... but nada
Still have the issues regarding the mappings of the buttons, even if button '0' has other designation, all the time it will decrease the GRIPER PWM. Is like there is a master mapping of the joystick, that has priority.
Anyway, I'll leave it like that because I can control the arm.
The default joint speed, is a little bit too high for my liking, so I've set the value '0.75', for the 'current' variable in joy_robot_control.py. Is not nice, but it works for now.
Btw, the code for the control of the robot is open source? Cannot be a part of Moveit? if I understand correctly, at the moment Moveit lacks this control option.

Strange that you're not able to map the gripper pwm inc/dec or the joint speed buttons correctly. Without actually having a ThrustM controller, I can't really help you debug that unfortunately.
Yes, the code for all of this is open source. If someone wanted to, they could create a pipeline to do this via MoveIt. I'd image the structure would be the joy_node (responsible for publishing joystick values to a topic) -> joy_mapper_node (responsible for mapping joystick buttons to desired Cartesian motion) -> robot_moveit_control_node (responsible for planning motion and executing using MoveIt's Move Group API).
However, in my experience, MoveIt is terrible at realtime motion and more often than not - will yell at you saying it can't find a desired pose - essentially making the whole joystick control node useless. You might be able to use this joystick pipeline to define desired waypoints for an arm to follow, and in the unlikely miracle that MoveIt actually is able to plan all the waypoints successfully - execute them - but it wouldn't be in real time.
That's one of the reasons I created my own Cartesian joystick node. It can be used in realtime and will never fail in finding the next desired goal pose (unless you're moving the end-effector out of the physical workspace of the arm).
Anyway, that's my rant on MoveIt. Thanks for listening :)

No problem, it was a useful rant :1st_place_medal:
Just after progressively I understand more about ROS, and in this particular case about Moveit, I can really appreciate add-ons for it.
Your idea is good, and also the implementation.
I'll use the node as it is, and maybe take a look at those mappings.
To keep the discussion into the joystick area, what model do you advise me to invest?
I've tried with ones borrowed from friends:

Joystick for Xbox one - very smooth, precise. It was hard until I get the proper drivers, and some mappings;
Joystick PS3 - less precise than the one for Xbox one, works out of the box;
Joystick Xbox 360,  works almost out of the box, the precision was weird... it was seriously warn out;
Joystick Thrustmater T-wireless, works mostly out of the box, and have the best coverage range from all tested. Lacks precision, but resists to the (un)wanted attacks of my twins :)

I haven't tried PS4 yet :)
If you have the time, can you please try mapping the joystick you have?
Now can I use your Cartesian joystick node to:

to put the robot in a desired position and get the joint position value?
can it be run almost 'simultaneous' with Moveit? To run the Setup Assistant for example, and to set several position that are wanted?

I've tested my code with both the PS3 and PS4 controllers, and verified that they work correctly. A different user added the Xbox360 controller, but I have not actually tested it. In my opinion, a PS4 controller is the way to go just because it's easily available, and I know it works (at least the official SONY version). It's hard to find an official SONY PS3 controller nowadays - but you can also go that route if money is an issue.

Closing due to lack of activity. Feel free to reopen if you still have questions.


RC3 compatibility

RC3 compatibility would be real nice. Any chance it will happen soon?

See https://github.com/Leaflet/Leaflet.draw/pull/596

Thank you!


Adjusting center on a fixture may not be working as intended.

Disclaimer: I'm fairly new to Box2D so my knowledge of the engine is based on quite a bit of reading on iForce2D
I'm trying to attach a sensor to a body by offsetting the sensors Center in a Fixture as shown in this Box2D guide however when doing so the center-offset is completely ignored and new objects are added to the center of an object, creating something that looks like this:

Link to Testbed

Sourcecode
planck.testbed('GroundSensor', function (testbed) {
    var Edge = planck.Edge, Box = planck.Box, Vec2 = planck.Vec2, Circle = planck.Circle, RevoluteJoint = planck.RevoluteJoint;
    var world = new planck.World(Vec2(0, -9.82));

    let edge = world.createBody();
    edge.createFixture(Edge(Vec2(-20, 0), Vec2(20, 0)));

    let body = world.createDynamicBody();
    body.createFixture(Box(1, 2));
    body.setFixedRotation(true);
    body.setPosition(Vec2(0, 10));

    let sensorFixture = body.createFixture(Box(0.6, 0.3), Vec2(0, -2), 0);
    sensorFixture.setSensor(true);
    sensorFixture.setUserData('player');https://i.imgur.com/7M8afG6.png
    return world;
});

The way I've found to combat this is using joints, but I feel like this isn't the right way.

Link to Testbed

Sourcecode
planck.testbed('GroundSensorJoint', function (testbed) {
    var Edge = planck.Edge, Box = planck.Box, Vec2 = planck.Vec2, Circle = planck.Circle, RevoluteJoint = planck.RevoluteJoint;
    var world = new planck.World(Vec2(0, -9.82));

    let edge = world.createBody();
    edge.createFixture(Edge(Vec2(-20, 0), Vec2(20, 0)));

    let body = world.createDynamicBody();
    body.createFixture(Box(1, 2));
    body.setFixedRotation(true);
    body.setPosition(Vec2(0, 10));

    let sensor = world.createDynamicBody();
    let sensorFixture = sensor.createFixture(Box(0.6, 0.3));
    sensorFixture.setSensor(true);
    sensorFixture.setUserData('player');
    sensor.setPosition(Vec2(0, 8));

    let joint = RevoluteJoint({}, sensor, body, Vec2(0, 8));
    world.createJoint(joint);

    return world;
});

Fixtures do not have geometry and you need to position shape, so here is the correct code:
body.createFixture(Box(0.6, 0.3, Vec2(0, -2), 0))

Note that I'm passing location and angle to Box instead of createFixture.

Fixtures do not have geometry and you need to position shape, so here is the correct code:
body.createFixture(Box(0.6, 0.3, Vec2(0, -2), 0))

Note that I'm passing location and angle to Box instead of createFixture.

By the way, there is a Discord channel which is more active than here, if you need faster response please feel free to use it. Also you are making great examples, they may help others learn something new with you.

By the way, there is a Discord channel which is more active than here, if you need faster response please feel free to use it. Also you are making great examples, they may help others learn something new with you.


Figure detected incorrect

While using extract_pages, whole pdf page is detected as Figure which is incorrect extraction.
The pdf file I used is:
N18-3011.pdf
My codes:
from pdfminer.high_level import extract_pages
for page_layout in extract_pages("N18-3011.pdf"):
    for element in page_layout:
        print(element)

My outputs:
<LTTextBoxHorizontal(0) 129.748,26.592,465.527,46.995 'Proceedings of NAACL-HLT 2018, pages 84–91\nNew Orleans, Louisiana, June 1 - 6, 2018. c(cid:13)2017 Association for Computational Linguistics\n'>
<LTTextBoxHorizontal(1) 293.317,53.182,304.226,64.091 '84\n'>
<LTFigure(Fm4) 0.001,0.001,595.276,841.889 matrix=[1.00,0.00,0.00,1.00, (0.00,0.00)]>
<LTTextBoxHorizontal(0) 293.317,53.182,304.226,64.091 '85\n'>
<LTFigure(Fm8) 0.001,0.001,595.276,841.889 matrix=[1.00,0.00,0.00,1.00, (0.00,0.00)]>
<LTTextBoxHorizontal(0) 293.317,53.182,304.226,64.091 '86\n'>
<LTFigure(Fm11) 0.001,0.001,595.276,841.889 matrix=[1.00,0.00,0.00,1.00, (0.00,0.00)]>
<LTTextBoxHorizontal(0) 293.317,53.182,304.226,64.091 '87\n'>
<LTFigure(Fm14) 0.001,0.001,595.276,841.889 matrix=[1.00,0.00,0.00,1.00, (0.00,0.00)]>
<LTTextBoxHorizontal(0) 293.317,53.182,304.226,64.091 '88\n'>
<LTFigure(Fm17) 0.001,0.001,595.276,841.889 matrix=[1.00,0.00,0.00,1.00, (0.00,0.00)]>
<LTTextBoxHorizontal(0) 293.317,53.182,304.226,64.091 '89\n'>
<LTFigure(Fm20) 0.001,0.001,595.276,841.889 matrix=[1.00,0.00,0.00,1.00, (0.00,0.00)]>
<LTTextBoxHorizontal(0) 293.317,53.182,304.226,64.091 '90\n'>
<LTFigure(Fm23) 0.001,0.001,595.276,841.889 matrix=[1.00,0.00,0.00,1.00, (0.00,0.00)]>
<LTTextBoxHorizontal(0) 293.317,53.182,304.226,64.091 '91\n'>
<LTFigure(Fm26) 0.001,0.001,595.276,841.889 matrix=[1.00,0.00,0.00,1.00, (0.00,0.00)]>

mark

same problem


wasmtime: Add a config knob to control the maximum number of stack frames captured in a backtrace

Would make capturing a backtrace go from O(n) where n is the number of Wasm frames on the stack to constant time for some user-defined acceptable constant. Max frames could be set to zero to disable backtraces completely. Would subsume the current (and deprecated) Config::wasm_backtrace method.
Follow up to #5049.

cc @Stebalien

FWIW, last time I looked, V8 would only capture 20 stack frames in its JS error stacks. Relevant precedent.


View notes part1 - Gateway and refactorings

Context
Back office users need ability to view notes on a beacon record.
Changes in this pull request
NotesGateway.ts, INotesResponse.ts, NotesResponseMapper.ts: pretty standard gateway back-office gateway type pattern.
NotesGateway.test.ts, NotesResponseMapper.test.ts: tests for the above
Also moved a bunch of files in to appropriate folders within the gateways folder, and fixed imports.
Guidance to review
not currently consumed anywhere
Link to Trello card
https://trello.com/c/t6q3ynN8/819-back-office-users-need-ability-to-view-notes-on-a-beacon-record

Accidentally superseded by #177 but will probably incorporate some good stuff into main from here like enum for NoteType!


Add example

Add Trussed Virt-based example and tests

See fuzzing example for the inspiration:

https://github.com/Nitrokey/oath-authenticator/commit/13c7c2d87997ce35b24f9a081d31476fe95fd093
https://github.com/Nitrokey/oath-authenticator/blob/main/fuzz/fuzz_targets/fuzz_target_1.rs


Correct source links

Corrects the URLs in the links to source for utcDay, unixDay, utcDays, unixDays.
I see that some of the links do not have Source as the title attribute. I did not change this because I don't know the current convention.

Thank you! The current convention is  · Source, Examples with a link to the source file and no line number or title. The "Examples" link goes to an observable notebook, if any is available.


Request executor schema disposal changes and test

Fixes closure issues over the Schema objects preventing correct disposal of evicted schemas.
Test demonstrating the resolution with heap inspection to prove correct disposal.
Of note, this test will likely need to run in isolation from other tests to avoid additional schema objects from other tests being misidentified. Happy for feedback on this if we need to get this test running alongside other tests, I'll need to look at multiple memory snapshots to exclude existing objects from other tests. Additionally, parallelized tests will cause all sorts of funky.
Closes #4863

Anything I need to do to make the CI/CD pipelines happy?


[ci]: move jobs to github runners

Description of the Change
Replace self-hosted runners inside tests workflows with the default ubuntu-latest Gihub Runners.
Issue
Some long tests are failed on self-hosted runners due to undetected reason.
Benefits
Long tests should not be failed when they are run on the deault GH Runners.
Possible Drawbacks
Should be none, but we have to re-implement self-hosted runners.

After some tests, I think that the problem isn't in runners.

Isn't the issue this PR is targeting #2780 @BAStos525 ? If so, it seems to be related to cargo-llvm-cov

Perhaps you are right. I can't say for sure what is the exact reason. I can just tell that it's not related to compute resources lack: I tried to run this job on 1 x Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz & 32GB RAM and it was the same.
But this workflow is successfully finished on this state.

Didn't seem to help much. We should revisit this later. For now, not having coverage is fine.

This PR is actual now: our colleague destroyed all AWS ephemeral self-hosted runners, so we can use only GH Actions default runners. We can create a new self-hosted runners in other cloud platform if they will be required in the feature.


🐛 Fix not emitting comments between sibling fields of object literals

Fixes #49784

This approach breaks many other test cases. So, I close it until I've found a viable solution.

@RyanCavanaugh The fix seems working now. However, the changes to the emitter affected 15 other test cases, fortunately, in a good way. That is, some trailing comments which were dropped in the reference baselines can now be retained as they were. Anyway, I totally get it that accepting any kind of change to old test cases requires careful consideration on your side.
For example, in assignmentCompatBug3.js, the comments below are now kept (as stated by the original test case file, assignmentCompatBug3.ts):
// assignmentCompatBug3.js
- get x() { return x; },
- get y() { return y; },
+ get x() { return x; }, // shouldn't be "void"
+ get y() { return y; }, // shouldn't be "void"

// assignmentCompatBug3.ts
get x() { return x;}, // shouldn't be "void"
get y() { return y;}, // shouldn't be "void"

Affected reference baselines
(15 cases)

arrowFunctionParsingDoesNotConfuseParenthesizedObjectForArrowHead.js
assignmentCompatBug3.js
bindingPatternCannotBeOnlyInferenceSource.js
callSignaturesWithParameterInitializers2.js
circularResolvedSignature.js
duplicateObjectLiteralProperty.js
excessPropertyCheckWithMultipleDiscriminants.js
excessPropertyCheckWithNestedArrayIntersection.js
indexSignatures1.js
objectLitArrayDeclNoNew.js
objectLiteralShorthandPropertiesErrorFromNoneExistingIdentifier.js
privateNameBadDeclaration.js
thisTypeInFunctions.js
thisTypeInFunctionsNegative.js
variableDeclaratorResolvedDuringContextualTyping.js

New test case
I've added a test case, commentsOnObjectLiteral5.ts (at tests/cases/compiler) to confirm that all comments are emitted as they are:
const a = {
    p0: 0, // Comment 0
    p1: 0, /* Comment 1
    A multiline comment. */
    p2: 0, // Comment 2
};

Nice one :+1: Thanks


add analyze and publish with dry run

Sorry to interrupt, we are planning to integrate mix panel into our project
But we found it is not yet published to pub.dev.
Any plan to release ?
Thanks

hi @thomaslee-u yes! we will release it by the end of this week. will let you know! Thanks for your interest!


Add support for threejs.LineSegments

I'd be happy to do a pull request to add lines to meshcat. Is that something you would like to include? If so, would you give me some pointers on how to go about adding that?

Definitely! I would certainly appreciate a PR :slightly_smiling_face:
Fortunately, this doesn't require any changes to the javascript side of the code. We just need to add a few new Python classes. In threejs, Line, LineSegments, and LineLoop are all types of objects (just like Mesh is). So, on the python side, we make classes for them just like we do for Mesh. Starting from https://github.com/rdeits/meshcat-python/blob/d26423c5efe776a811319a09c9fb0304211b5a2a/src/meshcat/geometry.py#L261-L262 , I think the classes should be as simple as:
class Line(Object):
    _type = u"Line"

and the same for LineSegments and LineLoop.
You would then be able to construct one like this:
import meshcat
import meshcat.geometry as g
vis = meshcat.Visualizer()
vis.set_object(g.LineSegments(g.PointsGeometry(vertices)))

where vertices is a 3xN numpy array. I haven't tested the above code, so it may need a few tweaks, but I think the general idea is right.
If that works, we can consider making a more convenient function to avoid needing to type LineSegments(PointsGeometry(...)) every time.

I’ll take a look! Thanks.

Added pull request #43.
I did not yet go ahead and add the convenience functions because I didn't know what you would prefer. Also the best names seem to have already been taken by the other classes. Maybe something like LineGroup as an equivalent to PointCloud might be good.

I also added the ability (in a separate pull request #44 ) to load mesh data which is already in memory. We didn't discuss this before hand but seems useful.

Closed by #43 !


Not able to add multiple CKEditors on same page

Not able to add multiple CKEditors on same page
Is there a way to do it because when trying to render same form multiple times in template I am getting a JS error

Error is maybe due to same name for 2 text editors but how can I resolve it

Yes it's possible. You have to use different IDs for the two elements.

Thanks for replying
but Id is added by default on my form when creating it by using RichTextField in models.py
Where I should change it

You could use forms with differing prefixes https://docs.djangoproject.com/en/3.1/ref/forms/api/#prefixes-for-forms

I'm closing this issue as its due to django form giving a default id to each form
Not related to django-ckeditor
And it can be solved by using your own forms using HTML and making a custom adaptor

Here is a quick fix to help people who face this problem in the future and stumble upon this issue while searching for a solution:
# Contents of admin.py
from django.contrib import admin
from django import forms
from ckeditor.widgets import CKEditorWidget
from .models import Article

class ArticleAdminForm(forms.ModelForm):
    """A form to allow different ids for the CKEditorWidgets

    By default, ckeditor.fields.RichTextField seems to assign the same id to 2 different
    ckeditor fields in the same form. See this issue:

    https://github.com/django-ckeditor/django-ckeditor/issues/637

    To circumvent this, we can assign different ids to the CKEditorWidgets.
    """

    content_en = forms.CharField(widget=CKEditorWidget(attrs={"id": "content_en"}))
    content_fr = forms.CharField(widget=CKEditorWidget(attrs={"id": "content_fr"}))

    class Meta:
        model = Article
        fields = "__all__"

@admin.register(Article)
class ArticleAdmin(admin.ModelAdmin):
    form = ArticleAdminForm

This is confusing. It works perfectly even without specifying the form field name twice, as follows:
    content_en = forms.CharField(widget=CKEditorWidget())
    content_fr = forms.CharField(widget=CKEditorWidget())

I'm not sure what's going on in your projects. But the ID which Django's forms automatically generate are already unique except if you're rendering the exact same form twice which cannot work anyway.


Module incompatibility with PHP 8.1

.+Please check the FAQ (frequently-asked questions) first. If you have other questions or something to report, please address the following:
+## PHP Driver version or file name
+5.10.0beta2
+## Client operating system
+Windows
+## PHP version
+8.1
+## Problem description
+I have an IIS server with PHP 8.1. Tried the 5.10.0beta2. Copied php_sqlsrv.dll to ext directory, and edited php.ini file to point to this. Restarted IIS and restarted computer. However, sqlsrv does not load in info.php. Running PHP in command prompt gives the following error:
PHP Warning:  PHP Startup: sqlsrv: Unable to initialize module
Module compiled with module API=20200930
PHP    compiled with module API=20210902
These options need to match
I read this error as that the PHP version is 8.1 (API=20210902), but sqlsrv.dll version is 8.0 (20200930).
I removed the previous .dll file and restarted IIS again, but the problem persists.
Edit:
I downloaded the dll file from https://pecl.php.net/package/sqlsrv/5.10.0beta2/windows
Specifically https://windows.php.net/downloads/pecl/releases/sqlsrv/5.10.0beta2/php_sqlsrv-5.10.0beta2-8.0-nts-vs16-x64.zip
Filename says 8.0 Non Thread Safe (NTS) x64. I'm wondering if I got the wrong version. Is there a way to get a windows dll for php 8.1?

@neurogroups , make sure you download the right binaries from 5.10 beta2

If you prefer the one in pecl, please download the one for 8.1 nts as well
https://windows.php.net/downloads/pecl/releases/sqlsrv/5.10.0beta2/

This should work- sorry I missed that link. I'll use this and update.
On Tue, Jan 18, 2022 at 4:43 PM Jenny Tam @.***> wrote:

@neurogroups https://github.com/neurogroups , make sure you download
the right binaries from 5.10 beta2
https://github.com/microsoft/msphpsql/releases/tag/v5.10.0-beta2
[image: image]
https://user-images.githubusercontent.com/21340756/150030512-49fac2a1-7d2a-4c5f-9bfe-a1d781fd2fbb.png
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/msphpsql/issues/1356#issuecomment-1015900158,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACOLO5Z4QAIZZPE4454XR3LUWXUJ7ANCNFSM5MIIE3AQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>

This worked for me. Thanks for pointing me in the right direction!

Great! I'll close this issue then.

Hello
can you help me to solve this problem please
Warning: PHP Startup: curl: Unable to initialize module
Module compiled with module API=20210902
PHP    compiled with module API=20230831
These options need to match
in Unknown on line 0
INFO  Running migrations.
2024_12_02_171812_create_locations_table ............................................... 63.60ms FAIL
Illuminate\Database\QueryException
SQLSTATE[42000]: Syntax error or access violation: 1067 Invalid default value for 'tracked_at' (Connection: mysql, SQL: create table locations (id bigint unsigned not null auto_increment primary key, created_at timestamp null, updated_at timestamp null, tracteur_id bigint unsigned not null, latitude
decimal(10, 8) not null, longitude decimal(11, 8) not null, altitude decimal(8, 2) null, tracked_at timestamp not null) default character set utf8mb4 collate 'utf8mb4_unicode_ci')
at vendor\laravel\framework\src\Illuminate\Database\Connection.php:825
821▕                     $this->getName(), $query, $this->prepareBindings($bindings), $e
822▕                 );
823▕             }
824▕
➜ 825▕             throw new QueryException(
826▕                 $this->getName(), $query, $this->prepareBindings($bindings), $e
827▕             );
828▕         }
829▕     }
1   vendor\laravel\framework\src\Illuminate\Database\Connection.php:571
PDOException::("SQLSTATE[42000]: Syntax error or access violation: 1067 Invalid default value for 'tracked_at'")
2   vendor\laravel\framework\src\Illuminate\Database\Connection.php:571
PDOStatement::execute()


Unable to update launch.json (via API) in multi-root workspace

I have some "project upgrade" functions that run when my extension activates to handle breaking changes. I merged my two debuggers today such that I need to update the type inside the launch.json.
I have the following code;
function convert_legacy_debug_types(workspace: vs.WorkspaceFolder) {
	// Read launch.json config.
	const launchFile = vs.workspace.getConfiguration("launch");

	const configs = launchFile.get<any[]>("configurations");
	if (!configs)
		return;

	let hasChanged = false;

	// Find Dart CLI items that might need upgrading.
	configs.filter((c) => c.type === "dart-cli" || c.type === "flutter").map((d) => {
		console.log("Found old dart-cli/flutter debug type, setting to dart...");
		d.type = "dart";
		hasChanged = true;
	});

	if (hasChanged)
		launchFile.update("configurations", configs);
}

This code works great if I open a single folder. However if I open a multi-root workspace there are two issues:

I only seem to get the launch configs for one of the projects
The update call at the end doesn't seem to persist to risk at all (not even for the one projects we modified)

I tried passing the workspace uris into getConfiguration however it warns:

Accessing a window scoped configuration for a resource is not expected. To associate 'launch' to a resource, define its scope to 'resource' in configuration contributions in 'package.json'.

What's the correct way to make the code above work for a multi-root workspace?

@DanTup please use the inspect instead of the get.
The same way I am doing in our core here
Let me know if that is not working and I will reopen this.
@sandy081 can correct me if I am wrong, but I believe this is the correct way to do it also for extensoins

@isidorn I'm confused; the doc for inspect suggests it only takes a single argument (string), how do I pass the workspaceFolder? (your example code seems to pass it as a second argument?)

@DanTup Example code from @isidorn is internal. What ever you did by passing the URI to getConfiguration is correct and gives you the configuration for that resource.
Warning should not be shown as it is applicable only when you access window configurations using a resource where launch is not.
I will fix the warning.

@sandy081 Aha, you're right. It does work, but I had to add WorkspaceFolder as the configuration target for the update at the end:
launchFile.update("configurations", configs, vs.ConfigurationTarget.WorkspaceFolder);

I'm not sure why (isn't the launchFile variable already for the workspace folder?), but it's working. Thanks!

@DanTup While writing default target is always Workspace and you can write launches into Workspace configuration file too in a MR workspace.

Yep, the warning is from reading the value with:
const launchFile = vs.workspace.getConfiguration("launch", workspace.uri);

launch and task are resource specific configurations by default. So accessing them needed a resource to be passed.
Updated the warning strategy to consider them resource scoped.


python311Packages.asyncsleepiq: 1.4.0 -> 1.4.1

Changelog: https://github.com/kbickar/asyncsleepiq/releases/tag/v1.4.1
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 277798 run on x86_64-linux 1

5 packages built:

home-assistant-component-tests.sleepiq
python310Packages.asyncsleepiq
python310Packages.asyncsleepiq.dist
python311Packages.asyncsleepiq
python311Packages.asyncsleepiq.dist

Result of nixpkgs-review pr 277798 run on x86_64-linux 1

5 packages built:

home-assistant-component-tests.sleepiq
python310Packages.asyncsleepiq
python310Packages.asyncsleepiq.dist
python311Packages.asyncsleepiq
python311Packages.asyncsleepiq.dist

Result of nixpkgs-review pr 277798 run on aarch64-darwin 1

4 packages built:

python310Packages.asyncsleepiq
python310Packages.asyncsleepiq.dist
python311Packages.asyncsleepiq
python311Packages.asyncsleepiq.dist


Add function markdownLink.

This makes a markdown link, with optional associated text, for programmatically inserting markdown links.

Good idea, thanks Rory


Issue including XpathGrammar.h

Tried compiling target=64bit of master
EDIT: should probably mention, I am on windows 64 and this is targeted to windows 64. Followed the guide on BUILDING.md except ....
I changed target_cpu in windows.py, rather than build.py as the CPUArch include is missing in build.py so "CPUArch" is undefined, figured it wouldn't matter. (Unless it does and causes this issue ... )
Either way, I guess from buildlib.common import CPUArch should be added by default or mentioned in the README?
Compile error:
[17193/24289] CXX obj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj
FAILED: obj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj
ninja -t msvc -e environment.x64 -- "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64/cl.exe" /nologo /showIncludes /FC @obj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj.rsp /c ../../third_party/WebKit/Source/core/xml/XPathParser.cpp /Foobj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj /Fd"obj/third_party/WebKit/Source/core/xml/xml_cc.pdb"
g:\chromium\ungoogled-chromium\build\sandbox\third_party\webkit\source\core\xml\xpathparser.cpp(31): fatal error C1083: Cannot open include file: 'core/XPathGrammar.h': No such file or directory
[17201/24289] CXX obj/third_party/WebKit/Source/core/workers/workers/WorkerThread.obj

Is this a known issue or did I do something wrong?
Ask if you need any more information.

I changed target_cpu in windows.py, rather than build.py as the CPUArch include is missing in build.py so "CPUArch" is undefined, figured it wouldn't matter. (Unless it does and causes this issue ... )

It really doesn't matter, but users are supposed to import CPUArch from buildilb.common. Terrible design is terrible. I can just add the import to build.py by default I guess.

Is this a known issue or did I do something wrong?

I built the Windows x64 build on Windows 10 Pro x64, but I don't think that's related to the issue. This is a weird error, since I don't know why the includes are wrong (for a file within the Chromium source tree, nonetheless).
Do your versions of Visual Studio and its components match exactly to what's listed in BUILDING? Do you have any other versions of Visual Studio installed?

I installed all the requirements correctly.
Visual studio has all the required things selected and I added all the needed binaries to the Path.
I now set builder.target_cpu = CPUArch.x64 and imported CPUarch in build.py
And I ran:
G:\chromium>git clone https://github.com/Eloston/ungoogled-chromium.git
G:\chromium>cd ungoogled-chromium
G:\chromium\ungoogled-chromium>C:\Python35\python.exe build.py

I did not change anything beside adding the target_cpu in build.py rather than windows.py, which you yourself said wouldn't matter.
And now I get this error:
DEBUG:root:Running: mc.exe -r c:\users\*\appdata\local\temp\tmp0abet4\gen\base/trace_event/etw_manifest -h c:\users\*\appdata\local\temp\tmp0abet4\gen\base/trace_event/etw_manifest -u -um G:\chromium\ungoogled-chromium\build\sandbox\base/trace_event/etw_manifest/chrome_events_win.man
Traceback (most recent call last):
  File "tools\gn\bootstrap\bootstrap.py", line 754, in <module>
    sys.exit(main(sys.argv[1:]))
  File "tools\gn\bootstrap\bootstrap.py", line 121, in main
    return run_build(tempdir, options)
  File "tools\gn\bootstrap\bootstrap.py", line 71, in run_build
    build_gn_with_ninja_manually(tempdir, options)
  File "tools\gn\bootstrap\bootstrap.py", line 183, in build_gn_with_ninja_manually
    'base/trace_event/etw_manifest/chrome_events_win.man')
  File "tools\gn\bootstrap\bootstrap.py", line 134, in write_compiled_message
    os.path.join(SRC_ROOT, source),
  File "tools\gn\bootstrap\bootstrap.py", line 44, in check_call
    subprocess.check_call(cmd, cwd=GN_ROOT, **kwargs)
  File "C:\Python27\lib\subprocess.py", line 181, in check_call
    retcode = call(*popenargs, **kwargs)
  File "C:\Python27\lib\subprocess.py", line 168, in call
    return Popen(*popenargs, **kwargs).wait()
  File "C:\Python27\lib\subprocess.py", line 390, in __init__
    errread, errwrite)
  File "C:\Python27\lib\subprocess.py", line 640, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
2016-12-21 11:24:54,721 - ERROR: GN bootstrap command returned non-zero exit code: 1

G:\chromium\ungoogled-chromium>

The older 55.x.x.x tags don't build on Windows. Only the latest one works (55.0.2883.87-1 as of now). The master branch should also work at the time of this comment (but there are no significant changes for Windows between the tag and the branch right now)

I figured, you only had to run "vcvarsall" in the "VS2015 x64 Native Tools Command Prompt"

Actually if you launch one of those command prompts, you don't need to run vcvarsall. The only reason I need to run in this command prompt is because the bootstrap build script for the GN tool won't build without the correct PATH containing the compilers.
However, this shouldn't affect the main build because Google already does the detection of the compilers during the GN build step separately from the currently executing terminal environment.
Did you install Visual Studio to the default location? I installed it to a custom location without spaces in it. Though I doubt this is the cause of the problems.

Its in the default folder. Tried Official chromium, builds fine. Still getting the include issue

Let's sum up: I can not start the build without using the Native Tools CMD Prompt, otherwise it errors
as seen in post3: https://github.com/Eloston/ungoogled-chromium/issues/156#issuecomment-268489583
As to your EDIT, I use "Default" and I checked out/Default/gen/ there is no gen/blink at all
nor is there a build.gn in third_party/WebKit/Source/core/ only has core_event_interfaces_file_list.tmp
Just in case, this time, here is the full output log:
https://paste.ubuntu.com/23667883/

And "vcvarsall" is not existing outside that cmd promt.
I figure you could prob just run it directly by specifying the full path in any cmd.exe process, but .. if you can just open the native tools cmd.. it doesnt matter.

Oh I misunderstood what you meant.  Yeah that's fine.

nor is there a build.gn in third_party/WebKit/Source/core/ only has core_event_interfaces_file_list.tmp

Sees image...
Corrects ambiguity

Bison is supposed to be generating this file according to third_party/WebKit/Source/core/BUILD.gn build/sandbox/third_party/WebKit/Source/core/BUILD.gn in make_core_generated_bison

That file should have this section in it

Sees image...
Corrects ambiguity
heh, ye I pre-written the entire text expecting that folder to not be there, but it eventually got created, but without the file "out/Default/gen/blink/core/XPathGrammar.h" which the image shows you as well.

But Now I'm lost, am I supposed to look for BUILD.gn in build/sandbox/third_party/WebKit/Source/core/
now? In which case, that file does not exist.

Then your build sandbox is broken, because it's still there in 55.0.2883.95

Which is odd because I seriously just followed the guide.. I mean I can't do much wrong beside that..
Considering official chromium compiles just fine without any issues.
And they use almost the same setup. Meaning, my compiler works fine. My python works fine.
Do you think it could be an issue with one of the binaries you have to install? (patch,bison,etc. see below)
Otherwise I really can't think of what I could have done wrong...
2016-12-22 10:14:27,869 - INFO: Checking Python 2 command...
2016-12-22 10:14:27,895 - DEBUG: Using Python version '2.7.6'
2016-12-22 10:14:27,895 - INFO: Checking ninja command...
2016-12-22 10:14:27,903 - DEBUG: Using ninja version '1.7.2'
2016-12-22 10:14:27,903 - INFO: Checking patch command...
2016-12-22 10:14:27,917 - DEBUG: Using patch command 'GNU patch 2.7.5'
2016-12-22 10:14:27,917 - INFO: Checking bison command...
2016-12-22 10:14:27,929 - DEBUG: Using bison command 'bison (GNU Bison) 2.4.1'
2016-12-22 10:14:27,929 - INFO: Checking gperf command...
2016-12-22 10:14:27,938 - DEBUG: Using gperf command 'GNU gperf 3.0.1'

(Going to repost edit so you don't have to keep checking messages you already read)
I'm still having a hard time believing that the file isn't there if the GN step was successful. What else is in that directory?

Think you can see that on the images.
I unfortunately do not have the full output anymore as I tested the official chromium build to see if that fails too. (Which takes a big chunk of my SSD, leaving no space for ungoogled)
But as I previously mentioned, I will re-run everything from scratch and report back again, only this way I can re-assure I did everything according to the guide (which I'm very very sure I did.)
Maybe you can read over the guide once too, see if really everything is correct, but then again no one else reported this issue, or barely anyone compiles this on/for Windows 64..

Think you can see that on the images.

Nope, those files are generated during the build. The path I'm talking about is part of the source code tree, not in the out directory.

I unfortunately do not have the full output anymore as I tested the official chromium build to see if that fails too. (Which takes a big chunk of my SSD, leaving no space for ungoogled)

Ah ok. ~50 GB of files is large indeed.

Maybe you can read over the guide once too, see if really everything is correct, but then again no one else reported this issue, or barely anyone compiles this on/for Windows 64..

nopjmp (who fixed Windows support for version 55) was able to get both 32-bit and 64-bit to build.
I'm starting to think that this could be a file writing/filesystem problem. From experiences I heard and had, I no longer find Windows a reliable OS for storing data. Do you have an antivirus enabled (including Windows Defender)? Do you know of any background service that could be affecting your data? I know of some PCs that come pre-installed with some special software for their SSDs that end up making things worse.
FYI, I run a fresh install of Windows 10 Pro inside a VirtualBox VM and with a lot of things turned off/tweaked. It's possible there's some external factor to your issue here.

Sorry about the closing, ignore that :)
But ye Defender is not installed/disabled, I have nothing running that could interfere, I run Windows 10 Pro too, actually a fresh install as well.

The plot thickens...
Maybe you could comment out the following in build.py so all of the build steps aren't executed:
        builder.setup_build_utilities()
        builder.generate_build_configuration()
        builder.build()
        builder.generate_package()

and see if build/sandbox/third_party/WebKit/Source/core/BUILD.gn is there at the end of it.

Alright, ran through, it has the above file, and the file also contains:
action_foreach("make_core_generated_bison") {
  script = "../build/scripts/rule_bison.py"
  sources = [
    "xml/XPathGrammar.y",
  ]
  outputs = [
    "$blink_core_output_dir/{{source_name_part}}.cpp",
    "$blink_core_output_dir/{{source_name_part}}.h",
  ]
  args = [
    "{{source}}",
    rel_blink_core_gen_dir,
    bison_exe,
  ]
  if (!use_system_xcode) {
    args += [ hermetic_xcode_path ]
  }

  deps = make_core_generated_deps
}

I assume now I should run the rest?
  ```

#builder.setup_build_utilities()
#builder.generate_build_configuration()
#builder.build()
#builder.generate_package()

Sure.

Alright, finished, was kinda hoping to see it work now
Steps I took:
G:\chromium>git clone https://github.com/Eloston/ungoogled-chromium.git
Cloning into 'ungoogled-chromium'...
remote: Counting objects: 2517, done.
remote: Compressing objects: 100% (40/40), done.
remote: Total 2517 (delta 6), reused 0 (delta 0), pack-reused 2476
Receiving objects: 100% (2517/2517), 1.28 MiB | 326.00 KiB/s, done.
Resolving deltas: 100% (1446/1446), done.

G:\chromium>"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64
G:\chromium>cd ungoogled-chromium
G:\chromium\ungoogled-chromium>C:\Python35\python.exe build.py

Error:
[16906/24289] CXX obj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj
FAILED: obj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj
ninja -t msvc -e environment.x64 -- "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64/cl.exe" /nologo /showIncludes /FC @obj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj.rsp /c ../../third_party/WebKit/Source/core/xml/XPathParser.cpp /Foobj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj /Fd"obj/third_party/WebKit/Source/core/xml/xml_cc.pdb"
g:\chromium\ungoogled-chromium\build\sandbox\third_party\webkit\source\core\xml\xpathparser.cpp(31): fatal error C1083: Cannot open include file: 'core/XPathGrammar.h': No such file or directory
[16914/24289] CXX obj/third_party/WebKit/Source/core/xml/xml/DocumentXSLT.obj

EDIT: btw, is there an easy way to re-run this without having to do all the patches again and getting errors about "already patched" etc? So that I can test theories ?
EDIT2: ^above indeed re-runs it just fine, tested whether something in the back interferes, and nope, nothing is now running, and it still produces the same error

Yay you figured out how to use buildlib the way it's intended to. Apparently not many people read through the entire BUILDING document...

EDIT3: btw there is this file, not sure if that helps in any way, it just stood out between all the headers and source files:

As you can see from the GN file I referenced a few times, that is the input file to Bison. Bison produces the header file and a corresponding C++ source file. The fact that the file is there is good, but Bison doesn't seem to be running for some reason.

EDIT4: is there any way we can make the build more verbose?

I'm assuming you want to increase verbosity of Ninja. In that case, pass in -v. You can add it in on this line of common.py

with -v:
Nothing new I that could help me, but maybe you.
[16954/24289] ninja -t msvc -e environment.x64 -- "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64/cl.exe" /nologo /showIncludes /FC @obj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj.rsp /c ../../third_party/WebKit/Source/core/xml/XPathParser.cpp /Foobj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj /Fd"obj/third_party/WebKit/Source/core/xml/xml_cc.pdb"
FAILED: obj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj
ninja -t msvc -e environment.x64 -- "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64/cl.exe" /nologo /showIncludes /FC @obj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj.rsp /c ../../third_party/WebKit/Source/core/xml/XPathParser.cpp /Foobj/third_party/WebKit/Source/core/xml/xml/XPathParser.obj /Fd"obj/third_party/WebKit/Source/core/xml/xml_cc.pdb"
g:\chromium\ungoogled-chromium\build\sandbox\third_party\webkit\source\core\xml\xpathparser.cpp(31): fatal error C1083: Cannot open include file: 'core/XPathGrammar.h': No such file or directory

Is there a way to debug bison / the bison command lines?

You can modify the Python script that they use to run bison. GN can't run executables directly; the only thing it can run are Python scripts.

Nothing new that could help me, but maybe it helps you.

Nope, sorry.

Interesting stuff:
BUILD.gn calls rule_bison.py here using bison_exe
action_foreach("make_core_generated_bison") {
  script = "../build/scripts/rule_bison.py"
  sources = [
    "xml/XPathGrammar.y",
  ]
  outputs = [
    "$blink_core_output_dir/{{source_name_part}}.cpp",
    "$blink_core_output_dir/{{source_name_part}}.h",
  ]
  args = [
    "{{source}}",
    rel_blink_core_gen_dir,
    bison_exe,
  ]
  if (!use_system_xcode) {
    args += [ hermetic_xcode_path ]
  }

  deps = make_core_generated_deps
}

in here we see that it just runs bison
third_party/WebKit/Source/build/script/rule_bison.py
prefix = {'XPathGrammar.y': 'xpathyy'}[inputName]

(inputRoot, inputExt) = os.path.splitext(inputName)

# The generated .h will be in a different location depending on the bison "`bisonExe = sys.argv[3]`"

# version.
outputHTries = [
    os.path.join(outputDir, inputRoot + '.cpp.h'),
    os.path.join(outputDir, inputRoot + '.hpp'),
]

Now the problem here is, where is bisonExe defined... as in the root of it, I can't see any other usage of bisonExe in the GN file

Now the problem here is, where is bisonExe defined... as in the root of it, I can't see any other usage of bisonExe in the GN file

bisonExe is defined at the top of the Python script. GN runs the Python script with a Python subprocess given the arguments as defined in the GN file.

I just checked the folders, rule_bison.py is never called (hence the lack of a pyc file and others having a pyc file, for example "make_style_builder.py" is being called by:
css_properties("make_core_generated_style_builder") {
  script = "../build/scripts/make_style_builder.py"

in the very same BUILD.gn file, do you have any clue what makes it call that but not
action_foreach("make_core_generated_bison") {
  script = "../build/scripts/rule_bison.py"
  sources = [
    "xml/XPathGrammar.y",

in fact make_core_generated_bison
appears in:
target(core_link_small_target_type, "core_generated") {
  sources = bindings_core_v8_files

  # Targets from above that generate outputs that need to be compiled.
  # All sources declared as outputs from these targets will be compiled into this
  # target.
  targets_generating_sources = [
    ":make_core_generated_bison",

and core_generated
appears in:
source_set("generated") {
  deps = [
    ":core_generated",

And thats as far as my guessing goes any other ideas?

More confusingly, make_core_generated_style_builder is in the same targets_generating_sources list as make_core_generated_bison of the core_generated target.
Why make_style_builder.py is invoked but not rule_bison.py makes no sense to me either.
Maybe we should try investigating ninja files. Does rule_bison.py show up in out/Default/toolchain.ninja?

yep, toolchain.ninja has::
rule __third_party_WebKit_Source_core_make_core_generated_bison___build_toolchain_win_x64__rule
  command = C$:/Python27/python.exe ../../third_party/WebKit/Source/build/scripts/rule_bison.py ${in} gen/blink/core bison.exe
  description = ACTION //third_party/WebKit/Source/core:make_core_generated_bison(//build/toolchain/win:x64)
  restat = 1
build obj/third_party/WebKit/Source/core/make_core_generated_bison.inputdeps.stamp: stamp ../../third_party/WebKit/Source/build/scripts/rule_bison.py obj/third_party/WebKit/Source/core/core_event_interfaces.stamp obj/third_party/WebKit/Source/core/generated_settings_macros.stamp obj/third_party/WebKit/Source/core/generated_testing_idls.stamp obj/third_party/WebKit/Source/core/generated_testing_idls_internal_runtime_flags.stamp obj/third_party/WebKit/Source/core/generated_testing_idls_settings.stamp

build gen/blink/core/XPathGrammar.cpp gen/blink/core/XPathGrammar.h: __third_party_WebKit_Source_core_make_core_generated_bison___build_toolchain_win_x64__rule ../../third_party/WebKit/Source/core/xml/XPathGrammar.y | obj/third_party/WebKit/Source/core/make_core_generated_bison.inputdeps.stamp

build obj/third_party/WebKit/Source/core/make_core_generated_bison.stamp: stamp gen/blink/core/XPathGrammar.cpp gen/blink/core/XPathGrammar.h

also:
.ninja.log
221313	221531	0	gen/blink/core/testing/InternalSettingsGenerated.idl	513b877cc043a3cd
221313	221531	0	gen/blink/core/testing/InternalSettingsGenerated.cpp	513b877cc043a3cd
221313	221531	0	gen/blink/core/testing/InternalSettingsGenerated.h	513b877cc043a3cd

After this, it should do xpathgrammar, but, no output

After this, it should do xpathgrammar, but, no output

Maybe the commands are out-of-order. The misordering should be reflected in the ninja files. Even if we figure out how ninja is doing it wrong, I'm not sure how this is going to help us figure out what GN is doing wrong.

EDIT2: I also just compared both BUILD.gn (official LKGR and ungoogled)
Some interesting differences are:

I don't think that's related. If you compare it to what we're working with, it looks like they just moved it out for better formatting/structuring.
@nopjmp Do you have any ideas why this issue is happening?

Have you read my big edit?

Do you mean EDIT2 and Before EDIT2 from here? Then yes. I replied to EDIT2 in the comment before this one.

Nope, big fat bold header # EDIT (be sure to read this first)

Ah. Apparently I needed to refresh the webpage in order to see edits. But if you stay on the page, new comments will show up. Going to add the new info to BUILDING.
@nopjmp Sorry for pinging you. It's all sorted out now.

That's a short reply .. heh.. hoped you'd say a little more.

That's a short reply .. heh.. hoped you'd say a little more.

shrugs
Well here's more then

Fun fact, this is the first time I've ever opened ungoogled chrome, since I've never got it to compile before.. be it due to this issue or the non-existent 64bit support before.

Could've ran the 32-bit version but I guess you like more bits and some more security.

just wonder why it doesn't show this in the build output at all.....

I think bison runs m4 (IIRC from the comment in the Python script), so even if the Python script were to redirect bison's standard I/O to its own, that error would only show up if bison spat out the error.

(And also wonder how to fix this, a work-around for now is using a less... spacy .. location, but that seems stupid.. should be fixed with quotation...)

There's not really much we can do, unless we patch the tools and build it. But what really bugs me is why bison doesn't throw an error if m4 doesn't exit successfuly.
I'm wondering how Google deals with this space issue. Maybe they patched their versions of the utilities? Or somehow used relative paths? Or they don't?

I'm wondering how Google deals with this space issue. Maybe they patched their versions of the utilities? Or somehow used relative paths? Or they don't?
Patches I guess? official source comes with its own bison.

Btw..offtopic issue I now face:

Every single page is grayed out and shows the error icon, even internal ones, any idea? (I can post this in a new issue if you want)

You need to run it on the C: drive because the sandboxing technologies used by Chromium aren't happy with executables running on non C: drives.

wow that sounds very sad (it was in C:/Pr..Files/Chromium-ungoogled/ actually)...
(now i just need ungoogled chromium for android...)

Oh saw your edit. Nevermind then.

(now i just need ungoogled chromium for android...)

Yep, would like this too. Too bad extensions aren't supported yet though.

chrome extensions on android are my dream

I'm just going to chime in some closing thoughts just in case someone else stumbles across this issue.
Spaces cause problems in a lot of ported tools on Windows. I always suggest used the VS2015 x86 Native Tools Command Prompt and VS2015 x64 Native Tools Command Prompt and then changing directories just because it prevents less knowledgeable users from making a mistake.
The grey page can also be caused by moving the ungoogled-chromium directory to a different path which is really dumb and needs to be fixed, but i'll open another issue when I get time (it's a holiday weekend for me).
I was actually going to work on getting Ungoogled Chromium on Android. It's going to take a lot more fiddling, but it's completely possible. CopperheadOS builds Chromium as part of their build process.

The grey page can also be caused by moving the ungoogled-chromium directory to a different path which is really dumb and needs to be fixed, but i'll open another issue when I get time (it's a holiday weekend for me).

Meaning its fixable from your side and not chromium?
(By patching chromium itself rather than making the chromium team fix it?)

If you extra that zip again then move it before running it works just fine. I think it's saving the sandbox binary location somewhere. I can upstream it later after its fixed. I'm a hobbyist c++ programmer so I should be able to fix it.
Sent from my android device.
-----Original Message-----
From: DeadSix27<EMAIL_ADDRESS>To: Eloston/ungoogled-chromium<EMAIL_ADDRESS>Cc: nopjmp<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Sent: Sun, 25 Dec 2016 1:11 PM
Subject: Re: [Eloston/ungoogled-chromium] Issue including XpathGrammar.h (#156)

The grey page can also be caused by moving the ungoogled-chromium directory to a different path which is really dumb and needs to be fixed, but i'll open another issue when I get time (it's a holiday weekend for me).

Meaning its fixable from your side and not chromium?
(By patching chromium itself rather than making the chromium team fix it?)
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/Eloston/ungoogled-chromium/issues/156#issuecomment-269134068

Well, so am I, altho all other languages more than c++, but does it really have to do where it saves the zip, does the entire build process remember "i saved my zip to G:\test""lets fail everywhere else except G:"
But ye I need more information to fully understand this, since if it saves the sandbox binary somewhere, shouldn't all we have to do be, moving said binary as well?

I'm just mentioning the behavior I noticed I haven't looked into it too much since made the pull request. I will look into it later. It takes way too long for me to build chromium on my machine. I'm waiting for my new machine to come in.
Sent from my android device.
-----Original Message-----
From: DeadSix27<EMAIL_ADDRESS>To: Eloston/ungoogled-chromium<EMAIL_ADDRESS>Cc: nopjmp<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Sent: Sun, 25 Dec 2016 1:48 PM
Subject: Re: [Eloston/ungoogled-chromium] Issue including XpathGrammar.h (#156)
Well, so am I, altho all other languages more than c++, but does it really have to do where it saves the zip, does the entire build process remember "i saved my zip to G:\test""lets fail everywhere else except G:"
But ye I need more information to fully understand this, since if it saves the sandbox binary somewhere, shouldn't all we have to do be, moving said binary as well?
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/Eloston/ungoogled-chromium/issues/156#issuecomment-269135157

No problem, was just curious (ye even on my 6700k it takes a long while, def needs more than 8 threads, more like 24 :p)
Be sure to quote me or reply here or so when you get more Information.

What "sandbox binary" are we talking about here? And zip file? I thought Chromium just made system API calls to get sandboxing to work

I think it's the sandbox process but it could be something else. I'm going to make a new issue or just submit a pull request to fix it. Most likely it only is a problem on Windows since macOS and Linux use different methods. macOS stores everything in the app and Linux uses a wrapper script.
Sent from my android device.
-----Original Message-----
From: Eloston<EMAIL_ADDRESS>To: Eloston/ungoogled-chromium<EMAIL_ADDRESS>Cc: nopjmp<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Sent: Sun, 25 Dec 2016 2:00 PM
Subject: Re: [Eloston/ungoogled-chromium] Issue including XpathGrammar.h (#156)
What "sandbox binary" are we talking about here? And zip file? I thought Chromium just made system API calls to get sandboxing to work
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/Eloston/ungoogled-chromium/issues/156#issuecomment-269135500

By the way, it does still work just fine, no matter which drive I put it on, after you mentioned it, I just took it for granted (that it won't work) but it does.

Huh, I remember that it wouldn't run on my secondary HDD attached to the VM, but it would run just fine on the C: drive. Maybe they changed it now?

They probably fixed it upstream. Windows filesystem can be crazy. UTF16 file names if I remember correctly.
Sent from my android device.
-----Original Message-----
From: Eloston<EMAIL_ADDRESS>To: Eloston/ungoogled-chromium<EMAIL_ADDRESS>Cc: nopjmp<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Sent: Sun, 25 Dec 2016 2:10 PM
Subject: Re: [Eloston/ungoogled-chromium] Issue including XpathGrammar.h (#156)
Huh, I remember that it wouldn't run on my secondary HDD attached to the VM, but it would run just fine on the C: drive. Maybe they changed it now?
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/Eloston/ungoogled-chromium/issues/156#issuecomment-269135781

I created a Gitter room. We should discuss more there instead of taking this issue off-track.


google fit rest api redirection url is returning FetchError: request to https://oauth2.googleapis.com/token failed, reason: connect ECONNREFUSED

I am trying to build a project in which I will fetch the user's step count by using the google fit Rest API. For this, I have created a project on google's developer console and specified a redirection url there. Have a look to the code snippet below :
exports.getUrl = async (req, res, next) => {
    try {
        const oauth2Client = new google.auth.OAuth2(
            process.env.GOOGLE_FIT_CLIENT_ID,
            process.env.GOOGLE_FIT_CLIENT_SECRET,
            process.env.GOOGLE_FIT_REDIRECTION_URL
        );
        console.log("oauth2Client", oauth2Client)
        // generate a url that asks permissions for fitness activity scopes
        const scopes = ["https://www.googleapis.com/auth/fitness.activity.read profile email openid"];

        const url = oauth2Client.generateAuthUrl({
            access_type: "offline",
            scope: scopes,
            include_granted_scopes: true,
            state: JSON.stringify({
                // callbackurl: req.body.callbackurl,
            })
        });
        console.log("url", url);
        var options = {
            url: url,
            proxy: 'http://proxy-url:port'
        }
        request(options, (err, response, body) => {
            if (response) {
                console.log("statuscode ", response && response.statusCode);
                //res.status(200).send(url);
                res.redirect(url);
            } else {
                console.log("err", err)
                res.status(500).send(err);
            }
        });
    } catch (err) {
        console.log("err", err)
        next(err);
    }
}

exports.getSteps = async (req, res, next) => {
    try {
        const queryUrl = new urlParse(req.url);
        const code = queryParse.parse(queryUrl.query).code;
        const oauth2Client = new google.auth.OAuth2(
            process.env.GOOGLE_FIT_CLIENT_ID,
            process.env.GOOGLE_FIT_CLIENT_SECRET,
            process.env.GOOGLE_FIT_REDIRECTION_URL
        );

        const token = await oauth2Client.getToken(code);
        oauth2Client.setCredentials(token);
        const result = await axios({
            proxy: {
                protocol: 'http',
                host: 'proxy-url',
                port: port
            },
            method: "POST",
            headers: {
                authorization: "Bearer " + token.tokens.access_token
            },
            "Content-Type": "application/json",
            url: "https://www.googleapis.com/fitness/v1/users/me/dataset:aggregate",
            data: {
                "aggregateBy": [{
                    "dataTypeName": "com.google.step_count.delta",
                    "dataSourceId": "derived:com.google.step_count.delta:com.google.android.gms:estimated_steps"
                }],
                "bucketByTime": { "durationMillis": 86400000 }, // This is 24 hours
                "startTimeMillis": startTime, // This startTime and endTime I am getting from DB
                "endTimeMillis": endTime
            }
        });

        if (result) {
            const response = [];
            let stepArray = result?.data?.bucket;
            for (const dataSet of stepArray) {
                for (const points of dataSet.dataset) {
                    for (const steps of points.point) {
                        response.push(steps?.value[0]?.intVal);
                    }
                }
            }
            res.status(200).send(response);
        } else {
            throw new Error('Data fetching failed!');
        }
    } catch (err) {
        next(err);
    }
}

The steps url is what I have mentioned as a redirection url on the google's developer console. I have used proxy because the urls which are getting called are not whitelisted on the server on which I am deploying the code.
Now, everything worked perfectly fine in localhost but on server, I am getting below error :
FetchError: request to https://oauth2.googleapis.com/token failed, reason: connect ECONNREFUSED <IP_ADDRESS>:443     at ClientRequest.<anonymous> (/home/node/app/node_modules/node-fetch/lib/index.js:1491:11)     at ClientRequest.emit (node:events:394:28)     at TLSSocket.socketErrorListener (node:_http_client:447:9)     at TLSSocket.emit (node:events:394:28)     at emitErrorNT (node:internal/streams/destroy:157:8)     at emitErrorCloseNT (node:internal/streams/destroy:122:3)     at processTicksAndRejections (node:internal/process/task_queues:83:21)
This error in coming in the steps api call. I have tried to set the proxy options like below but still same error.
google.options({     proxy: 'http://proxy-url:port' })
I have tried to use the https-proxy-agent and http-proxy-agent too but no outcome.
Also, one thing to note here is that the above error is coming when I am trying to get the data from chrome with --disable-web-security flag otherwise the first the route which is url itself is not working and throwing timeout error.
Please let me know if I am doing something wrong. Any response would be highly appreciated.

Hi @atulkrsngh is this issue still ongoing? Have you tried setting the environment variables http_proxy and/or https_proxy? If so, were the results the same?

Going to close this out. I would recommend trying the suggestion above and if this still does not work, please feel free to reopen.


[Question] How to suppress and/or increase the 15 seconds timeout message while posting to our messages endpoint?

Title says everything...
Thank you

@McBacker For reference, are you using .NET or Node SDK?
What problems are you experiencing? What is the context that you need to increase the timeout duration? To what specific endpoint?

.Net SDK
When a timeout occurs, I want to present to my Bot users a better message than "POST to XXX timed out after 15s".
Thank you

@McBacker What specific channels are you using with your bot? When does this error occur?

It happens when my on premise hosted messages endpoint is not reachable/running properly.
The channel doesn't matter on this scenario.

@nwhitmont, do you have any update on this?
Thank you

Anyone?

Anyone?

Is it possible to disable this messages?

Looks like not :-(

The timeout is from the connector services, and cannot be changed.  The message also comes from the connector services.
Bots are expected to respond with an acknowledgement (status 200) within 15 seconds.  Something that can be done on channels that allow proactive messages is to process the incoming message on a separate thread, and acknowledge the call immediately.
in .net:
public async Task<HttpResponseMessage> Post([FromBody]Activity activity)
 {
            if (activity.Type == ActivityTypes.Message)
            {
                    Task.Factory.StartNew(() => Conversation.SendAsync(activity, () => new Dialogs.RootDialog())); // process the message asyncronously
            }

            return Request.CreateResponse(HttpStatusCode.OK); //ack the call
}

Wouldn't it finish my Azure Function if I immediately return from this method?
I tried it and the Azure Function stopped running, and the real response to the chat never came. It's not a solution at all for the Function Bots

I ended up with this code for my Function Bot, which resolves the problem I just mentioned.
public static class Functions
{
    [FunctionName("messages")]
    [return: Queue("somequeue")]
    public static async Task<MessagePayload> Messages([HttpTrigger(WebHookType = "genericJson")]
        HttpRequestMessage req) =>
        // return from this Azure Function immediately to avoid timeout warning message
        // in the chat.
        // just put the request into "somequeue".
        // We can't pass the whole request via the Queue, so pass only what we need for
        // the message to be processed by Bot Framework
        new MessagePayload
        {
            Content = await req.Content.ReadAsStringAsync(),
            AuthScheme = req.Headers.Authorization.Scheme,
            AuthParameter = req.Headers.Authorization.Parameter
        };

    // Do the actual message processing in another Azure Function, which is
    // triggered by a message enqueued in the Azure Queue "somequeue"
    [FunctionName("processTheMessage")]
    public static async Task ProcessTheMessage([QueueTrigger("somequeue")]
        MessagePayload payload, TraceWriter logger)
    {
        // we don't want the queue to process this message 5 times if it fails,
        // so we won't throw any exceptions here at all, but we'll handle them properly.
        try
        {
            // recreate the request
            var request = new HttpRequestMessage
            {
                Content = new StringContent(payload.Content)
            };
            request.Headers.Authorization = new AuthenticationHeaderValue(payload.AuthScheme,
                payload.AuthParameter);

            // initialize dependency injection container, services, etc.
            var initializer = new SomeInitializer(logger);
            initializer.Initialize();

            // handle the request in a usual way and reply back to the chat
            await initializer.HandleRequestAsync(request);
        }
        catch (Exception ex)
        {
            try
            {
                // TODO: handle the exception
            }
            catch (Exception anotherException)
            {
                // swallow any exceptions in the exceptions handler?
            }
        }
    }

}

[Serializable]
public class MessagePayload
{
    public string Content { get; set; }
    public string AuthParameter { get; set; }
    public string AuthScheme { get; set; }
}

For those who run into this problem and built their bot using Node. A simple solution was to use async/await without the await. So it looks like this:
function async handleActivity(activity) {
   if(activity.attachments)
      doSomeLongRunningAsyncTask(activity);
   respond();
}

function async doSomeLongRunningAsyncTask(activity) {
   // do your long running task call here and send messages after it completes
}

@watadarkstar As long as there is no concern for state (no need to read or edit state, no use of dialogs or dialog stack), and you are simply sending a proactive message to the conversation, your method seems valid to me.  The documentation here: https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-howto-proactive-message?view=azure-bot-service-4.0&tabs=javascript#logic-to-start-a-job is still using TurnContext.getConversationReference (I cannot speak to getConversationReference moving to Activity.  I'm not sure what the plan is there. Even if it does move at some point in botbuilder-js, the existing method would still remain at least until a major version bump.)

@EricDahlvang Could you elaborate on "As long as there is no concern for state (no need to read or edit state, no use of dialogs or dialog stack)"
Specifically on the state part. As we do update our state in our database. Is that the kind of state you're talking about? And if so, why might this be problematic?

Thanks for the quick response @EricDahlvang! :) I think I know what you mean....The async nature of things could lead to odd race cases if both touching state.

return Request.CreateResponse(HttpStatusCode.OK);

what about this approach to keep task awaitable:
var timeout = 14000;
var task = Conversation.SendAsync(activity, () => new Dialogs.RootDialog());
await Task.WhenAny(task, Task.Delay(timeout));
return Request.CreateResponse(HttpStatusCode.OK);

@EricDahlvang @watadarkstar
I do not think sending the reply messages as proactive message for each incoming message is a good idea because your bot service has to then load the state from the storage twice (one to form the first turnContext in processActivity and second to form turnContext in adapter.continueConversation).
I tried below changes in my code to reduce API response time.
server.post(msgEndpointPath, (req, res, next) => {
    adapter.processActivity(req, res, async (turnContext) => {
        res.send(200);
        next();
        await bot.run(turnContext);
    });
});


Runner hangs with concurrency option set and failing browsers

Env: Windows 7 Pro 64bit, runner run via grunt-karma
When concurrency is set (to 2 in this example), and all browsers in the "last concurrency batch" fail to connect even after retries, the runner hangs.
This is the log:
D:\Herby\Work\gaff\gvp>grunt test
Running "testingbot-tunnel:tunnel" (testingbot-tunnel) task

Running "karma:testingbot" (karma) task
05 11 2015 12:16:48.286:INFO [karma]: Karma v0.13.15 server started at http://localhost:9000/
05 11 2015 12:16:48.304:INFO [launcher]: Starting browser firefox via Remote WebDriver
05 11 2015 12:16:48.337:INFO [launcher]: Starting browser googlechrome via Remote WebDriver
05 11 2015 12:16:56.603:INFO [Chrome 46.0.2490 (Windows 8.1 0.0.0)]: Connected on socket QVGhPwWhK4XA_y-cAAAA with id 7267888
05 11 2015 12:16:57.235:INFO [Firefox 38.0.0 (Windows 7 0.0.0)]: Connected on socket fRKIzbY1EZ-lFbMYAAAB with id 87610919
Firefox 38.0.0 (Windows 7 0.0.0): Executed 85 of 85 SUCCESS (6.966 secs / 5.569 secs)
Chrome 46.0.2490 (Windows 8.1 0.0.0): Executed 85 of 85 SUCCESS (7.252 secs / 6.244 secs)
05 11 2015 12:17:06.372:INFO [WebDriver]: Killed Karma test.
05 11 2015 12:17:06.379:INFO [launcher]: Starting browser microsoftedge via Remote WebDriver
05 11 2015 12:17:08.640:INFO [WebDriver]: Killed Karma test.
05 11 2015 12:17:08.648:INFO [launcher]: Starting browser iexplore via Remote WebDriver
05 11 2015 12:18:06.383:WARN [launcher]: microsoftedge via Remote WebDriver have not captured in 60000 ms, killing.
05 11 2015 12:18:08.486:INFO [WebDriver]: Killed Karma test.
05 11 2015 12:18:08.502:INFO [launcher]: Trying to start microsoftedge via Remote WebDriver again (1/2).
05 11 2015 12:18:08.676:WARN [launcher]: iexplore via Remote WebDriver have not captured in 60000 ms, killing.
05 11 2015 12:18:10.457:INFO [WebDriver]: Killed Karma test.
05 11 2015 12:18:10.463:INFO [launcher]: Trying to start iexplore via Remote WebDriver again (1/2).
05 11 2015 12:19:08.518:WARN [launcher]: microsoftedge via Remote WebDriver have not captured in 60000 ms, killing.
05 11 2015 12:19:10.485:WARN [launcher]: iexplore via Remote WebDriver have not captured in 60000 ms, killing.
05 11 2015 12:19:10.580:INFO [WebDriver]: Killed Karma test.
05 11 2015 12:19:10.588:INFO [launcher]: Trying to start microsoftedge via Remote WebDriver again (2/2).
05 11 2015 12:19:12.174:INFO [WebDriver]: Killed Karma test.
05 11 2015 12:19:12.177:INFO [launcher]: Trying to start iexplore via Remote WebDriver again (2/2).
05 11 2015 12:20:10.591:WARN [launcher]: microsoftedge via Remote WebDriver have not captured in 60000 ms, killing.
05 11 2015 12:20:12.186:WARN [launcher]: iexplore via Remote WebDriver have not captured in 60000 ms, killing.
05 11 2015 12:20:12.513:INFO [WebDriver]: Killed Karma test.
05 11 2015 12:20:12.517:ERROR [launcher]: microsoftedge via Remote WebDriver failed 2 times (timeout). Giving up.
05 11 2015 12:20:13.872:INFO [WebDriver]: Killed Karma test.
05 11 2015 12:20:13.876:ERROR [launcher]: iexplore via Remote WebDriver failed 2 times (timeout). Giving up.

After this, the runner is simply stuck (waited until 12:23 and Ctrl-C'd it). BTW, Edge and IE are stuck b/c of #1606.
When I use karma-testingbot-launcher instead of karma-webdriver-launcher, the result is the same.

I've the same issue, when setting concurency to 1, my tests will hang forever.
tried with browserstack launcher.

Same issue as @bvarga, browser launchs but stuck there. Instead with singleRun to false it works


Content search reference page - feedback column on right hand side

On this page the feedback column on the right hand side flows over the content in the middle column and you cant close the feedback tab to see the text. Even zooming out on the page just makes the right hand column wider.
Happens in Chrome and in Edge.

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e83b77e5-e3ec-6a5f-1f09-e232e2dda483
Version Independent ID: 0b263c72-95a2-d111-20a7-68857f288369
Content: Keyword queries and search conditions for Content Search - Microsoft 365 Compliance
Content Source: microsoft-365/compliance/keyword-queries-and-search-conditions.md
Service: o365-seccomp
GitHub Login: @markjjo
Microsoft Alias: markjjo

@chloegilbert Thanks for feedback, but not sure I understand, and can't seem to repro. Some of the tables in this topic have horizontal scroll bars, which is a result of column-width do to non-breaking lines. But not sure if that is what you're referring to. Maybe you could attach a screenshot, but I may not be about to do much if its related to the browser chrome.

@markjjo  Thank you very much for digging deep into it.  @chloegilbert We appreciate any feedback that improves the content in the Microsoft docs but we haven't had any response from you so we are going to close this issue.
Please feel free to re-open this issue if you see an improvement is required in the specific area of this documentation. We will certainly update this documentation with that valuable information. Thanks again, for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


Gradle 7 support

Hi,
Currently when running with Gradle 6 there is a deprecation warning that will turn into an error with Gradle 7 :
Internal API constructor DefaultPolymorphicDomainObjectContainer(Class<T>, Instantiator) has been deprecated. This is scheduled to be removed in Gradle 7.0. Please use ObjectFactory.polymorphicDomainObjectContainer(Class<T>) instead. See https://docs.gradle.org/6.7.1/userguide/custom_gradle_types.html#extensiblepolymorphicdomainobjectcontainer for more details.

I'm aware that this change would make the plugin compatible only with Gradle 6.1+. IMHO users with older Gradle version can still use the current version until they can migrate to Gradle 6.1 or newer.
Gradle 7 release is probably not far and it would be nice to support it on day one.

My bad this has already been fixed.

My bad this has already been fixed.


No way to look up or create Event Hubs via an extension on VS Code

Currently, there are extensions for IOT Hub, Event Grid, and Service Bus, and a third party extension for Event Hub.  There isn't a first party extension for Event Hub and so the flow of working in VSCode is broken a bit by this.
This is okay as I've talked to Matt Hernandez and the point of VSC isn't to remove the portal but to draw people in with popular/frequently used service extensions.  However, we're going to be doing some research to see how many people who actively use Event Hub in conjunction with Functions and VSCode to see if there would be a market for it.
Viewing Events in eventhub is difficult enough as it is, this would significantly improve the quality of a user that doesn't have to look for the events being sent in portal when all they have to do is look at it on VSC.

This is out of scope for an issue on the Functions repo. Creating/owning an entirely new extension is a lot of work and should be discussed offline, potentially with multiple teams to see who wants to own it.

This is out of scope for an issue on the Functions repo. Creating/owning an entirely new extension is a lot of work and should be discussed offline, potentially with multiple teams to see who wants to own it.


[Question] Can Factory Controls Workers Through State?

Hello, author!
First of all, thank you so much for providing us with such an excellent actor framework like Ractor. It has been incredibly helpful in our development process. Recently, I encountered a design scenario where I need to create a worker that sends API requests, implemented as a standalone actor. The functionality I’m aiming for is to allow this worker to continuously send concurrent requests. However, when it encounters a 429 Too Many Requests response from the server, the actor should pause its operations for a few seconds before resuming.
Here’s my initial approach: I implemented this within a single actor, using Tokio to spawn multiple threads while maintaining the actor's state. If a 429 response is received, the actor automatically transitions to a pending state to pause further requests.
The official documentation, however, recommends using a factory. I’m a bit unclear on how to manage multiple workers through a factory, especially when it comes to state management. Specifically, I’m wondering if a factory can maintain a shared state. For example, if multiple workers encounter a 429 response, could the factory facilitate pausing all workers simultaneously for a certain period of time?
Looking forward to your guidance and suggestions!

Hmmm I think this might be beyond the functionality of a factory and you'll likely want to hand build your own use case for this. However factories in ractor can definitely provide guidance for how to manage a child's lifecycle, etc
The only thing I could think of is if you write your own queue implementation and like pause releasing new items from the queue but that might break some assumptions on how a queue is used, since the queue wouldn't be empty etc.
I think you're probably best going your own way on this since the parent isn't just managing the lifecycle and message routing but has some higher level state involvement that we don't currently have the flexibility for.
P.s. thanks for using the crate and the feedback!

Hmmm I think this might be beyond the functionality of a factory and you'll likely want to hand build your own use case for this. However factories in ractor can definitely provide guidance for how to manage a child's lifecycle, etc
The only thing I could think of is if you write your own queue implementation and like pause releasing new items from the queue but that might break some assumptions on how a queue is used, since the queue wouldn't be empty etc.
I think you're probably best going your own way on this since the parent isn't just managing the lifecycle and message routing but has some higher level state involvement that we don't currently have the flexibility for.
P.s. thanks for using the crate and the feedback!

thanks for your response. but if I use tokio::spawn within an actor handle, it should work, right? Does it align with the ractor design philosophy?

I mean spawning inside an actor is fine as long as you manually wire up the failure flows. If a spawned task panicked it failed, is it handled and how does the actor respond?
Spawning child actors is probably how I'd expect it to be done so the supervision of the child tasks is automatic but that's up to you

I mean spawning inside an actor is fine as long as you manually wire up the failure flows. If a spawned task panicked it failed, is it handled and how does the actor respond?
Spawning child actors is probably how I'd expect it to be done so the supervision of the child tasks is automatic but that's up to you

Got it. BTW, in ractor, besides the actor name manually specified by the user, is there a unique actor_id generated automatically? If so, how can it be retrieved?

Yes and there's an accessor on the ActorRef.
https://github.com/slawlor/ractor/blob/main/ractor/src/actor/actor_cell.rs#L294


does the js lib support m1?

Hi, I enjoy taplo for a while and recently I switched to macbook pro m1 and found it stopped working with<EMAIL_ADDRESS>do you have an idea on this?

I'm using coc-toml btw.

Hi, unfortunately I have no access to macs, so I cannot investigate this one.
You could try to see if the cli works via either cargo install taplo-cli && taplo help and npx @taplo/cli help.

@tamasfe

Thanks, seems like a node issue on the arm macs.

@tamasfe tested works using<EMAIL_ADDRESS>I'll wait the next LTS before switching to it, thanks!