gRPC with Vault - why does server initiates a CSR with client CN?

Hello and first of all thank you for the great project, it's definitely saving me a LOT of time when dealing with certificates with grpc!
I have a general question regarding how the TLS handshake behaves with certify: why does the grpc Server is requesting a new certificate with client common name as Subject Alternative Name?
server1_1      | time="2020-02-08T23:59:56Z" level=error msg="Error getting server certificate" error="Error making API request.\n\nURL: PUT http://vault:8200/v1/pki/sign/admin\nCode: 400. Errors:\n\n* subject alternate name agent01.example.com not allowed by this role"

Of course, that can be easily fixed by changing my admin role and allowing that domain. But I am not sure to get why does the client CN is getting propagated to the server CSR SAN?
Is it because of tls.RequireAndVerifyClientCert and a way of doing mutual authentication?
My piece of code, for the client and server:

	renewableToken := &certifyVault.RenewingToken{
		Initial:     key.VaultToken,
	}

	issuer := &certifyVault.Issuer{
		URL: &url.URL{
			Scheme: vaultURL.Scheme,
			Host:   vaultURL.Host,
		},
		Role:       "agent01",
		AuthMethod: renewableToken,
		TLSConfig: &tls.Config{RootCAs: client.CaCertPool, ServerName: "agent01.example.com"},
	}

	client.Certify = &certify.Certify{
		CommonName:  "agent01.example.com",
		Issuer:      issuer,
		Cache:       certify.NewMemCache(),
		Logger: logrusadapter.New(log.New()),
	}

	tlsConfig := &tls.Config{
		GetClientCertificate: client.Certify.GetClientCertificate,
		RootCAs: client.CaCertPool,
		ServerName: client.Host,
	}

	conn, err := grpc.Dial(serverAddr, grpc.WithTransportCredentials(credentials.NewTLS(tlsConfig)))

Ands server:
	renewableToken := &certifyVault.RenewingToken{
		Initial: s.VaultClient.Token(),
	}

	issuer := &certifyVault.Issuer{
		URL: &url.URL{
			Scheme: vaultURL.Scheme,
			Host:   vaultURL.Host,
		},
		TimeToLive: time.Hour,
		Role:  "admin",
		AuthMethod: renewableToken,
		TLSConfig: &tls.Config{RootCAs: s.CaCertPool, ServerName: "admin.example.com"},
	}

	s.Certify = &certify.Certify{
		CommonName: "admin.example.com",
		Issuer:      issuer,
		Cache:       certify.NewMemCache(),
		Logger: logrusadapter.New(log.New()),

	listener, _ := net.Listen("tcp", s.addr)

	tlsConfig := &tls.Config{
		GetCertificate: s.Certify.GetCertificate,
		RootCAs: s.CaCertPool,
		ServerName: "admin.example.com",
		ClientCAs: s.CaCertPool,
		ClientAuth:     tls.RequireAndVerifyClientCert,
	}

	gs := grpc.NewServer(grpc.Creds(credentials.NewTLS(tlsConfig)))
	gs.Serve(listener)

Thanks a lot!

Hi Mathieu, thanks for your interest in the project!
That doesn't sound right at all, the server should be issuing a CSR with CN admin.example.com, according to your configuration. I noticed that you've configured the ServerName in your vault issuer TLSConfig, maybe this is what is causing the confusion? That field normally shouldn't be set, it should just use whatever the DNS is used for the Vault server.
Otherwise it looks fine, so if the problem persists, could you try to produce a minimal working example, or better yet, add a failing test to the tests?
Thanks!

Hi Johan, thank you very much for your reply. Indeed it was a DNS issue:
tlsConfig := &tls.Config{
		GetClientCertificate: client.Certify.GetClientCertificate,
		RootCAs: client.CaCertPool,
		ServerName: client.Host,
	}

My server name was wrong in the client, it should be the ServerName not Client. This is why the tls.ClientHelloInfo.ServerName was wrong on the server side as it was trying to issue the bad certificate. Just removing that parameter falls back to the actual resolved domain name.
If anyone else has the same issue, make sure all hosts are properly resolved too. In docker-compose it's pretty easy:
      networks:
        local:
          aliases:
            - vault.example.com

Thank you!


bundle the sqlite source code

It's possible to bundle the sqlite source code to simplify building with zig-sqlite.

Hello ! Thanks for this wrapper !
Wondered if this could allow some kind of "seamless" static builds on Zig's supported outputs ? And making zig-sqlite works on Windows too ? :)

Hello ! Thanks for this wrapper !
Wondered if this could allow some kind of "seamless" static builds on Zig's supported outputs ? And making zig-sqlite works on Windows too ? :)

Yes it should.
I haven't tested on a real Windows machine yet, but with the linked PR and building like this:
$ zig build test -Duse_bundled=true -Dtarget=native-windows-gnu

it builds correctly, however the test binary crashes in wine.
I'll have to do more testing.

Yes it should.
I haven't tested on a real Windows machine yet, but with the linked PR and building like this:
$ zig build test -Duse_bundled=true -Dtarget=native-windows-gnu

it builds correctly, however the test binary crashes in wine.
I'll have to do more testing.

Shall I open another issue for Windows support, linking to this one as a first step ?
I can help testing the test from time to time to my Windows machine, if it help.

Shall I open another issue for Windows support, linking to this one as a first step ?
I can help testing the test from time to time to my Windows machine, if it help.

Sure, and yes any testing is welcome.

Sure, and yes any testing is welcome.


Inconsistent input and disparity in Kitti2015Stereo (and others)

🐛 Describe the bug
In the official archive for Kitti2015Stereo, there are twice as input images for each camera than there are disparities.
Matching inputs and disparities now seems to depend solely on sorting the file names in respective directories which is not enough and inconsistent input image and disparities are served as a result.
The dataset also reports size 400 but iteration ends after 200 examples due to missing disparities.
A workaround is to get rid of the extra images in extracted archive. This would be good to add into docs.
A better solution would be to use the file names to get matching counterparts, e.g., to use the file names intersection from the relevant directories as examples to serve.
It affects any stereo dataset if sorting the files is not enough.
How to check/reproduce:
import matplotlib.pyplot as plt
import torch
import torchvision
import torchvision.transforms.functional as F

# Using https://s3.eu-central-1.amazonaws.com/avg-kitti/data_scene_flow.zip
dataset = torchvision.datasets.Kitti2015Stereo('data')
print(dataset)

for i, (left, right, disparity, _) in enumerate(dataset):
    print(f'{i} / {len(dataset)}')
    left = F.to_tensor(left).mean(dim=0, keepdim=True)
    right = F.to_tensor(right).mean(dim=0, keepdim=True)
    disparity = torch.as_tensor(disparity)
    grid = torchvision.utils.make_grid([left, right, disparity])
    plt.imshow(grid.numpy().transpose((1, 2, 0)))

Versions
Collecting environment information...
PyTorch version: 1.13.0+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.35
Python version: 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-58-generic-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Versions of relevant libraries:
[pip3] numpy==1.23.4
[pip3] torch==1.13.0
[pip3] torchvision==0.14.0
[conda] Could not collect
cc @pmeier

Can confirm that this is happening for split="train". For split="test" however this is not an issue, since we don't depend on the disparities there:
https://github.com/pytorch/vision/blob/dabb6d526174dbb04490af35c47ee75f339f578a/torchvision/datasets/_stereo_matching.py#L341-L346
We definitely need to fix this for split="train" though. Naming scheme regex for the images is \d{6}_1[0,1].png. For example, we have 000194_10.png and 000194_11.png. For the disparities it is \d{6}_10.png. So we can get away by just injecting _10 the file name globs:
https://github.com/pytorch/vision/blob/dabb6d526174dbb04490af35c47ee75f339f578a/torchvision/datasets/_stereo_matching.py#L337-L343
Doing that will effectively remove the *_11.png images and thus eliminate the disconnect between the images and disparities.
However, we also need to decide what we do for split="test". Do we continue to return the *_11.png images or do we leave them out there as well? Looking at them, it seems both images are either taken from different cameras or at different times. In any case, they both show basically the same scene with slight differences. @tpet Any insights if it would be useful to keep both images for testing?

I guess, the pairs are to be used in optical flow, so I would purge them out for stereo dataset.


doesn't display groups

On SharePoint 2010, from the main screen, selecting Groups, it never gets past the fetching groups; however stepping through the code, it gets the groups and processes them just never displays them.  Selecting Users works just fine.

so - guess I didn't wait long enough for the groups to load - but it takes significantly longer to load than edit users which was slow already.

How slow? I've had this deployed to various site collections with low
response times. Check network tab in dev tools and see if that's bottleneck
On Oct 5, 2016 9:49 PM, "vinny75"<EMAIL_ADDRESS>wrote:

so - guess I didn't wait long enough for the groups to load - but it takes
significantly longer to load than edit users which was slow already.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dhardin/SP-PermissionManagement/issues/11#issuecomment-251847648,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEsphdw4ZXbfJ3SnNxMb9qEI3QBEEGVfks5qxFOogaJpZM4KPc_r
.

After loading the initial page, then clicking Groups, this is the only network traffic I see:
URL	Protocol	Method	Result	Type	Received	Taken	Initiator	Wait	Start	Request	Response	Cache read	Gap
/_vti_bin/UserGroup.asmx	HTTPS	POST	200	text/xml	3.23 MB	15.61 s	XMLHttpRequest	0	0	15	1685	13916	-750552681
/_vti_bin/UserGroup.asmx	HTTPS	POST	200	text/xml	79.32 KB	421 ms	XMLHttpRequest	0	0	15	234	172	15195
however, it's taking just shy of 3 minutes to display

How many groups and users does your site collection have?

660 groups; 15008 users
I used SPServices to get those numbers, didn't take long
`  $().SPServices({
operation: "GetUserCollectionFromSite",
async: false,
completefunc: function(xData, Status) {
console.log($(xData.responseXML).find("User").length)
}
});
$().SPServices({
operation: "GetGroupCollectionFromSite",
async: false,
completefunc: function(xData, Status) {
console.log($(xData.responseXML).find("Group").length)
}
});
`

Alrighty, looks like the web service is one bottleneck because it's taking
around 15 seconds to retrieve but there is a possibility that there are
zombie views that are causing the rendering to take so long. 660 groups is
not too many for a single threaded environment and should be processed
rather quickly.  I'll see if I can take a look at it today.
On Oct 7, 2016 7:52 AM, "vinny75"<EMAIL_ADDRESS>wrote:

660 groups; 15008 users
I used SPServices to get those numbers, didn't take long
` $().SPServices({
operation: "GetUserCollectionFromSite",
async: false,
completefunc: function(xData, Status) {
console.log($(xData.responseXML).find("User").length)
}
});
$().SPServices({
operation: "GetGroupCollectionFromSite",
async: false,
completefunc: function(xData, Status) {
console.log($(xData.responseXML).find("Group").length)
}
});
`
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dhardin/SP-PermissionManagement/issues/11#issuecomment-252233303,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEspheIWkjkX8RXrXKgcjYfB-36foxDVks5qxjKGgaJpZM4KPc_r
.

Also there is the possibility that a group or user is corrupted. I've seen
that affect load times in the past and deleting/recreating the user/group
resolved the issue.
On Oct 7, 2016 7:52 AM, "vinny75"<EMAIL_ADDRESS>wrote:

660 groups; 15008 users
I used SPServices to get those numbers, didn't take long
` $().SPServices({
operation: "GetUserCollectionFromSite",
async: false,
completefunc: function(xData, Status) {
console.log($(xData.responseXML).find("User").length)
}
});
$().SPServices({
operation: "GetGroupCollectionFromSite",
async: false,
completefunc: function(xData, Status) {
console.log($(xData.responseXML).find("Group").length)
}
});
`
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dhardin/SP-PermissionManagement/issues/11#issuecomment-252233303,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEspheIWkjkX8RXrXKgcjYfB-36foxDVks5qxjKGgaJpZM4KPc_r
.

I wouldn't think it is the web service; since 15 seconds to retrieve the data then another 2:45 to display it.  Is there a way to see which group or user it's hanging on? Turned on testing in the config but didn't seem to display anymore information.  Happy to help test, this is a pretty useful app.

Yeah there's the possibility of the zombie views so it could be causing
numerous redraws and whatnot. It's been awhile since I looked at it but I
have 10 different clients using it at the moment and haven't seen the slow
render times unless it was a corrupt user (but that's typically only on the
fetching part which doesn't account for the other 2:45 minutes).
I'm gonna bring it up in a few and see but might not get time until next
week since I'm busy today and the weekend with best friend's wedding lol.
On Oct 7, 2016 8:08 AM, "vinny75"<EMAIL_ADDRESS>wrote:

I wouldn't think it is the web service; since 15 seconds to retrieve the
data then another 2:45 to display it. Is there a way to see which group or
user it's hanging on? Turned on testing in the config but didn't seem to
display anymore information. Happy to help test, this is a pretty useful
app.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dhardin/SP-PermissionManagement/issues/11#issuecomment-252236275,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEsphbc6PeNI-IpZYhm3HXZsFAOv3VG-ks5qxjZCgaJpZM4KPc_r
.

No worries - great support for a free app; I get a chance this weekend, I'll try stepping through it as well to see if I can find any issues


get road id and s/t coordinates given xyz point

Is it possible that provide a function named get_roadid_st_list_by_xy in C++

Please describe in more detail what function you need

I am sorry. I am not good at English. I am trying to describe clearly.
There is a function named get_xyz in Road.h. It return world xyz  with inputting s and t.
We need the  reverse function which return roadid,s,t with inputting xyz or return roadid,s,t list with inputting xy

Ok. I understand. I will be adding that query feature.

I am sorry. I am not good at English. I am trying to describe clearly. There is a function named get_xyz in Road.h. It return world xyz with inputting s and t. We need the reverse function which return roadid,s,t with inputting xyz or return roadid,s,t list with inputting xy

@saoraozhe3hao
Hi, you said get_xyz() function return world xyz , but in my case there is something wrong.. Coordinate is different from world coordinate. " tmp_xyz = odr_road.get_xyz(1.0, 0.0, 0.0); " I use this line to get world coordinate

When loading the map via odr::OpenDriveMap odr_map("data.xodr"); the default option is to center the map. So you would probably need to add OpenDriveMap::x_offs and y_offs to your point to get the correct world coordinate.

Ok. I understand. I will be adding that query feature.

is here has the feature？

I‘ve implemented the query feature, but i don't konw how to sync it.

@JuddiW looks good, I would like to try if it works. What do you mean by syncing? You can have a look at this guide: https://gist.github.com/Chaser324/ce0505fbed06b947d962

thank you for your help,i will try it.

@lrstttl > @JuddiW looks good, I would like to try if it works. What do you mean by syncing? You can have a look at this guide: https://gist.github.com/Chaser324/ce0505fbed06b947d962
Thank you for your help, I will try it.

So is the query feature merged?

Hello, I am also interested in this. Can someone make this function available?

I‘ve implemented the query feature, but i don't konw how to sync it to github.

Hello, regarding xy2TrackST(x,y), can I obtain your code for it?

@flschnepf @Damonyhk - This is a snippet of the code I use to do a similar thing (get s,t and the lane it belongs to)
the main function is the road.ref_line.match( point_.x, point_.y ); in the libOpenDRIVE code - then it is a bit more work to get the right sign of the t coordinate and the lane key.
I hope this helps
`
template
MapPoint
MapHelper::get_map_point_on_roads( const PointType &point_, bool search_all_roads, const std::setstd::string &road_ids_ ) const {
if( !search_all_roads && road_ids_.empty() ) {
    throw std::invalid_argument( "Road set is empty - can't find nearest point on an empty set of roads" );
}

double      t_coordinate    = std::numeric_limits<double>::max();
std::string closest_road_id = "";
double      closest_s       = 0;
odr::Vec3D  close_point{ 0.0, 0.0, 0.0 };

// Iterate over roads
for( const auto &road_pair : m_road_map.id_to_road ) {
    const auto &road = road_pair.second;

    if( !search_all_roads && road_ids_.count( road.id ) == 0 )
        continue;

    double     s             = road.ref_line.match( point_.x, point_.y );
    odr::Vec3D matched_point = road.ref_line.get_xyz( s );
    double     distance      = std::sqrt( pow( point_.x - matched_point[0], 2 ) + pow( point_.y - matched_point[1], 2 ) );
    if( distance < t_coordinate ) {
        t_coordinate    = distance;
        closest_road_id = road.id;
        closest_s       = s;
        close_point     = matched_point;
    }
}

// Check if the closest road ID was found
if( closest_road_id.empty() ) {
    std::cout << " point: " << point_.x << " " << point_.y << std::endl;
    throw std::runtime_error( "No closest road found." );
}

// Get the gradient/tangent vector at that point on the reference line
auto min_max_s = get_road_min_max_s( closest_road_id );
closest_s      = std::min( closest_s, min_max_s.second );
closest_s      = std::max( closest_s, min_max_s.first );

// Check if the road ID exists in the map
auto road_it = m_road_map.id_to_road.find( closest_road_id );
if( road_it == m_road_map.id_to_road.end() ) {
    throw std::runtime_error( "Closest road ID not found in the map." );
}

const auto &road = road_it->second;

odr::Vec3D grad = road.ref_line.get_grad( closest_s );

geometry::Point point_ahead( close_point[0] + grad[0], close_point[1] + grad[1] );
// Calculate the perpendicular angle of the gradient vector to calculate t
t_coordinate *= wind_left( geometry::Point( close_point[0], close_point[1] ), point_ahead, point_ );

// Check if the lanesection exists at closest_s
double lanesections0  = road.get_lanesection_s0( closest_s );
auto   lanesection_it = road.s_to_lanesection.find( lanesections0 );
if( lanesection_it == road.s_to_lanesection.end() ) {
    throw std::runtime_error( "Lanesection s0 not found in the road." );
}

const auto &lanesection = lanesection_it->second;

// Get the lane ID and check if it exists
int lane_id = lanesection.get_lane_id( closest_s, t_coordinate );
if( lane_id == 0 ) {
    lane_id = 1; // Prevent getting map location on center line
}
auto lane_it = lanesection.id_to_lane.find( lane_id );
if( lane_it == lanesection.id_to_lane.end() ) {
    throw std::runtime_error( "Lane ID not found in the lanesection." );
}

return MapPoint( odr::LaneKey( closest_road_id, lanesections0, lane_id ), closest_s, t_coordinate );

}`

@flschnepf @Damonyhk - This is a snippet of the code I use to do a similar thing (get s,t and the lane it belongs to)
the main function is the road.ref_line.match( point_.x, point_.y ); in the libOpenDRIVE code - then it is a bit more work to get the right sign of the t coordinate and the lane key.
I hope this helps
` template MapPoint MapHelper::get_map_point_on_roads( const PointType &point_, bool search_all_roads, const std::setstd::string &road_ids_ ) const {
if( !search_all_roads && road_ids_.empty() ) {
    throw std::invalid_argument( "Road set is empty - can't find nearest point on an empty set of roads" );
}

double      t_coordinate    = std::numeric_limits<double>::max();
std::string closest_road_id = "";
double      closest_s       = 0;
odr::Vec3D  close_point{ 0.0, 0.0, 0.0 };

// Iterate over roads
for( const auto &road_pair : m_road_map.id_to_road ) {
    const auto &road = road_pair.second;

    if( !search_all_roads && road_ids_.count( road.id ) == 0 )
        continue;

    double     s             = road.ref_line.match( point_.x, point_.y );
    odr::Vec3D matched_point = road.ref_line.get_xyz( s );
    double     distance      = std::sqrt( pow( point_.x - matched_point[0], 2 ) + pow( point_.y - matched_point[1], 2 ) );
    if( distance < t_coordinate ) {
        t_coordinate    = distance;
        closest_road_id = road.id;
        closest_s       = s;
        close_point     = matched_point;
    }
}

// Check if the closest road ID was found
if( closest_road_id.empty() ) {
    std::cout << " point: " << point_.x << " " << point_.y << std::endl;
    throw std::runtime_error( "No closest road found." );
}

// Get the gradient/tangent vector at that point on the reference line
auto min_max_s = get_road_min_max_s( closest_road_id );
closest_s      = std::min( closest_s, min_max_s.second );
closest_s      = std::max( closest_s, min_max_s.first );

// Check if the road ID exists in the map
auto road_it = m_road_map.id_to_road.find( closest_road_id );
if( road_it == m_road_map.id_to_road.end() ) {
    throw std::runtime_error( "Closest road ID not found in the map." );
}

const auto &road = road_it->second;

odr::Vec3D grad = road.ref_line.get_grad( closest_s );

geometry::Point point_ahead( close_point[0] + grad[0], close_point[1] + grad[1] );
// Calculate the perpendicular angle of the gradient vector to calculate t
t_coordinate *= wind_left( geometry::Point( close_point[0], close_point[1] ), point_ahead, point_ );

// Check if the lanesection exists at closest_s
double lanesections0  = road.get_lanesection_s0( closest_s );
auto   lanesection_it = road.s_to_lanesection.find( lanesections0 );
if( lanesection_it == road.s_to_lanesection.end() ) {
    throw std::runtime_error( "Lanesection s0 not found in the road." );
}

const auto &lanesection = lanesection_it->second;

// Get the lane ID and check if it exists
int lane_id = lanesection.get_lane_id( closest_s, t_coordinate );
if( lane_id == 0 ) {
    lane_id = 1; // Prevent getting map location on center line
}
auto lane_it = lanesection.id_to_lane.find( lane_id );
if( lane_it == lanesection.id_to_lane.end() ) {
    throw std::runtime_error( "Lane ID not found in the lanesection." );
}

return MapPoint( odr::LaneKey( closest_road_id, lanesections0, lane_id ), closest_s, t_coordinate );

}`

Thank you very much. I think your code is very useful, but I'm not quite clear about the wind_left() function. Could you please elaborate on it in more detail?

@Damonyhk
If you look into the match function you see that it calls a golden_section_search function which is a numerical method that will not give an exact answer but one with a given tolerance.

@markomiz Thank you for sharing this! Could you explain the get_road_min_max_s() function? I may be being naive, but shouldn't the minimum and maximum S-coordinate of a road be zero and the road's length respectively?

@markomiz Thank you for sharing this! Could you explain the get_road_min_max_s() function? I may be being naive, but shouldn't the minimum and maximum S-coordinate of a road be zero and the road's length respectively?

please try this code.
struct road_st
{
std::string roadId;
int laneId;
std::string laneType;
bool inRoad;
double s;
double t;
double hdg;
road_st()
{
    roadId = "-1";
    laneId = 0xFF;
    laneType = "unknown";
    inRoad = false;
    s = 0;
    t = 0;
    hdg = 0;
}

};
road_st Road::xy2RoadST(double x,double y)
{
road_st st;
double closestS = 0;
try
{
    closestS = ref_line.match(x,y);
    double x1 = ref_line.get_xyz(closestS)[0];
    double y1 = ref_line.get_xyz(closestS)[1];
    double hdg = ref_line.get_geometry(closestS)->get_hdg(closestS);

    auto CrossProduct2D = [&](double x1, double y1, double x2, double y2) ->double
    {
        return x1 * y2 - x2 * y1;
    };

    auto DotProduct2D = [&](double x1, double y1, double x2, double y2) -> double
    {
        return x1 * x2 + y1 * y2;
    };

    auto PointToLineDistance2D =[&](double px, double py, double lx0, double ly0, double lx1, double ly1) -> double
    {
        double l0x      = lx1 - lx0;
        double l0y      = ly1 - ly0;
        double cp       = GetCrossProduct2D(lx1 - lx0, ly1 - ly0, px - lx0, py - ly0);
        double l0Length = sqrt(l0x * l0x + l0y * l0y);
        return cp / l0Length;
    };

    double t = PointToLineDistance2D(x,y,x1,y1,x1+cos(hdg),y1+sin(hdg));

    auto lanes = this->get_lanesection(closestS).get_lanes();
    std::sort(lanes.begin(),lanes.end(),[](const Lane &a, const Lane &b){return a.id < b.id;});

    double x2 = std::cos(hdg) * 100 + x1;
    double y2 = std::sin(hdg) * 100 + y1;
    double va_x = x2 - x1;
    double va_y = y2 - y1;
    double vb_x = x - x1;
    double vb_y = y - y1;

    double dp = DotProduct2D(va_x,va_y,vb_x,vb_y);
    double va_val = std::sqrt(va_x * va_x + va_y * va_y);
    double vb_val = std::sqrt(vb_x * vb_x + vb_y * vb_y);
    double theta = dp / (va_val * vb_val);

    if(theta < -1 && theta > -2)
        theta = -1;
    if(theta > 1 && theta < 2)
        theta = 1;

    double degrees = acos(theta) * 180 / M_PI;
    if(fabs(90 - degrees) > 0.005 || lanes.empty() || t < lanes.front().outer_border.get(closestS) || t > lanes.back().outer_border.get(closestS))
    {
        return st;
    }

    st.s = closestS;
    st.t = t;
    st.hdg = hdg;

    auto lane = get_lanesection(st.s).get_lane(st.s,st.t);
    st.laneId = lane.id;
    st.laneType = lane.type;
    st.roadId = id;
    return st;

}
catch(const std::exception& e)
{
    printf("%s\n",e.what());
    return road_st();
}

}

@JuddiW thanks for sharing. The RoadGeometry class doesn't have a get_hdg() method - is this something else you've implemented?

@catmxt please try this code. struct road_st { std::string roadId; int laneId; std::string laneType; bool inRoad; double s; double t; double hdg;
road_st()
{
    roadId = "-1";
    laneId = 0xFF;
    laneType = "unknown";
    inRoad = false;
    s = 0;
    t = 0;
    hdg = 0;
}

};
road_st Road::xy2RoadST(double x,double y) { road_st st; double closestS = 0;
try
{
    closestS = ref_line.match(x,y);
    double x1 = ref_line.get_xyz(closestS)[0];
    double y1 = ref_line.get_xyz(closestS)[1];
    double hdg = ref_line.get_geometry(closestS)->get_hdg(closestS);

    auto CrossProduct2D = [&](double x1, double y1, double x2, double y2) ->double
    {
        return x1 * y2 - x2 * y1;
    };

    auto DotProduct2D = [&](double x1, double y1, double x2, double y2) -> double
    {
        return x1 * x2 + y1 * y2;
    };

    auto PointToLineDistance2D =[&](double px, double py, double lx0, double ly0, double lx1, double ly1) -> double
    {
        double l0x      = lx1 - lx0;
        double l0y      = ly1 - ly0;
        double cp       = GetCrossProduct2D(lx1 - lx0, ly1 - ly0, px - lx0, py - ly0);
        double l0Length = sqrt(l0x * l0x + l0y * l0y);
        return cp / l0Length;
    };

    double t = PointToLineDistance2D(x,y,x1,y1,x1+cos(hdg),y1+sin(hdg));

    auto lanes = this->get_lanesection(closestS).get_lanes();
    std::sort(lanes.begin(),lanes.end(),[](const Lane &a, const Lane &b){return a.id < b.id;});

    double x2 = std::cos(hdg) * 100 + x1;
    double y2 = std::sin(hdg) * 100 + y1;
    double va_x = x2 - x1;
    double va_y = y2 - y1;
    double vb_x = x - x1;
    double vb_y = y - y1;

    double dp = DotProduct2D(va_x,va_y,vb_x,vb_y);
    double va_val = std::sqrt(va_x * va_x + va_y * va_y);
    double vb_val = std::sqrt(vb_x * vb_x + vb_y * vb_y);
    double theta = dp / (va_val * vb_val);

    if(theta < -1 && theta > -2)
        theta = -1;
    if(theta > 1 && theta < 2)
        theta = 1;

    double degrees = acos(theta) * 180 / M_PI;
    if(fabs(90 - degrees) > 0.005 || lanes.empty() || t < lanes.front().outer_border.get(closestS) || t > lanes.back().outer_border.get(closestS))
    {
        return st;
    }

    st.s = closestS;
    st.t = t;
    st.hdg = hdg;

    auto lane = get_lanesection(st.s).get_lane(st.s,st.t);
    st.laneId = lane.id;
    st.laneType = lane.type;
    st.roadId = id;
    return st;

}
catch(const std::exception& e)
{
    printf("%s\n",e.what());
    return road_st();
}

}

What is the get_hdg function in the code?

@wangdemon I had the same problem, I implemented it like this:
double get_hdg(double gradient_x, double gradient_y)
{
    return atan2(gradient_x, gradient_y);
}

// ...

odr::Vec2D gradient = ref_line.get_geometry(s)->get_grad(s);
double hdg = get_hdg(gradient[0], gradient[1]);

@wangdemon I had the same problem, I implemented it like this:
double get_hdg(double gradient_x, double gradient_y)
{
    return atan2(gradient_x, gradient_y);
}

// ...

odr::Vec2D gradient = ref_line.get_geometry(s)->get_grad(s);
double hdg = get_hdg(gradient[0], gradient[1]);

Thank you for your reply
However, during testing, the atan2(gradient_x, gradient_y) resulted in a smaller t value
I use hdg0 instead of  atan2(gradient_x, gradient_y), as if the t value of t is closer

@wangdemon I had the same problem, I implemented it like this:
double get_hdg(double gradient_x, double gradient_y)
{
    return atan2(gradient_x, gradient_y);
}

// ...

odr::Vec2D gradient = ref_line.get_geometry(s)->get_grad(s);
double hdg = get_hdg(gradient[0], gradient[1]);

Thank you for your reply
However, during testing, the atan2(gradient_x, gradient_y) resulted in a smaller t value
I use hdg0 instead of atan2(gradient_x, gradient_y), as if the t value of t is closer

double Road::get_hdg(double s) {     auto geometry = ref_line.get_geometry(s);     if(geometry->type == odr::GeometryType_Line)     {         return geometry->hdg0;     }     else if(geometry->type == odr::GeometryType_Arc)     {         odr::Arc* arc = (odr::Arc*)geometry;         return (s - arc->s0) * arc->curvature + arc->hdg0;     }     else if(geometry->type == odr::GeometryType_Spiral)     {         odr::Spiral* spiral = (odr::Spiral*)geometry;         double x,y,theta;         double s1 = spiral->curv_start / spiral->c_dot + s - spiral->s0;         odrSpiral(s1,spiral->c_dot,&x,&y,&theta);         return spiral->hdg0 + theta;     }     else if(geometry->type == odr::GeometryType_ParamPoly3)     {         odr::ParamPoly3* parampoly3 = (odr::ParamPoly3*)geometry;         auto grad = parampoly3->get_grad(s);         return std::atan2(grad[0],grad[1]);      }     else     {         return geometry->hdg0;     } }


always force the file inside the tarball owned by root

similar to https://github.com/influxdata/telegraf/issues/1704
Required for all non-trivial PRs

[x] Rebased/mergable
[ ] Tests pass
[ ] CHANGELOG.md updated
[x] Sign CLA (if not already signed)

@lfarkas Thanks!, do we also want to do the same thing for the group?

yes that would be good.

@lfarkas I'll merge once that is done. Thanks!

closing in favor of #1660


[ECard Transactions] Fix EReceipt Hash

Fix the logic for defaulting when ""

@grgia Mind adding the Bug label here so payment automation works? Thanks!

Joel asked to add the awaiting payment label here. Looks like the PR has hit prod on Nov 10 so it should be ready for payment on the 17th. But I'll let @MitchExpensify confirm and add labels as appropriate 🥳

This is paid! https://github.com/Expensify/App/issues/24946

@MitchExpensify I think that's a different issue, this is the PR for this one.

Ok, sorry about that! So the idea is to use this issue to track payment of $250 for this PR, is that right?

No worries, thanks for reopening! Yeah I wasn't sure what to do otherwise when the automation fails as it did here :sweat_smile:  I think the payment would be $500 as that's the default price for an internal PR review, as far as I understand.

Cool, offer sent here! https://www.upwork.com/jobs/~0134f81a81ffe31846

@MitchExpensify Thanks, offer accepted! :bow:

closing as completed!
(@grgia testing something)


Autodetect if we talk to docker via tcp in the Makefile

This means people who build on a Mac, using docker machine, don't have to type make SUDO= all the time.

ifeq (,$(findstring tcp://,$(DOCKER_HOST)))
SUDO=sudo -E
else
SUDO=
endif

@tomwilkie says that this doesn't actually work with the latest Docker-on-Mac and that scope has "moved on".

"moved on"

I have updated this to match current scope.

@tom do you want to test this on OSX or shall I just merge it?

@awh works nicely.  Thanks.


Simplify face alignment procedure

Hello!
Currently, the face alignment procedure considers the eye positions and makes use of the Cosine Law to calculate the needed rotation angle. However, the angle can actually be calculated by a simple use of the np.arctan2 function.
In this PR, I proposed to vastly simplify the alignment_procedure function, while keeping the semantics of the function unchanged so that nothing will break, e.g., left-eye is the eye to the left of the viewer, as in the original code.
I have also prepare the following snippet, to illustrate that the angle calculated by the current method, and the newly proposed method, are the same. This snippet calculates the angle found by the two methods at different relative eye positions, and print out the angle values for comparison.
Please feel free to let me know if there are any concerns.
Thanks in advance for your consideration!
import math
import numpy as np

# --- Proposed simpler method

def get_alignment_angle_arctan2(left_eye, right_eye):
    """
    The left_eye is the eye to the left of the viewer,
    i.e., right eye of the person in the image.
    The top-left point of the frame is (0, 0).
    """
    return float(np.degrees(
        np.arctan2(right_eye[1] - left_eye[1], right_eye[0] - left_eye[0])
    ))

# --- Current method

def find_euclidean_distance(source_representation, test_representation):
    """
    https://github.com/serengil/deepface/blob/master/deepface/commons/distance.py#L11
    """
    if isinstance(source_representation, list):
        source_representation = np.array(source_representation)

    if isinstance(test_representation, list):
        test_representation = np.array(test_representation)

    euclidean_distance = source_representation - test_representation
    euclidean_distance = np.sum(np.multiply(euclidean_distance, euclidean_distance))
    euclidean_distance = np.sqrt(euclidean_distance)
    return euclidean_distance

def get_alignment_angle_cosine_law(left_eye, right_eye):
    """
    https://github.com/serengil/deepface/blob/master/deepface/detectors/FaceDetector.py#L85

    left eye is the eye appearing on the left (right eye of the person)
    left top point is (0, 0)
    """

    left_eye_x, left_eye_y = left_eye
    right_eye_x, right_eye_y = right_eye

    # -----------------------
    # find rotation direction

    if left_eye_y > right_eye_y:
        point_3rd = (right_eye_x, left_eye_y)
        direction = -1  # rotate same direction to clock
    else:
        point_3rd = (left_eye_x, right_eye_y)
        direction = 1  # rotate inverse direction of clock

    # -----------------------
    # find length of triangle edges

    a = find_euclidean_distance(np.array(left_eye), np.array(point_3rd))
    b = find_euclidean_distance(np.array(right_eye), np.array(point_3rd))
    c = find_euclidean_distance(np.array(right_eye), np.array(left_eye))

    # -----------------------

    # apply cosine rule

    if b != 0 and c != 0:  # this multiplication causes division by zero in cos_a calculation
        cos_a = (b * b + c * c - a * a) / (2 * b * c)
        angle = np.arccos(cos_a)  # angle in radian
        angle = (angle * 180) / math.pi  # radian to degree

        # -----------------------
        # rotate base image

        if direction == -1:
            angle = 90 - angle

    else:
        angle = 0

    img_rotate_angle = direction * angle

    return img_rotate_angle

def test():
    # Assume frame size is (10, 10).

    # Left-eye is to the left of the viewer,
    # i.e., right eye of the person in the image.

    # For simplicity, here we only change right-eye-y values to produce
    # different angles. While it's not exhaustive, it should be good enough
    # for illustration.
    right_eye_ys = np.arange(0, 10, 0.05)
    right_eye_xs = 7 * np.ones_like(right_eye_ys)
    left_eye_ys = 5 * np.ones_like(right_eye_ys)
    left_eye_xs = 3 * np.ones_like(right_eye_ys)

    left_eyes = zip(left_eye_xs, left_eye_ys)
    right_eyes = zip(right_eye_xs, right_eye_ys)

    for (left_eye, right_eye) in zip(left_eyes, right_eyes):

        angle_cosine_law = get_alignment_angle_cosine_law(left_eye, right_eye)
        angle_tangent = get_alignment_angle_arctan2(left_eye, right_eye)

        print(f"Angle cosine law: {angle_cosine_law}")
        print(f"Angle tangent: {angle_tangent}")
        print("-----------")

if __name__ == "__main__":
    test()

will test some edge cases before merging this, thank you for your contribution.

Hi @serengil !
Can I know the progress on this one?
Thanks again!


JanDeDobbeleer.OhMyPosh version 21.28.0

Checklist for Pull Requests

[ ] Have you signed the Contributor License Agreement?
[ ] Is there a linked Issue?

Manifests

[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] This PR only modifies one (1) manifest
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.6 schema?

Note: <path> is the directory's name containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

#142854

[Policy] Needs-Manual-Merge

Adding the label https://github.com/microsoft/winget-pkgs/labels/Needs-Manual-Merge adds https://github.com/microsoft/winget-pkgs/labels/Needs-Attention, which moves this out of my Squash-Ready search and into my VCNA and https://github.com/microsoft/winget-pkgs/labels/Needs-Manual-Merge searches.

https://github.com/microsoft/winget-pkgs/issues/142854


Can't publish with a very long string. I'm using ESP8266 controller.

Hi everyone!
I have a project that i need to gather a chain of data which in my case the requirement is N = 512 samples. These samples are in an int array as shown below. I converted it to a string for mqtt publishing. My problem is it doesn't publish. When I change the number of samples to just 16 samples (N = 16), it is perfectly sent without a problem. Is there a limitation in pubsubclient library or just my ESP8266? The necessary codes are shown below. Hoping for your great ideas. Thank you!
int N = 512;
// code for recording the data
void Record
{
for (int m = 0; m < N ; m++)
{
data = adc.readADC(0); //
dataarray[m] = data;
delayMicroseconds(34);
}
}
// convert data to string
for (int n = 0; n < N; n++) {
datastring += ", " + String(dataarray[n]);
}
//code for publishing
client.publish(Topic, String(datastring).c_str(), true);

The default max packet size is 128 bytes.
You need to increase this value by editing PubSubClient.h - http://pubsubclient.knolleary.net/api.html#configoptions


requires: check stat; unknown requires keyword - v2

For the requires keyword, check the new state, rules_skipped. Also add a test
for an unknown requires keyword to make sure the rule doesn't error out.
Feature: https://redmine.openinfosecfoundation.org/issues/6637

Merged in #1570, thanks!


Sidebar: Option to Hide "Open editor" if folder is open

My idea:

If the project folder is open and there are no open files outside the project:

If there is no open folder but any files are open:

You can right click on the Open Editrs section and just hide it. We try not to automatically hide things in the layout as it might confuse users.
However leaving this open as a feature request.


⭐ Template // Website content crafting and publishing: (Patterns/ Name of the new pattern name)

Objective
Provide Carbon for IBM.com adopter the best practices and guidance they need to adopt Carbon for IBM.com.
General publishing process
Please make sure you always start the content crafting process by viewing the 2020 website content publishing process box document first.
Publishing for Patterns section website page

Content crafting starts as soon as Initial research and publishing agreements are accepted by the DDS leadership team.
The content can be at the MVP level. MVP means the content meets the basic needs of an adopter for learning. It does not mean the content crafting will or should stop at this stage. Future enhancement should be working on with a separate task issue.
MVP content publishing template and requirement can be found in the Publishing template and best practices/_General page folder.
Content publishing to the live website can be code or markdowns. If you need guidance on how to start with markdowns, please see the step by step instruction in the Editing and publishing in Github Box document.
When the page is published, please also make sure the patterns table list on the Overview page is updated as part of the publishing task.

File naming and storage

Name the Sketch file with the following convention: Name of the category + components/pattern/guideline + version. ex: Components-Feature card block v1.0, Guidelines-Expressive theme v1.0, Help-Overview v1.0 and etc.
Please save all content publishing work files (box, markdowns link, or Sketch file) in the proper sub-folder under the ✍ Content + Design folder.

Publishing acceptance criteria

[ ] Content crafting is done.
[ ] Asset image creation is done.
[ ] Page design is done (as needed).
[ ] Publishing with markdown is done.
[ ] Update the Overview page component table list with the pattern name, link, description and status when the markdown is done.
[ ] QA'd by Jenny.
[ ] Web page preview is approved by Linda, Jeff, and Wonil.
[ ] PR merged by Jenny.
[ ] Web page is published and live on the website.

No longer needed.  Closing


how the model ensure semantic information goes to z rather than x_t?

Dear diffae team,
Thank you for sharing this great work, I really enjoy it.
I paly with the model a bit. When I fix z and randomly sample x_t, the output images are almost the same, with some small variaces. how the model ensure semantic information goes to z rather than x_t (starting gaussian noise)? Maybe because the model is trained with reconstruction loss, z is the only source for the target image information, and x_t is randomly sampled even for the same target image. So the model learn to 'ingore' x_t and therefore x_t only contains little semantic information.
In this repo, Justin takes the Unet from stable diffusion and the image encoder from CLIP, and train the model using image reconstruction loss. If my intuition in last paragraph is correct, then given the same image as input, we randomly sample x_t, the output images will be almost the same. But this not the case in Justin's model, changing x_t causes a large change (including semantic change) in output image. This mean the x_t in Justin's model contain more semantic information than the x_t in DA model. Could you tell me why this is happening?
Thank you for your help.
Best Wishes,
Zongze

This is the question we have been pondering for so long as well. I don't pretend to have the full answer. The following is the best intuition I can give you.
DiffAE doesn't learn a "semantic" latent code. It learns a latent code that packs "the most bang for the buck" under the implicit bias of the decoder, in this case, a diffusion model.
There is no reason to believe that a semantic latent code, corresponding to human judgment, is the same as one that packs the most bang for the buck, due to the strong emphasis of the reconstruction loss.
The fact that we observed DiffAE learn a semantically meaningful latent code should be considered accidental. Maybe, due to the dataset we used, FFHQ256, which is heavily cropped and properly aligned. We should not expect this quality of DiffAE to transfer to other datasets in general.
What to do if I want to control what's kept in the latent code?
There are two ways I could think of: change the loss function to something more semantic, or using a different encoder.
Using a different loss function resulting in a different latent code is easy to understand, but finding a substitute for the reconstruction loss is hard to do.
A more doable way is through the encoder. Let's consider a CLIP encoder. It was trained with correspondence between images and texts. It tends to capture high-level abstract concepts due to the text information given to describe an image. The latent code from this CLIP encoder is expected to capture high-level concepts rather than precise appearances.
One could extend this intuition to another encoder as ArcFace. Since ArcFace is trained to be invariant to most things expect the face identity. The latent code from this encoder should be deprived of most information except identity, i.e., the decoded face may not share the same pose an the original face.

Thx for your reply. It really helps me a lot.


Wrong formatted sql in Ray when using text search operator @@ in postgres query

Describe the bug
The formatted sql query shown when using the text search operator "@@" in postgres is wrong, it gets transformed into "@ @" which is incorrrect
Versions
Ray version (you can see this in "About Ray"): <IP_ADDRESS>
You can use composer show to get the version numbers of:

spatie/ray package version: 1.37.2
spatie/laravel-ray package version: 1.32.4

PHP version: 8.1
Laravel version: 9.52.7
To Reproduce
\DB::select("select to_tsvector('simple', 'some text') @@ to_tsquery('simple', 'text') as field");

Expected behavior
select
  to_tsvector('simple', 'some text') @@ to_tsquery('simple', 'text') as field

Screenshots

Desktop (please complete the following information):

OS: Windows 10

Hi @pablosebastianr
Version 2.6.5 of the Ray app will now try different sql dialects, including postgresql.
I'm closing this issue now but if you run into more formatting issues, feel free to tag me.


emp fails with self-signed certificates

emp fails when using a self-signed ssl certificate.  To generate and upload a new ssl certificate:
http://git.io/vYPHI
When that certificate is used for Empire, commands result in the following:
$ emp apps
error: Get https://empire.brianz.bz/apps: x509: certificate signed by unknown authority

This certificate was added to Keychain on my Mac, and Safari seems happy with it:

@phobologic suggests this may be an issue with the way Go handles certs:
https://groups.google.com/forum/#!topic/golang-nuts/v5ShM8R7Tdc

Please open in https://github.com/remind101/emp - thanks :)


Wrapper class and alias

❓ Questions and Help

How to explain the structure  D: BaseProviderWrapper = Wrapper()

Wrapper() is a wrapper for the inner object.
With it, users could replace the inner object without modifying the variable references to D.


As of changeset 870 the Game Settings dropbox for "Players per Team" no longer functions

It is stuck at 1-100 and unusable.
Imported from trac ticket #200,  created by Bard on 08-03-2010 at 20:55, last modified: 08-03-2010 at 23:01

The dropdown box does not work properly.
None of the new maxplayers settings can be selected.
Trac comment by Bard on 08-03-2010 at 21:18

Its based off of MaxPlayersPerGame setting defined by the server, which is 200.  The functionality has not changed in r870, only artwork.  Refer to the original reporter: pkk for further explanation if needed.
Trac comment by Imago on 08-03-2010 at 20:59

We just consulted in IRC about your problem and it seems you have the old artwork.  if you look at r807 you will see that it consists of mainly an artwork change.  Attached to this ticket is the artwork file you need.  Also note that you cannot change the max players for a default game, you have to create your own.
Trac comment by Imago on 08-03-2010 at 22:45

I've tested this and everything worked fine.
I were unable to change teamsize in a zonegame (maybe issue with #202), but it worked fine in a self created game.
Zonegames are wired anyway, see #202 #203
Trac comment by pkk on 08-03-2010 at 23:01

please regress your testing (with current R5 and previous) and consult with the reporter that triggered change r870 before re-opening.  To reiterate: this ticket is invalid - a defect was not introduced in R6 with change r870 as this ticket suggests.  As a matter of fact, this ticket contradicts r870's triggering ticket #185
Trac comment by Imago on 08-03-2010 at 22:05


Make rmarkdown files work with styler and lintr

See #2450

I have been working with rnoweb (.rnw) files recently and they work basically the same as R Markdown so might be worth adding as well if you were happy to.

I don't agree. For rnoweb files, I would rather have the linters/fixers related to latex.

Ah of course, I do agree, hadn't thought about that

Cheers! :beers:

Thanks!


Evtx.BinaryParser.OverrunBufferException raised unexpectedly

Hi,
I'm seeing exception Evtx.BinaryParser.OverrunBufferException raised unexpectedly when parsing event logs "Microsoft-Windows-Windows Firewall With Advanced Security%4Firewall.evtx",  "Microsoft-Windows-WindowsUpdateClient%4Operational.evtx", and "Microsoft-Windows-CAPI2%4Operational.evtx" (all extracted from a Windows 7 testing instance) using evtx_dump.py.
For Microsoft-Windows-Windows Firewall With Advanced Security%4Firewall.evtx, the same behaviour is seen across Windows with Python 2.7 and Ubuntu with Python 2.7 / 3.5. Traceback from Ubuntu with Python 3.5 as follows:
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 273, in unpack_byte
    return struct.unpack_from("<B", self._buf, o)[0]
struct.error: unpack_from requires a buffer of at least 1 bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.local/bin/evtx_dump.py", line 42, in <module>
    main()
  File "/home/user/.local/bin/evtx_dump.py", line 37, in main
    print(record.xml())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Evtx.py", line 481, in xml
    return e_views.evtx_record_xml_view(self)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 204, in evtx_record_xml_view
    return render_root_node(record.root())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 182, in render_root_node
    for sub in root_node.substitutions():
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 1001, in substitutions
    ofs = self.tag_and_children_length()
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 962, in tag_and_children_length
    for child in self.children():
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 952, in children
    return self._children(end_tokens=[SYSTEM_TOKENS.EndOfStreamToken])
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 153, in _children
    self._chunk, self)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 777, in __init__
    parent=self)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Evtx.py", line 398, in add_template
    self._load_templates()
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Evtx.py", line 380, in _load_templates
    token = self.unpack_byte(ofs - 10)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 275, in unpack_byte
    raise OverrunBufferException(o, len(self._buf))
Evtx.BinaryParser.OverrunBufferException: Tried to parse beyond the end of the file (read: 0x240f0ff7, buffer length: 0x101000)

For Microsoft-Windows-WindowsUpdateClient%4Operational.evtx, Python 3.5 on Ubuntu gives Evtx.BinaryParser.OverrunBufferException, while Python 2.7 on Windows / Ubuntu gives a MemoryError. Python 3.5 traceback as follows:
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 273, in unpack_byte
    return struct.unpack_from("<B", self._buf, o)[0]
struct.error: unpack_from requires a buffer of at least 1 bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.local/bin/evtx_dump.py", line 42, in <module>
    main()
  File "/home/user/.local/bin/evtx_dump.py", line 37, in main
    print(record.xml())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Evtx.py", line 481, in xml
    return e_views.evtx_record_xml_view(self)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 204, in evtx_record_xml_view
    return render_root_node(record.root())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 182, in render_root_node
    for sub in root_node.substitutions():
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 1006, in substitutions
    type_ = self.unpack_byte(ofs + 0x2)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 275, in unpack_byte
    raise OverrunBufferException(o, len(self._buf))
Evtx.BinaryParser.OverrunBufferException: Tried to parse beyond the end of the file (read: 0x101000, buffer length: 0x101000)

Python 2.7 traceback (from Ubuntu) as follows:
Traceback (most recent call last):
  File "/home/user/.local/bin/evtx_dump.py", line 42, in <module>
    main()
  File "/home/user/.local/bin/evtx_dump.py", line 37, in main
    print(record.xml())
  File "/home/user/.local/lib/python2.7/site-packages/Evtx/Evtx.py", line 481, in xml
    return e_views.evtx_record_xml_view(self)
  File "/home/user/.local/lib/python2.7/site-packages/Evtx/Views.py", line 204, in evtx_record_xml_view
    return render_root_node(record.root())
  File "/home/user/.local/lib/python2.7/site-packages/Evtx/Views.py", line 182, in render_root_node
    for sub in root_node.substitutions():
  File "/home/user/.local/lib/python2.7/site-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/home/user/.local/lib/python2.7/site-packages/Evtx/Nodes.py", line 1004, in substitutions
    for _ in range(sub_count):
MemoryError

And similarly for Microsoft-Windows-CAPI2%4Operational.evtx, Python 3.5 on Ubuntu gives Evtx.BinaryParser.OverrunBufferException, while Python 2.7 on Windows / Ubuntu BSODs / has process 'Killed'.  Traceback for Python 3.5 on Ubuntu as follows:
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 302, in unpack_word
    return struct.unpack_from("<H", self._buf, o)[0]
struct.error: unpack_from requires a buffer of at least 2 bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.local/bin/evtx_dump.py", line 42, in <module>
    main()
  File "/home/user/.local/bin/evtx_dump.py", line 37, in main
    print(record.xml())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Evtx.py", line 481, in xml
    return e_views.evtx_record_xml_view(self)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 204, in evtx_record_xml_view
    return render_root_node(record.root())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 191, in render_root_node
    return render_root_node_with_subs(root_node, subs)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 176, in render_root_node_with_subs
    rec(c, acc)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 126, in rec
    rec(child, acc)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 126, in rec
    rec(child, acc)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 166, in rec
    sub = render_root_node(sub.root())
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Views.py", line 182, in render_root_node
    for sub in root_node.substitutions():
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 64, in __call__
    cache[key] = self.func(*args, **kw)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/Nodes.py", line 1005, in substitutions
    size = self.unpack_word(ofs)
  File "/home/user/.local/lib/python3.5/site-packages/Evtx/BinaryParser.py", line 304, in unpack_word
    raise OverrunBufferException(o, len(self._buf))
Evtx.BinaryParser.OverrunBufferException: Tried to parse beyond the end of the file (read: 0x101000, buffer length: 0x101000)

I can forward the EVTX files in question if that assists?

I have the same problem, but in this instance, I can't share the EVTX files for confidentiality reasons.  I have the same parsing problem.  I tried to follow some of your steps above for leads, and I will paste the output below.  I'm running python 3.7.1.
`λ python D:\tools\Parse-Evtx\scripts\evtx_info.py "Microsoft-Windows-TerminalServices-LocalSessionManager%4Operational.evtx"
Information from file header:
Format version  : 3.1
Flags           : 0x00000001
File is         : dirty
Log is full     : no
Current chunk   : 8 of 9
Oldest chunk    : 1
Next record#    : 1018
Check sum       : pass
Traceback (most recent call last):
File "D:\tools\Parse-Evtx\scripts\evtx_info.py", line 114, in
main()
File "D:\tools\Parse-Evtx\scripts\evtx_info.py", line 57, in main
chunk_count = sum([1 for c in fh.chunks() if c.verify()])
File "D:\tools\Parse-Evtx\scripts\evtx_info.py", line 57, in
chunk_count = sum([1 for c in fh.chunks() if c.verify()])
File "C:\Python3\lib\site-packages\Evtx\Evtx.py", line 333, in verify
return self.check_magic() and
File "C:\Python3\lib\site-packages\Evtx\Evtx.py", line 308, in check_magic
return self.magic() == "ElfChnk\x00"
File "C:\Python3\lib\site-packages\Evtx\BinaryParser.py", line 211, in explicit_length_handler
return f(offset, length)
File "C:\Python3\lib\site-packages\Evtx\BinaryParser.py", line 475, in unpack_string
return self.unpack_binary(offset, length).decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 4: ordinal not in range(128)`
Here is the excerpt from evtx_structure.py where it failed.
`  Chunk
offset: 0x91000
magic:   ☻   ☻
file_first_record_number: 0x20048
file_last_record_number: 0x12c65ab
log_first_record_number: 0x2
log_last_record_number: 0x0
header_size: 0x20800
last_record_offset: 0x3
next_record_offset: 0x20000
data_checksum: 0x0
header_checksum: 0x0
verify: False
Unexpected token encountered
Unexpected token encountered
Traceback (most recent call last):
File "C:\Python3\lib\site-packages\Evtx\BinaryParser.py", line 273, in unpack_byte
return struct.unpack_from("<B", self._buf, o)[0]
struct.error: unpack_from requires a buffer of at least 1 bytes
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\tools\Parse-Evtx\scripts\evtx_structure.py", line 185, in
main()
File "D:\tools\Parse-Evtx\scripts\evtx_structure.py", line 180, in main
for line in formatter.format_header(log.get_file_header()):
File "D:\tools\Parse-Evtx\scripts\evtx_structure.py", line 66, in format_header
for line in self.format_chunk(chunk):
File "D:\tools\Parse-Evtx\scripts\evtx_structure.py", line 89, in format_chunk
yield self._l("templates: %d" % (len(chunk.templates())))
File "C:\Python3\lib\site-packages\Evtx\Evtx.py", line 411, in templates
self._load_templates()
File "C:\Python3\lib\site-packages\Evtx\Evtx.py", line 380, in _load_templates
token = self.unpack_byte(ofs - 10)
File "C:\Python3\lib\site-packages\Evtx\BinaryParser.py", line 275, in unpack_byte
raise OverrunBufferException(o, len(self._buf))
Evtx.BinaryParser.OverrunBufferException: Tried to parse beyond the end of the file (read: 0x43dbfc, buffer length: 0x101000)`

Experiencing the same exact issues. Any recommendations?

@williballenthin I believe this evt(x) file causes the same exception: https://send.firefox.com/download/1804ced0a1ad523c/#A_E9qDRE3rU1poCUk_zq4g
I hope this helps.
(it's produced by wusa.exe)


Count Issue in adding service in company

Describe the bug
When we are adding service to company

click on service count on company list it will open selected service for company page
click on +service button it will open add service page, select some service
filter that records with filter until that selected record is not visible
Select some other records again filter and again select some other records
Now you don't have idea how many records has selected

Expected behavior

there should be show some count how many service are adding
or if we filter records in add page all checkbox should uncheck

Screenshots
If applicable, add screenshots to help explain your problem.

Selected Count Removed @vinpahilwani
and Design Change Check Again

https://www.berrycast.com/conversations/5b56bf71-4f94-56ad-8cfb-ecb2c007055c


[tool_crash] StateError: Bad state: Future already completed

Command
flutter attach

Steps to Reproduce

...
...
...

Logs
StateError: Bad state: Future already completed
#0      _AsyncCompleter.complete (dart:async/future_impl.dart:45:31)
#1      FlutterDevice.connect.<anonymous closure> (package:flutter_tools/src/resident_runner.dart:321:17)
<asynchronous suspension>

[✓] Flutter (Channel stable, 2.5.3, on macOS 12.0.1 21A559 darwin-arm, locale zh-Hans-CN)
    • Flutter version 2.5.3 at /Library/flutter/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 18116933e7 (3 months ago), 2021-10-15 10:46:35 -0700
    • Engine revision d3ea636dc5
    • Dart version 2.14.4

[!] Android toolchain - develop for Android devices (Android SDK version 32.0.0)
    • Android SDK at /Users/chunma/Library/Android/sdk
    • Platform android-32, build-tools 32.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)
    ! Some Android licenses not accepted.  To resolve this, run: flutter doctor --android-licenses

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 13.1, Build version 13A1030d
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2020.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7249189)

[✓] Connected device (3 available)
    • M2007J17C (mobile)              • cb1b109c          • android-arm64  • Android 10 (API 29)
    • Mac Designed for iPad (desktop) • designed-for-ipad • darwin         • macOS 12.0.1 21A559 darwin-arm
    • Chrome (web)                    • chrome            • web-javascript • Google Chrome 96.0.4664.110

! Doctor found issues in 1 category.

Flutter Application Metadata
Type: module
Version: 1.0.0+1
Material: true
Android X: true
Module: true
Plugin: false
Android package: com.example.flutter_module_doctor
iOS bundle identifier: com.example.flutterModuleDoctor
Creation channel: stable
Creation framework version: 1d9032c7e1d867f071f2277eb1673e8f9b0274e3
Plugins
audio_session-0.1.6+1
audioplayers-0.20.1
camera-0.9.4+5
camera_web-0.2.1+1
connectivity-3.0.6
connectivity_for_web-0.4.0+1
connectivity_macos-0.2.1+2
file_picker-4.3.0
flutter_app_badger-1.3.0
flutter_boost-fe438a61ffd6df6af51226b5a48bb9f533b7a1bc
flutter_inappwebview-5.3.2
flutter_local_notifications-9.2.0
flutter_openim_sdk-1.1.0
flutter_plugin_android_lifecycle-2.0.5
image_gallery_saver-1.7.1
image_picker-0.8.4+4
image_picker_for_web-2.1.4
just_audio-0.7.5
just_audio_web-0.3.2
path_provider-2.0.8
path_provider_android-2.0.11
path_provider_ios-2.0.7
path_provider_linux-2.1.4
path_provider_macos-2.0.4
path_provider_windows-2.0.4
permission_handler-8.3.0
photo_manager-1.3.10
record-2.1.1
shared_preferences-2.0.11
shared_preferences_android-2.0.9
shared_preferences_ios-2.0.8
shared_preferences_linux-2.0.3
shared_preferences_macos-2.0.2
shared_preferences_web-2.0.2
shared_preferences_windows-2.0.3
sqflite-2.0.1
video_player-2.2.10
video_player_web-2.0.5
wakelock-0.5.6
wakelock_macos-0.4.0
wakelock_web-0.4.0
webview_flutter-2.8.0
webview_flutter_android-2.8.2
webview_flutter_wkwebview-2.7.1

Closing this as a duplicate of https://github.com/flutter/flutter/issues/77581


Suggestion about "complex_isar_initial_method"

I'm sorry if this is the wrong place to make a suggestion about isabelle-linter.
I heavily use isabelle-linter because it is very helpful to make clean proofs.
When using isabelle-linter, I noticed a bit strange behavior about "complex_isar_initial_method".
If I write proof (rule ***, simp), isabelle-linter correctly displays "complex_isar_initial_method".
However, the following patterns does not produce the same warning.

proof (rule ***, simp_all)
proof (rule ***; force?)

According to the reference "Gerwin's Style Guide for Isabelle/HOL", I think those patters should be avoided as well.
Is my understanding right?

Thank you for the reply.
I appreciate you maintaining this useful plugin.


Initial EncryptPersonalData support

new StoreOptions.EncryptPersonalData setting
new IPersonalDataEncryptor and IEncryptedUserStore interfaces that live in core
new AddPersonalDataEncryptor<MyEncryptor>() helper method on IdentityBuilder
The two concrete user stores in EF now implement IEncryptedUserStore
UserManager ctor throws if EncryptPersonalData is true if store is not IEncryptedUserStore, or if IPersonalDataEncryptor service not found
base IdentityUserContext hooks up ValueConverter using IPersonalDataEncryptor on UserName, NormalizedUserName, Email, NormalizedEmail, PhoneNumber columns when EncryptPersonalData is true.

So all of the normal store tests except one pass when turning this on with a dummy reverse encryptor.
There's a test that that directly uses the DbSet as an IQueryable to look for users like so: UserManager.Users.Count(u => u.UserName.StartsWith("CanFindUsersViaUserQuerable")) which fails because this probably bypasses the converters... @ajcvickers ?
cc @blowdart @javiercn

Add ToList for user manager.userName to work

@HaoK Looks good, IMO you can leave the KeyRotation part out. Having the interface doesn't really add much without an actual implementation. You can have something like the implementation of IPersonalDataProtector do the right thing by emdedding the key into the protected payload (the same way data protection does it today).
You should always protect with the current key and unprotect with whatever key was used at the time.
If a user wants to reencrypt data, they can do so by retrieving the data (unprotecting it) and protecting it with the new key.

The key rotation piece is actually part of core identity, without that they would have to implement that themselves on top of identity apis instead of the user manager taking care of doing lookups and automatically updating the encryption for the key in FindByName/Email.
We don't care what the implementation is, but without this support, they'd have a lot more things to change to support key rotation since they wouldn't be able to just use FindByName/Email anymore

I thought we were going to do a blind index for lookups, isn't that the case?

That's not the plan at this point.  The default protector will use a payload format of "keyid":"data", lookups will just try every key and update the payload to use the current key if needed.
They can still plug in the entire data protector if they want, but this should be good enough to allow them to only plug in encryption algorithms.  I still need to verify everything works using something more complicated than the fake encryption tests in this PR.

@javiercn I added 2 new functional tests for Download/Delete user can you review those, I mostly just copied the TwoFA manage test?  I didn't add any coverage of for custom users since the test website is currently using only IdentityUser, do we need a second test website that's setup more like the DefaultUI sample which lets us test the extensiblity flows as well?

Hi, I'm struggling to make this work with the Data Protection API and @HaoK I need your guidance. My goal is to protect all of the user information: email/username, cellphone and any other property I might have added to the ApplicationUser class and decorated with the [ProtectedPersonalData].
First thing I did was to enable protection of personal data in ASP.NET Core Identity.
services.AddDefaultIdentity<ApplicationUser>(o => {
  o.Stores.ProtectPersonalData = true;
})
.AddEntityFrameworkStores<ApplicationDbContext>()
.AddPersonalDataProtection<LookupProtector, LookupProtectorKeyRing>();

My implementation of the ILookupProtector is the following one. I thought it would be a good idea to use the IDataProtectionProvider to protect and unprotect strings since it would be so easy to implement.
public class LookupProtector : ILookupProtector
{
  private readonly IDataProtector protector;
  public LookupProtector(IDataProtectionProvider protectionProvider)
  {
      this.protector = protectionProvider.CreateProtector("UserData");
  }
  public string Protect(string keyId, string data)
  {
      return protector.Protect(data);
  }
  public string Unprotect(string keyId, string data)
  {
      return protector.Unprotect(data);
  }
}

When a users registers an account, his values are indeed encrypted in the database and shown as clear text when he visits his profile page.
But then, he can't login. I realized that the value returned by the Protect method of the IDataProtector is NOT deterministic. What can I do at this point? Is there any way to make the IDataProtector produce deterministic results? I don't want to leave the email address unprotected and I don't want to roll my own encryptor.
Thank you.

Yeah we probably need to document this at some point cc @blowdart @Rick-Anderson.
But ILookupProtector needs to be stable for logins to work, so you can't use data protection you will have to use something else

Thanks for your response. I'm concerned about this "something else" I might end up using. If you leave it to me, it will be for sure a sub-optimal solution, security-wise.
Instead, I wonder if the Data Protection API could be improved to allow AES with a static IV, as an additional option to the EncryptionAlgorithm enumeration. This way, I could perhaps get deterministic results out of the Protect method. Dictionary attacks could be mitigated by transparently adding/removing some static salt to the string. I know this might sound like a naive solution but I think still better than the one I could roll on my own.
The alternative is to choose proper usernames instead of e-mail addresses. I'm more comfortable with leaving usernames unprotected rather than e-mail addresses. Logins and FindByUsername would work again but then I need a configuration option to tell Identity I want unprotected usernames when invoking the  AddPersonalDataProtection extension method.

I wouldn't view that as an improvement to data protection at all I'm afraid. We deliberately designed it for maximum safety, adding a static IV or making it deterministic would move away from that design philosophy.

I understand you don't want to lower the security of a Microsoft branded API but there's a missing stepping stone here. The whole functionality provided by AddPersonalDataEncryptor is unusable unless I, the developer, provide a working implementation of an ILookupProtector. I agree that having a static IV is not the most secure option but then... how should I do it? Security experts at Microsoft should provide some guidance here. With GDPR enforcement just a few days away, developers will adopt the first quick and dirty way they come up with to implement the ILookupProtector and show their bosses the application is compliant.
Again, the other option is to leave usernames in clear text and, in order to do that, the Protect method should provide some context about what it is that I'm protecting. Maybe a third parameter with the property name would suffice.

I would not believe that Microsoft's commitment to the GDPR and personal data protection in general is limited to only HTTPS by default, Cookie Consent Bar and Default Data Control actions.... and leave this sensitive implementation to the individual develop....
@blowdart
if it's not possible to improve Data Protection Api, EncryptPersonalData should still have a default implementation of ILookupProtector, don't you think?
thank you.

I'm working on some samples, but you'll see when I get them done, that the real recommendation will be "Use encryption at rest on your data". MS SQL and SQL Azure provide encryption at REST with no changes needed to your code. For Linux the various databases there end up saying use full disk encryption on the volumes where databases reside, for which there are multiple options.
The protectors here are really a fallback of last resort.
And there cannot be a default implementation, because it will fall to you to manage keys, both in terms of rotation, size, even algorithms, based on your risk evaluation, which should be part of every GDPR review.

See GDPR Encryption at rest

@blowdart @Rick-Anderson
thank you for your suggestion, this makes me feel more comfortable

Is there any documentation for this yet?


removing all content for SEO

or rather so it doesn't show up in search engines
Signed-off-by: Lamont Granquist<EMAIL_ADDRESS>

@tas50


jeelizFaceFilter callback detectState.detected not working when camera streaming

Probably related to issue #14
Description
I'm trying to use a the face filter to map my face rotation to a circle going across the screen. All works well as expected until I try to use p5.js's createCapture to show the video on the canvas. Then the callback's detectState.detected value will only output very tiny values.
How to replicate
Here's a sketch you can run online.
Rotate your face to move the circle across the screen.
If you uncomment the preload function on top at lines 5-8, it will enable the camera and cause it to stop working.
My config
I'm on Windows 10, running the sketch above on Chrome.

This is a problem with p5.js, not with Jeeliz FaceFilter.
However, I can work on your code and on integration issues, but I charge for this.

Thanks for checking on this.
So to be clear is the problem with p5.js or my implementation?


How do you use the .zip file for Android

Your Question
I know that it has files  but how do i use the files?

try this
https://gofile.io/d/yMHabA
select the desired compiler in the installer

I meant Android


Lit.dev v2 documentation fork (undiscoverable)

Part of https://github.com/lit/lit.dev/issues/1074
First part of Lit.dev v3
This change has no user facing impact on the site, but instead sets us up for the future Lit.dev 3 user facing changes.
This change archives all the current V2 pages into a /docs/v2/** subdirectory, preserving cross linking. I also annotated all the redirects between the page versions so after a v3 switch is added, the pages nicely navigate between the v1, v2, and v3 versions.
Testing
This was tested manually. To test user facing behaviors use the second last preview URL before I removed the v3 selector:

https://pr1083-4bac296---lit-dev-5ftespv5na-uc.a.run.app/

Test by navigating to a page such as styles: https://pr1083-4bac296---lit-dev-5ftespv5na-uc.a.run.app/docs/components/styles/
Note that when switching between v1, v2, v3, the pages remain related to styles.

On the most recent preview I tested that the dropdown doesn't show v3, and that everything works like it currently works in production.
Thank you!
Checklist of Lit.dev 3 work

[x] V2 documentation forked
[x] Redirects between v1, v2 and v3 added.
[x] v2 cross links work correctly

These tasks will be moved into a separate PR:

[ ] Upgrade section is re-written
[ ] Browser Requirements section is re-written
[ ] v3 API works
[ ] Add back in the v3/v2 user facing UI.

@AndrewJakubowicz it'll be a little more work, but I think we should stage this as two separate PRs: one that makes v3 a new "next" version that's not the default, then another PR later that makes v3 the default. This would imply a /v3/ URL space for now, and maybe a banner there that this is pre-relase.
That way we can publish this upgrade guide along with the prerelease blog post while we're still in prerelease mode.
wdyt?

I've created this PR as an archive of the v2 documentation (undiscoverable) which we can later make discoverable. See this preview link (https://pr1083-4bac296---lit-dev-5ftespv5na-uc.a.run.app/) for the v1/v2/v3 functionality - although note that the default content is still v2 and not updated to v3. This was to test that everything works & make a change that can be landed.
Regarding your pre-release strategy - I love it! Maybe to reduce amount of work, wdyt about doing the following for the "next" version PR:

Make a banner announcing the pre-release of Lit 3. Link to a /docs/v3/upgrade guide or blog post that specifically addresses how to try out Lit 3.0.

Then when Lit 3.0 drops, unhide the v2 documentation & land the v3 docs(upgrade & requirements)/api?

I was a bit confused about what hiding v2 meant but that is hiding the v2 fork, and keeping the "v3" (which is really our existing v2 doc) as the default (and v2 dropdown)?
I agree with Justin that while more work, setting up the /v3 namespace right now (and hidden) seems better, and make our current default go to /v2.

Can we do this without copying, but moving the current docs to /docs/v2/ and updating the config to generate /docs/* and /docs/v2/* output?
I think when we add v3 we might want to have /docs/v1/, /docs/v2/ and /docs/v3/ folders, and then a config option for what the current docs version is.

Closing this PR. Thank you @justinfagnani for your awesome guidance here. Will pursue the path you suggested in a separate PR.


Stop using outdated manifest stanza; use random-route instead

Makes sense, going to test this and make sure it works for me locally, then will merge.

I've merged this into master, and it's now available in cfenv 1.2.0 on npm.
Testing it, I see that the "random-route" is now stable across pushes, which I'm pretty sure was not true with the older method, which is awesome!
Thanks so much for your contribution!


[#260] formatting of text messages on console

description

[x] textual format of messages on console

checklist

[x] compiles (cabal new-clean; cabal new-build)
[x] tests run successfully (cabal new-test)
[x] documentation added and created (cd docs; nix-shell --run make)
[x] link to an issue (#260)
[x] link to an epic
[x] add estimate points
[x] add milestone (the same as the linked issue)

rebased on develop

bors try

bors r+


Issue 27/add instanceprofile api

Something to note is that I can't really write integration tests for User-related methods because I cannot create/delete databricks users.

@reillydj hey can you up the middle version of the pom? I remember previously somebody suggested that whenever we modify an enum. Plus this is additional endpoints and features which I believe is supposed to be a middle version increase anyways.


Cross-talk highlighting and reset

I've noticed that the highlight capabilities between the graph and map using crosstalk do not display as expected. Is there additional specifications needed? Point selected on graph use to highlight on map (and vice versa)- but now does not seem to work. Also is there button to reset after selection has been made?

Or maybe I'm crazy and it never worked. In any case, it should be fixed now and the changes are pushed. I added a bullet point in the rmd with instructions for selecting and deselecting points. I'm not sure if there's a way to get it to work in the opposite direction (by selecting a plot in the map to get it to highlight a point in the graph) but I can look into it if you would like that funcitonality.

It did work before, but I recall it being somewhat buggy. I think I tried to fix the bug at one point and likely just broke the functionality altogether. It would be great if it could work oppositely as well - but not a deal breaker if cannot!


write_certificate is not exposed

I'd like to use this function to load in a certificate generated by an external authority but the only way I see to write a certificate is to generate a self signed one.
Is there any harm is building a pub wrapper around the pub(crate) write_certificate function to allow this?

I think it would be fine to change write_certificate to be pub.
Curious what @str4d thinks.


WIP - feat($resource): add support for cancelling requests using promises

Previously, it was not possible to use a promise as value for the timeout property of an action's config, because that value would be copied before being passed to $http.
This commit introduces a special value for the timeout, namely 'promise'.
Setting an action's timeout configuration property to 'promise', will create a new promise and pass it to $http for each request. The associated deferred, is attached to the resource instance, under $timeoutDeferred.
Example usage:
var Post = $resource('/posts/:id', {id: '@id'}, {
  query: {
    method: 'GET',
    isArray: true,
    timeout: 'promise'
  }
});

var posts = Post.query();   // GET /posts
...
// Later we decide to cancel the request
// in order to make a new one
posts.$timeoutDeferred.resolve();
posts.query({author: 'me'});   // GET /posts?author=me
...

Fixes #9332
Closes #13050

This is a POC to get some feedback.
Still missing/pending (if we decide to move this forward):

[ ] Documentation
[ ] Decide whether to keep or remove the $timeoutDeferred property once receiving the response.
[ ] Decide how to handle the fact that $timeoutDeferred might not be present.
[ ] Decide if $timeoutDeferred is an appropriate name.

I'm not qualified to review, but my opinion is that this is an excellent solution! Thanks a lot!

@gkalpak I think that the fix in #12657 makes this unnecessary?

Not quite (e.g. see https://github.com/angular/angular.js/pull/12657#issuecomment-134315124 and next couple of comments).
With #12657 you need to recreate the resource "class" every time you cancel a request or it won't work correctly. It's not very practical imo.
This is adding better support for cancelling resource requests.

I see! The feature is a good idea. I am not sure of the API, though.
What if we allowed people to pass in a promise to the instance methods?

I don't feel too strongly about the proposed API, but I would rather avoid the boiler plate of having to pass a promise every time (especially if we can easily avoid it :)).
Maybe we could just expose an extra method for cancelling the request and handle the promise creation/rejection internally (but then there's the overhead of creating a promise every time, just in case the user decides to use it).
I'm definitely open to better ideas though.

Let's have a think about it.

Here's another idea (thinking loud):

Keep the current behaviour, when using timeout: <NUMBER> (in $resource).
Deprecate using timeout: <PROMISE> (in $resource), since we can't make it work correctly.
Introduce a new boolean property (in $resource's config - either per method or per class): cancellable. Iff this is true, we will add a new cancel()/abort() method on the returned instance/array of each method (and handle the promise creation and resolving internally).

--
(I wonder what other libd

@gkalpak I like the sound of that better

From a quick look to other libs (Restangular, Angular Restmod, ModelCore, angular-restResource), they don't seem to have any special handling in place.
So, I'll go ahead and implement a quick POC as described in https://github.com/angular/angular.js/pull/13058#issuecomment-152182903.

Good stuff @gkalpak


Make render.RenderableNodes immutable

Part of #502, #854
Possibly, actually seems to slow things down, slightly... Still within range of being noise...

I'm not convinced this is needed - RenderableNodes collections are never shared or copied, right?  There use is quite restricted to the rendering pipeline.  ps.Maps are going to be slower here than normal go maps.
Worth taking 00afc20 and making it a separate PR though.

Either way this doesn't look urgent for v0.12, so can we park it and discuss later?

Decided to extract refactoring, and remove the render.RenderableNodes immutability bits, as they do not have a performance improvement.


[BUG] Oversized titlebar buttons in latest Nightly build

Describe the bug
the buttons are way too big, in the wrong order and don't hide the original windows buttons
To Reproduce
apply theme in firefox nightly
Expected behavior
buttons should be smaller and not cover back and forward page buttons
Screenshots

https://i.imgur.com/h7oghGi.mp4
Desktop (please complete the following information):

OS: Windows 10 21H2, build 19044
Firefox version Nightly 100.0a1
WhiteSur theme v1.4.1

Thanks for submitting the issue.
Will close as is a duplicate of #64


Adding functionality to "Terms of use" and "Privacy & Policy" Page at the bottom of the page.

I would like to contribute in your website . I want to Adding functionality to "Terms of use" and "Privacy & Policy"

Thanks for creating the issue in WildGuard! The issue has been assigned to you.
Before you start working on your PR, please make sure to:

⭐ Star the repository if you haven't already.
Pull the latest changes to avoid any merge conflicts.
Attach before & after screenshots in your PR for clarity.

We're excited to see your contribution as part of Hacktoberfest 2024! 🎉
Happy open-source contributing! 🚀


Support INameResolver for queueName argument

I'm trying to use this trigger in a WebJobs project where I need to provide the queue name in config settings.  I'll open a PR shortly to support this.

Any chance of getting this merged and published?


[Q] Why does your library require minsdk = 16?

This library is based on Android 4.0 NumberPicker.

Error:Execution failed for task ':app:processDebugManifest'.

Manifest merger failed : uses-sdk:minSdkVersion 15 cannot be smaller than version 16 declared in library [com.shawnlin:number-picker:2.3.0] C:\Users\alvar.android\build-cache\6ed1b3c5c955e4c0bc5a88b40bfa7fe276cae97b\output\AndroidManifest.xml
Suggestion: use tools:overrideLibrary="com.shawnlin.numberpicker" to force usage

Can you help me?

You can upgrade your minSdkVersion from 15 to 16 or add <uses-sdk tools:overrideLibrary="com.shawnlin.numberpicker"/> to your AndroidManifest.xml


automatic dependency management integration

from @acargol

How are you managing your dependencies? Is there anything like Bundler for Sails?

For client-side dependency management, I prefer npm and browserify or webpack. I don't like bower. However, when building Sails apps, I don't use sails to manage the client code at all. I use two different projects one for client (angular / ember / etc.) and one for the server side (sails).


Add POSTGRES_INITDB_ARGS env var

This closes #19 per @tianon's suggestion.
$ docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -e POSTGRES_INITDB_ARGS="--data-checksums" --rm postgres
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are enabled.
...

This also successfully enables data page checksums, but I have not tested it simultaneously with any other arguments.

LGTM

LGTM :+1:


Documentation/comments need to be updated to indicate support for vos0

Thanks for the HAL, this makes migrating from the G4 to the H7 so much less painful!
It appears that between #96  and #120  now have support for vos0, which is particularly helpful to help the STM32H7(2x|3x) series hit 550MHz (!). The documentation for pwr.rs suggests that "The voltage scaling mode is fixed at VOS1", which AFAICT is no longer the case given examples/vos0.rs, specifically let pwrcfg = example_power!(pwr).vos0(&dp.SYSCFG).freeze();. Still working on confirming that this does allow me to hit 550MHz on the NUCLEO-H723, but all signs point to "probably yes".

That's a good point, the documentation is indeed outdated after those PRs. With https://github.com/stm32-rs/stm32h7xx-hal/pull/316 (now on the master branch) I have tested the H725/735 parts at 520MHz!


Update txnbuild to support stellar-core 12 (part 2 of 2)

After v12 vote:

[ ] Release new operation PathPaymentStrictSend to SDK, implementing new v12 operation

This is part of #1737.

In progress in: https://github.com/stellar/go/pull/1776

I realized that this is split into 2 issues. Maybe we can merge this into #1771. I think we can do everything at once:

Add PathPaymentStrictSend (devs should be able to create ops before v12 for testing).
Create PathPaymentStrictReceive that is an alias for PathPayment (done in #1776 as explained in #1771).
Remove PathPayment after deprecation time.

@bartekn we can discuss again if you feel strongly, but in the last discussion we converged on doing this in two steps, and delaying the new operation, because a user calling the new operation would have a bad experience until the vote happens (and it wouldn't be clear why).

OK, we can talk tomorrow, maybe I misunderstand something.

After internal discussion we agree. Let's merge these and release all together, so that developers can try the new operations out on testnet prior to the pubnet rollout.


⚠ Thumbnails API Endpoint has degraded performance

In fefd3af, Thumbnails API Endpoint (https://thumbnails.roblox.com/v1/assets?assetIds=82738847&format=Png&isCircular=false&size=30x30) experienced degraded performance:

HTTP code: 200
Response time: 1103 ms

Resolved: Thumbnails API Endpoint performance has improved in 1ae2f71.


Feat/allow commitish epc

[x] I have read the contribution documentation for this project.
[x] I agree to follow the code of conduct that this project follows, as appropriate.
[x] The changes are appropriately documented (if applicable).
[x] The changes have sufficient test coverage (if applicable).
[x] The testsuite passes successfully on my local machine (if applicable).

Summarize your changes:

Replace home-made RegEx for testing whether or not the version string of electron-prebuild-compile with exact-version.

Since this functionality is not closely related to electron / electron-forge, I think it makes sense to use and external library instead. Even though it is relatively small, there are a lot of subtleties that should have test coverage.
The existing implementation does not correctly identify strings like 1.x, 1.7.0 || 1.8.3, or 1.0.0 - 1.5.0 as a range.

Obtain electron version via new getElectronVersion util, which reads the electron-prebuilt-compile's package.json (using the existing util readPackageJSON), which is guaranteed (by package.json / npm publish semantics) to be in a predictable format.

Why? One reason is that it should make it much easier to try the latest version of electron before a corresponding version of the official electron-prebuilt-compile module is published to NPM.

@malept Oops, I was pushing changes while you were reviewing, sorry. Feel free to edit/fix/squash as needed (you should be able to write to this branch) or to let me know if you want me to make any changes myself. Thanks for checking it out!

Oops, I'll add those missing awaits right now (originally I had it as a sync function).

@malept I've revised, rebased, and pushed. Please take another look when you have the chance.

@jacobq I talked to @MarshallOfSound briefly and we decided to hold off on merging this after we've merged the v6 branch to master, due to the significant amount of refactoring in that branch.

@malept OK, but I hope you'll keep this in mind / come back to it later. Thanks.

We're not going to close this PR :smile: we're more likely going to rebase this on the merged master after v6 is merged.

Also, I'm not immune to this, I have a PR that's being deferred for the same reason.

I'm going to merge this into the 5.x branch and then forward-port the change into the 6.x branch.


Use explicit cast on forwarding in an attempt to make asan happier

What
Hitting some spurious asan issues on ci: Some Linux runs end up spamming AddressSanitizer:DEADLYSIGNAL indefinitely with no additional information.
This started with:

#5544

so this PR tries to wiggle that piece of code a bit by making it more obvious what's happening both to human and machine (? - jury still out on that).
Defined a FwdType alias to keep things shorter
Checklist

[x] I have read and agree to Contributor Guide and the Code of Conduct
[x] I've included a screenshot or gif (if applicable)
[x] I have tested the web demo (if applicable):

Using newly built examples: app.rerun.io
Using examples from latest main build: app.rerun.io
Using full set of examples from nightly build: app.rerun.io

[x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG
[x] If applicable, add a new check to the release checklist!

PR Build Summary
Docs preview
Examples preview
Recent benchmark results
Wasm size tracking

did not work so far https://github.com/rerun-io/rerun/actions/runs/8326523953/job/22782380753?pr=5557

Looks like explicitly disabling worked. Triggering two more manual runs to confirm.
The other changes are a net improvement I think, so keeping them

extra runs:

https://github.com/rerun-io/rerun/actions/runs/8327256543

disabling ASAN for the changed functions doesn't get us any further it seems.

another attempt:

https://github.com/rerun-io/rerun/actions/runs/8328547083
https://github.com/rerun-io/rerun/actions/runs/8328545313


Prompting for password for p12 when there is none

I created my certificates using PEM which creates p12 files without passwords. If I use the p12 on others services it works fine, but Pusher seems to ask for a password, and if I leave it blank i get this error "Unable to read p12 file: PKCS12 data wrong password" and it doesn't show up in the list of certificates.

Hi @jlubeck,
Thanks for reporting this issue. I was able to reproduce it, but unable to fix.
Unfortunately Apple's p12 import function (https://developer.apple.com/reference/security/1396915-secpkcs12import) does not seem to support files without password. I did add support for it in the Mac app though (commit ff57680), but it will results in a password required error.
There's an Open Radar post on this issue: http://www.openradar.me/22909471. This is referenced on stackoverflow: http://stackoverflow.com/questions/39392338/secpkcs12import-when-p12-requries-an-empty-passphrase
For now I suggest adding a password.

Ohh, that sucks on Apple's side... glad to hear you were able to have a look at it though. Hopefully they will fix it eventually. Thanks for an awesome tool!

you're welcome, thanks for using it :)


Implement a test runner for Hardhat stack traces & console log messages

Follow up from https://github.com/NomicFoundation/edr/issues/241
Implement a test runner for Hardhat stack trace console log messages.
Definition of Done
Implement a test runner for these JSON files. Also, see README

See https://github.com/NomicFoundation/edr/issues/266#issuecomment-2100963981


Device stop to work in HA after renaming Name/Location in Control Panel

Hi,
Adding device works perfect, Shows in HA, Integration as well (when doing manual/cp/ha on/off it updates
on both), But when changing the Name/Location in Control Panel, Rarely the update gets to HA,
And the integration dies for that device (HA shows it switch off, but nothing happens and set back to On).
Looking on MQTT Explorer, Even when renaming the device, From time to time you will see the old name (DeviceID_x) appearing there.

@ezeitoun It's correct, you should rediscover it once you change it's name/location as it's name/location are part of it's topic.
For the old entities, you should use mqtt explorer to delete the retained topics if there are

Thanks, Rediscover does partial help,
When its a simple switch, Everything is updated correctly,
When its a "Smart" Switch (which include sensors), The Switch entities of the device will be update,
The sensors wont be updated (will either remind with old name, or when deleted and rediscovered wont appear at all)

You have to press on rediscover node, have you pressed that?

Yup, Rediscover Node bring only the Switch Entities, Not the Sensor ones.
On MQTT Explorer i do see the sensors are being created, So we are good on that part.
MQTT Explorer:

Not sure if its an issue/bug or settings, Just in case..., My Settings are:

Thanks for the Help, Appreciate it!

@ezeitoun Could you please try to restart the application?
Usually my suggestion is to first address all devices (give them the name/location and modify the discovery if needed) than store the device configuration of each node, when you change the name/location of a device usually by doing rediscover node it fix the new topics but if not an application restart should do it. Let me know and thanks for sponsoring my work :heart:

Nothing like a good Restart! 8-)
(Not sure why the sensors are unknown as for status, but i dont think its related to you)

Good Product + Great Support = Sponsoring, You Deserve it!

Not sure why the sensors are unknown as for status

Maybe they are the old ones?

MQTT Retain did the job...
(well at least as soon as i modified i see the values)
Is there a document regarding all the settings to make sure i have the optimized configuration ?

@ezeitoun Here you find everything needed: https://github.com/OpenZWave/Zwave2Mqtt#robot-home-assistant-integration-beta