Document the Streaming API

While there is support in the Ruby code for the Streaming API, the docs are completely empty. We should fix that by filling them in. Anyone want to help?

Streaming seems to not work on that endpoint anymore. Even for old cars.
Opening Stream to: https://streaming.vn.teslamotors.com/stream/XX/?values=speed,odometer,soc,elevation,est_heading,est_lat,est_lng,power,shift_state,range,est_range,heading
Stream open
Stream data:
Stream data: <title>502 Bad Gateway</title>
Stream data:
Stream data:
502 Bad Gateway

Stream data:
nginx
Stream data:
Stream data:
Stream error: nil```

Having the same issue as @jonasman

Seems like i can get a few values out of the streaming before the 502 starts.

I see one possible thing to test out here:
https://www.lifewire.com/502-bad-gateway-error-explained-2622939
«Clear your browser cache [and/or cookies]. Outdated or corrupted files that are being stored by your browser could be causing 502 Bad Gateway issues ... »

I tried the streaming API recently and it still doesn't work.

The HTTP streaming endpoint is completely dead. It 502's for my car, which was still supported as of last year.
I've updated the gem to use the new WebSocket endpoint: 353cfc57bdcf7e411dcc115b5637da6c582536f7

My experience is that the 502's began in May/19 and have persisted.  At that point I switched to a web-socket streaming implementation by dirkvm  ( https://github.com/dirkvm/teslams )  forked from teslams.  That worked until late summer at which time I began getting "data:error" messages indicating "vehicle_disconnected".  At that point the code close the socket and retires with no success.  The complete message from the data:subscribe attempt is:
data:error { msg_type: 'data:error', tag: 'XXX my vid XXX', value: 'disconnected',  error_type: 'vehicle_disconnected' }
Are there any current streaming endpoint descriptions available, that are known to work?

Mine works. It's actively running as I write this, for a project I'm working on right now.

Will look into integrating into my processes.  Thanks.

I will also implement this on TeslaSwift lib as soon as i get some time. Will report if it works. The HTTPSteamEvent stream is totally dead even for older cars.

A couple of things to add to this:
You can send a "data:subscribe_oauth" msg_type instead of "data:subscribe". This uses the oauth token string instead of the token returned with the vehicle details.
If you respond to the "data:error" with a "data:subscribe_oauth", it seems to continue sending data so it does not look like closing and reopening the socket is necessary. Probably works the same with "data:subscribe" but I have not tested that.
My guess is that there's probably some message that needs to be sent in response to the "data:update" to indicate that further updates are wanted. Either that or the vehicle sends one update and then disconnects if it is parked and locked? (Would make sense - I've not yet tried streaming any data and had the vehicle in motion but I'll give that a try at some point.)

I've been playing with this today and i've come to the conclusion that data:error vehicle_disconnected is a time out issue, albeit a very small timeout window.
If I connect to the stream while my car is parked, I get 1 result with data, then a couple of seconds later I get the data:error.
If i'm driving when I connect, it streams fine.
If I come to a stop, but have my heating on, it streams fine as the power(kw) keeps changing between 1 & 2 which pushes data to the stream.
If I then turn off my heater, so none of the data from the stream is changing (Same location, same speed 0, same power etc..) then a few seconds later it disconnects with the error.
I can't yet confirm if the timeout is coming from Tesla or from my code. The web socket connection is greeted with {"msg_type":"control:hello","connection_timeout":0} so I wonder if there is a control command we can send to extend the timeout, or else an alternative command we can send to keep the socket alive before timeout.

There is a control:ping on the Summon API. You might try that.

Yes I tried that amongst other things. I tried sending the ping every second and although it didn’t return any error,  it still timed out after 10s.
The connection to the socket doesn’t close, just the subscription to vehicle data.
What works is simply sending the subscribe command again after you receive vehicle_disconnected.

Hi,
My experience with the timeout matches yours.  I'm using the dirkvm version of streamws which originally closed the socket when a "data error" occurred.  I removed the close and now the program just carries on after logging the error.  Seems to work just fine except for the occasional hiccup.

From: ddaddy<EMAIL_ADDRESS>Sent: November 19, 2019 4:54 AM
To: timdorr/tesla-api<EMAIL_ADDRESS>Cc: TheSpaceCowboy<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [timdorr/tesla-api] Document the Streaming API (#97)
Yes I tried that amongst other things. I tried sending the ping every second and although it didn’t return any error, it still timed out after 10s.
The connection to the socket doesn’t close, just the subscription to vehicle data.
What works is simply sending the subscribe command again after you receive vehicle_disconnected.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/timdorr/tesla-api/issues/97?email_source=notifications&email_token=AKJNQEETE73UBEVO3FRF2BTQUOZUXA5CNFSM4GJYU3PKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEENRZQQ#issuecomment-555424962, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKJNQEHK3OW72LLXIYGLOETQUOZUXANCNFSM4GJYU3PA.

My partner is borrowing the car today for a short journey so I had an opportunity to observe the data whilst the car was being driven.
Sometimes after a vehicle_disconnected message, even subscribing again does not seem to work, so my previous suggestion might not be useful after all.
Observations:
Generally you'll get 1 or 2 messages if the car is parked and awake before it sends the vehicle_disconnected message. Certain things trigger more frequent messages to be sent (the car being in motion is the obvious one, but simply having the driver-side door open seemed to result in more frequent messages being sent.)
While the car was being driven, I'd get multiple messages per second. The car stopped at a red light and this resulted in a vehicle_disconnected after I received a message indicating its speed being 0. My script sent several subscribe messages 10 seconds apart, and after a minute gave up and reconnected. It then did not receive any messages after attempting to subscribe (on the new connection), and again sent several subscribe messages 10 seconds apart, giving up after a minute. The third connection attempt was successful and I started getting messages, however the car was already moving and had made significant progress along the road at that point.
I suspect if I had just disconnected and reconnected immediately each time upon receiving the vehicle_disconnected message, this would've worked.

I want to confirm if anyone else is seeing this behaviour when using the streaming API in a third-party app like Teslamate while also using the (official) Tesla mobile app.
It appears the streaming API only supports one client at a time, which the Tesla mobile app also uses. While a third-party app is using the streaming API, the Tesla mobile app seems to have issues with the location display while the car is in motion. Frequently it would error (I assume it doesn't handle error responses well) and display the car's location in a random location on the map like in the middle of the ocean or in another continent.
For example while my car was in Melbourne, Australia, it is displayed as in the middle of the Pacific Ocean

If this is a widespread issue, I think it is worth adding some warnings to this API that it may cause issues with the Tesla mobile app.

I want to confirm if anyone else is seeing this behaviour when using the streaming API in a third-party app like Teslamate while also using the (official) Tesla mobile app.
It appears the streaming API only supports one client at a time, which the Tesla mobile app also uses. While a third-party app is using the streaming API, the Tesla mobile app seems to have issues with the location display while the car is in motion. Frequently it would error (I assume it doesn't handle error responses well) and display the car's location in a random location on the map like in the middle of the ocean or in another continent.
For example while my car was in Melbourne, Australia, it is displayed as in the middle of the Pacific Ocean

If this is a widespread issue, I think it is worth adding some warnings to this API that it may cause issues with the Tesla mobile app.

I have noticed same issue and my Android phone doesn't show real time update when my family member is driving Tesla Model 3. I am using TeslaMate which uses real time streaming I guess and that shows real time update on my computer/browser but not Tesla App on my Android phone.

I was doing something with the streaming API the other day, and half way through the day I stopped getting any data returned from the car.
I've not been able to get any streaming data since.
I receive the
{"msg_type":"control:hello","connection_timeout":0}

handshake and send the subscribe command
{"msg_type":"data:subscribe","tag":"1305444632","token":"Base64(<email:1stToken>)","value":"speed,soc,power"}

but get absolutely nothing until it times out.
I've tried sitting in the car and messing with the AirCon, driving forwards and backwards. Nothing.
Anyone else seen this before?

One user reported a bug in the car location in the app. The car was always in fremont lol.
He contacted Tesla and tesla forced a map update. It could be that your car has a buggy map and this can happen too.

On 20 Oct 2020, at 13:56, Darren Jones<EMAIL_ADDRESS>wrote:
I was doing something with the streaming API the other day, and half way through the day I stopped getting any data returned from the car.
I've not been able to get any streaming data since.
I receive the
{"msg_type":"control:hello","connection_timeout":0}
handshake and send the subscribe command
{"msg_type":"data:subscribe","tag":"1305444632","token":"Base64(email:1stToken)","value":"speed,soc,power"}
but get absolutely nothing until it times out.
I've tried sitting in the car and messing with the AirCon, driving forwards and backwards. Nothing.
Anyone else seen this before?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/timdorr/tesla-api/issues/97#issuecomment-712767715, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANOMXTBVAW2YYGFFQMOOB3SLVUE3ANCNFSM4GJYU3PA.

Thanks.
The Tesla app seems to work fine with the correct location. I just get absolutely nothing from the socket without any changes.
I haven't had an update for a while, so hoping things aren't just stuck in the car.

I keep seeing so many conflicting stories on this subject.  Is this something that is still working, any links to something working for android or Windows (I am clueless on ruby)?

Despite combing through others code in languages/dialects I have no experience with, but confident I've implemented the same interface (in C++) Ive seen nothing except  endless {"msg_type":"control:hello","connection_timeout":0} on the streaming websocket. And just today I'm now getting a 408 on the (REST) wake_up endpoint unless I first poke the API with the Tesla app, whereas It was working a week ago

@coleged, I got some code going today but looks like the wss stopped working due to tesla changing something or some kind of problem.  I am not the only one getting the error from what I am reading, which is "Basic Auth Disabled".  You get it right after you send the "subscribe" request.

They must have shut down the basic auth. Send the oauth header instead https://github.com/timdorr/tesla-api/issues/97#issuecomment-549613978

Yeah, that makes sense with the new SSO system going live (must have been what took MFA so long). I'll get the Ruby gem updated soon.

Yep, I can confirm you need a new connect message. Here's what we generate in Ruby:
    def streaming_connect_message
      {
        msg_type: 'data:subscribe_oauth',
        token: client.access_token,
        value: 'speed,odometer,soc,elevation,est_heading,est_lat,est_lng,power,shift_state,range,est_range,heading',
        tag: self['vehicle_id'].to_s,
      }
    end

So, just send the msg_type: 'data:subscribe_oauth' and the token set to the Oauth access token you use for the Owner API. Pretty simple change, luckily.
Now I wonder what they're doing with the tokens from the vehicle data... 🤔

@timdorr, How do I get the "Oauth access token you use for the Owner API"?  I cannot seem to find a way to do this.  Thanks

https://tesla-api.timdorr.com/api-basics/authentication

Lol, I swear I tried that yesterday using my vehicle token.   I will try again.

Wow, now I am getting "Got HTTP status: 403 Forbidden for . Response: {"response":"unauthorized"}`","error_type":"client_error"" with a valid bearer token.  Im about to give up.

@timdorr I've updated my code to use the new connect message format but an receiving an Error: Can't validate token response. Any ideas?

You shouldn't be using one of the two tokens entries from the vehicles endpoint. Make sure you're using the same token as the Bearer token used on the Owner API for all endpoints.

@timdorr, thanks for posting the code up, finally figured out what I was doing wrong.  I used the "id" from the vehicle list like you do on the /vehicle API instead of using the other "vehicle_id" and it now works.  Not sure if it is as expected but I do get the following and maybe you could tell me if that is what happens?
**This is with the car parked and awake.

Connect to the socket and get "{"msg_type":"control:hello","connection_timeout":0}"
Send the "subscribe_oauth" message.
Get 1 "msg_type":"data:update" message from the server with all the data fields populated
About 10 seconds later I get a
""msg_type":"data:error","tag":"xxxxx","value":"disconnected","error_type":"vehicle_disconnected"}"
60 seconds later my socket gets closed with CloseCode=1002 Reason="No more WebSocket frame from the server."

Really appreciate the help

On my MCU1 Model S, I get a single frame of data for my car's current position. I don't know the behavior of other vehicle and/or MCU versions. They may not emit any data if they are in deep sleep.
Unfortunately, I haven't yet gotten the MCU2 upgrade, so I can't verify any behavior differences. I would recommend trying it with the car in drive. If you can get someone else to drive it around for a bit while testing, that would be best.

Ok, I will do that, thanks again for all the help!

@timdorr I had already made the change to use the bearer OAuth token from a successful login. That is why I am confused.
{
msg_type: "data:subscribe_oauth",
token: options.authToken,
value: options.values.join(','),
tag: options.vehicle_id.toString()
}

@timdorr OK, belay that. in debugging through I discovered that further upstream I was not passing in the correct token. I am now receiving a data:update response. Thanks!

@timdorr, thanks for posting the code up, finally figured out what I was doing wrong.  I used the "id" from the vehicle list like you do on the /vehicle API instead of using the other "vehicle_id" and it now works.  Not sure if it is as expected but I do get the following and maybe you could tell me if that is what happens?
**This is with the car parked and awake.

Connect to the socket and get "{"msg_type":"control:hello","connection_timeout":0}"

Send the "subscribe_oauth" message.

Get 1 "msg_type":"data:update" message from the server with all the data fields populated

About 10 seconds later I get a

""msg_type":"data:error","tag":"xxxxx","value":"disconnected","error_type":"vehicle_disconnected"}"

60 seconds later my socket gets closed with CloseCode=1002 Reason="No more WebSocket frame from the server."

Really appreciate the help

The socket only sends data when something changes and times out after 10 seconds.
So you get 1 hit of the current state, then updates after that.
Try turning your heating or air con on and you'll get more data as the power use changes. Or drive around so the speed and gps changes.
No data after 10 seconds, it times out. I can't remember if you have to reconnect or just send the subscribe again.

@ddaddy, It appears that if I keep pinging the server every 20 to 30 seconds with the connection it stays connected and then when I wake the vehicle out of a deep sleep I do get a "data:update".  Im curious if there are any other values we can poll from the "value" field, charging would be nice so I could monitor when it starts and stops (mostly curious about stop since I set the start time).

These are the ones I found:
speed,odometer,soc,elevation,est_heading,est_lat,est_lng,power,shift_state,range,est_range,heading,est_corrected_lat,est_corrected_lng,native_latitude,native_longitude,native_heading,native_type,native_location_supported

power will tell you if charging or not as it goes negative. Be wary that Regen braking also makes it go negative.
How are you pinging it? I tried all sorts of ping commands and none seemed to work.

Currently while testing I am using a library called "com.neovisionaries:nv-websocket-client" for android until I get this stuff setup.  They have a func/property you can set called "setPingInterval", I set it to 20 seconds and it has kept me connected.  So see if you can find a similar property in your socket.

@ddaddy, I added some function to monitor what the ping/pong was actually sending and it looks like it is just sending a rolling integer, starting with "1" (Ascii 49), and the server answers with the same integer.

Thanks, I'll try the ping.
Did you make sure something is changing for it to report? Open the Tesla app and switch the air con on or start charging. Or go to the car and open the door.

Tried pings every 4, 5 & 8 seconds, I still get the vehicle_disconnected after 10 seconds and I have to resent the subscribe_oauth message

@ddaddy, Sorry I didnt explain that very well, I still get the vehicle_disconnected (if we didn't then that would mean the vehicle would be awake 24/7) but I can maintain the socket connection using the ping so that I can ask for an update with the data:subscribe anytime I want.

Ah ok. How long have you kept the connection alive for?
I automatically send the subscribe message when I receive the vehicle disconnected message and it tends to stay alive for about 5 minutes.

I let it stay connected for 3 hours without issue.

Greetings!
I have set up a client locally (Eclipse/Java) using the com.neovisionaries:nv-websocket-client library as did @ktoonsez .  I am able to connect and get the control:hello message.
When I submit the data:subscribe_oauth JSON, the stream replies with "Can't validate token."
In the token field, I provide "Bearer ".  I use the same token for the vehicle_data API and receive a valid response.
In the tag field, I provide the value found in the id_s field from the vehicle_data API.
I refreshed the token and created a new token; I receive the same error response from the stream.
I have an M3 purchased new and delivered in Mar 2020.
Thoughts or ideas on what I might try to get past this error?
Thanks!
Joe

Has anyone noticed any issues today with the streaming API?
After I receive the control:hello and I send my data:subscribe_oauth it disconnects. I've tried another 3rd party library and it does the same.

Can anyone please confirm if their streams are still working?
I'm still getting disconnects as soon as I send the subscribe message after having this working for months.

Can anyone please confirm if their streams are still working?
I'm still getting disconnects as soon as I send the subscribe message after having this working for months.

Working fine here

Thanks for confirming. So something i'm doing must be wrong.
I receive the hello, then I send a payload like this:

then the socket disconnects:

Is your payload similar to this? I've tried setting tag as both the vehicle_id and theid_s which is different and longer.
I've also tried with and without Authorization in the connection headers:

It looks like I might be missing a step in the auth process.
I've just proxied a working node library, and the token used is much shorter.
Looking at their auth flow, after getting tokens from the oauth2/v3/token endpoint, they also fetch different tokens from oath/token which gives a shorter token.
I'm going to try and add this to my flow and see if it works.

Fixed it :)
So the OAuth token from https://auth.tesla.com/oauth2/v3/token seems to be all you need for all the POST requests, but the streaming API needs the token from https://owner-api.teslamotors.com/oauth/token.

@timdorr what is the current status of the streaming API? I’m using python and would appreciate some pointers on where to start.

I'm playing with the streaming API. I don't know if anyone has been able to pull the charging data from the web socket. I've been passing code words for at least 30 mins without success.

While there is support in the Ruby code for the Streaming API, the docs are completely empty. We should fix that by filling them in. Anyone want to help?

Forked.  I'll work on it in the next week or so.  This might help some in the mean time.

So for example if it was on a moving transport it would stream.

I believe there is a transport mode, so streaming might be stopped in this particular case.

As far as I have seen, the criteria seems to be "something changing". E.g. speed, power consumption, gps and so on.

@TeslaOwnerTips I'd imagine it's dependent on speed per the linked notes:

You'll notice when the car is in motion (assuming good network connectivity, etc), you'll get a record every ~250ms. But as you roll up to a stop light, that starts to taper off. When stopped at a stop light, the stream of data ends and Tesla will eventually hang-up on your websocket.

GPS coordinates would be less accurate than speed and you'd prob still be in Drive at a stop light (though "Hold" could be a criteria in addition to Park). It also seems record intervals are determined by speed. So prob leaning towards speed here.

Good questions.  I'll present a scenario from one of my recent drives.  See data from log file at the end.  I'll include more examples like this in the PR for the documentation.
@TeslaOwnerTips - to your question specifically, the likely scenario transport service provider will put the vehicle in service mode.  When that happens, any call to vehicle_data or similar endpoint will fail.  If that's a situation you'll need to handle,... you might want to try a GPS tracker or some other mechanism.
General comments about the sample

This is a situation where I approach a red light and stop.  When it turns green, I turn left and continue the drive.  The last few lines of the log cover me coming out of the turn.  A visual is also attached below.
The difference between the timestamp reported in the websocket record (e.g.<PHONE_NUMBER>438 for the 1st record) is roughly 300-350 milliseconds ahead of the log file timestamp at the start of each log line.
In most cases, each subsequent record is received almost exactly 250 milliseconds after the record before it.
As data readers of the websocket, regardless if it's carson, tesla-api, whoever, we do not have control over the velocity of the data.  We simply subscribe and read/handle records as they are fed to us.

Lines 24 - 26

I'm coming to a stop.  Line 24 says my speed is 1 mph.  Line 25 is the first reported record where my speed is 0.  Am I really at 0 mph?  Dunno.  The next record, line 26 is not at its usual 250 milliseconds.  Instead, it's exactly 500 milliseconds (according to the timestamp in the record itself, not the log timestamp).
Maybe I was going < 0.5 mph.  Or maybe the accuracy on the car's GPS was resolved to a better certainty yielding a longitude of -96.619813 instead of -96.619814 on the record before it.
This is just a guess (I haven't calculated this), but I'm assuming since there was a change in longitude, it was enough for their system to report a coursing change from 267 deg to 266 deg.
Take note of line 26's lat/lon of 33.175984,-96.619813.  I'll reference this later.

Line 27

Exactly ten seconds later, a disconnect occurs.  This is a characteristic of carson.  I had experienced situations where longer wait times at a light would never wake back up.  Either Tesla simply stopped talking or there was something stuck in a buffer somewhere, I couldn't read any more data from the socket, despite having continued the drive which should have sent more data.
Given the ten second timeout, you'll see a new websocket connection was established.

Line 30

First record received since subscribed on new websocket.
The timestamp reported in the record (again, not the log timestamp) is somewhere between 10.9 and 11.0 seconds since the last record received before the timeout.
Remember on line 26 I said to note lat/lon values of 33.175984,-96.619813?  Notice the values here on line 30?  Hold that thought...

Lines 31 - 32

The timestamp in the record on line 31 is 9.001 seconds after value reported on line 30.  But line 32 is back to a familiar 250 millisecon interval.
Still at the light.  Stopped.  Only difference appears to be power output increased from 2 on line 30 to 3 and 7 for lines 31 and 32 respectively.

Lines 33 - 35

Looks like I'm moving again.  You can see my speed increased to 5 mph.  Getting consistent 250 millisecond intervals again.
Okay, we need to talk... notice lat/lon is still 33.175984,-96.619813! ... the heck?

So...
I had assumed for a while, you probably noticed this too, the same heading, coursing, lat, and lon values are reported in what appear to be batches of four.  It sounds very reasonable to me that the car's GPS unit only reports on 1HZ frequency.  That's fine.  But this doesn't explain the incongruency between lines 26 and 35 where clearly speed/power values are changing.  You can't blame a lame/stale socket buffer because we tore it down and established a new connection entirely.
The point of all of that is simply to highlight this is for recreational use only.  You'll probably need to write your own patterns for what you are trying to achieve by using this data.  Let's document the basics (I'm working on a PR and will submit to Tim if he thinks any of this has value).  But it's unlikely you'll get a one-size-fits-all implementation out of any library, regardless of the language/interface.  Data doesn't lie, unless the data is lying or has already lied.  And this isn't autopilot or black box data.  Personally, I'm grateful Tesla has allowed us to geek-out on the data we have.
#  Line
01 2022-07-07 00:51:24,764 D cruiser  b'1657180284438,27,17195.7,68,169,270,33.175982,-96.619492,-50,D,235,245,271'
02 2022-07-07 00:51:25,009 D cruiser  b'1657180284688,26,17195.7,68,169,270,33.175982,-96.619492,-48,D,235,245,271'
03 2022-07-07 00:51:25,264 D cruiser  b'1657180284938,25,17195.7,68,169,271,33.175985,-96.619613,-45,D,235,245,271'
04 2022-07-07 00:51:25,515 D cruiser  b'1657180285188,23,17195.7,68,169,271,33.175985,-96.619613,-43,D,235,245,271'
05 2022-07-07 00:51:25,764 D cruiser  b'1657180285438,22,17195.7,68,169,271,33.175985,-96.619613,-40,D,235,245,271'
06 2022-07-07 00:51:26,014 D cruiser  b'1657180285688,20,17195.7,68,169,271,33.175985,-96.619613,-36,D,235,245,271'
07 2022-07-07 00:51:26,264 D cruiser  b'1657180285938,19,17195.7,68,169,271,33.175987,-96.619714,-33,D,235,245,271'
08 2022-07-07 00:51:26,514 D cruiser  b'1657180286188,17,17195.7,68,169,271,33.175987,-96.619714,-30,D,235,245,271'
09 2022-07-07 00:51:26,764 D cruiser  b'1657180286438,16,17195.7,68,169,271,33.175987,-96.619714,-27,D,235,245,271'
10 2022-07-07 00:51:27,014 D cruiser  b'1657180286689,14,17195.7,68,169,271,33.175987,-96.619714,-24,D,235,245,271'
11 2022-07-07 00:51:27,264 D cruiser  b'1657180286938,13,17195.7,68,169,270,33.175988,-96.619778,-21,D,235,245,271'
12 2022-07-07 00:51:27,514 D cruiser  b'1657180287188,11,17195.7,68,169,270,33.175988,-96.619778,-18,D,235,245,271'
13 2022-07-07 00:51:27,766 D cruiser  b'1657180287438,10,17195.7,68,169,270,33.175988,-96.619778,-15,D,235,245,271'
14 2022-07-07 00:51:28,014 D cruiser  b'1657180287688,8,17195.7,68,169,270,33.175988,-96.619778,-12,D,235,245,271'
15 2022-07-07 00:51:28,264 D cruiser  b'1657180287939,7,17195.7,68,169,268,33.175987,-96.619809,-10,D,235,245,270'
16 2022-07-07 00:51:28,515 D cruiser  b'1657180288188,5,17195.7,68,169,268,33.175987,-96.619809,-7,D,235,245,270'
17 2022-07-07 00:51:28,764 D cruiser  b'1657180288438,4,17195.7,68,169,268,33.175987,-96.619809,-3,D,235,245,270'
18 2022-07-07 00:51:29,014 D cruiser  b'1657180288688,3,17195.7,68,169,268,33.175987,-96.619809,-1,D,235,245,270'
19 2022-07-07 00:51:29,265 D cruiser  b'1657180288938,2,17195.7,68,169,266,33.175985,-96.619818,0,D,235,245,268'
20 2022-07-07 00:51:29,514 D cruiser  b'1657180289188,2,17195.7,68,169,266,33.175985,-96.619818,1,D,235,245,268'
21 2022-07-07 00:51:29,764 D cruiser  b'1657180289439,1,17195.7,68,169,266,33.175985,-96.619818,1,D,235,245,268'
22 2022-07-07 00:51:30,014 D cruiser  b'1657180289688,1,17195.7,68,169,266,33.175985,-96.619818,2,D,235,245,268'
23 2022-07-07 00:51:30,264 D cruiser  b'1657180289939,1,17195.7,68,169,266,33.175984,-96.619814,2,D,235,245,267'
24 2022-07-07 00:51:30,514 D cruiser  b'1657180290188,1,17195.7,68,169,266,33.175984,-96.619814,2,D,235,245,267'
25 2022-07-07 00:51:30,764 D cruiser  b'1657180290438,0,17195.7,68,169,266,33.175984,-96.619814,2,D,235,245,267'
26 2022-07-07 00:51:31,284 D cruiser  b'1657180290938,0,17195.7,68,169,266,33.175984,-96.619813,2,D,235,245,266'
27 2022-07-07 00:51:41,284 I cruiser  Disconnected. timeouts=17 disconnects=13 msg_count=6759 waypoints=6759 msg={'msg_type': 'data:error', 'tag': '25...90', 'value': 'disconnected', 'error_type': 'vehicle_disconnected'}
28 2022-07-07 00:51:41,981 I cruiser  Starting stream loop #31. Vehicle('photon' state='online' id=12...89 is_user_present=True miles=17,195 software='2<IP_ADDRESS>' battery_level=68 charging_state='Disconnected' shift_state='D' speed=0)
29 2022-07-07 00:51:41,982 D cruiser  subscribe='{"msg_type":"data:subscribe_oauth","token":"abc...xyz","value":"speed,odometer,soc,elevation,est_heading,est_lat,est_lng,power,shift_state,range,est_range,heading","tag":"25...90"}'
30 2022-07-07 00:51:42,847 D cruiser  b'1657180301861,0,17195.7,68,169,266,33.175984,-96.619813,2,D,235,245,266'
31 2022-07-07 00:51:51,184 D cruiser  b'1657180310862,0,17195.7,68,169,266,33.175984,-96.619813,3,D,235,245,266'
32 2022-07-07 00:51:51,434 D cruiser  b'1657180311112,0,17195.7,68,169,266,33.175984,-96.619813,7,D,235,245,266'
33 2022-07-07 00:51:51,684 D cruiser  b'1657180311362,2,17195.7,68,169,266,33.175984,-96.619813,12,D,235,245,266'
34 2022-07-07 00:51:51,934 D cruiser  b'1657180311612,3,17195.7,68,169,266,33.175984,-96.619813,19,D,235,245,266'
35 2022-07-07 00:51:52,184 D cruiser  b'1657180311862,5,17195.7,68,169,266,33.175984,-96.619813,25,D,235,245,266'
36 2022-07-07 00:51:52,435 D cruiser  b'1657180312112,7,17195.7,68,169,266,33.175984,-96.619819,30,D,235,245,266'
37 2022-07-07 00:51:52,685 D cruiser  b'1657180312362,8,17195.7,68,169,266,33.175984,-96.619819,33,D,235,245,266'
38 2022-07-07 00:51:52,934 D cruiser  b'1657180312612,10,17195.7,68,169,266,33.175984,-96.619819,37,D,235,245,266'
39 2022-07-07 00:51:53,184 D cruiser  b'1657180312862,12,17195.7,68,169,266,33.175984,-96.619819,44,D,235,245,266'
40 2022-07-07 00:51:53,435 D cruiser  b'1657180313112,13,17195.7,68,169,264,33.175981,-96.619860,54,D,235,245,266'
41 2022-07-07 00:51:53,684 D cruiser  b'1657180313362,15,17195.7,68,169,264,33.175981,-96.619860,58,D,235,245,266'
42 2022-07-07 00:51:53,934 D cruiser  b'1657180313612,16,17195.7,68,169,264,33.175981,-96.619860,35,D,235,245,266'
43 2022-07-07 00:51:54,185 D cruiser  b'1657180313862,17,17195.7,68,169,264,33.175981,-96.619860,24,D,235,245,266'
44 2022-07-07 00:51:54,434 D cruiser  b'1657180314112,18,17195.7,68,168,260,33.175971,-96.619938,22,D,235,245,264'
45 2022-07-07 00:51:54,684 D cruiser  b'1657180314362,19,17195.7,68,168,260,33.175971,-96.619938,22,D,235,245,264'
46 2022-07-07 00:51:55,177 D cruiser  b'1657180314862,20,17195.7,68,168,260,33.175971,-96.619938,22,D,235,245,264'
47 2022-07-07 00:51:55,427 D cruiser  b'1657180315112,20,17195.7,68,168,252,33.175950,-96.620034,23,D,235,245,260'
48 2022-07-07 00:51:55,681 D cruiser  b'1657180315362,21,17195.7,68,168,252,33.175950,-96.620034,39,D,235,245,260'
49 2022-07-07 00:51:55,934 D cruiser  b'1657180315612,22,17195.7,68,168,252,33.175950,-96.620034,45,D,235,245,260'
50 2022-07-07 00:51:56,185 D cruiser  b'1657180315862,23,17195.7,68,168,252,33.175950,-96.620034,38,D,235,245,260'
51 2022-07-07 00:51:56,422 D cruiser  b'1657180316112,23,17195.7,68,168,240,33.175913,-96.620131,20,D,235,245,260'
52 2022-07-07 00:51:56,684 D cruiser  b'1657180316362,24,17195.7,68,168,240,33.175913,-96.620131,17,D,235,245,260'
53 2022-07-07 00:51:56,934 D cruiser  b'1657180316612,24,17195.7,68,168,240,33.175913,-96.620131,31,D,235,245,260'
54 2022-07-07 00:51:57,184 D cruiser  b'1657180316862,25,17195.7,68,168,240,33.175913,-96.620131,44,D,235,245,260'
55 2022-07-07 00:51:57,442 D cruiser  b'1657180317112,26,17195.7,68,168,229,33.175875,-96.620173,57,D,235,245,253'
56 2022-07-07 00:51:57,685 D cruiser  b'1657180317362,27,17195.7,68,168,229,33.175875,-96.620173,65,D,235,245,253'
57 2022-07-07 00:51:57,936 D cruiser  b'1657180317612,28,17195.7,68,168,229,33.175875,-96.620173,69,D,235,245,253'
58 2022-07-07 00:51:58,172 D cruiser  b'1657180317862,29,17195.7,68,168,229,33.175875,-96.620173,71,D,235,245,253'
59 2022-07-07 00:51:58,422 D cruiser  b'1657180318112,30,17195.7,68,167,212,33.175773,-96.620270,73,D,235,245,226'
60 2022-07-07 00:51:58,673 D cruiser  b'1657180318362,32,17195.7,68,167,212,33.175773,-96.620270,75,D,235,245,226'
61 2022-07-07 00:51:58,927 D cruiser  b'1657180318612,33,17195.7,68,167,212,33.175773,-96.620270,75,D,235,245,226'
62 2022-07-07 00:51:59,184 D cruiser  b'1657180318862,34,17195.7,68,167,212,33.175773,-96.620270,69,D,235,245,226'
63 2022-07-07 00:51:59,434 D cruiser  b'1657180319114,35,17195.7,68,167,203,33.175635,-96.620357,69,D,235,245,213'
64 2022-07-07 00:51:59,684 D cruiser  b'1657180319362,35,17195.7,68,167,203,33.175635,-96.620357,71,D,235,245,213'
65 2022-07-07 00:51:59,935 D cruiser  b'1657180319612,36,17195.7,68,167,203,33.175635,-96.620357,73,D,235,245,213'

@mloyd thank you for your hard work on this.
So we need FSD to become street legal so that we can sit in the car to code and debug while in motion 😂 or wait for our significant other to go for a drive 😀
Personally I'm interested in streaming to see if a pi zero with a relay could be used to open the garage door for me when my Tesla gets close enough. I'm currently using our arrival home on Apple Home which works quite well. It also opens the garage door when we come home from a walk but I changed my habits to use the garage door to enter the house. I've tried BLE devices as well. I've also tried a dedicated iPod Touch placed in the car but I need to get back to that since I didn't debug why it didn't work.

Teslamate, MQTT, Nodered are a set of perfectly working softwares for this
on my raspberry to open the garage door but also monitor my calendar to set
the preconditioning, charge amps and timing depending on the location and
power company contract, and much more... like overheat protection while
sentry is active, smartphone remote, ...
Le ven. 15 juil. 2022 à 17:29, TeslaOwner.tips @.***> a
écrit :

@mloyd https://github.com/mloyd thank you for your hard work on this.
So we need FSD to become street legal so that we can sit in the car to
code and debug while in motion 😂 or wait for our significant other to go
for a drive 😀
Personally I'm interested in streaming to see if a pi zero with a relay
could be used to open the garage door for me when my Tesla gets close
enough. I'm currently using our arrival home on Apple Home which works
quite well. It also opens the garage door when we come home from a walk but
I changed my habits to use the garage door to enter the house. I've tried
BLE devices as well. I've also tried a dedicated iPod Touch placed in the
car but I need to get back to that since I didn't debug why it didn't work.
—
Reply to this email directly, view it on GitHub
https://github.com/timdorr/tesla-api/issues/97#issuecomment-1185655723,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADUNWNKFLCX57GJ6774RLUTVUF7VJANCNFSM4GJYU3PA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

I also try to use the Streaming API.
I send this data to authenticate:
{
   "msg_type": "data:subscribe_oauth",
    "token": "eyJh...", // access_token from login
    "value": "speed,odometer,soc,elevation,est_heading,est_lat,est_lng,power,shift_state,range,est_range,heading",
    "tag": "9296..." // my vehicle_id
}

But I always get a 403 back :( :
{
    "msg_type":"data:error",
    "tag":"9296...",
    "value":"owner_api error: `Got HTTP status: 403 Forbidden for . Response: {\"response\":\"unauthorized\"}`",
    "error_type":"client_error"
}

Any ideas why this happens?

arrrggg... Found it...
It's not the vehicle.id that we need for the API Requests, it's the vehicle_id...

Is here some actually using the streaming API, because I have some questions...
It seems, that the streaming API is just for driving and charging, right?
1.) Detect the state:
I detect, that the car is driving, when the shift state is != nil.
I detect, that the car is charging, when shift state is == nil and power is < 0.
Is this correct?
2.) About when to connect to the streaming?
When the car goes to sleep, the WebSocket closes.
How do you decide, when to reopen the connection?
3.) Is it for charging?
When Charging, after some time the API stops senden data. So perhaps, the streaming API is just for driving?
4.) What data does the streaming API offer?
Can I request more data than this: speed,odometer,soc,elevation,est_heading,est_lat,est_lng,power,shift_state,range,est_range,heading
Thanks for you help :)

Hello @Urkman!
The streaming code is an example of how to get streaming up and running.  As you pointed out, it would require additional code to detect specific states of the vehicle's operation.
In my experience, Tesla stops streaming when there is no change in the streaming data.  I've seen the streaming stop when the vehicle remains at a stop light for longer than 10 seconds.  Additional code is required to determine if the streaming should start again.
I wrote my streaming in Java based on this model.  I implemented a state machine to accommodate vehicle operation while driving.  I found it both complex and rewarding to develop.  It is still not perfect.
Joe

Hello @Urkman!
I have also seen the streaming stop while driving.  When that happens, the socket receives data:error message.  I start a timer that waits 20 seconds before attempting to subscribe again.
If there is no response, I wait 60 seconds and try again.  If there is no response, I wait 80 seconds and try again.  If there is no response, I close the socket.
This generates an onStateChange event on the socket where I disconnect from the socket.  This generates an onDisconnect event on the socket where I set the socket object to null and create another socket object.  Everything starts again on the new socket.
It is a lot of messing around but usually, the socket responds after the first attempt to subscribe.  Open to suggestions for simplifying the error recovery process.  I have read somewhere that waiting to reconnect is the best method to start the subscription again.  I have been using a third-party socket implementation that I would like to simplify but I have been distracted by other projects.
Joe

Curious if the disconnects are from network issues between cell towers. The behavior may change when connected to mobile hotspot. Without more detail on how the output signals are generated, we're all literally just (intelligent-ish-ly?) guessing.  We only get to see the outputs after all.


fipsEnable should be consistent with previous name

fix for issue ibm-ingress-nginx operator fips param should be consistent with previous name
Signed-off-by: qpdpQ<EMAIL_ADDRESS>

/retest

/retest


Refactor: Settings SCSS

Resolves #1048
Proposed Changes

Broke SCSS out into styled-components. I needed to keep the input-slider classnames for the 3rd party component, but I broke it out into its own file and included it in PollingSliderContainer so it's not too hard to read.
Deleted SettingsGrid.scss and references

How to Test

Navigate to SettingsGrid, check that there are no regressions

Please update your snapshots before merging.


Find a method to add conda installation to NGC images, etc.

The NGC images currently have no method of installing conda packages during runtime because they have a non-base conda environment installed.
While this can be solved by rebuilding the image with the new requirements included, this is time-consuming.
Find a way to use the base environment, which is the only one capable of installation, in the runtime environment.
However, this may cause trade-offs with having a clean new environment with only the packages specified in the environment.yaml file installed.

Simply installing mamba into a non-base environment simply will not work.

This tuns out to be too difficult.


numba_issue1

Hey @jstac , I have added x and y labels to two graphs in the lecture numba and set the font size of the labels to 12.

Thanks @Harveyt47 .
@shlff and @sayaikegawa , could you please add your review?  (You can just say "looks OK" if you think it's fine.)

Hi @jstac , it looks good.

Hi @jstac , it looks ok.

Thanks @Harveyt47 @shlff and @sayaikegawa, I'll merge now.


question  about drive command

Hi, thank you for sharing the great work!
How does the "drive command" integrate into the current task?  ''drive command"means turn left、turn right or go ahead command.
Bests.

Hi @dswangabel, thank you for your interest. We don't need the "drive command" in our task because we already have a pre-planned route for the vehicle to follow. The simulator provides us with the necessary route information, including "turn left" or "go ahead" instructions. Therefore, you don't have to worry about giving any driving commands.

Thanks a lot !  If " pre-planned route " that you said is  the  'data['route_lanes']'  corresponding  in the project code?  And i also have some questions? (1)I visualize the data output, the output result of lane looks curved，so i add the maximum number of points per feature and  change the interpolation method, but It doesn't seem to make a significant difference.  (2)The processed data doesn't seem to match the 2.8 million data points mentioned in the paper, despite using the provided data processing script for the full NuPlan data.

About the "pre-planned route", the data['route_lanes'] comes from the dataset but differs slightly from the route used in the planner. I added the "route_lanes" in the predictor to better predict the initial plan of the ego vehicle, but to ensure the planning performance, you'll need a more sophisticated path planner to generate the route or reference path.
About the curved lanes, I don't know the exact reason, and it seems from the dataset itself. But I don't think it should mess with the model's performance.
As for the amount of data, how many data points do you have? We didn't set any specific scenario types in the challenge and used all the data we could get. If you're looking for more data points, you could try removing the scenario type restriction.


wish: toolkitize

After reviewing the source code to create #8, I can see how personal the project is.
The concept of "helix-wezterm.sh" is generally useful, but a number of the specifics here for the commands used are rather personal choices.
I wonder if it's practical to split this project into two layers:

First, a toolkit that makes it easy create splits and interact with Wezterm from Helix.
Second, something more like a configuration layer that declares which commands you want to run and whether they are split to the side or down.

This 100%. I would love some sort of configuration overlay for this set up. Btw I love it, use it as daily driver! Very cozy 🛏

@markstos Thank you for your suggestion.
Could you please try this and let me know what do you think?


Why does fastward in Firefox stick to old version 0.1946

Domain
https://www.google.com.br/
Link
https://www.google.com.br/
Version
0.1946
What browsers are you seeing the problem on?
Firefox
What OS are you seeing the problem on?
Linux
Anything else?
Olá pessoal
Hey guys
Why does Firefox have version 0.1946, while on the official page the application available has FastForward version 0.2219?

Olá :wave:,
O Firefox suspendeu a nossa extensão temporariamente, mas em breve ela voltará a ser atualizada na loja de extensões do Firefox.
A extensão, mesmo em uma versão antiga, vai funcionar corretamente, ja que o script para pular os sites é obtido pelo Github, e é atualizado automaticamente.

@princessmortix why did FF remove the extension? Same response as Google?
¿Por qué FF eliminó la extensión? ¿Misma respuesta que Google?

It is not being allowed to install the extension manually on FF, as described on the website:
https://github.com/FastForwardTeam/FastForward/blob/main/INSTALLING.md#troubleshooting
See attached error message:

Firefox does not allow unsigned extensions; you can use Firefox Nightly or Developer Edition until we fix the issue with Mozilla.


fix units of time objects

Fix units of time for chrono objects and throttled messages.
This is one of the fixes that we made in experiment-20201208-final

@mitsudome-r @TakaHoribe FYI you can use rclcpp::Duration::from_seconds (https://docs.ros2.org/foxy/api/rclcpp/classrclcpp_1_1Duration.html#a39ce5aa6d0bb190d7d59857aa7a9af1a) to convert from a double to an rclcpp::Duration and from there to a std::chrono::Duration or seconds or nanoseconds.


[3.x] Use qualified name

[x] Changes have been thoroughly tested to not break existing functionality.
[x] New functionality has been documented or existing documentation has been updated to reflect changes.
[x] Visual changes are explained in the PR description using a screenshot/recording of before and after.

When following the (Docs)[https://filamentphp.com/docs/3.x/panels/resources/relation-managers#searching-the-options-to-attach-across-multiple-columns] on how to use AttachAction, an error occurs: SQLSTATE[23000]: Integrity constraint violation: 1052 Column 'id' in where clause is ambiguous.
This PR makes sure to always qualify column name for AttachAction & AssociateAction.

There is another issue regarding AttachAction, when using getLabelUsing it doesn't work, one great example is using first_name, last_name on the database and using name attribute on the back-end without using recordTitleAttribute.


Getting S/N and Port #

Hi  ArekKubacki,
I have a HoyMiles install and I have been playing with the ModBus TCP directly.
I figured out how to get the S/N and Port numbers from the data.
Here is how I did it.
You need values 0-3, convert each of them to binary. Zero pad so they are length 16.
So now you will have 4 values that look like this:<PHONE_NUMBER>010001
Concatenate them together so you have something like this:
0000110000010001000011000001000100001100000100010000110000010001
Next split them into chunks of 4 so you have:
[0] = 0000
[1] = 1100
[2] = 0001
[3] = 0001
. . .
[15] = 0001
Convert chunks 2-13 to decimal and you will have the S/N, convert chunk 15 to decimal and you will have the port #.
Please let me know if you have any questions, I hope this helps so you can include S/N + Port in a future version.

Hello,
Please tell me why you need the S/N information in HA? Currently I don't see any point in creating an entity in HA that doesn't change and has a constant S/N number.

The idea is to show the production of each panel and their performance over time. You show each panel, but don't identify it by S/N + Port which is what you would need if a panel needs service or replacing.
I don't use this for myself at all, I was just sharing a way for you to get the info to improve other user's experience. I am writing my own custom tracking system unrelated to HA.

The entity name for each panel is sensor.hoymiles_pv_'sn'pv'port'_pv_power. Each panel has exact identification.

Ok cool


n00b question

Sorry, I'm just a hack trying to upload a database backup to dropbox. API v1 worked fine but now I'm forced to migrate.
Here's the code I used to use, with the old API v1 commands and the new API v2 ones.
Not working - can someone please help?
use Dropbox\Dropbox;
use Dropbox\DropboxApp;
use Dropbox\DropboxFile;
$app = new DropboxApp(1,2,3);
$dropbox = new Dropbox($app);
$backupFile = "/home/edplorga/backups/temp/db_" . date('d-m-Y').".zip";
$backupFilename = "/db_" . date('d-m-Y').".zip";
//Old stuff
$appInfo = dbx\AppInfo::loadFromJsonFile(DIR."/config.json");
$dbxClient = new dbx\Client($accessToken, "EDPL_SQL_Backups");
$f = fopen($backupFile, "rb");
$result = $dbxClient->uploadFile($backupFilename, dbx\WriteMode::force(), $f);
fclose($f);
//New stuff
$dropboxFile = DropboxFile::createByStream($backupFilename, $backupFile);

You're correct till the: "creating a DropboxFile instance" part. Now you need to pass this instance to the upload() method in order to actually upload the file to your Dropbox. Here's how you can do it:
try {
    // Upload the File
    $file = $dropbox->upload($dropboxFile, $backupFilename);
   // File uploaded
   echo $file->getName() . " was successfully uploaded on your Dropbox";
} catch (DropboxClientException $e) {
    echo $e->getMessage();
}

See Upload and Download Files documentation


Zero-sized fork window and certifications

Launching Cutecoin this morning, I could see I had 3 certifications with "0/1" confirmations:

However, I know these certifications are already written in every node of the network (it is written in the blockchain).
Shouldn't cutecoin directly write 1/1, or just nothing like in this screen?

It should be nothing like in the second screen. It's a bug :)

Fixed.


improve spatial maps

fixed a problem with calc_data_map that it wasn't returning the new positions, and a bug with the calculation of xy positions.
added some tests and an example that explain how to use calc_data_map to interpolate spatial data

also added an option for transparency to plot_spatial_map


suppress warnings

there are cases where warnings in model development, etc. need to be suppressed.

convergence warning is suppressed after PR #67
suppression is still needed for:
/home/tjones/.local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py:523: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
tensor_proto.tensor_content = nparray.tostring()
/home/tjones/.local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py:523: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
tensor_proto.tensor_content = nparray.tostring()
I tried suppression using warning around the tensorflow call in modeling_utilities.train_model:
warnings.filterwarnings('ignore', message='tostring() is deprecated')
es = tf.keras.callbacks.EarlyStopping(monitor=monitor, patience=patience, mode=mode)
warnings.filterwarnings('default')
but it didn't work
I also tried the same around the scipy call in anomaly_utilities.set_dynamic_threshold:
z = norm.ppf(1 - alpha / 2)
but it didn't work either


Build(deps-dev): Bump @types/node from 17.0.21 to 17.0.22 (#3169)

Pull request opened by the merge tool on behalf of #3169

Merged in 48eb75140018b263f602a4c8d8aa5d785a343edc


I didn't see any command related to IntactKV

how to get the quantized model with IntactKV_[B] and awq together？

Hi, IntactKV_[B] does not require any training, so you can directly run the evaluation shell script to get the evaluation results following commands in README. e.g.
# evaluate w3g128 AWQ-quantized LLaMA-2-7B model on GPU0, port 29500
bash ./scripts/eval/eval.sh llama-2 7b awq 3 16 ppl 29500 0

intactkv_eval.py provides the code for incorporating IntactKV into AWQ pseudo-quantized model.

Thanks a lot!!! So how can I use the IntactKV to get a inference script for AWQ?

Certainly, inference scripts are in ./scripts/eval.


RFE: Page outline / navigate to subsection?

It would be nice if qman could somehow extract the page structure and present an "outline view" of the page, with potential fuzzy search capability for locating a (sub)section by name.
For instance, man.archlinux.org is able to extract (sub)section titles and other items into id= attributes, which then allows easy linking and navigation to a specific manpage section or even toa a specific option or subcommand described in the man:

https://man.archlinux.org/man/tmux.1#respawn-pane

It would be useful if qman could perform a similar process and present an outline view for navigation.
To be clear, in the above example of tmux(1) man page I'm expecting a hierarchical outline view similar to the "Table of contents" at the link above, but with added subitems for each tmux subcommand:

tmux(1)
├ NAME
├ SYNOPSIS
├ ...
├ WINDOWS AND PANES
│ ├ append-selection
│ ├ append-selection-and-cancel
│ ├ ...
│ └ unlink-window
├ KEY BINDINGS
│ ├ bind-key
│ ├ ...
│ ├ send-prefix
│ └ unbind-key
├ ...
└ AUTHORS

Would it be feasible to implement such a feature?

Hi intelfx, thanks for your suggestion.
Support for inline links to section headings is in the works. In fact, the program already supports such links in whatis/apropos pages, and in the index page. However, the way these are implemented wouldn't work for man pages without reworking the code a bit.
So, this is a feature that will definitely happen at some point.
And yes, the next logical step after that would be having some sort of searchable table of contents (and perhaps a table of links) for the entire page. Implementing this will take some effort, though.

I'm trying to consider the possible performance issues for this.
Currently, Qman works by retrieving the formatted manual page from man and then going through it once to discover links and blocks of text that should be highlighted. This is all currently done in a single loop.
(1) Identifying all (sub)sections and creating a table of contents for them can be done in the same loop. So, that should be OK, although it will still add several ms to processing time.
(2) Going through the text to "discover" potential internal links to the various (sub)sections would be considerably costlier, as it would require looping through the text a second time, searching for text that matches them. Having such a feature, however, would be nice. For text like "please refer to the CONFIGURATION section for more details" we'll be able to click on "CONFIGURATION" to jump there.
(3) Finally, there's the list of available languages for the manual page that man.archlinux.org also provides. I don't know if that would be useful for someone, however I suppose it could be done upon request, with the available languages appearing in a pop-up menu. That wouldn't really impact performance.
Anyway, I think I should start with (1) and take it from there.

On 20 Sep 2024, at 13:58, Pantelis Panayiotou @.***> wrote:
I'm trying to consider the possible performance issues for this.
Currently, Qman works by retrieving the formatted manual page from man and then going through it once to discover links and blocks of text that should be highlighted. This is all currently done in a single loop.
(1) Identifying all (sub)sections and creating a table of contents for them can be done in the same loop. So, that should be OK, although it will still add several ms to processing time.
(2) Going through the text to "discover" potential internal links to the various (sub)sections would be considerably costlier, as it would require looping through the text a second time, searching for text that matches them. Having such a feature, however, would be nice. For text like "please refer to the CONFIGURATION section for more details" we'll be able to click on "CONFIGURATION" to jump there.We can always build an Aho-Corasick automaton and make it just one more pass linear in the man page text length :)
(3) Finally, there's the list of available languages for the manual page that man.archlinux.org also provides. I don't know if that would be useful for someone, however I suppose it could be done upon request, with the available languages appearing in a pop-up menu. That wouldn't really impact performance.
Anyway, I think I should start with (1) and take it from there.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Feature implemented in devel branch and appears to be functioning correctly.

Sorry, I just missed the previous comment here.
Indeed, works nicely with the traditional man macros (like the .TP one in ls(1)). I wonder if it could be extended to mdoc macros (.Bl/.It), like in tmux(1)?

Yeah, well, the .Bl/.It/El commands have a very complex syntax, and produce all sorts of lists and tables. Trying to properly handle them would take us into the realm of proper parsing, which goes beyond the scope of a simple utility such as Qman, IMHO.
Please note that I'm not saying that it can't be done, only that (in my opinion) it's out of scope.
While building Qman, I actually started having ideas about building a completely new and modern help system for Linux, perhaps one that is based on Markdown and incorporates modern features, such as links, tags, metadata, images/media, proper indexing, etc. And also community features, such as online editing, comments, TL/DR, and so on.
Obviously, such a system would include a full groff_man parser that would translate all the 'legacy' manual pages into the new format. But it would be an enormous undertaking, requiring community effort. Certainly not the kind of project an old dude could do by himself.
It would be useful, though. The current man system dates from the 1970s.


Roll Flutter Engine from 51d4402f17b4 to d50cae6f1d34 (1 revision)

https://github.com/flutter/engine/compare/51d4402f17b4...d50cae6f1d34
2024-09-08<EMAIL_ADDRESS>Manual roll Dart SDK from 0f783bdb7a4d to b6c3cd9b93f1 (1 revision) (flutter/engine#55031)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC codefu@google.com,rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


Bug in command worker

I have mojo app with minion worker in which i run external process.
I want to have ability to handle stdout when it occur. So, i use Mojo::IOLoop::ReadWriteFork to get this functionality.
When i start external program with Mojo::IOLoop::ReadWriteFork the "close" event not arrive.
Below two example of code. First example have right behaviour (without mojo app), second example (with mojo app) have wrong behaviour.
I don't know which module have bug, this is not obvious, so @jhthorsen maybe you would be interested.
1-st example
#!/usr/bin/perl

use Mojo::IOLoop::ReadWriteFork;
use Minion;

my $minion = Minion->new(Pg => 'postgresql://logioniz@/test');

$minion->add_task(x => sub {
  my ($job, @args) = @_;

  my $fork = Mojo::IOLoop::ReadWriteFork->new;

  $fork->on(close => sub {
    my($fork, $exit_value, $signal) = @_;
    Mojo::IOLoop->stop;
  });

  $fork->start(program => sub { warn $$; `ls` });
  Mojo::IOLoop->start;

  $job->finish;
});

$minion->enqueue('x');

my $worker = $minion->worker;
my $job = $worker->register->dequeue;
my $pid = $job->start;
warn $pid;
while (1) {
  last if $job->is_finished($pid);
  sleep 1;
}

print "Ok!\n";

2-nd example
#!/usr/bin/perl

use Mojolicious::Lite;
use Mojo::IOLoop::ReadWriteFork;

app->plugin('Minion' => {Pg => 'postgresql://logioniz@/test'});

app->minion->add_task(x => sub {
  my ($job, @args) = @_;

  my $fork = Mojo::IOLoop::ReadWriteFork->new;

  $fork->on(close => sub {
    my($fork, $exit_value, $signal) = @_;
    Mojo::IOLoop->stop;
  });

  $fork->start(program => sub { warn $$; `ls` });
  Mojo::IOLoop->start;

  $job->finish;
});

app->start;

Second example never completed its work.
$ perl 2.pl minion job -e x
36
$ MOJO_READWRITE_FORK_DEBUG=1 perl 1.pl minion worker
[22304] Child starting (CODE(0x25f12d8) )
[22304] Starting CODE(0x25f12d8)
[22304] >>> 22338 at 1.pl line 18, <DATA> line 660.\n

$ ps ax | grep  22338
22338 pts/10   Z+     0:00 [perl] <defunct>

You see that child process is zombie process.

Afraid i don't know much about Mojo::IOLoop::ReadWriteFork, so won't be able to do much here. But maybe @jhthorsen can help.

@Logioniz I guess you could add more lines from the worker command to your first example, one by one, to locate the actual problem.

@Logioniz: what happens if you run the the Mojo::IOLoop::ReadWriteFork code by itself? I mean as a standalone script, without using minion?
This test looks pretty much like the code you have, so I'm not sure why the "close" event is not called: https://github.com/jhthorsen/mojo-ioloop-readwritefork/blob/master/t/close.t
Also, I'm curious of why MOJO_READWRITE_FORK_DEBUG=1 doesn't report the STDERR/OUT from the "program" sub.

Also, which version of Mojo::IOLoop::ReadWriteFork and which operation system are you using?
Oh! Maybe you need Mojo::IOLoop->reset inside the task sub, since the IOLoop is already running the main application..? (wild guess)

@jhthorsen when run script without minion all works well. In first example worker works well too.
I use latest version of both modules (minion and readwritefork).
Command ls not show result because i call it in backtick (it's a new process), so ls return string without showing on console.
Mojo::IOLoop->reset doesn't have any positive affect.
Need to take advice of @kraih and continue investigate worker.
$ perl -MMojo::IOLoop::ReadWriteFork -le 'print $Mojo::IOLoop::ReadWriteFork::VERSION'
0.13
$ perl -MMinion -le 'print $Minion::VERSION'
2.03
$ uname -a
Linux Vena 3.19.0-30-generic #34-Ubuntu SMP Fri Oct 2 22:08:41 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

@Logioniz: Do you have EV installed?
$ mojo version
...
OPTIONAL
EV 4.0+                 (4.21)

If so, can you try with MOJO_REACTOR=Mojo::Reactor::Poll

Yes. Also i add on close warn message "close event". Results are differents:
$ MOJO_REACTOR=Mojo::Reactor::Poll MOJO_READWRITE_FORK_DEBUG=1 perl 1.pl minion worker
[25397] Child starting (CODE(0x2173518) )
[25397] Starting CODE(0x2173518)
[25397] >>> 25397 at 1.pl line 20, <DATA> line 660.\n
[25397] Child is dead (0/) 0/0
close event at 1.pl line 16, <DATA> line 660.

"Different" = It works with MOJO_REACTOR=Mojo::Reactor::Poll ?
Sorry, I don't understand what "different" means.

In example 2 close event not called. Program never completed its work. But when set Mojo::Reactor::Poll the close event was called and mojo::ioloop->stop was called and program has finished

May be need to say that with Mojo::Reactor::Poll  works well?)

I honestly have no idea why it's different... What I know is that you hit this code which does some funky stuff using EV: https://github.com/jhthorsen/mojo-ioloop-readwritefork/blob/master/lib/Mojo/IOLoop/ReadWriteFork.pm#L375
I'm sad to say I don't know the inner workings of EV very well. What I do know is that I'm not pulling that code out, since it makes RWF work a lot smoother in cases I use it for.
My suggestion is that we close this issue, since it doesn't seem to be specific to Minion and that you open a new issue in https://github.com/jhthorsen/mojo-ioloop-readwritefork/issues. And if you want to speed up things from my side, it would be awesome if you can make a failing case, without Minion, since I don't know Minion either.
One quick fix would be to add a constant to RWF which disable the EV::child call, but still allow you to use EV for the rest of your application.

Agreed, this does not seem like a Minion specific problem.

I can't write example without Minion which not working.
Any example with RWF and Reactor::EV works fine. Even example with extra fork (like minion):

use Mojo::IOLoop::ReadWriteFork;

my $pid = fork;
die $! unless defined $pid;

if ($pid) {
  warn "Parent: $$";
  waitpid $pid, 0;
} else {
  warn "Child: $$";
  my $fork = Mojo::IOLoop::ReadWriteFork->new;

  $fork->on(close => sub {
    my($fork, $exit_value, $signal) = @_;
    warn 'close';
    Mojo::IOLoop->stop;
  });

  $fork->start(program => 'ls');

  Mojo::IOLoop->start;
}

I think that problem at the junction of Minion and RWF.

I just scanned though the Minion code very briefly, and I can't see I spot any reason why it would make any specific issues. (Mostly focused the reading on https://github.com/kraih/minion/blob/master/lib/Minion/Job.pm)

Even tried to drop DBD::Pg and waitpid into the mix: https://ssl.thorsen.pm/paste/9dd154dda66c

@Logioniz: $SIG{CHLD} = "DEFAULT" was the issue. Fixed in 3be062eae189ee518b4a53192f52568215490432.

@kraih, @jhthorsen  very thanks.
@kraih now we have another problem.

I have Minion version 2.04

Looks like the change needs to be reverted.

As far as i can see, the actual problem here is that EV takes over SIGCHLD when it is loaded, and will greedily fetch the exit/wait status of all children. This kills our process management and means that the local $SIG{CHLD} = 'DEFAULT' line is absolutely necessary.

Yeah, I think you're right.
use strict;
use POSIX 'WNOHANG';

$SIG{CHLD} = sub {
  my $pid = waitpid -1, WNOHANG;
  warn "SIGCHLD: $pid = $?\n" if $pid > 0;
};

if (my $pid = fork) { # parent
  while (1) {
    local $SIG{CHLD} = 'DEFAULT'; # commenting this line out makes the process hang forever
    my $chld = waitpid $pid, WNOHANG;
    warn "while waitpid $chld: $?\n" if $chld;
    last if $chld > 0;
    sleep 1;
  }
}
else { # child
  print "Child $$\n";
  sleep 1;
  exit;
}

Would you be interested in a EV patch, or is it too ugly having EV specific code in Minion? Or how about adding Mojo::Reactor::child($pid, $ch) ..?

I can't think of another solution than going back to $SIG{CHLD} = "IGNORE". I've got a commit which makes this work: https://github.com/jhthorsen/mojo-ioloop-readwritefork/commit/8ecab592dfcfcfbc676807ec78bfb85abad2d23f
Can you install that version @Logioniz and try it out?
$ cpanm https://github.com/jhthorsen/mojo-ioloop-readwritefork/archive/master.tar.gz

Also, can you look at the test I wrote @kraih and give me feedback if I'm doing something too crazy?

@jhthorsen this version works. Test looks fine.
But i can't understand why we have problem with $SIG{CHLD} = "DEFAULT" in minion module...

@Logioniz: With EV, my code allows EV to "hijack" $SIG{CHLD}. When this is done, waitpid() is not working properly, since the SIGCHLD handler is reaping the children, which again results in waipid() to block. This is just the way things work... Nothing special about RWF, Mojolicious or Minion: You need to choose a global strategy for handling dead child processes.

POSTing upload for Mojo-IOLoop-ReadWriteFork-0.14.tar.gz to https://pause.perl.org/pause/authenquery
PAUSE add message sent ok [200]

Enjoy!

@jhthorsen  very thanks.


Copy-DbaDatabase - Add EA to internal function

Adding EA to the Start-BitsTransfer so the catch block and final Copy-Item is properly hit.

Type of Change

[x] Bug fix (non-breaking change, fixes #6915 )
[ ] New feature (non-breaking change, adds functionality, fixes # )
[ ] Breaking change (effects multiple commands or functionality, fixes # )
[ ] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[ ] Pester test is included
[ ] If new file reference added for test, has is been added to github.com/sqlcollaborative/appveyor-lab ?
[ ] Nunit test is included
[ ] Documentation
[ ] Build system

Thank you! 🥳

If we don't want to deal with the new format style that Code enforces with PSSA we should add exclusion to turn that off for our workspace.
Can't remember when it started doing that.

That'd be great! It supposedly shouldn't be impacting us. Well, our tests aren't failing about it because we're using an older version, but new versions are doing it. Niph suggested we submit a bug report to PSScriptAnalzyser, but I haven't had a chance yet. Is that something we can ignore using our .vscode settings?

It's not a bug in formatting really because it is the proper casing for the parameters. They have always been in all caps. Code or the extension just never enforced it.
If we don't want it to be I'll see if I can find a way to block it.


Speed-Up Plagiarism Detection

Describe the bug
Currently, plagiarism detection is very slow, which sometimes causes problems (server not reachable, no result shown, etc.). The origin of the problem is twofold:

The download of all repositories happens sequential (related: https://github.com/ls1intum/Artemis/issues/1226)
JPlag is run in normal mode instead of parallel mode.

To Reproduce
Run plagiarism check on an exercise with many participants (600+).
Expected behavior
Run all steps in parallel and thus speed up execution.
Screenshots
No response
What browsers are you seeing the problem on?
Firefox
Additional context
JPlag currently only distinguishes between normal and parallel mode and does not allow the specification of a max. number of used processors. Thus, extending JPlag with an option to limit parallelism is desirable to avoid excessive resource consumption.
Relevant log output
No response

We will try to speedup the process, however please keep in mind that plagiarism checks have a lower priority than other processes on Artemis. Performing too many operations at the same time (e.g. while others have large exams) can lead to an overall unstable system performance.
I think we would need to synchronize plagiarism operations in a way that e.g. only one operation per course is running at the same time, but instructors could queue up operations and get notified when they are finished. In case of active exams in large courses, the operations could be performed after the exam has finished...

The general performance was improved the last couple of weeks.


Faster writing of Json-objects by not boxing properties

Right now the object-reflection is rather simple, and converts all properties into object. Then afterwards the object is serialized into the expected format (json / xml / etc.)
When handling properties of value-type, then this means boxing (memory allocation). Could be nice if the serializer could perform the object-reflection without boxing all value-type-properties.

@snakefoot , just to confirm before I take a look at this, you're referring to NLog.Targets.DefaultJsonSerializer.SerializeObjectProperties(),  https://github.com/NLog/NLog/blob/2c6b59ead3474ac18b0af16b2c33b5b13b7120f5/src/NLog/Targets/DefaultJsonSerializer.cs#L708 ?

Yes that is the spot.

And are you ok with an implementation that basically is just a switch statement with specific handling for each primitive value type? I think that's basically how I've seen this handled in various other libraries.
And do you have a current size or complexity that you've seen this have speed issues, or is it just something you know could or should be faster? I'll be setting up some kind of proof of speed improvement to verify changes, but would like to know any context you have.

Think it would be a great start if there was handling of ValueTypes that implements IConvertible without boxing (int, bool, double, decimal, datetime etc.)

Because the process to get the value of the property within the cache lookup logic has to use PropertyInfo.GetValue(Object, Object[]), that returns System.Object and does not offer a generic PropertyInfo.GetValue() version, then the primitive value will already be boxed, and I am unaware of a way to get around that.
If you were to find a way, you would then also have to modify the PropertyValue struct that is being returned from the cache lookup to create some form of generic  implementation, and you won't be able to cast the value into an IConvertible, because that will also cause boxing.

I suppose you might be able to compile an Expression, specific to the property, then provide a GetValueExpression<T>() method on the PropertyValue struct, that could then be called from within a generic method inside the serializer. I don't think this would box the value. It would be quite a rework of code though. Is this a path you'd want to go down?

Yes somehow the ObjectReflectionCache needs to build an object-serialization-contract (that is cached), so the object-reflection-cache can query the json-serializer what types supports "non-boxed"-serialization (only once for each type)
So the json-serializer must implement a method that receives a PropertyInfo and can then resolve to a expression-tree-lambda (static method that takes object-value and json-serializer and stringbuilder as input parameter).
When json-serializer enumerates the special-property-list (object-serialization-contract) then it will call a method like propertyValue.TrySerializer(this) and if the special-property-list has resolved a serializer-method, then it is used and returns true. But when "non-boxed" then it returns false, and standard serialization happens as normal.

The code needing to be changed keeps going deeper and deeper.
There are 3 constructors in PropertyValue, and 2 of them have already boxed the primitive values prior to the call to the constructor.
                public PropertyValue(string name, object value, TypeCode typeCode)
                {
                    Name = name;
                    Value = value;
                    _typecode = typeCode;
                }

                public PropertyValue(object owner, PropertyInfo propertyInfo)
                {
                    Name = propertyInfo.Name;
                    Value = propertyInfo.GetValue(owner, null);
                    _typecode = TypeCode.Object;
                }

                public PropertyValue(object owner, FastPropertyLookup fastProperty)
                {
                    Name = fastProperty.Name;
                    Value = fastProperty.ValueLookup(owner, null);
                    _typecode = fastProperty.TypeCode;
                }

The first constructor is used in ObjectPropertyList.Enumerator, which has a few layers of containers that have all boxed to object already.
The second constructor could be left alone from the caller's perspective, but it appears to only be used for non primitive values because it's hardcoded to set the TypeCode to Object.
The FastPropertyLookup's current expression tree used in the third constructor also boxes values.
I feel that essentially the entirety of ObjectReflectionCache, and all the associated classes, will have to be redone from scratch to ensure a path of preventing the boxing, and then build back in all the various handling for different object and property types. I'm not sure how many of the special cases are covered in the current JSON serializer tests?

Essentially the entirety of ObjectReflectionCache, and all the associated classes, will have to be redone from scratch to ensure a path of preventing the boxing

Also my observation. The ObjectReflectionCache was a very naive attempt to support structured-logging. Could also be that we should focus on opening NLog to make use of System.Text.Json, since it has the ability to serialize without boxing.


LOG-375. Move CLO and EO to v1

This bumps CLO and EO to v1 of the CRD and 4.1.0 of release
Ref: https://jira.coreos.com/browse/LOG-375
Depends on:

https://github.com/openshift/cluster-logging-operator/pull/130
https://github.com/openshift/elasticsearch-operator/pull/106
https://github.com/openshift/cluster-logging-operator/pull/136

/hold

/hold cancel

@SamiSousa can we approve and merge this?  https://github.com/openshift/cluster-logging-operator/pull/136 is in the merge que and the last hold out

@jcantrill Are we waiting on the other PR to merge first?

@jcantrill Are we waiting on the other PR to merge first?

No. Logging is already broken for CRD version mismatch so this wont break it any further

@jcantrill Is this version of the operator working, or is this also broken?
Have these changes been tested on OpenShift?

@jcantrill Is this version of the operator working, or is this also broken?
Have these changes been tested on OpenShift?

Yes.

Testing now

/lgtm

Elasticsearch-operator and cluster-logging have been updated in quay.io


Updating node-debugger error

Updating the package through Atom on Win 10 x64 consistently receives the following error.

Updating to<EMAIL_ADDRESS>failed.
EPERM, unlink 'C:\Users\Username\.atom\packages\node-debugger\debugger.log'

Workaround solution is to uninstall the package and reinstall from fresh.

Hmm seems like the log file is locked and this preventing the uninstall to succeed.


Fail state is missing CausePath field

Describe the bug
The schema for Catch is missing the field "CausePath" field which is a ReferencePath a subset of the IntrinsicFunctions
To Reproduce
clone existing fail test and add both Cause and CausePath.
Expected behavior
Report an error if both properties are defined
report an error if CausePath is not a ReferencePath or one of the intrinsic functions that produces a string
Version:
latest
Additional context
https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-fail-state.html

I am experiencing the same issue, seems like ErrorPath isn't supported either.

Fixed by https://github.com/ChristopheBougere/asl-validator/pull/148 and released in 3.8.2
Thanks @massfords


[BACKLOG-8492] Added MDI Support for Get Table Names

Build completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands:

ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f engine/build.xml jacoco-integration checkstyle publish-local

Cleanup Commands:
rm -r ~/.ivy2/local || echo no publish local to remove
Changed files:
engine/src/org/pentaho/di/trans/steps/gettablenames/GetTableNamesMeta.java
engine/src/org/pentaho/di/trans/steps/gettablenames/messages/messages_en_US.properties
engine/test-src/org/pentaho/di/trans/steps/gettablenames/GetTableNamesMetaInjectionTest.java
Newly Broken Tests:
org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoaderMetaTest.testSerialization:
java.lang.RuntimeException
java.lang.RuntimeException: Error validating autoSchema
	at org.pentaho.di.trans.steps.loadsave.LoadSaveTester.validateLoadedMeta(LoadSaveTester.java:193)
	at org.pentaho.di.trans.steps.loadsave.LoadSaveTester.testXmlRoundTrip(LoadSaveTester.java:238)
	at org.pentaho.di.trans.steps.loadsave.LoadSaveTester.testSerialization(LoadSaveTester.java:199)
	at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoaderMetaTest.testSerialization(MonetDBBulkLoaderMetaTest.java:133)
Caused by: org.pentah

Still Broken Tests:
org.pentaho.di.trans.steps.rest.RestIT.testNoContent:
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException: 11264
	at org.objectweb.asm.ClassReader.readClass(Unknown Source)
	at org.objectweb.asm.ClassReader.accept(Unknown Source)
	at org.objectweb.asm.ClassReader.accept(Unknown Source)
	at com.sun.jersey.spi.scanning.AnnotationScannerListener.onProcess(AnnotationScannerListener.java:136)
	at com.sun.jersey.core.spi.scanning.JarFileScanner.scan(JarFileScanner.java:97)
	at com.sun.jersey.core.spi.scanning.JarFileScanner$1.f(JarFileScanner.java:74)
	at com.sun.j

org.pentaho.di.trans.steps.rest.RestIT.testResponseHeader:
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException

Unit test coverage change:
org.pentaho.di.job.entries.evalfilesmetrics.JobEntryEvalFilesMetrics:

Branch Change : -.39%
Complexity Change : -.51%
Instruction Change : -.08%

org.pentaho.di.job.entries.evaluatetablecontent.JobEntryEvalTableContent:

Branch Change :  + 1.01%
Complexity Change :  + 1.06%
Instruction Change :  + .17%

org.pentaho.di.trans.step.StepMeta:

Instruction Change : -.19%
Line Change : -.27%

org.pentaho.di.trans.steps.checksum.CheckSumMeta:

Instruction Change :  + .12%

org.pentaho.di.trans.steps.gettablenames.GetTableNamesMeta:

Complexity Change :  + .53%
Instruction Change :  + .41%
Line Change :  + .73%
Method Change :  + .29%

org.pentaho.di.trans.steps.groupby.GroupByMeta:

Branch Change : -1.04%
Complexity Change : -1.05%

org.pentaho.di.trans.steps.ifnull.IfNullMeta:

Branch Change : -2.38%
Complexity Change : -1.64%
Instruction Change : -.17%

org.pentaho.di.trans.steps.insertupdate.InsertUpdateMeta:

Instruction Change :  + .05%

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta:

Instruction Change :  + .06%

org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoaderMeta:

Branch Change : -6.85%
Complexity Change : -4.38%
Instruction Change : -14.09%
Line Change : -11.53%
Method Change : -6.25%

org.pentaho.di.trans.steps.rowgenerator.RowGeneratorMeta:

Instruction Change : -.08%

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta:

Instruction Change : -.18%

org.pentaho.di.trans.steps.systemdata.SystemDataMeta:

Branch Change : -2.04%
Complexity Change : -2.04%
Instruction Change : -.32%
Line Change : -.81%

org.pentaho.di.trans.steps.textfileinput.TextFileInputMeta:

Branch Change : -.53%

Checkstyle violations:
See: https://github.com/pentaho/pentaho-coding-standards
File /home/buildguy/project/engine/src/org/pentaho/di/trans/steps/gettablenames/GetTableNamesMeta.java

Message: ',' is preceded with whitespace. at Line: 60 Column: 90
Message: ',' is preceded with whitespace. at Line: 60 Column: 103
Message: Line has trailing spaces at Line: 66
Message: Line has trailing spaces at Line: 69
Message: Line has trailing spaces at Line: 73
Message: Line has trailing spaces at Line: 76
Message: Line has trailing spaces at Line: 79
Message: Line has trailing spaces at Line: 85
Message: Line has trailing spaces at Line: 88
Message: Line has trailing spaces at Line: 91
Message: Line has trailing spaces at Line: 94
Message: Line has trailing spaces at Line: 97
Message: Line has trailing spaces at Line: 100
Message: Line has trailing spaces at Line: 103
Message: Line has trailing spaces at Line: 106
Message: Line has trailing spaces at Line: 111
Message: File contains tab characters (this is the first instance). at Line: 287 Column: 1
Message: Line has trailing spaces at Line: 289

Build completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands:

ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f engine/build.xml jacoco-integration checkstyle publish-local

Cleanup Commands:
rm -r ~/.ivy2/local || echo no publish local to remove
Changed files:
engine/src/org/pentaho/di/trans/steps/gettablenames/GetTableNamesMeta.java
engine/src/org/pentaho/di/trans/steps/gettablenames/messages/messages_en_US.properties
engine/src/org/pentaho/di/trans/steps/getvariable/messages/messages_en_US.properties
engine/test-src/org/pentaho/di/trans/steps/gettablenames/GetTableNamesMetaInjectionTest.java
Newly Fixed Tests:
org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoaderMetaTest.testSerialization:
java.lang.RuntimeException
java.lang.RuntimeException: Error validating autoSchema
	at org.pentaho.di.trans.steps.loadsave.LoadSaveTester.validateLoadedMeta(LoadSaveTester.java:193)
	at org.pentaho.di.trans.steps.loadsave.LoadSaveTester.testXmlRoundTrip(LoadSaveTester.java:238)
	at org.pentaho.di.trans.steps.loadsave.LoadSaveTester.testSerialization(LoadSaveTester.java:199)
	at org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoaderMetaTest.testSerialization(MonetDBBulkLoaderMetaTest.java:133)
Caused by: org.pentah

Still Broken Tests:
org.pentaho.di.trans.steps.rest.RestIT.testNoContent:
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException: 11264
	at org.objectweb.asm.ClassReader.readClass(Unknown Source)
	at org.objectweb.asm.ClassReader.accept(Unknown Source)
	at org.objectweb.asm.ClassReader.accept(Unknown Source)
	at com.sun.jersey.spi.scanning.AnnotationScannerListener.onProcess(AnnotationScannerListener.java:136)
	at com.sun.jersey.core.spi.scanning.JarFileScanner.scan(JarFileScanner.java:97)
	at com.sun.jersey.core.spi.scanning.JarFileScanner$1.f(JarFileScanner.java:74)
	at com.sun.j

org.pentaho.di.trans.steps.rest.RestIT.testResponseHeader:
java.lang.ArrayIndexOutOfBoundsException
java.lang.ArrayIndexOutOfBoundsException

Unit test coverage change:
org.pentaho.di.job.entries.sql.JobEntrySQL:

Instruction Change : -3.28%
Line Change : -3.87%

org.pentaho.di.repository.kdr.KettleDatabaseRepositoryCreationHelper:

Branch Change : -1.64%
Complexity Change : -.69%
Instruction Change : -2.43%
Line Change : -3.70%

org.pentaho.di.trans.steps.checksum.CheckSumMeta:

Instruction Change : -.12%

org.pentaho.di.trans.steps.datagrid.DataGridMeta:

Instruction Change :  + .09%

org.pentaho.di.trans.steps.dbproc.DBProcMeta:

Instruction Change : -.11%

org.pentaho.di.trans.steps.dimensionlookup.DimensionLookupMeta:

Branch Change : -.66%
Complexity Change : -.41%
Instruction Change : -.50%
Line Change : -.24%

org.pentaho.di.trans.steps.gettablenames.GetTableNamesMeta:

Complexity Change :  + .53%
Instruction Change :  + .41%
Line Change :  + .73%
Method Change :  + .29%

org.pentaho.di.trans.steps.groupby.GroupByMeta:

Branch Change : -1.04%
Complexity Change : -1.05%

org.pentaho.di.trans.steps.ifnull.IfNullMeta:

Instruction Change :  + .08%

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta:

Instruction Change : -.06%

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta:

Branch Change :  + 2.50%
Complexity Change :  + 2.82%

org.pentaho.di.trans.steps.monetdbbulkloader.MonetDBBulkLoaderMeta:

Branch Change :  + 6.85%
Complexity Change :  + 4.38%
Instruction Change :  + 14.09%
Line Change :  + 11.53%
Method Change :  + 6.25%

org.pentaho.di.trans.steps.rssinput.RssInputField:

Branch Change : -2.27%
Complexity Change : -1.54%
Instruction Change : -.29%

org.pentaho.di.trans.steps.setvalueconstant.SetValueConstantMeta:

Instruction Change :  + .14%

org.pentaho.di.trans.steps.singlethreader.SingleThreaderMeta:

Instruction Change : -.08%

org.pentaho.di.trans.steps.symmetriccrypto.symmetriccryptotrans.SymmetricCryptoTransMeta:

Branch Change : -3.33%
Complexity Change : -1.85%
Instruction Change : -.28%

org.pentaho.di.trans.steps.synchronizeaftermerge.SynchronizeAfterMergeMeta:

Instruction Change :  + .04%

org.pentaho.di.trans.steps.systemdata.SystemDataMeta:

Branch Change : -2.04%
Complexity Change : -2.04%
Instruction Change : -.32%
Line Change : -.81%

org.pentaho.di.trans.steps.tableoutput.TableOutputMeta:

Branch Change :  + .91%
Complexity Change :  + .82%

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta:

Instruction Change :  + .18%


fix(onAction): more onAction after can run

The main problem is that the parent and child components $onAction at the same time will only trigger the last after callback, but in the actual scenario, we may need to use this after callback.

Multiple $onAction are registered, but only the last after is running.

You’re supposed to call after only once. Is that what you meant to fix?
If not I think a boiled down repro would help because this dix complexifies a lot the implémentation

You’re supposed to call after only once. Is that what you meant to fix?
If not I think a boiled down repro would help because this dix complexifies a lot the implémentation

You’re supposed to call after only once. Is that what you meant to fix? If not I think a boiled down repro would help because this dix complexifies a lot the implémentation

I use onAction in multiple places, why only one after is executed? So if you follow the current thinking, why do you need to execute onAction multiple times instead of executing multiple times his callback? Because I may have done monitoring in different components, I may need to do some callback processing. For example, I use a pop-up window to log in, and I want to monitor user changes in the header component, and then make some other data loading requests. Is this unreasonable? At the same time, I also provided 3 scenarios for testing. I think scenario 3 should be used frequently in actual development.

I'm revisiting the tests and in this case
    const spy = jest.fn()
    store.$onAction(({ after }) => {
      after(spy)
    })
    store.$onAction(({ after }) => {
      after(spy)
    })

the test is not valid because you should not be passing the same function twice to the after of one given store. Do you have a use case for that? We could maybe warn against this though.
However, this one should work and currently it isn't:
    const spy1 = jest.fn()
    const spy2 = jest.fn()
    store.$onAction(({ after }) => {
      after(spy1)
    })
    store.$onAction(({ after }) => {
      after(spy2)
    })

    await expect(store.asyncUpperName()).resolves.toBe('EDUARDO')
    expect(spy2).toHaveBeenCalledTimes(1)
    expect(spy1).toHaveBeenCalledTimes(1)

Currently, the plugin allows multiple onAction subscriptions to be executed, and as a result, only one after callback is triggered. But there is a scenario where the parent component triggers the action, the child component gets the return value through after, and the parent component cannot get the action return value. For child components, this after return value is useless, but for parent components, it is very useful. The child component may use this after callback to do some other things, and its function can be completely isolated from the parent component. Therefore, I think this is an unavoidable user behavior. If the user can write after, it means it has his purpose. We should not prohibit him.

Thank you for this, unfortunately, I had to revisit quite a few things, please refer to the commit to see the differences


pass node to data bag helper

data bag helper needs to access the node to read platform and platform_version and to read the local services directory attribute.

Looks ok.


Reverse Camera2D zoom

Bugsquad edit: Implements and closes https://github.com/godotengine/godot-proposals/issues/3888
This PR changes how zoom property of Camera2D works, so e.g. "zoom = 2" is now "zoom-in 2x", which is much more intuitive (previously it would zoom out).
Alternatively we could rename the property to view_rect_scale or similar, because that's what it actually does right now.

The documentation should be updated too.
https://github.com/godotengine/godot/blob/e53e357a2bf738d624b1fe29ee967603395efdd0/doc/classes/Camera2D.xml#L157-L159

Closing in favor of #57392


Incorrect documentation for dynamic_rnn

The help for tf.nn.dynamic_rnn states that the optional initial_state kwarg, if cell.state_size is an integer, must be a Tensor of appropriate type and shape [batch_size x cell.state_size]. This is incorrect; the shape must be actually be [batch_size, cell.state_size].

Thanks for the report!  I'll update this issue once the fixed documentation has been merged in.

OK the docs have been updated, thanks @vladfi1 for filing the issue!


GH-41190: [C++] support for single threaded joins

When I initially added single threading support, I didn't do asof joins and sorted merge joins, because the code for these operations uses threads internally. This is a small check-in to add support for them. Tests run okay in single-threaded, I'm pushing it here to run full tests and check I didn't break the threaded case.
I'm pushing this now because making this work saves adding a load of threading checks in python (this currently breaks single-threaded python i.e. emscripten).

GitHub Issue: #41190

This is not a MINOR change.
See also the auto generated comment https://github.com/apache/arrow/pull/41125#issuecomment-2047730935 and our MINOR definition https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes in the comment.
Could you open a new issue for this?

Fair enough, #41190 is the corresponding issue.

@kou any chance we could merge this - it's needed for #37822

@joemarshall, will you be able to pick these changes up any time soon? Do you need more funding to pull this back up the priority queue?

@kou - I couldn't quite get rid of process_task_, because it is used as a flag to detect finishing of data in various ways, and keeping it in makes it much easier to keep consistent behaviour compared to the threaded version, but I got rid of process_ and process_thread_ by using a push function as you described.

And did the same in sorted merge.
There are some weird errors in python asof join on some platforms (e.g. mac ), which seem to happen in the main build also. I don't think I broke them.

There are some weird errors in python asof join on some platforms (e.g. mac ), which seem to happen in the main build also. I don't think I broke them.

Yes, there are indeed some flaky asof join tests at the moment (https://github.com/apache/arrow/issues/40675 and https://github.com/apache/arrow/issues/41149)

Author of sorted merge here. Just to say it, thank you for taking this on!

Hey @kou / @westonpace, I think this should be ready for another pass when you have time. It looks like the errors happening before were resolved as @jorisvandenbossche mentioned, but weren't a result of any changes @joemarshall made.

@westonpace done that fix above now


Migrate to IndexedDB

Probably IndexedDB is a better choice for the app. Further investigation is needed

I'll take this issue.

Probably we should read this and decide what works for us:
https://paultman.com/from-localstorage-to-indexeddb/
https://paultman.com/best-library-for-indexeddb-localforage-idb-keyval-or-idb/

Any progress here?

Good example of CRUD operations for indexeddb:
https://javascript.plainenglish.io/working-with-indexeddb-in-typescript-react-ad504a1bdae3

Done with implementing CRUD and created functions for inserting and fetching data to/from database.

Please open a PR or Draft PR?

Done with PR changes, please check.

Closed by https://github.com/cofeed-19/cofeed-19.github.io/pull/25


Add some build tips encountered on Raspberry Pi.

Most of this has suffered from bit rot, as phantomjs isn't even being used anymore, so I'll close this.
Just some notes:

https://github.com/openstreetmap/id#prerequisites doesn't say anything about dependencies, but https://github.com/openstreetmap/iD/blob/develop/CONTRIBUTING.md#tests says that karma needs CHROME_BIN pointing at the path to chromium, which seems good.
The phantomjs install really was that flaky, and would leave the node_modules in a broken state that npm install didn't fix.
The low memory situation is still valid, but not really worth mentioning. (But wow, Karma manages to consume 1.5G of memory just to tell me that it can't find Chrome.)


Can I load my segment key if I use the self-hosted analytics.js?

I don't want to include that big code snippet documented in the QuickStart into my HTML file. I'd rather load the UMD analytics.js module that I've installed with JSPM and load it using SystemJS.
If I load it this way, how can I tell analytics.js what my segment key is?

I'm in the same situation. There are too many moving parts in my application, I need segment loaded synchronously and immediately, which means I need to package it with my application.

Same issue. Is this repository maintained? We're paying for this product...

The open source version of analytics.js doesn't need/use a write key. You provide settings for all the providers directly.
Note that self hosting your own analytics.js as opposed to using the CDN means that you lost a big benefit of using Segment, which is on the fly updates to your analytics anytime you make changes. Self hosting means that you need to configure the settings for each integration directly and will need to redeploy your analytics.js any time you want to make any changes to your settings.
In my honest opinion, it's always better/easier to just use the CDN.

I'm in the same situation. There are too many moving parts in my application, I need segment loaded synchronously and immediately, which means I need to package it with my application.

This means that if any of your integrations' services are down, or have downtime or have slow servers, your app may be blocked which is probably not what you want :)


Bump dependencies.

Fix issues reported by new rule unicorn/no-fn-reference-in-iterator.
semver-major package updates:

globby ^6.0.0 to ^7.1.1
eslint-plugin-unicorn ^2.1.0 to ^3.0.1
eslint-config-xo-react ^0.13.0 to ^0.15.0

See also for information about new linter rules:
https://github.com/sindresorhus/eslint-plugin-unicorn/releases/tag/v3.0.0
https://github.com/sindresorhus/eslint-config-xo-react/commits/v0.15.0

Thanks, but I bump the dependencies before releasing a new version to reduce churn. There will be more dependencies to update by the time I publish the next XO version.

Do you not want me to submit a PR that fixes the unicorn/no-fn-reference-in-iterator errors (this change excluding the package.json changes)?  It seems like that would make things easier when eslint-plugin-unicorn is eventually updated.


Add support for Fork CMS ^6

For Fork CMS 6 we've made some changes to make it possible to install modules or themes through composer
Could you add us to the list here?

Thanks