ConnectException: Connection refused when passing Request or Uri to Client.expect

object Main extends IOApp {
  override def run(args: List[String]): IO[ExitCode] = for {
    x <- BlazeClientBuilder[IO](global).resource.use { client =>
      client.expect[String]("https://www.bbc.com")
    }
    _ <- IO(println(x))

  } yield ExitCode.Success
}

The above works while
object Main extends IOApp {
  override def run(args: List[String]): IO[ExitCode] = for {
    x <- BlazeClientBuilder[IO](global).resource.use { client =>
      client.expect[String](uri = Uri(path = "https://www.bbc.com"))
    }
    _ <- IO(println(x))

  } yield ExitCode.Success
}

doesn't. I also tried
client.expect[String](uri = Uri(path = "www.bbc.com", scheme = Option(Scheme.https)))

and passing in a Request. In all those cases I'm getting
java.net.ConnectException: Connection refused
	at sun.nio.ch.UnixAsynchronousSocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishConnect(UnixAsynchronousSocketChannelImpl.java:252)
	at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:198)
	at sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:213)
	at sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:293)
	at java.lang.Thread.run(Thread.java:748)

I'm using Debian 9, Scala 2.12, I'll paste my current dependecies below, although I've had this issue for a while and tried multiple other combinations.
val http4sVersion = "0.20.1"

libraryDependencies += "org.typelevel" % "cats-core_2.12" % "1.6.0"
libraryDependencies += "org.typelevel" % "cats-effect_2.12" % "1.3.1"
libraryDependencies += "io.circe" % "circe-core_2.12" % "0.11.1"
libraryDependencies += "io.circe" % "circe-generic_2.12" % "0.11.1"
libraryDependencies += "io.circe" % "circe-parser_2.12" % "0.11.1"
libraryDependencies += "io.circe" % "circe-java8_2.12" % "0.11.1"
libraryDependencies += "org.http4s" % "http4s-circe_2.12" % http4sVersion
libraryDependencies += "org.http4s" % "http4s-dsl_2.12" % http4sVersion
libraryDependencies += "org.http4s" % "http4s-blaze-client_2.12" % http4sVersion

This is a flaw in our URI class: the URI is a case class with various components, including a scheme (e.g., http, https), an authority (e.g., www.bbc.com), and a path.  When you call withPath, it circumvents the parsing done by fromString, and the client doesn't know that the URI refers to www.bbc.com.
Workaround: if you use uri"http://www.bbc.com" (works for String literals), or Uri.fromString("http://www.bbc.com") (puts the result in a ParseResult[Uri], so you handle any errors), it should work.
Long term: we're going to replace the URI with something friendlier.

Long term: we're going to replace the URI with something friendlier.

Thanks for your reply. I think In the meantime it would help to remove references to the deprecated Uri.uri from the documentation. Looking for a better alternative  is what lead me down this path (altough admittedly I should've paid more attention to the docs and also should've realised there is probably a deprecated annotation on Uri.uri with further instructions attached)

It looks to me like tut is configured with fatal warnings and deprecation.  I'm not sure how this slipped through, but point well taken on the docs.  I'll file some followups.

I think the original issue is a duplicate of #2487.


3pl predict function returns predictions with values that exceed 1

The predict function in three_param_logistic.py returns:
return lambdas + (1 - lambdas / (1 + np.exp(-discs * (abilities - diffs))))
Which will return predicted scores ranging from 1-2. This contrasts with the 2pl model which appears to correctly return predictions ranging from 0-1. I believe this is missing some parentheses and should be:
return lambdas + ((1 - lambdas) / (1 + np.exp(-discs * (abilities - diffs))))
to match the formula here: https://en.wikipedia.org/wiki/Item_response_theory#Three_parameter_logistic_model

Fixed! This is now in the latest version. Thanks for your patience.


Middleware returns int(1)

Disclaimer: Most likely not an issue, just lack of insight on my part.
I am using v3 development version and my project relies on things like:

Subrequests (killer feature, makes it a HMVC framework)
Application-wide middleware for things like wrapping page routes inside layout views etc.

All was fine and dandy until I updated to latest commit of the develop branch, and middleware calls started to act strange:

Return integer 1, instead of Response object
Leak output

For example, a call like this "$newResponse = $next($req, $res)" inside application middleware would:

Return integer 1 as the "$newResponse" variable
Dump the output (the contents of what should have been response body) of the "$next" callable to the screen

Are you able to create a small example project that shows the problem?

If I invoke /test, it prints "some nice output" and crashes with exception:
'''
UnexpectedValueException: Middleware must return instance of \Psr\Http\Message\ResponseInterface'
#0 C:\projects\flsite\vendor\slim\slim\Slim\MiddlewareAware.php(106): Slim\App->Slim{closure}(Object(Slim\Http\Request), Object(Slim\Http\Response))
#1 C:\projects\flsite\vendor\slim\slim\Slim\App.php(346): Slim\App->callMiddlewareStack(Object(Slim\Http\Request), Object(Slim\Http\Response))
#2 C:\projects\flsite\index.php(16): Slim\App->run()
#3 {main}
'''
require 'vendor/autoload.php';
$app = new \Slim\App();
$app->get('/test', function($req, $res, $args) {
echo 'some nice output';
});
$app->add(function($req, $res, $next) use($app) {
$newResponse = $next($req, $res);
return $newResponse;
});
'''

You have to return a Response from the route's action callable:
$app->get('/test', function($req, $res, $args) {
   return $response->write('some nice output');
});

But this will force me to manually do output buffering (start my callable with ob_start and end with writing contents of ob_get_clean into response body) in each and every route callable. Previously this was handled automatically by framework.
My applications route definitions are structured like this:
$app->get('/', function($req, $res, $args) { return include('pages/home.php'); });
$app->get('/search', function($req, $res, $args) { return include('pages/search.php'); });
$app->get('/fragments/users', function($req, $res, $args) { return include('fragments/users.php'); });
etc.

Where each included file handles everything (setting page title, invoking 404 handler, outputting response body with echo or plain text...)
I am not using templating engine. Include files serve both controller and view functionality with plain PHP. This approach actually releaves me of unnecesary overhead of building variable arrays for passing to template object etc. And, believe or not, I am
I have two types of routes - "page routes" which represent a full page URL that visitor may visit and "fragment routes" which output HTML of some fragment of a page (think "partial views").
I use one application middleware callable which looks at the request URL and, if it does NOT start with "/fragments/", calls the next middleware (which corresponds to the actual page route callabe), saves its response body in a variable and pastes that variable inside a layout view (think "master view"), which, itself is just a plain PHP template.

Fixed in #1206.


Docker run

Use client instead of bash

Ah consistency.
Line https://github.com/eWaterCycle/leakybucket-bmi/blob/main/Dockerfile#L23 show you how to start and interact with it using BmiClientDocker.
Line https://github.com/eWaterCycle/leakybucket-bmi/blob/main/Dockerfile#L27 is to debug when things dont work.
I think the README should contain both parts. Showing how to use it is a very nice thing.


[FEATURE] Dynamic damage on attacks

Is your feature request related to a problem? Please describe.
A lot of summons increases the damage to attacks when the spell is upcast. Currently it's not possible to increase the damage on attacks on an actor sheet through the summoning. (to my knowledge, but it might be possible with attributes?)
Describe the solution you'd like
An option to add damage to certain attacks based on level.

The summoning spell's roll data is made available to the summoned actor, so you can add @summoner.item.level into a damage formula to get the spell level that was used for the summoning. Check out the readme for more info.


Add a subscription_list_title_prefix

All other finders have this, and we have a problem with our topic
only displaying as '#' on email signup (they still work).
Also, remove the topics key containing some/dummy-topic. It isn't
required.

👍


Response speed slowed with Elasticsearch-6.

Consider measures to improve processing speed.

Released v1_es6.6.1_3


Rotating Animation Navigation

Project title
Rotating Animation Navigation
Project Description
A site that rotates fully on landing page.
Stack:
And in order to tick the check box just but x inside them for example - [x] like this. Please delete options that are not relevant.

[x] Html
[x] CSS
[x] JavaScript
[ ] ReactJS

/assign


Tease apart archetype and catalog generation, so we can generate the Catalog after the artifacts are built so we only include built archetypes in the catalog.

[lgtm][merge]

Code not merged because @chirino (Hiram Chirino) is not in the Admin list.

[merge]

[merge]

BTW to make the archetypes get built first we could just change the order in the parent pom
https://github.com/fabric8io/ipaas-quickstarts/blob/master/pom.xml#L101


mouse event not working while playing video

VlcControl add on WinForm, There is no any mouse event fire while playing video

You'll need to use: https://github.com/higankanshi/xZune.Vlc

I notice that https://github.com/higankanshi/xZune.Vlc is only for wpf, no for WinForm. Any solution to this issue?

I have the same problem in a WinForm App .It seems that the released vlc lib has not throw the mouse event .You should modify the source code of vlc to make it  response .

+1

Use this VLC control which i have created. It supports clicks, mouseclicks, mouse move, scroll etc and also it has auto cursor hide after 2 seconds just like how it does in actual VLC player. I am creating a video player with this VLC dot net will upload soon. :)
VlcControlV2.zip

Mouse events are automatically captured by the Vlc control. We probably could add a way to let events fall through by implementing libvlc_video_set_mouse_input
see https://github.com/videolan/vlc/blob/9dce7032797ed9ab54e46f944bb7f800bd69cee8/include/vlc/libvlc_media_player.h#L1098

Hi,
Sorry for my inappropriate question,
Last update this issue have "28 Nov 2017". At this moment 16.05.2018.
Do you know ho fixing this issue or find a workaround?

This still need to be implemented in Vlc.DotNet, either by me or by any volunteer.
As for a workaround, I suppose that you could show an invisible control on top of the Vlc control that would handle the right click. I agree, this is far from perfect.

Hi jeremyVignelles.
Sorry but i am very bad developer, and my english is also very bed.
But i found how to create "invisible control" and it is work. Link

Hi @aap4ixx . Did you try using my control?
https://github.com/ZeBobo5/Vlc.DotNet/files/1010749/VlcControlV2.zip

Hi @krrrishh.
No i didn't use you code. Because i found normal code.
Thanks.

@krrrishh  I do use your control,but it don't work, Can you give us  a full demo,thanks

I've downloaded the new files but still mouse events doesn't work.
is it fixed for you ?

Did you try the 3.0.0 prerelease?

yes I use 3.0.0 prerelease,
according to dear @CrookedFingerGuy's guidence in this page
my mouse and keyboard event handlers are working right now.
 public Sample()
        {
            InitializeComponent();
            myVlcControl.Video.IsMouseInputEnabled = false;
            myVlcControl.Video.IsKeyInputEnabled = false;
        }

Thanks you all


Two password reset emails being sent (custom PasswordsController)

I'm having an issue in production where users are requesting pw resets and getting two emails instead of one. It has caused confusion because each time at least one of the pw reset tokens conveyed in the multiple emails is of course invalid and users who don't realize that more than one was sent and try the wrong one get an invalid token error when trying to use the link in the email.
I customized the Devise::PasswordsController to respond both to a normal html request as well as an Ajax request for use on a different login form that I didn't want the user to be redirected away from. I'm assuming this customization introduced the problem but I have been reviewing my code and it isn't clear to me what I would've done to cause this behavior.
Here are my server logs from a normal HTML request that is generating two emails:
Started POST "/users/password" for 174.xx.xx.xxx at 2014-11-20 20:18:46 -0500
Processing by PasswordsController#create as HTML
Parameters: {"utf8"=>"✓",  "authenticity_token"=>"grk1shn96l4fHhKxzhuVDmala4FYbMhHRjrESTxdhZg="<EMAIL_ADDRESS>"commit"=>"Send me reset password instructions"}

Rendered users/mailer/reset_password_instructions.html.haml (0.4ms)
Sent mail to<EMAIL_ADDRESS>(5586.8ms)
Rendered users/mailer/reset_password_instructions.html.haml (0.4ms)
Sent mail to<EMAIL_ADDRESS>(342.2ms)
Redirected to http://myapp.com/users/sign_in`

As you can see, two emails were sent before the redirect took place. Now here is a log from when I made the pw reset request as JS from the other form:
Started POST "/users/password" for 174.xx.xx.xxx at 2014-11-20 20:33:10 -0500
Processing by PasswordsController#create as JS
Parameters: {"utf8"=>"✓",   "authenticity_token"=>"PASKZu73jiKs7+9NpNZLLL8W63vBOmsq36FRWFn4hVE=", "user"=><EMAIL_ADDRESS>"commit"=>"Email reset instructions"}

Rendered users/mailer/reset_password_instructions.html.haml (0.3ms)
Sent mail to<EMAIL_ADDRESS>(6673.2ms)
Rendered passwords/new_job_pw_reset.js.erb (0.3ms)`

With the JS request, only one pw reset email was generated. Here is my controller:
class PasswordsController < Devise::PasswordsController
def create
	self.resource = resource_class.send_reset_password_instructions(resource_params)
	yield resource if block_given?

	respond_to do |format|
		if successfully_sent?(resource)
			format.js   { render template: "passwords/new_job_pw_reset" }
			format.html { super }
		else
			format.js { render template: "passwords/new_job_pw_reset_fail" }
			format.html { super }
		end
	end
end
end

I used the standard controller here at Devise's GitHub as a template. The part I changed is everything after the line yield resource if block_given?.
Can anyone help me figure out what I've done here that causes two emails to be sent from an HTML request?
Rails 3.2.18
Devise 3.2.4

I am also facing same problem.
Here is my PasswordsController.
class PasswordsController< Devise::PasswordsController
layout 'admin_dashboard'
respond_to :json, only: [:create]
prepend_before_filter :require_no_authentication
# Render the #edit only if coming from a reset password email link
append_before_filter :assert_reset_token_passed, only: :edit
# GET /resource/password/new
def new
	self.resource = resource_class.new
end
# POST /resource/password
def create
	user = resource_class.where(resource_params).first
 	if user
 		data = { status: 200, email: user.email}
 		resource_class.send_reset_password_instructions(user)
 		render json: data
	else
		data = { status: 400,errors: "Email not found"}
		render json: data
	end
end
# # GET /resource/password/edit?reset_password_token=abcdef
def edit
	self.resource = resource_class.new
	set_minimum_password_length
	resource.reset_password_token = params[:reset_password_token]
end
# # PUT /resource/password
def update
	self.resource = resource_class.reset_password_by_token(resource_params)
	yield resource if block_given?
	if resource.errors.empty?
		if Devise.sign_in_after_reset_password
			flash_message = resource.active_for_authentication? ? :updated : :updated_not_active
			set_flash_message(:notice, flash_message) if is_flashing_format?
			sign_in(resource_name, resource)
		else
			set_flash_message(:notice, :updated_not_active) if is_flashing_format?
		end
		respond_with resource, location: bvadmin_dashboard_path
	else
		respond_with resource
	end
end

protected

def after_resetting_password_path_for(resource)
	Devise.sign_in_after_reset_password ? after_sign_in_path_for(resource) : new_admin_session_path(resource_name)
end
# # The path used after sending reset password instructions

# def after_sending_reset_password_instructions_path_for(resource_name)
# 	admin_session_path(resource_name) if is_navigational_format?
# end

# # Check if a reset_password_token is provided in the request
def assert_reset_token_passed
	if params[:reset_password_token].blank?
	set_flash_message(:alert, :no_token)
	redirect_to new_admin_session_path(resource_name)
	end
end

def translation_scope
'devise.passwords'
end

def resource_params
	params.require(:user).permit(:email,:reset_password_token, :password, :password_confirmation)
end

end


Frontend/feature/signup v2

Frontend checklist

[x] Formatted my code with yarn format && yarn lint --fix
[x] There are no warnings from yarn lint
[x] There are no console warnings when running the app
[ ] Added any new components to storybook
[x] Added tests where relevant
[ ] All tests pass
[x] Clicked around my changes running locally and it works
[x] Checked Desktop, Mobile and Tablet screen sizes

Are there more details on how this new sign up flow works?

There are comments in the proto files

Also the tests are failing in CI crashing on createEvent - perhaps some async stuff never got returned when you expect them to or forgot an await somewhere?

Ugh these are super hard to debug. I will try running with -i to see what is causing the issue

When I run tests sequentially, I don't get any error. When I run --all, I get failures and the createEvent issue but at the end, all tests have passed. At a bit of a loss now.

Says the tests failed again... any idea which tests are being flaky?

I really have no idea. Is there a way we can proxy the api to find out what is not being mocked?

Do you mean similar to what you did with the msw? Might be tricky with grpc. But everything interacts with the API is done through service and that is mocked, so everything should be mocked? Maybe there is some async update we aren't waiting for in the tests before letting it finish? Seems to be the most common answer when searching for this error...

I mean proxy the service later so we know what is being called. Like in storybook, if you make an API call that doesn't have a mock implementation, you get a console warning

The best I can think of is probably iterating through all of their service functions and give it a default mock implementation that log the warning if it's then not replaced by the individual test files later on.

This is a bit of a weird one as I still get the warning (using yarn test-ci) even on develop, but it's fine on CI...


Add Torrent Tracker Lists

Hello I was looking for the option of adding either 'custom' torrent trackers, or just making the default addTorrent function on line 235, make an additional callback and add an additional list of torrent trackers.
I have the payload of what the UI makes adding these via the webGui. Was hoping we could work together on it. Happy to shoot a PR if you can provide a little guidance. On where to add an additional post would be needed to the "core.set_torrent_trackers".
Hopefully its clear enough above, but happy to create screenshots of adding trackers manually.

Sounds pretty straightforward. Could you post an example payload for that
request?
ginman
github https://github.com/ginman86
linkedIn http://www.linkedin.com/pub/greg-inman/46/1ba/849
On Tue, Nov 27, 2018 at 6:45 AM guy16510<EMAIL_ADDRESS>wrote:

Hello I was looking for the option of adding either 'custom' torrent
trackers, or just making the default addTorrent function on line 235, make
an additional callback and add an additional list of torrent trackers.
I have the payload of what the UI makes adding these via the webGui. Was
hoping we could work together on it. Happy to shoot a PR if you can provide
a little guidance. On where to add an additional post would be needed to
the "core.set_torrent_trackers".
Hopefully its clear enough above, but happy to create screenshots of
adding trackers manually.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ginman86/deluge/issues/17, or mute the thread
https://github.com/notifications/unsubscribe-auth/ADWixGYHHA659DJWtys5geUZi5YBSm-xks5uzUIUgaJpZM4Y1hVG
.

Sweet, thanks man. So I generally grab the torrent trackers from: http://www.torrenttrackerlist.com/torrent-tracker-list/
I believe the params array ("120e24d62d0daf2537b4b0ae696f60931a6217b2") is the GUID of the torrent. I mentioned above, happy to help where I can sir. Thank you.
POST http://<IP_ADDRESS>:8112/json
Payload:
{"method":"core.set_torrent_trackers","params":["120e24d62d0daf2537b4b0ae696f60931a6217b2",[{"tier":0,"url":"http://tracker.trackerfix.com:80/announce"},{"tier":0,"url":"udp://9.rarbg.me:2710"},{"tier":0,"url":"udp://9.rarbg.to:2710"},{"tier":1,"url":"udp://public.popcorn-tracker.org:6969/announce"},{"tier":2,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":3,"url":"http://<IP_ADDRESS>:2710/announce"},{"tier":4,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":5,"url":"udp://tracker.ilibr.org:80/announce"},{"tier":6,"url":"http://atrack.pow7.com/announce"},{"tier":7,"url":"http://bt.henbt.com:2710/announce"},{"tier":8,"url":"http://mgtracker.org:2710/announce"},{"tier":9,"url":"http://mgtracker.org:6969/announce"},{"tier":10,"url":"http://open.touki.ru/announce.php"},{"tier":11,"url":"http://p4p.arenabg.ch:1337/announce"},{"tier":12,"url":"http://pow7.com:80/announce"},{"tier":13,"url":"http://retracker.krs-ix.ru:80/announce"},{"tier":14,"url":"http://secure.pow7.com/announce"},{"tier":15,"url":"http://t1.pow7.com/announce"},{"tier":16,"url":"http://t2.pow7.com/announce"},{"tier":17,"url":"http://thetracker.org:80/announce"},{"tier":18,"url":"http://torrentsmd.com:8080/announce"},{"tier":19,"url":"http://tracker.bittor.pw:1337/announce"},{"tier":20,"url":"http://tracker.dutchtracking.com:80/announce"},{"tier":21,"url":"http://tracker.dutchtracking.nl:80/announce"},{"tier":22,"url":"http://tracker.edoardocolombo.eu:6969/announce"},{"tier":23,"url":"http://tracker.ex.ua:80/announce"},{"tier":24,"url":"http://tracker.kicks-ass.net:80/announce"},{"tier":25,"url":"http://tracker1.wasabii.com.tw:6969/announce"},{"tier":26,"url":"http://tracker2.itzmx.com:6961/announce"},{"tier":27,"url":"http://www.wareztorrent.com:80/announce"},{"tier":28,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":29,"url":"udp://eddie4.nl:6969/announce"},{"tier":30,"url":"udp://explodie.org:6969/announce"},{"tier":31,"url":"udp://shadowshq.eddie4.nl:6969/announce"},{"tier":32,"url":"udp://shadowshq.yi.org:6969/announce"},{"tier":33,"url":"udp://tracker.eddie4.nl:6969/announce"},{"tier":34,"url":"udp://tracker.mg64.net:2710/announce"},{"tier":35,"url":"udp://tracker.sktorrent.net:6969"},{"tier":36,"url":"udp://tracker2.indowebster.com:6969/announce"},{"tier":37,"url":"udp://tracker4.piratux.com:6969/announce"},{"tier":38,"url":"http://atrack.pow7.com/announce"},{"tier":39,"url":"http://bt.henbt.com:2710/announce"},{"tier":40,"url":"http://mgtracker.org:2710/announce"},{"tier":41,"url":"http://mgtracker.org:6969/announce"},{"tier":42,"url":"http://open.touki.ru/announce.php"},{"tier":43,"url":"http://p4p.arenabg.ch:1337/announce"},{"tier":44,"url":"http://pow7.com:80/announce"},{"tier":45,"url":"http://retracker.krs-ix.ru:80/announce"},{"tier":46,"url":"http://secure.pow7.com/announce"},{"tier":47,"url":"http://t1.pow7.com/announce"},{"tier":48,"url":"http://t2.pow7.com/announce"},{"tier":49,"url":"http://thetracker.org:80/announce"},{"tier":50,"url":"http://torrentsmd.com:8080/announce"},{"tier":51,"url":"http://tracker.bittor.pw:1337/announce"},{"tier":52,"url":"http://tracker.dutchtracking.com/announce"},{"tier":53,"url":"http://tracker.dutchtracking.com:80/announce"},{"tier":54,"url":"http://tracker.dutchtracking.nl:80/announce"},{"tier":55,"url":"http://tracker.edoardocolombo.eu:6969/announce"},{"tier":56,"url":"http://tracker.ex.ua:80/announce"},{"tier":57,"url":"http://tracker.kicks-ass.net:80/announce"},{"tier":58,"url":"http://tracker.mg64.net:6881/announce"},{"tier":59,"url":"http://tracker.tfile.me/announce"},{"tier":60,"url":"http://tracker1.wasabii.com.tw:6969/announce"},{"tier":61,"url":"http://tracker2.itzmx.com:6961/announce"},{"tier":62,"url":"http://tracker2.wasabii.com.tw:6969/announce"},{"tier":63,"url":"http://www.wareztorrent.com:80/announce"},{"tier":64,"url":"udp://bt.xxx-tracker.com:2710/announce"},{"tier":65,"url":"udp://eddie4.nl:6969/announce"},{"tier":66,"url":"udp://shadowshq.eddie4.nl:6969/announce"},{"tier":67,"url":"udp://shadowshq.yi.org:6969/announce"},{"tier":68,"url":"udp://tracker.eddie4.nl:6969/announce"},{"tier":69,"url":"udp://tracker.mg64.net:2710/announce"},{"tier":70,"url":"udp://tracker.mg64.net:6969/announce"},{"tier":71,"url":"udp://tracker.opentrackr.org:1337/announce"},{"tier":72,"url":"udp://tracker.sktorrent.net:6969"},{"tier":73,"url":"udp://tracker2.indowebster.com:6969/announce"},{"tier":74,"url":"udp://tracker4.piratux.com:6969/announce"},{"tier":75,"url":"udp://tracker.coppersurfer.tk:6969/announce"},{"tier":76,"url":"http://tracker.opentrackr.org:1337/announce"},{"tier":77,"url":"udp://zer0day.ch:1337/announce"},{"tier":78,"url":"udp://zer0day.to:1337/announce"},{"tier":79,"url":"http://explodie.org:6969/announce"},{"tier":80,"url":"udp://tracker.leechers-paradise.org:6969/announce"},{"tier":81,"url":"udp://9.rarbg.com:2710/announce"},{"tier":82,"url":"udp://9.rarbg.me:2780/announce"},{"tier":83,"url":"udp://9.rarbg.to:2730/announce"},{"tier":84,"url":"udp://p4p.arenabg.com:1337/announce"},{"tier":85,"url":"udp://tracker.sktorrent.net:6969/announce"},{"tier":86,"url":"http://p4p.arenabg.com:1337/announce"},{"tier":87,"url":"udp://tracker.aletorrenty.pl:2710/announce"},{"tier":88,"url":"http://tracker.aletorrenty.pl:2710/announce"},{"tier":89,"url":"http://tracker.bittorrent.am/announce"},{"tier":90,"url":"udp://tracker.kicks-ass.net:80/announce"},{"tier":91,"url":"http://tracker.kicks-ass.net/announce"},{"tier":92,"url":"http://tracker.baravik.org:6970/announce"},{"tier":93,"url":"udp://torrent.gresille.org:80/announce"},{"tier":94,"url":"http://torrent.gresille.org/announce"},{"tier":95,"url":"http://tracker.skyts.net:6969/announce"},{"tier":96,"url":"http://tracker.internetwarriors.net:1337/announce"},{"tier":97,"url":"udp://tracker.skyts.net:6969/announce"},{"tier":98,"url":"http://tracker.dutchtracking.nl/announce"},{"tier":99,"url":"udp://tracker.yoshi210.com:6969/announce"},{"tier":100,"url":"udp://tracker.tiny-vps.com:6969/announce"},{"tier":101,"url":"udp://tracker.internetwarriors.net:1337/announce"},{"tier":102,"url":"udp://mgtracker.org:2710/announce"},{"tier":103,"url":"http://tracker.yoshi210.com:6969/announce"},{"tier":104,"url":"http://tracker.tiny-vps.com:6969/announce"},{"tier":105,"url":"udp://tracker.filetracker.pl:8089/announce"},{"tier":106,"url":"udp://tracker.ex.ua:80/announce"},{"tier":107,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":108,"url":"https://www.wareztorrent.com/announce"},{"tier":109,"url":"http://www.wareztorrent.com/announce"},{"tier":110,"url":"http://tracker.filetracker.pl:8089/announce"},{"tier":111,"url":"http://tracker.ex.ua/announce"},{"tier":112,"url":"http://tracker.calculate.ru:6969/announce"},{"tier":113,"url":"udp://tracker.grepler.com:6969/announce"},{"tier":114,"url":"udp://tracker.flashtorrents.org:6969/announce"},{"tier":115,"url":"udp://tracker.bittor.pw:1337/announce"},{"tier":116,"url":"http://tracker.tvunderground.org.ru:3218/announce"},{"tier":117,"url":"http://tracker.grepler.com:6969/announce"},{"tier":118,"url":"http://tracker.flashtorrents.org:6969/announce"},{"tier":119,"url":"http://retracker.gorcomnet.ru/announce"},{"tier":120,"url":"http://bt.pusacg.org:8080/announce"},{"tier":121,"url":"http://<IP_ADDRESS>:8080/announce"},{"tier":122,"url":"udp://tracker.kuroy.me:5944/announce"},{"tier":123,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":124,"url":"http://tracker.kuroy.me:5944/announce"},{"tier":125,"url":"http://retracker.krs-ix.ru/announce"},{"tier":126,"url":"http://open.acgtracker.com:1096/announce"},{"tier":127,"url":"udp://open.stealth.si:80/announce"},{"tier":128,"url":"udp://<IP_ADDRESS>:8000/announce"},{"tier":129,"url":"http://tracker.dler.org:6969/announce"},{"tier":130,"url":"http://bt2.careland.com.cn:6969/announce"},{"tier":131,"url":"http://open.lolicon.eu:7777/announce"},{"tier":132,"url":"http://tracker.opentrackr.org:1337/announce"},{"tier":133,"url":"http://explodie.org:6969/announce"},{"tier":134,"url":"http://p4p.arenabg.com:1337/announce"},{"tier":135,"url":"http://tracker.aletorrenty.pl:2710/announce"},{"tier":136,"url":"http://tracker.bittorrent.am/announce"},{"tier":137,"url":"http://tracker.kicks-ass.net/announce"},{"tier":138,"url":"http://tracker.baravik.org:6970/announce"},{"tier":139,"url":"http://torrent.gresille.org/announce"},{"tier":140,"url":"http://tracker.skyts.net:6969/announce"},{"tier":141,"url":"http://tracker.internetwarriors.net:1337/announce"},{"tier":142,"url":"http://tracker.dutchtracking.nl/announce"},{"tier":143,"url":"http://tracker.yoshi210.com:6969/announce"},{"tier":144,"url":"http://tracker.tiny-vps.com:6969/announce"},{"tier":145,"url":"http://www.wareztorrent.com/announce"},{"tier":146,"url":"http://tracker.filetracker.pl:8089/announce"},{"tier":147,"url":"http://tracker.ex.ua/announce"},{"tier":148,"url":"http://tracker.calculate.ru:6969/announce"},{"tier":149,"url":"http://tracker.tvunderground.org.ru:3218/announce"},{"tier":150,"url":"http://tracker.grepler.com:6969/announce"},{"tier":151,"url":"http://tracker.flashtorrents.org:6969/announce"},{"tier":152,"url":"http://retracker.gorcomnet.ru/announce"},{"tier":153,"url":"http://bt.pusacg.org:8080/announce"},{"tier":154,"url":"http://<IP_ADDRESS>:8080/announce"},{"tier":155,"url":"http://tracker.kuroy.me:5944/announce"},{"tier":156,"url":"http://retracker.krs-ix.ru/announce"},{"tier":157,"url":"http://open.acgtracker.com:1096/announce"},{"tier":158,"url":"http://bt2.careland.com.cn:6969/announce"},{"tier":159,"url":"http://open.lolicon.eu:7777/announce"},{"tier":160,"url":"https://www.wareztorrent.com/announce"},{"tier":161,"url":"udp://<IP_ADDRESS>:1337/announce"},{"tier":162,"url":"http://<IP_ADDRESS>:1337/announce"},{"tier":163,"url":"udp://<IP_ADDRESS>:1337/announce"},{"tier":164,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":165,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":166,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":167,"url":"udp://<IP_ADDRESS>:2710/announce"},{"tier":168,"url":"udp://<IP_ADDRESS>:1337/announce"},{"tier":169,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":170,"url":"http://<IP_ADDRESS>:1337/announce"},{"tier":171,"url":"udp://<IP_ADDRESS>:2710/announce"},{"tier":172,"url":"http://<IP_ADDRESS>:2710/announce"},{"tier":173,"url":"http://<IP_ADDRESS>/announce"},{"tier":174,"url":"udp://<IP_ADDRESS>:80/announce"},{"tier":175,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":176,"url":"http://<IP_ADDRESS>/announce"},{"tier":177,"url":"http://<IP_ADDRESS>:6970/announce"},{"tier":178,"url":"udp://<IP_ADDRESS>:1337/announce"},{"tier":179,"url":"http://<IP_ADDRESS>/announce"},{"tier":180,"url":"udp://<IP_ADDRESS>:80/announce"},{"tier":181,"url":"http://<IP_ADDRESS>/announce"},{"tier":182,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":183,"url":"http://<IP_ADDRESS>:1337/announce"},{"tier":184,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":185,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":186,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":187,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":188,"url":"udp://<IP_ADDRESS>:1337/announce"},{"tier":189,"url":"udp://<IP_ADDRESS>:2710/announce"},{"tier":190,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":191,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":192,"url":"udp://<IP_ADDRESS>:8089/announce"},{"tier":193,"url":"udp://<IP_ADDRESS>:80/announce"},{"tier":194,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":195,"url":"https://<IP_ADDRESS>/announce"},{"tier":196,"url":"http://<IP_ADDRESS>/announce"},{"tier":197,"url":"http://<IP_ADDRESS>:8089/announce"},{"tier":198,"url":"http://<IP_ADDRESS>/announce"},{"tier":199,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":200,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":201,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":202,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":203,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":204,"url":"udp://<IP_ADDRESS>:1337/announce"},{"tier":205,"url":"http://<IP_ADDRESS>:3218/announce"},{"tier":206,"url":"http://<IP_ADDRESS>:6881/announce"},{"tier":207,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":208,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":209,"url":"http://<IP_ADDRESS>:8080/announce"},{"tier":210,"url":"http://<IP_ADDRESS>/announce"},{"tier":211,"url":"http://<IP_ADDRESS>:8080/announce"},{"tier":212,"url":"http://<IP_ADDRESS>:8080/announce"},{"tier":213,"url":"udp://<IP_ADDRESS>:5944/announce"},{"tier":214,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":215,"url":"http://<IP_ADDRESS>:5944/announce"},{"tier":216,"url":"http://<IP_ADDRESS>:1337/announce"},{"tier":217,"url":"http://<IP_ADDRESS>/announce"},{"tier":218,"url":"http://<IP_ADDRESS>:1096/announce"},{"tier":219,"url":"udp://<IP_ADDRESS>:80/announce"},{"tier":220,"url":"udp://<IP_ADDRESS>:2710/announce"},{"tier":221,"url":"udp://<IP_ADDRESS>:8000/announce"},{"tier":222,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":223,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":224,"url":"http://<IP_ADDRESS>/announce"},{"tier":225,"url":"http://<IP_ADDRESS>:7777/announce"},{"tier":226,"url":"udp://tracker.coppersurfer.tk:6969/announce"},{"tier":227,"url":"udp://zer0day.ch:1337/announce"},{"tier":228,"url":"udp://tracker.leechers-paradise.org:6969/announce"},{"tier":229,"url":"udp://9.rarbg.com:2710/announce"},{"tier":230,"url":"udp://p4p.arenabg.com:1337/announce"},{"tier":231,"url":"udp://tracker.sktorrent.net:6969/announce"},{"tier":232,"url":"udp://tracker.aletorrenty.pl:2710/announce"},{"tier":233,"url":"udp://tracker.kicks-ass.net:80/announce"},{"tier":234,"url":"udp://torrent.gresille.org:80/announce"},{"tier":235,"url":"udp://tracker.skyts.net:6969/announce"},{"tier":236,"url":"udp://tracker.yoshi210.com:6969/announce"},{"tier":237,"url":"udp://tracker.tiny-vps.com:6969/announce"},{"tier":238,"url":"udp://tracker.internetwarriors.net:1337/announce"},{"tier":239,"url":"udp://mgtracker.org:2710/announce"},{"tier":240,"url":"udp://tracker.filetracker.pl:8089/announce"},{"tier":241,"url":"udp://tracker.ex.ua:80/announce"},{"tier":242,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":243,"url":"udp://tracker.grepler.com:6969/announce"},{"tier":244,"url":"udp://tracker.flashtorrents.org:6969/announce"},{"tier":245,"url":"udp://tracker.bittor.pw:1337/announce"},{"tier":246,"url":"udp://tracker.kuroy.me:5944/announce"},{"tier":247,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":248,"url":"udp://open.stealth.si:80/announce"},{"tier":249,"url":"udp://<IP_ADDRESS>:8000/announce"},{"tier":250,"url":"udp://tracker.coppersurfer.tk:6969/announce"},{"tier":251,"url":"http://tracker.opentrackr.org:1337/announce"},{"tier":252,"url":"udp://zer0day.ch:1337/announce"},{"tier":253,"url":"http://explodie.org:6969/announce"},{"tier":254,"url":"udp://tracker.leechers-paradise.org:6969/announce"},{"tier":255,"url":"udp://9.rarbg.com:2710/announce"},{"tier":256,"url":"udp://p4p.arenabg.com:1337/announce"},{"tier":257,"url":"udp://tracker.sktorrent.net:6969/announce"},{"tier":258,"url":"http://p4p.arenabg.com:1337/announce"},{"tier":259,"url":"udp://tracker.aletorrenty.pl:2710/announce"},{"tier":260,"url":"http://tracker.aletorrenty.pl:2710/announce"},{"tier":261,"url":"http://tracker.bittorrent.am/announce"},{"tier":262,"url":"udp://tracker.kicks-ass.net:80/announce"},{"tier":263,"url":"http://tracker.kicks-ass.net/announce"},{"tier":264,"url":"http://tracker.baravik.org:6970/announce"},{"tier":265,"url":"udp://tracker.piratepublic.com:1337/announce"},{"tier":266,"url":"udp://<IP_ADDRESS>:1337/announce"},{"tier":267,"url":"http://<IP_ADDRESS>:1337/announce"},{"tier":268,"url":"udp://<IP_ADDRESS>:1337/announce"},{"tier":269,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":270,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":271,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":272,"url":"udp://<IP_ADDRESS>:2710/announce"},{"tier":273,"url":"udp://<IP_ADDRESS>:1337/announce"},{"tier":274,"url":"udp://<IP_ADDRESS>:6969/announce"},{"tier":275,"url":"http://<IP_ADDRESS>:1337/announce"},{"tier":276,"url":"udp://<IP_ADDRESS>:2710/announce"},{"tier":277,"url":"http://<IP_ADDRESS>:2710/announce"},{"tier":278,"url":"http://<IP_ADDRESS>/announce"},{"tier":279,"url":"udp://<IP_ADDRESS>:80/announce"},{"tier":280,"url":"http://<IP_ADDRESS>:6969/announce"},{"tier":281,"url":"http://<IP_ADDRESS>/announce"}]],"id":75}


[WIP] Ability to auto-bootstrap module requirements on-demand

Proposal: Ability to auto-bootstrap module requirements on-demand
Author: Sorin Sbarnea <@ssbarnea> IRC: zbr
Date: 2019-03-08

Status: New
Proposal type: core
Targeted release: <future release #, ex. 2.2>
Associated PR: <link to GH PR in ansible/proposals if PR was submitted>
Estimated time to implement: <X days, weeks, etc.>

Motivation / Problems
Quite often ansible users discover that their (core) module is not working on the target system because they are missing a python package on this system.
Obviously that the presence of the required python modules must be cached as a fact as we do not want to re-check their presence on each execution.
Solution proposal 1
Fixing this requires extra bootstrapping tasks which require additional time to write/test/install and we could probably improve the user experience by gradually adding auto-bootrapping support to modules.
For example openstack modules do require shade python library so they should expose the required module in their metadata and ansible should attempt to verify their presense and run the bootstrapping code(role?) before running the module.
Solution proposal 2
Alternatively we could think about a different approach where the module itself could raise a special exception similar to ImportError when an import fails that triggers the bootstrapping logic in Ansible.
If bootstrapping fails module fails, nothing is changed. Still, if bootstrapping suceeds the same task is called again and will likely succeed.
Testing (optional)
TBD
Documentation (optional)
Does / should this require documentation? If so, describe here. This section is optional but could be helpful.
Anything else?
If you'd like to add anything else beyond the above required and optional sections, you are welcome to do so.

There are a few things to consider here:

pip vs. system package manager for installing deps
for installation using pip, system level or --user?
what about python packages that require a compiler and/or C libraries/headers?
do we automatically upgrade/downgrade python libs that already exist in the env?
how do modules report to the user that they took these extra steps?
how does this work for utility programs needed by a module?
how does this work outside of python modules (such as powershell)?

A few more things:

installing via pip/npm/etc or via OS package manager is very dependent on 'policy' in the user's environment
The user 'using the module' might not have rights to install anything and can get into trouble if they do 'install locally' .. many environments have strict policies against that.
The module/plugin author would now have to supply all possible 'package names', they can be very different across platforms for the same library
Version management is also different for the platforms, which is a problem for when a specific version or range of versions is required.
What do we do when the requirement is the 'interpreter version'? i.e python >= 3.5, ruby <=2.3

I originally posted over in https://github.com/ansible/proposals/issues/57 about this, but I think it fits more here—Execution Environments are in need of a standard way to define requirements for collection plugins and other content, and they've started to use this layout: https://github.com/ansible-collections/community.kubernetes/pull/211

Yes, there is some stuff in-progress at https://github.com/ansible/ansible-builder, you may want to skim over the top-level README file, and then any feedback would be appreciated.

Quite often ansible users discover that their (core) module is not working on the target system because they are missing a python package on this system.

A big distinction between that effort and this discussion is that I strongly want the role of ansible-builder to be limited to the control machine, as opposed to the targeted hosts. For many plugin types like lookup plugins, inventory plugins, etc. you need the dependencies installed in the environment where ansible-playbook is called. You also really don't want that call to make unprompted changes to the system. That same machine may still run modules that interact with web APIs as local actions.
For modules running on some target host, it makes sense that you may want to use other modules in order to assure that dependencies are satisfied. That's a very different form of problem in my mind.

Aside from asnible-builder and execution environments, with collections now you can also have 'boostrap plays' included in the plays directory. I would encourage a 'install_runtime_dependencies.yml' convention and tags based on plugin/module required.

Why not extending requirements.yml with new properties in addition to roles and collections?

because not all requirements are a simple  in many environments you MUST use OS package, in others you don't have access to pypi.org .. and requirements are not all just python libs.
There is no universal format to handle all the transliterations of a library's/cli tool's package (pyyaml vs libpyyaml vs pythonX-pyyaml), why sometimes a playbook with many options will be only way to cover all.

That makes sense, it as not clear that we were talking about a bootstrapping playbook.
Still, this means that it should be stored inside playbooks/ folder, just to follow the already documented layout. I do not fancy the idea of diverging from that layout.

not adding new layout, just naming convention


Simple way to make individual attributes immutable?

I'm a bad functional programmer and want to mutate parts of my classes. But it'd be nice if there were still a way to mark the parts of my class that are immutable as being immutable in a few key strokes. I usually won't bother if I have to work for it (I'm also lazy), but might catch some bugs. It seems this fits into attrs' general style. attr.ib(readonly=True)? Could be implemented as either a __setattr__ hook like frozen, or with a property.

Well, hm.  It would be frozen=True to mirror the main option but it really sounds like a lot of complexity/rewriting…

This turns out to be super useful when retrofitting existing codebases to use attrs.

I suggested readonly instead of frozen because I thought frozen might suggest that the object stored there was immutable? Either could work really. And I can't comment on the implementation complexity, I just thought I'd throw it out there as an idle wish :-)

Quoth @lukasa: I’d merge a (good) PR. ;)

Could you provide some pointers as to where one would start, and what a good PR might look like?

Maybe @Tinche can be more helpful but I'd guess that we could adapt _frozen_setattrs or something?
The reason for my handwavevey delegation is mostly b/c I dunno. :)

I'm not really sure how you'd go about implementing this.
One way: for a field x, implement a shadow field _x and create a property without a setter. The equivalent of
class A:
    def __init__(self, x):
        self._x = x
    @property
    def x(self):
        return self._x

This is how it's normally done when using manual classes.
Another way would be to install a custom __setattr__ (and __delattr__) that basically looks like this:
class A:
    def __setattr__(self, name, value):
        if name in __frozen_attr_names__:
            raise FrozenAttributeError('{} is frozen'.format(name))
        else:
            object.__setattr__(self, name, value)

This will impose a penalty on every assignment and every __init__ (since you'd have to use object.__setattr__ in __init__ to set the initial value).
I think the first approach is a little better.
Inheritance needs to be taken into account as well.
Maybe there are more ingenious approaches, so comments welcome :)

I was approaching it from the descriptor angle, to be honest. I'm not interested in blocking serious workaround, just making readonly properties relatively easy:
class Attribute:
    def __set__(self, obj, value):
        raise FrozenAttributeError()

Would that not be a direction to start in?

How about using having a frozen instance and use evolve for mutable attributes? Too much overhead?

class FrozenAttribute:
    def __set__(self, obj, value):
        raise AttributeError

class A:
    x = FrozenAttribute()

    def __init__(self, x):
        self.__dict__['x'] = x

>>> a = A(1)
>>> a.x
1
>>> a.x = 2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "tt.py", line 9, in __set__
    raise AttributeError
AttributeError

Something like this, you mean? It's an interesting approach, but it won't work with slot classes. Maybe it could be made to support slot classes with a little more investigation though.

Gah, if I ever meet @gvanrossum at a conference, I will buy him a drink and then continue to complain for 10 minutes about how immutability sucks in Python.

Alright, I've been poking at frozen attributes for slot classes for a few hours now.
Good news: I actually understand Python descriptors now! (I think.) 🤓
Bad news: I don't think we can have frozen attributes for slot classes without speed penalties. 😡
Mediocre news: With a bit of C code we could have perfectly immutable zero-overhead slot classes and attributes. 🤔
Let me elaborate.
Slot classes store their instance contents in a hidden array. As far as I can tell, this space is basically inaccessible directly from Python. This is different from normal classes, which use instance.__dict__, and we can poke at this from Python with no issues.
When a slot class is defined, this space is allocated somewhere and a descriptor is made for each attribute. This descriptor is the only way to actually access this space. The descriptors are then placed into the class __dict__ (descriptors are always placed there).
These descriptors are of type member_descriptor, and are defined in descobject.c. The member_descriptor class/struct is very limited though. I tried subclassing the type - they can't be subclassed. I can wrap them with a different descriptor, hiding the __set__ and __delete__ and proxying __get__, but I can't use their __get__ directly, so there's always the overhead of wrapping so attribute access becomes ~4x slower, which isn't really acceptable to me.
I don't really know how to write C extensions, nor am I sure attrs would allow C extensions. However writing a descriptor in C that duplicates __get__ from member_descriptor and throws exceptions in other methods would get us overhead-free frozen slot classes and attributes. Any takers? :eyes:
It's also possible I've made a mistake somewhere, this stuff isn't exactly the simplest.

I think this is out of question for the reason alone that it will explode in our faces on anything that isn’t CPython, correct?

I have a pure Python proof of concept, but it slows down attribute access by a lot. Presumably PyPy would take care of the speed bump. Maybe I'll do some benchmarks.

Hm, there's a performance penalty on PyPy too. 0.10 ns +- 0.00 ns vs 2.46 ns +- 0.07 ns. The numbers are so small I'm not even sure the measurements are good.

Whatever y’all do: it must not affect the performance of other at all.
I’m not gonna stop people driving around with handbrakes on but don’t take any hostages. :)

It seems pretty clear to me that this isn't going to affect the performance of other cases.  We can use our favorite tool to prevent this - subclassing! :)
FrozenAttribute could be a subclass of Attribute which implements the descriptor protocol to reject duplicate writes, and to read from a private storage attribute.  The main difference between the __slots__ and non-__slots__ case is that the descriptor that you delegate to in the non-__slots__ case would be a synthetic thing that just writes to __dict__, the attribute that you write to in the __slots__ case in the storage decorator.
This would be a performance hit for the attributes that use it, of course, but, that's fine; the purpose of this is mostly just to prevent errors anyway, and I bet this is exactly the sort of no-op that PyPy will happily JIT away.  And for the attributes that don't use it, nothing would change.

Separate thought based on investigating this implementation - _CountingAttr is private, Attribute is public. Yet, Attribute has an undocumented, but public, from_counting_attr method.  It would be easier to implement this, as well as just more cleanly factored, to have an as_attribute method on _CountingAttr instead - one which could select its type between FrozenAttribute and Attribute.

Ugh, from_counting_attr is not prefixed by accident.  I think I didn’t consider making Attribute public in the beginning and so it snook in.  Doesn’t matter, that one is private and if this doesn’t go anywhere, I’ll prefix it.
Is the descriptor approach possible before we stop shaving off Attribute definitions?  FWIW, I could live with shaving them off if someone uses FrozenAttributes.
P.S. I don’t believe in blackmailing but I’ve taken a screenshot of your suggestion to use subclassing just in case. ;)

Yes, we'll need descriptors for this. I don't see why we'd use Attributes for this though.

Is the descriptor approach possible before we stop shaving off Attribute definitions?

What does this mean?

I meant “start”.

Ah. Yes it could be done, by making Attributes descriptors. When does the depreciation period end exactly? If it's soon it's not really worth it.

2017-08-30

I've started picking away at this over at https://github.com/python-attrs/attrs/pull/172.

I have a usecase where I want the attribute to be publicly readonly but settable from inside the class.
Essentially:
@attr.s(auto_attribs=True)
class Foo:
  _bar: int = attr.ib(default=0, public_getter=True, public_setter=False)

Couldn't we simply attach properties to the class and call it a day?

@thedrow thats what name-mangled "private" variables are for

@RonnyPfannschmidt if you're talking about:
...    _abc = attr.ib()
Then its  not the same thing unfortunately.
People want this instead:
...    abc = attr.ib(frozen=True)
Where abc cannot be changed once its been initialized.

Just checking in to say that there's still plenty of demand for this

Everyone interested in this please look at #645.

…and more importantly #660 which implements all of this.

I believe this is fixed.


sl-drop-button not showing dropdown options when clicked

The sl-drop-button is not showing the dropdown options when clicked because it's missing the data-toggle="dropdown" attribute

@notmessenger please close this has been merged in https://github.com/softlayer/sl-ember-components/commit/667ac69eb2cd15732b72e822156bf25a9dd6f056


Import & Use in Swift

I install this library in pod by this :
pod 'YTXAnimations', :subspecs => ["AnimateCSS", "CSShake", "MagicCSS"]
I want using in swift , I try use with below line to import but not successfully :
import YTXAnimations //or import AnimateCSS //or  import YTXAnimateCSS
Please help me for use this library in swift . Tnx

Why closed?
Try this http://stackoverflow.com/questions/24002369/how-to-call-objective-c-code-from-swift


Add helper for running class_exec with keywords when needed

This is an extraction from a pr on rspec-core but its going to be a repeated pattern to fix keyword arguments across the gem so I'm adding it here. I expect to add more as I find other places that keyword arguments don't work quite right.
The aim will be to keep the string eval to a minimum and this will all be replaced with proper **kwargs in RSpec 4, which I want to release in the later half of this year before the next version of Ruby.

Merged because something odd is going on with the sub builds which means this wasn't picked up on the pinned branch.

On this subject https://discuss.rubyonrails.org/t/new-2-7-3-0-keyword-argument-pain-point/74980 (cc @JonRowe @pirj )

I've added a reply


Added arrow functions to js/ts

Hello.
Can you please add arrow functions?
Something like that
const bla = () => {     return  }

Hey, thanks for the suggestion. Do you use nvim-autopairs by any chance ? That's what I use to automatically expand arrow functions and I really like it. One advantage is that you don't need to trigger any snippet with a key, it automatically happens once the regex is matched.
The rule is added like this:
        local pairs = require "nvim-autopairs"
        local Rule = require "nvim-autopairs.rule"

        pairs.add_rules {
            -- arrow key on javascript, typescript
            Rule(
                "%(.*%)%s*%=>$",
                " {}",
                { "typescript", "typescriptreact", "javascript", "javascriptreact" }
            ):use_regex(true):set_end_pair_length(1),
        }

I liked this solution more since I don't need to press a key to trigger the snippet.


doc: correct source lines

Sample codes in documentation guide/threads a bit strange. This PR corrects those line referenced. Following screenshots are between original version and this PR version.

before
with this PR

Landed in 99f8ffcc, thanks!


[BUG] Mars web got stuck when chunk graph is large

Describe the bug
After a tileable graph with small chunk size is submitted and after a while, Mars Web got stuck.
To Reproduce
Use code below to reproduce:
import mars
import mars.tensor as mt
sess = mars.new_session('http://localhost:13042/')
a = mt.ones((1000, 1000), chunk_size=10)
b = (a + 1) * 2 + (a - 3) ** 2
b.execute()

Maybe we can produce a SIGSEGV and see core dumps to discover what is actually happening using LLDB.


Permission

Hello. I am getting this error in some commands even though I have authorized myself. can you help me?

How did you click to show that form?
example) action > Block,  action > Command,  condition > Block ...


refactor : Transform SaveModal to typescript

SUMMARY
Transform SaveModal to Typescript

@evans @junlincc

Codecov Report

Merging #11951 (726e7fe) into master (a7bba92) will decrease coverage by 1.45%.
The diff coverage is 9.09%.

@@            Coverage Diff             @@
##           master   #11951      +/-   ##
==========================================
- Coverage   64.45%   63.00%   -1.46%
==========================================
  Files         937      472     -465
  Lines       45319    16780   -28539
  Branches     4317     4342      +25
==========================================
- Hits        29211    10572   -18639
+ Misses      15948     6031    -9917
- Partials      160      177      +17

Flag
Coverage Δ

cypress
?

javascript
63.00% <9.09%> (+<0.01%)
:arrow_up:

python
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...uperset-frontend/src/components/Checkbox/index.tsx
100.00% <ø> (ø)

...et-frontend/src/dashboard/components/SaveModal.tsx
19.60% <9.09%> (ø)

superset-frontend/src/dashboard/App.jsx
0.00% <0.00%> (-100.00%)
:arrow_down:

superset-frontend/src/dashboard/index.jsx
0.00% <0.00%> (-100.00%)
:arrow_down:

superset-frontend/src/setup/setupColors.js
0.00% <0.00%> (-100.00%)
:arrow_down:

superset-frontend/src/chart/ChartContainer.jsx
0.00% <0.00%> (-100.00%)
:arrow_down:

...et-frontend/src/dashboard/containers/Dashboard.jsx
0.00% <0.00%> (-100.00%)
:arrow_down:

...-frontend/src/visualizations/presets/MainPreset.js
0.00% <0.00%> (-100.00%)
:arrow_down:

...tend/src/dashboard/containers/DashboardBuilder.jsx
0.00% <0.00%> (-100.00%)
:arrow_down:

superset-frontend/src/setup/setupFormatters.ts
0.00% <0.00%> (-75.00%)
:arrow_down:

... and 506 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a7bba92...726e7fe. Read the comment docs.

@etr2460 Refactored.

I think all questions/issues have been resolved! Thanks, everyone!


Charge level regression line

this is an enhancement for the charge level dashboard and is built on top of #4186
by enabling the regression transformation feature toggle in grafana it allows showing a regression line for the charge level

details on the feature and feature toggles in general
https://github.com/grafana/grafana/pull/78457
https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/feature-toggles/

@JakobLichterfeld - still experimenting a little bit with some added lines to add value to the charge level dashboard.
next to having a longer history in #4186 i tried adding some moving median / percentile values within the dashboard to show how battery usage changes over time.
moving percentile is a little bit tricky within postgres as percentile_disc is not supported as a window function out of the box. i still would like to improve some calcuation steps, therefore converted this pr to a draft.
you really can get lost in the details - e.g. time weighting the averages (as we are having infrequent data via positions)
https://www.timescale.com/blog/what-time-weighted-averages-are-and-why-you-should-care/

Thanks for your suggestion and effort!

could you please guide on what is favored? regression via plugin or showing moving percentiles within the graph?

Personally, I don't think we need a forecast for the charge level (which would be a benefit of using regression). I think moving percentiles are well suited as we want more transparency about the distribution of the charge states and detailed insights into the charging behavior.

Making use of TimescaleDB Hyperfunctions would have made the query way easier to write and read but I guess switching our DB is not an option currently.

As TimescaleDB Hyperfunctions is an extenstion to PosgreSQL I could imagine doing the migration effort if we do need the additional performance. I personally do not yet have extensive knowledge of TimescaleDB Hyperfunctions.

before updating the PR - could someone run the query below (6 months+ history) and note the query time please?

I changed it to car_id = 1 but didn't get any output, can you double check the syntax?

Hi @JakobLichterfeld - the query should output 1472 rows even if you are selecting a non existing car right now.
could you run the query again and append an EXPLAIN ANALYZE in front of the query?

with sample_set as (
	select date, battery_level from positions p where p.car_id = 2 and p.battery_level is not null
	-- union all
	-- select date, battery_level, usable_battery_level from charges c inner join charging_processes cp on c.charging_process_id = cp.id and cp.car_id = 2
), date_series as (
	select generate_series('2024-03-08T15:19:38.536Z', '2024-09-08T14:19:38.536Z', (10800000 / 1 || ' milliseconds')::INTERVAL) as series_id
), date_series_bucketing as (
	select series_id, lead(series_id) over (order by series_id asc) as next_series_id from date_series
), bucketing as (
	select
		series_id, avg(battery_level) as battery_level
	from date_series_bucketing
	left join sample_set on
		sample_set.date >= date_series_bucketing.series_id
		and sample_set.date < date_series_bucketing.next_series_id
	group by series_id
), locf_intermediate as (
	select series_id, battery_level, count(battery_level) over (order by series_id) as i from bucketing
), bucketing_gapfill as (
	select series_id, max(battery_level) over (partition by i) as battery_level from locf_intermediate
), trick as (
	select series_id, battery_level, array_agg(battery_level) over (rows between 125 preceding and 125 following) as arr from bucketing_gapfill
)
select series_id,
    case when battery_level is null then null else(SELECT percentile_cont(0.075) within group (ORDER BY s)
    FROM unnest(arr) trick(s)) end as p0075,
    case when battery_level is null then null else(SELECT percentile_cont(0.5) within group (ORDER BY s)
    FROM unnest(arr) trick(s)) end as p050,
    case when battery_level is null then null else(SELECT percentile_cont(0.925) within group (ORDER BY s)
    FROM unnest(arr) trick(s)) end as p0925
from trick

Sorry for not providing the non-existent error message in first place :-)

Super Strange... what version of postgres are you using? @DrMichael could you maybe run the query?

Super Strange... what version of postgres are you using?

PostgreSQL16

Please terminate the Query with ; 🫣

Please terminate the Query with ; 🫣

😀 That's what I was looking for with

can you double check the syntax?

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Subquery Scan on trick  (cost=112.45..219.45 rows=200 width=32) (actual time=34.540..346.705 rows=1472 loops=1)
   ->  WindowAgg  (cost=112.45..120.45 rows=200 width=72) (actual time=34.533..344.951 rows=1472 loops=1)
         ->  Subquery Scan on bucketing_gapfill  (cost=112.45..117.95 rows=200 width=40) (actual time=34.022..37.424 rows=1472 loops=1)
               ->  WindowAgg  (cost=112.45..115.95 rows=200 width=48) (actual time=34.019..36.203 rows=1472 loops=1)
                     ->  Sort  (cost=112.45..112.95 rows=200 width=48) (actual time=30.719..31.332 rows=1472 loops=1)
                           Sort Key: locf_intermediate.i
                           Sort Method: quicksort  Memory: 90kB
                           ->  Subquery Scan on locf_intermediate  (cost=55.28..104.80 rows=200 width=48) (actual time=5.984..28.824 rows=1472 loops=1)
                                 ->  WindowAgg  (cost=55.28..104.80 rows=200 width=48) (actual time=5.980..27.491 rows=1472 loops=1)
                                       ->  GroupAggregate  (cost=55.28..101.80 rows=200 width=40) (actual time=5.946..20.159 rows=1472 loops=1)
                                             Group Key: (generate_series('2024-03-08 15:19:38.536+00'::timestamp with time zone, '2024-09-08 14:19:38.536+00'::timestamp with time zone, ('10800000 milliseconds'::cstring)::interval))
                                             ->  Nested Loop Left Join  (cost=55.28..94.30 rows=1000 width=10) (actual time=5.905..15.620 rows=1472 loops=1)
                                                   Join Filter: ((p.date >= (generate_series('2024-03-08 15:19:38.536+00'::timestamp with time zone, '2024-09-08 14:19:38.536+00'::timestamp with time zone, ('10800000 milliseconds'::cstring)::interval))) AND (p.date < (lead((generate_series('2024-03-08 15:19:38.536+00'::timestamp with time zone, '2024-09-08 14:19:38.536+00'::timestamp with time zone, ('10800000 milliseconds'::cstring)::interval))) OVER (?))))
                                                   ->  WindowAgg  (cost=54.85..72.35 rows=1000 width=16) (actual time=2.687..9.706 rows=1472 loops=1)
                                                         ->  Sort  (cost=54.85..57.35 rows=1000 width=8) (actual time=2.659..3.492 rows=1472 loops=1)
                                                               Sort Key: (generate_series('2024-03-08 15:19:38.536+00'::timestamp with time zone, '2024-09-08 14:19:38.536+00'::timestamp with time zone, ('10800000 milliseconds'::cstring)::interval))
                                                               Sort Method: quicksort  Memory: 56kB
                                                               ->  ProjectSet  (cost=0.00..5.02 rows=1000 width=8) (actual time=0.363..1.611 rows=1472 loops=1)
                                                                     ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.003..0.005 rows=1 loops=1)
                                                   ->  Materialize  (cost=0.43..4.46 rows=1 width=10) (actual time=0.003..0.003 rows=0 loops=1472)
                                                         ->  Index Scan using positions_car_id_index on positions p  (cost=0.43..4.45 rows=1 width=10) (actual time=3.193..3.193 rows=0 loops=1)
                                                               Index Cond: (car_id = 2)
                                                               Filter: (battery_level IS NOT NULL)
   SubPlan 1
     ->  Aggregate  (cost=0.16..0.17 rows=1 width=8) (never executed)
           ->  Function Scan on unnest trick_1  (cost=0.00..0.10 rows=10 width=32) (never executed)
   SubPlan 2
     ->  Aggregate  (cost=0.16..0.17 rows=1 width=8) (never executed)
           ->  Function Scan on unnest trick_2  (cost=0.00..0.10 rows=10 width=32) (never executed)
   SubPlan 3
     ->  Aggregate  (cost=0.16..0.17 rows=1 width=8) (never executed)
           ->  Function Scan on unnest trick_3  (cost=0.00..0.10 rows=10 width=32) (never executed)
 Planning Time: 36.497 ms
 Execution Time: 349.111 ms
(34 rows)

~
(END)

Can you change car_id to 1 again to get results?

Can you change car_id to 1 again to get results?

For sure, aborted one after 9 minutes right now, as I do not want the Rpi to hang while the car is charging, will redo later today.

sure - takes less than 1 second on my instance with 2 cars, 25k km in total. let's see what explain analyze returns, hopefully i can find a way to improve speed for lower spec instances.

I've executed your query a few times and it took about 21 to 24 seconds every time (with car_id=1)

let's see what explain analyze returns, hopefully i can find a way to improve speed for lower spec instances.

The Rpi reboots during the query, so no output. Will try with shorter timeframe

Will try with shorter timeframe

Tried with
EXPLAIN ANALYZE with sample_set as (
        select date, battery_level from positions p where p.car_id = 1 and p.battery_level is not null
        -- union all
        -- select date, battery_level, usable_battery_level from charges c inner join charging_processes cp on c.charging_process_id = cp.id and cp.car_id = 1
), date_series as (
        select generate_series('2024-08-01T00:00:01.000Z', '2024-09-13T23:59:59.000Z', (10800000 / 1 || ' milliseconds')::INTERVAL) as series_id
), date_series_bucketing as (
        select series_id, lead(series_id) over (order by series_id asc) as next_series_id from date_series
), bucketing as (
        select
                series_id, avg(battery_level) as battery_level
        from date_series_bucketing
        left join sample_set on
                sample_set.date >= date_series_bucketing.series_id
                and sample_set.date < date_series_bucketing.next_series_id
        group by series_id
), locf_intermediate as (
        select series_id, battery_level, count(battery_level) over (order by series_id) as i from bucketing
), bucketing_gapfill as (
        select series_id, max(battery_level) over (partition by i) as battery_level from locf_intermediate
), trick as (
        select series_id, battery_level, array_agg(battery_level) over (rows between 125 preceding and 125 following) as arr from bucketing_gapfill
)
select series_id,
    case when battery_level is null then null else(SELECT percentile_cont(0.075) within group (ORDER BY s)
    FROM unnest(arr) trick(s)) end as p0075,
    case when battery_level is null then null else(SELECT percentile_cont(0.5) within group (ORDER BY s)
    FROM unnest(arr) trick(s)) end as p050,
    case when battery_level is null then null else(SELECT percentile_cont(0.925) within group (ORDER BY s)
    FROM unnest(arr) trick(s)) end as p0925
from trick;

And aborted after 1h11m33s

@JakobLichterfeld - can you try the following two queries? (sorry but it's hard to improve runtimes without knowing timings and what is slowing it down in your system. query execution is 300ms here...
select count(*) from positions p where p.car_id = 2 and date between '2024-09-01T00:00:01.000Z' and '2024-09-13T23:59:59.000Z' and p.battery_level is not null

EXPLAIN ANALYZE with sample_set as (
        select date, battery_level from positions p where p.car_id = 2 and date between '2024-09-01T00:00:01.000Z' and '2024-09-13T23:59:59.000Z' and p.battery_level is not null
        -- union all
        -- select date, battery_level, usable_battery_level from charges c inner join charging_processes cp on c.charging_process_id = cp.id and cp.car_id = 1
), date_series as (
        select generate_series('2024-09-01T00:00:01.000Z', '2024-09-13T23:59:59.000Z', (10800000 / 1 || ' milliseconds')::INTERVAL) as series_id
), date_series_bucketing as (
        select series_id, lead(series_id) over (order by series_id asc) as next_series_id from date_series
), bucketing as (
        select
                series_id, avg(battery_level) as battery_level
        from date_series_bucketing
        left join sample_set on
                sample_set.date >= date_series_bucketing.series_id
                and sample_set.date < date_series_bucketing.next_series_id
        group by series_id
), locf_intermediate as (
        select series_id, battery_level, count(battery_level) over (order by series_id) as i from bucketing
), bucketing_gapfill as (
        select series_id, max(battery_level) over (partition by i) as battery_level from locf_intermediate
), trick as (
        select series_id, battery_level, array_agg(battery_level) over (rows between 125 preceding and 125 following) as arr from bucketing_gapfill
)
select series_id,
    case when battery_level is null then null else(SELECT percentile_cont(0.075) within group (ORDER BY s)
    FROM unnest(arr) trick(s)) end as p0075,
    case when battery_level is null then null else(SELECT percentile_cont(0.5) within group (ORDER BY s)
    FROM unnest(arr) trick(s)) end as p050,
    case when battery_level is null then null else(SELECT percentile_cont(0.925) within group (ORDER BY s)
    FROM unnest(arr) trick(s)) end as p0925
from trick;

Yeah sorry, I was thinking about splitting the query to find the root cause as well.

@JakobLichterfeld - can you try the following two queries? (sorry but it's hard to improve runtimes without knowing timings and what is slowing it down in your system. query execution is 300ms here...

select count(*) from positions p where p.car_id = 1 and date between '2024-09-01T00:00:01.000Z' and '2024-09-13T23:59:59.000Z' and p.battery_level is not null

                                                                                QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Aggregate  (cost=127766.34..127766.35 rows=1 width=8) (actual time=47428.881..49319.385 rows=1 loops=1)
   ->  Gather  (cost=1000.00..127766.33 rows=1 width=0) (actual time=47230.977..49274.355 rows=108817 loops=1)
         Workers Planned: 2
         Workers Launched: 2
         ->  Parallel Seq Scan on positions p  (cost=0.00..126766.23 rows=1 width=0) (actual time=42907.622..42979.595 rows=36272 loops=3)
               Filter: ((battery_level IS NOT NULL) AND (date >= '2024-09-01 00:00:01'::timestamp without time zone) AND (date <= '2024-09-13 23:59:59'::timestamp without time zone) AND (car_id = 1))
               Rows Removed by Filter: 2077526
 Planning Time: 11.502 ms
 JIT:
   Functions: 11
   Options: Inlining false, Optimization false, Expressions true, Deforming true
   Timing: Generation 29.442 ms, Inlining 0.000 ms, Optimization 480.674 ms, Emission 2836.354 ms, Total 3346.471 ms
 Execution Time: 49342.482 ms
(13 rows)

@JakobLichterfeld - there is an index in positions over date that should have been used for the first query - i wonder why it's not in your case...

could you run:
SELECT indexname AS index_name,
       tablename AS table_name
FROM pg_indexes
WHERE schemaname = 'public' and tablename = 'positions';

if the result shows an index named positions_date_index -> please run ``reindex index positions_date_index;` afterwards.

there is an index in positions over date that should have been used for the first query - i wonder why it's not in your case...

I think you may have hit on something there. We had a user who reported that the position table was not restored correctly from the backup, I could not find the issue/pr at the moment.
teslamate=# SELECT indexname AS index_name,
       tablename AS table_name
FROM pg_indexes
WHERE schemaname = 'public' and tablename = 'positions';
       index_name       | table_name
------------------------+------------
 positions_pkey         | positions
 positions_car_id_index | positions
(2 rows)

I'll prepare an overview of all indexes and a sql script to add all missing indexes later today. If others are affected we could add it to the docs section.
I restored from a backup last week (testing pg17) - worked without any issue here.
Let's see how the query performs afterwards, tried some other indexes, seeing enough potential to speed it and get this feature over the line!
I do have more time tomorrow, may ping you with some queries if ok for you.

I'll plug https://github.com/jheredianet/Teslamate-CustomGrafanaDashboards/raw/main/dashboards/DatabaseDashboadInfo.json for details about the database

Nice, thanks for pointing this out.

Nice, thanks for pointing this out.

@JakobLichterfeld - looks like you are missing 8 indexes when comparing to my instance:
could you execute the sql below and afterwards run the two select statements mentioned here again?
https://github.com/teslamate-org/teslamate/pull/4187#issuecomment-2349222316
@coreGreenberet / @sdwalker - can you confirm a total of 36 indexes within your environments of teslamate via select count(*) from pg_indexes where schemaname = 'public';
corrective sql
CREATE UNIQUE INDEX IF NOT EXISTS addresses_osm_id_osm_type_index ON public.addresses USING btree (osm_id, osm_type);
CREATE UNIQUE INDEX IF NOT EXISTS addresses_pkey ON public.addresses USING btree (id);
CREATE UNIQUE INDEX IF NOT EXISTS car_settings_pkey ON public.car_settings USING btree (id);
CREATE UNIQUE INDEX IF NOT EXISTS cars_eid_index ON public.cars USING btree (eid);
CREATE UNIQUE INDEX IF NOT EXISTS cars_pkey ON public.cars USING btree (id);
CREATE UNIQUE INDEX IF NOT EXISTS cars_settings_id_index ON public.cars USING btree (settings_id);
CREATE UNIQUE INDEX IF NOT EXISTS cars_vid_index ON public.cars USING btree (vid);
CREATE UNIQUE INDEX IF NOT EXISTS cars_vin_index ON public.cars USING btree (vin);
CREATE INDEX IF NOT EXISTS charges_charging_process_id_index ON public.charges USING btree (charging_process_id);
CREATE INDEX IF NOT EXISTS charges_date_index ON public.charges USING btree (date);
CREATE UNIQUE INDEX IF NOT EXISTS charges_pkey ON public.charges USING btree (id);
CREATE INDEX IF NOT EXISTS charging_processes_address_id_index ON public.charging_processes USING btree (address_id);
CREATE INDEX IF NOT EXISTS charging_processes_car_id_index ON public.charging_processes USING btree (car_id);
CREATE UNIQUE INDEX IF NOT EXISTS charging_processes_pkey ON public.charging_processes USING btree (id);
CREATE INDEX IF NOT EXISTS charging_processes_position_id_index ON public.charging_processes USING btree (position_id);
CREATE INDEX IF NOT EXISTS drives_end_geofence_id_index ON public.drives USING btree (end_geofence_id);
CREATE INDEX IF NOT EXISTS drives_end_position_id_index ON public.drives USING btree (end_position_id);
CREATE INDEX IF NOT EXISTS drives_start_geofence_id_index ON public.drives USING btree (start_geofence_id);
CREATE INDEX IF NOT EXISTS drives_start_position_id_index ON public.drives USING btree (start_position_id);
CREATE INDEX IF NOT EXISTS trips_car_id_index ON public.drives USING btree (car_id);
CREATE INDEX IF NOT EXISTS trips_end_address_id_index ON public.drives USING btree (end_address_id);
CREATE UNIQUE INDEX IF NOT EXISTS trips_pkey ON public.drives USING btree (id);
CREATE INDEX IF NOT EXISTS trips_start_address_id_index ON public.drives USING btree (start_address_id);
CREATE UNIQUE INDEX IF NOT EXISTS geofences_pkey ON public.geofences USING btree (id);
CREATE INDEX IF NOT EXISTS positions_car_id_index ON public.positions USING btree (car_id);
CREATE INDEX IF NOT EXISTS positions_date_index ON public.positions USING btree (date);
CREATE INDEX IF NOT EXISTS positions_drive_id_date_index ON public.positions USING btree (drive_id, date);
CREATE UNIQUE INDEX IF NOT EXISTS positions_pkey ON public.positions USING btree (id);
CREATE UNIQUE INDEX IF NOT EXISTS schema_migrations_pkey ON public.schema_migrations USING btree (version);
CREATE UNIQUE INDEX IF NOT EXISTS settings_pkey ON public.settings USING btree (id);
CREATE UNIQUE INDEX IF NOT EXISTS "states_car_id__end_date_IS_NULL_index" ON public.states USING btree (car_id, ((end_date IS NULL))) WHERE (end_date IS NULL);
CREATE INDEX IF NOT EXISTS states_car_id_index ON public.states USING btree (car_id);
CREATE UNIQUE INDEX IF NOT EXISTS states_pkey ON public.states USING btree (id);
CREATE UNIQUE INDEX IF NOT EXISTS tokens_pkey ON public.tokens USING btree (id);
CREATE INDEX IF NOT EXISTS updates_car_id_index ON public.updates USING btree (car_id);
CREATE UNIQUE INDEX IF NOT EXISTS updates_pkey ON public.updates USING btree (id);

looks like you are missing 8 indexes when comparing to my instance:

Yeah, with a quick look into the migrations, I could not find one for positions_date_index for example.
Creating the indexes right now and will store a backup without the indexes for potential future testing.

run the two select statements mentioned here again?

teslamate=# select count(*) from positions p where p.car_id = 1 and date between '2024-09-01T00:00:01.000Z' and '2024-09-13T23:59:59.000Z' and p.battery_level is not null;
 count
--------
 108817
(1 row)

and
                                                                                QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Subquery Scan on trick  (cost=129128.92..129235.92 rows=200 width=32) (actual time=1295.137..1444.189 rows=104 loops=1)
   ->  WindowAgg  (cost=129128.92..129136.92 rows=200 width=72) (actual time=1293.132..1294.773 rows=104 loops=1)
         ->  Subquery Scan on bucketing_gapfill  (cost=129128.92..129134.42 rows=200 width=40) (actual time=753.184..753.685 rows=104 loops=1)
               ->  WindowAgg  (cost=129128.92..129132.42 rows=200 width=48) (actual time=753.163..753.590 rows=104 loops=1)
                     ->  Sort  (cost=129128.92..129129.42 rows=200 width=48) (actual time=753.020..753.078 rows=104 loops=1)
                           Sort Key: locf_intermediate.i
                           Sort Method: quicksort  Memory: 21kB
                           ->  Subquery Scan on locf_intermediate  (cost=55.28..129121.28 rows=200 width=48) (actual time=5.183..752.722 rows=104 loops=1)
                                 ->  WindowAgg  (cost=55.28..129121.28 rows=200 width=48) (actual time=5.164..752.541 rows=104 loops=1)
                                       ->  GroupAggregate  (cost=55.28..129118.28 rows=200 width=40) (actual time=5.030..751.688 rows=104 loops=1)
                                             Group Key: (generate_series('2024-09-01 00:00:01+00'::timestamp with time zone, '2024-09-13 23:59:59+00'::timestamp with time zone, ('10800000 milliseconds'::cstring)::interval))
                                             ->  Nested Loop Left Join  (cost=55.28..123350.78 rows=1153000 width=10) (actual time=4.805..680.663 rows=108895 loops=1)
                                                   ->  WindowAgg  (cost=54.85..72.35 rows=1000 width=16) (actual time=4.543..5.374 rows=104 loops=1)
                                                         ->  Sort  (cost=54.85..57.35 rows=1000 width=8) (actual time=4.457..4.569 rows=104 loops=1)
                                                               Sort Key: (generate_series('2024-09-01 00:00:01+00'::timestamp with time zone, '2024-09-13 23:59:59+00'::timestamp with time zone, ('10800000 milliseconds'::cstring)::interval))
                                                               Sort Method: quicksort  Memory: 18kB
                                                               ->  ProjectSet  (cost=0.00..5.02 rows=1000 width=8) (actual time=4.259..4.350 rows=104 loops=1)
                                                                     ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.027..0.028 rows=1 loops=1)
                                                   ->  Index Scan using positions_date_index on positions p  (cost=0.43..111.75 rows=1153 width=10) (actual time=0.110..5.777 rows=1046 loops=104)
                                                         Index Cond: ((date >= (generate_series('2024-09-01 00:00:01+00'::timestamp with time zone, '2024-09-13 23:59:59+00'::timestamp with time zone, ('10800000 milliseconds'::cstring)::interval))) AND (date < (lead((generate_series('2024-09-01 00:00:01+00'::timestamp with time zone, '2024-09-13 23:59:59+00'::timestamp with time zone, ('10800000 milliseconds'::cstring)::interval))) OVER (?))) AND (date >= '2024-09-01 00:00:01'::timestamp without time zone) AND (date <= '2024-09-13 23:59:59'::timestamp without time zone))
                                                         Filter: ((battery_level IS NOT NULL) AND (car_id = 1))
   SubPlan 1
     ->  Aggregate  (cost=0.16..0.17 rows=1 width=8) (actual time=0.471..0.471 rows=1 loops=104)
           ->  Function Scan on unnest trick_1  (cost=0.00..0.10 rows=10 width=32) (actual time=0.072..0.121 rows=104 loops=104)
   SubPlan 2
     ->  Aggregate  (cost=0.16..0.17 rows=1 width=8) (actual time=0.472..0.472 rows=1 loops=104)
           ->  Function Scan on unnest trick_2  (cost=0.00..0.10 rows=10 width=32) (actual time=0.072..0.121 rows=104 loops=104)
   SubPlan 3
     ->  Aggregate  (cost=0.16..0.17 rows=1 width=8) (actual time=0.475..0.476 rows=1 loops=104)
           ->  Function Scan on unnest trick_3  (cost=0.00..0.10 rows=10 width=32) (actual time=0.072..0.121 rows=104 loops=104)
 Planning Time: 15.153 ms
 JIT:
   Functions: 57
   Options: Inlining false, Optimization false, Expressions true, Deforming true
   Timing: Generation 22.986 ms, Inlining 0.000 ms, Optimization 32.070 ms, Emission 507.753 ms, Total 562.809 ms
 Execution Time: 4231.983 ms
(36 rows)

(END)

Yeah, with a quick look into the migrations, I could not find one for positions_date_index for example.

I assume, this one should normally create the index:
https://github.com/teslamate-org/teslamate/blob/363e5a661612a090cfac5bf89715f7e9de39f420/priv/repo/migrations/20230417225712_composite_index_to_position.exs

Btw. love the speed on the Rpi 3B with the 6 additional indexes 😃

great to hear, wonder if i might should switch to slower hardware and look for additional improvements by adding an index here and there 😆
finalized the query for charge level - works great here
@JakobLichterfeld - could you first add an index:
CREATE INDEX "positions_car_id_date_ideal_battery_range_km_IS_NOT_NULL_index" ON public.positions USING btree (car_id, date, ((ideal_battery_range_km IS NOT NULL))) WHERE (ideal_battery_range_km IS NOT NULL)

and afterwards run this query:
-- To be able to calucate percentiles for unevenly sampled values we are bucketing & gapfilling values before running calcuations
with positions_filtered as (
    select
        date,
        battery_level
    from
        positions p
    where
        p.car_id = '1'
        -- p.ideal_battery_range_km condition is added to reduce overall amount of data and avoid data biases while driving (unevenly sampled data)
        and p.ideal_battery_range_km is not null
),
gen_date_series as (
    select
        -- series is used to bucket data and avoid gaps in series used to determine percentiles
        generate_series(to_timestamp(1710497281), to_timestamp(1726391281), concat(7200, ' seconds')::interval) as series_id
),
date_series as (
    select
        series_id,
        -- before joining, get beginning of next series to be able to left join `positions_filtered`
        lead(series_id) over (order by series_id asc) as next_series_id
    from
        gen_date_series
),
positions_bucketed as (
    select
        -- simple average can result in loss of accuracy, see https://www.timescale.com/blog/what-time-weighted-averages-are-and-why-you-should-care/ for details
        series_id,
        avg(battery_level) as battery_level
    from
        date_series
    left join positions_filtered on
        positions_filtered.date >= date_series.series_id
        and positions_filtered.date < date_series.next_series_id
    group by
        series_id
),
-- PostgreSQL cannot IGNORE NULLS via Window Functions LAST_VALUE - therefore use natural behavior of COUNT & MAX, see https://www.reddit.com/r/SQL/comments/wb949v/comment/ii5mmmi/ for details
positions_bucketed_gapfilling_locf_intermediate as (
    select
        series_id,
        battery_level,
        count(battery_level) over (order by series_id) as i
    from
        positions_bucketed

),
positions_bucketed_gapfilled_locf as (
    select
        series_id,
        max(battery_level) over (partition by i) as battery_level
    from
        positions_bucketed_gapfilling_locf_intermediate
),
-- PostgreSQL cannot use PERCENTILE_DISC as Window Function - therefore use ARRAY_AGG and UNNEST, see https://stackoverflow.com/a/72718604 for details
positions_bucketed_gapfilled_locf_percentile_intermediate as (
    select
        series_id,
        battery_level,
        array_agg(battery_level) over w as arr,
        avg(battery_level) over w as battery_level_avg
    from
        positions_bucketed_gapfilled_locf
    window w as (rows between (1296000 / 7200) preceding and (1296000 / 7200) following)
)

select
    series_id,
    case when battery_level is null then null else (select percentile_cont(0.075) within group (order by s) from unnest(arr) trick(s)) end as "30 Day Moving 7,5% Percentile",
    case when battery_level is null then null else (battery_level_avg) end as "30 Day Moving Average",
    case when battery_level is null then null else (select percentile_cont(0.5) within group (order by s) from unnest(arr) trick(s)) end as "30 Day Moving Median",
    case when battery_level is null then null else (select percentile_cont(0.925) within group (order by s) from unnest(arr) trick(s)) end as "30 Day Moving 92,5% Percentile"
from
    positions_bucketed_gapfilled_locf_percentile_intermediate;

great to hear, wonder if i might should switch to slower hardware and look for additional improvements by adding an index here and there 😆

😄 I only found a slow query after that, see #4199

could you first add an index:

Created with ; at the end

and afterwards run this query:

                                                                                QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Subquery Scan on positions_bucketed_gapfilled_locf_percentile_intermediate  (cost=1556588.22..1556695.72 rows=200 width=64) (actual time=7442.816..17583.645 rows=2208 loops=1)
   ->  WindowAgg  (cost=1556588.22..1556596.72 rows=200 width=104) (actual time=7442.779..8347.987 rows=2208 loops=1)
         ->  Subquery Scan on positions_bucketed_gapfilled_locf  (cost=1556588.22..1556593.72 rows=200 width=40) (actual time=3659.174..3675.169 rows=2208 loops=1)
               ->  WindowAgg  (cost=1556588.22..1556591.72 rows=200 width=48) (actual time=3659.142..3672.381 rows=2208 loops=1)
                     ->  Sort  (cost=1556588.22..1556588.72 rows=200 width=48) (actual time=3658.844..3660.370 rows=2208 loops=1)
                           Sort Key: positions_bucketed_gapfilling_locf_intermediate.i
                           Sort Method: quicksort  Memory: 155kB
                           ->  Subquery Scan on positions_bucketed_gapfilling_locf_intermediate  (cost=55.28..1556580.57 rows=200 width=48) (actual time=8.954..3653.613 rows=2208 loops=1)
                                 ->  WindowAgg  (cost=55.28..1556580.57 rows=200 width=48) (actual time=8.934..3650.616 rows=2208 loops=1)
                                       ->  GroupAggregate  (cost=55.28..1556577.57 rows=200 width=40) (actual time=8.802..3632.143 rows=2208 loops=1)
                                             Group Key: (generate_series('2024-03-15 10:08:01+00'::timestamp with time zone, '2024-09-15 09:08:01+00'::timestamp with time zone, (concat(7200, ' seconds'))::interval))
                                             ->  Nested Loop Left Join  (cost=55.28..1392872.85 rows=32740444 width=10) (actual time=8.643..3598.592 rows=22537 loops=1)
                                                   ->  WindowAgg  (cost=54.85..72.35 rows=1000 width=16) (actual time=4.922..24.914 rows=2208 loops=1)
                                                         ->  Sort  (cost=54.85..57.35 rows=1000 width=8) (actual time=4.778..7.353 rows=2208 loops=1)
                                                               Sort Key: (generate_series('2024-03-15 10:08:01+00'::timestamp with time zone, '2024-09-15 09:08:01+00'::timestamp with time zone, (concat(7200, ' seconds'))::interval))
                                                               Sort Method: quicksort  Memory: 99kB
                                                               ->  ProjectSet  (cost=0.00..5.03 rows=1000 width=8) (actual time=0.207..3.379 rows=2208 loops=1)
                                                                     ->  Result  (cost=0.00..0.01 rows=1 width=0) (actual time=0.026..0.028 rows=1 loops=1)
                                                   ->  Index Scan using "positions_car_id_date_ideal_battery_range_km_IS_NOT_NULL_index" on positions p  (cost=0.42..1065.40 rows=32740 width=10) (actual time=0.070..1.600 rows=9 loops=2208)
                                                         Index Cond: ((car_id = '1'::smallint) AND (date >= (generate_series('2024-03-15 10:08:01+00'::timestamp with time zone, '2024-09-15 09:08:01+00'::timestamp with time zone, (concat(7200, ' seconds'))::interval))) AND (date < (lead((generate_series('2024-03-15 10:08:01+00'::timestamp with time zone, '2024-09-15 09:08:01+00'::timestamp with time zone, (concat(7200, ' seconds'))::interval))) OVER (?))))
   SubPlan 1
     ->  Aggregate  (cost=0.16..0.17 rows=1 width=8) (actual time=1.418..1.419 rows=1 loops=2187)
           ->  Function Scan on unnest trick  (cost=0.00..0.10 rows=10 width=32) (actual time=0.229..0.397 rows=348 loops=2187)
   SubPlan 2
     ->  Aggregate  (cost=0.16..0.17 rows=1 width=8) (actual time=1.393..1.393 rows=1 loops=2187)
           ->  Function Scan on unnest trick_1  (cost=0.00..0.10 rows=10 width=32) (actual time=0.232..0.399 rows=348 loops=2187)
   SubPlan 3
     ->  Aggregate  (cost=0.16..0.17 rows=1 width=8) (actual time=1.389..1.389 rows=1 loops=2187)
           ->  Function Scan on unnest trick_2  (cost=0.00..0.10 rows=10 width=32) (actual time=0.230..0.396 rows=348 loops=2187)
 Planning Time: 30.443 ms
 JIT:
   Functions: 57
   Options: Inlining true, Optimization true, Expressions true, Deforming true
   Timing: Generation 23.334 ms, Inlining 775.072 ms, Optimization 1706.078 ms, Emission 1301.190 ms, Total 3805.675 ms
 Execution Time: 21119.696 ms
(35 rows)

(END)

@coreGreenberet / @sdwalker - can you confirm a total of 36 indexes within your environments of teslamate via select count(*) from pg_indexes where schemaname = 'public';

Yes
teslamate=# select count(*) from pg_indexes where schemaname = 'public';
 count
-------
    36
(1 row)

closed in favor of #4200
@JakobLichterfeld - should we take any action for missing potentially missing indexes on other instances?

@JakobLichterfeld - should we take any action for missing potentially missing indexes on other instances?

Yeah, let's track it in #4201

@JakobLichterfeld
@coreGreenberet can you confirm a total of 36 indexes within your environments of teslamate via select count(*) from pg_indexes where schemaname = 'public';

actually I'm having 640 😁
I've partitioned the positions table and I've also added my own indexes.
Each line got skipped from your create index snippet.

Ok, that doesnt help. If there are indexes that are benefitial for all please feel free to open an issue / pr!
Why have you partioned the positions table, also to speed up queries ?
@JakobLichterfeld would be another argument for switching to timescaledb ;) partitions by default ...

@swiffer I've overseen your new index "positions_car_id_date_ideal_battery_range_km_IS_NOT_NULL_index", but that didn't really speed up the process.
yes I've partitioned it for performance reasons.
  SubPlan 1
    ->  Aggregate  (cost=0.16..0.17 rows=1 width=8) (actual time=1.765..1.766 rows=1 loops=2208)
          ->  Function Scan on unnest trick  (cost=0.00..0.10 rows=10 width=32) (actual time=0.265..0.388 rows=346 loops=2208)
  SubPlan 2
    ->  Aggregate  (cost=0.16..0.17 rows=1 width=8) (actual time=1.749..1.749 rows=1 loops=2208)
          ->  Function Scan on unnest trick_1  (cost=0.00..0.10 rows=10 width=32) (actual time=0.264..0.390 rows=346 loops=2208)
  SubPlan 3
    ->  Aggregate  (cost=0.16..0.17 rows=1 width=8) (actual time=1.765..1.765 rows=1 loops=2208)
          ->  Function Scan on unnest trick_2  (cost=0.00..0.10 rows=10 width=32) (actual time=0.265..0.400 rows=346 loops=2208)
Planning Time: 152.184 ms
JIT:
  Functions: 570
  Options: Inlining true, Optimization true, Expressions true, Deforming true
  Timing: Generation 598.691 ms, Inlining 257.288 ms, Optimization 50413.348 ms, Emission 26995.150 ms, Total 78264.477 ms
Execution Time: 94728.314 ms

@coreGreenberet - could you try the updated dashboard provided in grafana image of #4200 ?

@swiffer
I'm getting this error 2 times on the new dashboard
A
Status: 500. Message: db query error: pq: function date_bin(interval, timestamp without time zone, timestamp with time zone) does not exist

@coreGreenberet thanks, seems reasonable when comparing to timing from jakob


chore(package): update ava to version 0.21.0

Closes #3

Coverage remained the same at 100.0% when pulling 0bba560948070047caab27d7fcc09567e23a781d on greenkeeper/ava-0.21.0 into 0d59bc473f63d2155aa0ff24d81951d134a74ea2 on master.


Implemented #604

1 TASK - 1 PR. Please don't let messy/useless code behind when removing UI parts. I did not saw a shipmentViewModel clean up


devops(tooling): Fix path to README.md in driver downloader

I followed the directions in CONTRIBUTING.md and ran ./build.sh --download-driver.
When it ran dotnet run --project ./src/tools/Playwright.Tooling/Playwright.Tooling.csproj -- download-drivers --basepath . I got this following exception:
WARNING: Could not update the browser versions in the README file.
This is usually due to the readme file not yet existing for 1.46.0.
Could not find a part of the path 'D:\Code\RenderMichael\playwright\README.md'.

This fixes the issue for me. I assume it hasn't been hit before because everyone else also has the upstream playwright project as well.

This would end being a noop, since it would read from the same README.md like it writes it to.
We could add a flag like --update-readme and only use this flag internally when rolling a new version via ./build.sh --roll 1.46.0.

Ah got it. Sorry I didn't look closely enough at what the code was doing. My bad.
I can add a flag for that, or to avoid disruptions to flow I can add the inverse --no-update-readme flag, and add guidance to use that in CONTRIBUTING.md. Whatever y'all think is best.

I'll close it since it seem stale. Its just a non-user facing warning, so I think we don't need to perform any action. We could change the text of it if you are interested in making it more clear.


default.profraw file created when running Sourcery 0.9.0

A default.profraw file is created when running Sourcery at its 0.9.0 version.
I tried with 0.8.0 and this does not happen.

I'm seeing this as well.

I don't think it's something that we should look into. As far as I understand this is a file with information about tests results.

My point is that this is creating a file in the users' repository for nothing.
As far as I know, we are not executing "tests" when we use Sourcery to perform the code generation 😉

Ah, sorry @Liquidsoul I misunderstood the task, I thought you mean if you run Sourcery.xcworkspace. In case of the user - you're right 👍

We have similar problem and setting CLANG_ENABLE_CODE_COVERAGE = NO for release build solved the problem.

@ivira thanks for the tip 😉

I don't think this issue was fixed, I'm still seeing that file.

Resolved with #445

Guys, how about releasing this as 0.9.1 update? Seems like necessary bugfix update, this file is changing it's content and showing in git diffs everytime. Requires constant attention and raises questions every time.

@DenHeadless thanks for raising that question. I'll try to release it tonight.
@krzysztofzablocki @ilyapuchka @Liquidsoul  anything we want to add particularly for the release?

@ilyapuchka will try.

@DenHeadless done =)

Thanks guys! Confirmed this fix works 🍻


Autodetect and set appropriate pragmas when parsing Haskell modules

I've implemented reading of language pragmas from the top of Haskell files.  I've created #35 to cover reading extensions from cabal files.