Permanent image feed disabling

I currently disable the image feed on every launch, because I don't need it and it takes up lots of window space.
Can you add a default option "Minimized" for "Image Feed Location"?
Thank you.

Once you close it, it will already remember the closed state
Do you have your browser clearing all data on close or similar?
Last time this was raised, the problem was that the image feed wasnt from this repo but from comfy_mtb so can you please confirm the image feed is from this repo - there should NOT be a load session button if it is mine

This is exactly my issue. My browser is set to not remember anything.
I tried to set an exception for comfy, but it didn't work. I also don't know what I would need to keep saved so the browser can remember.
This is why I'm currently looking for a solution that can save this options as default in the server, so that's why I asked if you could add an extra option.
I had neither mtb nor custom scripts installed before, but on comfyui repo, they said your repo could help, so I installed it.
I think the feed has now a new "resize feed" button and I can't see "load session" on the feed.
Also, due to this, I finally have a tag autocomplete, which is very good.

It's quite disappointing that there isn't an option to turn this off.
Just like @MortalSage, I also make sure to clear my browser cache when I exit (I prefer not to have meta and others tracking my online activities). For those who might find themselves in a similar situation, I've found a workaround by renaming the file to prevent the image feed from loading.
Here are the commands I used:
cd ComfyUI/custom_nodes/ComfyUI-Custom-Scripts/web/js
mv imageFeed.js imageFeed.js.old

Why don't you please stop saving settings in the browser cache instead of using configuration files. For users like me, who use colab and a different URL each time, browser cache is useless.
Thank you

You can now select "hidden" as an option in the settings


Legal issues regarding youtube-dl

Hi,
I was recently looking for an all-in-one solution that would allow me to search & play YouTube videos without even starting a browser to avoid using bloatware (Chrome instances eat up 300 MB if I open up the YouTube search page.. big shame of the 21st century).
I found Minitube and it wasn't that bad, but lacking a few basic features, like selecting output audio device and keyboard shortcuts, so I chose not to purchase and use it. One of the facts I read about made me raise my head. There was a discussion about a removed download feature of MiniTube. It used to be able to download videos, but Google lawyers might have done a great job threatening the author to remove the download capability. I'm just curious why can't they address youtube-dl to remove YouTube download feature for enabling people watch & download copyrighted videos without seeing advertisements and stuff.
Can all liability be excluded if it's an open-source / freeware product?
How does this work?
Best
Pál Ács

We are not lawyers, and thus cannot answer legal questions, especially those that do not specify which country's laws should be applied.
Pass in --include-ads to download advertisements as well.

I don't think you would have to be a lawyer to answer my questions. I did not ask for a legal statement. I am just curious. A lot of YouTube downloading sites, applications, extensions, etc. getting cease and desist letters to remove download capability. Haven't any of you got one of these? How would you react? Could an open-source app like this be killed by some megacorporation like Google? If I have to choose a country, let it be home country, Germany, where (as far as I know) it's prohibited to download copyrighted content and almost all music videos are inaccessible on YouTube.

Can all liability be excluded if it's an open-source / freeware product?

Could an open-source app like this be killed by some megacorporation like Google?

Nobody knows. All maintainers/contributors try their best to solve existing problems and implement new features, while nothing is guaranteed.

By the way, here's a technical issue: --include-ads applies to Brightcove-based websites only.

Doesn't fear from big copyright-enforcing recording companies keep you away from doing this work?

No for me. But anyway it doesn't matter. As a technical project,  to fear or not to fear does not affect it's future.

I don't think you would have to be a lawyer to answer my questions. I did not ask for a legal statement. I am just curious. [...]

There seems to be a slight misconception about copyright here. Copyright is a legal construct that grants exclusive rights to (re-)usage and (re-)distribution of original creative work to the holder, with certain limitations end exceptions, e.g. fair use etc.
It's not obvious if or to which extent copyright applies here.
And since you mention Germany: No, it's not per se prohibited to download copyrighted content, because the act itself of downloading content is something where copyright does not apply directly. There are some other laws, and there have been reforms over the last years that tried to tackle this "issue" a little bit, but they all focus on (technical) measures of protection or enforcement of copyright, and the legality to circumvent said measures, if present.
But downloading itself, no. There is another legal angle to this regarding the storage of copyrighted content, which complicates this matter, but this can't be seen as a complete substitute, the laws don't really overlap, and so the legal gray area still remains..
That many music videos are not available for German users of YouTube is an entirely different issue, please bear that in mind. This is the result of a longstanding and still ongoing legal battle between YouTube and a nice (sarcasm) German organization called GEMA (You can thank Joseph Goebbels for that). But the very same music videos are available on other sites, run by other companies (German companies in this case, actually), for example.

Doesn't fear from big copyright-enforcing recording companies keep you away from doing this work?

Not sure where you see the problem here. As I said, this is somewhat of a legal gray area. This means that there is no clear-cut precedent which would settle these issues. And if you ask me, I think the reason that we don't see more litigation aiming in this direction carried out by some corporations (By the way, YouTube/Google for example is not the copyright holder in this whole scenario) is the fear of setting a new legal precedent, that may be unfavorable to the monetary interests of the corporations involved, if you catch my drift.
Oh, and before I forget it: Participation in an open-source project like this is another topic, there is no connection to possible or theoretical violations of copyright statutes from the project itself. If you crush your thumb with a hammer, do you think the manufacturer of the hammer is responsible?

One of the former contributors is at Google right now. That explains a lot :)


Provide a Typescript type declaration

It would be great to activate type checking using Typescript via your declaration file :)

I'm not a TypeScript regular user, if anybody is interested in this I'm happy to accept a PR 🙂

I'll be happy to write one @Kikobeats 😄

@geopic please do it!

Good stuff, I'll get started on it soon.


Couchdb 2.3 unsupported due to clustered "purge_seq" field transition to string value

Bug Description
Couchdb 2.3 changed the purge_seq field in the DbInfo response to be a string (this possibly applies to other sequences as well)
1. Steps to reproduce and the simplest code sample possible to demonstrate the issue
Try to configure a client against a CouchDB 2.3 server and issue and query that'll call the  info() method from a Database instance.
2. What you expected to happen
This should have worked
3. What actually happened
Gson deserialization failed with the following stacktrace:
Caused by: com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: For input string: "0-g1AAAAFTeJzLYWBg4MhgTmEQTM4vTc5ISXIwNDLXMwBCwxygFFOSA5BMiv___39WIgMedXksQJLhAJACKr2PX22SAshMe4JmJjIkyRNpcQPE4vnEqH0AUUuUuQsgavcT8FACyEP1EDOzADZYXo4"
	at com.google.gson.internal.bind.TypeAdapters$11.read(TypeAdapters.java:306)
	at com.google.gson.internal.bind.TypeAdapters$11.read(TypeAdapters.java:296)
	at io.gsonfire.gson.ExcludeByValueTypeAdapterFactory$ExcludeByValueTypeAdapter.read(ExcludeByValueTypeAdapterFactory.java:99)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:131)
	at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:222)
	at io.gsonfire.gson.ExcludeByValueTypeAdapterFactory$ExcludeByValueTypeAdapter.read(ExcludeByValueTypeAdapterFactory.java:99)
	at io.gsonfire.gson.HooksTypeAdapter.deserialize(HooksTypeAdapter.java:86)
	at io.gsonfire.gson.HooksTypeAdapter.read(HooksTypeAdapter.java:54)
	at com.google.gson.Gson.fromJson(Gson.java:927)
	at com.google.gson.Gson.fromJson(Gson.java:865)
	at com.cloudant.client.org.lightcouch.internal.CouchDbUtil.getResponse(CouchDbUtil.java:229)
	at com.cloudant.client.org.lightcouch.CouchDbClient.get(CouchDbClient.java:384)
	at com.cloudant.client.api.Database.info(Database.java:1333)

Environment details

CouchDB versions

2.3

Version(s) that are affected by this issue.

2.13.1

Java version including vendor and platform

JDK 11 Linux (Azul) , but relevant to all platforms

Using the optional okhttp dependency.

No

Thanks, we are aware of this issue and have plans to resolve it prior to it impacting Cloudant services.

@ricellis  thank you for your response. Do you have an estimated timeline to when might this happen?

I'd like it to be in the next release, which I think will be in January.

@sheinbergon The 2.14.0 release includes the fix and is now available from Maven Central.


"Auto Failover" feature

If one or more of the agent server is down during the process, will the rest functioning agents pick up its left job ?

no. currently the job will fail.


Appstudio update test-component-pac-oaen

Pipelines as Code configuration proposal
To start the PipelineRun, add a new comment with content /ok-to-test
For more detailed information about running a PipelineRun, please refer to Pipelines as Code documentation Running the PipelineRun
To customize the proposed PipelineRuns after merge, please refer to Build Pipeline customization

Pipelines as Code CI/test-component-pac-oaen-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
20 seconds
init

✅ Succeeded
24 seconds
clone-repository

✅ Succeeded
1 minute
build-container

✅ Succeeded
36 seconds
deprecated-base-image-check

✅ Succeeded
34 seconds
inspect-image

✅ Succeeded
1 minute
clamav-scan

✅ Succeeded
3 minutes
clair-scan

✅ Succeeded
31 seconds
sbom-json-check

✅ Succeeded
9 seconds
label-check

✅ Succeeded
7 seconds
show-summary

✅ Succeeded
7 seconds
show-sbom

Pipelines as Code CI/test-component-pac-oaen-on-pull-request has failed.

StatusDurationName

✅ Succeeded
5 seconds
init

✅ Succeeded
18 seconds
clone-repository

🏃 Running
---
build-container

Pipelines as Code CI/test-component-pac-oaen-on-pull-request has failed.

StatusDurationName

✅ Succeeded
5 seconds
init

✅ Succeeded
18 seconds
clone-repository

🏃 Running
---
build-container

Pipelines as Code CI/test-component-pac-oaen-on-pull-request has failed.

StatusDurationName

✅ Succeeded
5 seconds
init

✅ Succeeded
18 seconds
clone-repository

✅ Succeeded
32 seconds
build-container

Pipelines as Code CI/test-component-pac-oaen-on-pull-request has successfully validated your commit.


method call is put a new line although there is enough horizontal space

impl Foo {
    fn foo() {
        self.report.add_non_formatted_ranges(visitor.skipped_range.clone());
    }
}

 impl Foo {
     fn foo() {
-        self.report.add_non_formatted_ranges(visitor.skipped_range.clone());
+        self.report
+            .add_non_formatted_ranges(visitor.skipped_range.clone());
     }
 }

This is expected, as the width of the chain exceeds chain_width (which is 60 by default).

@topecongiro what is chain_width? It is not documented in https://github.com/rust-lang/rustfmt/blob/master/Configurations.md

I think that is the property of the width_heuristics, but it is listed under not user-facing in the config.
https://github.com/rust-lang/rustfmt/blob/c0e616bc1d19eb9b08a5db2f265b8332569e7259/src/config/mod.rs#L150
https://github.com/rust-lang/rustfmt/blob/c0e616bc1d19eb9b08a5db2f265b8332569e7259/src/config/options.rs#L190-L191


On Degger.js of latest version: Uncaught SyntaxError: Cannot use 'import.meta' outside a module

Configuration:

Web browser and its version: Chrome Version 104.0.5112.101
Operating system and its version: Ubuntu 18.04
PDF.js version:  2.16.105
Is a browser extension: No

I used the viewer.js in my Flask application. In the viewer.html, I have
<link rel="resource" type="application/l10n" href="<path_to_locale.propertie>"/>
<script type="text/javascript" src="<path_to_pdf.js>"></script>
<script type="text/javascript" src="<path_to_viewer.js>"></script>
<script type="text/javascript" src="<path_to_debugger.js"></script>

I don't have to include     <script type="text/javascript" src="<path_to_debugger.js"></script> in viewer.html to work. But I wanted to include it so that I don't miss any features. Then I got this error on my console:
VM496:1 Uncaught SyntaxError: Cannot use 'import.meta' outside a module
I think this is a bug in debugger.js. Am I wrong? What am I supposed to with that?

web/debugger.js and its accompanying CSS-file is only intended for debugging/development purposes, as indicated by the filename, hence it's not normally something that you need/want in a production environment.

<script type="text/javascript" src="<path_to_debugger.js"></script>

Please note that that file has never been intended to be loaded unconditionally in that way[1], since the default viewer has always been loading it lazily; see https://github.com/mozilla/pdf.js/blob/95e3f5d49510bfa3425d7bfb9ce8be48deac30c0/web/app.js#L2212-L2220
Hence what you're trying to do here is neither intended, or supported, unfortunately.

[1] Although I suppose that setting type="module" instead probably "works", but again it doesn't really make sense to load it with a script-tag.


Update to more performant source mapping

rationale: since inline-source-map generates 1:1 line-column mappings it is much more expensive to generate... dev mode can use cheap-module-eval-source-map algorithm as it provides huge performance gains by only mapping the line numbers from original source
@mxstbr this change has been removed from intl PR :)

Coverage remained the same at 98.293% when pulling 031309a9d890857882ebe78b7e36535898b44068 on justingreenberg:patch-5 into 4a7d9ed93cfc682946a1130450f3ae1921d7f56b on mxstbr:master.

Awesome, thanks so much @justingreenberg!


UserCredential with ADFS error: AADSTS70002: The request body must contain the following parameter: 'client_secret or client_assertion'.

Hi,
I'm runing this sample with AFDS flow - When I'm trying to get the token - I'm getting an error:
 Inner Exception : AADSTS70002: The request body must contain the following parameter: 'client_secret or client_assertion'. Trace ID: 90ed39ca-2403-4efe-be94-6e237a060c00 Correlation ID: 94c8938b-5bf1-4103-8900-e0df04937f01 Timestamp: 2018-05-17 09:06:30Z
Can I use the  UserCredential flow to AcquireToken when I have ADFS Flow ?

Yes, you should
are you sure you have registered your application as a public client application (See https://github.com/Azure-Samples/active-directory-dotnet-native-headless#register-the-client-app-todolistclient-headless, the application type needs to be Native)

Closing as there was no answer.


use activityResultRegistry in libgdx for android

hi, how i can use activityResultRegistry in activity that extend from AndroidApplication?
problem is:
my class (activity) is
class InAppBlillingBazaar: AndroidApplication() {}
AndroidApplication class extend from Activity, but activityResultRegistry exist in AppCompatActivity. and i cant edit AndroidApplication because its part of libgdx libs.
public class AndroidApplication extends Activity implements AndroidApplicationBase {}
Can anyone solve this?

I think you can instead use AndroidFragmentApplication to drop the libgdx app into an AppCompatActivity.

Closing this out, if using AndroidFragmentApplication didn't work, feel free to reopen it.


Implement sorted order for each view

For the "All" view:

Group first by line
For each line, group alerts by Ongoing Alerts, then Upcoming Alerts
Within those groupings, group alerts by End Date Exists, then No End Date
Then within those groupings, order by effective date ascending.

For the "Ongoing"/"Upcoming" views:

Group first by line
Within those groupings, group alerts by End Date Exists, then No End Date
Then within those groupings, order by effective date ascending.

Examples:
Line 2

Line 105:

Line 158

Line 207


test(client): filter using null value in nullable self-relation

Should provide a test case for #3354 #3352
Could be merged with relations test

Looks like for some reason the tests didn't run and now it failed in CI, so I'm reverting it and will figure out why the test didn't run 😄


Fix postgres preflight condition

Conflict needs to be resolved!

/azp run

Fixed. However, this will still block the installation in edge cases, such as re-applying after changing the number of single_machine nodes from 1 to 2, and in case of custom mappings for PostgreSQL:
apply:
    kubernetes_master:
      count: 1
    postgresql:
      count: 0

---
kind: configuration/feature-mapping
title: Feature mapping to roles
name: default
provider: aws
specification:
  roles_mapping:
    kubernetes_master:
    - kubernetes-master
    - helm
    - applications
    - node-exporter
    - filebeat
    - postgresql

:x: re-apply:
    kubernetes_master:
      count: 1
    postgresql:
      count: 1

---
kind: configuration/feature-mapping
title: Feature mapping to roles
name: default
provider: aws
specification:
  roles_mapping:
    kubernetes_master:
    - kubernetes-master
    - helm
    - applications
    - node-exporter
    - filebeat
    - postgresql


apiserver fails to build due to missing dependency

When building the apiserver, ./build.sh local produces the following error:
Building apiserver-linux-amd64
vendor/k8s.io/apimachinery/pkg/util/httpstream/spdy/connection.go:25:2: cannot find package "github.com/docker/spdystream" in any of:
	/home/ubuntu/go/src/github.com/ndslabs/apiserver/vendor/github.com/docker/spdystream (vendor tree)
	/usr/local/go/src/github.com/docker/spdystream (from $GOROOT)
	/home/ubuntu/go/src/github.com/docker/spdystream (from $GOPATH)
Building apictl-linux-amd64

This package is notably absent from the Fetching... output:
[INFO]	--> Fetching github.com/davecgh/go-spew
[INFO]	--> Fetching github.com/docker/docker
[INFO]	--> Fetching github.com/ghodss/yaml

Dependency versions likely need to be updated, and ideally pinned.

Fixed by #330


Comments on Blocks

Comment pane doesn't always disappear when you change focus.  Also, trying to delete a block with a comment is a pain... since when you click on a block the comment pane gets the focus.

The problem is that if comments are not automatically opened, users never looked at them. The gearwheel is just not intuitive. We should prevent taking the focus when poping up the comment.

@samelhusseini can we close this with the new comments?

Yes


Bind mount source path has changed and unable to "ddev start"

Hi, I've installed DDEV 1.1.1 via Homebrew on MacOS 10.13.6 with Docker 18.06.1-ce-mac73 (26764). Docker had the basic setup. I've simply installed a current Drupal 8 install with composer create-project drupal-composer/drupal-project:8.x-dev ddevsite --stability dev in ~/sites for initial testing purposes. But the install was creeping slow and so was the loading of each new page afterwards - the CPU peaked between 100 to 160% constantly and the fans where steadily running at max.everything took forever. I performed a ddev stop afterwards. After some research I then went with the recommendations of the following post over in the docker for-mac issue tracker
https://github.com/docker/for-mac/issues/1759#issuecomment-412486531 . Kubernetes got disabled, switched off experimental features on daemon, set under advanced to 1 cpu, to4gb out of 8gb of memory and left swap to 1gb and removed under file sharing /users and /volumes and replaced the former with /users/myuser/sites (the directory containing all the web related projects utilizing docker). problem is when i try to restart the ddevsite project with ddev start afterwards i get:
Unable to get project(s): a project (web container) in exited state already exists for ddevsite that was created at /Users/myuser/sites/ddevsite
When i take a look into the .ddev subfolder inside the ddevsiteproject folder i am unable to find any hint to a path containing /Users/myuser/sites/ddevsite neither in the config.yaml nor in the docker-compose.yaml. I also looked into /Users/myuser/sites/ddevsite/web/sites/default/settings.ddev.php also nothing in that regard. Is there a way to update the bin mount source path so i am able to utilize the already present setup so that i dont end up having any unused old skeletons in the closet? Thanks Ralf

Somehow you ended up with a web container created in a different directory. You should be able to use ddev rm -R ddevsite to get rid of it; if not, docker rm -f $(docker ps -aq) will kill off all containers (including non-ddev containers). docker ps -a shows all containers including non-ddev containers.

at first thanks for the fast reply!
hmmmm strange thing is I never moved anything?! I just removed /usersand/volumesfrom the file sharing setting in the docker prefs as I told while ddevsite was stopped. So the site is still recognized:
$> ddev list
NAME      TYPE     LOCATION          URL(s)                       STATUS
ddevsite  drupal8  ~/sites/ddevsite  http://ddevsite.ddev.local   stopped
                                     https://ddevsite.ddev.local

DDEV ROUTER STATUS: not found
The router is not currently running. Your sites are likely inaccessible at this time.
Try running 'ddev start' on a site to recreate the router.

Odd thing is that the router container isn't found and shown when I perform:
$> docker ps -a
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS                    PORTS               NAMES
a0c4cbb23bf1        drud/phpmyadmin:v1.1.0       "/run.sh phpmyadmin"     13 hours ago        Exited (0) 13 hours ago                       ddev-ddevsite-dba
158fad8af18a        drud/ddev-webserver:v1.1.0   "/start.sh"              13 hours ago        Exited (0) 13 hours ago                       ddev-ddevsite-web
cd15c4138d39        drud/ddev-dbserver:v1.1.0    "/docker-entrypoint.…"   13 hours ago        Exited (0) 13 hours ago                       cd15c4138d39_ddev-ddevsite-db

the router container isn't showing up there.  ddev doesn't have anything else where it stores anything? everything could be found either with docker ps -aon the container side or within the project inside .ddev? So safest and cleanest would be simply destroying those three containers and deleting the ddevsite folder and start fresh?

I don't think anything will work if you removed /Users from the docker share. And do docker rm -f $(docker ps -aq) to get rid of the stopped containers.

I recommend that you do a docker reset-to-defaults as a starting point; if you do that you don't even have to remove those containers. It will give you a decent starting point. That will also delete the docker volume that has your database on it, so that will be clean as well.

thanks a lot! indeed the reset was the fastest and most convenient way. thanks for the heads up! will try later to find a configuration without fans running and faster execution on the docker side of things :)


fix: fill defaults for non-required field of type "record"

Fixes #334
Adds a check for required while recusrively filling defaults for subconfig of type record.
Current behaviour of KIC is out of sync from Kong Lua implementation.
There are some field "required" checks in Lua code: here and here
These checks are missing form go code

thanks @nishant95! I just tested your branch and I still have the same issue, my field is set to null.  My fix in #333 works understand that our PRs don't fix exactly the same thing. To support default with set of record, we'll need to merge both, right?

@aboudreault Yes, I raised this PR for issue #334 , this deals with record type only.
Your PR looks good and should fix it for type "set".
But shouldn't there be a similar handling for type "map" as well? Similar to this in Lua.

@nishant95 this is possible yeah. Let's try to release this fix and #333, I will create a ticket for the map type.

Overriding linter failure and merging cause it ain't worth waiting to round trip to the upstream branch and the error is inscrutable:
Check failure on line 2262 in kong/utils_test.go
GitHub Actions / lint

File is not `gci`-ed with --skip-generated -s standard -s default -s prefix(github.com/kong) (gci)

golangci, please just print the actual command you need to run. Furthermore, the import block didn't change between those commits; it shouldn't start failing. This is not an import >:(
10:47:38-0700 esenin $ gci diff --skip-generated  -s standard -s default -s "prefix(github.com/kong)" utils_test.go
--- utils_test.go
+++ utils_test.go
@@ -2259,4 +2259,4 @@
 			}
 		})
 	}
-}
\ No newline at end of file
+}

Going to fix in the release changelog commit.


Cannot apply to link elements

The script, when I apply .jGravity() to <div> elements with link tags inside them, does not drop those div's.

@danksky I'm sorry, I haven't actively maintained this. However, if you create a PR, I'll take a look. Thank you for reporting

Closing for inactivity


support for go 1-23

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 88.47%. Comparing base (becfc50) to head (0b90f09).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #74   +/-   ##
=======================================
  Coverage   88.47%   88.47%
=======================================
  Files          14       14
  Lines        1180     1180
=======================================
  Hits         1044     1044
  Misses        111      111
  Partials       25       25

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


chore: workflow automation

Mark inactive issues as stale after 30 days
Close stale issues after 30 days

Mi Gusta!

:tada: This PR is included in version 1.31.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[Core]: Support destroying all KV cache during runtime

Implements #10714
API Design:

Destroy (this PR implements): vllm.LLM().llm_engine._destroy_kv_caches()
ReInitialize (already have): vllm.LLM().llm_engine._initialize_kv_caches()
Stop loop (already have): vllm.LLM().llm_engine.model_executor.stop_remote_worker_execution_loop()

This PR only implements _destroy_kv_caches for GPU executor and workers, as I don’t have other available hardware, feel free to take over this PR to implement others, and once we finish all the implementations, we can make destroy_cache() an abstract method.
Also, since the engine won’t generate without KV Caches (will throw errors), this PR assumes that the developers will handle everything on their side so that no request will be sent to generate after _destroy_kv_caches() and before _initialize_kv_caches() (in sleep mode)
Code for testing:
import ray, time
from ray.util.placement_group import placement_group
from ray.util.scheduling_strategies import PlacementGroupSchedulingStrategy

@ray.remote
class LLMRayActor:
    def __init__(self, *args, **kwargs):
        import vllm

        if not kwargs["tensor_parallel_size"] == 1:
            kwargs["worker_use_ray"] = True

        self.llm = vllm.LLM(*args, **kwargs)

    def generate(self, *args, **kwargs):
        return self.llm.generate(*args, **kwargs)

    def destroy_cache(self):
        self.stop_remote_worker_execution_loop()
        self.llm.llm_engine._destroy_kv_caches()

    def load_cache(self):
        self.stop_remote_worker_execution_loop()
        self.llm.llm_engine._initialize_kv_caches()

    def stop_remote_worker_execution_loop(self):
        self.llm.llm_engine.model_executor.stop_remote_worker_execution_loop()

def create_vllm_engines(
    num_engines: int,
    tensor_parallel_size: int,
    model: str,
):
    vllm_engines = []
    for _ in range(num_engines):
        num_gpus = int(tensor_parallel_size == 1)
        scheduling_strategy = None

        if tensor_parallel_size > 1:
            bundles = [{"GPU": 1, "CPU": 1}] * tensor_parallel_size
            pg = placement_group(bundles)
            ray.get(pg.ready())

            scheduling_strategy = PlacementGroupSchedulingStrategy(
                placement_group=pg, placement_group_capture_child_tasks=True, placement_group_bundle_index=0
            )

        vllm_engines.append(
            LLMRayActor.options(
                num_cpus=1,
                num_gpus=num_gpus,
                scheduling_strategy=scheduling_strategy,
            ).remote(
                model,
                tensor_parallel_size=tensor_parallel_size,
            )
        )

    return vllm_engines

if __name__ == "__main__":
    # engines = create_vllm_engines(2, 2, "meta-llama/Llama-3.1-8B-Instruct")
    engines = create_vllm_engines(4, 1, "meta-llama/Llama-3.1-8B-Instruct")

    ref = []
    for engine in engines:
        ref.append(engine.generate.remote("San Francisco is a"))
    print(f"output: {ray.get(ref)}")

    ref = []
    for engine in engines:
        ref.append(engine.destroy_cache.remote())
    ray.get(ref)

    time.sleep(5)

    ref = []
    for engine in engines:
        ref.append(engine.load_cache.remote())
    ray.get(ref)

    ref = []
    for engine in engines:
        ref.append(engine.generate.remote("New York is a"))
    print(f"output: {ray.get(ref)}")

Also, since the engine won’t generate without KV Caches (will throw errors), this PR assumes that the developers will handle everything on their side so that no request will be sent to generate after _destroy_kv_caches() and before _initialize_kv_caches() (in sleep mode)

Maybe another possible way to get this handled is to check if we have initialized the KV cache when the engine receives the request, if not, initialize it so that we can get rid of manual intervention. But this didn't get implemented in this PR.


Change path to import test modules

Summary
Fix #131.

Codecov Report

Merging #132 (b3efbce) into develop (8805a70) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #132   +/-   ##
========================================
  Coverage    96.53%   96.53%
========================================
  Files           21       21
  Lines         2570     2570
========================================
  Hits          2481     2481
  Misses          89       89

Impacted Files
Coverage Δ

ssspy/algorithm/projection_back.py
100.00% <ø> (ø)

ssspy/bss/ilrma.py
98.29% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


tf.io.gfile.glob is extremely slow

Click to expand!
Issue Type
Bug
Source
binary
Tensorflow Version
2.9 and 2.10
Custom Code
No
OS Platform and Distribution
macOS 12
Mobile device

Python version
3.9
Bazel version

GCC/Compiler version

CUDA/cuDNN version

GPU model and memory
No response

Current Behaviour?
In my environments, tf.io.gfile APIs are extremely slow.
Standalone code to reproduce the issue
For instance:
>>> import tensorflow.io.gfile as gfile  # type: ignore
>>> gfile.glob('gs://tfds-data/*.jsonl')
2022-10-01 22:59:07.324892: W tensorflow/core/platform/cloud/google_auth_provider.cc:184] All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "NOT_FOUND: Could not locate the credentials file.". Retrieving token from GCE failed with "FAILED_PRECONDITION: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Could not resolve host: metadata".
['gs://tfds-data/community-datasets-list.jsonl']

The above takes around 70 seconds to query. Note that gs://tfds-data/ bucket has only ~7 files on the root directory. Would glob walk the entire subtree recursively?
In contrast, gsutil commands are quite fast.
❯❯❯ time noglob gsutil ls gs://tfds-data/*.jsonl
gs://tfds-data/community-datasets-list.jsonl
noglob gsutil ls gs://tfds-data/*.jsonl  0.62s user 0.16s system 78% cpu 0.997 total

Relevant log output
No response

@gadagashwini
I was able to reproduce the issue on Colab using TF v2.10. Please find the gist here for reference.
Thank you!

This is still replicating in tf-nightly. Here's the gist

Hi,
Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.
The Tensorflow team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.
Please follow the release notes to stay up to date with the latest developments which are happening in the Tensorflow space.


Updated README.md

Corrected grammatical error in a sentence.

@surajphulara I'd love to accept your PR. A couple things:
You need to sign the CLA in this thread and your PR needs to conform to our commit format (no upper case letters). Also use the present tense (not updated). something like content: fix grammer in readme would work.

I have read the CLA Document and I hereby sign the CLA

recheckcla

recheckcla

@surajphulara I'd like to merge your PR and close it. Can you rebase and be sure that the CI passes so I can do so...

@surajphulara I'd like to merge your PR and close it. Can you rebase and be sure that the CI passes so I can do so...

How to make it pass CI

@suraj142 all you need to do is rebase and merge to the latest develop

@suraj142 thank you for the contribution.
I cherry-picked and merged your changes, with some minor changes to the commit message to follow our conventions.
Made sure your attribution is maintained. https://github.com/ockam-network/website/commit/731b88db08730af97d00337df3b8ba58866af809

@suraj142 thank you for the contribution.
I cherry-picked and merged your changes, with some minor changes to the commit message to follow our conventions.
Made sure your attribution is maintained. https://github.com/ockam-network/website/commit/731b88db08730af97d00337df3b8ba58866af809


Fix when first child tag is not NODE_ELEMENT

Hi, thanks for pull request.
But there are few comments:

For leaflet 0.7x use branch "leaflet-0.7.3", it not maintained, but if you want use this version - you can update this branch and send PR here.
Need tests for changed firstChild element.
Please send PR again to develop branch

Hi!

This pull requests affects not only 0.7 branch, but 1.x too
I'm not spec in karma, I can send you a valid xml from WMS server that fails
Ok, as soon as you answer to me


[FEATURE] Explore Page

Problem
Currently, rounds are just shared, we need a way actually to discover rounds
User Story(s):
As a potential voter, I want to see all the upcoming and past round, so that I can discover places where I can vote
Description
Description of solution
Acceptance Criteria

filter by upcoming rounds, application open, past rounds
show closed round
show vote
direct to #15 page
have a description
make this the home page
show which rounds are vaulted vs just suggested

Limitations

dependent on design #4
need to query active rounds from factory round #11 contract (maybe slow), dependent on how we expose ehlper functions on contract

Resources

https://github.com/PotLock/grantpicks/pull/34


NEW UPDATES

New Updates in Solo Mode, Cards and more but maybe pixeltris stopped updating his mod

Added in v1.9.


Any plans to use fater-whisper?

Hi, thanks for the good work. SInce there was no placholder for discussion thread I am using this as a feature suggestion: whould you have any plans to use fater-whisper models --for the purpose reducing VRAM and processing time? Thanks.

Thanks for the suggestion. I may add this in future!

@dgoryeo added the faster-whisper. works like a charm!

Wonderfull!!
Side question: can one specify large-v2 instead of large? I presume large will use large-v3, which unfortunately has bot been very good.


create_model fails to use type annotation of typing.Annotated

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
Using create_model fails with an "A non-annotated attribute" error when using an annotation created with typing.Annotated. This same annotation works as expected when used to define a field as a subclass of BaseModel. The difference in behavior is apparent in the FieldInfo of the two examples; in the BaseModel subclass example the field's FieldInfo annotation is bool, whereas in the create_model example the field's FieldInfo annotation is NoneType.
Example Code
from typing import Annotated

from pydantic import BaseModel, create_model
from pydantic.fields import Field

annotated = Annotated[bool, Field(alias="field_alias", description="this is a description")]

print(annotated)
# typing.Annotated[typing.Optional[bool], FieldInfo(annotation=NoneType, required=True, alias='field_alias', alias_priority=2, description='this is a description')]

class Model(BaseModel):
    field: annotated

print(Model.model_fields)
# {'field': FieldInfo(annotation=bool, required=True, alias='field_alias', alias_priority=2, description='this is a description')}

create_model('CreatedModel', field=annotated)
# pydantic.errors.PydanticUserError: A non-annotated attribute was detected: `field = typing.Annotated[bool, FieldInfo(annotation=NoneType, required=True, alias='field_alias', alias_priority=2, description='this is a description')]`. All model fields require a type annotation; if `field` is not meant to be a field, you may be able to resolve this error by annotating it as a `ClassVar` or updating `model_config['ignored_types']`.

Python, Pydantic & OS Version
pydantic version: 2.5.3
        pydantic-core version: 2.14.6
          pydantic-core build: profile=release pgo=false
               python version: 3.12.1 (main, Dec  7 2023, 20:45:44) [Clang 15.0.0 (clang-15<IP_ADDRESS>)]
                     platform: macOS-14.2.1-x86_64-i386-64bit
             related packages: typing_extensions-4.9.0

@trippersham,
Thanks for suggesting this. I'm going to mark this as a feature request - this isn't something we've supported in the past afaik, so it's something we'll have to add.
Also marking as help wanted if anyone wants to work on this 😄.

Thanks @sydney-runkle for the re-categorization. I saw this pattern used in the dyntamic JSON Schema to Pydantic Model generator and assumed (incorrectly!) that this was a regression.

Hello, wanted to work on this issue if nobody is currently working on it. But wanted to get the problem and expected outcome straightened out.
from typing import Annotated

from pydantic import BaseModel, create_model
from pydantic.fields import Field

annotated = Annotated[bool, Field(alias="field_alias", description="this is a description")]

Given the annotated,
create_model('CreatedModel', field=annotated)

should behave equally as
create_model('CreatedModel', field=(bool, FieldInfo(alias="field_alias", description="this is a description"))

However, the current behavior seems that it is unable to interpret the field type and assuming it is a NoneType?

@wannieman98,
That's correct. We're excited for you to contribute! Ping me whenever you have a PR you'd like me to review :), or if you have any more questions.

I'm seeing the following error trying to use Pydantic with the Python 3.13 alpha:
Traceback (most recent call last):
  File "/tmp/3.13.0a3/bin/llm", line 5, in <module>
    from llm.cli import cli
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/llm/__init__.py", line 18, in <module>
    from .plugins import pm
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/llm/plugins.py", line 37, in <module>
    mod = importlib.import_module(plugin)
          ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/Users/simon/.pyenv/versions/3.13.0a3/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/llm/default_plugins/openai_models.py", line 7, in <module>
    import openai
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/openai/__init__.py", line 8, in <module>
    from . import types
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/openai/types/__init__.py", line 5, in <module>
    from .image import Image as Image
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/openai/types/image.py", line 5, in <module>
    from .._models import BaseModel
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/openai/_models.py", line 408, in <module>
    class FinalRequestOptions(pydantic.BaseModel):
    ...<59 lines>...
            model_construct = construct
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/pydantic/_internal/_model_construction.py", line 178, in __new__
    set_model_fields(cls, bases, config_wrapper, types_namespace)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/pydantic/_internal/_model_construction.py", line 452, in set_model_fields
    fields, class_vars = collect_model_fields(cls, bases, config_wrapper, types_namespace, typevars_map=typevars_map)
                         ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/pydantic/_internal/_fields.py", line 214, in collect_model_fields
    field_info = FieldInfo.from_annotated_attribute(ann_type, default)
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/pydantic/fields.py", line 372, in from_annotated_attribute
    if _typing_extra.is_annotated(annotation):
       ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/pydantic/_internal/_typing_extra.py", line 103, in is_annotated
    return origin is not None and lenient_issubclass(origin, Annotated)
                                  ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/tmp/3.13.0a3/lib/python3.13/site-packages/pydantic/_internal/_utils.py", line 76, in lenient_issubclass
    return isinstance(cls, type) and issubclass(cls, class_or_tuple)
                                     ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/Users/simon/.pyenv/versions/3.13.0a3/lib/python3.13/typing.py", line 486, in __subclasscheck__
    raise TypeError(f"{self} cannot be used with issubclass()")
TypeError: typing.Annotated cannot be used with issubclass()

Could it be that this issue needs addressing for compatibility with Python 3.13?
Full steps to reproduce in this issue comment:

https://github.com/simonw/llm/issues/436#issuecomment-1992678618

We haven't added support for Python 3.13 yet (to my knowledge), so I wouldn't say that this issue is unique in that sense.


'All Channels' information under 'Guide' are not loading

Hi
I have tried to fix it by reinstalling KODI but channels information are not loading.
Tried everything to fix but no success.
Kindly look into this.
Thank you
Abhishek

Now, some Channels' guide is not loading. Ex. Colours

This issue is coming again and again. No Guide information is loading.

There are few updates by KODI. It means, there is a need to make changes in Add-ons.
I do not know much, it's what I think.

https://kodi.tv/addons/matrix/pvr.iptvsimple/
New version of PVR available https://kodi.tv/addons/matrix/pvr.iptvsimple/

Issue not resolved

Someone please check


Vercel run duplicated

We are currently running 2 Vercel runs in parallel:

blockstacks-docs
docs

We should move all the functionality to docs.

@criadoperez what are the next steps on this? Maybe @wileyj can help here?

Maybe the easiest option would be for someone with permissions to the blockstacks-docs to transfer the project to docs (of the team "stacks-team").
That way we don't have to reprogram the functionality of docs.
Anybody with permissions for this?

Maybe the easiest option would be for someone with permissions to the blockstacks-docs to transfer the project to docs (of the team "stacks-team"). That way we don't have to reprogram the functionality of docs. Anybody with permissions for this?

We can also check on Hiro's end (cc/ @CharlieC3 ) -- though I would recommend just creating a new Vercel project that is under Foundation / community governance for this repo.

I would recommend just creating a new Vercel project that is under Foundation / community governance for this repo.

That's exactly what we did! :+1:  Now we just need to transfer the project from one user to the other.

@criadoperez If you already created your own Vercel project for this repo, wouldn't it be better if we just deleted the corresponding project Hiro owns instead of transferring it?

I don't mind. Whatever is easier... I would have thought that transferring would be easier but I may be wrong. The main thing is that our Vercel account should be able to deploy on docs.stacks.co
Our Vercel project in publishing on the following URLs:
docs-stacks-team.vercel.app
docs-one-sigma.vercel.app
docs-git-master-stacks-team.vercel.app
blockstacks-docs is publishing on the following URLs:
blockstack-docs.vercel.app
docs.stacks.co
blockstack-docs-blockstack.vercel.app
blockstack-docs-git-master-blockstack.vercel.app
docs.blockstack.org
docs-staging.blockstack.org
If you delete blockstack-docs, will it then allow me to enable docsto deploy on docs.stacks.co or I'll need some sort of permissions for that?

@criadoperez ah so it seems the two projects aren't exact copies due to the domain differences. Also, it doesn't seem like I'm able to transfer the project over unless we're a part of the stacks-team. I've requested access.

I accepted the request.

Transferred over! Let me know if it looks good on your end.

Thanks @CharlieC3 ! That seems to have worked.
For the record, we know have only one Vercel project called stacks-docs that replaces both docs and `blockstacks-docs'


feat: add integration tests & integration test framework

Description
This PR adds integration tests and establishes an integration test framework using terraform tests and python, for observe-agent testing.
Checklist

[x] Created tests which fail without the change (if possible)
[x] Extended the README / documentation, if necessary

are we cleaning up the ec2 instances afterwards? can you point me at that part?

are we cleaning up the ec2 instances afterwards? can you point me at that part?

Yeap. Tf tests automatically sets up and destroys as part of in-memory state
https://developer.hashicorp.com/terraform/tutorials/configuration-language/test
"Terraform tests let you validate your module configuration without impacting your existing state file or resources. Testing is a separate operation that is not part of a plan or apply workflow, but instead builds ephemeral infrastructure and tests your assertions against in-memory state for those short-lived resources"


DOC: Incorrect Vector Store Provider Name in Documentation - Should be 'chromadb' Instead of 'chroma'

Issue with current documentation:
Bug Report: Incorrect Vector Store Provider Name in Documentation
Description:
The Mem0 documentation lists chroma as a supported vector store provider, but the correct provider name is chromadb. This discrepancy causes a ValueError when trying to configure Mem0 with chroma.
Steps to Reproduce:

Follow the Mem0 documentation to configure Mem0 with chroma as the vector store provider.
Run the script to initialize Mem0.

Expected Behavior:
Mem0 should initialize successfully with chroma as the vector store provider.
Actual Behavior:
A ValueError is raised indicating that chroma is an unsupported vector store provider.
Error Message:
ValueError: Unsupported VectorStore provider: chroma

Proposed Solution:
Update the documentation to use the correct provider name chromadb.
Environment:

Mem0 version: 0.1.120
Python version: 3.10
Operating System: [Mac OSX Sonoma 14.5 (23F79)]
Mac M3 Max Pro

Additional Context:
Here is the relevant section of the configuration that causes the error:
config = {
    "vector_store": {
        "provider": "chroma",
        "config": {
            "collection_name": "test",
            "path": "./chroma_langchain_db"
        }
    },
    "llm": {
        "provider": "ollama",
        "config": {
            "model": "test"
        }
    },
    "embedder": {
        "provider": "ollama",
        "config": {
            "model": "test"
        }
    }
}

Changing chroma to chromadb resolves the issue:
config = {
    "vector_store": {
        "provider": "chromadb",
        "config": {
            "collection_name": "test",
            "path": "./chroma_langchain_db"
        }
    },
    "llm": {
        "provider": "ollama",
        "config": {
            "model": "test"
        }
    },
    "embedder": {
        "provider": "ollama",
        "config": {
            "model": "test"
        }
    }
}

Thank you for addressing this issue!

Hey @jameshgrn, This issue is fixed in the latest version. Please install that try it out again.
Let me know if you still face any issues!


The tutorial is dated and inaccurate

[Enter feedback here]
The tutorial is not updated some of the functions are missing
Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 74bf3716-f9bc-237b-395b-c61eb5c91dbf
Version Independent ID: 7e7bcfee-80a6-760d-76ab-bd779160cf43
Content: PlayFab authentication with Google Play Games Sign-In in Unity - PlayFab
Content Source: playfab-docs/features/authentication/platform-specific-authentication/google-sign-in-unity.md
Service: playfab
GitHub Login: @joannaleecy
Microsoft Alias: joanlee

Thank you for taking the time to report the problem. We are doing a full review of this page for required updates.

Page has been updated. Please let us know if you have any further feedback. Thanks!


Add new JWT renew token without revoking existing token feature to documentations

Is your suggestion related to a missing or misleading document? Please describe.

Currently, when HA enable for a service provider and if each node request for a token, When the token type is opaque (default), IS keep the token in the database and return the same token for combination of application, scope, binding, user. But when it comes to JWT tokens, we cannot keep the token in the database due to the different lengths in different JWT tokens. Hence when application instances in HA request for a JWT token for the same application, scope, binding, user combination, current, IS will revoke the previous issued token and issue a new token. In that case, in HA environment, only one token will be in active state.

Describe the improvement

To solve this issue, we are adding new binding type called request binding type. When we enable this feature from configurations, we will add a new binding type to JWT tokens based on configured values. We will set a random UUID as the value of this new binding type, so when a HA environment request a token for the combination application, scope, binding, user, the binding value will be different and it will issue a new different token for each request without revoking existing token.

We can enable this feature by adding following config to the deployment.toml file.
[oauth.jwt.renew_token_without_revoking_existing]
enable = true

We need to document this new feature improvement in IS docs
Products: IS-5.11.0, IS-6.0.0, IS-6.1.0

5.11.0 update level <IP_ADDRESS>

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Proxy

https://github.com/WhatsApp/proxy/pull/293#issue-2469210729

Thanks all cack indonesian whatshapp
Pada Rab, 28 Agu 2024 20.47, Juanbel0 @.***> menulis:

@.**** approved this pull request.
—
Reply to this email directly, view it on GitHub
https://github.com/WhatsApp/proxy/pull/293#pullrequestreview-2266328177,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A772VOI6ES3DLIHEAIYLZT3ZTXIHNAVCNFSM6AAAAABMTEPZ7WVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDENRWGMZDQMJXG4
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Ok


No decoration view

No decoration View means we cannot add any other subviews in the cell apart from the imageView..?
And please try to write comments in the class so that one trying to understand your code can do so!!

+1

@GABHISEKBUNTY were you able to find a workaround?


How to use lax.scan() with xs as a tuple?

Hi,
The documentation for lax.scan() suggests that xs "can be an array or any pytree (nested Python tuple/list/dict) thereof with consistent leading axis sizes." Does this imply that I can provide multiple arrays for scan() to operate over, by calling with something like:
lax.scan(f, init, (xs, ys))

When I try this, I get a TypeError: 'JaxTuple' object does not support indexing.
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-09d2a81e4d1d> in <module>
----> 1 lyr_force(input_t, target_t, bias, tau, np.zeros(20))

~/Development/Projects/LearningToLearn_GIT/fullforce.py in lyr_force(input_t, force_t, bias, tau, x0, H, dt)
     96         return x, H(x, bias)
     97
---> 98     _, res_output_t = scan(net_step, x0, (input_t, force_t))
     99     return res_output_t
    100

~/Development/jax/jax/lax/lax_control_flow.py in scan(f, init, xs)
    555   out_aval = core.AbstractTuple((carry_aval, y_aval))
    556   jaxpr = core.TypedJaxpr(lifted_jaxpr, (), in_avals, out_aval)
--> 557   length = _leading_dim_size(xs)
    558   out = scan_p.bind(core.pack(consts), init, xs,
    559                     forward=True, length=length, jaxpr=jaxpr)

~/Development/jax/jax/lax/lax_control_flow.py in _leading_dim_size(xs)
    456 def _leading_dim_size(xs):
    457   if isinstance(xs, core.JaxTuple):
--> 458     return _leading_dim_size(xs[0])
    459   else:
    460     return xs.shape[0]

TypeError: 'JaxTuple' object does not support indexing

Could you please point me to an example of using lax.scan() over multiple arrays simultaneously?

Thanks for raising this!
You're using it as intended, but there was a bug that led to this behavior. It was just recently squashed.
What's your jax.version.__version__? It should be fixed on jax 0.1.34 and later, and also fixed on the master branch.

(I just bumped the version on pypi to 0.1.35 because there were some other scan bugs we squashed yesterday.)

This was on 0.1.31. Thanks for the bug fix!


Flashing screen

Running on Nokia 3 Android 8.0 the screen flashes on and off when the sensor is covered. Is there anything to try to do to fix it?

Is this happening on the latest version from the Play Store? Does it happen in both the default and AMOLED modes?

Latest version from F-Droid. The AMOLED modes black out the screen. Are there any logs that can be retrieved?

There's no explicit log calls from the application itself so it would probably be difficult to diagnose that way. I suppose the best I can hope for would be for someone with the same problem or same device configuration to build the debug version and step through the service with a debugger.

If you provide instructions then I can try, running Linux here so if you can direct me then I shall do it.

Great! 😃
If you can get the project building with Android Studio then we would want to debug the app with the service active. If I am understanding correctly, only the default proximity sensor wakelock mode is having the issue?
https://github.com/ssaqua/ProximityService/blob/74997e1ae16500189272169a40c38c285718e0f0/app/src/main/kotlin/ss/proximityservice/ProximityService.kt#L265-L270
We would want to set breakpoints on the wakeLock.acquire() and wakeLock.release() lines and just see if there is anything strange happening like the wakeLock being released while the proximity sensor is still covered. It may also be helpful to set breakpoints on the first lines of the onNear() and onFar() functions for the same reasons.

There might be a potential solution here: https://dontkillmyapp.com/nokia
Closing this issue, please re-open or comment if there's anything new.

I had discovered that and have all evenwell apps (user and system) disabled, but that does not fix the problem unfortunately.


[SPARK-13852][YARN]handle the InterruptedException caused by YARN HA switch

when sc stops, it will interrupt thread using to monitor app status.
the thread will throw an InterruptedException if YARN is switch as there is a sleep method in retry logic.
If YARN is switch between active and standby, sc.stop will return YarnApplicationState.FAILED as the InterruptedException is not caught.

Test build #53060 has started for PR 11692 at commit 9d6de23.

Test build #53060 has finished for PR 11692 at commit 9d6de23.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53060/
Test PASSed.

@srowen thanks for your comments. I've changed it, please check.

Test build #53072 has started for PR 11692 at commit 27203d8.

Test build #53072 has finished for PR 11692 at commit 27203d8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53072/
Test PASSed.

Merged build finished. Test PASSed.

OK by me

@vanzin @jerryshao  does that sound right?

@WangTaoTheTonic  Can you please clarify exactly what is going on here?
Does this happen in client or cluster mode or both?
You are saying if YARN RM fails over from active to standby then our client logic can no longer connect to RM and gets an interrupted Exception?  Who is interrupting the monitor thread?    If its the spark context then how do you know its success and not failure?
I'm not sure reporting success is the right thing to do here if we don't know the real status that is why I want to understand exactly what is going on.

Also switch sleep are you referring to because the place you put the try/catch isn't around the        Thread.sleep(interval)  in monitorApplication, its only around getApplicationReport

hi @tgravescs , it happened when sc stop normally in client mode. As sc.stop will stop dagscheduler -> stop taskscheduler -> stop scheduler backend -> interrupt the monitor thread, the monitor thread will enter into a retry logic in which there is a sleep intervals(which is not the sleep here) waiting for RM's switching.
The sleep methods will throw an InterruptedException when it is interrupted, so we need to catch it because it will log the application failed as treated as NonFatal(e).

for another concern about the final application status returned, we don't need too much worry as it is barely used by the codes who invoke this.

So you are saying that if spark context in yarn client mode is cleanly exiting while the RM is switching to the standby node, the call to getApplicationReport in Yarn can internally retry and sleep, since sc.stop() was called it ends up calling the scheduler backend stop interrupting the monitoring thread.
The MonitorThread has a catch for interruptedException and should be printing an info message " Interrupting monitor thread", you are seeing this?
Interrupted Exception is not a NonFatal error so it shouldn't be catching it:
From scaladoc:
Extractor of non-fatal Throwables. Will not match fatal errors like VirtualMachineError (for example, OutOfMemoryError and StackOverflowError, subclasses of VirtualMachineError), ThreadDeath, LinkageError, InterruptedException, ControlThrowable.

Also is this just what is being printed by the client or is the YARN final status (in RM) actually failed?

I've only observed exception caused by InterruptedException but not itself directly, thought it should be wrapped internally. The status in RM is ok as it is decided by ApplicationMaster not spark client.
In my recall i didn't see the message "Interrupting monitor thread" but not 100% sure. I will try to reproduce it and confirm.

@WangTaoTheTonic , would you please elaborate specific problem you met when InterruptedException is thrown?
From my understanding, it will only throw some exceptions mentioned that this application is failed, is that right?

@tgravescs I reproduce it and the error message like:

16/03/16 10:29:33 INFO YarnClientSchedulerBackend: Shutting down all executors
16/03/16 10:29:33 ERROR Client: Failed to contact YARN for application application_1457924833801_0003.
java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy10.getApplicationReport(Unknown Source)
at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getApplicationReport(YarnClientImpl.java:431)
at org.apache.spark.deploy.yarn.Client.getApplicationReport(Client.scala:221)
at org.apache.spark.deploy.yarn.Client.monitorApplication(Client.scala:882)
at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend$MonitorThread.run(YarnClientSchedulerBackend.scala:144)
Caused by: java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:155)
... 5 more
16/03/16 10:29:33 INFO YarnClientSchedulerBackend: Asking each executor to shut down
16/03/16 10:29:33 ERROR YarnClientSchedulerBackend: Yarn application has already exited with state FAILED!
16/03/16 10:29:33 INFO SparkContext: SparkContext already stopped.
16/03/16 10:29:33 INFO YarnClientSchedulerBackend: Stopped

There's no "Interrupting monitor thread" and the exception is UndeclaredThrowableException caused by InterruptedException.

@jerryshao yes, the problem is that client side's log will throw exception and show app is failed. more details are in log i pasted.

But from my understanding, this exception does no harm to your application, since your application is about to finish itself, also this may happen occasionally.
Also does it relate to RM HA, from my understanding, this InterruptedException will be thrown in any case where the code run into sleep, no matter in HA or not. Is there any special thing for RM HA?

the application is finished successfully(RM UI also show success state) but log shows it failed, that's the problem i think.
yeah you're right sleep method can throw InterruptedException. this pr is trying to fix the problem we find in RM HA switching.
what i am trying to say is that interrupting a monitor thread should not print the failed message in log.

I see your point, so the real issue should only be the log issue.
But marking the state as FINISHED with SUCCEED should be open to question, since here we don't know the real state of application, if it is finished with failure, say job is aborted due to continuous stage failure, should we still mark this as SUCCEED? So here I'm conservative to this change, because:

This issue is happened rarely and does no harm to the result of application.
We cannot get the real exit state of application, so marking as SUCCEED should be open to question.

monitorApplication returns 2-tuple, in which FinalApplicationStatus is not used when sc stops normally. that's why it doesn't matter what status we set it to.

So inside of hadoop in the getApplicationReport call, it was in RetryInvocationHandler which was doing a sleep and got an interrupted exception.  That ended up throwing a java.lang.reflect.UndeclaredThrowableException up to monitorApplication which is why it was handled by the NonFatal catch.
I need to look at it a bit closer.

so, how about it guys?

I have had time to look further to see what we should be doing, but as I read the exception you listed above,  the fix you are proposing here won't work.  Its not getting an InterruptedException back to the monitorApplication routine, its getting an UndeclaredThrowableException.

have you tried to reproduce the scenaro and see what happend? The UndeclaredThrowableException will be caught by e.getCause.isInstanceOf[InterruptedException], i think.

sorry I didn't read the diff close enough thought you were just catching that type.  I don't have an RM HA setup to quickly test it.
I think there are cases this can return the wrong thing still (success when failure). Like if you control-c out of spark-shell during this same time period.  Yes it probably doesn't make much difference, but at the same time I don't see it printing the exception to the log on shutdown as that big of a deal either.
Is this actually causing you an issue or just annoying in the log?

It will not impact on actuall result, but a error stacktrace and log showing failure will make user confused and easy to believe that the application is failed and needed to be submitted again.
Like I said above, we return 2-tuple in which the last one (FinalApplicationStatus ) is not used, the first one said application is finished so it is ok.

Wow this is old. @WangTaoTheTonic I'm not sure this is the right fix. The code in YarnClientSchedulerBackend is already catching InterruptedException:
      try {
        val (state, _) = client.monitorApplication(appId.get, logApplicationReport = false)
        logError(s"Yarn application has already exited with state $state!")
        allowInterrupt = false
        sc.stop()
      } catch {
        case e: InterruptedException => logInfo("Interrupting monitor thread")
      }

It seems it just needs to be tweaked to also handled the UndeclaredThrowableException case.

Hi @vanzin, would this be then a soft-suggestion for closing this if there is no objection for about a week?

Either close or make the right fix. As it is, the PR is not doing the right thing.

Let me try to propose to close this after a week if the author seems not active on this.


chore (RingTheory.Congruence): make RingCon extend AddCon and Con

Currently RingCon extends Add and Mul. This changes it to extend AddCon and Con directly.

!bench

That doesn't look like good news to me

Total instructions over the whole library are essentially flat.
The red one is small.
But you are correct in that it doesn’t represent a performance improvement (yet).
I think the main argument is stylistic consistency.

I am skeptical of most improvements beyond instructions broadly but this does seem better.


Get rid of specialization

~The last step for #5 is getting rid of specialization. I've remove most uses in 7c0379b1 and subsequent commits, except for the following ~17~ ~10~ ~9~ ~7~ 4:~ https://github.com/rust-lang/rust/pull/64564 changed the rules so there are again many default fn.
src/class/descr.rs
76:    default fn tp_descr_get() -> Option<ffi::descrgetfunc> {
97:    default fn tp_descr_set() -> Option<ffi::descrsetfunc> {
130:    default fn tp_as_descr(_type_object: &mut ffi::PyTypeObject) {}

src/class/buffer.rs
46:    default fn tp_as_buffer() -> Option<ffi::PyBufferProcs> {
74:    default fn cb_bf_getbuffer() -> Option<ffi::getbufferproc> {
110:    default fn cb_bf_releasebuffer() -> Option<ffi::releasebufferproc> {

src/class/gc.rs
27:    default fn update_type_object(_type_object: &mut ffi::PyTypeObject) {}
72:    default fn tp_traverse() -> Option<ffi::traverseproc> {
124:    default fn tp_clear() -> Option<ffi::inquiry> {

src/class/iter.rs
49:    default fn tp_as_iter(_typeob: &mut ffi::PyTypeObject) {}
71:    default fn tp_iter() -> Option<ffi::getiterfunc> {
94:    default fn tp_iternext() -> Option<ffi::iternextfunc> {

src/class/basic.rs
152:    default fn tp_as_object(_type_object: &mut ffi::PyTypeObject) {}
153:    default fn nb_bool_fn() -> Option<ffi::inquiry> {
183:    default fn tp_getattro() -> Option<ffi::binaryfunc> {
252:        default fn set_attr() -> Option<ffi::setattrofunc> {
274:        default fn del_attr() -> Option<ffi::setattrofunc> {
296:        default fn set_del_attr() -> Option<ffi::setattrofunc> {
324:    default fn tp_str() -> Option<ffi::unaryfunc> {
344:    default fn tp_repr() -> Option<ffi::unaryfunc> {
364:    default fn tp_hash() -> Option<ffi::hashfunc> {
389:    default fn nb_bool() -> Option<ffi::inquiry> {
409:    default fn tp_richcompare() -> Option<ffi::richcmpfunc> {

src/class/mapping.rs
83:    default fn tp_as_mapping() -> Option<ffi::PyMappingMethods> {
116:    default fn mp_length() -> Option<ffi::lenfunc> {
139:    default fn mp_subscript() -> Option<ffi::binaryfunc> {
162:    default fn mp_ass_subscript() -> Option<ffi::objobjargproc> {
187:    default fn mp_del_subscript() -> Option<ffi::objobjargproc> {
201:    default fn det_set_dispatch() -> Option<ffi::objobjargproc> {

src/class/pyasync.rs
94:    default fn tp_as_async() -> Option<ffi::PyAsyncMethods> {
121:    default fn am_await() -> Option<ffi::unaryfunc> {
144:    default fn am_aiter() -> Option<ffi::unaryfunc> {
167:    default fn am_anext() -> Option<ffi::unaryfunc> {

src/conversion.rs
107:    default fn with_borrowed_ptr<F, R>(&self, py: Python, f: F) -> R

src/class/sequence.rs
135:    default fn tp_as_sequence() -> Option<ffi::PySequenceMethods> {
168:    default fn sq_length() -> Option<ffi::lenfunc> {
190:    default fn sq_item() -> Option<ffi::ssizeargfunc> {
239:        default fn set_item() -> Option<ffi::ssizeobjargproc> {
285:        default fn del_item() -> Option<ffi::ssizeobjargproc> {
328:        default fn del_set_item() -> Option<ffi::ssizeobjargproc> {
372:    default fn sq_contains() -> Option<ffi::objobjproc> {
394:    default fn sq_concat() -> Option<ffi::binaryfunc> {
416:    default fn sq_repeat() -> Option<ffi::ssizeargfunc> {
438:    default fn sq_inplace_concat() -> Option<ffi::binaryfunc> {
465:    default fn sq_inplace_repeat() -> Option<ffi::ssizeargfunc> {

src/class/number.rs
625:    default fn tp_as_number() -> Option<ffi::PyNumberMethods> {
692:    default fn nb_add() -> Option<ffi::binaryfunc> {
714:    default fn nb_subtract() -> Option<ffi::binaryfunc> {
736:    default fn nb_multiply() -> Option<ffi::binaryfunc> {
758:    default fn nb_matrix_multiply() -> Option<ffi::binaryfunc> {
780:    default fn nb_true_divide() -> Option<ffi::binaryfunc> {
802:    default fn nb_floor_divide() -> Option<ffi::binaryfunc> {
824:    default fn nb_remainder() -> Option<ffi::binaryfunc> {
846:    default fn nb_divmod() -> Option<ffi::binaryfunc> {
868:    default fn nb_power() -> Option<ffi::ternaryfunc> {
890:    default fn nb_lshift() -> Option<ffi::binaryfunc> {
912:    default fn nb_rshift() -> Option<ffi::binaryfunc> {
934:    default fn nb_and() -> Option<ffi::binaryfunc> {
956:    default fn nb_xor() -> Option<ffi::binaryfunc> {
978:    default fn nb_or() -> Option<ffi::binaryfunc> {
1000:    default fn nb_inplace_add() -> Option<ffi::binaryfunc> {
1022:    default fn nb_inplace_subtract() -> Option<ffi::binaryfunc> {
1044:    default fn nb_inplace_multiply() -> Option<ffi::binaryfunc> {
1066:    default fn nb_inplace_matrix_multiply() -> Option<ffi::binaryfunc> {
1088:    default fn nb_inplace_true_divide() -> Option<ffi::binaryfunc> {
1110:    default fn nb_inplace_floor_divide() -> Option<ffi::binaryfunc> {
1132:    default fn nb_inplace_remainder() -> Option<ffi::binaryfunc> {
1154:    default fn nb_inplace_power() -> Option<ffi::ternaryfunc> {
1176:    default fn nb_inplace_lshift() -> Option<ffi::binaryfunc> {
1198:    default fn nb_inplace_rshift() -> Option<ffi::binaryfunc> {
1220:    default fn nb_inplace_and() -> Option<ffi::binaryfunc> {
1242:    default fn nb_inplace_xor() -> Option<ffi::binaryfunc> {
1264:    default fn nb_inplace_or() -> Option<ffi::binaryfunc> {
1287:    default fn nb_add_fallback() -> Option<ffi::binaryfunc> {
1309:    default fn nb_sub_fallback() -> Option<ffi::binaryfunc> {
1331:    default fn nb_mul_fallback() -> Option<ffi::binaryfunc> {
1353:    default fn nb_matmul_fallback() -> Option<ffi::binaryfunc> {
1375:    default fn nb_truediv_fallback() -> Option<ffi::binaryfunc> {
1397:    default fn nb_floordiv_fallback() -> Option<ffi::binaryfunc> {
1419:    default fn nb_mod_fallback() -> Option<ffi::binaryfunc> {
1441:    default fn nb_divmod_fallback() -> Option<ffi::binaryfunc> {
1463:    default fn nb_pow_fallback() -> Option<ffi::ternaryfunc> {
1485:    default fn nb_lshift_fallback() -> Option<ffi::binaryfunc> {
1507:    default fn nb_rshift_fallback() -> Option<ffi::binaryfunc> {
1529:    default fn nb_and_fallback() -> Option<ffi::binaryfunc> {
1551:    default fn nb_xor_fallback() -> Option<ffi::binaryfunc> {
1573:    default fn nb_or_fallback() -> Option<ffi::binaryfunc> {
1595:    default fn nb_negative() -> Option<ffi::unaryfunc> {
1618:    default fn nb_positive() -> Option<ffi::unaryfunc> {
1640:    default fn nb_absolute() -> Option<ffi::unaryfunc> {
1662:    default fn nb_invert() -> Option<ffi::unaryfunc> {
1684:    default fn nb_int() -> Option<ffi::unaryfunc> {
1706:    default fn nb_float() -> Option<ffi::unaryfunc> {
1728:    default fn nb_index() -> Option<ffi::unaryfunc> {

src/instance.rs
387:    default fn to_managed_py_ref<'p>(&self, py: Python<'p>) -> ManagedPyRef<'p, Self> {
396:    default fn drop_impl(borrowed: &mut ManagedPyRef<Self>) {

src/types/sequence.rs
268:                default fn extract(obj: &'a PyAny) -> PyResult<Self> {
307:    default fn extract(obj: &'a PyAny) -> PyResult<Self> {

~Once those are removed, we pyo3 can be stable on rust 1.30! I'm happy to take pull requests tackling one or more of those default fn uses~

The codegen in pyo3-derive-backend is a problem once the above functions are removed. For example:
https://github.com/PyO3/pyo3/blob/5100676497f2f97fac44acd8e3f0d41c755273ff/pyo3-derive-backend/src/py_class.rs#L329-L336
https://github.com/PyO3/pyo3/blob/264f642f19005a7a65640d95e48ced1b958edf59/pyo3-derive-backend/src/py_impl.rs#L33-L40
I am not familiar enough with it to have a solution.

I've also already ran against that and haven't found a good solution either :/
I think the most viable way forward would be not to have a default implementation of PyMethodsProtocolImpl, but instead have #[pymethods] implement it. For cases where there is no pymethods block, one could use #[pyclass(methods = false)] which then implements py_methods().

Is this still an issue given #332 being merged? I'm interested in helping push this to stable, and would appreciate advice for where to start.

@bspeice
I think now  the problem is around #426.
I'm looking for a better design but have not come up with it.

@bspeice https://github.com/PyO3/pyo3/issues/210#issuecomment-437684539 is fixed by #332, while some specialization are still there.
The main question is how we can rewrite the following so we don't have a conflicting implementation without specialization anymore:
pub trait FooImpl {
    fn foo() -> Option<ffi::Foo> {
        None
    }
}

impl<T> FooImpl for T {}

impl<'p, T> FooImpl for T
where
    T: Bar<'p>,
{
    fn foo() -> Option<ffi::PySequenceMethods> {
        return bar();
    }
}

(You can see a lot of those in the error message by removing #![feature(specialization)] from lib.rs)

As a contributor to Mercurial where we started using Rust a few months ago, I can say that we very much look forward to using PyO3, and the progress towards a build on stable is really encouraging.
However I wonder if that last issue is fixable at all, considering that it is exactly the use case for specialization.

It is technically fixable. We mainly have two options:
Inventory
Instead of implementing traits, the protocol implementations are submit to an inventory and added collected at class initialization time. The disadvantage here is that this essentially a well established linker hack, which can cause some weird interactions with the c api (e.g. https://github.com/PyO3/pyo3/issues/341).
Inventory is already used for pymethods blocks (so that we can have 0 to n pymethods blocks).
Manual annotations
You could force explicit annotations on pyclass blocks about which protocols will be implemented. The pyclass block would add default implementations for all protocol you didn't specify. If we wanted to completely get rid of inventory, we'd also need to forbid multiple pymethod block (or add an awkward syntax for numbering them) and annotate whether a struct has one or zero pymethod blocks.

I feel like #2 is not a particularly hard burden, especially with the outlook that it will get removed in some future.
For me a few more manual steps are certainly worth it for going to stable.

Is there anything I can do to help? I'm not sure how much time I can set aside, but compiling on stable would be a huge deal to the Mercurial project.

Not an expert on this, but as far as I understood https://github.com/PyO3/pyo3/pull/552 is a WIP PR to address part of this that could use some help.

@Alphare There are mainly two categories of traits that still use specialization: The first are already protocol implementation traits, which are used to implement the dunder methods. If I haven't made a grave mistake, extending #552 to all Py*Protocol traits should give us protocols without specialization.
The other category are the conversion traits, specifically FromPyObject. This requires a lot of trait refactoring. I think the way to tackle this is trying to move everything over to PyTryInto and IntoPy. E.g. in #583 I've replace IntoPyObject with IntoPy.
A way to get a (very verbose) todo list is to remove the #![feature(specialization)] from src/lib.rs and run cargo check.

@dtolnay to the rescue?  https://www.reddit.com/r/rust/comments/dm371q/towards_overflower_10/f4wjajf

@dtolnay to the rescue https://github.com/dtolnay/case-studies/blob/master/autoref-specialization/README.md :D
https://www.reddit.com/r/rust/comments/dmivsc/case_study_technique_for_doing_specialization_on/

I'm having a play with that technique to see if I can get it working for an Add method as an initial example. If someone else beats me to it I would not complain at all! Once Pyo3 is stable I expect serious uptick in usage.

rg "default .*fn" seems to have ~60 hits now not 18, we're further away then when the issue was raised :-( .
#552 doesn't compile - did it ever or was that the work in progress bit?
Most of the specialised functions are static taking no args - all of @dtolnay 's excellent specialisation examples dependend upon selecting the specialisation based upon the type of the argument.
So that's the bad news out of the way. Anyone got any good news / suggestions?

rg "default .*fn" seems to have ~60 hits now not 18, we're further away then when the issue was raised :-( .

It's because of #614

#552 doesn't compile - did it ever or was that the work in progress bit?

It's also because of #614

Most of the specialised functions are static taking no args - all of @dtolnay 's excellent specialisation examples dependend upon selecting the specialisation based upon the type of the argument.

Yeah, we cannot use it.

So that's the bad news out of the way. Anyone got any good news / suggestions?
(Also is there a good chat room discord/irc/zullip to talk pyo3?)

We have a gitter channel.
And what I can say is...
We have 2 kinds of specialization usages.

For ObjectProtocol
For distinguishing Python native object and pyclass

I think the later one should be resolved by changing the object layout of pyclass.
And, for the former one, I still have not decided what kind of approach we should use.
@konstin's 'unified slots' approach is good but still needs default.
I'm thinking about specialized struct approaches but... now it's too difficult to explain shortly here.

And, for the former one, I still have not decided what kind of approach we should use.
@konstin's 'unified slots' approach is good but still needs default.

I'm pretty sure that the approach using inventory still works, but so far no one volunteered to work in that.

I’m happy doing the leg work. If you can guide me a bit as to what needs to
be done then will do my best. I really think python-rust integration is as
important as wasm-rust  integration but alas doesn’t get as much focus. All
pointers gratefully received!!
On Fri, 15 Nov 2019 at 14:50, konstin<EMAIL_ADDRESS>wrote:

And, for the former one, I still have not decided what kind of approach we
should use.
@konstin https://github.com/konstin's 'unified slots' approach is good
but still needs default.
I'm pretty sure that the approach using inventory still works, but so far
no one volunteered to work in that.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/PyO3/pyo3/issues/210?email_source=notifications&email_token=AAGEJCCN6EKZ7PQEU77HLZLQT2ZMFA5CNFSM4FRR7SY2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEFUWVY#issuecomment-554388311,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAGEJCGWK2N37KNPYULLJI3QT2ZMFANCNFSM4FRR7SYQ
.

I have some alternative solutions that I've been hacking away at here: https://github.com/wickerwaka/pyo3/tree/removing-specialization
I still have a about 30% of the tests stubbed out, but it is building and running with stable rust. The solution I settled on is having the proc_macros provide default implementations for the various protocol traits that the user has not implemented. For any missing implementations in #[pyproto] blocks that was relatively simple, but there wasn't an easy solution for entirely missing protocols because there was no point at which the proc_macros could know the user was finished implementing protocols. So the solution I settle on there is requiring a list of protocols in the #[pyclass] proc_macro like so:
#[pyclass(protocols=[PyNumberProtocol, PyObjectProtocol])]
Which instructs the pyclass proc_macro to provide default implementations for everything except PyNumberProtocol and PyObjectProtocol.
Generally it has been working pretty well. I am currently trying to work around issues with Sel, Del and SetDel implementations for the sequence and mapping protocols, since the current implementations rely on specialization to make a compile time decision over which methods to use.
Again, this is still very hacky and is just a proof of concept and learning experience for me, it is not in a mergable state.

Nice hacking! One thing I really like about your approach: being more dependent on proc macros (compile time) rather than being more dependent on inventory (runtime) seems like a simpler solution.

@wickerwaka
Looks great!
Thank you.

So the solution I settle on there is requiring a list of protocols in the #[pyclass] proc_macro like so:
#[pyclass(protocols=[PyNumberProtocol, PyObjectProtocol])]

Yeah, there's a trade-off between one-by-one macro-generated impl and specialization.
A possible approach is to make #[pyproto] fill a global variable, like konstin's approach
But..., personally I want to resolve pyclass layout problem first, rather than ObjectProtocol.
The underlying problem, I think, is our whole ObjectProtocol system is too complex.
Refactoring the#[pyclass] system, I believe, would make it easier to simplify ObjectProtocols.

"Refactoring the #[pyclass] system" - if we're going to do it, earlier breaking changes are better than later when the communtiy has built lots on top of it. Do you have a branch that you've hacked some ideas on or a plan as to how you'd like it to look?

@gilescope
Yeah.
https://github.com/kngwyu/pyo3/blob/pyclass-new-layout/src/pyclass.rs
But it's highly WIP and I don't recommend anyone to look at :roll_eyes:

@kngwyu
I've been wondering about helping out with pyo3 for a while. I've got a fair chunk of experience using (and debugging) the C++ equivalent pybind11.
Perhaps I can help you design a solution for the pyclass layout? You'll have to give me some background of the issues with the existing layout, and then I would be happy to give my input as to how pybind11 solves similar problems.

@davidhewitt
Thank you.
I'll cc you when I file an issue or PR for related topics.

I'm not familiar with the problem so just passing information along, would this be of any use? https://github.com/dtolnay/case-studies/blob/master/autoref-specialization/README.md
Found it here: https://github.com/rust-lang/rust/issues/31844#issuecomment-557918823

I'm not familiar with the problem so just passing information along, would this be of any use? https://github.com/dtolnay/case-studies/blob/master/autoref-specialization/README.md
Found it here: rust-lang/rust#31844 (comment)

alas, no
https://github.com/PyO3/pyo3/issues/210#issuecomment-554231313

Most of the specialised functions are static taking no args - all of @dtolnay 's excellent specialisation examples dependend upon selecting the specialisation based upon the type of the argument.

Most of the specialised functions are static taking no args - all of @dtolnay 's excellent specialisation examples dependend upon selecting the specialisation based upon the type of the argument.

FWIW I don't see the presence or absence of arguments as being relevant to the technique. You can always turn a no-arg function into a method by dispatching it on PhantomData::<T>.
For example instead of:
trait Trait {
    fn f();
}

impl<T: Display> Trait for T {
    default fn f() {...}
}

impl Trait for String {
    fn f() {...}
}

you would dispatch it as:
impl<T: Display> DisplayKind for &PhantomData<T> {}
impl StringKind for PhantomData<String> {}

Just an experimental note:
I'm considering the use of 'provider pattern' which uses switch implementation by generics, instead of trait specialization: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=8131e41b3c816bb317391c252f831ab0
In this pattern, we prepare 2 kinds of structs, which corresponds to default implementation and real protocol implementation.
trait LenProvider: Sized {
    fn len() -> Option<Box<dyn Fn(PyObject) -> usize>> {
        None
    }
}

struct DummyProvider;
impl LenProvider for DummyProvider {}

struct LenProviderImpl<T: LenImpl>(PhantomData<T>);
impl<T: 'static + LenImpl> LenProvider for LenProviderImpl<T> {
    fn len() -> Option<Box<dyn Fn(PyObject) -> usize>> {
        fn wrap<T: LenImpl>(obj: PyObject) -> usize {
            let obj = T::from_obj(obj);
            obj.__len__().into().unwrap()
        }
        Some(Box::new(wrap::<T>))
    }
}

And then we can select which implementation to use in proc-macro code.
If a user implement __len__, we pass LenProviderImpl<T> to the initialization function.
If he doesn't, we use DummyProvider.

I’m probably naive here, but why can’t we have a trait with all the dunder
methods on it with default implementations of unimplemented! and the we
just override methods we want to impl? Is it because you might be able to
write blanket impls more easily? Or is it that all the impl would have to
be in one place? I’m just trying to understand why the easy route is ruled
out.
On Mon, 9 Dec 2019 at 06:50, Yuji Kanagawa<EMAIL_ADDRESS>wrote:

Just an experimental note:
I'm considering the use of 'provider pattern' which uses switch
implementation by generics, instead of trait specialization:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=8131e41b3c816bb317391c252f831ab0
In this pattern, we prepare 2 kinds of structs, which corresponds to
default implementation and real protocol implementation.
trait LenProvider: Sized {
fn len() -> Option<Box<dyn Fn(PyObject) -> usize>> {
None
}
}
struct DummyProvider;impl LenProvider for DummyProvider {}
struct LenProviderImpl<T: LenImpl>(PhantomData);impl<T: 'static + LenImpl> LenProvider for LenProviderImpl {
fn len() -> Option<Box<dyn Fn(PyObject) -> usize>> {
fn wrap<T: LenImpl>(obj: PyObject) -> usize {
let obj = T::from_obj(obj);
obj.len().into().unwrap()
}
Some(Box::new(wrap::))
}
}
And then we can select which implementation to use in proc-macro code.
If a user implement len, we pass LenProviderImpl to the
initialization function.
If he doesn't, we use DummyProvider.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PyO3/pyo3/issues/210?email_source=notifications&email_token=AAGEJCC2RRTOBX246MLWF3DQXXTEDA5CNFSM4FRR7SY2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGIBIMQ#issuecomment-563090482,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAGEJCABJBZ7V5E6RMH3PBDQXXTEDANCNFSM4FRR7SYQ
.

@gilescope
I'm not sure what you are talking about, but if you are suspicious about specific protocol traits(e.g., LenImpl in my example or PyMappingLenProtocol), they exist so that users do not need to give associated type definitions for methods they don't need.
We can skip implementaiton of traits by unimplemented!, but can't do so for associated types.

I’m probably naive here, but why can’t we have a trait with all the dunder methods on it with default implementations of unimplemented! and the we just override methods we want to impl? Is it because you might be able to write blanket impls more easily? Or is it that all the impl would have to be in one place? I’m just trying to understand why the easy route is ruled out.

Those protocols aren't just normal methods, but use different "tp slots" (s. https://docs.python.org/3/c-api/typeobj.html), and each protocol trait is for one of those slots. The python interpreter also will behave differently for methods that aren't there than for those that are implemented as error (and especially unimplemented!() will crash the whole application instead of raising an exception).

One thing to consider is moving to min_specialization instead of specialization: See https://github.com/rust-lang/rust/issues/31844#issuecomment-617013949

+1 on considering min_specialization. I have taken a brief look at it and I think it'll still require reworking a fair amount of existing uses of specialization, but at least then we'll be on a sound subset of specialization which might be stabilized sooner rather than later.

For now, I think we can remove specialization by using inventory for what is called PyMethod in our proc-macro codes. I'll create a patch for this if I can work for it this weekend.

I would love it if you have the chance to make that PR! That's easily the biggest use of it and then it should be relatively easy for us to get to stable 👍
Don't worry about my WIP branches breaking with changes you make - I will rebase them after 😂

STATUS: Now I'm experimenting with a new idea to re-implement object protocols without specialization.
At last, my 'provider pattern' failed since it's really difficult to pass marker types in proc-macro code.
Now I'm implementing inventory-like, but task-specific and efficient storage to store FFI functions.
Though recently I have lots of other tasks and cannot complete the implementation immediately, I'll open a PR till June 5th or so.

@kngwyu have you got the provider pattern branch anywhere? I'd be curious to read the code later and see if I could offer any ideas

@davidhewitt

have you got the provider pattern branch anywhere?

No.

Alas, no worries. Many thanks for your continued work on this! (And of course take your time - we have waited a long time already, can wait a few more months :smile: )

:tada:


Add a License

Hi there Max - would you be able to add a license to this software so it can be used elsewhere? Thanks.

https://choosealicense.com/

@harrisonlingren I did a PR
I hope @max-barry will merged the PR, so that other developers are able to use this.

Hi @harrisonlingren
A license is added.
@max-barry you could close this issue.

Nice, thanks! (PR #15 for reference)


can you pls add downloading?

Summary
can you add download button
movies : just a normal download button
tv shows: a download all button and a download one ep button
and quality selection of download
Please confirm the following

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.

Do not except to be soon, but it's something I also would like to implement