Built-in private registry?

I've been poring over past (closed) issues, and see no explicit mention of this, so thought it best to ask even if it's only to have it filed away for future reference:

Even given that running a registry can be considered an entirely different proposition from orchestrating containers, wouldn't it make sense for Rancher to have a built-in registry, even if there was no UI for it?

This would be an added value (i.e., deploy Rancher, have a complete solution) and might also do away with a fair amount of issues of various kinds regarding trusting private registries.

Docker registry runs with Docker very easily, I am not sure of the plus value, maybe with some hooks that for instance, could restart Rancher services when you push a new image.

"very easily" doesn't really go too well with registry 2.0. Setting up the required certificates and the backing store is mandatory if you intend to actually use instead of just playing around with it, so I think there's plenty of added value there.
It's not just docker run registry:2, it actually requires some sane defaults, plus each Rancher host has to be manually configured to trust a new registry.

I've just been setting up a registry and for me there would be a lot of value if Rancher could:

set up multiple redudant registries across physical hosts with a shared data store and load balancing
simplify addition of the certificate and key through the UI
automatically configure itself to consume images from the registry service

+1 @JamesBarwell

+1

Keep the +1's coming :).  If we get enough requests will do something along the lines of what @JamesBarwell suggested.
FYI @cloudnautique

+1

Yeah +1. I see this feature almost like resting on a distributed volume
concept being defaulted for rancher. Cos there still many apps that may
come in the future that will still make use of this concept
On Aug 27, 2015 1:21 PM, "Jan Guth"<EMAIL_ADDRESS>wrote:

+1
—
Reply to this email directly or view it on GitHub
https://github.com/rancher/rancher/issues/1738#issuecomment-135497667.

+1

+1

:+1:

+1

:+1:

tl;dr
Don't see built-in private registry as an option just because you have to setup storage. That means that you would need to have some configuration on admin panel before starting it.
But since I've read the issue #1272 a few minutes ago, it seems a better solution for this case.

Templates still need to ultimately deploy a docker image, which comes from a registry, and people don't necessarily want their image in public DockerHub...

@cusspvz I do not see quite the connection between docker registry and templates. A built in docker registry in rancher would mean to me

easy self managed (private) docker registry. (no hassle with certs etc.)
no need of external traffic (image pull) > faster.
integrated functionality like restarting of running containers if new images are available. (maybe easy configuration with container labels)
and all what JamesBarwell mentioned.

@vincent99 yep, but registry image is public, which mean you could have a template that creates a private registry service on your hosts, allows to configure storage (to prevent issues with reboots later reboots) and its up and running as it was built-in. You could even launch multiple private registries with for different teams or stacks. :)

@fentas

easy self managed (private) docker registry. (no hassle with certs etc.)

That should be useful, but rancher allows you to add any registry right?

no need of external traffic (image pull) > faster.

My idea covers that also, well, as it runs as a normal service, you could also say in which hosts should the registry run.

integrated functionality like restarting of running containers if new images are available. (maybe easy configuration with container labels)

That should be triggered by the CD software, not by the registry. Example: tests workflow could vary between companies, if a company needs to push images so it could be tested by worker services, this would be a hassle. So, Continuous Development Software should be the one who triggers an upgrade (not difficult if you use rancher-composer to do a rolling upgrade after testing and pushing the image to the registry).

and all what JamesBarwell mentioned.
set up multiple redudant registries across physical hosts with a shared data store and load balancing

Allowable by the templating feature

simplify addition of the certificate and key through the UI
automatically configure itself to consume images from the registry service

A fork of registry could be created for rancher to do just that

+1 :D

+1!

+1

+1

+1

There is a nice project to manage private docker registry v2 token authentication named Portus. May be it can be integrated into rancher.

@gbisheimer I quickly scoped Protus and it has several configuration issues which prevent it running in production mode just from docker-compose.yml and one nasty bug when tagging images.
:+1:
To keep it simple I would suggest registry:2 behind nginx proxy with htpasswd read/write ACL and use SSL certs stored in Rancher.

+1

I agree with @mishak87. I tried standing Portus up on rancher, and it would require a few modifications in order to run properly w/o initialization by the user. It also locked up one of my hosts by filling up the HD. I'm not sure why though; I didn't do a postmortem.

@mishak87, before finding Portus I also found this blog that explains how to setup docker_auth to use the new docker registry v2 token-authentication. This is a nice feature that allows to have more fine-grained access control to the registry images.
The nice thing about Portus is that it uses token authentication and manages namespaces, teams and users, and allow to limit the access level (owner, collaborator, viewer) of different team members to the registry. Sure it's not usable in production right now, and also it may need to be ported from Rails before including it in Rancher, but it's a starting point.
Registry proxy cache is also a nice feature that should be included

+1

+1 here as well.
I've been trying to set up registry:2 with certs and a GUI and it is not so simple.  If Rancher had one supplied it would be much simpler, particularly if the UI were integrated and you could use the existing Environment permissions to grant access to maintain said registry.
Of course, once you get RancherOS + Rancher + Registry + Registry UI + ... you end up with Rancher becoming more of a full end-to-end PaaS solution than a Docker cluster (eg: OpenShift).  It might be a bit soon to go that far before Rancher itself has even reached 1.0.

+1

We definitely want to make setting up a registry easier.  UI and integrated authz is probably not something we would consider before 1.0.
I think the best route atm would be to create a catalog entry for registry:2.  @cloudnautique has been very busy building out elk, Jenkins, gluster, Hadoop, and others in the catalog.  We will probably get to registry but we would love some community contribution to speed up the process.  If anybody is interested in helping let us know.  I'd love to see a one click install of registry:2.

+1

+1

Here's our contribution! A scalable registry compose :)
Be sure to have a valid htpasswd on your btsync share. before starting it.
# Stack name: docker

btsync:
  labels:
    io.rancher.scheduler.affinity:container_label_ne: io.rancher.stack_service.name=docker/btsync
  command:
  - bootstrap
  - ${BTSYNC_SHARED_KEY}
  - data
  image: cusspvz/btsync:2.0.1
  privileged: true
  volumes:
  - ${EPHEMERAL_STORAGE}/docker/registry:/data
  stdin_open: true

registry-redirect:
  environment:
    REDIRECT: https://${DOMAIN}
  tty: true
  image: cusspvz/redirect:0.0.1
  stdin_open: true

redis:
  image: redis:latest

registry:
  environment:
    REGISTRY_AUTH: htpasswd
    REGISTRY_AUTH_HTPASSWD_PATH: /data/htpasswd
    REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
    REGISTRY_HTTP_HOST: ${DOMAIN}
    REGISTRY_HTTP_SECRET: ${HTTP_SECRET}
    REGISTRY_REDIS_ADDR: redis-server:6379
    REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data/storage
    REGISTRY_STORAGE_CACHE_BLOBDESCRIPTOR: redis
  image: registry:2
  links:
  - redis:redis-server
  - btsync:nfs-server
  entrypoint:
  - /bin/bash
  command:
  - -c
  - apt-get -y -qq update; apt-get install -y -qq nfs-common; mkdir -p /data; mount -t nfs4 nfs-server:/data /data; while [ ! -d "/data/.sync" ]; do echo "waiting for sync process to be completed"; sleep 2; done; exec registry /etc/docker/registry/config.yml

http-and-https:
  ports:
  - '443'
  - '80'
  links:
  - registry:registry
  - registry-redirect:registry-redirect
  labels:
    io.rancher.scheduler.global: 'true'
    io.rancher.loadbalancer.ssl.ports: '443'
    io.rancher.loadbalancer.target.registry-redirect: ${DOMAIN}:80=80
    io.rancher.loadbalancer.target.registry: ${DOMAIN}:443=5000
  tty: true
  image: rancher/load-balancer-service
  stdin_open: true

@cusspvz - that's interesting.  I've something similar that uses nginx, registry:2 and one of the frontends, and works with certificates; however the problem I have is that there seems to be no decent web frontend that supports a registry with authentication enabled.  Also, registry:2 does not seem to allow me to use LDAP authentication.
The trick to do the NFS mount on the registry container is handy, too.  I'd originally thought to do that using a sidekick.
The Holy Grail to me is a stack that has registry, web admin interface, certificates, LDAP authentication, and NFS mounting of the registry backend storage...

+1

+1

+1

The Holy Grail to me is a stack that has registry, web admin interface, certificates, LDAP authentication, and NFS mounting of the registry backend storage...

Agree with those, but I suppose as it comes built-in with rancher to authenticate against rancher's authentication. It should support then LDAP, GitHub, Local users and future Rancher's authentication methods.

The trick to do the NFS mount on the registry container is handy, too. I'd originally thought to do that using a sidekick.

Nice to hear that!

however the problem I have is that there seems to be no decent web frontend that supports a registry with authentication enabled.

There is one, Portus - made by Suse, think someone already talked about it over here. I've already tested it, but unfortunately at that time, it wasn't mature enough. Maybe things have changed.

I've tried installing Portus - it looks ike it could be very good, especially if wrapped up with a mysql container and registry into a stack with a load balancer to offload the certificate...  however I just cannot make it work with our AD LDAP.  If it ever decides to play nicely then I'll create a Catalog definition, but I suspect it needs a bit of work.  If anyone else has managed this then please share...

+1

+1

Here's what is now working for me.  I've wrapped up everything into a Catalogue definition which you can access from https://github.com/sshipway/rancher-catalogue.git  (just point your Rancher at this catalogue to get a couple of goodies including this)
This Catalogue item will give you a secure, SSL-protected and authenticated Registry, with a web UI for administration (Portus) which can link to LDAP for authentication.  It works on our setup (behind F5 load balancer, AD LDAP) but YMMV.  Please give me feedback on the Github Issues page at https://github.com/sshipway/rancher-catalogue/issues if you have problems.
Ideally, of course, this would be built-in to Rancher, share the Rancher authentication and certificates, allow deletions, etc. but this is the best I've managed to date.

+1

+1

+1

@cusspvz  @sshipway @ibuildthecloud
A word on auth: I feel hashicorp vault could come handy in here as well, if it is decided to leverage it's features, see suggestion here: https://github.com/rancher/rancher/issues/3267

A word on templates: I don't see one single reason, why a template is not the right place for this sort of first class citicens.

:+1:

+1

+1

+1

+1

+1
If one day I go to catalog and registry:2 is there by default I will cry tears of joy you have no idea.

If someone is building catalog this could be useful component for authentication https://github.com/cesanta/docker_auth

:+1:

@mishak87 - I built mine using Portus (which has built-in auth support for registry:2), however this only links to LDAP for auth.  The one you mention seems to have Google auth support as well as LDAP and static.  One problem with the Portus approach is that (if you use LDAP) then all users need to be in LDAP, so you can't define an additional static user creds for Rancher to use (its not so easy for me to add an LDAP identity because of the way our LDAP synchs to our main identiy service)
Also, Portus does not have an official Docker build as yet, so I had to do some tweaks and produce my own... it does, however, have a web UI which is very helpful for user management and defining goup/namespace access.  Catalogue at https://github.com/sshipway/rancher-catalogue.git if you want to see how I put ours together.

@sshipway I was thinking more like this: It is opensource project so I could (or better convince here someone else to) hack support for GitLab (it has oauth) to support my team. :smile:
I don't like Portus for several reasons one of them is that it is written in ruby and tries to do multiple jobs instead of one very well.

@mishak87 - I would far rather Rancher itself were able to support the registry auth, so as to use the single definitions... however with Portus at least we can link both to our LDAP for authentication and only authorization is separate.  Also, we have some valid use cases where we would want a registry to be managed by different people to the ones who have environment member rights, so that actually helps.

@sshipway It is hidden in examples but cesanta/docker_auth supports multiple backends via failover so you could in theory implement your usecase by providing more sources (custom, ops etc.). Still you would need some UI to manage that source.

+1 here and I'll see what we can do to help in contributing to this!

I'm implementing in-house auth service using GitLab as SSoT with mapping namespace/project 1:1 namespace/image. I might make an open-source version based on my experience building it.
So far I just realized how much opinionated and variable the registry setup can get. And for orgs with LDAP Portus is probably the fastest way to get started.
FYI GitLab is discussing adding private registry too.

+1

why bothering to integrate docker registry in rancher, when you can have it fully implemented using portus from rancher community catalog : https://github.com/rancher/community-catalog/tree/master/templates/registry
now that community catalog have a template for it,
i don't think it's still necessary to have a registry built-in.
that's my 2cents :)

@xinity because it's an absolutely massive image and for those that require just a lightweight solution to keep images private, docker-registry makes a lot of sense! :-D

The registry stack in community catalog is too heavy and all-encompassing for my requirements. There is still a gap for a simple lightweight registry:2 solution.
I agree there is still scope for this but I'm not convinced it needs to be built in. Perhaps an alternative registry catalog entry would suffice. If you use the letsencrypt stack and terminate the SSL with a load balancer its quite simple to set up a registry. The only missing piece is basic auth. However, I'm toying with the idea of leaving the registry internal so that it can only be accessed by VPNing into the rancher network.
For me the end goal would be to have a CI such as Drone pushing to the registry anyway, which would happen internally to the Rancher network.

Exactly my thoughts for a setup. Think it would work really nice and would
minimise security concerns.
On 22 March 2016 at 17:01, djskinner<EMAIL_ADDRESS>wrote:

The registry stack in community catalog is too heavy and all-encompassing
for my requirements. There is still a gap for a simple lightweight
registry:2 solution.
I agree there is still scope for this but I'm not convinced it needs to be
built in. Perhaps an alternative registry catalog entry would suffice. If
you use the letsencrypt stack and terminate the SSL with a load balancer
its quite simple to set up a registry. The only missing piece is basic
auth. However, I'm toying with the idea of leaving the registry internal so
that it can only be accessed by VPNing into the rancher network.
For me the end goal would be to have a CI such as Drone pushing to the
registry anyway, which would happen internally to the Rancher network.
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/rancher/rancher/issues/1738#issuecomment-199907945

--
Lewis Taylor
Managing Director
Danger Farms
<EMAIL_ADDRESS>+44 75522 31643
Office 1.05, 1st Floor, 20 Ropemaker Street, London, EC2Y 9AR

:+1:

+1

Just a thought, it might be nice if the Rancher API could be used as the token authentication server for a registry, fulfilling a similar role to https://github.com/cesanta/docker_auth

@djskinner - I agree with you on that, I think the subject came up once before somewhere else, too.  This would be a great solution as we could just use registry:2 but have unified authnz in Rancher.  It might need to have some more configuration in Rancher, though, as you'd want to be able to grant permissions to the registry on a namespace basis, independently of environment access

+1

👍

:+1:

:+1:  for the complete built-in solution.

+1

GitLab completed their roll-out of this feature, per the comments above. It has some room to grow but could be an option for a self-hosted solution with access control.

Yeah.. I'm using the GitLab registry and it works great!

We considered doing a private registry, but given that we had done a lot of work on Docker Hub previously, it was a non-starter due to having to update all our images to use the private registry URL, instead of just the repository namespace.
However we did go forward using a --registry-mirror container, in Rancher, which is working great to speed up our pulls. Not sure if doing something like that would fall under this issue, but I'd +1 a private registry catalog, and a +1 for a registry mirror catalog, or even a built-in registry mirror.

I would definitely like a stable registry mirror or registry catalog item, possibly with VMWare Harbor. I use Dockerhub when I can, but sometimes it's not an option.  Gitlab's container registry is nice, but it limits you to 1 image per repo and they don't have a date of when that'll be changed.

Just for your info.

I would definitely like a stable registry or registry mirror catalog item.

GitLab Registry works great for me (and it is nicely integrated into GitLab and GitLab CI).
For the Docker Hub registry mirroring NGINX with caching works perfectly and it is very lightweight, here is the NGINX config:
proxy_cache_path /tmp/dockerhub-nginx-cache levels=1:2 keys_zone=DOCKERHUB:10m inactive=72h max_size=10g;

server {
    server_name dockerhub-mirror.com;

    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_certificate_key /etc/letsencrypt/live/dockerhub-mirror.com/privkey.pem;
    ssl_certificate /etc/letsencrypt/live/dockerhub-mirror.com/fullchain.pem;
    listen 443 ssl;

    # DockerHub Caching Proxy
    location /v2/ {
        proxy_pass https://registry-1.docker.io;
        proxy_cache            DOCKERHUB;
        proxy_cache_valid      200  1d;
        proxy_cache_use_stale  error timeout invalid_header updating
                               http_500 http_502 http_503 http_504;
        proxy_intercept_errors on;
        proxy_cache_lock on;
        error_page 301 302 307 = @handle_redirect;
    }
    location @handle_redirect {
        set $saved_redirect_location '$upstream_http_location';
        proxy_pass $saved_redirect_location;
        proxy_cache            DOCKERHUB;
        proxy_cache_valid      200  1d;
        proxy_cache_use_stale  error timeout invalid_header updating
                               http_500 http_502 http_503 http_504;
        proxy_cache_lock on;
        proxy_cache_lock_age 600s;
        proxy_cache_lock_timeout 600s;
        proxy_cache_key $scheme$proxy_host$uri;
    }

}

Once the mirror is up and running, you should add --registry-mirror https://dockerhub-mirror.com to the Docker daemon on every server where you plan to use the mirror.

Gitlab's container registry is nice, but it limits you to 1 image per repo and they don't have a date of when that'll be changed.

I just (ab)use tags if I need to put several images, e.g. dockerhub.mygitlab.com/group/project:latest-service1.

We use this:
gitlab stack
GITLAB_URL=gitlab.company.com
DOCKER_GITLAB_URL=docker.company.com
GITLAB_REGISTRETION_TOKEN='' #get from you gitlab

version: '2'
services:
  server:
    image: gitlab/gitlab-ce:latest
    hostname: ${GITLAB_URL}
    environment:
      GITLAB_OMNIBUS_CONFIG: |-
        external_url 'https://${GITLAB_URL}';
        nginx['listen_port'] = 80;
        nginx['listen_https'] = false;
        nginx['proxy_set_headers'] = {
          "X-Forwarded-Proto" => "https",
          "X-Forwarded-Ssl" => "on"
        };
        gitlab_rails['gitlab_shell_ssh_port'] = 2222;
        gitlab_rails['smtp_enable'] = true;
        gitlab_rails['smtp_address'] = "smtp.yandex.ru";
        gitlab_rails['smtp_port'] = 587;
        gitlab_rails['smtp_user_name'] =<EMAIL_ADDRESS>        gitlab_rails['smtp_password'] = "passw0rd";
        gitlab_rails['smtp_domain'] = "www.yandex.ru";
        gitlab_rails['smtp_authentication'] = "login";
        gitlab_rails['smtp_enable_starttls_auto'] = true;
        gitlab_rails['smtp_openssl_verify_mode'] = 'peer';
        gitlab_rails['gitlab_email_from'] =<EMAIL_ADDRESS>        gitlab_rails['gitlab_email_reply_to'] =<EMAIL_ADDRESS>        # Settings used by Registry application;
        registry_external_url 'https://${DOCKER_GITLAB_URL}';
        registry['enable'] = true;
        gitlab_rails['registry_enabled'] = true;
        registry_nginx['listen_port'] = 80;
        registry_nginx['listen_https'] = false;
        registry_nginx['proxy_set_headers'] = {
          "X-Forwarded-Proto" => "https",
          "X-Forwarded-Ssl" => "on"
        };
        ];
    volumes:
    - /gitlab/config:/etc/gitlab
    - /gitlab/logs:/var/log/gitlab
    - /gitlab/data:/var/opt/gitlab
    labels:
      io.rancher.container.pull_image: always
  cleanup:
    image: quay.io/gitlab/gitlab-runner-docker-cleanup
    environment:
      DEFAULT_TTL: 10m
      USE_DF: '1'
      LOW_FREE_SPACE: 4G
      EXPECTED_FREE_SPACE: 8G
    stdin_open: true
    volumes:
    - /var/run/docker.sock:/var/run/docker.sock
    tty: true
    labels:
      io.rancher.container.pull_image: always
      io.rancher.scheduler.global: 'true'
  ci-node:
    privileged: true
    image: gitlab/gitlab-runner
    environment:
      CI_SERVER_URL: https://${GITLAB_URL}/ci
      REGISTER_NON_INTERACTIVE: 'true'
      REGISTRATION_TOKEN: ${GITLAB_REGISTRETION_TOKEN}
    stdin_open: true
    entrypoint:
    - sh
    - /config/entrypoint.sh
    volumes:
    - /var/run/docker.sock:/var/run/docker.sock:rw
    tty: true
    volumes_from:
    - ci-node-conf
    command:
    - --executor
    - docker
    - --docker-image
    - docker
    - --docker-privileged
    labels:
      io.rancher.sidekicks: ci-node-conf
      io.rancher.container.pull_image: always
      io.rancher.scheduler.affinity:container_label_ne: io.rancher.stack_service.name=gitlab/server
  ci-node-conf:
    image: negash/dockerize
    environment:
      CI_SERVER_URL: https://${GITLAB_URL}/ci
      DOCKERIZE_GET: negash.ru/s/gitlab/config.toml.tmpl;negash.ru/s/gitlab/entrypoint.sh
      REGISTRATION_TOKEN: ${GITLAB_REGISTRETION_TOKEN}
    stdin_open: true
    volumes:
    - /config
    - /etc/gitlab-runner
    - /usr/local/share/ca-certificates
    tty: true
    - /etc/gitlab-runner
    labels:
      io.rancher.container.start_once: 'true'
      io.rancher.container.pull_image: always
      io.rancher.scheduler.affinity:container_label_ne: io.rancher.stack_service.name=gitlab/server

+1

FWIW, we had to stop using our --registry-mirror setup because when we were bringing up a new hosts with many containers scheduled (30+, over about 15 services), pulling that many images simultaneously ended up saturating the network and caused other issues, in addition to making the host take forever/time out coming up.
Probably not as large an issue for smaller installations or infrastructures with lighter docker images (several of ours our 1GB+), but in our case, we're going to have to move to a high-network registry mirror that lives outside the rancher/overlay network ecosystem to avoid issues.

+1

would be probably the most awesome feature ever

Acceptance Criteria for Alpha
[ ] Have a HA cluster with local cluster enabled, add a PV using NFS to the local cluster. Enable Harbor using the PV option. - Only Admin
[ ] Enable Harbor, selecting the S3 option - Only Admin
[ ] Add images to Harbor - Cluster Owner/Cluster Member and Project Owner/Project Member
[ ] Add username and password to the global registry - Cluster Owner/Cluster Member and Project Owner/Project Member
[ ] Deploy workloads using the private images - Cluster Owner/Cluster Member and Project Owner/Project Member

verified on rancher:master

Rancher setup - HA with local cluster.
Enabled Harbor using

Storage Backend for Registry - S3
Config Database Type - external - Postgres
Config Redis Type - external - Redis

Able to enable Harbor Registry by Admin.
Other user, other than Global Admin - Cluster Owner, Cluster member and Project Owner - Able to create an account in Harbor UI, push images to the Harbor registry and deploy workload using these private images

verified on rancher: master

Enabled Harbor on the Rancher HA setup using self-signed certs
Enabled Harbor by using EBS storage class
as a project member, using Global Registry on EKS, AKS and imported GKE cluster

Known issue:

using NFS to for PV in the local cluster: https://github.com/rancher/rancher/issues/20681

verified on rancher: master

Enabled Harbor on the Rancher HA setup using Let's Encrypt

verified on rancher: master
enable Harbor using EBS as the Storage Class

Add images to Harbor - Cluster Owner/Cluster Member and Project Owner/Project Member
Add username and password to the global registry - Cluster Owner/Cluster Member and Project Owner/Project Member
Deploy workloads using the private images - Cluster Owner/Cluster Member and Project Owner/Project Member


DefaultActivityIndicator not showing on Dark Mode

The DefaultActivityIndicator not showing on Dark Mode.

Hi, thanks for the ticket! I tried to improve the Dark Mode handling in this commit 9a5900104a2c9a5bde32dabb35b07da30ee7bc0e, so it should now work correctly.

Hi, thanks for the ticket! I tried to improve the Dark Mode handling in this commit 9a5900104a2c9a5bde32dabb35b07da30ee7bc0e, so it should now work correctly.


SetOrchestrationServiceState CLI Request

Hi,
We have a new compute API that we want to release by March 31 2020.
So far, the swagger and SDK changes are not merged yet in public repo but its estimated to be merged by end of week or next week.
The swagger changes: https://github.com/Azure/azure-rest-api-specs/pull/8613
SDK changes: https://github.com/Azure/azure-sdk-for-net-pr/pull/967/commits
Powershell cmdlet design approved: https://github.com/Azure/azure-powershell-cmdlet-review-pr/issues/479
Could someone let me know if its possible to get this done by March 31 release? Let me know if more information is needed.
@yungezz

put it in S167

automatic instance repair related

the feature is released in latest 2.3.0 version, pls give it a try and let us know if any other issue. close the issue.


Fix R# Unit Tests error when shadow copying is enabled

Current coverage is 72.34%

Merging #305 into master will decrease coverage by -3.66% as of 25fa22b

@@            master    #305   diff @@
======================================
  Files           41      42     +1
  Stmts         4026    4230   +204
  Branches       349     367    +18
  Methods
======================================
  Hit           3060    3060
  Partial        146     146
- Missed         820    1024   +204

Review entire Coverage Diff as of 25fa22b

Powered by Codecov. Updated on successful CI builds.

Reference:
http://www.nunit.org/index.php?p=settingsDialog&r=2.6.3
Shadow Copy

https://github.com/Microsoft/SparkCLR/issues/304


UI testing

The architecture of the program is well-suited for unit testing (every UI state can be reached by an appropriate sequence of Actions, which is extremely easy to encode versus something like "click on this part, wait for it to load, activate this button"). In principle, it should be easy to write unit tests to ensure that it behaves nicely using this approach, and we should, but it's not something that we do currently.

@zrho knows more about this, anything to add?


feat: splash screen support

Follows @xlanex6 work (#4)
Resolves #3
This PR will be squashed and merged with the following commit description to keep credits on @xlanex6 :
Co-authored-by: Alex Duval<EMAIL_ADDRESS>

@xlanex6 You can check my changes, I think it's ready. I finally added back the splash.backgroundColor option, as well as targetDir (default: splash).
You can have a try and give your feedback :)
Also don't hesitate to ask anything regarding my changes.

@kevinmarrec
Funny that you back to my initial idea to make a root options splash with all options into.
You make it better way than I could. thank you

@xlanex6 Yeah I got confused with some of my thoughts at some point. You had it in the good direction mate 👍

@xlanex6 I don't think I'll make one, because overall it will land in https://github.com/nuxt-community/pwa-module as official module. I'm working at NuxtLabs so it will be just about time that things get more official. Therefore the documentation will be on the official module :)


One Agent Can monitor multiple pg instances?

We have multiple pg instance，if one agent can monitor multiple pg instances，it will be very practical！

@tripitakav, unfortunately no, pg-agent is designed to monitor a single Postgres instance.
We've followed the Prometheus guideline that recommends "each exporter to monitor exactly one instance of an application".

@tripitakav, unfortunately no, pg-agent is designed to monitor a single Postgres instance. We've followed the Prometheus guideline that recommends "each exporter to monitor exactly one instance of an application".

ok，thanks


Multiple buttons in Bitbucket

Hi,
Create a pull request, navigate between activity <-> commits
-> adds two new buttons each time

duplicate of #560
This is already fixed but not yet released.


Fixed issue with ViewControllers embedded in NavigationControllers

I found out firsthand that if you create a constraint based on a ViewController's view and that ViewController is embedded in a NavigationController then having the view's translatesAutoresizingMaskIntoConstraints set to false caused issues. I have provided screenshots so you can see what I am talking about. Removing this code seemed to fix everything based on my tests. All constraints should work properly once created.

You are right. Thanks for the fix. 💯  Please checkout TinyConstraints 2.1.1 💪

Thank you for accepting it! I love this framework, just trying to do what I can to make it better.


Electric Vehicle(kWh) and Non-Full Refuel support

Add support for KWH in place of consumption units(gallons, liters) and also the ability to defer calculation for fuel mileage until the next full fill up.

completed.


onSlidingStart to go along with onSlidingComplete

Describe the Feature
It would be useful to have a prop onSlidingStart that act like onSlidingComplete with a callback but for when the slider starts instead. This library has this feature: https://github.com/jeanregisser/react-native-slider
Possible Implementations
<Slider onSlidingStart={() => console.log("start")} />

I like the idea! Perhaps you want to send a PR with this functionality?

Fixed


docker-squash image after build

I recommend using docker-squash on the built image. I had great experience with the Python based docker-squash tool:
https://github.com/goldmann/docker-squash
In theory we are putting as many commands on one line as possible, but still a docker-squash into a single layer is good choice before release.
Later docker versions have integrated the squash tool, so if we are using such docker version we might be able to use the built-in one.

Right now, docker-squash compresses my image from 241 MB to 177 MB, which is quite a lot. Also, less layers = better Docker performance.


Back out "Fix jump-to-null issue for ObjC async calls."

The decision to revert a different bugfix as in https://github.com/apple/swift/pull/58397 to fix the less common jump-to-null issue was hasty. It turns out there are reasonable workarounds for the latter, but not the former.
This PR reverts https://github.com/apple/swift/pull/58397 in favor of fixing the SIL verifier so it catches the jump-to-null miscompilation. This lets us more easily identify the miscompile as we provide a proper fix.

@swift-ci please smoke test and merge


socks5 does not work well

Hi. When I'm using proxy = {'http': 'socks5://IP:PORT', 'https': 'socks5://IP:PORT'}, I got "AssertionError: Not supported proxy scheme socks5". I had upgraded my requests( version:2.11.1)  and env is mac osx.

proxies = {'http': 'socks5://<IP_ADDRESS>:9050',
                   'https': 'socks5://<IP_ADDRESS>:9050'}
requests.get(url, proxies=proxies)

it works for me. I suggest you debug to find error.

@PegasusWang This suggests that your PySocks isn't working well. pip install -U pysocks==1.5.6 should resolve your problem.


Application does not conform to Dark System Theme

Pull requests welcome, if anyone is inspired to help address this. Thanks!


Performance/Caller: Clearer message

I have the following line of code
caller[0][/`.*'/]

which RuboCop 0.49.1 flags with
caller.rb:5:11: C: Performance/Caller: Use caller(n..n) instead of caller[n].
    caller[0][/`.*'/]

Looking at the suggested solution I'd expect to change the code to
caller(0..0)[/`.*'/]

but that isn't feature equivalent because
caller[0] #=> "caller.rb:13:in `<main>'"
caller(0..0) #=> ["caller.rb:9:in `modified'"]

caller(0..0) not only returns an Array instead of a String, but even if we use the first item in the returned Array we get a different result. In short; the default message is confusing and potentially misleading (at least to me).
Expected behavior
I'd expect RuboCop to guide me to a correct solution, or not at all. The pipe dream is

Use caller(1..1).first instead of caller[0].

but that's a bit specific to my concrete case, I reckon. Perhaps we could settle for:

Use caller(n..n).first instead of caller[n].

but that might hide the changed behavior too much. More precisely:

Use caller((n + 1)..(n + 1)).first instead of caller[n].

but that's hardly concise or easily understandable. How about some form of

Use arguments with caller instead of returning the full array

¯\_(ツ)_/¯
There is some discussion ongoing on the now closed pull request as well: https://github.com/bbatsov/rubocop/pull/4078
Actual behavior
RuboCop suggest

Use caller(n..n) instead of caller[n].

Steps to reproduce the problem

Create a file containing caller[0]
Run RuboCop on it

RuboCop version
$ rubocop -v
0.49.1

Thanks for the reporting! 👍
I'll work it.

Isn't this rather the tail wagging the dog?
In any case, when is caller called in time-critical situations? This is like the time someone optimised an RPC call failure mode by microseconds, ignoring that the network call was going to take milliseconds anyway.

For those of you arriving via a google search as to what this cryptic message could possibly mean (and why this bizarre syntax is "preferable"), the answer seems to be along these lines:
# bad
caller[0] # extracts only one element after receiving an enormous array

# good
caller(0..0).first # extracts only one element after receiving a single-element array

I hope the array size is actually something to be concerned about here, because this is a very odd edit for a style checker to suggest.

# bad
caller[0] # extracts only one element after receiving an enormous array

Since when has passing a huge or tiny array involved anything more than passing a pointer to something, in any language whatsoever?

You're right, I'm going to update that comment


Windows 11 support

Description
There is no support for Windows 11
Neofetch version
7.1.0
Screenshot

Config file
default
Verbose log
none

neofetchlog.txt


Make pull request for the hash to curve implementation

We would like that our implementation of hash to curve gets integrated into original arkworks algebra repo.

https://github.com/arkworks-rs/algebra/issues/343 got merged.


Add Paul Li to Contributor List

Added my name to the github contribution list.

@illuap, I'm quite elated about your pull request. I wanna evolve this project to addresses various problems faced by first-time contributors. I'd love to learn about your journey in open source community, the problems, pain points you had etc.
Could you explain how you felt when you went through the tutorial, made a pull request and learned that I merged it?
We’ve recently added social share  to our web app. Could you please go to https://roshanjossey.github.io/first-contributions/#social-share and share your first contribution to open source?
Also, check out projects with easy issues while you’re there.


[Impeller] Enable RuntimeStage tests for the Vulkan backend.

The backend been implemented but tests disabled since the test harness only creates *.iplr files for the host backend instead of all enabled backends.
Related to https://github.com/flutter/flutter/issues/105538 for OpenGL ES. Filing a separate issue for more granular tracking.

E.g. https://github.com/flutter/engine/pull/42146 has it failing.

Ok, it was failing there because I actually broke it - however, in a local run it seems to pass, even though it's desginated as being skipped (it's not actually skipping)

Depends on https://github.com/flutter/flutter/issues/129659. Today, the packaged SL is not SPIRV.


Sprint 21 updates

taken from slide deck and items labeled “needs changelog”
[skip ci}

@Fryguy  once we have botnvinnik released how does the changelog change?  Do we start a new one?
@jrafanie @gmcculloug @blomquisg @dclarizio @abellotti @roliveri please review.

once we have botnvinnik released how does the changelog change? Do we start a new one?

That's a good question.  I think what we do is created a top level header named "Botvinnik" with the release date.  Then we can either 1) collapse all the sprint sections into one giant section or 2) just leave them, but remove the Unreleased label

From outside-in, probably best to collapse them, but divided into categories (UI, Providers,.....)

  I knew something didn't look right.
Sent from my iPhone

On Apr 2, 2015, at 5:04 PM, Jason Frey<EMAIL_ADDRESS>wrote:
Your skip ci in the commit message there is wrong :)
—
Reply to this email directly or view it on GitHub.


New man pages

st-flash
st-info

Will add st-term as well (In another PR probably)
Some st-flash options are marked as TODO because I'm not quite sure what they do yet

I did take the latest master branch, the Makefile was still there... I mess something up, will take a fresh clone and try again

Moved to #467.


Small performance fix

Hello!
This PR is fix for #1227, now we cache fatty jQuery object, and this works from 20% to 90% faster.
(20 for selection by id) (90 for selection by class)

LGTM

:+1:

Thanks @korun!


Add documentation for new changefeed features

.between.changes
.orderby.limit.changes
.filter.changes
.map.changes
squash

@mlucy - since you assigned this to yourself I figured you intended to write up a little bit about these yourself first and then bounce that to me?

Re-assigning back to me, as @mlucy wrote up some docs for squash. Are the other mentioned points things that need to be documented beyond a note in the changes() documentation about working with those commands now?

@mlucy suggested a few days ago that we could put an example or note about changefeeds in the documents of the commands that you can apply changes() to.
For example we could add an example with changes() to the documentation of the max term, or to get ("... You can also listen to changes on a document by opening a changefeed: r...get(...).changes()").
I don't think this is necessary before the 1.16 release though. We can extend the documentation in that respect later.
I think apart from documenting the squash parameter, noting the now working combinations in the changes() docs is the only critical thing.

What's the state of this @chipotle ?


Allow dev to set SameSite & MaxAge attributes on cookie

https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#security
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie#strict

Relevant line in sk-auth: https://github.com/Dan6erbond/sk-auth/blob/4bf8cc9fa025df6e75505142f1aa7be67cdca139/src/auth.ts#L133
Maybe also relevant? Do JWT and cookie need to be set to the same expiration? https://github.com/Dan6erbond/sk-auth/blob/4bf8cc9fa025df6e75505142f1aa7be67cdca139/src/auth.ts#L100

Possible solution would be to have add something like this to the config:
cookie: {
   maxAge: 1234,  // jwtExpires and cookie probably should be set the same automatically
   sameSite: 'Strict', // Lax is browser default if unspecified according to Mozilla link above
}


v1.11 Backports 2023-05-03 Cont

This PR is a successor of #25250. I didn't make any changes for those who are already approved in the old PR, so you don't have to review this again.
For those who are notified even if you are not listed as a reviewer now, sorry for the spam. I mistakenly open the PR against main at the beginning, and that made a huge diff.

[x] #25043 (@harsimran-pabla)

⚠ Already ported

[ ] #25046 (@nbusseneau)
[ ] #24770 (@aditighag)

⚠ Dropped

[x] #25033 (@giorio94)
[x] #25109 (@sayboras)
[x] #25192 (@kaworu)
[ ] #25174 (@joestringer)
[x] #25219 (@tommyp1ckles)

Once this PR is merged, you can update the PR labels via:
for pr in 25043 25046 24770 25033 25109 25192 25174 25219; do contrib/backporting/set-labels.py $pr done 1.11; done

or with
make add-labels BRANCH=v1.11 ISSUES=25043,25046,24770,25033,25109,25192,25174,25219

@aditighag I resolved this comment, but I still see the test failure. I'll dig into more details, but please let me know if you have any clue.
https://github.com/cilium/cilium/pull/25250#discussion_r1187486629

For those who are notified even if you are not listed as a reviewer now, sorry for the spam. I mistakenly open the PR against main at the beginning, and that made a huge diff.

Don't be sorry. Use the scripts.

@YutaroHayakawa Please skip the PR backport. I'll backport it manually to 1.11. Thanks!

Dropped Aditi's PR.

/test-backport-1.11

k8s-1.21-kernel-4.9: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.21-kernel-4.9/2621/
k8s-1.22-kernel-4.19: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.22-kernel-4.19/1665/
They are known flakes as MLH mentioned.

/test-1.21-4.9

/test-1.22-4.19

Conformance AWS-CNI: https://github.com/cilium/cilium/actions/runs/4942916632
Timeout while connecting to EKS kubernetes service. Probably, the AWS side of the issue. Let me retry.

/ci-awscni-1.11

All green. Ready to merge.

@YutaroHayakawa Run the script to mark backports done. Noticed that you had dropped #24770, marked it back to needs-backport-1.11, but should make sure if there was another determination what to do with it?

@jrajahalme Thanks for running the script. #24770 will be handled by Aditi.


Missing Images

I downloaded the data from the google drive link, and several image files are empty (0 bytes), including:
./train/img/1807.04686v1.1.png
./train/img/1610.02534v1.3.png
./train/img/0804.1441v3.3.png
./train/img/1611.02944v1.9.png
./train/img/1712.01039v2.4.png
./train/img/1705.03385v1.2.png
./train/img/1702.00552v1.1.png
./train/img/0705.1956v1.13.png
./train/img/1803.01529v1.3.png
./train/img/1703.09695v1.1.png
./train/img/1803.04786v1.4.png
./train/img/1402.1107v1.1.png
./train/img/1711.01711v11.3.png
./train/img/1808.02152v1.1.png
./train/img/1611.01056v2.11.png
./train/img/1510.03820v4.1.png
./train/img/1603.07603v1.1.png
./train/img/1805.10499v1.1.png
./train/img/1605.03821v2.3.png
./train/img/1412.1842v1.3.png
./train/img/1604.08553v4.5.png
./train/img/1311.4166v1.6.png
./train/img/1210.6912v1.6.png

It is possible because the script cropping image from pdf has some problem in it. It cannot deal with all the pdfs properly. You can just skip the empty images or check the pdfs manually. We will update the dataset as soon as possible.


don't let sync producer hang when delivery report is not received

This pull request fixes the long-lived #449 by adding a pending_timeout_ms kwarg to the Producer. This kwarg indicates the amount of time that should be spent waiting for delivery reports / delivery marks before returning from calls that wait on them. This allows the producer to handle the case in which the delivery report is never received, possibly due to poor connection between broker and client. This PR also adds a test to exercise this use case.

Codecov Report

Merging #769 into master will decrease coverage by 9.42%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #769      +/-   ##
==========================================
- Coverage    84.5%   75.08%   -9.43%
==========================================
  Files          27       27
  Lines        3556     3556
  Branches      532      495      -37
==========================================
- Hits         3005     2670     -335
- Misses        413      736     +323
- Partials      138      150      +12

Impacted Files
Coverage Δ

pykafka/rdkafka/helpers.py
0% <0%> (-93.34%)
:arrow_down:

pykafka/rdkafka/simple_consumer.py
0% <0%> (-89.39%)
:arrow_down:

pykafka/rdkafka/producer.py
10.9% <0%> (-83.64%)
:arrow_down:

pykafka/rdkafka/__init__.py
50% <0%> (-50%)
:arrow_down:

pykafka/handlers.py
73.88% <0%> (-23.14%)
:arrow_down:

pykafka/utils/compat.py
70.27% <0%> (-14.87%)
:arrow_down:

pykafka/topic.py
74.46% <0%> (-10.64%)
:arrow_down:

pykafka/broker.py
81.28% <0%> (-9.95%)
:arrow_down:

pykafka/simpleconsumer.py
74.82% <0%> (-9.84%)
:arrow_down:

pykafka/managedbalancedconsumer.py
75.33% <0%> (-8.67%)
:arrow_down:

... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5b860fd...41e1038. Read the comment docs.


⚠ Bofrost.de has degraded performance

In 08dea82, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1414 ms

Resolved: Bofrost.de performance has improved in 092dc0b.


TODO Aasim review with Spencer the service locator being used to muddle the lifetimes by injecting IServiceProvider into IPageFactory... does it cause lifetime corruption?

TODO document how clients can register the PageFactory Dictionary delegates to resolve all types; nameof(TPage) => provider.GetRequiredService();

https://github.com/DFE-Digital/pageobjects-testing-infrastructure/pull/4/files
Need to start writing tests with both providers that this works.

TODO after https://github.com/DFE-Digital/pageobjects-testing-infrastructure/commit/a76ac33db4e210ca4ce21bbc6e6c494f5feab6f9
Expand how a user can inject a ComponentFactory into their application components.
Expand how a user can see which components are available.
Expand on selection strategies with IElementSelector ->

IElementSelector, (XPath, ChildXPath, Css, Id)
FindWithinAScope,
Overwrite the default mapping selector for a known component

#6 succeeding for remaining points.


PhotoImage with PNG files

Type: Bug
Hello, during my usage of Tkinter, I can't import PNG files sometimes, like it's randomly when I star the app I don't know if it'll be able to run my project or not.
It's really really annoying because after every try I have to restart my computer.
I have this problem only with PNG and JPG files, not GIF.
Thanks for reading this feedback.
Cordially.
VS Code version: Code 1.86.0 (05047486b6df5eb8d44b2ecd70ea3bdf775fd937, 2024-01-31T10:28:19.990Z)
OS version: Windows_NT x64 10.0.19045
Modes:

System Info

Item
Value

CPUs
AMD Ryzen 5 3600 6-Core Processor               (12 x 3600)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: enabled_ondirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_graphite: disabled_offvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: enabled

Load (avg)
undefined

Memory (System)
15.95GB (7.85GB free)

Process Argv
--crash-reporter-id 01fbaeaf-ee87-4a05-8a3f-a4b7c763c7d2

Screen Reader
no

VM
0%

Extensions (10)

Extension
Author (truncated)
Version

vscode-language-pack-fr
MS-
1.85.2024013109

debugpy
ms-
2024.0.0

isort
ms-
2023.10.1

python
ms-
2024.0.0

vscode-pylance
ms-
2023.12.1

jupyter
ms-
2024.1.0

jupyter-keymap
ms-
1.1.2

jupyter-renderers
ms-
1.0.17

vscode-jupyter-cell-tags
ms-
0.1.8

vscode-jupyter-slideshow
ms-
0.1.5

A/B Experiments
vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805cf:30301675
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:30899288
c4g48928:30535728
azure-dev_surveyone:30548225
a9j8j154:30646983
962ge761:30951796
pythongtdpath:30769146
welcomedialogc:30910334
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
pythontestfixt:30902429
pythonregdiag2:30936856
pyreplss1:30897532
pythonmypyd1:30879173
pythoncet0:30885854
pythontbext0:30879054
accentitlementsc:30887149
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
dsvsc019b:30953937
3ef8e399:30949928

Thanks for creating this issue! We figured it's missing some basic information or in some other way doesn't follow our issue reporting guidelines. Please take the time to review these and update the issue.
Happy Coding!

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


cad-120 dateset

Aprreciate for your nice work! Can you share me with CAD-120 dataset? It seemed that there are some incomplete CAD-120 dataset on the Internet. And paper(LIGHTEN-Learning-Interactions-with-Graphs-and-Hierarchical-TEmporal-Networks-for-HOI) used the features that it has encoded. Does CAD-120 dataset keep secret?

Hi @mytk2012 Our work doesn't involve experiments on CAD-120 dataset. Please refer to this link for some help.

Yeah, but still appreciate for your work due to your novel dataset, it give me another choice instead of only CAD-120 dataset. Hope your dataset and model can be widespread soon.


Remove SafeMode from Count and Detect

This is in response to #69. As noted in the issue, SafeMode has outlived its usefulness. Making use of being disabled by default or setting this as unsafe for auto-correction make more sense.
I have left in the functionality that automatically disables these cops when the Rails cops are enabled. I'm not sure if that is functionality that we would like to maintain or not. These are the only 2 cops that function this way. If we remove the functionality, we can remove the SafeMode module from the core project.

I have left in the functionality that automatically disables these cops when the Rails cops are enabled. I'm not sure if that is functionality that we would like to maintain or not.

Yeah, I'd like to investigate whether rails? method can be removed. Anyway, I agree with SafeMode should be replaced by SafeAutoCorrect.

Yeah, I'd like to investigate whether rails? method can be removed.

I think there is an argument for removing it based on having SafeAutoCorrect set to false and having the reasons why it is considered unsafe being well documented.

Should I update the VersionChanged on these cops?

Yes.

@bbatsov what are your thoughts on removing the rails? method in favor of SafeAutoCorrect: false and documentation on why it is unsafe?

That sounds like a good approach to me. Let's make it happen!

Cool, I'll update this code to remove the rails? check. I'll then create a PR that removes the SafeMode module from RuboCop core.

@koic this is ready for review again.


Indexer is not working

Hi,
I tried the following tutorial
ColBERTv2: Indexing & Search Notebook
I used Nvidia V100 GPU for it.
I checked the status of gpu usage with the "nvidia-smi" command and it was increased temporally.
But there was no result and only "Starting". It is still running 😂.

Could give me some tips for it?
Thanks.

How long has it been? Are you running off the conda environment in the readme?

How long has it been? Are you running off the conda environment in the readme?

Hi. It has been a few days. I saw the recommendation of using conda but I executed it without conda environment actually.
I installed some requirements before running.
Did I do it wrong...?

Maybe try running again. But this time index a smaller collection. You can do that by passing collection=collection[:3000] in the cell you show. Let’s see if this runs. It should finish in one or two minutes

Maybe try running again. But this time index a smaller collection. You can do that by passing collection=collection[:3000] in the cell you show. Let’s see if this runs. It should finish in one or two minutes

Thank you for your comments. I tried indexing process with sampled data before but it was not finished. Maybe, there are some problems of environments.

I am having the same problem.  The indexer takes forever to finish it.

Hi,
I tried the following tutorial ColBERTv2: Indexing & Search Notebook
I used Nvidia V100 GPU for it. I checked the status of gpu usage with the "nvidia-smi" command and it was increased temporally. But there was no result and only "Starting". It is still running 😂.
Could give me some tips for it?
Thanks.

I am having the same problem. The indexer takes forever to finish it.

Hi,
I tried the following tutorial ColBERTv2: Indexing & Search Notebook
I used Nvidia V100 GPU for it. I checked the status of gpu usage with the "nvidia-smi" command and it was increased temporally. But there was no result and only "Starting". It is still running 😂.
Could give me some tips for it?
Thanks.

Hi,
I can't still solve it too 😅

Hi,
I faced the same issue a while back, it keeps on running here without giving any error, to see the error try to run it in VS code or something similar and see its output(which is on the left of terminal) or use ctrl+shift+U(in VS CODE). On jupyter notebook, it doesn't give any error.

Could you re-run and set the environment variable COLBERT_LOAD_TORCH_EXTENSION_VERBOSE=True? It's possible that you need to erase your torch extensions cache to enable the Torch extension code to compile.

Like I said earlier, though, this time index a smaller collection. You can do that by passing collection=collection[:3000] in the cell you show. Let’s see if this runs. It should finish in one or two minutes

Hi Omar,
I am having the same issue. I took a really small tsv file(16 pages of text) and created a collection. When I try to create an index, the code keeps running forever on a GPU. I created the conda environment based on conda_env.yml .  Something doesnt seem right. Are you able to help? Attaching the output log while the code is running.
Log_output.pdf

Hi Omar,
I am getting the same issue. I took a really small tsv file(16 pages of text) and created a collection. I have used the conda environment based on the conda_env.yml file. When I try to create the index, there seems to be a runtime error
but the code keeps running forever even on a GPU. Attaching the output for your reference. Appreciate your help.
Log_output.pdf

By the way, here's a free Colab notebook in which you can run indexing and search: https://colab.research.google.com/github/stanford-futuredata/ColBERT/blob/main/docs/intro2new.ipynb

@okhat im also facing same issue just tried few documents & indexer running forever

tried running collab notebook but for smaller data.

I think RAGatouille has a fix for this now — it avoids forking the process, I suggest using RAGatouille’s for ColBERT development for apps. (For research this repo is better.)

I think this issue has been fixed in 2162aee by setting avoid_fork_if_possible = True in RunConfig:
with Run().context(RunConfig(nranks=1, experiment='notebook', avoid_fork_if_possible = True)):


Public method for signing a transaction

We are developing a software in .NET for interacting with IC using the EdjCase.ICP.Agent nuget package (https://www.nuget.org/packages/EdjCase.ICP.Agent). Our software basically consists of two components: the server side app (webapi) and the client side app (a console app), that securely holds the private key.
In our flow, the transaction submission process is divided into 4 steps:

on the server side - prepare the transaction to be signed,
on the client side - retrieve the (nonsigned) transaction from the server side, sign it and send the signed transaction back to the server-side,
on the server side - submit the signed transaction to the IC using HTTP.API,
on the server side - check the status of the transaction.

In the EdjCase.ICP.Agent library, steps 1-3 are performed within a private SendAsync method:
https://github.com/BoomDAO/ICP.NET/blob/93072497d0ea15e1460bb4161c4e39b7066c5851/src/Agent/Agents/HttpAgent.cs#L229.
What we would need to accomplish our flow, is to split this private method into 3 separate public methods:

Can this functionality be added to the EdjCase.ICP.Agent nuget package?

Its possible, i think I would need a bit more information
The hardest part would be allowing access to these lower level APIs and keeping the library abstracted from HTTP
From what I understand, you really just are sending it with an identity that is on a different computer, so it just needs to be signed somewhere else. There are 2 ways I can think of maybe implementing this

Delegated Identity would be the easiest, because that can be done now. Essentially you generate a key on your api server, then that is sent to your client, which then signs a delegation chain to allow the api server key to send the request(s) using its private key with a limited scope/time.
Remote identity. This would have to be added, but essentially we could make it so that there is a new identity type that makes a call to the client to sign the request, then on response, the server could use that signature and send it off. This would require more work and maybe API changes but i think it would be possible.

I guess a question is how much is the client checking, or is it just signing whatever it is given, just to have the key seperate. Or is there high security validation that the request is perfectly correct by the user?


[Do not Merge] {Backup} HANA Target container

Description

To fix an ICM -  Error in DB restore command of Azure backup for HANA using Azure CLI. Target VM Restore fails during alterante VM
Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change.
[Component Name 2] az command b: Add some customer-facing feature.

This checklist is used to make sure that common guidelines for a pull request are followed.

[Yes ] The PR title and description has followed the guideline in Submitting Pull Requests.

[Yes] I adhere to the Command Guidelines.

Backup


404 dask dashboard

When running jupyterlab on a local ubuntu machine, I notice the following in the logs.
What happened:

What you expected to happen:
I expected the dashboard to work and not encounter the 404.
Anything else we need to know?:
When I look try to access https://localhost:8787, I encounter the following

Environment:

Dask version: 2022.2.1
Distributed version: 2022.2.1
Bokeh version: 2.4.2
Python version: 3.9.10
Operating System: Ubuntu 20.04
Install method (conda, pip, source): conda (conda-forge channel)

@nikhil003 Welcome and thanks for reporting!
I believe this is similar to: https://github.com/dask/dask-labextension/issues/164?
If so, I'd suggest moving this question to that issue and closing this one. :) You can also try out some solutions described there like clearing the URL and updating the version of dask-labextension.

Closing in favor of https://github.com/dask/dask-labextension/issues/164


[Question] How SaveDraft in wizard form

Hi, Ihave been working with formio but i haven't found a way to save wizard draft using  directive. Save draft is only allowed to FormManager???

You will need to create a button and set the Action as 'Save in State' and set the 'Save in state' setting to 'draft'. You will then need to create a new submission action and add 'Submission State = "draft";'. You need to ensure that the same name that you have in the 'Save in state' setting is the same as what the submission state is equal to.
Here is a screenshot for the save as draft button: https://share.getcloudapp.com/YEuAY1bk
Here is a screenshot of the Submission Actions: https://share.getcloudapp.com/JruWYb5w
Here is some additional information: https://formio.github.io/formio.js/app/examples/savedraft.html

@osantos1910
Thank you for your patience while we got an example of this working for you.
Here it is:

I hope that this helps.

Closing this thread as it is outdated. Please re-open if it is still relevant. Thank you for your contribution!


Cranelift: Infinite loop in cranelift-frontend's variable ssa state machine

I'm experimenting with some cranelift codegen and I'm seeing a hang/OOM related to the variable ssa resolver.
Here's a minimal reproducer (for cranelift/frontend/src/frontend.rs):
#[test]
fn ssa_state_machine_hang() {
    let sig = Signature::new(CallConv::SystemV);

    let mut fn_ctx = FunctionBuilderContext::new();
    let mut func = Function::with_name_signature(ExternalName::testcase("sample"), sig);
    {
        let mut builder = FunctionBuilder::new(&mut func, &mut fn_ctx);

        let a = Variable::new(0);
        builder.declare_var(a, I32);

        let entry = builder.create_block();
        let block1 = builder.create_block();
        let use_block = builder.create_block();

        // fill entry block
        builder.switch_to_block(entry);
        use cranelift_codegen::ir::TrapCode;
        builder.ins().trap(TrapCode::User(0));

        // fill block1 with back edge
        builder.switch_to_block(block1);
        let cond = builder.ins().iconst(I32, 0);
        builder.ins().brnz(cond, block1, &[]);
        builder.ins().jump(use_block, &[]);

        // use variable in use_block
        builder.switch_to_block(use_block);
        let _ = builder.use_var(a);
        builder.ins().return_(&[]);

        println!("sealing blocks...");
        builder.seal_all_blocks(); // hangs
        builder.finalize();
    }
    println!("{}", func.display(None).to_string());
}

It hangs in seal_all_blocks -> run_state_machine with a huge/growing self.calls vec.
Versions and Environment
Cranelift version or commit: v0.73.0 and current main

I think the problem here is that block1 has a single predecessor (itself) and it tries to handle this single predecessor first

It looks like this was also discovered in #3094 and fixed in 6a9378e.


[DATASTD-1415] [DATASTD-1450]

updating xml namespace to http://ed-fi.org/0320c for descriptors, sample xml, and schemas
[DATASTD-1415] add Survey domain to Ed-Fi Data Standard
[DATASTD-1450] create initial Person model

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Allow the user's "open pane" bindings to open a pane with the current pane

Currently, the split pane keybindings will always open the default profile.
Perhaps a user will want to open a pane with the currently active pane's profile instead?
This is not #998, which is more about opening a pane with a specific profile, while this is about changing the default split behavior.

Bear with me for a second. Somebody suggested this in another issue, and I wonder if we should consider it here.

The new pane is focused, and put in a [number entry] or [search for profiles by name] mode, just like one of our earlier issues suggested.
We probably want that UI anyway, since it has some benefits. Thoughts?

Hmmmmmmm. I don't hate the idea. That's a good idea.
Lets take it another step. With the context of #1142, we could add an arg(s) that says "open the picker dialog" or "open my default" or "open this specific profile", and default to the "open the picker" setting. Then, the user could have one keybinding for "split the default" and another for "split with the picker" (which is the settings I would use :P)

My first inclination is to ask the new pane to inherit everything from the parent pane - the shell, its current working directory, environment variables (maybe?), etc, the reason being that I mentally want to split the existing shell, as-is.
I can see how others might want to split the pane into something completely new, so the idea of asking which shell to use is ok as well, either thru UI or settings in the keybindings.
So, having different commands and keybindings for them seems to be the better solution.
I'll argue that there are many different cases when splitting the pane:

start the same shell, inheriting the environment of the parent pane (profile,working dir, environment var, etc).
start the same shell, but do not inherit the current environment (ie, same as starting a new tab).
start the default shell
start an arbitrary shell

Just chiming in;

_ (profile,working dir, environment var, etc)._

Anything about the actual process on the other end is, in the general case, impossible to replicate. The connected process could be ssh.exe, whose environment variables and working directory have no bearing on the detectable environment and working directory from the terminal side. The same actually, weirdly enough, applies to WSL. It doesn't use "working directory" and it doesn't expose its environment variables to interested Windows processes in any way.
Powershell doesn't even set the current working directory, so its directory can't be detected (!) either.

Other related threads with thoughts on this discussion:
https://github.com/microsoft/terminal/issues/2315#issuecomment-519317472
https://github.com/microsoft/terminal/issues/1536#issuecomment-519107586

I really like the idea to change the default to use the current profile as default when creating a new pane with the default shortcut, and not the 'default profile'.
Using another shortcut (not the main shortcut though), should open a pane and ask for the profile to open there (#3586).

Just chiming in;

_ (profile,working dir, environment var, etc)._

Anything about the actual process on the other end is, in the general case, impossible to replicate. The connected process could be ssh.exe, whose environment variables and working directory have no bearing on the detectable environment and working directory from the terminal side. The same actually, weirdly enough, applies to WSL. It doesn't use "working directory" and it doesn't expose its environment variables to interested Windows processes in any way.
Powershell doesn't even set the current working directory, so its directory can't be detected (!) either.

Was interested in this but seems like it's not going to happen. Just curious if something like 'start' in cmd can't work in terminal. Is this the case ?

@Surya-06 could you clarify what use case you're looking for? start is a pretty powerful command that ends up calling ShellExecute.
Are you just looking for the ability to open a new terminal in the current directory? That'll work with #4023 and wt -d . Though, opening new panes/tabs in the current Terminal window like that is much trickier.

It seems like, at very least, we need an option to specify "current profile" when executing the splitPane action. I can configure different key bindings for each profile, or remember which order they are and use the index. But just having something I can specify on the splitPane action like:
    "keybindings": [
        {
            "keys": ["ctrl+|"],
            "command": {
                "action": "splitPane",
                "split": "vertical",
                "useCurrentProfile": true
            }
        }
    ]

would be good enough. I do think this should be the default ("splitting the current profile into two instances of the same profile") and that keeping the environment between them would be nice. But in the interim, just being able to split the current profile at all would be helpful.

@zadjii-msft Hey, has there been any update in this situation I'm trying to use terminal but this problem is the main blocker against it. It would be great if there was a way around. Thanks !

@Surya-06 Unfortunately no updates here. I know what needs to be done, and it's not terribly difficult, there's just other work that's a higher priority at the moment unfortunately.
If someone wanted to do this they'd need to:

add a type argument to the SplitPane action, that accepts two values: (as documented in #998)

"manual" (default): Assume that the user provided a profile parameter.
"duplicate": Use the currently focused pane's profile as the profile for the new pane
see ActionArgs.h, ActionArgs.idl, SplitPaneArgs

Update the logic in TerminalPage and AppActionHandlers to handle the new type parameter. We'd need to query the active tab to find the active profile if there is one, and use that, instead of the profile that CascadiaSettings::BuildSettings usually finds.

See TerminalPage::_HandleSplitPane, TerminalPage::_SplitPane, and CascadiaSettings::BuildSettings would probably also need some modification.

Add tests in KeybindingsTests.cpp

@zadjii-msft I'd like to take this up in my spare time if it's ok. 😉

@Surya-06 Go for it 😄

@zadjii-msft cool. will work on this. will ping here if I run into any issues. Thanks!

@zadjii-msft Hello. I've started working on the project and am making the changes. I could make the changes fine but am unable to test it out since the build has been failing. I don't use visual studio so I'm editing on VS Code and building using cmd. I ran the instructions in dev help and am running into the following error:

I've raised a draft PR: #4683
Can you please provide some help on this? Thanks!

I don't build the entire solution from the commandline myself very often, so I've never seen that particular error. However, I have a feeling that it's basically the same thing as #4211. I bet if you close all your browser windows, you should be able to free up enough memory to get past this step.
Once you get a successful build once, make sure to use bz or bx to avoid doing a clean rebuild on subsequent builds. This error only comes from building the pch.h file, and once it's built once, you shouldn't need to rebuild it unless you change it.

I've restarted my machine so that there are no running applications and I have 32gb of physical RAM. It has been several retries already. Is there any way to get around this?

Oh well that's pretty unexpected 😕. Do you have VS installed? As much as I don't love using it either, Visual Studio sometimes might be able to provide a better error message for something like this.

@zadjii-msft  Hey, thanks for the tip. I tried with Visual Studio and tried building all the packages which failed for almost 20 projects and then tried building only the Cascadia Package ( x64 ) and it failed too. Are there detailed build instructions somewhere I could use? Also I couldn't debug it as mentioned in the support guide because of the following error.

I'm almost done with the changes but am not able to get the build running so I can't test my changes.

I'm glad you were able to get it figured out! I can honestly say I've never seen those errors before, so your SDK install must have been pretty messed up 😅

@Surya-06 could you clarify what use case you're looking for? start is a pretty powerful command that ends up calling ShellExecute.
Are you just looking for the ability to open a new terminal in the current directory? That'll work with #4023 and wt -d . Though, opening new panes/tabs in the current Terminal window like that is much trickier.

Is there a way currently to have the new pane have the same working directory as the old pane? I tried doing "startingDirectory": "%CD%" for the splitPane keybinding but that failed.

Just chiming in with my thoughts as I just ran into this problem after installing powershell 7 and it's not my default shell... so all my splits end up being my default rather than ps7.   I think it should split by forking the active shell as much as possible.  But also have an option to change the active splits shell.  Also to be able to swap the position of two splits.  Also (heh..) to be able to take any shell from any tab and putting it into a split. Also number the splits and have a quick number hotkey jump to split. Also https://github.com/microsoft/terminal/issues/4692  . Also pop split into its own tab....    Also save splits into a profile (remembering size), and then open a split profile which can be configured to automatically run specific programs.  Then I think splits will be pretty cool, you can run semi ide like and open tabs with "split pane profiles" with active running terminal programs that are "widgets" (its nice you can resize a pane down to one line) showing timezones / server status / build status etc...


nbdev_release_gh fails with no feedback

Calling nbdev_release_gh I just get this, and I don't know where even to start to debug or create a minimal example:
❯ nbdev_release_gh
Traceback (most recent call last):
  File "/Users/klay6683/miniforge3/envs/py310/bin/nbdev_release_gh", line 10, in <module>
    sys.exit(release_gh())
  File "/Users/klay6683/miniforge3/envs/py310/lib/python3.10/site-packages/fastcore/script.py", line 119, in _f
    return tfunc(**merge(args, args_from_prog(func, xtra)))
  File "/Users/klay6683/miniforge3/envs/py310/lib/python3.10/site-packages/nbdev/release.py", line 127, in release_gh
    Release().changelog()
  File "/Users/klay6683/miniforge3/envs/py310/lib/python3.10/site-packages/nbdev/release.py", line 57, in __init__
    if not token: raise Exception('Failed to find token')
Exception: Failed to find token

My repo is here:  https://github.com/michaelaye/nbplanetary
nbdev version: 2.3.12
OS: macOS 12.6.1

ok this is weird, this is related to the Github token, which I clearly did create a while ago, as evident by the .gitignore that already includes the token ignore, but my token file isn't there. Very weird.


Draft 5: handshake and protocols

This branch adds a handshake packet and protocols packet so protocol extensions can be created. Please provide feedback, this is a draft.

This increases the complexity of the protocol significantly without much benefit. Needing the client to reply also increases overall latency during the handshake. Also, every existing Wisp implementation would have to adapt.

Wisp is already deployed in production by multiple services, so it's too late to many any breaking changes to the protocol.

I agree with @ading2210 personally,
I still believe using the connect packet as the handshake using the stream ID is the best approach since wisp is now in several production proxies

It's both a bit too early yet a bit too late for a breaking change already


Added dispatch for LeapInternalFrame

Summary
This PR ensures the LeapInternalFrame event acts as expected, similarly to the LeapFrame event.
Contributor Tasks
These tasks are for the merge request creator to tick off when creating a merge request.

[ ] Pair review with a member of the QA team.
[ ] Add any release testing considerations to the MR for the next release.
[x] Check any relevant CHANGELOG files have been updated.
[x] Ensure documentation requirements are met e.g., public API is commented.
[x] Consider any licensing/other legal implications for this MR e.g. notices required by any new libraries.
[x] Add any relevant labels such as breaking to this MR.
[ ] If this MR closes a Jira issue, make sure the fix version on the JIRA issue is set to the correct one.

Reviewer Tasks
Use emojis in review threads to communicate intent and help contributors.

[ ] Code reviewed.
[ ] Non-code assets e.g. Unity assets/scenes reviewed.
[ ] Documentation has been reviewed. Includes checking documentation requirements are met and not missing e.g., public API is commented.
[ ] Checked and agree with release testing considerations added to MR for the next release.
[ ] Check this provides valid hand data (info on how in the comments of this PR)

Closes JIRA Issue
If this MR closes any JIRA issues list them below in the form Closes PROJECT-#

To check the validity of the data we get from this event, I have created a test script that acts as a post-process provider and replaces and incoming data with the previous event data from LeapInternalFrame:
`using UnityEngine;
using Leap;
using Leap.Unity;
using LeapInternal;
public class TestingScript : PostProcessProvider
{
LeapXRServiceProvider provider;
Controller controller;
Frame newFrame = new Frame();

public override void ProcessFrame(ref Frame inputFrame)
{
    if (newFrame != null)
        inputFrame = newFrame;
    else if(Application.isPlaying)
        Debug.Log("No frame");
}

private void Awake()
{
    provider = FindObjectOfType<LeapXRServiceProvider>();
    controller = provider?.GetLeapController();

    if (controller != null)
    {
        controller.InternalFrameReady += InternalFrameReady;
    }
}

private void InternalFrameReady(object sender, InternalFrameEventArgs e)
{
    if (provider != null && newFrame != null)
    {
        LEAP_TRACKING_EVENT trackingEvent = e.frame;

        Frame lastFrameref = new Frame().CopyFrom(ref trackingEvent);
        newFrame.CopyFrom(lastFrameref);
    }
}

}`

Is there a Jira ticket for this @MattGrayUL?

@MaxPalmer-UH There isn't but i'll make one :)


Updated Check your user’s identity credential matches the level of co…

…nfidence needed section here   to add clarity around 'vot'claim as requested via slack and jira
Why
To improve the docs and make it clearer that RPs need to look for the 'vot' claim in the ID token and then make sure the expected and achieved level of protection matches or exceeds the level of confidence you need
What
Updated the Check your user’s identity credential matches the level of confidence needed with input from developers via slack and the Jira tickets ATO-456 and DTW-97
Technical writer support
Yes it would be nice for a tech writer to give it a once over
How to review
Confirm

[ ] I have checked if any docs change here also requires updates to other repositories (ADRs / RFCs, README.md, Team Manual, elsewhere in these docs)
[ ] Where there is any overlap I have updated or opened a PR for corresponding changes

I'm not sure this fully solves the problem. We also want RPs to check the VOT on auth only journeys. I think we need to update the "response to ID token" section: https://docs.sign-in.service.gov.uk/integrate-with-integration-environment/authenticate-your-user/#receive-response-for-make-a-token-request
This section:

Does not have a correct example ID token (missing VOT)
Does not include instructions to check VOT matches the expected value


bug: "references" preview pane shows tree-sitter highlights after scrolling

Check the following:

[X] I have checked the behavior after setting "vscode-neovim.neovimClean" in VSCode settings.
[X] I have read the vscode-neovim docs.
[X] I have searched existing issues.

Neovim version (nvim -v)
v0.9.5
Operating system/version
Fedora 39
Describe the bug
When opening the "references" pane, highlighting will (usually) show up correctly. However, once I click on an item to scroll to, the tree-sitter highlights return. A video is worth a thousand words:
https://github.com/vscode-neovim/vscode-neovim/assets/977151/63d35917-6dfb-420d-b374-eb1975e24d56
This seems related to #1739 (and perhaps #1940?).
Steps To Reproduce

Go to a file where you will have multiple references for a single variable
Run "go to references"
Click on one of the jump-points, and notice that the highlight colors change.

Expected Behavior
The tree-sitter highlights should not bleed through.

Hm, I've been experimenting a little bit, and I wonder... do we have any reason to store highlight groups for of kind syntax? We always just disable treesitter when a buffer opens. I worry this would have knock-on effects, but it sure feels like a "simple" solution.
https://neovim.io/doc/user/ui.html

kind: always present. One of the following values: "ui": Builtin UI highlight. highlight-groups "syntax": Highlight applied to a buffer by a syntax declaration or other runtime/plugin functionality such as nvim_buf_add_highlight() "terminal": highlight from a process running in a terminal-emulator. Contains no further semantic information. ui_name: Highlight name from highlight-groups. Only for "ui" kind. hi_name: Name of the final :highlight group where the used attributes are defined. id: Unique numeric id representing this item.

I did a git-bisect from v1.11.1 to master, and came up with #1944 being the culprit, which is interest because it seems to try and fix this same problem. I'll dig into this when I get a spare cycle (but of course wouldn't mind a hand)

Nice, this seems to reproduce pretty reliably for me as well (like 75% of the time), but only when I'm running v1.11.3, so I think it helped! I'll keep an eye out for any other highlight issues (seems like there was some discussion on that PR) but this definitely seems like it's helping for now, thanks! 🎉


Adds MMProperty class - a view onto a core property with a nicer API

after repeatedly calling mmcore.doSomething(device, property), this is a convenience class that lets you avoid doing a lot of searching for the property related API on the core object.
In [1]: from pymmcore_plus import MMProperty

In [2]: prop = MMProperty('Objective', 'Label')

In [3]: prop.isValid()  # doesn't need to be loaded in core
Out[3]: False

In [4]: prop.core.loadSystemConfiguration()

In [5]: prop.isValid()  # now it's loaded in core
Out[5]: True

In [6]: prop.value
Out[6]: 'Nikon 10X S Fluor'

In [7]: prop.value = 'Objective-2'  # property.setter

In [8]: prop.value
Out[8]: 'Objective-2'

In [9]: prop.value == prop.core.getProperty(prop.device, prop.name)  # stays in sync
Out[9]: True

In [10]: prop.isReadOnly()
Out[10]: False

In [11]: prop.hasLimits()
Out[11]: False

In [12]: prop.type()
Out[12]: <PropertyType.String: 1>

In [13]: prop  # nice repr
Out[13]: <Property Label on device Objective: value='Objective-2'>

In [14]: prop.dict()
Out[14]:
{
    'valid': True,
    'value': 'Objective-2',
    'type': 'string',
    'device_type': 'StateDevice',
    'read_only': False,
    'pre_init': False,
    'range': None,
    'allowed': (
        'Nikon 10X S Fluor',
        'Nikon 20X Plan Fluor ELWD',
        'Nikon 40X Plan Flueor ELWD',
        'Objective-2',
        'Objective-4',
        'Objective-5'
    )
}

ok, removed all the core setting stuff.  Added core.getPropertyObject and core.iterProperties() (which is an iterator i've wanted for a while... it also includes as_object, device_type, and property_type filters.)

It would be nice to have the ability to get all the properties as objects for a specific device.

added a device_label filter to iterProperties()

Only remaining thought is about getDeviceProperties

actually I'd propose we remove that method.  can do in a later PR though


Question about Pre-training

Dear zihan,
Thank you for your impressive work!
I have been following your work and I have successfully run the code with the setting 'model_type="LViT"', which does not require a pretrained model. However, I am facing some challenges in understanding how to obtain the pretained model for the setting 'model_type="LViT"'.
As discussed in Section 2.1 of your instructions:

It appears that a U-Net model might be a prerequistite for the LViT model. Could you please clarifiy if this is the case? If so, does this imply I need to train a U-Net model first before proceeding with 'LViT_pretrain'? Also, in such a scenario, should I change the model type and write the corresponding code here?

Furthermore, I am also curious about how to load the pretrained U-Net model once it is obtained. Is this U-Net model directly applicable to LViT_pretrain, or are there additional steps or modifications required?
Your guidance on these matters would be greatly appreciated, as it would greatly assist me in understanding and utilizing your work more effectively.
Thank you for your time and consideration. I am looking forward to your valuable insights.
Best regards,
Pengyu Zhao

Hi Pengyu,
You can use the UNet network code we provide to train a UNet model and use it as a pre trained model. Don't worry, the UNet pretrain model can be directly applied to the pretraining of LViT.

Hi Zihan,
Thank you for your patient guidance!
Now, I have successfully trained the UNet network on the 'MoNuSeg' task using the code you provided. If I want to used it for the pretraining of LViT to get the results you show here.

Should I rewite the pretrained_UNet_model_path here to import the pretrained weight?

Please don't close the issue for now. I will close it promptly when it's resolved.
Thank you again for your professional guidance!
Best regards,
Pengyu

Yes, you should change the load path to your own path.

Hi Zihan,
I have changed the load path to my own path as shown here:

The model, best_model-UNet.pth.tar,  was obtrained by training the UNet model on the MoNuSeg dataset. I used this pretrained UNet model for the pretraining of LViT. Could you please confirm if this approach is appropriate?
Now, I have got the test results of LViT and LViT w/o pretrain on the MoNuSeg task as shown here:

Upon evaluating the results of my experiments, I observed a decrease in the performance of the LViT model after loading the UNet pretrained model. This was an unexpected outcome. To understand this issue better, I delved into the training logs. Interestingly, I found that while the convergence speed of LViT was indeed faster with the pretrained model (achieving optimal performance at epoch 89) compared to LViT without pretraining (achieving optimal performance at epoch 130), the validation results told a different story. Specifically, the mean dice score for the LViT model without pretraining was 0.8062, while the pretrained LViT model only achieved a score of 0.8052, indicating a decrease in validation performance.
LViT w/o pretrain training log:

LViT training log:

This outcome is puzzling to me, and I'm currently exploring potential reasons for this discrepancy. I am considering whether adjusting the learning rate from 1e-3 to 3e-4 and increasing the early_stopping_patience might lead to better results. I would greatly value your opinion on this approach.
Do you think these changes could potentially address the issues I'm encountering?
Additionally, are there other modifications you would recommend?
Any guidance you can provide would be immensely helpful.
Best regards,
Pengyu

The optimal performance may be related to the GPU used, and you should search for the optimal hyperparameters based on the platform you are using. And also, you can decrease the learning rate and increase the early stop patience to get a better result.

I am wondering if the formula about calculating IOU is correct. if N is the number of pixels, so it's an average IoU or cumulative IoU? Looking forward to your reply. Thank you.


ncp: add flag to pass RLOC16 IPv6 traffic to host.

Add SPINEL_PROP_THREAD_RLOC16_DEBUG_PASSTHRU to allow forwarding of RLOC16 traffic to HOST, including receipt of ping responses to RLOC16.

LGTM 👍

LGTM 👍


[FEATURE] Display max instances count in wash get inventory and washboard output

Affected project(s)

[ ] documentation
[ ] examples
[ ] wasmCloud host
[X] wasmCloud CLI (wash)
[X] wasmCloud dashboard UI (washboard)
[ ] capability providers
[ ] provider bindgen
[ ] control interface client
[ ] other / not sure

Is your feature request related to a problem? Please describe.
In order to know the scale of my components, I need to put the inventory out in JSON output mode.
Describe the solution you'd like
I would love to see the concurrency that's set for a component in the host inventory output
For example:
➜ wash get inventory

  Host Inventory (NALS3CXBO7IK57655LUN4NIPA3BLCX5KCXYV3WE3G7O5LPC6KK5COTKZ)

  hostcore.os          linux
  hostcore.arch        x86_64
  hostcore.osfamily    unix

  Component ID         Name                   Image Reference                                   Max
  hello                http-hello-world       wasmcloud.azurecr.io/http-hello-world:0.1.0       999

  Provider ID          Name                   Image Reference
  http-server          http-server-provider   ghcr.io/wasmcloud/http-server:0.20.1

Describe alternatives you've considered
No response
Additional context
I actually wouldn't mind getting rid of the Name now that we have a unique ID that generally covers it, especially since claims are no longer required metadata.

I think I know how to add this feature, can I have this issue assigned please? Thank you!

So the plan is to add a new column called "Max count" which will show max instances, then i'll reposition "Name" with "ID" and get rid of "Name"

would it be alright if we got rid of Name from the Provider table too ? just so everything appears uniform


Links webpart ID issue

Hi,
I tried to use only links Webpart  and its give me error loading always  when I do gulp serve ?
[SPLoaderError.loadComponentError]:
***Failed to load component "89978bff-19d9-4fbc-87bc-42ab73384dbf" (LinksWebPart).
Original error: ***Failed to load entry point from component "89978bff-19d9-4fbc-87bc-42ab73384dbf" (LinksWebPart).
Original error: Error loading https://component-id.invalid/89978bff-19d9-4fbc-87bc-42ab73384dbf_0.0.1
Cannot find module "ControlStrings"

@krmoanh can you check if the @pnp/spfx-controls-react dependency is installed? You should be there on the following path: node_modules/@pnp/spfx-controls-react.

Hi @estruyf  thanks for replay. I already install the package  npm i --save @pnp/spfx-controls-react
.
but still not working. and also I install  npm i --save @pnp/spfx-property-controls  and same issue

I'm sure that you ran a npm install, but I want to be sure the path to the locale file which is failing exists or not. Could you please check if the following location exists: node_modules/@pnp/spfx-controls-react/lib/loc/.

@estruyf  yes this is with PowerShell getting all the modules
PS F:\akram\links> npm list --depth=0
<EMAIL_ADDRESS>F:\akram\links
+--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>+--<EMAIL_ADDRESS>`--<EMAIL_ADDRESS>

Ok, seems you have the wrong version. Do you use npm v5? Or an older version?
Can you try to install npm i @pnp/spfx-controls-react@next -S.

I just install it and test the webpart    gulp serve  and its give me this
[SPLoaderError.loadComponentError]:
***Failed to load component "89978bff-19d9-4fbc-87bc-42ab73384dbf" (LinksWebPart).
Original error: ***Failed to load entry point from component "89978bff-19d9-4fbc-87bc-42ab73384dbf" (LinksWebPart).
Original error: Error loading https://component-id.invalid/89978bff-19d9-4fbc-87bc-42ab73384dbf_0.0.1
Cannot find module "PropertyControlStrings"

Just pushed out a new major release for the React Controls. Can you update the @pnp/spfx-controls-react and @pnp/spfx-property-controls dependency versions in the package.json file to the following:
"@pnp/spfx-controls-react": "1.4.0",
"@pnp/spfx-property-controls": "1.6.0"

I update it  both  to
spfx-controls-react": "1.4.0",
pnp/spfx-property-controls": "1.6.0"
and its gives me after gulp serve
[SPLoaderError.loadComponentError]:
***Failed to load component "89978bff-19d9-4fbc-87bc-42ab73384dbf" (LinksWebPart).
Original error: ***Failed to load entry point from component "89978bff-19d9-4fbc-87bc-42ab73384dbf" (LinksWebPart).
Original error: Error loading https://component-id.invalid/89978bff-19d9-4fbc-87bc-42ab73384dbf_0.0.1
Cannot find module "PropertyControlStrings"
***INNERERROR:
***Failed to load entry point from component "89978bff-19d9-4fbc-87bc-42ab73384dbf" (LinksWebPart).
Original error: Error loading https://component-id.invalid/89978bff-19d9-4fbc-87bc-42ab73384dbf_0.0.1
Cannot find module "PropertyControlStrings"
***CALLSTACK:
Error
at SPLoaderError.SPError [as constructor] (https://localhost:4321/node_modules/@microsoft/sp-loader/dist/sp-loader-assembly_en-us.js:10183:24)
at new SPLoaderError (https://localhost:4321/node_modules/@microsoft/sp-loader/dist/sp-loader-assembly_en-us.js:4211:28)
at Function.ErrorBuilder.buildErrorWithVerboseLog (https://localhost:4321/node_modules/@microsoft/sp-loader/dist/sp-loader-assembly_en-us.js:3821:21)
at Function.ErrorBuilder.buildLoadComponentError (https://localhost:4321/node_modules/@microsoft/sp-loader/dist/sp-loader-assembly_en-us.js:3743:21)
at https://localhost:4321/node_modules/@microsoft/sp-loader/dist/sp-loader-assembly_en-us.js:7952:38

On which type of machine are you running this? Node and NPM version?
Manually check if this path is correct: node_modules/@pnp/spfx-property-controls/lib/loc/. In the loc folder you should find a JS file.

@estruyf  I already checked the node_modules/pnp/spfx-property-controls/lib/loc and also  for react  both are .js file are there.  when I use  the whole package of  sp-starter-kit  the links webpart works well. but what I need is im traying to use only the links webpart code  not the whole sp-starter-kit  same what I did in worldtime webpart and weatherInformation and its work fine for me. iam using windows 10 dell machine and when I start create the links webpart I choose react  the second option.  Node version is v6.12.3  , and NPM version is 3.10.10  .

If I understand correctly, did you copy the code to a separate project? If you did that, you will have to do some configuration for the controls. Check the installation/configuration procedure for the property pane controls which you seem to be missing: https://sharepoint.github.io/sp-dev-fx-property-controls/#installation

yes that true. it was missing the configuration for the controls in config.json and now the webpart it works perfectly. thanks a lot estruyf


Update helloworld.py

Another exclamation mark! (won't actually merge this one)

Perfect, made a new image.


Update vue.js version to 1.0.16

As title, just update the file from original vue github

Sorry this seemed to have slipped off my to-do list. I've shipped this update. Thanks for your contribution.


Improve check error reporting

The ci process ends up with a list of failed projects; it should also list the failed checks 😅

moved to https://github.com/coveo/stew/issues/20


Add reading and writing with bigger data types

Make an optional optimization, where each thread reads element sizes that are 2x or 4x the size of the real data element size.
This should be aplicable to data types that are multiple of 2. May we do it for data types of size 3 at some point.

Things pending to do:

The flag to activate the thread fusion has to be in the GridPattern.
For the support of only first read and last write operations, we need to find a way to have the same number of elements per thread in both the read and the write operations.
We need to either detect that there is a MidWriteOperation and disable the thread fusion, or add support to apply the thread fusion also to the MidWriteOperation.
What happens if one of the parameters it's a matrix, and we need to read each of the elements of that matrix to multiply one on one with each of the elements of the other matrix.


fix grammatical issues and single installer

Not sure what "either" applies to as only Homebrew is now listed.
Advise if the script on demand is no longer supported . ie.
curl -L https://aka.ms/InstallAzureCli | bash

While that script does work, it's no longer clearly visible on the macOS install guide. So, is Homebrew the only way supported in the future? Please say "NO" - many of us are not full Brew fans.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-conceptual/install-azure-cli-macos.md
:white_check_mark:Succeeded
View (azure-cli-latest) View (azure-cli-2017-03-09-profile)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

macOS is a version of UNIX supported by the installation script, so you can still follow the Linux manual instructions.

I get that, but current Doc doesn’t say that. Should it be added back in? What’s the path forward?

The official recommendation is that if a package manager is available, it should be used instead of any manual installation.


Problem with swapping columns in Example 7

I haven't dug into this issue yet,  but I did notice that example 7 has a bug with updating the view:

Click on a cell in the Priority column and notice that it cycles through the options
Drag the Priority column to the left of the title column
Click on a cell in the Priority column and notice that it does not change, although it can be edited or right-clicked
Click on a cell in the Title column to change the priority.

Fixed in https://github.com/mleibman/SlickGrid/commit/adac3d8f861ee11ac290250e5d3667b23405cda7


Add an option to see chart of all tx methods combined

Currently, for each tx method, a chart is rendered. Add an extra option to render by combining all tx method balance data.

Add another option beside the list of the tx methods on the chart page which can also be enabled and disabled
Properly handle the max value in the chart

Most of the code is already in place, and should only require minor modification for this. If anyone wants to contribute, feel free to comment here!

Resolved on #90