Add VEP annotation via subworkflow

PR checklist

[ ] This comment contains a description of changes (with reason).
[ ] If you've fixed a bug or added code that should be tested, add tests!
[ ] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
[ ] Make sure your code lints (nf-core lint).
[ ] Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
[ ] Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
[ ] Usage Documentation in docs/usage.md is updated.
[ ] Output Documentation in docs/output.md is updated.
[x] CHANGELOG.md is updated.
[ ] README.md is updated (including new tool citations and authors/contributors).

Problem so far: The index file is older than the vcf.gz file...
executor >  local (6)
[53/d41436] process > QBICPIPELINES_VCFTOMAF:VCFTOMAF:UNTAR (vep.tar.gz)     [100%] 1 of 1 ✔
[d1/ad6aac] process > QBICPIPELINES_VCFTOMAF:VCFTOMAF:ENSEMBLVEP_VEP (test2) [100%] 2 of 2 ✔
[0c/80a19f] process > QBICPIPELINES_VCFTOMAF:VCFTOMAF:TABIX_TABIX (test2)    [100%] 2 of 2 ✔
[7a/b3cb1c] process > QBICPIPELINES_VCFTOMAF:VCFTOMAF:BCFTOOLS_VIEW (test)   [ 50%] 1 of 2, failed: 1
[-        ] process > QBICPIPELINES_VCFTOMAF:VCFTOMAF:GUNZIP                 -
[-        ] process > QBICPIPELINES_VCFTOMAF:VCFTOMAF:VCF2MAF                -
[-        ] process > QBICPIPELINES_VCFTOMAF:VCFTOMAF:MULTIQC                -
[DUMP: vcf-before] [['id':'test', 'index':true, 'normal_id':'testN', 'vcf_normal_id':'testN'], /home/paifb01/vcftomaf/work/80/5d0858f352f14c963373766c511701/test.vcf.gz]
[DUMP: vcf-sfter] [['id':'test', 'index':true, 'normal_id':'testN', 'vcf_normal_id':'testN'], /home/paifb01/vcftomaf/work/80/5d0858f352f14c963373766c511701/test.vcf.gz, /home/paifb01/vcftomaf/work/81/bbdcd6d89e3166b3629350da6aea6e/test.vcf.gz.tbi]
[DUMP: vcf-before] [['id':'test2', 'index':false, 'tumor_id':'testT', 'vcf_tumor_id':'testT'], /home/paifb01/vcftomaf/work/d1/ad6aacd5607f07379d37d7619767f5/test2.vcf.gz]
[DUMP: vcf-sfter] [['id':'test2', 'index':false, 'tumor_id':'testT', 'vcf_tumor_id':'testT'], /home/paifb01/vcftomaf/work/d1/ad6aacd5607f07379d37d7619767f5/test2.vcf.gz, /home/paifb01/vcftomaf/work/0c/80a19f8f4a5a74ae77648d7ff60263/test2.vcf.gz.tbi]
Execution cancelled -- Finishing pending tasks before exit
-[qbic-pipelines/vcftomaf] Pipeline completed with errors-
ERROR ~ Error executing process > 'QBICPIPELINES_VCFTOMAF:VCFTOMAF:BCFTOOLS_VIEW (test)'

Caused by:
  Process `QBICPIPELINES_VCFTOMAF:VCFTOMAF:BCFTOOLS_VIEW (test)` terminated with an error exit status (9)

Command executed:

  bcftools view \
      --output test.vcf.gz \
      --regions-file genome.bed \
       \
       \
      --output-type z \
      --threads 4 \
      test.vcf.gz

  cat <<-END_VERSIONS > versions.yml
  "QBICPIPELINES_VCFTOMAF:VCFTOMAF:BCFTOOLS_VIEW":
      bcftools: $(bcftools --version 2>&1 | head -n1 | sed 's/^.*bcftools //; s/ .*$//')
  END_VERSIONS

Command exit status:
  9

Command output:
  (empty)

Command error:
  Error: BCF read error

Work dir:
  /home/paifb01/vcftomaf/work/7a/b3cb1cf4b5c49a62635b8e8284521c

Container:
  quay.io/biocontainers/bcftools:1.20--h8b25389_0

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting

 -- Check '.nextflow.log' file for details


query retry not working

Describe the bug
QueryClient defaultOptions.queries retry is not working., defaultOptions.mutations works as expected.
Your minimal, reproducible example
Not provided
Steps to reproduce
const queryClient = new QueryClient({
	defaultOptions: {
		queries: {
			notifyOnChangeProps: 'all',
			refetchOnMount: 'always',
			refetchOnWindowFocus: true,
			refetchOnReconnect: true,
			retry: (failureCount: number, error: any) => {
				return (
					(!error.response ||
						error.response.status === 404 ||
						(error.response.status >= 500 && error.response.status <= 510)) &&
					failureCount <= 3
				);
			},
			retryDelay: attemptIndex => Math.min(1000 * 2 ** attemptIndex, 15000),
		},
		mutations:{
			retry: (failureCount: number, error: any) => {
				return (
					(!error.response ||
					error.response.status === 404 ||
					(error.response.status >= 500 && error.response.status <= 510)) &&
					failureCount <= 3
				);
			},
			retryDelay: attemptIndex => Math.min(1000 * 2 ** attemptIndex, 15000),
		}
	},
});

Expected behavior
That the retry for all queries works as expected.
How often does this bug happen?
Every time
Screenshots or Videos
No response
Platform
iOS, Android
Tanstack Query version
4.19.1
TypeScript version
4.9.4
Additional context
No response

my fault, the axios method returns a empty error on error


The Injectable API documentation is incorrect

See original issue posted here: https://github.com/angular/angular/issues/12098#issuecomment-251801431
The API documentation of Injectable says:

A marker metadata that marks a class as available to Injector for creation.
[...]
Injector will throw NoAnnotationError when trying to instantiate a class that does not have @Injectable marker

This is wrong: the injector needs Injectable only if the service itself has dependencies that must be injected. The API documentation should thus be updated.
The guide about DI, despite correctly explaining later when the annotation is really needed, repeats this incorrect documentation:

@Injectable() marks a class as available to an injector for instantiation. Generally speaking, an injector will report an error when trying to instantiate a class that is not marked as @Injectable().

This should be fixed as well, because it's misleading. The section just after just explains that what is written before is actually not true.

I think this is intentional.
According to the style guide (read months ago) @Injectable() should be added also when the constructor doesn't have parameters, because it often causes issues when later parameters are added but not also the @Injectable() decorator.
It's an easier and less error prone rule to just add @Injectable() to all classes that are injected.

I have no problem with that recommendation, which is a good one. But saying that it's needed to make it possible for the injector to create the service is still wrong and confusing, IMHO.

I agree, we should rewrite the DI guide.

Please raise your quarrel about the API documentation on angular. This repo is for issues about angular documentation.
In our defense, the DI guide is correct in saying:

@Injectable() marks a class as available to an injector for instantiation. Generally speaking, an injector will report an error when trying to instantiate a class that is not marked as @Injectable().

Generally speaking you'll get an error if you try to inject a service class without the @Injectable  because, generally speaking services have constructor dependencies and you will get an error if you neglect the decorator on a service with constructor dependencies.
Now that we have that out of the way, we can concede that the wording should be much better.
We intend to revisit the two DI chapters and fix many things ... including the wording around @Injectable.  We probably won't take up this particular issue before the rewrite.
I'll leave this open for now.


ERROR: for mlflow  Container "477acbc097fb" exited with code 124.

I'm trying to setup a mlflow server on my raspberry pi 4b with Ubuntu 22.10 but when doing docker-compose up -d the following error shows up.
Creating network "mlflow-docker_internal" with the default driver
Creating network "mlflow-docker_public" with driver "bridge"
Creating mlflow-docker_s3_1          ... done
Creating mlflow_db          ... done
Creating mlflow-docker_wait-for-db_1       ... done
Creating mlflow-docker_create_s3_buckets_1 ... done

ERROR: for mlflow  Container "477acbc097fb" exited with code 124.
ERROR: Encountered errors while bringing up the project.```

Minio server works good but MLFlow is not working at all

hi @helloerikaaa! Thanks for bringing this up.
Please send me your logs from docker compose logs. and docker compose logs mlflow.
Also please try to run it one more time:
docker compose down
docker compose up --build

I dont have a raspberry to test it there, but running docker compose on a fresh machine with windows 10 i've got this success logs.
PS C:\repo\mlflow-docker> docker compose logs mlflow
tracker_mlflow  | 2023/01/03 10:00:31 INFO mlflow.store.db.utils: Creating initial MLflow database tables...
tracker_mlflow  | 2023/01/03 10:00:31 INFO mlflow.store.db.utils: Updating database tables
tracker_mlflow  | INFO  [alembic.runtime.migration] Context impl MySQLImpl.
tracker_mlflow  | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade  -> 451aebb31d03, add metric step
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade 451aebb31d03 -> 90e64c465722, migrate user column to tags
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade 90e64c465722 -> 181f10493468, allow nulls for metric values
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade 181f10493468 -> df50e92ffc5e, Add Experiment Tags Table
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade df50e92ffc5e -> 7ac759974ad8, Update run tags with larger limit
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade 7ac759974ad8 -> 89d4b8295536, create latest metrics table
tracker_mlflow  | INFO  [89d4b8295536_create_latest_metrics_table_py] Migration complete!
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade 89d4b8295536 -> 2b4d017a5e9b, add model registry tables to db
tracker_mlflow  | INFO  [2b4d017a5e9b_add_model_registry_tables_to_db_py] Adding registered_models and model_versions tables to database.
tracker_mlflow  | INFO  [2b4d017a5e9b_add_model_registry_tables_to_db_py] Migration complete!
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade 2b4d017a5e9b -> cfd24bdc0731, Update run status constraint with killed
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade cfd24bdc0731 -> 0a8213491aaa, drop_duplicate_killed_constraint
tracker_mlflow  | WARNI [0a8213491aaa_drop_duplicate_killed_constraint_py] Failed to drop check constraint. Dropping check constraints may not be supported
by your SQL database. Exception content: (pymysql.err.ProgrammingError) (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'CHECK status' at line 1")
tracker_mlflow  | [SQL: ALTER TABLE runs DROP CHECK status]
tracker_mlflow  | (Background on this error at: https://sqlalche.me/e/14/f405)
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade 0a8213491aaa -> 728d730b5ebd, add registered model tags table
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade 728d730b5ebd -> 27a6a02d2cf1, add model version tags table
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade 27a6a02d2cf1 -> 84291f40a231, add run_link to model_version
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade 84291f40a231 -> a8c4a736bde6, allow nulls for run_id
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade a8c4a736bde6 -> 39d1c3be5f05, add_is_nan_constraint_for_metrics_tables_if_necessary
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade 39d1c3be5f05 -> c48cb773bb87, reset_default_value_for_is_nan_in_metrics_table_for_mysql
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade c48cb773bb87 -> bd07f7e963c5, create index on run_uuid
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade bd07f7e963c5 -> 0c779009ac13, add deleted_time field to runs table
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade 0c779009ac13 -> cc1f77228345, change param value length to 500
tracker_mlflow  | INFO  [alembic.runtime.migration] Running upgrade cc1f77228345 -> 97727af70f4d, Add creation_time and last_update_time to experiments table
tracker_mlflow  | INFO  [alembic.runtime.migration] Context impl MySQLImpl.
tracker_mlflow  | INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
tracker_mlflow  | [2023-01-03 10:00:33 +0000] [23] [INFO] Starting gunicorn 20.1.0
tracker_mlflow  | [2023-01-03 10:00:33 +0000] [23] [INFO] Listening at: http://<IP_ADDRESS>:5000 (23)
tracker_mlflow  | [2023-01-03 10:00:33 +0000] [23] [INFO] Using worker: sync
tracker_mlflow  | [2023-01-03 10:00:33 +0000] [24] [INFO] Booting worker with pid: 24
tracker_mlflow  | [2023-01-03 10:00:33 +0000] [25] [INFO] Booting worker with pid: 25
tracker_mlflow  | [2023-01-03 10:00:33 +0000] [26] [INFO] Booting worker with pid: 26
tracker_mlflow  | [2023-01-03 10:00:33 +0000] [27] [INFO] Booting worker with pid: 27

hi @Toumash thank you for your reply.
When doing docker compose up --build the message I get is the following:
Attaching to mlflow-docker-create_s3_buckets-1, mlflow-docker-s3-1, mlflow-docker-wait-for-db-1, mlflow_db, tracker_mlflow
mlflow_db                          | exec /entrypoint.sh: exec format error
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:02Z INF [TCP] Checking the db:3306 ...
mlflow-docker-create_s3_buckets-1  | Added `minio` successfully.
mlflow-docker-create_s3_buckets-1  | mc: <ERROR> Unable to make bucket `minio/mlflow`. Your previous request to create the named bucket succeeded and you already own it.
mlflow_db exited with code 0
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:05Z ERR Expectation failed error="timed out while making a tcp call, caused by: dial tcp <IP_ADDRESS>:3306: i/o timeout" timeout=3s
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:05Z INF [TCP] Checking the db:3306 ...
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:05Z ERR Expectation failed error="failed to establish a tcp connection, caused by: dial tcp: lookup db on <IP_ADDRESS>:53: server misbehaving"
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:05Z INF [TCP] Checking the db:3306 ...
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:05Z ERR Expectation failed error="failed to establish a tcp connection, caused by: dial tcp: lookup db on <IP_ADDRESS>:53: server misbehaving"
mlflow-docker-create_s3_buckets-1 exited with code 0
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:06Z INF [TCP] Checking the db:3306 ...
mlflow_db exited with code 0
mlflow_db exited with code 1
mlflow_db exited with code 1
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:09Z ERR Expectation failed error="timed out while making a tcp call, caused by: dial tcp <IP_ADDRESS>:3306: i/o timeout" timeout=3s
mlflow_db exited with code 1
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:09Z INF [TCP] Checking the db:3306 ...
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:09Z ERR Expectation failed error="failed to establish a tcp connection, caused by: dial tcp: lookup db on <IP_ADDRESS>:53: server misbehaving"
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:09Z INF [TCP] Checking the db:3306 ...
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:09Z ERR Expectation failed error="failed to establish a tcp connection, caused by: dial tcp: lookup db on <IP_ADDRESS>:53: server misbehaving"
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:09Z INF [TCP] Checking the db:3306 ...
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:09Z ERR Expectation failed error="failed to establish a tcp connection, caused by: dial tcp: lookup db on <IP_ADDRESS>:53: server misbehaving"
mlflow-docker-wait-for-db-1        | 2023-01-03T19:34:10Z INF [TCP] Checking the db:3306 ...

It's the same log if I use the command docker compose logs, if I try to use docker compose logs mlflow it returns nothing.

So the root problem is with the database not booting.
I've googled the issue and it looks like it could be the problem with mysql not being compatible with your processor's architecture which is ARM.
I've found a project that compiled mysql 5.5 onto arm platform. I would suggest to give it a try. Try to swap
    image: mysql/mysql-server:5.7.28

with
    image: hypriot/rpi-mysql

The other thing i've found out is that the mysql now has support for arm v8 in the latest, 8 version
image: mysql:8-oracle

Source: https://hub.docker.com/layers/library/mysql/8-oracle/images/sha256-cfddf275c8b1ae1583c0f6afb4899d4dbe14111a6462699559a1f4dc8f4d5f6e?context=explore.
@helloerikaaa could you please check if the following docker-compose.yml works for you? Its starting up ok, but i cant see to get mlflow/python/conda working on my windows machine.
version: "3.9"
services:
  s3:
    image: minio/minio:RELEASE.2021-11-24T23-19-33Z
    restart: unless-stopped
    ports:
      - "9000:9000"
      - "9001:9001"
    environment:
      - MINIO_ROOT_USER=${AWS_ACCESS_KEY_ID}
      - MINIO_ROOT_PASSWORD=${AWS_SECRET_ACCESS_KEY}
    command: server /data --console-address ":9001"
    networks:
      - internal
      - public
    volumes:
      - minio_volume:/data
  db:
    image: mysql:8-oracle
    restart: unless-stopped
    container_name: mlflow_db
    expose:
      - "3306"
    environment:
      - MYSQL_DATABASE=${MYSQL_DATABASE}
      - MYSQL_USER=${MYSQL_USER}
      - MYSQL_PASSWORD=${MYSQL_PASSWORD}
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
    volumes:
      - db_volume:/var/lib/mysql
    networks:
      - internal
  mlflow:
    container_name: tracker_mlflow
    image: tracker_ml
    restart: unless-stopped
    build:
      context: ./mlflow
      dockerfile: Dockerfile
    ports:
      - "5000:5000"
    environment:
      - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
      - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
      - AWS_DEFAULT_REGION=${AWS_REGION}
      - MLFLOW_S3_ENDPOINT_URL=http://s3:9000
    networks:
      - public
      - internal
    entrypoint: mlflow server --backend-store-uri mysql+pymysql://${MYSQL_USER}:${MYSQL_PASSWORD}@db:3306/${MYSQL_DATABASE} --default-artifact-root s3://${AWS_BUCKET_NAME}/ --artifacts-destination s3://${AWS_BUCKET_NAME}/ -h <IP_ADDRESS>
    depends_on:
      wait-for-db:
        condition: service_completed_successfully
  create_s3_buckets:
    image: minio/mc
    depends_on:
      - "s3"
    entrypoint: >
      /bin/sh -c "
      until (/usr/bin/mc alias set minio http://s3:9000 '${AWS_ACCESS_KEY_ID}' '${AWS_SECRET_ACCESS_KEY}') do echo '...waiting...' && sleep 1; done;
      /usr/bin/mc mb minio/${AWS_BUCKET_NAME};
      exit 0;
      "
    networks:
      - internal
  wait-for-db:
    image: atkrad/wait4x
    depends_on:
      - db
    command: tcp db:3306 -t 90s -i 250ms
    networks:
      - internal
networks:
  internal:
  public:
    driver: bridge
volumes:
  db_volume:
  minio_volume:


Flag to Prevent Villager Trading

I've asked this on WorldGuard Github with no response since June.
Looking to see what you guys have to say about this one!
My players complain that people sneak into their bases and use their villagers to trade with. In a server where you use villager trades for your player-shop for economy reasons this is a way of stealing from someone's villagers they worked hard to get good trades from.
I'm amazed that to this day neither WG or WGExtraFlags has this option. I've had to resort to using deny-entry flags on everyone's villager area and I don't think that should be necessary. I hope this is something simple that can be added.
Essentially deny interacting with a villager, no other flags work for this I've tried everything and looked everything up.

Any change this might get implemented here? The fork mentioned by @Invvk is discontinued.

In general this would require hooking to WorldGuards internals to make it work nicely with other flags. Restricting areas where people can go is easy way to mitigate it. If you want to globally disable it then there are plugins for it. For specific areas where you have control over the villagers, you can just set the offers list to empty.

I was thinking more like not letting player Mark trade with the villagers in player Josh’s region unless he was granted access to that region (member, owner).
Kinda like making the trading halls private and exclusive to the players that own them.

That would fall under the restricting areas part. You can easily block off the area to only allow teleporting inside it. Or you can use doors or buttons and redstone to achieve that. Interacting with item frames requires build access to the region so thats one way to achieve more strict access.

Any player can visit this room as a guest and use sethome unnoticed. It is also possible to use a pearl.
I made my first server 7 years ago and even then I didn't understand why there was no such function. But I never wrote about it, because it is easy to implement with a mini-plugin - it was simpler and faster :)


Previous query not erased when closing "PowerToys Run" using Alt+Space command ("Erase previous query" option enabled)

Microsoft PowerToys version
0.49.1
Running as admin

[ ] Yes

Area(s) with issue?
PowerToys Run
Steps to reproduce

Check "Clear previous query on launch" in PowerToys Run menu.
Press Alt + Space and type something such that no results are shown.
Hide the Search Bar by pressing Alt + Space / Esc / Clicking somewhere else.
Press Alt + Space again and the previous query will be still there.

✔ Expected Behavior
Expected the previous query to be erased even through it showed no results.
❌ Actual Behavior
The previous query is not erased if the typed matter shows no results.
Other Software
N.A.

Failed to reproduce this. Can you try exiting PowerToys from the tray icon, check your settings again, and see if the issue persists.

Failed to reproduce this. Can you try exiting PowerToys from the tray icon, check your settings again, and see if the issue persists. Thank you!

I will attach a video about i
https://user-images.githubusercontent.com/91146818/141606106-77e79e91-6e58-402e-abeb-3d841de11693.mp4
t.

Could you attach a full bug report file so we can investigate this?
Right-click on the tray icon and click Report Bug. Then drag-and-drop the generated file into the GitHub comment to upload it. Thanks!

PowerToysReport_2021-11-13-14-25-33.zip
Here's the zip folder. Thanks

I've been unable to replicate this, as well.
Nothing weird on the logs, as well.
The way we're currently doing this seems to be by cleaning the search when we close PowerToys Run. A more resilient solution might be to clean it when we activate it again as well.
It's weird that it gets cleaned for your case when using Esc but not the shortcut.

Thank You for looking into this issue 😄

I notice that it happens if the query contains at least one ` (backtick symbol) or the input are only whitespaces or there is a whitespace between text in the query. #14642

No, there's neither a ` nor a whitespace, just "hello" is what I've typed

In my case, it's different. But I closed it since it's a similar bug report (#14642) to yours. I just added the details of mine here in case the bug report I sent will be ignored due to its closure.
Does this occur always when the query is "hello"?

Not just hello, that was just for example. It happens no matter what I type in there.

I've been unable to replicate this, as well. Nothing weird on the logs, as well.
The way we're currently doing this seems to be by cleaning the search when we close PowerToys Run. A more resilient solution might be to clean it when we activate it again as well. It's weird that it gets cleaned for your case when using Esc but not the shortcut.

This can easy be repro with a search query with no results.
Regarding the current solution for clear the last search look at https://github.com/microsoft/PowerToys/pull/10291
Is far from perfect but was even worse in the past.
That said, the OP issue can be easy fixed.

Not just hello, that was just for example. It happens no matter what I type in there.

Does the query get cleared if the search have visible results?

Does the query get cleared if the search have visible results?

Oh yes it does!

Investigating a fix for this issue

This is fixed in 0.53, get it at https://aka.ms/installPowerToys


items grid card url missing prefix

What happened?
ulr prefix missing when clicking on item image cards

Please reproduce in steps
check video
Expected Behavior
prefix should be there
What browsers are you seeing the problem on?
No response
At which address did you encounter bug?
No response
Are you logged in?
None
Which wallet you are using?
No response
At which chain did you encounter bug?
No response
Screenshots
https://github.com/kodadot/nft-gallery/assets/44554284/604d3efa-6fb3-4967-90af-44b561537179
Relevant log output
No response
Payment link for reward
https://kodadot.xyz/ksm/transfer/?target=<My_Kusama_Address_check_https://github.com/kodadot/nft-gallery/blob/main/CONTRIBUTING.md#creating-your-ksm-address>
Code of Conduct

[X] I agree to follow this project's Code of Conduct

👋

@yangwao any way of increasing the limit ? , maybe if you x amount of pr merged .


Benefits of using electron-serve in Angular

Hi,
I am currently developing an Electron application that renders an Angular application using standard win.loadURL('file://…'). It uses the router module from Angular internally for routing tasks and it works ok. What is the advantage of using this library over the standard approach?
Thanks

I don't know anything about Angular development, but I assume the router module is using hashbang instead of real URLs then? That might come with certain downsides.
For example, in the React world, the downsides are:

IMPORTANT NOTE: Hash history does not support location.key or location.state. In previous versions we attempted to shim the behavior but there were edge-cases we couldn't solve. Any code or plugin that needs this behavior won't work. As this technique is only intended to support legacy browsers, we encourage you to configure your server to work with  instead.
- https://github.com/ReactTraining/react-router/blob/master/packages/react-router-dom/docs/api/HashRouter.md

@sindresorhus :pray: for the clarification.


Request: Graphite backend

I see there's a statsd backend which is awesome. Requesting a graphite backend so that I can set my carbon-relay to stream metrics to bluefood in the same way we can stream to blueflood-statsd.
Would really allow us (and I'm sure many others) to hook in to blueflood much easier.
We have too many metrics going to graphite (not via statsd) right now that switching to blueflood is going to be super difficult and time consuming. A graphite backend would make this process much easier to get started, allowing us to slowly convert everything else in the future, while still getting the immediate benefits.

Hey Nick,
I know you already know this, but we have a carbon-relay that we're working on:
https://github.com/rackerlabs/graphite-http-wrapper
It will be single-tenant at the moment, but should be a good way for people who already employ graphite to simply push their data into Blueflood.

Following up: We've tested the graphite-http-wrapper and it works great!
Our graphite node is relaying metrics to a graphite-http-wrapper node, which then injects them in to blueflood. We can then see our metrics in blueflood via grafana. Everything's working very well for our needs. Thanks so much!
From my perspective, I think we can close out this issue, unless you still need it open on your side?

Sounds great!  I'll close this issue for now.


Fix 'H' timeframe support

Fix to support "H" timeframes.
The issue is explained in #114.

Thanks! We're organizing this in a separate branch which is why we did not include in this version but this is super helpful.

@lenak25 this looks good to me so I will be merging into develop and test. Please take a look as well on Sunday.

Thanks @vonpupp for your PR!
This indeed seems to fix the live case.
We will need to apply some adjustments to address this issue in backtest as well.
Thanks again!

Thank you @lenak25.
Yes, I am aware of such things, however I handle them internally on an intermediate layer I made for my algos on top of Catalyst. Unfortunately this layer doesn't fit within Catalyst itself, otherwise I would PR it also.


"abilities" do not have descriptions

"Abilities" have a "description" field but it is always blank.

Again, V1 is deprecated so we won't be adding data to V1.


Update matchers.md: include and namespace instructions

Added a note on which header to include to use the built-in matchers to the top of the page.
Moved a sentence about the namespace to the top of the page.

Fixes #4604

Marchers.md was just an example. You need to do this for all of the files in the documentation, and double check that the name spaces are also specified.


[MM-25833] - Add button common component

WIP since this is a part of a larger feature, will be tested by PM and QA when we have linked it all together.
Summary
As part of our new Menu component, and preparation for the Compass design system, this PR adds one of the subelements of the Menu, the Button Common component. It renders the Icon Common component and Label common component (to be added). Structure is:

leftIconGlyph: the name of the desired left icon glyph, using constants (eg. Icon.EMOTICON_HAPPY_OUTLINE)

rightIconGlyph: the name of the desired right icon glyph, using constants (eg. Icon.EMOTICON_HAPPY_OUTLINE)

style: the desired style of the Button (primary, secondary, tertiary), using constants (eg. Button.PRIMARY)

size: the desired size of the Button (small, standard, large), using constants (eg. Button.SMALL)

label: text to use for button label and accessibility, uses aria-label

link: a string value that renders the button with an anchor tag instead of the default button tag

fullWidth: a boolean value that stretches the button to the full width of it's parent container

disabled: a boolean indicating whether the Button should be disabled or not

Snapshots should run when we have the main element created. More tests will be added after components are merged together
Ticket Link
https://mattermost.atlassian.net/browse/MM-25833
Related PR's
#5728
#5732
#5730

By the way, the WIP label prevents the PR from being merged, so it's not needed if you're expecting that this will be reviewed by PMs after it's merged into the feature branch

A few things below and the following:

We still need to add the constants for the acceptable values for style and size. Exported typescript string enums maybe?
We need to talk about how we are going to actually import the css (global , module etc.)

Didn't actually intend to create suggestions for each time a css variable was used, but I got partway through forgetting there were so many and just kept going. 😛

I agree, I mentioned in the comment above to have a doc shared with UX for all the variables? What do you think? I can schedule a call with @andrewbrown00
I like the idea of a module. It keeps things into their own folders. What do you think?
That's a good start into the naming though :)

I agree, I mentioned in the comment above to have a doc shared with UX for all the variables? What do you think? I can schedule a call with @andrewbrown00

Hey @nevyangelova, I'd be happy to hop on a call and talk through this...

I agree, I mentioned in the comment above to have a doc shared with UX for all the variables? What do you think? I can schedule a call with @andrewbrown00

Hey @nevyangelova, I'd be happy to hop on a call and talk through this...

Thanks, I will schedule a call with you and @deanwhillier when we are done with sidebar and release work.

Closing as we are restarting this effort with a slightly different approach


API endpoint for session staff details

I'd like an API endpoint which will return the session chairs and runners. Whatever is the absolute simplest to implement which I can cross reference against the conference.json feed:
GET /schedule/session-staff.json
[
    {
        "conf_key": 123, // fk into conference.json
        "chair_name": "Jane Smith",
        "chair_email"<EMAIL_ADDRESS>        "runner_name": "John Doe",
        "runner_email"<EMAIL_ADDRESS>    },
]

Since it contains volunteer names and emails, it should only be available to staff / superusers.
Use case: I have a script which generates a green-room schedule. This API endpoint will let me include the runner names and email addresses in that printout.

Duplicate of #390.


xtensa/esp32[|s2|s3]: Fix task backtrace dump

Summary

Documentation: Document stack and backtrace dump for Espressif SoCs

Stack and backtrace dump for Espressif's SoCs (ESP32, ESP32-S2, ESP32-S3, ESP32-C3, ESP32-C6 and ESP32-H2) is now documented in a new section for each chip entry page.

xtensa/esp32[|s2|s3]: Fix task backtrace dump

Fix MAKE_PC_FROM_RA macro to consider the instruction region base address;
Add sanity check for calculated PC and SP registers;
Check if the stack pointer is within the interrupt stack to enable backtrace dump if an exception occurs during the ISR;
Update the script for decoding the backtrace dump to consider the syntax of the backtrace format and properly handle the dump when SMP is enabled;

xtensa/Kconfig: Fix dependency for backtrace dump on Xtensas

CONFIG_XTENSA_INTBACKTRACE is necessary to enable backtrace dump for the tasks because exceptions are treated like interrupts (even when an exception occurs during a normal task execution). It's now automatically selected when CONFIG_SCHED_BACKTRACE is enabled. This commit also removes outdated Kconfig options.

Impact
Fix backtrace dump on assertions considering assertions triggered by tasks or during an interrupt service.
Testing
Internal CI testing with HW on all Xtensa-based devices.
How to Test on ESP32:
Use esp32-devkitc:nsh. Enable CONFIG_SCHED_BACKTRACE and CONFIG_SYSTEM_DUMPSTACK. On nsh, run dumpstack:
nsh> dumpstack
sched_dumpstack: backtrace| 2: 0x400ee9bc 0x400ef817 0x400e2d0b 0x400e1d63 0x40000000

Save this output to a file and use ./tools/espressif/btdecode.sh esp32 /tmp/backtrace.txt to check the backtrace dump:
./tools/espressif/btdecode.sh esp32 /tmp/backtrace.txt
Backtrace for task 2:
0x400ee9bc: sched_dumpstack at ??:?
0x400ef817: dumpstack_main at ??:?
0x400e2d0b: nxtask_startup at ??:?
0x400e1d63: nxtask_start at ??:?
0x40000000: ?? ??:0

Before this PR, the output would generate wrong addresses:
Backtrace for task 2:
0x000ee998: ?? ??:0
0x000ef7f3: ?? ??:0
0x000e2ce7: ?? ??:0
0x000e1d3f: ?? ??:0

[Experimental Bot, please feedback here]
Squash The Commits: This PR contains 3 Commits. Please Squash the Multiple Commits into a Single Commit.
NuttX PR Requirements Review
This PR appears to meet the NuttX requirements. Here's a breakdown:
Summary:

Clear explanation of the change: The summary clearly states the purpose of the change, addressing a fix for the backtrace dump functionality specifically for Espressif SoCs.
Functional area:  The affected area is clearly identified (stack and backtrace dump for xtensa/esp32).
Mechanism:  The summary outlines how the change works, including fixes to the MAKE_PC_FROM_RA macro, sanity checks, and script updates.
Issue references: While not explicitly mentioned, the summary implies addressing an existing issue with backtrace dumps on the mentioned architectures. It would be beneficial to add specific issue references if applicable.

Impact:

Feature change: This PR focuses on fixing an existing feature (backtrace dump), not introducing a new one.
User impact: No user-facing changes are described, implying minimal impact.
Build impact: No build process changes are mentioned.
Hardware impact: The changes are specific to Xtensa-based devices, specifically ESP32 and its variants, which is clearly stated.
Documentation impact: The summary explicitly mentions updating documentation for each affected chip.
Security impact: No security implications are mentioned.
Compatibility impact:  No backward or forward compatibility issues are mentioned.

Testing:

Verification: The author confirms testing on local setup with positive results.
Host and Target:  While "Internal CI testing" is mentioned,  listing specific host OS, compiler versions, and target boards used for testing would strengthen the validation.
Logs: The PR provides clear steps to reproduce the issue and verify the fix on an ESP32. Including snippets of actual logs before and after the change would further enhance the testing section.

Overall:
This PR demonstrates a good understanding of NuttX requirements.  Adding specific issue references and detailing the testing environment would further strengthen it.

@xiaoxiang781216 , I could not find a reason for arm-05 being failing...

@xiaoxiang781216 , I could not find a reason for arm-05 being failing...

here is fix: https://github.com/apache/nuttx/pull/13548

@xiaoxiang781216 , I could not find a reason for arm-05 being failing...

here is fix: #13548

Thanks. Rebased.


docs: search functionality partly broken

Documentation:
https://katex.org/docs/api.html
Type an arbitrary search phrase into the search input field, e.g. auto-render.
A list with 5 occurrences of àuto-render` is presented. Click on the link given in the first row. This forwards you to a non existing page:
https://katex.org/docs/0.10.1/autorender.html#docsNav

Thank you for the report! We're using an external search engine, Algolia DocSearch, and since we removed versioned docs, it seems it couldn't index our documentation. I've sent a pull request at https://github.com/algolia/docsearch-configs/pull/3932.

The issue should be resolved now. Thanks again.


ContainerRegistry tag for docker hub does not work

Hi,
Regarding authentication against DockerHub
when trying for example
 dotnet publish --os linux -c Release -p:PublishProfile=DefaultContainer **-p:ContainerImageName=hellodocker-api-modern** -p:ContainerImageTag=1.0.1
login to docker hub using the 1st option  "registry.hub.docker.com" is not sufficient
The ContainerImageName should contain the login/ContainerImageName
So maybe this needs to be indicated in the authentication part of this repo to help anyone having issues

Thanks, this is a good call-out. I've opened #420 to add a note on this. This is a fairly common need, though - for example Github Packages has the same requirement.


Fixed issues with Mamba finetune pipeline

PR Type
Fix
Short Description
Updated the Mamba finetune pipeline
Tests Added
...

@amrit110 Please take a look when you have the time!


Fix issue arrow function with type

Fixes https://github.com/vinissimus/next-translate/issues/459
cc: / @vimutti77

It works 🎉 🎉 🎉

It works 🎉 🎉 🎉

Thanks for the feedback @vimutti77 !! 😊

Thanks for the feedback @vimutti77 !! 😊


State Type

I am using class as my state. However Verge says, you cannot use class type for state type. Can we make it?

Thank you, I get what you mean, I will change state as value type.


add tests for routes

created "tests" branch.                 TESTS
/
FEATURES/UI - MODEL/UI
/
MAIN -> DEVELOP            STATIC/UI

PARALLAX/PLAYGROUND

Test files can be removed


Added Java JDBC topic

@Janwilx727  Thank you for this, can I ask that you add code examples like below,
JDBC templates
long id = 1;
string name = "umuzi"
jdbcTemplate.update("insert into User (id, name) values(?,?)", new Object[]{id,name});

JPA equivalent
user = new User(2, "umuzi");
userRepository.save(user);

Also all of this which you shared with me on slack
Both Springboot and VanillaJS require a middle man to be able to talk to a database.
So JDBC can be used with either
However, I believe that Javascript applications might make more use of ODBC and JDBC, I suppose it's interchangeable.
Springboot however, I believe is more reliable on JDBC becuase it has no way to communicate with a database by itself, it needs to communicate via JDBC driver

Did some small modification, looks awesome thank you


Entities reference guide

Signed-off-by: Nikki Attea<EMAIL_ADDRESS>Reference guide documentation for Entities.
Closes #198

LGTM


feat(middleware): v4 experimental middleware

@jptosso Care to take a look at comments?

And provide a description about what this PR does?


Default sorting

Hi! Just implemented this project on my self hosted website in an evening and I love it! There's one thing that I kind of miss: default sorting. Now the sorting is new->old (as far as I can see) but this does not work for the pictures of my wedding. I would like to show them old->new. Is this something that I could easily add?
Thanks!
Jesse

Hey there! I am afraid this currently is not an option. You could enable ALLOW_ORDER_SORT_CHANGE though which will show a button so your wedding guests can switch the sorting order on their own.
If you don't mind a little hack, you could even try changing the /_includes/javascript.html file and add a few lines right before the lazyload(); call like that:
      document.addEventListener('load', (event) => {
          reverseSorting();
      }

Just an idea and not tested!
Anyway, this sounds like a nice and relatively simple feature to be added in the future.

Thanks! I'll try that and report back!

That almost works, with a small edit it works fine!
I added this on line 177:
  document.addEventListener('load', (event) => {     reverseSorting();   }, true);
Thanks again!

#43 is merged and will be included in new releases.


Support for UDT (hierarchyid, geometry and geography)

Adds support for hierarchyid, geometry and geography SQL type.
Currently a panic is thrown when the system tries to scan any UDT (user defined type)
This has been brought up many times (e.g. 167, 79)
Doesn't make sense to have a official driver that doesn't support all official SQL types.
There are some forks such as this one to try to solve this issue and the solutions seems to work correctly for every case.
I have tested the solution for all proposed types and have implemented a library to handle hierarchyid encoding and decoding in Go, that has been developed with the modified driver.

@microsoft-github-policy-service agree

Codecov Report
Attention: Patch coverage is 0% with 17 lines in your changes missing coverage. Please review.

Project coverage is 74.54%. Comparing base (9b84d9b) to head (e29c4cf).

Files
Patch %
Lines

types.go
0.00%
17 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #216      +/-   ##
==========================================
- Coverage   74.79%   74.54%   -0.25%
==========================================
  Files          32       32
  Lines        6379     6396      +17
==========================================
- Hits         4771     4768       -3
- Misses       1325     1344      +19
- Partials      283      284       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

I agree the driver shouldn't panic and hierarchyid seems useful in non-CLR uses. What are the non-CLR use cases for geometry values not converted to string?

Thank you!
Geometry and geography are useful for example for spatial filtering. for example if you define the geometries for houses you can esily get wich house is at a location X, Y (by intersection) or for example get the closest house to a point etc.
Geography and geometry are similar the main diference between them is that geometry is a more generic for any 2D in a generic X, Y referential and geography restricts its usage to earth coordinates taking account the earth curvature the hemispheres etc

I agree the driver shouldn't panic and hierarchyid seems useful in non-CLR uses. What are the non-CLR use cases for geometry values not converted to string?

Thank you!
Geometry and geography are useful for example for spatial filtering. for example if you define the geometries for houses you can esily get wich house is at a location X, Y (by intersection) or for example get the closest house to a point etc.
Geography and geometry are similar the main diference between them is that geometry is a more generic for any 2D in a generic X, Y referential and geography restricts its usage to earth coordinates taking account the earth curvature the hemispheres etc

I meant - how does a Go app consume the binary values? AFAIK those are serialized CLR objects. Maybe I've misunderstood that.

I meant - how does a Go app consume the binary values? AFAIK those are serialized CLR objects. Maybe I've misunderstood that.

Yes i belive so, there are at leat two libs to support serialization of these in go go-mssqlclrgeo and  go-geom


Allow user to replay events starting from a specific replay ID

Currently the user can play upcoming or all past events.
It would be nice to be able to replay events starting from a specific replay ID.

Implemented in v3.3.0.


OLS-508: Ability to control collector via file existance

Description
Ability to control collector via file existence
Type of change

[x] New feature

/retest

/retest

/retest

/retest

/retest

/lgtm
/approve


Initialize set of circular dependencies to be folded with those from self

There may be a case in which nixfromnpm has detected a circular dependency for a package but circularClosure turns up empty handed. This likely means nixfromnpm is incorrectly detecting a circular dependency.
However, when this happens it means the package may not have a required dependency because circularDepClosure winds up with an empty set if circularClosure doesn't find its cycle.
Therefore, due to the way the circular dependency resolution works, it is technically safe to resolve a circular dependency that may not be circular by starting the fold function over the final closure with the the circularDependencies of self.

@adnelson it was for the webpack-3.11.0 npm package. I will create an issue ticket mentioning this and I'll try to reproduce in isolation.


Add Spark-3.1 trouble shooting

       ❨?❩

Why are the changes needed?

More details could see the config spark.sql.legacy.setCommandRejectsSparkCoreConfs
How was this patch tested?

this needs to be fixed. a trouble-shooting is necessary but not enough. for the first session that creates the engine, it's very like that the queue will be passed here

Yes, I considered to fix it, but it's a little hard to list the total illegal configs(need to collect the config in all module). We could fix it in next major release since it's have enough time.

let's create an issue to track this first


Toolbar icons should not be bold

Bold styling is distracting and draws excess and unnecessary visual attention to the toolbar.

Yup, I smooth it a notch:

Currently, deployed.


Cannot get the data out of FormData object

I want to perform the test against the FormData object but it seems that there is no FormData.get method available.
I read the FormData api from here https://developer.mozilla.org/en/docs/Web/API/FormData. So I wonder why doesn't it has get method. How can I get the data from the FormData object?

The #get method for FormData has just been recently added to chrome and firefox. I've updated https://github.com/form-data/form-data/issues/124 which strives for api symmetry with the browser to include these new methods. You can track progress there.

Thank you @DylanPiercey for informing. Your lib is very useful for Node. I will be looking forward to see the new release. :)

hello all, when do you think it will be possible to do the "get" method to extract data from the FormData?


Could the mapper be pure?

Looking at TreeMapper, I wonder if we can mark map() as @psalm-pure, allowing us to force the design of all internal callbacks (specifically userland ones) to not perform I/O or side-effects in general.
This would greatly improve the safety of the mapper, even if it is reduced in scope.

Sounds like a great idea indeed.
I'm currently using it in a customer's project to fetch entities directly from the DB, but I was already asking myself if I was not abusing the library's capabilities…
I'll take some time to think about it. Also, I'd love to have comments/feedback from other users and contributors on this suggestion; I'll allow myself one ping, hope that's ok with you guys 😅  — @Mopolo @ @Slamdunk @brandonsavage @brayniverse @shulard (others are obviously more than welcome to join the discussion!)

Hi @romm, I'm not using this library on any active projects at the moment but I appreciate the consideration. I have no objections to map() being marked as @psalm-pure.


Consider replacing std::optional by const url*

A pointer can default on null.

This is already completed. Closing.


http.Get do not redirect

$ go version
go version go1.16.3 windows/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\aaa\AppData\Local\go-build
set GOENV=C:\Users\aaa\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\aaa\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\aaa\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.16.3
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=NUL
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\aaa\AppData\Local\Temp\go-build848562969=/tmp/go-build -gno-record-gcc-switches

What did you do?
func Download(url, dir string) error {
    resp, err := http.Get(url)
    if err != nil {
        return err
    }
    defer resp.Body.Close()
    filename := filepath.Base(url)
    filepath := path.Join(dir, filename)
    out, err := os.Create(filepath)
    if err != nil {
        return err
    }
    defer out.Close()
    _, err = io.Copy(out, resp.Body)
    return err
}

url  := "https://prdownloads.sourceforge.net/expat/expat-2.2.10.tar.xz"
Download(url, ".")

What did you expect to see?
I expect tar.xz content downloaded
What did you see instead?
html content is downloaded because web has <meta http-equiv="refresh" content="5; ..."> and http.Get do not redirect.

http-equiv is part of html and not http. also the client cannot read the request body.
Closing as working as intended


ExportTableTest::CheckContent Does Not Provide Useful Feedback

The export_table_test fails randomly on Jenkins. Thanks to @crd477, we can actually see the debug output now for the failed test:
http://jenkins.db.cs.cmu.edu:8080/blue/organizations/jenkins/terrier/detail/PR-938/54/tests/
This says that we are failing ExportTableTest::CheckContent. This function is responsible for reading the data back in from the exported file to see whether it matches what we expect:
https://github.com/cmu-db/terrier/blob/master/test/storage/export_table_test.cpp#L268
The problem, however, is that the test only checks whether CheckContent returns true or false. There is no useful feedback to say which of the checks inside of CheckContent is failing.
We need to refactor this function so that the asserts are inside of CheckContent rather than just returning false for the entire function invocation. This will help us begin to understand why this test case is failing randomly.

Happened to see this issue -- sorry for not providing enough debuggability :(. If you don't mind it I'll refactor the code and try to reproduce the failure to see what went wrong.

@YuzeLiao That would be great! Just let us know if you don't have the time for it so that I can assign it to another student.

Fixed in #1019


High memory usage after hours of use

I just let probeSniffer.py running for a few hours (4 hours I think) and when I return I found out that 3/4(three quarter) of my main memory (8GB) was fully used.
I killed probeSniffer.py (with CTRL-C) and I was able to get that memory back, freeing it.
Is there some memory leak, or is it the expected behavior given that it runs for hours ?

Hey!
Thats weird... Thank you for reporting, I will look into it.
Were there a lot of requests/devices when your memory got filled up?

I got a total of 127 logged mac address/ssid during those 4 hours but I modifed the script to log duplicates. So it's not that much I would say. I only got 18 unique mac_address/ssid
Note: I copied the lines 219-221 and pasted them after line 228
If I run the script and run htop I can see the memory increasing from the start of the script

Sorry for the late reply but I belive I fixed the memory leak issue in commit b8f8612b9431a42a151edc58285c95a3be6a6acf.
Please try it and see if the issue is fixed for you.


Fill or kill orders stuck in executable state

I have been placing FILL_OR_KILL orders through Flumine and noticed that the order stream update for setting the order status to "Execution complete" is sometimes severely delayed. A FILL_OR_KILL order is either matched instantaneously or rejected, so it should go straight from "Executable" to "Execution complete". Normally, it looks like this, and one can see that the delta between the last two updates is 52 ms:
Time                       Selection ID  Bet ID        Order status        Side      Matched    Remaining  Trade status
-----------------------  --------------  ------------  ------------------  ------  ---------  -----------  --------------
2023-08-13 10:35:25,142        55258054  None          Pending             BACK         0            2     Live
2023-08-13 10:35:25,340        55258054<PHONE_NUMBER>80  Executable          BACK         2            0     Pending
2023-08-13 10:35:25,392        55258054<PHONE_NUMBER>80  Execution complete  BACK         2            0     Live

However, for a sizeable proportion of the time, the "Execution complete" status update is received several minutes later, when the market gets resulted. Note that the bet is already fully matched when it reaches the status of "Executable", but it takes almost 9 minutes for the "Exection complete" update to arrive.
Time                       Selection ID  Bet ID        Order status        Side      Matched    Remaining  Trade status
-----------------------  --------------  ------------  ------------------  ------  ---------  -----------  --------------
2023-08-13 10:11:58,988        58973557  None          Pending             BACK            0            2  Live
2023-08-13 10:11:59,296        58973557<PHONE_NUMBER>67  Executable          BACK            2            0  Pending
2023-08-13 10:20:39,839        58973557<PHONE_NUMBER>67  Execution complete  BACK            2            0  Live

The issue has been reproduced on other machines and traced down to https://github.com/betcode-org/flumine/blob/master/flumine/order/process.py#L88. The problem is that "order.bet_id is None with status PENDING, current_order.bet_id is set and status is EXECUTION_COMPLETE."

I have just done an experiment with placing FOK bets and they immediately go to EXECUTION_COMPLETE in the order stream, regardless of whether they are matched or not. Here is the data. The first order is matched, the second is cancelled ("sm"/"sc"). They both go straight to "EC" for status, which stands for EXECUTION_COMPLETE.
{'id': '1.216407624',
 'orc': [{'id': 39674645,
          'uo': [{'id': '317109322476',
                  'p': 1.94,
                  's': 1,
                  'side': 'L',
                  'status': 'EC',
                  'pt': 'L',
                  'ot': 'L',
                  'pd':<PHONE_NUMBER>000,
                  'md':<PHONE_NUMBER>000,
                  'avp': 1.94,
                  'sm': 1,
                  'sr': 0,
                  'sl': 0,
                  'sc': 0,
                  'sv': 0,
                  'rac': '',
                  'rc': 'REG_GGC',
                  'rfo': '',
                  'rfs': ''}],
          'ml': [[1.94, 1]]}]}
{'id': '1.216407624',
 'orc': [{'id': 39674645,
          'uo': [{'id': '317109323812',
                  'p': 1000,
                  's': 1,
                  'side': 'B',
                  'status': 'EC',
                  'pt': 'L',
                  'ot': 'L',
                  'pd':<PHONE_NUMBER>000,
                  'sm': 0,
                  'sr': 0,
                  'sl': 0,
                  'sc': 1,
                  'sv': 0,
                  'rac': '',
                  'rc': 'REG_GGC',
                  'rfo': '',
                  'rfs': '',
                  'cd':<PHONE_NUMBER>000}]}]}

In the example in the first post, the order was fully matched, so both the order stream and order placement report should have returned EXECUTION_COMPLETE status. Under no circumstances should this order ever be in EXECUTABLE state.


Fixed expected error text in test case

Description
Fixed expected error text in failing test case
Issues Resolved
Closes #78
Check List

[x] New functionality includes testing.

[x] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has documentation added

[x] Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Hi @zethuman, this is fixed in PR #84 and merged to main.

Fixed in another PR


BatchWriteError Error when dy import

I'm encountering an issue while trying to import data into a table using Dynein. The process starts, but eventually fails with a BatchWriteError. It seems like the problem might be related to the aws-sdk-rust. Below are the details of the attempts and the errors received.
$ dy import -t <table_name> -i import_data.json
WARN: For the best performance on import/export, dynein recommends OnDemand mode. However the target table is Provisioned mode now. Proceed anyway?
[WARN: For the best performance on import/export, dynein recommends OnDemand mode. However the target table is Provisioned mode now. Proceed anyway? yes
8350 items processed (1201.56 items/sec)Error: BatchWriteError(DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source:CredentialsNotLoaded(CredentialsNotLoaded { source: Some("no providers in chain provided credentials") }), connection: Unknown } }))

$ dy import -t <table_name> -i import_data.json
1975 items processed (1026.56 items/sec)Error: BatchWriteError(DispatchFailure(DispatchFailure { source: ConnectorError { kind: Other(None), source:ProviderError(ProviderError { source: Unexpected(Unexpected { source: hyper::Error(Connect, HttpTimeoutError { kind: "HTTP connect", duration: 1s })}) }), connection: Unknown } }))

Thank you for using dynein and reporting the error. A credential provider's throttling might cause this issue. The current implementation does not reuse clients. I think that the code change on our side should fix this issue.
https://github.com/awslabs/dynein/blob/main/src/batch.rs#L240

Thank you for your response.
Due to the need for immediate use in business, I modified the context function in app.rs to load the authentication information and instantiate the DynamoDbSdkClient. Additionally, by reusing the instantiated client within the context function, I confirmed that the above-mentioned bug no longer occurs.

Thank you for sharing your status. I am working on improving the import/export command, and I expect the new implementation will resolve this issue. Therefore, I do not have plans to address this issue individually. I will update this thread once the work is complete. Is this acceptable to you? Please feel free to let me know if you have any other concerns.

There's no problem with this. I am looking forward to the improvements.


How to disable UIpageviewcontroller bounce in parchment swift ?

You can disable bouncing by setting:
pagingViewController.pageViewController.scrollView.bounces = false

In my case some how its still not working


New build of app not being copied to simulator

Hi,
I just made some changes to one of my screens to load some dummy data, when I run snapshot it seems to be building correctly but the changes are not being to the simulator. If I delete the app in the simulator its correctly installed, but if I just run snapshot it doesn't seem to push the new app.
Have you seen anything like this? Any suggestion? Logging?
Murray

@KrauseFx Is there any progress on this issue?
Or can I use somehow your latest snippet in setup_for_device_change?

Sorry, I haven't got that to be fully working yet... You can probably use something like the snippet you linked in the mean time :+1:

I've got Unknown method com error with the following in my Snapfile:
setup_for_device_change do |device|
  udid = find_simulator(device)
  com("killall 'iOS Simulator'")
  com("xcrun simctl boot '#{udid}'")
  com("xcrun simctl uninstall '#{udid}' 'com.logmein.authenticator'") # TODO: Dynamic
  sleep 3
  com("xcrun simctl install '#{udid}' '#{@app_path}'")
  com("xcrun simctl shutdown '#{udid}'")
end

@tamastimar of course, you forgot to copy the method you want to call:
def find_simulator(name)
  all = `instruments -s`.split("\n")
  all.each do |current|
    return current.match(/\[(.*)\]/)[1] if current.include?name
  end
  raise "Could not find simulator '#{name}' to install the app on."
end

No, I didn't.
That's right above the setup_for_device_change callback. The error is because the com method is unknown. Should I include anything?

oh, right, add
def com(cmd)
  puts `#{cmd}`
end

above your code

I further investigated, this is not a caching problem of xcodebuild, but really a simulator/instruments issue. I'll further investigate, since this issue must be fixed soon!

Looks like my current solution works, I'll keep trying a few apps and release a new version later today :+1:

I'm happy to say, this issue is finally fixed: https://github.com/KrauseFx/snapshot/releases/tag/0.4.5
Update to the latest version and let me know if it works for you too. By default it will now re-install the app. In case you don't want this, see https://github.com/KrauseFx/snapshot/releases/tag/0.4.5
:ship:

I still have problems where the app run in instrument is not the builded version.
Instead of calling reinstall_app I do :
com("killall 'iOS Simulator'")
com("instruments -w '#{device}'")
com("xcrun simctl install booted '#{@app_path}'")

That way when instruments is started a second time, it is loaded correctly

So, you run this instead of the few com commands snapshot currently uses?

yes. I don't want to reset the simulator but you could add this step with an option.


IRSA Support for Bare Metal

What would you like to be added: IRSA Support for Bare Metal clusters
Why is this needed: Users with Bare Metal clusters want to send metrics back to AWS using ADOT w/IRSA.
Today, if a user tries to add a serviceAccountIssuer in their cluster spec, it's not passed through to the kube-api config so the pod-identity-webhook does not work.

We definitely have plans to build, support and probably use this, not sure what that is going to look like though

I also don't know what bare metal changes might be needed if any to support this.

currently using this workaround:

during cluster creation, after control plane node is created but before PXEing other nodes, edit /etc/kubernetes/manifests/kube-apiserver.yaml to add another --service-account-issuer argument for your OIDC provider BEFORE the existing default one. Then proceed with cluster build.

See IRSA docs here for a description of the kube-api server flags required.
So far it seems to work, as long as the rest of the IRSA setup is correct. (you have a service role that's properly annotated and you've got correct keys in your OIDC provider bucket, etc)

@evdevr
can you please expand on what you mean by add another --service-account-issuer argument for your OIDC provider BEFORE the existing default one? do you mean to keep the old one as is and just add the new issuer argument even though it will be a duplicate?

currently using this workaround:

during cluster creation, after control plane node is created but before PXEing other nodes, edit /etc/kubernetes/manifests/kube-apiserver.yaml to add another --service-account-issuer argument for your OIDC provider BEFORE the existing default one. Then proceed with cluster build.

See IRSA docs here for a description of the kube-api server flags required.
So far it seems to work, as long as the rest of the IRSA setup is correct. (you have a service role that's properly annotated and you've got correct keys in your OIDC provider bucket, etc)

I tried to update the /etc/kubernetes/manifests/kube-apiserver.yaml file with the above mentioned arguments and the kube-apiserver does not restart not matter how I try

I restarted the docker container
I tried to run eksctl anywhere upgrade command which is not gonna work as the kube-apiserver pod is down

The only way I can start kube-apiserver is by manually running the command .usr/local/bin/kube-apiserver --advertise-address=<IP_ADDRESS> ......
FYI: I'm running eksctl anywhere cluster in an EC2 instance
One thing that seems to partially work is to update the kubectl edit KubeadmControlPlane --namespace eksa-system CLUSTER_NAME -o yaml and adding the OIDC config but I'm seeing a ton of API Server: Unable to authenticate the request due to an error: invalid bearer token errors in apiserver

@balusarakesh
You're exactly right: I add a new argument in before the one that's already in the file. I don't have the file in front of me, but you'll end up with something like this in the command spec:
  --service-account-issuer=<your $ISSUER_HOSTPATH value>
  --service-account-issuer=<original cluster.local value>

The important part is to make sure the new one you add is first in the list. The kube-apiserver docs state "When this flag is specified multiple times, the first is used to generate tokens and all are used to determine which issuers are accepted." My understanding is: you need to put it first so that your kube-apiserver will create tokens using the OIDC provider, but you need to leave the second one there so that tokens for accounts in your cluster not using IRSA will still be accepted by the API.
After updating the file, it looked like kube-apiserver picked up the change and restarted itself after a few minutes. In addition, I ran the following commands to restart services on the control plane node before proceeding with the cluster build.
systemctl daemon-reload
systemctl restart kubelet
systemctl restart containerd

@evdevr
I updated the config and ran the above systemctl commands and the kube-apiserver process still does not start (I also waited for more than 5 minutes)
FYI: I can start kube-apiserver by manually running the command usr/local/bin/kube-apiserver --advertise-address=<IP_ADDRESS> --allow-privileged=true --audit-log-maxage=30 --audit-log-maxbackup=10.........
Let me know if there is a way I can debug why it did not start

Thank you


Lead Score Form [2/2]

Added

LeadScore form (#160168488)

[Delivers #160168488]

@wnowicki Ready for another look


use pre-created secret instead of embed in syncset

As requested in https://jira.coreos.com/browse/SREP-1831
And discussed with @wanghaoran1988 , he suggested that we should do the clean up things manually on the running clusters.

/retest

@jewzaam
I have added some test for the new changes, and also addressed the deletion part.
Please help review again.  Thanks.

The reason about why the delete secret. I assumed that the test "Test Deleting" should cover that, but actually it does not.
So I updated the test "Test Managed Cluster that later sets noalerts label", and now the deletion of secret should be covered.
And I also renamed some of the function name and arg name for the created secret to "refSecret", to avoid confusion.

/approve
/lgtm


map all error flags to decimal array in L1calibratie

Now comes out as a binary string... but is very slow


Deprecate UrlCodingUItils methods

I think this will resolve #2936. This adds back UrlCodingUtils methods, annotates them as deprecated, and forwards to corresponding Uri methods.

Hmm, looks like build 9704.2 failed due to a test failure:
[error]   x behave and not deadlock on failures with parSequence (30 seconds, 108 ms)
[error]    None is not Some (BlazeClientSpec.scala:198)
[error] org.http4s.client.blaze.BlazeClientSpec.$anonfun$new$35(BlazeClientSpec.scala:198)

I think the test exceeded the 30 second timeout?

I also had an issue with one of the tests in BlazeClientSpec yesterday (link) in which one of the tests timed out (5 seconds). The problem resolved itself when I did another build without any changes... I was not able to recreate it locally so I wonder if Travis is just spotty on this?

Going to close and reopen to trigger another build.


Monitoring and alerting for cilium

Cilium is currently running on MCs and CAPI WCs but scraping is disabled and there is no alerting.
Enable scraping of metrics and basic alerting as well as a grafana dashboard if needed

[ ] Bump MCs to at least 14.4.0
[x] Adjust prometheus-meta-operator
[x] Create grafana dashboard
[ ] Setup alerting

waiting for MCs to be upgraded to 14.5.0+

alert released with prometheus-rules v2.56.0


VectorInterp/ScalerInterp VSE nodes missing from the new UI (3.2 dev).

Godot version:
Godot 3.2 Dev.Calinou at commit https://github.com/godotengine/godot/commit/a0180fa86ec54ac4f7e1b7c8f154364f0525b76a
OS/device including version:
Win10 64 bit, an Nvidia 1060 GPU, AMD Ryzen 2700X CPU.
Issue description:
The enhanced UI does not appear to list the VectorInterp and the ScalerInterp nodes. The nodes themselves in other shaders remain in place, so it's just a case of them not being accessible.
Steps to reproduce:
Create a visual shader and try to find the nodes in the Create Shader Node  menu, the search term "interp" does not return anything and the nodes simply cannot be found.

They are use Mix term now

I could've sworn I tried typing 'mix' before and not find anything, but I did find them now.
However, the name on the nodes themselves have yet to be updated (so for consistency, that needs to be done).

Well, I could rename the title, but if I renames it completly this will be serious break of the compatibility

What else needs to be done after #28516?

I've leave it opened in case of renaming the internal node name itself for 4.0.. However in the UI perspective its already been renamed, so you right and this should be closed.


Null object exception at Microsoft.AspNetCore.Mvc.ViewFeatures.Internal.ExpressionMetadataProvider when TagHelpers process form input

I am consistently getting another null object exception when TagHelpers process form input. It’s just a text box input with asp-for tag to a data field of the viewmodel.
(lldb) clrstack
OS Thread Id: 0xa2db (16)
    Child SP               IP Call Site

00007FFF577F91D8 00007FFF7CECF3CD DynamicClass.lambda_method(System.Runtime.CompilerServices.Closure, StarterMvc.ViewModels.Account.LoginViewModel)
00007FFF577F91E0 00007FFF7E0CCC73 Microsoft.AspNetCore.Mvc.ViewFeatures.Internal.ExpressionMetadataProvider+<>c__DisplayClass0_0`2[[System.__Canon, mscorlib],[System.__Canon, mscorlib]].b__0(System.Object)
00007FFF577F9230 00007FFF7E0A3705 Microsoft.AspNetCore.Mvc.ViewFeatures.ModelExplorer.get_Model()
00007FFF577F9250 00007FFF7E0CCA13 Microsoft.AspNetCore.Mvc.TagHelpers.InputTagHelper.GenerateTextBox(Microsoft.AspNetCore.Mvc.ViewFeatures.ModelExplorer, System.String, System.String)
00007FFF577F92A0 00007FFF7E0CB9A3 Microsoft.AspNetCore.Mvc.TagHelpers.InputTagHelper.Process(Microsoft.AspNetCore.Razor.TagHelpers.TagHelperContext, Microsoft.AspNetCore.Razor.TagHelpers.TagHelperOutput)
00007FFF577F9310 00007FFF7E0B1693 Microsoft.AspNetCore.Razor.TagHelpers.TagHelper+d__5.MoveNext()
00007FFF577F9330 00007FFF7E0B14E1 System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Microsoft.AspNetCore.Razor.TagHelpers.TagHelper+d__5, Microsoft.AspNetCore.Razor.Runtime]](d__5 ByRef)
00007FFF577F9360 00007FFF7E0B138B Microsoft.AspNetCore.Razor.TagHelpers.TagHelper.ProcessAsync(Microsoft.AspNetCore.Razor.TagHelpers.TagHelperContext, Microsoft.AspNetCore.Razor.TagHelpers.TagHelperOutput)
00007FFF577F93E0 00007FFF7E0B0594 Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperRunner+d__0.MoveNext()
00007FFF577F9420 00007FFF7E0B017B System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.__Canon, mscorlib]].Start[[Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperRunner+d__0, Microsoft.AspNetCore.Razor.Runtime]](d__0 ByRef)
00007FFF577F9450 00007FFF7E0B00E0 Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperRunner.RunAsync(Microsoft.AspNetCore.Razor.Runtime.TagHelpers.TagHelperExecutionContext)
00007FFF577F94E0 00007FFF7E0BED1D Asp.ASPV__Views_Account_Login_cshtml+<b__39_0>d.MoveNext()
00007FFF577F96C0 00007FFF7E0BA431 System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[Asp.ASPV__Views_Account_Login_cshtml+<b__39_0>d, ozwgotzv.fkw]](<b__39_0>d ByRef)
(lldb) sos DumpObj 00007fff594d4c48
Name:        Microsoft.AspNetCore.Mvc.ViewFeatures.ModelExplorer
MethodTable: 00007fff7d1bfa48
EEClass:     00007fff7d1af4d8
Size:        72(0x48) bytes
File:        /mnt/test/StarterMvc/bin/debug/dnxcore50/ubuntu.14.04-x64/Microsoft.AspNetCore.Mvc.ViewFeatures.dll
Fields:
          MT    Field   Offset                 Type VT     Attr            Value Name

00007fff7cc686e8  40000cd        8 ...lMetadataProvider  0 instance 00007fff58102be0 _metadataProvider
00007fff7c609e00  40000ce       10        System.Object  0 instance<PHONE_NUMBER>000000 _model
00007fff7cd16880  40000cf       18 ...bject, mscorlib]]  0 instance 00007fff594d4c08 _modelAccessor
00007fff7c7ae4d8  40000d0       20          System.Type  0 instance<PHONE_NUMBER>000000 _modelType
0000000000000000  40000d1       28                       0 instance<PHONE_NUMBER>000000 _properties
00007fff7d1bfa48  40000d2       30 ...res.ModelExplorer  0 instance 00007fff590f42f8 k__BackingField
00007fff7d0847f8  40000d3       38 ...ing.ModelMetadata  0 instance 00007fff592945f8 k__BackingField
(lldb) sos DumpObj 00007fff590f42f8
Name:        Microsoft.AspNetCore.Mvc.ViewFeatures.ModelExplorer
MethodTable: 00007fff7d1bfa48
EEClass:     00007fff7d1af4d8
Size:        72(0x48) bytes
File:        /mnt/test/StarterMvc/bin/debug/dnxcore50/ubuntu.14.04-x64/Microsoft.AspNetCore.Mvc.ViewFeatures.dll
Fields:
          MT    Field   Offset                 Type VT     Attr            Value Name

00007fff7cc686e8  40000cd        8 ...lMetadataProvider  0 instance 00007fff58102be0 _metadataProvider
00007fff7c609e00  40000ce       10        System.Object  0 instance<PHONE_NUMBER>000000 _model
00007fff7cd16880  40000cf       18 ...bject, mscorlib]]  0 instance<PHONE_NUMBER>000000 _modelAccessor
00007fff7c7ae4d8  40000d0       20          System.Type  0 instance<PHONE_NUMBER>000000 _modelType
0000000000000000  40000d1       28                       0 instance<PHONE_NUMBER>000000 _properties
00007fff7d1bfa48  40000d2       30 ...res.ModelExplorer  0 instance<PHONE_NUMBER>000000 k__BackingField
00007fff7d0847f8  40000d3       38 ...ing.ModelMetadata  0 instance 00007fff590f4128 k__BackingField
dfThe problem seems to be here:
    public object Model

    {

        get

        {

            if (_model == null && _modelAccessor != null)

            {

                Debug.Assert(Container != null);

                _model = _modelAccessor(Container.Model); -----àContainer.Model is always null here

    public static ModelExplorer FromLambdaExpression<TModel, TResult>(

        Expression<Func<TModel, TResult>> expression,

        ViewDataDictionary<TModel> viewData,

        IModelMetadataProvider metadataProvider)

    {

…
        Func<object, object> modelAccessor = (container) =>

        {

            try

            {

                return CachedExpressionCompiler.Process(expression)((TModel)container); -----àWould hit the exception a lot

            }

            catch (NullReferenceException)

            {

                return null;

            }

        };

It seems that very likely the Container.Model would be null, and thus causing the accessor to keep throw and catch nullreferenceexception.

@rynowak @sivagms @SajayAntony

@yuwaMSFT are you observing first-chance Exceptions? The current code is intentional and container == null is not a reliable special case. E.g. the following would be exactly wrong:tm: if the expression were m => someVariable.
Func<object, object> modelAccessor = (container) =>
{
    if (container == null)
    {
        return null;
    }

    try
    {
        return CachedExpressionCompiler.Process(expression)((TModel)container);
    }
    catch (NullReferenceException)
    {
        return null;
    }
};

That said, throwing and catching Exceptions for expected cases is not particularly efficient or fun to debug. We've considered and, in previous MVC versions, prototyped other approaches. But we haven't hit on something that's both correct and more efficient.

@dougbu  Thanks! Yes it's first chance exception. I see your point that return null directly is not correct in certain cases. But considering the most common usage of simple member access, would it be possible to optimize out the exception throwing for this common case, while leave the other cases in a less efficient way?

would it be possible to optimize out ...

Yes, it's possible. But as I said, we have not found a solution which is both correct and more efficient.
E.g. I believe one option prototyped a while back was to rewrite the Linq expression tree to make most nodes null conditional (like ?.). That turned out to grow the tree enough to slow things down in the also-common case where everything is non-null. On balance it just didn't work out.


arrow example

Description

adds arrow + connector x example
fixes how items are counted in writers + arrow
incremental optimizations
drop unix timestamp autodetect

How does connector-x handle larger than memory datasets. In a regular SQL source, you chunk results from the cursor/server in proper generator fashion. this on the other hand is most like to one shot the entire result on the first call to next() right. I love the idea of pulling data directly to arrow in memory via rust, but am concerned about the lack of a generator-like interface (which I am not sure is possible via the FFI?)

How does connector-x handle larger than memory datasets. In a regular SQL source, you chunk results from the cursor/server in proper generator fashion. this on the other hand is most like to one shot the entire result on the first call to next() right. I love the idea of pulling data directly to arrow in memory via rust, but am concerned about the lack of a generator-like interface (which I am not sure is possible via the FFI?)
Consider a warning for users if the above holds true. Regardless, I would be extremely keen on some benchmarking results against the existing sql source, I bet its blazingly fast by comparison 🆒

they load everything into memory first - pre-allocating the needed space. so you need to chunk the stream yourself. that's quite unfortunate. if you have any good index, it should be easy though. I do that in the example here.
https://github.com/sfu-db/connector-x#how-does-connectorx-download-the-data


False positives for no-constant-condition

Errors on increment/decrement.
let len = 10

while (len--) {}

The error was fixed and it will not happen anymore in the next release.

@blakeembrey v1.6.0 released


 D3DTOP_SELECTARG2 : D3DTOP_MODULATE);

  texture is not defined

Hey @danielkrupinski , am i the only one whose game crashes and gives errors here:
            g_pd3dDevice->SetTexture(0, texture);
            g_pd3dDevice->SetTextureStageState(0, D3DTSS_COLOROP, texture == g_FontTexture ? D3DTOP_SELECTARG2 : D3DTOP_MODULATE);

  texture is not defined

This code is crashing my game all of the time pl s fi x

This code is crashing my game all of the time pl s fi x


Force datasource refresh manually

Is it possible to force a simple dashboard refresh programmatically?
My dashboard has a custom date selector, and currently the whole dashboard needs to be reloaded when the date range is changed because that's how I programmed my custom JS. This is not the ideal way to refresh, since the widgets re-animate each time and it's just a little clunky.
I looked in freeboard_plugins.js but didn't see a simple refresh function that I can call from my own JS when my date range is changed.   Further, I have an "autoplay" function that loops through various date ranges, and this clunkiness is exposed in all it's glory!
UPDATE
The function "serialize" shows promise... is this the way to force refresh?

Here's my solution. I added this simple custom function in freeboard+plugins.js:
this.customRefresh = function() {
	var datasources = [];
	_.each(self.datasources(), function(datasource)
	{
		datasource.updateNow();
	});
}

Then I simply call this from my main dashboard page (index.html) when I want to manually refresh the widgets with new data.


When I run animation.ipynb, it report error

When I run animation.ipynb
    data = np.load(pjoin(src_dir, npy_file))
    save_path = pjoin(tgt_ani_dir, npy_file[:-3] + 'mp4')
    if os.path.exists(save_path):
        continue    plot_3d_motion(save_path, kinematic_chain, data, title="None", fps=20, radius=4)

it report the error
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_813/1233317043.py in <module>
      5         continue
      6 #   You may set the title on your own.
----> 7     plot_3d_motion(save_path, kinematic_chain, data, title="None", fps=20, radius=4)

/tmp/ipykernel_813/1547562698.py in plot_3d_motion(save_path, kinematic_tree, joints, title, figsize, fps, radius)
     87     ani = FuncAnimation(fig, update, frames=frame_number, interval=1000/fps, repeat=False)
     88
---> 89     ani.save(save_path, fps=fps)
     90     plt.close()

~/miniconda3/lib/python3.8/site-packages/matplotlib/animation.py in save(self, filename, writer, fps, dpi, codec, bitrate, extra_args, metadata, extra_anim, savefig_kwargs, progress_callback)
   1076                                _is_saving=True, manager=None):
   1077             for anim in all_anim:
-> 1078                 anim._init_draw()  # Clear the initial frame
   1079             frame_number = 0
   1080             # TODO: Currently only FuncAnimation has a save_count

~/miniconda3/lib/python3.8/site-packages/matplotlib/animation.py in _init_draw(self)
   1696                 )
   1697                 return
-> 1698             self._draw_frame(frame_data)
   1699         else:
   1700             self._drawn_artists = self._init_func()

~/miniconda3/lib/python3.8/site-packages/matplotlib/animation.py in _draw_frame(self, framedata)
   1718         # Call the func with framedata and args. If blitting is desired,
   1719         # func needs to return a sequence of any artists that were modified.
-> 1720         self._drawn_artists = self._func(framedata, *self._args)
   1721
   1722         if self._blit:

/tmp/ipykernel_813/1547562698.py in update(index)
     57     def update(index):
     58         #         print(index)
---> 59         ax.lines = []
     60         ax.collections = []
     61         ax.view_init(elev=120, azim=-90)

AttributeError: can't set attribute```

You may have different matplotlib version
On Mon, Aug 28, 2023 at 8:59 AM elba @.***> wrote:

When I run animation.ipynb
`for npy_file in tqdm(npy_files):
data = np.load(pjoin(src_dir, npy_file))
save_path = pjoin(tgt_ani_dir, npy_file[:-3] + 'mp4')
if os.path.exists(save_path):
continue
You may set the title on your own.
plot_3d_motion(save_path, kinematic_chain, data, title="None", fps=20, radius=4)`
it report the error
` 0%| | 0/29228 [00:00<?, ?it/s]/tmp/ipykernel_813/1547562698.py:38:
MatplotlibDeprecationWarning: Axes3D(fig) adding itself to the figure is
deprecated since 3.4. Pass the keyword argument auto_add_to_figure=False
and use fig.add_axes(ax) to suppress this warning. The default value of
auto_add_to_figure will change to False in mpl3.5 and True values will no
longer work in 3.6. This is consistent with other Axes classes.
ax = p3.Axes3D(fig)
/tmp/ipykernel_813/1547562698.py:19: MatplotlibDeprecationWarning: The 'b'
parameter of grid() has been renamed 'visible' since Matplotlib 3.5;
support for the old name will be dropped two minor releases later.
ax.grid(b=False)
0%| | 0/29228 [00:00<?, ?it/s]
AttributeError Traceback (most recent call last)
/tmp/ipykernel_813/1233317043.py in
5 continue
6 # You may set the title on your own.
----> 7 plot_3d_motion(save_path, kinematic_chain, data, title="None",
fps=20, radius=4)
/tmp/ipykernel_813/1547562698.py in plot_3d_motion(save_path,
kinematic_tree, joints, title, figsize, fps, radius)
87 ani = FuncAnimation(fig, update, frames=frame_number,
interval=1000/fps, repeat=False)
88
---> 89 ani.save(save_path, fps=fps)
90 plt.close()
~/miniconda3/lib/python3.8/site-packages/matplotlib/animation.py in
save(self, filename, writer, fps, dpi, codec, bitrate, extra_args,
metadata, extra_anim, savefig_kwargs, progress_callback)
1076 _is_saving=True, manager=None):
1077 for anim in all_anim:
-> 1078 anim._init_draw() # Clear the initial frame
1079 frame_number = 0
1080 # TODO: Currently only FuncAnimation has a save_count
~/miniconda3/lib/python3.8/site-packages/matplotlib/animation.py in
_init_draw(self)
1696 )
1697 return
-> 1698 self._draw_frame(frame_data)
1699 else:
1700 self._drawn_artists = self._init_func()
~/miniconda3/lib/python3.8/site-packages/matplotlib/animation.py in
_draw_frame(self, framedata)
1718 # Call the func with framedata and args. If blitting is desired,
1719 # func needs to return a sequence of any artists that were modified.
-> 1720 self._drawn_artists = self._func(framedata, *self._args)
1721
1722 if self._blit:
/tmp/ipykernel_813/1547562698.py in update(index)
57 def update(index):
58 # print(index)
---> 59 ax.lines = []
60 ax.collections = []
61 ax.view_init(elev=120, azim=-90)
AttributeError: can't set attribute
`
—
Reply to this email directly, view it on GitHub
https://github.com/EricGuo5513/HumanML3D/issues/84, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKRYNB5SH7B7UWVMVASPVYTXXSW5PANCNFSM6AAAAAA4BRUMII
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

You may have different matplotlib version
…
On Mon, Aug 28, 2023 at 8:59 AM elba @.> wrote: When I run animation.ipynb for npy_file in tqdm(npy_files): data = np.load(pjoin(src_dir, npy_file)) save_path = pjoin(tgt_ani_dir, npy_file[:-3] + 'mp4') if os.path.exists(save_path): continue You may set the title on your own. plot_3d_motion(save_path, kinematic_chain, data, title="None", fps=20, radius=4) it report the error 0%| | 0/29228 [00:00<?, ?it/s]/tmp/ipykernel_813/1547562698.py:38: MatplotlibDeprecationWarning: Axes3D(fig) adding itself to the figure is deprecated since 3.4. Pass the keyword argument auto_add_to_figure=False and use fig.add_axes(ax) to suppress this warning. The default value of auto_add_to_figure will change to False in mpl3.5 and True values will no longer work in 3.6. This is consistent with other Axes classes. ax = p3.Axes3D(fig) /tmp/ipykernel_813/1547562698.py:19: MatplotlibDeprecationWarning: The 'b' parameter of grid() has been renamed 'visible' since Matplotlib 3.5; support for the old name will be dropped two minor releases later. ax.grid(b=False) 0%| | 0/29228 [00:00<?, ?it/s] AttributeError Traceback (most recent call last) /tmp/ipykernel_813/1233317043.py in 5 continue 6 # You may set the title on your own. ----> 7 plot_3d_motion(save_path, kinematic_chain, data, title="None", fps=20, radius=4) /tmp/ipykernel_813/1547562698.py in plot_3d_motion(save_path, kinematic_tree, joints, title, figsize, fps, radius) 87 ani = FuncAnimation(fig, update, frames=frame_number, interval=1000/fps, repeat=False) 88 ---> 89 ani.save(save_path, fps=fps) 90 plt.close() ~/miniconda3/lib/python3.8/site-packages/matplotlib/animation.py in save(self, filename, writer, fps, dpi, codec, bitrate, extra_args, metadata, extra_anim, savefig_kwargs, progress_callback) 1076 _is_saving=True, manager=None): 1077 for anim in all_anim: -> 1078 anim._init_draw() # Clear the initial frame 1079 frame_number = 0 1080 # TODO: Currently only FuncAnimation has a save_count ~/miniconda3/lib/python3.8/site-packages/matplotlib/animation.py in _init_draw(self) 1696 ) 1697 return -> 1698 self._draw_frame(frame_data) 1699 else: 1700 self._drawn_artists = self._init_func() ~/miniconda3/lib/python3.8/site-packages/matplotlib/animation.py in _draw_frame(self, framedata) 1718 # Call the func with framedata and args. If blitting is desired, 1719 # func needs to return a sequence of any artists that were modified. -> 1720 self._drawn_artists = self._func(framedata, *self._args) 1721 1722 if self._blit: /tmp/ipykernel_813/1547562698.py in update(index) 57 def update(index): 58 # print(index) ---> 59 ax.lines = [] 60 ax.collections = [] 61 ax.view_init(elev=120, azim=-90) AttributeError: can't set attribute — Reply to this email directly, view it on GitHub <#84>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKRYNB5SH7B7UWVMVASPVYTXXSW5PANCNFSM6AAAAAA4BRUMII . You are receiving this because you are subscribed to this thread.Message ID: @.>

Thanks, I change version and it works

You may have different matplotlib version
…
On Mon, Aug 28, 2023 at 8:59 AM elba @.> wrote: When I run animation.ipynb for npy_file in tqdm(npy_files): data = np.load(pjoin(src_dir, npy_file)) save_path = pjoin(tgt_ani_dir, npy_file[:-3] + 'mp4') if os.path.exists(save_path): continue You may set the title on your own. plot_3d_motion(save_path, kinematic_chain, data, title="None", fps=20, radius=4) it report the error 0%| | 0/29228 [00:00<?, ?it/s]/tmp/ipykernel_813/1547562698.py:38: MatplotlibDeprecationWarning: Axes3D(fig) adding itself to the figure is deprecated since 3.4. Pass the keyword argument auto_add_to_figure=False and use fig.add_axes(ax) to suppress this warning. The default value of auto_add_to_figure will change to False in mpl3.5 and True values will no longer work in 3.6. This is consistent with other Axes classes. ax = p3.Axes3D(fig) /tmp/ipykernel_813/1547562698.py:19: MatplotlibDeprecationWarning: The 'b' parameter of grid() has been renamed 'visible' since Matplotlib 3.5; support for the old name will be dropped two minor releases later. ax.grid(b=False) 0%| | 0/29228 [00:00<?, ?it/s] AttributeError Traceback (most recent call last) /tmp/ipykernel_813/1233317043.py in 5 continue 6 # You may set the title on your own. ----> 7 plot_3d_motion(save_path, kinematic_chain, data, title="None", fps=20, radius=4) /tmp/ipykernel_813/1547562698.py in plot_3d_motion(save_path, kinematic_tree, joints, title, figsize, fps, radius) 87 ani = FuncAnimation(fig, update, frames=frame_number, interval=1000/fps, repeat=False) 88 ---> 89 ani.save(save_path, fps=fps) 90 plt.close() ~/miniconda3/lib/python3.8/site-packages/matplotlib/animation.py in save(self, filename, writer, fps, dpi, codec, bitrate, extra_args, metadata, extra_anim, savefig_kwargs, progress_callback) 1076 _is_saving=True, manager=None): 1077 for anim in all_anim: -> 1078 anim._init_draw() # Clear the initial frame 1079 frame_number = 0 1080 # TODO: Currently only FuncAnimation has a save_count ~/miniconda3/lib/python3.8/site-packages/matplotlib/animation.py in _init_draw(self) 1696 ) 1697 return -> 1698 self._draw_frame(frame_data) 1699 else: 1700 self._drawn_artists = self._init_func() ~/miniconda3/lib/python3.8/site-packages/matplotlib/animation.py in _draw_frame(self, framedata) 1718 # Call the func with framedata and args. If blitting is desired, 1719 # func needs to return a sequence of any artists that were modified. -> 1720 self._drawn_artists = self._func(framedata, *self._args) 1721 1722 if self._blit: /tmp/ipykernel_813/1547562698.py in update(index) 57 def update(index): 58 # print(index) ---> 59 ax.lines = [] 60 ax.collections = [] 61 ax.view_init(elev=120, azim=-90) AttributeError: can't set attribute — Reply to this email directly, view it on GitHub <#84>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKRYNB5SH7B7UWVMVASPVYTXXSW5PANCNFSM6AAAAAA4BRUMII . You are receiving this because you are subscribed to this thread.Message ID<EMAIL_ADDRESS>
Thanks, I change version and it works

which version did you use? i change the matplotlib to 3.3.4.  the error still exits.


Recurrentshop 1

@abhaikollara Please test recurrentshop-1 branch on TF backend.

@farizrahman4u Weird, I'm getting segmentation faults on all three tests

Update TF and Keras to latest version.

All tests are passing for me on TF / python 3.5.. except for teacher forcing.

No joy, here's traceback for test_recurrent_sequential
Traceback (most recent call last):
  File "tests/test_recurrent_sequential.py", line 219, in <module>
    model.fit([np.random.random((32, 7, 5)), np.random.random((32, 7, 5))], np.random.random((32, 5)))
  File "/home/abhai/keras/keras/engine/training.py", line 1486, in fit
    initial_epoch=initial_epoch)
  File "/home/abhai/keras/keras/engine/training.py", line 1141, in _fit_loop
    outs = f(ins_batch)
  File "/home/abhai/keras/keras/backend/tensorflow_backend.py", line 2101, in __call__
    session = get_session()
  File "/home/abhai/keras/keras/backend/tensorflow_backend.py", line 155, in get_session
    _initialize_variables()
  File "/home/abhai/keras/keras/backend/tensorflow_backend.py", line 308, in _initialize_variables
    sess.run(tf.variables_initializer(uninitialized_variables))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 767, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 965, in _run
    feed_dict_string, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1015, in _do_run
    target_list, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1035, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: The node 'recurrent_sequential_14/while_1/Variable_1/Assign' has inputs from different frames. The input 'recurrent_sequential_14/while_1/Const_1' is in frame 'recurrent_sequential_14/while_1/recurrent_sequential_14/while_1/'. The input 'recurrent_sequential_14/while_1/Variable_1' is in frame ''.
[1]    15461 segmentation fault  python tests/test_recurrent_sequential.py

Thats the test for teacherforcing failing. I think the rest of the file is commented out?

I tried commenting out the teacher forcing part, now I get this
*** Error in `python': free(): invalid pointer: 0x000000000178e320 ***
[1]    21185 abort      python tests/test_recurrent_sequential.py

But did it train the rest of the models?

Yeah, it did

cool.
Next : Documentation.

Hold on, I've been testing on python2.7


adds refinement list filters

Codecov Report

Merging #7 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master       #7   +/-   ##
=======================================
  Coverage   94.72%   94.72%
=======================================
  Files          27       27
  Lines         379      379
  Branches       44       44
=======================================
  Hits          359      359
  Misses          5        5
  Partials       15       15

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7297e28...77b9534. Read the comment docs.


Add node-inspector rc file

Check out the save-live-edit flag. =)
https://www.npmjs.com/package/node-inspector#rc-configuration

Maybe we should recommend iron-node? Anyway, this seems like it may be out of scope.


Support exFAT format on source/destination keys

I'm not sure pmount can do that, need to check that out.

I did more research into this and tested - the current version of pmount cannot handle exFAT keys for read-only or read-write. It looks like there have been patches for exFAT support via fuse (similar to how pmount supports NTFS) but they haven't been merged. There might be a way to use mount without root permissions for exFAT keys, but that's probably not a road we want to go down at the moment considering 99% of USB keys in the wild are FAT formatted.

In 2019 I'm seeing a lot of exFAT devices and it seems like pmount is not maintained? Would it be possible to switch to or add support for FUSE? exfact-fuse and exfact-utils seems to be the usual way at the moment.

Very, very probably, yes.
I'll try to work on it as soon as possible, but if you have the chance to give it a go, a PR is more than welcome :)