[Keyvault][2.0.6] "az keyvault secret list" doesn't show "name" of secret, but download/show require it

Description
To use keyvault as a rsa keystore, I'll need to use download and show when I setting up new local machines (let alone servers in Azure). But both of those require --name of the secret. Unfortunately, the name value is not displayed by the output of list.
Result: I can't script directly from list, because I have to put a sed from xargs in there to calculate the name from the id value that is returned.
GOAL: the metadata from list has the name value already in it so that you can easily script the keys from there without interpolating xargs/sed.
EXAMPLE:
az keyvault secret list --vault-name xxx
[
  {
    "attributes": {
      "created": "2017-05-28T09:50:49+00:00",
      "enabled": true,
      "expires": null,
      "notBefore": null,
      "updated": "2017-05-28T09:50:49+00:00"
    },
    "contentType": "Password-less private key for dcos demos",
    "id": "https://xxx.vault.azure.net/secrets/<name of key>",
    "managed": null,
    "tags": {
      "file-encoding": "utf-8"
    }
  }

Environment summary
Install Method: How did you install the CLI? (e.g. pip, interactive script, apt-get, Docker, MSI, nightly)
Answer here:
sudo -H pip install azure-cli
CLI Version: What version of the CLI and modules are installed? (Use az --version)
Answer here:
╰─ az --version
azure-cli (2.0.6)
acr (2.0.4)
acs (2.0.6)
appservice (0.1.6)
batch (2.0.4)
cdn (0.0.2)
cloud (2.0.2)
cognitiveservices (0.1.2)
command-modules-nspkg (2.0.0)
component (2.0.4)
configure (2.0.6)
core (2.0.6)
cosmosdb (0.1.6)
dla (0.0.6)
dls (0.0.6)
feedback (2.0.2)
find (0.2.2)
interactive (0.3.2)
iot (0.1.5)
keyvault (2.0.4)
lab (0.0.4)
monitor (0.0.4)
network (2.0.6)
nspkg (3.0.0)
profile (2.0.4)
rdbms (0.0.1)
redis (0.2.3)
resource (2.0.6)
role (2.0.4)
sf (1.0.1)
sql (2.0.3)
storage (2.0.6)
vm (2.0.6)
Python (Darwin) 2.7.13 (default, Apr  4 2017, 08:47:57)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)]
Python location '/usr/local/opt/python/bin/python2.7'
OS Version: What OS and version are you using?
Answer here:
macOS Sierra, 10.12.5
Shell Type: What shell are you using? (e.g. bash, cmd.exe, Bash on Windows)
Answer here:
oh-my-zsh

Unfortunately, this happens because the service, at present, only returns the key ID. cc/ @schaabs

Is there a movement afoot to change that?

@squillace yes we are currently looking to change the service so that these fields are returned in addition to the ID.  However, I'm not sure when this fix will be available.  I'm also looking into adding methods into the cli to parse the ID, currently you can do this through the python azure-keyvault SDK using the KeyVaultId class.

Hi @schaabs, not sure we should have you do the work for that if the server side is updated in the future. I just wanted THAT updated. But parsing the ID is 'ok' for now. I'd let this be the issue to fix the server and close it when that is done.

I would agree with @squillace.  If you really wanted to, you could simulate the server fix by introducing an output transformer for the KeyVault data plane commands. That would be a more seamless workaround than introducing ID parse commands (unless they have a role in some other usage scenario).

Sounds good @squillace could you possibly share a sample of the work around you used here for any others that encounter this issue in the meantime?

Cleaning u old issues, reopen if this is still active.

This is very important, making az keyvault almost useless to me - I cannot enumerate the secrets in a keyvault and then act on them, I need to know what their name is.
az keyvault secret list --vault-name <vault-name>
returns
  {
    "attributes": {
      "created": "2017-11-10T21:55:15+00:00",
      "enabled": true,
      "expires": null,
      "notBefore": null,
      "recoveryLevel": "Purgeable",
      "updated": "2017-11-10T21:55:15+00:00"
    },
    "contentType": null,
    "id": "https://<vault-name>.vault.azure.net/secrets/<secret-name>",
    "managed": null,
    "tags": null
  }

but to do anything to the secret, I need <secret-name> - ...

@schaabs, when is this service-side fix coming?  If it will be far in the future, I'd recommend implementing the client-side workaround.

Until this is fixed, here's a workaround I use for reading secrets using bash and jq:
# this prints all the secrets in the keyvault
VAULT_NAME=my-key-vault
for i in `az keyvault secret list --vault-name $VAULT_NAME | jq -r '.[]|.id' | cut -d'/' -f 5`
do
  az keyvault secret show --vault-name $VAULT_NAME --name $i
done | jq '.id,.value'

I appreciate the workaround. I had one, but yours is equivalent. I had missed the earlier request for mine.

I have added a work around for this issue which is in the above mentioned PR.  I have added an '--id'  parameter to secret show/download (as well as other command where it makes sense) so that you can use the id property of a secret directly rather than having to parse into vault-name name and version.
$ az keyvault secret show -h

Command
    az keyvault secret show : Get a specified secret from a given key vault.
        The GET operation is applicable to any secret stored in Azure Key Vault. This operation
        requires the secrets/get permission.
Arguments

Id Arguments
    --id           : Id of the secret.  If specified all other 'Id' arguments should be omitted.
    --name -n      : Name of the secret. Required if --id is not specified.
    --vault-name   : Name of the key vault. Required if --id is not specified.
    --version -v   : The secret version. If omitted, uses the latest version.

Global Arguments
    --debug        : Increase logging verbosity to show all debug logs.
    --help -h      : Show this help message and exit.
    --output -o    : Output format.  Allowed values: json, jsonc, table, tsv, yaml.  Default: json.
    --query        : JMESPath query string. See http://jmespath.org/ for more information and
                     examples.
    --subscription : Name or ID of subscription. You can configure the default subscription using
                     `az account set -s NAME_OR_ID`.
    --verbose      : Increase logging verbosity. Use --debug for full debug logs.

These changes are currently available by installing the keyvault-preview cli extension.  You can find instructions on how to install the extension as well as file any issues encountered here: https://github.com/azure/azure-keyvault-cli-extension/
These changes should be released in the next update of the azure-cli-keyvault command module.

Since the PR has been merged, I'm closing this.


Not authorized to access Exercises or Lectures

Describe the bug
Opening the Exercise or Lectures tab of an existing course causes a "You are not authorized to access this page." error message to appear. This message appears for all users when they have access to something (meaning if they truly don't have access to any exercise yet, then they won't see this message in the exercise tab) including Administrators.
It does not seem like we actually lost access to anything. We only get this message.
To Reproduce

Go to "Course Overview"
Click on a course with an exercise (or a lecture)
Click on "Exercises" (or Lectures) on the left side
See error

Expected behavior
No error message
Screenshots
No response
Which version of Artemis are you seeing the problem on?
7.5.5
What browsers are you seeing the problem on?
Firefox
Additional context
This started happening after upgrading from 7.5.2 to 7.5.5
Relevant log output
No response

I have been reinstalling 7.5.5 twice and this error seems to be magically gone now. Feel free to close this issue if you don't find anything to fix.


Add validation of CRD types

We should validate the applied configuration of resources to provide faster feedback to user.

Closing with: #184 #180


Tensorflow or cuda not giving back gpu memory after session closes

I am tying to install tensorflow correctly and I am getting memory allocation erros.
I am using:
Ubuntu 16.04
tf = 1.5.0 from pip install tensorflow-gpu
CUDA 9.0
CUDNN 7.0.5
starting python in a command terminal and running the following commands:
import tensorflow as tf
sess = tf.Session()
sess.close()
If I start a session it is fine the first time it says total memory: 7.72Gib free Memory: 7.50GiB
The next time in the same terminal I start python again ti says freeMemory: 279.44MiB
and finally if I start again it says:
freeMemory 122.50MiB
failed to allocate 72.50M from device: CUDA_ERROR_OUT_OF_MEMORY
What can I do to fix this?
I have pasted the entire sequence below:
teves@teves:~$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import tensorflow
import tensorflow as tf
sess = tf.Session()
2018-02-15 11:06:55.708721: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-02-15 11:06:55.846202: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:895] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-02-15 11:06:55.846656: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1105] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.695
pciBusID: 0000:01:00.0
totalMemory: 7.92GiB freeMemory: 7.50GiB
2018-02-15 11:06:55.846685: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
sess.close()

[1]+  Stopped                 python
teves@teves:~$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import tensorflow as tf
sess = tf.Session()
2018-02-15 11:07:34.144528: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-02-15 11:07:34.351426: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:895] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-02-15 11:07:34.351699: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1105] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.695
pciBusID: 0000:01:00.0
totalMemory: 7.92GiB freeMemory: 279.44MiB
2018-02-15 11:07:34.351732: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
sess.close()

[2]+  Stopped                 python
teves@teves:~$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import tensorflow as tf
sess = tf.Session()
2018-02-15 11:08:43.527818: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-02-15 11:08:43.877301: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:895] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-02-15 11:08:43.877577: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1105] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.695
pciBusID: 0000:01:00.0
totalMemory: 7.92GiB freeMemory: 122.50MiB
2018-02-15 11:08:43.877610: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
2018-02-15 11:08:44.047980: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 72.50M (76021760 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
tf.version
'1.5.0'

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

Have I written custom code: No
OS Platform and Distribution: Ubuntu 16.04
TensorFlow installed from: pip install tensorflow-gpu
TensorFlow version 1.5.0
Bazel version: 0.9.0
CUDA/cuDNN version: cuda 9.0 cudnn 7.0.5
GPU model and memory: gtx 1070 8gb
Exact command to reproduce
open pthon
import tensorflow as tf
sess = tf.Session()
sess.close()
close python
repeat three times. get out of memory error
I think it has something to do with the nvidia driver. I went down to the 390.12 version from 390.3 and the problem got better. I am able to close the terminal and open a new one now and i do not get the issue. However if I dont close the terminal but close the python session and start a new one I still get the error.

By default TensorFlow allocates GPU memory for the lifetime of the process, not the lifetime of the session object. More details at: https://www.tensorflow.org/programmers_guide/using_gpu#allowing_gpu_memory_growth
Thus, if you want memory to be freed, you'll have to exit the Python interpreter, not just close the session.
Hope that helps.

I'm currently building out a workflow using Celery to perform a set of image analytics in parallel. I ran into this issues since the celery daemon, once started, never released GPU resources after it completed processing images that used TensorFlow. As a result, I quickly depleted all of my GPU resources. My interim solution to the problem, was to wrap my analytic in a Pool that spawns a single Process. It's not pretty, but it at least solves the problem in releasing GPU resources if you have a long running process like a celery daemon.
import tensorflow as tf

from multiprocessing import Pool

def _process(image):
    sess = tf.Session()
    sess.close()

def process_image(image):
    with Pool(1) as p:
        return p.apply(_process, (image,))

The problem is that even if I exit python in the terminal (ctl -Z) and
restart another python process the memory is still occupied. I have to
close the terminal for the memory to release.
On Fri, Feb 23, 2018 at 12:41 PM, Nathan Douglas<EMAIL_ADDRESS>wrote:

I'm currently building out a workflow using Celery to perform a set of
image analytics in parallel. I ran into this issues since the celery
daemon, once started, never released GPU resources after it completed
processing images that used TensorFlow. As a result, I quickly depleted all
of my GPU resources. My interim solution to the problem, was to wrap my
analytic in a Pool that spawns a single Process. It's not pretty, but it at
least solves the problem in releasing GPU resources if you have a long
running process like a celery daemon.
import tensorflow as tf
from multiprocessing import Pool
def _process(image):
sess = tf.Session()
sess.close()
def process_image(image):
with Pool(1) as p:
return p.apply(_process, (image,))
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/17048#issuecomment-368082470,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJsMvwR_3Po8S7ANCWN0MJNkZIOfGm9wks5tXvhGgaJpZM4SHHWg
.

When you're "Ctrl+Z"ing, you're not killing the python process, you're just putting it into the background. So it doesn't release the memory. You want to exit the python interpreter to kill the process.

Oh I see. Just to clarify in case you weren't aware, Ctrl-Z will put the python process in the background which isn't the same as doing Ctrl-D from the python terminal. If you open 2 terminals and in one run watch nvidia-smi and in the other you run your python terminal to start a tensorflow session, you'll see your python process allocate space and then drop off with Ctrl-D, where as the Ctrl-Z command won't release those resources.

Thanks Nathan. Yes with ctrl-D it works properly.
On Fri, Feb 23, 2018 at 5:43 PM, Nathan Douglas<EMAIL_ADDRESS>wrote:

Oh I see. Just to clarify in case you weren't aware, Ctrl-Z will put the
python process in the background which isn't the same as doing Ctrl-D from
the python terminal. If you open 2 terminals and in one run watch
nvidia-smi and in the other you run your python terminal to start a
tensorflow session, you'll see your python process allocate space and then
drop off with Ctrl-D, where as the Ctrl-Z command won't release those
resources.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/17048#issuecomment-368159242,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJsMv3ZK9ZS3jEyWbTIF5TwnAFQ3i_tPks5tXz8CgaJpZM4SHHWg
.

Closing this as things are working as intended.

I'm having a similar issue. I want to run indefinite training cycles back to back, because I'm using an evolutionary algorithm to optimize the hyperparameters. However, over time, the GPU "waits" for greater and greater times between training cycles (afterburner visualization):

I assume this is a memory issue in a similar vein. Any ideas of a workaround? I found an attempt at a solution using Keras to reset GPU memory after each run, and have tried this, but it has not solved the problem (it may have somewhat shortened the lag between training runs, but not enough to constitute a fix). Keras "fix"
What do you suppose is going on here? I.e. what may be the source of the problem, and resources that could help me address it?
Thanks!

I use numba to release the gpu. With tensorflow I can not find a effect method.

import tensorflow as tf
from numba import cuda

a = tf.constant([1.0,2.0,3.0],shape=[3],name='a')
b = tf.constant([1.0,2.0,3.0],shape=[3],name='b')
with tf.device('/gpu:1'):
    c = a+b

TF_CONFIG = tf.ConfigProto(
gpu_options=tf.GPUOptions(per_process_gpu_memory_fraction=0.1),
  allow_soft_placement=True)

sess = tf.Session(config=TF_CONFIG)
sess.run(tf.global_variables_initializer())
i=1
while(i<1000):
        i=i+1
        print(sess.run(c))

sess.close()
cuda.select_device(1)
cuda.close()
with tf.device('/gpu:1'):
    c = a+b

TF_CONFIG = tf.ConfigProto(
gpu_options=tf.GPUOptions(per_process_gpu_memory_fraction=0.5),
  allow_soft_placement=True)

sess = tf.Session(config=TF_CONFIG)

sess.run(tf.global_variables_initializer())
while(1):
        print(sess.run(c))

@androidbeepboop : What you're alluding to seems different from the issue the original author of this issue had. Could you please file a separate issue with all details (and ideally a minimal, complete, verifiable example)?

@asimshankar
Thx for your answer, if I create and close session in the thread not the process, will tensorflow release gpu memory? My situation is that I use tf in c++, when calling tf is finished, gpu memory is not released, if I call tf in thread, will it release gpu memory when thread terminates? or some other solutions can I take

@TanLingxiao were you able to find any other method? numba is a great way with the drawback being that once you run cuda.close(), you can no longer run your process again in the same process/session. As alluded above and in other threads, Was hoping that tensorflow has config option to free GPU Memory after the processing ends.

Solution found in this link [https://stackoverflow.com/questions/15197286/how-can-i-flush-gpu-memory-using-cuda-physical-reset-is-unavailable]
sudo fuser -v /dev/nvidia*
Your output will look something like this:
                    USER        PID  ACCESS COMMAND
/dev/nvidia0:        root      1256  F...m  Xorg
                    username   2057  F...m  compiz
                    username   2759  F...m  chrome
                    username   2777  F...m  chrome
                    username   20450 F...m  python
                    username   20699 F...m  python

sudo kill -9 PID

I've elaborated on the trick used by @nathandouglas to create a decorator that will transparently launch your tf training/prediction function in a subprocess. It also takes care of hiding CUDA devices from TF so that it only uses the desired number of devices (you can easily remove that functionality if you don't need it). You'll need to wrap any code that creates a session to ensure that your GPU memory is released as it seems that TF carries out all the CUDA initialisation and device memory allocation when the first session is created. For example:
@RunAsCUDASubprocess(num_gpus=1)
def train_model():
  ...

train_model()

https://gist.github.com/ed-alertedh/85dc3a70d3972e742ca0c4296de7bf00

@calannap due to the way the multiprocessing module works you need to make sure that the arguments passed into your wrapped function can be pickled and unpickled. As a workaround try creating the swig objects inside your function (if possible).

@asimshankar I think there needs to be a way to get around this in the python instance. e.g. if I want to make a python script which consecutively runs several models with different params.
I have tried manually calling garbage collection but still resource usage increases:
import gc, resource
from .my.module import run_model

def wrapper_run_model(params)
   run_model(params)
   print(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
   gc.collect()

# my/module
def run_model(params):
    with sess as tf.Session():
        # do stuff
        # sess.close() <--- shouldn't need to add this as called when exiting with statement,
        # but tried just in case it helps

If I can not free resource from python, then how should I launch a series of experiments?

The problem is that even if I exit python in the terminal (ctl -Z) and restart another python process the memory is still occupied. I have to close the terminal for the memory to release.
…
On Fri, Feb 23, 2018 at 12:41 PM, Nathan Douglas @.***> wrote: I'm currently building out a workflow using Celery to perform a set of image analytics in parallel. I ran into this issues since the celery daemon, once started, never released GPU resources after it completed processing images that used TensorFlow. As a result, I quickly depleted all of my GPU resources. My interim solution to the problem, was to wrap my analytic in a Pool that spawns a single Process. It's not pretty, but it at least solves the problem in releasing GPU resources if you have a long running process like a celery daemon. import tensorflow as tf from multiprocessing import Pool def _process(image): sess = tf.Session() sess.close() def process_image(image): with Pool(1) as p: return p.apply(_process, (image,)) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#17048 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AJsMvwR_3Po8S7ANCWN0MJNkZIOfGm9wks5tXvhGgaJpZM4SHHWg .

For me Ctrl+Shift + C kills the python process and frees the gpus

Using numba causes Segmentation fault:
https://stackoverflow.com/questions/58792739/tensorflow-model-wrapper-that-can-release-gpu-resources

as stated in the link, DO NOT use memory growth:
"Once the memory is allocated, it will not be released to avoid memory fragmentation. So even if your process starts on low memory, it will acquire more memory gradually but it will not release when it don’t need it."
https://medium.com/ibm-data-ai/memory-hygiene-with-tensorflow-during-model-training-and-deployment-for-inference-45cf49a15688
use First Option — Specifically Set The Memory.

as stated in the link, DO NOT use memory growth:
"Once the memory is allocated, it will not be released to avoid memory fragmentation. So even if your process starts on low memory, it will acquire more memory gradually but it will not release when it don’t need it."
https://medium.com/ibm-data-ai/memory-hygiene-with-tensorflow-during-model-training-and-deployment-for-inference-45cf49a15688
use First Option — Specifically Set The Memory.

try tf.keras.backend.clear_session() as it will del the model that has been occupying memory.
not explicitly flush memory till empty but at least reducing model in memory.
"If you are creating many models in a loop, this global state will consume an increasing amount of memory over time, and you may want to clear it. Calling clear_session() releases the global state: this helps avoid clutter from old models and layers, especially when memory is limited."


[CINFRA-579] Enabling range deletes on followers

Scope & Purpose
This PR enables range deletes for truncate operations on followers. Before this, the followers always used remove operations, regardless of what the leader did.

[ ] :hankey: Bugfix
[x] :pizza: New feature
[ ] :fire: Performance improvement
[ ] :hammer: Refactoring/simplification

I remember that range deletes caused performance issues in the past. Is this still a thing?

From the tests I ran it is much faster than using remove operations.


fix(dataset-api): get_or_create creates a dataset for an existing table_name but different schema

SUMMARY
At Pinterest we were trying to use the get_or_create endpoint to automate the integration between our MetricsLayer and Superset. During our tests we've encountered the following issue: https://github.com/apache/superset/issues/30377
The issue happens because at the moment the get_or_create code does not account for the payload schema attribute, doing a search by table_name only.
This PR changes get_or_create to take the schema into account:

A new get_table_by_schema_and_name was added to the DatasetDAO class
get_or_create now checks if the dataset exists by calling DatasetDAO.get_table_by_schema_and_name (instead of DatasetDAO.get_table_by_name)

TESTING INSTRUCTIONS

Integration tests were added for the new behaviour
Manual tests: reproducing the steps described on the issue

Case 1 - False Positive

Go to the datasets page
Pick any existing dataset name and prepare a payload as follows (example using the users datasets)

{
  'table_name': 'users',
  'schema': 'other',
  'database_id': 1,
}

Submit this payload via a POST request to /api/v1/dataset/get_or_create
A new dataset is created. The API returns with a 200 pointing to the new dataset. No false positives anymore.

Case 2 - Internal Server Error

Create 2 or more datasets with the same table_name and different schemas (either via UI or create dataset API)
Try to create a new dataset. Again, with same table_name but a different schema. Payload example:

{
  'table_name': 'users',
  'schema': 'any_new_schema_name',
  'database_id': 1,
}

Submit this payload via a POST request to /api/v1/dataset/get_or_create
A new dataset is created. The API returns with a 200 pointing to the new dataset. No 500 errors anymore.

Backward Compatibility

Go to the datasets page
Pick any existing dataset name and prepare a payload as follows (without passing the schema)

{
  'table_name': 'users',
  'database_id': 1,
}

Submit this payload via a POST request to /api/v1/dataset/get_or_create
No new dataset is created. The API returns with a 200 with the response body pointing to the existing dataset.

ADDITIONAL INFORMATION

[x] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

Marking this PR as ready for review. test-sqlite is the only test failing.
However, it's failing with sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: tables.table_name, which is inherently opposed to the nature of this PR.
Furthermore, there are some signals that this constraint should be dropped:

it doesn't align with other databases (MySQL/Postgres) or with the overall Superset functionalities (as it does allow duplicate table_names for different databases/schemas)
The Github action logs says: SQLite Database support for metadata databases will be removed in a future version of Superset.
There's this migration that is removing the uniqueness constraint on the table_name, but is doing it for MySQLDialect only

I could use some guidance to decide whether or not dropping this constraint is the right decision. And if so, how to do it.
Thank you in advance.


improved apis for easier adoption

Pull Request Test Coverage Report for Build 882298456

43 of 45 (95.56%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+1.7%) to 81.725%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

matrix.go
43
45
95.56%

Totals

Change from base Build 882075514:
1.7%

Covered Lines:
398

Relevant Lines:
487

💛  - Coveralls


🐛 [Bug] Encountered RuntimeError: CUDA out of memory. bug when using TRTorch

Bug Description

I'm trying to compile two esrgan-based models but getting GPU OOM errors for both of them. They compile and run inference fine with torchscript.
Real-esrgan error:

INTERNAL_ERROR: [TRTorch Conversion Context] - [virtualMemoryBuffer.cpp::resizePhysical::65] Error Code 2: OutOfMemory (no further information)
ERROR: [TRTorch Conversion Context] - Requested amount of GPU memory<PHONE_NUMBER> bytes) could not be allocated. There may not be enough free memory for allocation to succeed.
torch.Size([1, 3, 2016, 2016])
Error The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
  File "code/__torch__/basicsr/archs/rrdbnet_arch.py", line 29, in forward
    input = torch.reshape(_8, [1, 12, 1008, 1008])
    _9 = (_7).forward(input, )
    input0 = torch.add(_9, (_5).forward((_6).forward(_9, ), ))
                                         ~~~~~~~~~~~ <--- HERE
    input1 = torch.upsample_nearest2d(input0, None, [2., 2.])
    _10 = (_3).forward((_4).forward(input1, ), )
  File "code/__torch__/torch/nn/modules/container.py", line 54, in forward
    _21 = getattr(self, "1")
    _22 = (getattr(self, "0")).forward(argument_1, )
    _23 = (_20).forward((_21).forward(_22, ), )
           ~~~~~~~~~~~~ <--- HERE
    _24 = (_18).forward((_19).forward(_23, ), )
    _25 = (_16).forward((_17).forward(_24, ), )
  File "code/__torch__/basicsr/archs/rrdbnet_arch/___torch_mangle_62.py", line 12, in forward
    argument_1: Tensor) -> Tensor:
    _0 = self.rdb3
    _1 = (self.rdb2).forward((self.rdb1).forward(argument_1, ), )
                              ~~~~~~~~~~~~~~~~~~ <--- HERE
    _2 = torch.mul((_0).forward(_1, ), CONSTANTS.c0)
    return torch.add(_2, argument_1)
  File "code/__torch__/basicsr/archs/rrdbnet_arch/___torch_mangle_47.py", line 26, in forward
    input1 = torch.cat([argument_1, _5, _6, _7], 1)
    _8 = (_4).forward3((_1).forward(input1, ), )
    input2 = torch.cat([argument_1, _5, _6, _7, _8], 1)
             ~~~~~~~~~ <--- HERE
    _9 = torch.mul((_0).forward(input2, ), CONSTANTS.c0)
    return torch.add(_9, argument_1)

Traceback of TorchScript, original code (most recent call last):
anaconda3/lib/python3.8/site-packages/basicsr/archs/rrdbnet_arch.py(37): forward
anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1039): _slow_forward
anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1051): _call_impl
anaconda3/lib/python3.8/site-packages/basicsr/archs/rrdbnet_arch.py(59): forward
anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1039): _slow_forward
anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1051): _call_impl
anaconda3/lib/python3.8/site-packages/torch/nn/modules/container.py(139): forward
anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1039): _slow_forward
anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1051): _call_impl
anaconda3/lib/python3.8/site-packages/basicsr/archs/rrdbnet_arch.py(114): forward
anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1039): _slow_forward
anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py(1051): _call_impl
anaconda3/lib/python3.8/site-packages/torch/jit/_trace.py(952): trace_module
anaconda3/lib/python3.8/site-packages/torch/jit/_trace.py(735): trace
Real-ESRGAN/realesrgan/utils.py(68): process
Real-ESRGAN/realesrgan/utils.py(197): enhance
anaconda3/lib/python3.8/site-packages/torch/autograd/grad_mode.py(28): decorate_context
inference_realesrgan.py(84): main
inference_realesrgan.py(102): <module>
RuntimeError: CUDA out of memory. Tried to allocate 374.00 MiB (GPU 0; 9.78 GiB total capacity; 551.39 MiB already allocated; 329.94 MiB free; 1.19 GiB reserved in total by PyTorch)

Esrgan error:
ERROR: [TRTorch Conversion Context] - Requested amount of GPU memory<PHONE_NUMBER> bytes) could not be allocated. There may not be enough free memory for allocation to succeed.
INTERNAL_ERROR: [TRTorch Conversion Context] - [virtualMemoryBuffer.cpp::resizePhysical::65] Error Code 2: OutOfMemory (no further information)
INTERNAL_ERROR: [TRTorch Conversion Context] - [virtualMemoryBuffer.cpp::resizePhysical::65] Error Code 2: OutOfMemory (no further information)
ERROR: [TRTorch Conversion Context] - Requested amount of GPU memory<PHONE_NUMBER> bytes) could not be allocated. There may not be enough free memory for allocation to succeed.
Segmentation fault (core dumped)

To Reproduce
Steps to reproduce the behaviour:
compile settings for traced_real_esrgan_half_cuda_2016
https://drive.google.com/file/d/1ByTicU48vl8v6Fxd_k5sni38g7z_Mt7w/view?usp=sharing
compile_settings = {
            "inputs": [trtorch.Input(
                    shape=[1, 3, 2016, 2016],
                    dtype = torch.half,
            )],
            "enabled_precisions": {torch.half}, # Run with FP16
            "workspace_size": 1 << 20,
            "truncate_long_and_double": True,
        }
        trt_ts_module = trtorch.compile(self.model, compile_settings)

compile settings for esrgan_traced
https://drive.google.com/file/d/1e7Tk5YwRH8TfjYlhrbh0LYVZrctOD0p7/view?usp=sharing
compile_settings = {
    "inputs": [trtorch.Input(
            min_shape = [1, 3, 16, 16],
            opt_shape = [1, 3, 2016, 1512],
            max_shape = [1, 3, 2016, 2016],
            dtype = torch.half,
    )],
    "enabled_precisions": {torch.half}, # Run with FP16
    "workspace_size": 1 << 20,
    "truncate_long_and_double": True,
}
trt_ts_module = trtorch.compile(model, compile_settings)

Expected behaviour

Compile and run like torchscript
Environment

Build information about the TRTorch compiler can be found by turning on debug messages

TRTorch Version (e.g. 0.2.0): 0.4.0a0
PyTorch Version (e.g. 1.0): 1.9
CPU Architecture: x86-64
OS (e.g., Linux): 20.04
How you installed PyTorch (conda, pip, libtorch, source): conda
Build command you used (if compiling from source): python3 setup.py install
Are you using local sources or building from archives: local
Python version: 3.8
CUDA version: 11.1
GPU models and configuration: RTX 3080
Any other relevant information: TensorRT <IP_ADDRESS>

Additional context

The TensorRT building stage can take more memory than is purely required for running the model. This may be the cause. Try adjusting your workspace size and see if that helps. I am interested in the fact that you got a segfault with this error. Any possibility of giving us the GDB backtrace there?

Hi, I'm having the same issue. Has anyone got a solution? Thanks!

+1

I have a 1.7 million param model and I am also getting the same issue. I cannot reduce the input size as it would imply retraining my model.


Support underscore in project name

To harmonize with crate naming I think Shuttle supporting underscore in the project name as well would be reasonable. After all, underscore is a legal character in a domain name.
MRE:
cargo install<EMAIL_ADDRESS>git clone git@github.com:shuttle-hq/shuttle.git
cd shuttle/examples/axum/hello-world
rm Shuttle.toml
sed -i.bkp 's/hello-world/hello_world/g' Cargo.toml
cargo shuttle run

Last line gives the following output:
Error:
`hello_world` is an invalid project name. project name must
1. not start or end with `-`.
2. not contain any characters outside of the alphanumeric range, except for `-`.
3. not be empty.

I can take this one

Closes by #299


There is a deepscan plug-in VScode used to scan all react code for unused variables, code practice and warnings. I am implementing that on our code base so that there might be some cleanup

Describe the bug
A clear and concise description of what the bug is. If your issue is regarding an error in the data, it's best reported through this form.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Please follow the issue format


Feature: sealos apply

sealos apply -f Clusterfile

Isn't this feature already implemented?

Yes,But there should be some problems.


[Feature Request] 测试下载/生成功能实现效果

Checklist

[X] I searched exising issues and no one else requests similar feature.
[X] I think that 25%+ users are positive towards this feature.

Describe the feature
测试 下载/生成功能效果
Additional context
No response

OK


about Inference

how can i inference with my own local images？

it seems like test all the testdataset?

We have a demonstration of arbitrary pose in the supplementary materials (haven't included in our arXiv preprint), which is in line with inferencing with our own images.
I will clean it up and upload that .ipynb file later this week.

thanks !

We upload the jupyter notebook for inference/reasoning. You could modify it as you want, e.g. replacing the conditional image with your customized one and randomly sampling a target pose from the test dataset.

Is there no way to use my own target pose?


Issues using hcxdumptool in conjuction with ALFA AWUS036ACH

GitHub is for bugs and features - not for support
Questions must be posted in the discussion board https://github.com/ZerBea/hcxdumptool/discussions
Check the FAQ
Please review the discussions
Comment output of:

$ sudo hcxdumptool -v

hcxdumptool 6.3.4-17-g904a2a8 (C) 2024 ZeroBeat
running on Linux kernel 6.6.31+rpt-rpi-v8
running GNU libc version 2.36
compiled by gcc 12.2.0
compiled with Linux API headers 6.6.31
compiled with GNU libc headers 2.36
enabled REALTIME DISPLAY
enabled GPS support
enabled BPF compiler

$ sudo hcxdumptool -l

  0	  3	e45f0166cd50	e45f0166cd50	-	wlan0           	brcmfmac	NETLINK
  1	  4	00c0cab035be	00c0cab035be	+	wlan1           	rtl88XXau	NETLINK

Describe the bug
When using hcxdumptool with an ALFA AWUS036ACH adapter running the aircrack-ng rtl8812au driver, hcxdumptool launches without any warnings, and starts running. When running, hcxdumptool doesn't populate the RDS with any access points or clients. Upon termination, I'm hit with the following output:
0 ERROR(s) during runtime
0 Packet(s) captured by kernel
0 Packet(s) dropped by kernel
Warning: too less packets received (monitor mode may not work as expected)
Possible reasons:
 no transmitter in range
 frames are filtered out by BPF
 driver is broken
Warning: no PROBERESPONSES received (frame injection may not work as expected)
Possible reasons:
 no AP in range
 frames are filtered out by BPF
 driver is broken
 driver does not support frame injection

exit on watchdog

All of these errors point to a driver malfunction but I've seen others have success with the aircrack-ng rtl8812au driver.
To Reproduce
sudo hcxdumptool -i wlan1 -F --rcascan=a
Expected behavior
I expected hcxdumptool to startup without errors and begin scanning for surrounding access points and clients.
Screenshots
If applicable, add screenshots to help explain your problem.
Note: Instead of individual screenshots, I recorded this video showcasing the bug.
https://github.com/ZerBea/hcxdumptool/assets/91102919/f49d07fc-206d-4642-a390-bb51e17f8100
Additional context
Add any other context about the problem here.
All of the evidence points towards a driver malfunction/error but hcxdumptool does behave in different ways when different drivers are used. For example, this driver by astsam is old and causes hcxdumptool to terminate before it starts scanning. I'm also aware that hcxdumptool does not support 3rd party drivers, I'd just like to figure out where the problem is occurring in my setup.

Your guess is correct. This are driver issues and exactly that is the reason why I do not recommend to use chipsets that depend on third party drivers.
README.md
No support for a third party driver which is not part of the official Linux kernel (https://www.kernel.org/)
Report related issues to the site, from which you downloaded the driver.

Each driver coming from git exhibits a different behavior (especially in combination with different kernel versions). Nearly all of them depend on the old hal modules and absolutely none of them will find its way into the Linux kernel tree.
Please read this:
https://github.com/morrownr/USB-WiFi/issues/314
and this:
https://github.com/morrownr/USB-WiFi/issues/449
Luckily there is work in progress (new rtw88 stack) to add RTL8812au support to the Linux kernel:
https://github.com/morrownr/8821au-20210708/issues/133#issuecomment-2159223160
I'm monitoring this rtw88 driver too, as well as the maintainer of the aircrack-ng driver (kimocoder)
I've tested the driver in combination with a RTL8811 device and it is working as expected
I recommend to test this driver and to report possible problems there.

@Jake-Grafton @ZerBea
The new 8812au driver is ready for initial testing. Read the first message at the following link for details:
https://github.com/morrownr/8812au-20210820/issues/27
We really need testers to beat up monitor mode.

First of all, a very sad news:
https://lore.kernel.org/linux-wireless/ds6wc3svkyre4p2rwg4f76o67xndazaeoyjwblhuzichscyxoz@5ttdvbymxr55/T/#mfd2f4928e0e013c10375ca766035c1385e18f8e0
RiP Larry and thanks for your great work. The community will remember you!

@Jake-Grafton
At the moment I only can test 8811au. Maybe I can put my hands on a cheap 8812au device, soon.

@ZerBea
Understood. I just tested my 8812au based device with the new driver and it worked just fine! (see attached screenshot for hcxdumptool output)

Glad to hear that it is working.
Latest Linux kernels need latest driver. There is no place for the ancient "hal" ones.
BTW:
rt88/rtw99 will be continued.

An additional information:
None of my mt6 based combined WiFi / BT devices is really working as expected on USB3 (AMD mainboards).
For me it is mandatory to prevent that btusb will be loaded:
echo "install btusb /bin/true" >> /etc/modprobe.d/blacklist.conf
Otherwise they do not work:
[11321.446538] usb 5-2.4: new high-speed USB device number 5 using xhci_hcd
[11321.541550] usb 5-2.4: New USB device found, idVendor=0e8d, idProduct=7961, bcdDevice= 1.00
[11321.541557] usb 5-2.4: New USB device strings: Mfr=6, Product=7, SerialNumber=8
[11321.541560] usb 5-2.4: Product: Wireless_Device
[11321.541561] usb 5-2.4: Manufacturer: MediaTek Inc.
[11321.541563] usb 5-2.4: SerialNumber: 000000000
[11321.770836] Bluetooth: Core ver 2.22
[11321.770866] NET: Registered PF_BLUETOOTH protocol family
[11321.770867] Bluetooth: HCI device and connection manager initialized
[11321.770872] Bluetooth: HCI socket layer initialized
[11321.770874] Bluetooth: L2CAP socket layer initialized
[11321.770878] Bluetooth: SCO socket layer initialized
[11321.807855] usbcore: registered new interface driver btusb
[11321.810801] Bluetooth: hci0: HW/SW Version: 0x008a008a, Build Time:<PHONE_NUMBER>1427
[11324.591334] Bluetooth: hci0: Device setup in 2718184 usecs
[11324.591341] Bluetooth: hci0: HCI Enhanced Setup Synchronous Connection command is advertised, but not supported.
[11326.646123] Bluetooth: hci0: Opcode 0x0c03 failed: -110
[11328.779479] Bluetooth: hci0: Failed to read MSFT supported features (-110)
[11330.913067] Bluetooth: hci0: AOSP get vendor capabilities (-110)
[11331.002514] usb 5-2.4: reset high-speed USB device number 5 using xhci_hcd

The faulty BT module blocks the USB port. As a result neither WiFi nor BT is working!

@ZerBea

None of my mt76 based combined WiFi / BT devices is really working as expected on USB3 (AMD mainboards).

I have seen reports of this but had not been able to duplicate it. You added something to the conversation that I had not seen before because most users probably did not consider it to be relevant: AMD mainboards
I very recently got a new Mini PC with an AMD CPU. So I decided to test my AXML. I was able to duplicate the problem. So as to not derail this thread, I'll take this email.


Async methods to fix freezing LiveSplit when doing requests, escaped characters.

Thanks for this!
I'm still not as fluent in C#, so haven't worked with async methods "properly" yet - though i had thought about this before.
I'll double-check the changes tonight; from a glance, I'm not 100% sure if it may cause issues with a discrepancy between time-of-split and time-of-check for the resolution.
This partially addresses #1

I'll publish these changes as part of v2.1 at some point soon, including the Release Binary.


Is this still being maintained?

 It would likely require quite a few significant changes to get Strongback working with the 2019 version of WPILib. There are, however, many similar libraries that are being actively maintained this year:

https://github.com/TeamMeanMachine/meanlib
https://github.com/ezAuton/ezAuton
https://github.com/DurhamAcademy/kyberlib
https://github.com/sertain/sertain
https://github.com/team401/SnakeSkin

@andewda, thanks for ur response. Sorry I didn't see it until now! I'll check those out for next season.


Can't select text inside LineEdit

Needs to be implemented.
Should support but both Shift+Arrow keys and mouse selection.

I'll try and take care of this too, I kinda know how chat works at this point.

This needs to be closed


Missing information_schema tables from MySQL gateway

Hi there,
I tried pointing Tableau at the MySQL gateway (harsh, I know) and unsurprisingly it failed.  It did manage to handshake and identify the databases, but then it failed when trying to get the schema information.  There are at least 2 problems and I'm logging them separately.  The first is that it is looking for a key_column_usage table in the information_schema that does not exist.  I've included the first connection where all appears well in case it has any information that is useful, but the error happens on connection 2.

Query:
SELECT COLUMN_NAME,
 REFERENCED_TABLE_NAME,
 REFERENCED_COLUMN_NAME,
 ORDINAL_POSITION,
 CONSTRAINT_NAME
FROM information_schema.KEY_COLUMN_USAGE
WHERE REFERENCED_TABLE_NAME IS NOT NULL
 AND TABLE_NAME = 'client_logs'
 AND TABLE_SCHEMA = 'plyql1'
If you believe this query should work please create an issue on PlyQL and include this section
Issue url: https://github.com/implydata/plyql/issues
Message:
could not resolve $KEY_COLUMN_USAGE
Stack:
Error: could not resolve $KEY_COLUMN_USAGE
    at RefExpression._fillRefSubstitutions (/home/ben/imply-1.2.1/imply-1.2.1/dist/pivot/node_modules/plywood/build/plywood.js:6683:23)
    at ChainExpression._fillRefSubstitutions (/home/ben/imply-1.2.1/imply-1.2.1/dist/pivot/node_modules/plywood/build/plywood.js:7111:49)
    at ChainExpression.Expression.referenceCheckInTypeContext (/home/ben/imply-1.2.1/imply-1.2.1/dist/pivot/node_modules/plywood/build/plywood.js:6183:18)
    at ChainExpression.Expression.referenceCheck (/home/ben/imply-1.2.1/imply-1.2.1/dist/pivot/node_modules/plywood/build/plywood.js:6176:25)
    at ChainExpression.Expression._initialPrepare (/home/ben/imply-1.2.1/imply-1.2.1/dist/pivot/node_modules/plywood/build/plywood.js:6277:18)
    at /home/ben/imply-1.2.1/imply-1.2.1/dist/pivot/node_modules/plywood/build/plywood.js:6318:45
    at _fulfilled (/home/ben/imply-1.2.1/imply-1.2.1/dist/pivot/node_modules/q/q.js:834:54)
    at self.promiseDispatch.done (/home/ben/imply-1.2.1/imply-1.2.1/dist/pivot/node_modules/q/q.js:863:30)
    at Promise.promise.promiseDispatch (/home/ben/imply-1.2.1/imply-1.2.1/dist/pivot/node_modules/q/q.js:796:13)
    at /home/ben/imply-1.2.1/imply-1.2.1/dist/pivot/node_modules/q/q.js:604:44

I guess there is no going around adding all the information_schema tables

Please note that this project is now deprecated. Please see the Druid SQL docs for information on using SQL syntax with Druid.


Enhancement of Song Data Storage and Access

The current song_data.py stores songs in a simple dictionary with only song names and YouTube links. To improve functionality, I propose expanding the song data to include details like artist, genre, mood, and duration, which would support more sophisticated features such as filtering and recommendations. Additionally, moving this data to an external JSON or CSV file would enhance scalability and ease of management. Implementing search functions and caching with functools.lru_cache will improve data retrieval efficiency. Lastly, adding documentation and type hints will enhance code readability and maintainability. Feedback on this enhancement is welcome, particularly regarding additional metadata or structure.

can you assign me this isssue i want to work on it as gssoc contributor.

Hi
Thank you for your thoughtful proposal regarding the enhancement of
song_data.py. I agree that expanding the song metadata to include
additional details like artist, genre, mood, and duration would
significantly improve functionality, especially for filtering and
recommendations. Moving the data to a JSON or CSV file is also a great idea
for scalability.
You're welcome to work on this issue as a GSSoC contributor! Please proceed
with the following steps:

Expand the song data structure to include the new metadata fields.
Move the data to an external JSON or CSV file for ease of management.
Implement search and filtering functions, with caching using
functools.lru_cache to improve retrieval efficiency.
Ensure type hints and documentation are added to enhance readability
and maintainability.
Create a detailed PR, outlining all the changes and improvements.

Let me know if you need any further guidance or clarification. Looking
forward to seeing your implementation!
Best regards,
Avanish Singh
Project Admin, GSSoC
On Tue, 1 Oct, 2024, 8:43 pm Satyam Singh, @.***> wrote:

can you assign me this isssue i want to work on it as gssoc contributor.
—
Reply to this email directly, view it on GitHub
https://github.com/suryanshsk/Python-Voice-Assistant-Suryanshsk/issues/1#issuecomment-2386281544,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AVBBZFL2ZMYSDN4CLDYOQQTZZK3ZDAVCNFSM6AAAAABPFT2LOKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBWGI4DCNJUGQ
.
You are receiving this because you are subscribed to this thread.Message
ID: <suryanshsk/Python-Voice-Assistant-Suryanshsk/issues/1/2386281544@
github.com>

Hi
Thank you for your thoughtful proposal regarding the enhancement of
song_data.py. I agree that expanding the song metadata to include
additional details like artist, genre, mood, and duration would
significantly improve functionality, especially for filtering and
recommendations. Moving the data to a JSON or CSV file is also a great idea
for scalability.
You're welcome to work on this issue as a GSSoC contributor! Please proceed
with the following steps:

Expand the song data structure to include the new metadata fields.
Move the data to an external JSON or CSV file for ease of management.
Implement search and filtering functions, with caching using
functools.lru_cache to improve retrieval efficiency.
Ensure type hints and documentation are added to enhance readability
and maintainability.
Create a detailed PR, outlining all the changes and improvements.

Let me know if you need any further guidance or clarification. Looking
forward to seeing your implementation!
Best regards,
Avanish Singh
Project Admin, GSSoC
On Tue, 1 Oct, 2024, 7:14 pm GCJ0310, @.***> wrote:

The current song_data.py stores songs in a simple dictionary with only
song names and YouTube links. To improve functionality, I propose expanding
the song data to include details like artist, genre, mood, and duration,
which would support more sophisticated features such as filtering and
recommendations. Additionally, moving this data to an external JSON or CSV
file would enhance scalability and ease of management. Implementing search
functions and caching with functools.lru_cache will improve data retrieval
efficiency. Lastly, adding documentation and type hints will enhance code
readability and maintainability. Feedback on this enhancement is welcome,
particularly regarding additional metadata or structure.
—
Reply to this email directly, view it on GitHub
https://github.com/suryanshsk/Python-Voice-Assistant-Suryanshsk/issues/1,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AVBBZFNEMWOGZPPNDON2BJTZZKRK7AVCNFSM6AAAAABPFT2LOKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGU2TSMRZGU4TAOI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Inconsistent inference results from clearml-serving

Hello,
I deployed a model using clearml-serving, but it generate inconsistent results across same HTTP requests.
To recreate:

I deployed a self-hosted clearml server in my local kubernetes (from docker image allegroai/clearml:1.4.0).
Reused the pytorch MNIST example from https://github.com/allegroai/clearml-serving/tree/main/examples/pytorch.
Went through the model training process.
Installed clearml-serving with helm (helm repo: NAME allegroai/clearml-serving, CHART VERSION 0.4.1, APP VERSION 0.9.0).
Deployed the MNIST model to a serving endpoint.
Tested the endpoint "http://ip:port/serve/test_model_pytorch" using POSTMAN

Everything goes well as the readme.md from https://github.com/allegroai/clearml-serving/tree/main/examples/pytorch instructed.
But mysteriously, the HTTP responses are not consistent! (The MNIST model occasionally returns different "digits" from the same input image)
I'm quite confused here, and have no idea if any random process happens during the model inference.
Thanks for any help!

Hi @Muscle-Oliver , sorry for the late reply.
Do you have an update on this, or has it not changed since?
Do you think you could try running the model just in python, not in clearml serving and seeing if the same thing happens there?
ClearML Serving uses Triton on the backend, so it could be that it is an issue with Triton. Can you try the newest version of ClearML Serving (1.2 was just released) and see if the issue persists?


use uuid instead of crypto for session-key generation.

This fixes a race condition when sessions are created at the same time.
Since the session has not been saved yet, it has no id, thus sessions
created at the exact same time have the exact same session-key. Using
the existing node-uuid module is a clean, simple solution.

Hmmm... I think we need to do something better here -- uuid's help with concurrent session creation, but neither they nor our current strategy are unguessable (which the session keys really need to be).  I think we need to introduce either some entropy or an hmac here.

Pardon me, I was confusing sesion-key with sock-key.  Guessability requirements for session-key are much less; uuid's are probably adequate.


Revert "improve the scrollbar behavior when viewport size changed"

Reverts flutter/flutter#76102
A customer test in google is broken by this change. The customer code needs to be updated as this appears to have exposed a bug on their end. Once the bug is fixed we should be able to re-land. :)
FYI @xu-baolin

@Piinks Hi, good day, any news about the customer's tests? : )

@Piinks Hi, good day, any news about the customer's tests? : )

Hey @xu-baolin ! Feel free to re-stage this and I will double check before we land it again. :) Thank you!


❯ ligature in terminal

Neovide does ligatures rendering in normal mode (NERDTree) but fails to render "❯" ligature in terminal mode.
I have guifont set to set FiraCode\ Nerd\ Font:h12.
Terminal nvim:

neovide:

I could very well be wrong, but I think thats not a ligature? It looks like a special symbol in which case the bug is font fallback not working or the character not being present in the default fallback fonts

Yes indeed you are right, it's not ligature, it's a special character printed by starship.
I have the same font in iTerm2 and guifont of Neovide but only Neovide fails to render it...

yes likely iterm2 uses font fallback because the character doesn't exist in your configured font. Unfortunately due to the font management library options in rust at the moment, there isn't a good way to take advantage of operating system font fallback. SO to get around this I recently added a manual font fallback feature to the guifont configuration.
If you can find a font that for sure has that glyph supported, you can add multiple font names separated by commas in the guifont format.
For example: let guifont=Consolas,FiraCode:h14
This will set the first font to consolas, use firacode as a fallback, and set the font size to 14.

Perfect, thanks!


Data:,

the program only openn multiple times chrome with the text "Data:," in adress bar but never opens the right link

This is my config
{
"http_api": {
"enabled": true,
"host": "<IP_ADDRESS>",
"port": 5000
},
"database": true,
"views": 100,
"minimum": 85.0,
"maximum": 95.0,
"proxy": {
"category": "f",
"proxy_type": false,
"filename": "C:\Users\Mohammed Nehan\Desktop\proxy.txt",
"authentication": false,
"proxy_api": false,
"refresh": 0.0
},
"background": false,
"bandwidth": true,
"playback_speed": 1,
"max_threads": 5,
"min_threads": 2
}

You have to put in something into search.txt file
"name of the video :::: "

You have to put in something into search.txt file
"name of the video :::: "
Do you know how the program works with premium socks5 ip?
I always got an error connection, using socks5 proxy.

Works for me better just using premium and http .  Change your port to http
on your proxy and change the setting to use http.
On Sun, Mar 13, 2022 at 4:46 PM nashv95 @.***> wrote:

You have to put in something into search.txt file
"name of the video :::: "
Do you know how the program works with premium socks5 ip?
I always got an error connection, using socks5 proxy.
—
Reply to this email directly, view it on GitHub
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_MShawon_YouTube-2DViewer_issues_342-23issuecomment-2D1066188281&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=fYWo8lDJdmUZ4ZkdM7LJPljTTNubPbg5EVwIzEWmhH8&m=61fy5CoVjz0CSO6Fm-gFPQKU2ZIp_6wz91a0h6Ehv6nCJguuk45GdL_zDjunoUh0&s=2Dj3d7wN2nrS0O_Pd-x6QTzzooyRobgZo2v4m0stac4&e=,
or unsubscribe
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AMUYSBTDYB723OJ52UPWTADU7ZO3FANCNFSM5QQG5GTA&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=fYWo8lDJdmUZ4ZkdM7LJPljTTNubPbg5EVwIzEWmhH8&m=61fy5CoVjz0CSO6Fm-gFPQKU2ZIp_6wz91a0h6Ehv6nCJguuk45GdL_zDjunoUh0&s=-0B_UAgQW_5FXAXrs4-Ddi_R4bYUSwDsBKEkZeHtE9Q&e=
.
Triage notifications on the go with GitHub Mobile for iOS
https://urldefense.proofpoint.com/v2/url?u=https-3A__apps.apple.com_app_apple-2Dstore_id1477376905-3Fct-3Dnotification-2Demail-26mt-3D8-26pt-3D524675&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=fYWo8lDJdmUZ4ZkdM7LJPljTTNubPbg5EVwIzEWmhH8&m=61fy5CoVjz0CSO6Fm-gFPQKU2ZIp_6wz91a0h6Ehv6nCJguuk45GdL_zDjunoUh0&s=L_EtoJwfcI-uvzDwEYtw1clSia-FWreM8gVVjq-zsKY&e=
or Android
https://urldefense.proofpoint.com/v2/url?u=https-3A__play.google.com_store_apps_details-3Fid-3Dcom.github.android-26referrer-3Dutm-5Fcampaign-253Dnotification-2Demail-2526utm-5Fmedium-253Demail-2526utm-5Fsource-253Dgithub&d=DwMCaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=fYWo8lDJdmUZ4ZkdM7LJPljTTNubPbg5EVwIzEWmhH8&m=61fy5CoVjz0CSO6Fm-gFPQKU2ZIp_6wz91a0h6Ehv6nCJguuk45GdL_zDjunoUh0&s=zorY8WdYJ1m4NAHYrlI3Ir8U6V8Oa1ZiLaE5Jw6sFBo&e=.
You are receiving this because you commented.Message ID:
@.***>

--
http://www.verizon.com
Logan Schoessow
Engr IV Cslt-Ntwk Eng&Ops
Wireline Network Ops & Eng
M:<PHONE_NUMBER>
http://www.facebook.com/verizon http://twitter.com/verizon
http://www.linkedin.com/company/verizon
http://www.instagram.com/verizon

@letsfootball  what's in your urls.txt and search.txt?


Does the voice call demo use the same model trained by this repo

HI, thanks for your work. I have tried this voice call and found it can take streaming audio as the input if I am correct.
I am wondering (1) if the voice call demo use the same model trained by this repo? (2) how do you implement the voice call demo.
Thanks very much!

@viyjy yes, it uses the same model. You can do the same thing using our apis: docs.ultravox.ai


[AutoPR billing/resource-manager] [Microsoft.Billing]: Updating swagger with rename changes required.

Created to sync https://github.com/Azure/azure-rest-api-specs/pull/4495

This PR has been merged into https://github.com/Azure/azure-sdk-for-js/pull/555


docker: for builds

i have Dockerfile that builds all the projects in a container. if you're wanting to include it in the official build ci, i can go ahead and clean it up and make it ready for general use.
might help to reduce compiler errors that intermittently show up. the only limitation that i can think of; the build will run on linux, unless MS have a dev. image that runs in Docker?

@mbana we use travis to build in docker on checkin. Locally we build using the tsc compiler. Did you experience any problems with this. For me using docker locally to compile is a little bit an overkill since the tsc setup is easy with npm.

i was thinking of having something akin to a pre-commit hook that would compile and push, iff, it didn't err.
ps. w.r.t to WebSockets,
if you don't mind could you feedback your thoughts about http://azure.bana.io/monaco-go/. it's using go-langserver on the backend side.
i'd love for some of the patches that i have in my branch to be merged upstream.
are you still busy?

As said we use travis for builds and they even run when you create a branch. I will close this since I don't see big advantage of supporting docker locally.


async normalizer use case

You want your users to enter correct phone numbers, in a format such as the international format (starts with +), regardless of the fact that the users might not be aware of that format and still insist on entering local numbers.
As a solution, think about formatting the phone numbers on the backend using a library like libphonenumber. When users enter the phone, async validation will either reject the phone, or will replace the field value with formatted phone number content.
So therefore, a sync normalization on phone input would not be sufficient.
I hope this is a good motivation :)

This is a workaround for validating multiple phone numbers on the form, using asyncValidate. It can be used for async normalizing any number of fields.
It was simpler than I originally thought, thanks to redux form.
const asyncValidate = (values, dispatch) => {
    let err = {};

    return Promise.all(
    asyncValidateFieldNames.map(field =>
        validator.getVerifiedPhoneNumber(values[field], values.chooseCountry).then(function(verifiedNumber){
            const verNum = Object.assign({},verifiedNumber)  as PhoneVerificationResponse;
            if (verNum.isValid) {
                dispatch(change('signupForm', field, verNum.formattedNumber));
            } else  {
                err[field]= "Number looks invalid";
            }
        })
    )).then(() => {
        if(err !== {})
            throw( err)

    }   )
}

Because this issue saw almost no activity for a few months, I'm closing it.

I have a similar issue where a user clicks on a checkbox and confirmation box is displayed. I need to wait for the user click event to decide on the return value.. though there are workarounds..it would be really great to have a async normalizer. Please reopen this issue.

+1 for async normalizer


Allowing hosts to share the mounted volumes as either read only or as SELinux write compatiable volume.

Changelog
Fill this in and update CHANGELOG.md
Issues fixed or closed
Questions (open the PR then click the check boxes)
Did you update the documentation related to your changes?

[x] Yes
[ ] My changes were not already documented

Did you run make before committing code and opening this PR?

[x] Yes
[ ] I didn't change any code

Did you run porter create-stack and porter sync-stack to verify provisioning
works?

[x] Yes
[ ] N/A

Looks good just run make and commit any changes

I have added the formatting changes.

👍

@davepersing can you take a look?

👍


docs: link to checklist in voring document that is linked in every email

Platforms affected
coho
Motivation and Context

We link to the voting guidelines in every vote email. This PR adds the new checklist as a TLDR at the top to make it more accessible and encourage votes.
Description

Just a link in docs.
Testing

Checklist

[ ] I've run the tests to see all new and existing tests pass
[ ] I added automated test coverage as appropriate for this change
[ ] Commit is prefixed with (platform) if this change only applies to one platform (e.g. (android))
[ ] If this Pull Request resolves an issue, I linked to the issue in the text above (and used the correct keyword to close issues using keywords)
[ ] I've updated the documentation if necessary

Codecov Report

Merging #287 (37d7bd5) into master (277dcc7) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #287   +/-   ##
=======================================
  Coverage   58.73%   58.73%
=======================================
  Files           8        8
  Lines         492      492
=======================================
  Hits          289      289
  Misses        203      203

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 277dcc7...37d7bd5. Read the comment docs.


feat: Disable rules already checked by the TypeScript compiler

Fixes #144

:tada: This PR is included in version 11.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


did i miss something , still get the same  error  java.lang.NoClassDefFoundError  io.indexr.compress.bh.BhcompressLibrary

i run the shell
sh script/setup_lib.sh
sh script/setup_indexr-segment.sh
sh script/setup_indexr-query.sh
and check it ,BhcompressLibrary.java  was  generated. and recompile the sorce .
indexr-segment-0.1.0.jar   also  has the class file

so i  repalce the jar file  in  every node . including the indextools .
and  restart the drill cluster.    after period  of a time , the error is found  when save in memory rows to disk.
did i miss some step ?
the error msg ==>
16:47:28.981 [IndexR-RT-Handle-4] ERROR io.indexr.segment.rt.RTSGroup - Save in memory rows to disk failed. [table: auth_data4, rtsg: rt/rtsg.201701111551.87dee7ce-877d-471c-a9f3-dc4197dcef5d.seg, segment: rts.201701111557.d1838032-d9df-4f96-bdd4-be32434e587e.seg]
java.lang.NoClassDefFoundError: Could not initialize class io.indexr.compress.bh.BhcompressLibrary
at io.indexr.compress.bh.BHCompressor.compressInt(BHCompressor.java:68) ~[indexr-segment-0.1.0.jar:na]

@AisTop
 Could not initialize class io.indexr.compress.bh.BhcompressLibrary may cause by cpp libs not correctly setting up.
Make sure you have done this:

Deploy all boost libs in /usr/local/lib/ to all nodes, not only the libboost_system.so
Deploy libbhcompress.so to /usr/local/lib/ in all nodes.

And it turns out that the old deployment document is misleading people. You should deploy all boost libs in /usr/local/lib to all nodes, not only the libboost_system.so. I have correct it now.
Really sorry for that.

It's ok. think you very much.


Pinecone metadata filtering - "cannot provide both filter and this.filter"

I am going to work on a pull request to fix this issue. An empty object is being passed in as filter by default on similaritySearchWithScore, so I am not able to pass filter into fromExistingIndex().
Here is where the problem is in the langchain source:
pinecone.ts
async similaritySearchVectorWithScore(
    query: number[],
    k: number,
    filter?: object
  ): Promise<[Document, number][]> {
    if (filter && this.filter) {
      throw new Error("cannot provide both `filter` and `this.filter`");
    }
    const _filter = filter ?? this.filter;
    const results = await this.pineconeIndex.query({
      queryRequest: {
        includeMetadata: true,
        namespace: this.namespace,
        topK: k,
        vector: query,
        filter: _filter,
      },
    });

vectorstores/base.ts
async similaritySearch(
    query: string,
    k = 4,
    filter: object = {}
  ): Promise<Document[]> {
    const results = await this.similaritySearchVectorWithScore(
      await this.embeddings.embedQuery(query),
      k,
      filter
    );

    return results.map((result) => result[0]);
  }

  async similaritySearchWithScore(
    query: string,
    k = 4,
    filter: object = {}
  ): Promise<[object, number][]> {
    return this.similaritySearchVectorWithScore(
      await this.embeddings.embedQuery(query),
      k,
      filter
    );
  }

Thanks for fixing this @ElektrikSpark !


Feature 84 docker centos7 - change from centos8

Pull Request Testing
Loaded data to database from Docker container.  Made README file more detailed, matching test.

[ ] Describe testing already performed for these changes:

[ ] Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

[ ] Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes or No]

[ ] Do these changes include sufficient testing updates? [Yes or No]

[ ] Will this PR result in changes to the test suite? [Yes or No]
If yes, describe the new output and/or changes to the existing output:

[ ] Please complete this pull request review by [Fill in date].

Pull Request Checklist
See the METplus Workflow for details.

[ ] Review the source issue metadata (required labels, projects, and milestone).
[ ] Complete the PR definition above.
[ ] Ensure the PR title matches the feature or bugfix branch name.
[ ] Define the PR metadata, as permissions allow.
Select: Reviewer(s)
Select: Organization level software support Project or Repository level development cycle Project
Select: Milestone as the version that will include these changes
[ ] After submitting the PR, select Linked issues with the original issue number.
[ ] After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
[ ] Close the linked issue and delete your feature or bugfix branch from GitHub.

I tried to build a container using the command:
docker build .
The build finished with exit code: 1 and the error message:
executor failed running [/bin/sh -c yum -y update  && yum -y install epel-release  && yum -y install 'dnf-command(config-manager)'  && yum-config-manager --set-enabled powertools]: exit code: 1
and
#5 136.6 Loaded plugins: fastestmirror, ovl
#5 136.6 Command line error: no such option: --set-enabled
#5 136.6 Usage: yum-config-manager [options] [section ...]

I corrected the powertools line


Update theme.toml

I was getting this error using while flowing the hugo quick start, changing to a string fixed it.
Error: failed to read module config for "ananke" in "/Users/theyes/Projects/hugo/demo/themes/ananke/theme.toml": unmarshal failed: Near line 11 (last key parsed 'min_version'): Invalid float value: "0.30.2"

#186 is the same, I'll close this.


Bump core-js for es5 package

Fixes #2187
Changelog

Fix #2187. Bump core-js and update core-js modules on index-es5.js, by @corinagum in PR #2195

Description
Changes made by adding eslint rules caused core-js modules to fail on the es5 build of Web Chat. This fix bumps core-js in bundle and modifies the imported modules to the index-es5 module.
Specific Changes

bump core-js
update core-js modules in index-es5.ts

This will be patched to 4.5

[ ] Testing Added

Coverage remained the same at 63.308% when pulling bc10372648716a4eef0f64cea1301c4595b87d44 on corinagum:fix-2187 into 6e5ca98f66598af6ecd3cb3ac2aa6b30a3556c37 on microsoft:master.


Obligee, Obligor, and Owner Roles - Where are they?

@mbrochhausen - last year we discussed 'obligee role', 'obligor role', and 'owner role' for extending ICO for clinical consent documents. When searching in Ontology Lookup Service, these terms are still discoverable under an IAO purl (see below image). However, they do not appear to be classes in IAO anymore - further they are not discoverable as obsoleted classes. Can you or someone from IAO help me find documentation about where these terms went?
Example

Hi Liz,
regarding d-acts, ontology lookup service is not a locus of authority
(sadly). Even more sadly, Ontobee (currently) cannot not be used for this
purpose, due to an PURL issue created by the OBO Foundry at the time. I am
working with James Overton to resolve that matter, but it might be a few
weeks until that is done.
To look up the newest version of d-acts, please refer to
http://purl.obolibrary.org/obo/iao/d-acts.owl
Best,
Mathias
On Fri, Mar 26, 2021 at 3:22 PM Elizabeth (Liz) Umberfield <
@.***> wrote:

@mbrochhausen https://github.com/mbrochhausen - last year we discussed 'obligee
role'
https://www.ebi.ac.uk/ols/ontologies/omrse/terms?iri=http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FIAO_0020026,
'obligor role'
https://www.ebi.ac.uk/ols/ontologies/omrse/terms?iri=http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FIAO_0020025,
and 'owner role'
https://www.ebi.ac.uk/ols/ontologies/omrse/terms?iri=http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FIAO_0020027
for extending ICO for clinical consent documents. When searching in
Ontology Lookup Service, these terms are still discoverable under an IAO
purl (see below image). However, they do not appear to be classes in IAO
anymore - further they are not discoverable as obsoleted classes. Can you
or someone from IAO help me find documentation about where these terms went?
Example
[image: image]
https://user-images.githubusercontent.com/44447480/112688029-aca4b200-8e4e-11eb-89cb-770f63516d17.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/information-artifact-ontology/IAO/issues/243, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ACF6DLTSZSGVCA2QRYVWEDLTFTUJVANCNFSM4Z4BGKIQ
.

Hi Mathias,
Thanks for the speedy reply. I appreciate you sending me the most recent version.
I've also noticed quite a few errors that populate when searching in Ontobee lately, not just with IAO classes. Hoping this gets sorted soon.
Best,
Liz


Create 11. ContainerWithMostWater.cpp #461

Pull Request Template
What have you Changed(must)
Added solution for 11. container with most water problem
Issue no.(must) - #461
Pr will be close and marked as spam. If issue number not found or issue was assigned to someone else.
Marking as spam can block your account from HacktoberFest.
Self Check(Tick After Making pull Request)

[x] This issue was assigned to me if not it will be marked as spam.
[x] My file is in the proper folder
[x] I am following clean code and Documentation.
[x] I have added the title and what program will do first in the file
[x] I have named the file as (Problem #). (Problem Title(in Camel Case)).(extension)
ex. 2. AddToNumbers.py

README - How to Contribute

Please review and merge

@sau-rav I've assigned this issue to you now but make sure you wait to get assigned before making a PR.

Okay I will make sure from now on :)


Dynamic features support bug.

Sceneform doesn't work in dynamic features due to its way of addressing resources.
Unable to load Texture registryId='android.resource://com.arfeature.arprofile/drawable/sceneform_plane'     java.util.concurrent.CompletionException: java.lang.IllegalStateException: java.io.FileNotFoundException: No package found for authority: android.resource://com.arfeature.arprofile/drawable/sceneform_plane     java.util.concurrent.CompletionException: java.io.FileNotFoundException: No package found for authority: android.resource://com.arfeature.arprofile/raw/sceneform_plane_shadow_material
Demo project with proper resource addressing and demo of the issue: https://goo.gl/GmufL3
This is a follow-up to https://github.com/google-ar/sceneform-android-sdk/issues/33 that was closed claimed the issue was fixed while it wasn't.

I am running in to same issue. Were you able to find any solution/ work around for this ?

@kunalmshah85 nope, not yet. Still waiting for @tpsiaki or @abramyan for comment :(

We have the same issue. Looks, like nobody is going to fix that :(

Sorry about the late response. We're looking into it.

Thanks, Lucy! @abramyan

Thanks Lucy @abramyan  for looking in to this. my scenario is little bit different than what @istima has. My "Dynamic feature" module is just a facade which is dependent on xyz.aar and xyz.aar is using scene form. ny base module downloads dynamic feature at run time which brings in aar with sceneform code along with it. This scenario also results in to same error.

It's madness - the fix require only a couple lines of code and still, it isn't implemented...

@tpsiaki @abramyan. Thanks for looking in to this and identifying it as bug. Would you guys be able to share details around time line for this bug fix.  or do you guys have any suggestions or workarounds on how to use Sceneform sdk and ArFragment with dynamic feature. we are working on AR project where we would like to leverage power of Dynamic Feature to reduce app size. if you can share timeline for this for this bug fix it would help us to better plan our project.

@tpsiaki @abramyan  I"m joining to the question @kunalmshah85  asked.

Fixed in 1.6

I receive an error
"com.google.ar.core.exceptions.FatalException: Application manifest must contain meta-data com.google.ar.core" when using ARCore in my dynamic feature module.


Optimise performance - version 1 (demo) release

Enable/disable these in the menu graphics settings:

cap the frame rate in project settings 30 lo 60 hi (done in graphics settings)
turn off tyre dust particles (done via menu low graphics setting)
Turn off vehicle parts in slow motion death - definitely do this (done via menu low graphics setting)
Disable directional light shadows (done via menu low graphics setting)
turn off physics_process for power_ups after set in place (done)
Cinder particles - reduce (300 hi, 30 lo) - (done via graphcis settings)

What prob won't make any diff, so ignore or blocked:

turn off/reduce flames particles (ignore for now)
turn off explosion smoke particles? Shouldn't be much of a problem, so maybe ignore this
If performance is low during nighttime, might be the headlight shadows - maybe reduce shadow atlas size (doesn't look possiblein 3.5 from script - only project settings

Any other physics processes we can turn off?

v1.18.1-demo  - W laptop - 10-13 FPS on high graphics setting


Fix GenericVector and optimize some code which used GenericVector::init_to_size

There might be more cases where init_to_size is not needed.


Compatible with comfy ui?

Hi! I saw your message about your model. What is your model based on? Do you have a version compatible with comfy ui or draw things on Mac? Can it run on Mac MPS?

hey, i'm not really related to this repo or anything, but i came across this issue and wanted to tell you that there is an extention.
https://github.com/gameltb/ComfyUI_stable_fast


Move libxslt to (optional) peerDependencies, devDependencies

Addresses #180 (though slightly differently). I had meant to address this in #171, but it must have slipped my mind.
As an optional peer dependency, client-only usage will be able to skip installing libxslt entirely. Server transform users (including enketo-core and enketo-express, though core can and should be updated to use the client transform) will need to explicitly install libxslt directly.
This is technically a breaking change, and maybe should be released as semver-major. But I think the impact should be fairly minimal because npm will message about the uninstalled peer dependency, and Node will error if importing libxslt fails.

This looks like a good approach to me.
Regarding the decision for semver major vs minor, will Core and Express or their install instructions need to change? I think not, right? If not, minor seems reasonable. @JGreenlee interested in your thoughts on this as well.

Sure, seems like this is basically to the same effect as optionalDependencies. LGTM

Chatted briefly with @eyelidlessness about the release plan. We're going to make the Transformer release with this change semver major to reflect the fact that projects that depend directly on it will need to explicitly depend on libxslt if they want to continue using server-side transformation.
Core only uses Transformer for dev and test. We'll have those use client-side transforms so that it doesn't depend on libxslt at all.
Express will add an explicit dependency on libxslt when it gets this update so that it can continue doing server-side transforms for the time being.
We'll release this now so that @JGreenlee and others can benefit from it but we won't update Core and Express immediately.


How to use it?

I have downloaded the repository, but I don't know how to run the script in order to uniquize my video. May I get your support?

Use any filters


Bump fastrlock requirement

Closes #4629

Jenkins, test this please

Jenkins CI test (for commit 523ac2b1e7d84b4bfe855f808e870af22470f2de, target branch master) succeeded!

Resolved conflict. pfnCI, test this please.

Jenkins CI test (for commit 4b133e9a767cf188e166b65dc8eb5003e10ff348, target branch master) failed with status FAILURE.

Jenkins, test this please

Jenkins CI test (for commit 4b133e9a767cf188e166b65dc8eb5003e10ff348, target branch master) succeeded!


Save to file

Add exporting to JSON + importing from JSON. The files maintain their title, even if renamed, as its stored in a JSON. Currently includes a semver version, coded to 1.0 at the minute. This is also the case for local storage items.
Also added a quick bit to update the current program selection in the dropbox based on what is currently loaded.

That's a nice feature!  I played around with the new UI controls for a bit and seems to work just fine!  I did notice one thing that we might want to address though.
If the user wants to export the current blockly program, but didn't supply a program title before clicking on 'Export to File', shall we maybe use the file name as the program title?  Currently the "title" field in the JSON file will be left empty.
The reason I am asking this is because otherwise the program dropdown list looks confusing when a user imports the program that was exported earlier.
Exact steps to trigger this behaviour

Start simulator
Do not provide a name in 'Program name' input field.
Select 'Export to File' => File Dialog comes up
Provide a file name, say foobar.json
Click 'Save' => File Dialog closes
Re-start simulator
Click 'Import from File ...' and import foobar.json
Look at program dropdown lists, the program doesn't have a title.

Let me know what you think, maybe it's not worth handling this better ...

Made the above change, lemme know if you find any other weirdness!


Protecting the endpoints on the backend (Node.js)

Currently on the backend, I have an auth.js file that has an endpoint that checks if a user is authorized to make an API call to the server.
Endpoints should be protected so no one not logged in or users who don't have a token can use it.
I've tried a few ways but I'm currently not able to implement it currently.
I believe you just have to add the auth to each endpoint in the router and then in the frontend, you would get the token from cookies and include it in the headers, could be wrong.

Can I pick this one?

@ksharma20 Hey so the auth file got deleted in a merge but what you could do is make a new auth file and protect the current endpoints that we have right now.
You would need to run the code to create a token when a user creates a username in the setting.jsx page then store it in to cookies and any api call will include the token.

Going to be closing this issue. This is no longer needed.


ci: standard linter config

Signed-off-by: Long<EMAIL_ADDRESS>Description
Ref https://github.com/dapr/dapr/issues/3537
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #[issue number]
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[x] Code compiles correctly
[x] Created/updated tests
[ ] Extended the documentation / Created issue in the https://github.com/dapr/docs/ repo: dapr/docs#[issue number]

When the comment is a command, JSON or XML, then we want the content to be as-is and not have a period. Is there a way to disable lint for those?

Add //nolint, details see https://golangci-lint.run/usage/false-positives/#nolint.


Build ARM-compatible version of this Image

On Mac (ARM)
docker run -p 26657:26657 -p 26659:26659 ghcr.io/celestiaorg/local-celestia-devnet:main

Result
Unable to find image 'ghcr.io/celestiaorg/local-celestia-devnet:main' locally
2main: Pulling from celestiaorg/local-celestia-devnet
3docker: no matching manifest for linux/arm64/v8 in the manifest list entries.
4See 'docker run --help'.

Link to snippet

The third line of the error message, docker: no matching manifest for linux/arm64/v8 in the manifest list entries., indicates that Docker is unable to find a matching manifest for the specified image and platform (Linux, architecture: arm64, variant: v8).
This error could occur because the image doesn't exist on the specified repository or the specified platform is not supported by the image. You can try pulling the image from a different repository, or you can specify a different platform that is supported by the image.
You can use the docker run command with the --platform option to specify the desired platform. For example:
docker run --platform linux/amd64 ghcr.io/celestiaorg/local-celestia-devnet:main

When specifying the amd64 platform on a mac, it does not run natively on Mac. Also, this approach is not possible for RPIs.
I propose reopening this issue and changing the local-celestia-devnet build to a multi-arch one.
See here: https://github.com/celestiaorg/.github/blob/main/.github/workflows/reusable_dockerfile_pipeline.yml#L97

hmm, it worked for me on my Mac 🤔

Yes, modern Macs have an arm processor but can emulate amd processes.

ah, cool. what are you on? I'd imagine intel macs can not use this docker run --platform linux/amd64 ghcr.io/celestiaorg/local-celestia-devnet:main

also, I looked into the Dockerfile. I think it may be celestia-app that is not built correctly? do you know what the fix is for this?

Soon there will be a new common workflow in place, that will be used by celestia-app. Then this will also be built on arm.
Is this time sensitive?

Nope, I have a note on the tutorials that says it's only been tested on AMD


Bring bridges together in zulip & improve+publish matrix bridge

There are 3 main thrusts to this PR:

Bring the 3 existing bridges together into zulip/integrations and rename them in a cohesive way (including moving the "interrealm bot" for zulip<->zulip from zulip_bots)
Make the matrix bridge easier to use generally;
Include (install) the matrix bridge by default in the zulip package.

Potential issues:

the zulip-zulip bridge (bridge_between_zulips) uses python3 type annotations, whereas the rest of the API uses python2 code; this could be reverted, assuming the code is compatible, or we could add a run-time check for python3?
the matrix bridge adds a dependency; this is just one, but should we require it just for the API? This suggests that maybe the integrations/bridges belong in yet another package?
the error-handling commit modifies the caught exception in matrix_to_zulip to just be Exception, since I highly doubt that the Zulip API raises a MatrixRequestError; however, are the exceptions potentially thrown by the (python) Zulip API clearly defined?
tests for the script would be useful; I've previously achieved this using subprocess (Popen) calls, but this can be slow to run many tests, so I've held off on writing tests for this at this point.

This does not include setting up the existing IRC or inter-realm bot at this stage, based on the recent work on this by @rheaparekh and the push to get it out and used/tested, though the latter seems like a good next step in this series.
Potential reviewers:
@rheaparekh - you've worked on this most-recently :)
@showell @timabbott - this includes the renames and script install we discussed in #integrations.

I merged one small refactor commit when I had a quick look.  I would have much preferred doing the renames at the end of the branch (actually, this is a general rule, of putting potentially controversial things at the end), since I'm not sure I'm happy with the naming scheme and want to think about it more, whereas many of the other cleanups are good to merge immediately.  If it's not hard, it'd be great if you can reorder things.  A few notes on the code changes:

I like the configparser migration (agree we need to update the docs when we make that change) and the shift to writing a sample config, though for that latter bit, I think it'd be a good idea to have it take a zuliprc file as an argument, so that we can provide good defaults for email/apikey/site.
Adding matrix_client as a dependency seems not totally ideal, given how minimal we've been with dependencies for the core zulip package, but might be worth it.  I want to think about that.

I've made some improvements, but will hold off pushing the rebased version until a little more feedback, to avoid losing the discussion points.

Here's an updated set of commits:

hopefully in an order which is easier to merge incrementally
with some README.md amendments
with renames/moves after most of the matrix changes
with matrix tests at the end

Note that the order of commits is not as shown above, as is typical for github after re-pushing.
The tests aren't in a test script at the moment, since I've been developing using pytest and they run fine with that :) I can place those in another PR along with a runner if that makes it easier.

Cool, this is great; I merged the first 8 commits (which is enough that we should do a PR to the zulip.git docs ASAP with the new config format).
The --from-zuliprc option doesn't work for me:
Traceback (most recent call last):
  File "./zulip/integrations/matrix/matrix_bridge.py", line 315, in <module>
    main()
  File "./zulip/integrations/matrix/matrix_bridge.py", line 257, in main
    write_sample_config(options.sample_config, options.zuliprc)
  File "./zulip/integrations/matrix/matrix_bridge.py", line 238, in write_sample_config
    sample_dict['zulip']['email'] = zuliprc_config['api']['email']
  File "/usr/lib/python3.5/configparser.py", line 956, in __getitem__
    raise KeyError(key)
KeyError: 'api'

We should also be sure to use six.moves.configparser; not sure if that's related to this issue.

@timabbott My guess is that your zuliprc file did not exist? I've updated that commit with a check for this condition. I'm running it as something similar to the following:
zulip-matrix-bridge  --from-zuliprc ~/zuliprc --write-sample-config /tmp/foo`

I also added a commit to add a separate test for this case, and another to separate some of the sys.exit calls from outputting the error via print, to enable that test (and potentially others).
The rebased version should be an improvement :)

Well, as above, I'm running the command using ~/zuliprc, so that seems unlikely to be the issue?

@timabbott I appreciate you're travelling, but since the sample config code is not checked yet, I wondered if I reorder these tests then we could commit those? They run with pytest, and I can add a runner pretty easily.
@showell FYI
I'd like to add some extra tests on the core functionality too, after the bug report on czo#general.

@neiljp I'm not really sure what you're asking about here, since I don't have a ton of context on the bridges changes, but if you have some commits that you are eager to get merged to un-block other work, I can try to help.  This seems like a long PR, though, so I'm a bit wary.

@showell I'm mainly wanting to get the tests merged, since I'd like to get more coverage there. The tests from here (and a few more) are now in #437, though currently the integrated runner doesn't run them (test-zulip), though it does with the changes from #431.

I just tested this PR by creating a bridge and sending messages over it. I used all the config parsers ( --from-zuliprc,  --write-sample-config, -c) and they seem to work fine.
For the first commit bridges: Bring inter-realm (zulip) bridge into zulip/ & rename others., I had a suggestion. Would it be a good idea to create a new directory bridges and have the folder irc, matrix, zulip-interrealm? I think this would be more clear than having bridge-with-*.
For the third commit zulip: Update setup.py to install matrix bridge & add dependencies., as @timabbott mentioned, there is still a pending decision whether matrix_client should be added as a dependancy.
Other than this, the second, fourth and fifth commits look fine to me. I'll look at the tests in the other PR. I think as a priority the fourth commit matrix bridge: Add optional from-zuliprc to use with write-sample-config. should be merged.

Good to hear your experience with the parsers/config-files was positive :)
This was part of the discussion on czo, and we discussed various options; this can certainly be adjusted if that is preferable
I agree that the dependency on matrix is pending; all of the bridges have dependency files (I actually missed updating the matrix one, it seems), but if we want the matrix (or others) to be installed as scripts then we need dependency installation. Perhaps one approach could be a sort of common 'provision' option that each bridge/integration can use to optionally install the dependencies required? Then the core zulip package doesn't depend on eg. matrix_client, but zulip-matrix-bridge --provision would install it?
I just reread and I missed @timabbott wanting six for configparser in the current code; it's probably best to get the simpler and less 'controversial' #437 merged first, then I'll rebase this and make that change then.

@timabbott Rebased and amended as per your requests.
I'm unsure if we need configparser as a dependency now?
There are not currently any extra docs PRs, but they will need changing slightly in line with the change to the config-file generation.

I removed the explicit configparser and merged this.  Please do the documentation update soon.  I imagine we'll want to advertise the IRC/Matrix integration in our Zulip 1.9 release notes, so we should be sure the docs work :).  Thanks @neiljp!


logging: fix formatting bug

When logging with both alternative-forms and zero padding, place the zero padding after the alternative form prefix.
As an example: LOG_ERR("%#010x", 0xabcd);
Should write 0x0000abcd to the log and not 00000xabcd

@jeffkellam thanks for the PR, please take a look at the commit formatting guidelines here:
https://docs.zephyrproject.org/latest/contribute/guidelines.html#commit-guidelines

Please change commit prefix to lib: os: cbprintf:  since fix is in that library. Also, would be good to extend tests/unit/cbprintf test suite to cover that case.

I've added the changes suggested by nordic-krch, and hopefully adjusted to be in compliance with the submission guidelines.

Code looks correct to me.    Just read through "git log" for literally thousands of examples of how this works.  Yes, the rules are pedantic and needlessly overprecise, but in a big project things get really out of hand without rules.
Also the second patch should specify that it's adding test cases for the bug, not just restate the bug.